For the third successive away game, Arnautovic found the back of the net for the Hammers, his latest strike coming 11 seconds into the second half at the John Smith's Stadium. Lolley would make up for his embarrassing moment, with a terrific individual finish in the 40th minute to equal the contest.
The win takes the Hammers above Town in the Premier League table, with the Terriers four points above the relegation zone.
However, it was West Ham who broke the deadlock just before the half hour after a mistake by Lolley from Lossl's short goal kick.
Quique Sanchez Flores rules himself out of vacant Stoke job
Flores was successful in the Premier League with Watford in 2016 and he managed to secure a mid-table position for the newly promoted club.
Awful, awful goal from Town's perspective on 25 minutes. The first is an individual mistake that should never happen.
Cheikhou Kouyaté then flicked the ball on to Arnautovic, who deftly turned Tommy Smith before firing past Jonas Lossl...
On his first start in the PL he cut inside from the right and curled a beauty into the far corner with his left foot. 1-1.
West Ham United moved themselves to the brink of the top-half of the Premier League after a hugely impressive victory. A series of quick passes left Manuel Lanzini through on goal and the Argentinian finished with aplomb to send the Hammers 3-1 ahead.
West Ham were understandably delighted with the result and were equally impressed with the performance from Arnautovic during the game, taking to Twitter to provide their reactions.