Arsenal took the lead twice through Thierry Henry and Gilberto Silva, only for Laurent Robert and Olivier Bernard to peg them back.
It took a penalty ten minutes from time - converted by Henry - to seal the points after Jermaine Jenas handled a Robert Pires corner.
Arsenal dominated the opening exchanges and Sylvain Wiltord, Freddie Ljungberg and Henry all went close before the opening strike.
The Gunners broke the deadlock on 18 minutes thanks to a calamitous error from Titus Bramble.
Lauren whipped a cross in from the right, but Bramble swung wildly at the ball and it deflected off his left foot and straight to Henry at the back post, who finished with great aplomb past Shay Given.
Arsenal looked comfortable but on 25 minutes they were rocked by an injury to captain Patrick Vieira who was forced off with a thigh strain - and within 60 seconds the visitors were level.
Newcastle had done nothing to suggest they could level the game, but a stunning break down the right saw them grab the equaliser.
Craig Bellamy received the ball on the edge of the area and he flicked it straight into the path of the on-rushing Kieron Dyer and he squared the ball for Robert, who could not miss from five yards out.
The second half began in a similar vein to the first with Arsenal on top and they should have took the lead with Ljungberg after just five minutes, but his chip over Given cannoned off the post and to safety.
Another injury blow then struck Arsenal, when Ashley Cole made an early departure to the dressing room, but the loss of the England full back did not have too much affect on the home side.
Robert Pires was introduced by Arsene Wenger on the hour and he created the second goal for Arsenal on 70 minutes with a wicked free kick which Silva headed in from close range after losing his marker Jenas.
Again, though, Newcastle hit Arsenal on the break with a second goal just a minute later, when Dyer broke clear and ran unchallenged for 20 yards before threading a wonderful ball into the run of Olivier Bernard and the Frenchman blasted the ball into the top corner of Jens Lehmann's net. 
A deflated Arsenal were then gifted the lead again when Jenas inexplicably handled Pires's corner when he had already out leaped the attackers for the ball.
Mike Riley spotted Jenas's misdemeanour and Henry (pictured) took advantage from spot, as he coolly clipped the ball down the middle of the goal past the helpless Given.
Click below for match summary:
Arsenal 3-2 Newcastle United
Click here for Premiership table
