Pierre-Emerick Aubameyang scored with a sensational bicycle kick and had a chance to seal the tie with the final kick of the game, but somehow fired wide from six yards out.
Alexandre Lacazette said that Pierre-Emerick Aubameyang has been Arsenal's best player this season.
Arsenal climbed above Everton into ninth place and sit four points behind fifth-placed Manchester United.
Arsenal vs Newcastle: Arsenal Roar Back Into Champions League Contention With 4-0 Win Over Newcastle
Just a second league win for the Gunners under Mikel Arteta lifts Arsenal into 10th in the Premier League, but just six points adrift of Tottenham in fifth.
Pierre-Emerick Aubameyang's opening goal was cancelled out by a Jordan Ayew strike in the second half.
Arsenal defeated West Ham 3-1 on Monday to end their worst winless run in 42 years.
Man Utd vs Arsenal: A return of just nine points from their opening seven Premier League games is Manchester United's worst league start for 30 years.
Europa League: Arsenal manager Unai Emery made seven changes to the team which started Sunday's 2-2 draw at Watford.
Pierre-Emerick Aubameyang's strike ruined Newcastle manager Steve Bruce's first match in charge of the club he supported as a boy.
Luis Suarez scored the winner in the 89th minute in front of nearly 99,000 people, who packed the stands to see new signings Antoine Griezmann and Frenkie De Jong.
Unai Emery insisted his side's Europa League semi-final against Valencia is still "50-50" despite Arsenal taking a 3-1 first leg lead to Spain.
Arsenal held out for a first clean sheet away from home in the Premier League this season to move ahead of Chelsea on goal difference into fourth.
Apart from Pierre-Emerick Aubameyang, substitute Alex Iwobi scored the third goal for Arsenal while Ashley Barnes scored the lone goal for Burnley.
A video has emerged that appears to show a number of Arsenal stars inhaling nitrous oxide.
A banana skin was hurled from the away end at the Emirates Stadium as Pierre-Emerick Aubameyang celebrated scoring in front of the Tottenham fans.
Aubameyang's superb long-range equaliser sparked a second-half surge from Arsenal.
Arsenal have been revitalised since Unai Emery replaced Arsene Wenger in the close-season.
Arsenal take on Sporting CP while Chelsea take on Bate Borisov in their Europa League encounters.
