Sergio Aguero scored two scruffy goals as City won 2-1 at Burnley and Liverpool followed suit by downing Sunderland 2-0 after losing Brazilian playmaker Philippe Coutinho to a worrying injury.
"It was an intense game for us and a deserved win at the end," said Liverpool manager Jurgen Klopp.
"It (Coutinho's injury) is an ankle injury, but we can't say anymore until we see a scan."
Chelsea can reclaim top spot if they secure a seventh successive victory when they host London rivals Tottenham Hotspur in the day's late game.
Swansea City claimed an extraordinary 5-4 win over fellow strugglers Crystal Palace, while champions Leicester City narrowly avoided a seventh defeat of the campaign in a 2-2 draw with Middlesbrough.
Having seen City prevail at Turf Moor in the early game, Liverpool were under pressure to produce the goods against a Sunderland team who had won their two previous matches.
Former Liverpool captain Steven Gerrard was in attendance at Anfield, two days after announcing his retirement from football.
But he could only look on with concern as the influential Coutinho was stretchered off with his right ankle in a brace after hurting himself in a challenge by Didier Ndong.
Origi took the Brazil international's place and he gave Liverpool reward for their dominance in the 75th minute with his first league goal of the season.
Jordan Henderson picked the Belgian forward out on the left and he cut inside onto his right foot and swept a shot inside the far post.
Milner completed victory from the penalty spot in the first minute of injury time.
Burnley took a shock 14th-minute lead against City when Dean Marney met Nicolas Otamendi's headed clearance with a thumping 25-yard volley that nestled in the bottom-left corner.
But Aguero equalised in the 37th minute, stabbing home after Burnley failed to clear a corner, and the hosts' luck continued to desert them as both Marney and Johann Berg Gudmundsson went off injured.
A defensive mix-up allowed City to net the decisive goal in the hour, with Fernandinho's cross striking Aguero and going in after Ben Mee and Stephen Ward had got in each other's way.
"We are so satisfied for the points and the performance, especially two days after the Champions League," said City manager Pep Guardiola, whose side host Chelsea next weekend.
"All of the players showed huge character and a lot of personality –- and after going 1-0 down at a stadium where only Chelsea are better at home. We are so happy."
Swansea upped the pressure on Palace manager Alan Pardew by condemning the south London club to a sixth straight defeat in a madcap game that gave Bob Bradley his first win since becoming manager.
Palace led 1-0, fell 3-1 down, fought back to go 4-3 up, but eventually succumbed courtesy of a remarkable brace of injury-time goals from Spanish striker Fernando Llorente.
"It was mad," said Swansea midfielder Jack Cork.
"At 3-1 we thought the game was finished. At 4-3 we were a bit deflated, but somehow we managed to get the win."
The victory took Swansea off the foot of the table, where they were replaced by Sunderland, and left Palace above the bottom three on goal difference.
Leicester reached the Champions League last 16 in the week and they were almost brought down to earth with a bump by Middlesbrough at the King Power Stadium.
Alvaro Negredo twice put Boro ahead, but Riyad Mahrez cancelled out his first goal with a 34th-minute penalty and Islam Slimani rescued a draw with another spot-kick in the fourth minute of added time.
Elsewhere, Gareth McAuley's header was cancelled out by Michael Dawson as West Bromwich Albion were held to a 1-1 draw at Hull City.