The Reds became the first side to overturn a two-goal first leg home deficit to gloriously and historically progress to the quarter-finals on away goals after Rashford nervelessly thumped home the spot-kick in the fourth minute of stoppage time. Referee Dario Skomina had initially given a corner but reviewed the video technology after Diogo Dalot's shot had deflected off the arm of Presnel Kimpembe.
Missing ten first team players and going to the formidale setting of the Parcs des Princes 0-2 behind after defeat at Old Trafford, few - if any - had given Ole's side even a prayer of getting through. PSG, after all, were widely considered as among the favourites for the Champions League crown.
They had reckoned without this club's famed powers of recovery, and it took United only two minutes to halve the arrears. A waywardly loose backpass from Thilo Kehrer was seized upon by the lurking Romelu Lukaku, and the in-form Belgian raced through, rounded the advancing Gigi Buffon and slid the ball in from the tightest of angles.
That shook Thomas Tuchel's side into life and thereafter came a period of sustained pressure as United were forced to dig deep and hang on. United survived a couple of close calls but were unable to prevent Juan Bernat levelling the proceedings; the Spanish left-back arrived at the far post to convert a low cross from Kylian Mbappe, who had got in behind the Reds’ defence on PSG’s right.
Angel Di Maria flashed an effort wide and then had a goal correctly chalked off by the linesman. David de Gea saved well from Bernat after Dani Alves had played him in, before speedster Kylian Mbappe went close with a snapshot.
Having weathered the predicted Parisian storm, and despite the hosts boasting 86% of the ball, United went ahead again on the half-hour mark. Rashford's speculative effort should have been easy for Buffon, but the legendary veteran instead spilled the ball straight into the path of Lukaku who gobbled up the chance for his six goal in three games. United now only needed one more to go through, as hope had turned to belief and then a semblance of expectation.
Lukaku became the first Red since Ronaldo in 2006 to score a brace in three successive matches.
Diogo Dalot replaced the hapless Eric Bailly and the Reds looked more comfortable and at ease.
Ashley Young switched to a more familiar right-back role and made hugely important interception on the hour mark when the always dangerous Mbappe tried to find Di Maria.
Ole insisted all his young players were ready should they be needed and Tahith Chong, fresh from coming on against Southampton, joined the action 10 minutes from time in place of Andreas Pereira, while 17-year-old Mason Greenwood made his United debut soon after in place of Young.
De Gea blocked from substitute Thomas Meunier, a last ditch challenge from Chris Smalling thwarted Mbappe and Bernat then fired off a post.
Time had begun to run out for the Reds, with PSG clinging to their slender 3-2 aggregate advantage and seemingly on course for the last eight.
Then came the moment United had been looking for when Dalot's shot struck Kimpembe's arm and deflected away for a corner. After the referee agonisingly deliberated for what felt like an eternity, he gave the spot-kick. Rashford showed nerves of steel and guts aplenty to take responsibility for the kick, and he smashed it high beyond Buffon to prompt wild celebrations on the pitch, on the touchline, in the stands and all the way back to Manchester. United were forced to survive an inexplicable ten added minutes before the miracle of Paris was remarkably complete.
Overall team performance: 9/10
United Faithful Man of the Match: There were fourteen Herculean heroes pretty in pink out there but we're giving this to Romelu Lukaku. He can't stop scoring.