Pompey conceded deep into stoppage-time to cruelly be denied a point at Oxford after having to play most of the game with just 10 men.
Michael Jacobs immediately cancelled out Luke McNally’s early opener, only for Joe Morrell to then be harshly sent off.
But the Blues were on course for an unlikely victory when Ronan Curtis slotted them into the lead after the break.
Cameron Brannagan levelled in the closing stages of normal time, however, before substitute Nathan Holland struck at the death.
Danny Cowley made five changes from the side that were beaten by Charlton on home soil on Monday night.
Two of them were enforced, with the injured Mahlon Romeo (ankle) and Shaun Williams (back) replaced by Kieron Freeman and Louis Thompson.
Marcus Harness, Tyler Walker and Clark Robertson all dropped to the bench to make way for Jacobs, Curtis and Connor Ogilvie.
Also named among the substitutes were deadline day signing Aiden O’Brien and a fit-again Ryan Tunnicliffe.
Former Blues players Ryan Williams and Steve Seddon both started for the hosts, who also had Simon Eastwood in their matchday squad.
Anybody who got into the ground late would certainly have missed some key incidents, as the opening 16 minutes saw two goals and a red card.
The hosts took an early lead when McNally rose highest in the box to convert a corner from Billy Bodin.
But Pompey quickly hit back, as a long kick from Gavin Bazunu was flicked on by George Hirst and Curtis did well to keep it in play.
His neat back-flick was collected from Freeman, who then squared the ball for JACOBS to slot home from close range.
The celebrations did not last long, however, as the disbelieving visitors were soon reduced to 10 men.
Morrell caught Brannagan and there was no surprise that a free-kick was awarded, but the midfielder was left open-mouthed when referee Samuel Barrott brandished a red card.
United looked to make the most of their numerical advantage and got plenty of men forward to apply pressure.
They came close to restoring their advantage when Brannagan was sent racing through, but Bazunu was off his line quickly to make a key save.
Williams then ballooned a good chance over the bar, while Herbie Kane rolled a shot narrowly wide and saw another attempt held by Bazunu.
But the Blues almost took the lead themselves just before the break after Jacobs had stolen possession.
He rolled the ball across for Hirst, but the striker’s low shot was kept out by the foot of keeper Jack Stevens.
Half Time: Oxford 1 Pompey 1
The second half began more than 20 minutes later than scheduled because of an unspecified ‘incident’.
And when the action did get started, there was a cagey opening – before Pompey helped explode the contest back into life.
Hirst did well to get forward and use his strength brilliantly to hold of a challenge before finding Jacobs.
He then slipped a clever pass through to CURTIS, who was left in space to calmly roll the ball past Stevens and into the net.
Oxford tried to hit back and Bodin’s shot was blocked after both Matty Taylor and Seddon had failed to connect with Williams’ centre.
Bazunu then dived to his left to push behind Brannagan’s angled strike and was able to deal with the resulting corner.
And Pompey were forced to make a change on the hour mark when Freeman took a ball to the face and was replaced by Robertson.
It was the home side that were unsurprisingly seeing more of the ball and a couple of corners caused a few nerves, while Bazunu made a regulation save to deny Mark Sykes.
Cowley then made his second switch on 71 minutes, bringing some fresh legs into the engine room by swapping Thompson with Tunnicliffe.
Bazunu was soon called into action again, getting down to his right to push clear a low Brannagan shot that was creeping towards the bottom corner.
But the Blues’ resistance was finally broken late on, courtesy of a fine 25-yard strike from Brannagan that flew into the net.
O’Brien was then brought on for his debut in place of Hirst and the delay caused by Freeman’s injury, along with various other stoppages, meant 10 minutes were added on.
With the defender’s departure deemed a concussion substitute, Denver Hume made way for Reeco Hackett.
But the Blues were unable to hang on and Holland’s effort from outside the box clipped the inside of the post on its way into the net.
Oxford (3-1-4-2): Stevens; McNally, Moore (c), Brown (Holland 55); Kane; Williams (Winnall 63), Sykes, Brannagan, Seddon (Mousinho 90+8); Bodin (McGuane 87), Taylor
Goals: McNally 6, Brannagan 82, Holland 90+6
Booked: Brannagan, Seddon, Kane, Moore
Subs not used: Eastwood, Forde, Smyth
Pompey (3-4-1-2): Bazunu; Carter, Raggett (c), Ogilvie; Freeman (Robertson 60), Thompson (Tunnicliffe 71), Morrell, Hume (Hackett 90+5); Jacobs; Hirst (O’Brien 82), Curtis
Goals: Jacobs 10, Curtis 51
Sent off: Morrell
Booked: Jacobs, Hirst, Hume, Curtis, Bazunu, Tunnicliffe, Raggett
Subs not used: Webber, Harness, Walker
Referee: Samuel Barrott
Attendance: 10,373 (1,754 Pompey fans)