Callum Lang came off the bench to score a dramatic debut goal, but it was not enough to secure victory for Pompey at Oxford.
His late strike had turned the contest on its head after a scrappy Colby Bishop effort cancelled out Tyler Goodrham’s opener.
But it was another substitute, James Henry, who had the final say with a 90th-minute leveller for the hosts.
John Mousinho made one change from the side that had won on the road at Port Vale at the weekend.
Marlon Pack missed that game through illness, but the skipper recovered to replace Tom Lowery in midfield.
Lang was named on the bench, although fellow new arrival Tom McIntyre did not sign in time to feature.
Pompey, as in recent weeks, controlled possession throughout the first half, but they struggled to convert that into many meaningful chances.
In fact, neither goal was really tested for most of the opening period, as the contest struggled to get going.
Not that there was any lack of commitment, with Connor Ogilvie and Joe Morrell both sliding in to win key challenges in the middle of the pitch.
There was an opportunity for the visitors after Pack had sprayed a fine pass out to Terry Devlin on the right.
He sent in a cross that Abu Kamara had to stoop to get his head to, with the ball flying narrowly wide of the post.
But the Blues were forced to make a change when Morrell went down and limped from the pitch to be replaced by Lowery.
Pompey continued to build patiently around the Oxford box, but they were just unable to fashion a opening.
And it was their opponents who broke the deadlock just before break, as Marcus McGuane found Goodrham, who had time to advance and send a precise shot into the bottom corner.
Half Time: Oxford 1 Pompey 0
United had already been forced to bring off defender Fin Stevens in the first half and they had to make another change at the break.
Jamie Cumming had appeared in some discomfort and did not return, with former Pompey keeper Simon Eastwood replacing him.
The visitors had an early chance to get back on level terms when Kamara met Paddy Lane’s fine cross, but he could only direct his header into Eastwood’s arms.
Lowery then brought the ball down neatly outside the box and sent a shot dipping not too far over the bar.
The midfielder was soon caught in possession to set up an Oxford attack, but Mark Harris’ pass to Tyler Burey was poor, allowing Ogilvie to make the block.
Mousinho decided to freshen things up, with Lang brought on for Devlin to make his Blues bow just past the hour mark.
Pompey continued to pass the ball around and were searching for gaps to exploit, with their opponents defending resolutely.
But they did grab an equaliser on 69 minutes – and none of the away fans cared that it was an extremely scrappy affair.
Kamara did well to skin Goodrham on the right and send a cross in towards Lang, whose touch led to an almighty scramble.
Eastwood was unable to grab hold of the ball and BISHOP showed plenty of persistence to force it over the line.
Goodrham soon tried to make amends by breaking up the pitch and finding Cameron Brannagan, whose shot deflected narrowly over off Kamara.
The United winger was in the thick of it again, seeing penalty appeals waved away after hitting the turf under pressure from Conor Shaughnessy, with boss Des Buckingham booked for his protests.
But the noisy Blue army were sent wild with delight on 80 minutes, thanks to a moment of magic from a new favourite.
LANG collected a fine Pack pass into the box and held off challenges from both Greg Leigh and Brannagan before firing into the net.
It looked like that would be enough to secure all three points, but United had other ideas and grabbed a late leveller.
Will Norris did well to get down to a low Harris shot and make the save, only for Henry to get to the loose ball first and nod it home from close range.
Both sides still sensed they could find a winner during five minutes of added time, but had to settle for a share of the spoils.
Oxford (4-4-2): Cumming (Eastwood 46); Stevens (Smyth 43), Moore (c), Brown, Leigh; Burey (O’Donkor 69), McGuane (Henry 85), Brannagan, Goodrham; Rodrigues, Harris
Goals: Goodrham 45, Henry 90
Subs not used: Negru, Thorniley, McEachran
Pompey (4-2-3-1): Norris; Devlin (Lang 62), Shaughnessy, Raggett, Ogilvie (Towler 86); Pack (c), Morrell (Lowery 34); Kamara, Peart-Harris, Lane; Bishop
Goals: Bishop 69, Lang 80
Booked: Bishop, Peart-Harris
Subs not used: Macey, Sparkes, Martin, Saydee
Referee: Darren Drysdale
Attendance: 8,864 (1,485 away fans)