Pompey conceded a late goal to be denied all three points in a midweek contest at Oxford.
John Mousinho's men dominated possession in the first half, only to fall behind to Tyler Goodrham's strike just before the break.
But Colby Bishop scrambled home a leveller and Callum Lang then came off the bench to fire the Blues ahead on his debut.
James Henry was first to a loose ball to nod United level late on, however, and ensure the contest finished all square at 2-2.