It looked like Pompey were going to experience their first Fratton defeat of the season.
But trailing 2-1 to Burton deep into stoppage-time, Ellis Harrison was bundled to the turf and Brett Pitman converted from the spot.
That capped a dramatic fightback, with Ronan Curtis initially inspiring hope after Joe Sbarra and Kieran Wallace had both struck for the visitors inside six minutes.
The Brewers then had John-Joe O’Toole dismissed for two bookable offences midway through the first half.
And although Kenny Jackett was pleased with the manner in which his side took control of the contest from that point, he was disappointed by their sloppy opening.
The Blues boss said: “It was a frustrating evening and a very poor start from us, which gave us a mountain to climb.
“You can look at both goals we conceded and we need to be much more solid, even if there was a big deflection for the second.
“But I still felt positive that we could get something out of the game andperhaps even win it because we had enough time and enough of the ball.
"For all of the domination we had with the ball, we couldn’t find that killer instinct until right at the end."
“We swapped to three at the back to give Haunstrup and Evans more width in those crossing positions.
“There were some big chances for us in the second half and the keeper made a really good save from Ronan, who played very well.
“Marquis had one that went just a whisker wide and there was another decent penalty appeal for a foul on Cannon.
“But for all of the domination we had with the ball, we couldn’t find that killer instinct until right at the end.
“It was a penalty on Ellis and Brett’s put it away very well. It would have been nice to have a few more minutes after that to try to score the winner.”
There were some boos from the Fratton faithful at the final whistle and Jackett understands their annoyance.
He said: “I’m obviously frustrated as well because six points from six games is not where we want to be.
“We have to keep working away and need a series of wins to turn our season around. It’s up to us to make sure we do that.”