Pompey appeared to be cruising to victory, but will ultimately be happy with a point.
They created a hatful of chances in the first half against Plymouth at a packed Fratton Park, leading at the break thanks to a fine Josh Koroma finish.
But a Sean Raggett own-goal and Niall Ennis strike saw the visitors turn the contest on its head in the space of just five minutes.
Brendan Galloway was then dismissed for the Pilgrims late on and substitute Reeco Hackett nodded home a dramatic stoppage-time equaliser.
Danny Cowley was pleased with the way his side kept fighting until the end, but was unhappy with the manner of the goals they conceded.
The Blues boss said: “I thought we played really well for the first 70 minutes and just weren’t ruthless enough.
“There were a lot of positive moments, particularly on the transition, against a Plymouth side we have a lot of respect for.
“They’re as good as we’ve played so far this season and will certainly be right up there, let me tell you that.
“But goals change games and we concede a poor one. We didn’t defend the 1v1 situation well enough, then allow the cross to come in and the mistake happens.
"The players on the pitch – and the supporters off it – did what they did best. They dug in and found a way."
“That turned the match on its head and they scored another that was disappointing from our point of view.
“All of a sudden we’re behind and you have to decide what you stand for. But the players on the pitch – and the supporters off it – did what they did best. They dug in and found a way.
“There was the tactical smartness of Marlon to find Joe in space and it was a fantastic cross from him, followed by a fine header from Reeco. He’s six foot three and owes me some headers!”
There was a controversial moment when the contest was still goalless, with Raggett’s strike blocked by the arm of Nigel Longwijk and then appearing to cross the line.
Cowley said: “There’s a handball and it’s about two yards over, but I haven’t had a chance to speak to the referee yet.
“We’re in the 21st century and to help the officials, I think we should have the technology in professional football.
“I don’t know why we don’t have it, but his watch would most definitely have been vibrating if we did.”
Tom Lowery limped off injured late in the first half, although his replacement Joe Morrell impressed in midfield.
Cowley said: “It’s such a shame because he was training so well. It looks like the outside of his hamstring, but we’re hoping it’s not too bad.
“Joe hasn’t trained after having an extra injection into his groin on Wednesday, so to come on and immediately go to 100mph is testament to the player he is.”