Pompey were denied victory by a stoppage-time own-goal, despite a positive performance at Bristol Rovers.
Gareth Evans opened the scoring from the penalty spot and the visitors doubled their lead after the break, as Ronan Curtis headed home a John Marquis cross.
But Alex Roadman reduced Rovers' deficit and then, right at the death, Liam Sercombe's corner hit both Ross McCrorie and the bar before going in off the head of Craig MacGillivray.