Pompey were beaten at Bristol Rovers to suffer just a second League One away defeat of the season.
John Marquis had opened the scoring, but Jonah Anuya's brace and a Luke Leahy penalty sealed a 3-1 win for the hosts.
The side were led by Joe Gallen in Kenny Jackett's absence, with Marquis seeing a stoppage-time penalty saved.