Pompey dropped a place to fifth in the League One standings after losing to fellow promotion hopefuls Doncaster, despite a late rally at the Keepmoat Stadium.
An early goal from Reece James was all that separated the sides at the break and the Blues looked much more of a threat in the second half.
But they were unable to take their chances and a finish from Rovers substitute Fejiri Okenabirhie seemingly ended their hopes.
John Marquis reduced the deficit deep into stoppage-time, however, and the visitors almost managed to force home an equaliser with the last kick of the ball.
Kenny Jackett made two changes from the side that drew with Gillingham on home soil at the weekend.
Jack Whatmough and Charlie Daniels both returned to the starting line-up, as Andy Cannon and Lee Brown dropped to the bench.
It led to a switch to a 3-4-1-2 system, with Marcus Harness floating just behind the front two of Ellis Harrison and Ronan Curtis.
There was a familiar face in the opposition team, with former Pompey loanee Omar Bogle leading the attack.
It was almost a dream start for the visitors, who came close to taking the lead with just 15 seconds on the clock.
They stole possession from the kick-off and a cross from Harrison was brought down by Curtis, who turned in the box and fired a shot inches past the post.
Doncaster hit back almost immediately and Brad Halliday angled the ball narrowly wide following good work from the evergreen James Coppinger.
And the hosts did forge ahead on 12 minutes, when Haji Mnoga was unable to cut out John Bostock’s pass and James raced through to score.
It was certainly a lively opening – a bit too much for referee Ross Joyce, who showed yellow cards to Jon Taylor and Harness for late challenges.
Jack Whatmough had an opportunity to level when Daniels’ deep free-kick was nodded down, but could not keep his effort down.
Pompey thought they should have won a penalty midway through the first half after Mnoga received the ball back from Harness and went down in the box under Tom Anderson’s challenge.
The Rovers skipper certainly looked relieved when Joyce shook his head to dismiss the teenager’s appeals.
They were forced to make a change soon after, with Taylor getting injured by a crunching – yet fair – challenge from Sean Raggett.
Bostock tried to double the home team’s advantage with an ambitious strike from 35 yards that was well held by Craig MacGillivray.
But the Blues finished the half with a flourish, with an attempt from Harness deflected wide before a Harrison header was claimed by keeper Louis Jones.
And Harrison was inches away from connecting with a fine Curtis delivery in stoppage-time, only for an apparent flick by Anderson to deny him.
Half Time: Doncaster 1 Pompey 0
Jackett made a double switch at the interval, with Mnoga and Curtis taken off and replaced by Cannon and Marquis.
It meant a more traditional 4-4-2 formation for the Blues, albeit with Whatmough filling in at right-back.
The latter soon received a fine return pass from Harvey White, but his low ball into the centre was intercepted and cleared by Bostock.
White was looking much more assured and a stinging drive from the edge of the area just cleared the angle of post and bar.
Marquis then thought he got a push as he tried to reach a teasing low Whatmough cross, but Joyce disagreed.
And the striker soon bundled ball and keeper into the net after Rasmus Nicolaisen had nodded Daniels’ free-kick towards the target, with a foul awarded against the striker.
Pompey continued to knock at the door and the hosts were relieved to see the ball land safe when Joe Wright and Jones made a mess of dealing with White’s cross.
Instead it was the visitors who scored against the run of play on 70 minutes, courtesy of a fine finish from Okenabirhie, who somehow forced a shot home from the tightest of angles.
Brown immediately replaced Daniels, but it was a cross from Harness that was agonisingly out of reach for Marquis just moments later.
Jordy Hiwula was the latest player introduced, coming on for Harrison to offer some fresh legs to the attacking line.
It was clearly not Pompey’s night, however, and the was best summed up in the closing stages when an on target Cannon strike was deflected wide by team-mate White.
But they did have something to show for their efforts in the fourth minute of added time, as MARQUIS met Harness' cross, spun sharply and neatly found the net.
And they almost grabbed an unlikely leveller with the last kick off the ball, but White could not quite force the ball home in a packed box.
Doncaster (4-2-3-1): Jones; Halliday, Wright, Anderson (c), James; Bostock (Robertson 64), Smith; Taylor (Okenabirhie 28), Coppinger (Richards 76), Sims (Lokilo 64); Bogle
Goals: James 12, Okenabirhie 70
Booked: Taylor, Smith
Subs not used: Balcombe, John, Greaves
Pompey (3-4-1-2): MacGillivray; Whatmough, Raggett, Nicolaisen; Mnoga (Marquis 46), Naylor (c), White, Daniels (Brown 70); Harness; Curtis (Cannon 46), Harrison (Hiwula 76)
Goals: Marquis 90+4
Booked: Harness, Nicolaisen, Marquis
Subs not used: Ward, Byers, Williams
Referee: Ross Joyce