Pompey were cruelly denied a point at Rotherham after conceding a goal in stoppage-time.
It looked like they had done enough to secure a draw after surviving plenty of second half pressure.
But two of their former players combined for the Millers’ late winner, with Michael Smith teeing-up Joe Mattock for a headed finish.
Kenny Jackett made two changes from the side that drew with Scunthorpe on home soil the previous week.
Sylvain Deslandes was handed a debut at left-back, while Matty Kennedy also returned, as Oli Hawkins and Connor Ronan dropped to the bench.
Jackett was facing the club where he had a short spell as manager last term, while former Blues boss Richie Barker was in the opposition dug-out.
The ball was a regular visitor to both ends of the pitch in the early stages, although a lack of quality in the final third meant a goal did not appear likely.
Dion Donohue – in a natural midfield role – made a block to deny Will Vaulks, while Kal Naismith sent a volley wildly over the bar after meeting Kennedy’s cross.
A fierce effort from Jamal Lowe was on target, although it was straight at Marek Rodak and the Millers keeper made a comfortable save.
Pompey did look bright following a quick break up the pitch on 13 minutes, as Deslandes collected the ball from Ben Close on the overlap, only for his cross to be just too high for Naismith.
And a Donohue free-kick that was flighted into the box somehow evaded both Christian Burgess and Matt Clarke as the defensive duo dashed in at the far post.
Both sides then had decent chances to break the deadlock in the space of a few minutes – with the Blues first to threaten.
Lowe chipped a pass up to Brett Pitman and the skipper tried to deftly flick the ball home, but he did not get enough on his header and it flew safely into Rodak’s arms.
David Ball soon had a dig for Rotherham at the other end, with his curling effort taking a slight deflection and sailing narrowly over the bar.
There were certainly not too many clear-cut attempts, though, with the home fans growing frustrated by some wasteful finishing.
Donohue had one last effort before the break and although his drive took a deflection off Vaulks, it was not enough to deceive Rodak.
Half Time: Rotherham 0 Pompey 0
The first half had drifted towards a conclusion, but Rotherham came charging from the blocks at the start of the second.
First, Luke McGee had to leap on the ball when Ryan Williams’ delivery was turned back into the centre by Richie Towell.
A series of corners then caused chaos in the Blues box, with the Millers going closest to scoring from the first of them.
Richard Wood’s header was blocked in a crowded box and Nathan Thompson then charged down Vaulks’ follow-up effort.
Another flag-kick was soon sent back into the danger zone, though, and this time an unmarked Wood skewed wide.
But Pompey then had an opportunity of their own, with a commanding Rodak punch preventing Naismith from converting Lowe’s cross.
Jackett made a double switch on 66 minutes, with Pitman and Deslandes withdrawn for Hawkins and Ronan.
Smith soon worked his way into the Blues box, but Burgess was there to make a block and deny his former team-mate.
Corners were certainly proving a problem for Pompey and when one was met by Smith, the ball pinged around the area before eventually being hacked to safety.
Another soon followed and it was Thompson to the rescue once more, with the defender throwing himself in front of Semi Ajayi’s effort.
Rotherham applied more pressure as the seconds ticked down, but the visitors were holding firm at the back.
They might have even snatched victory late on when the ball fell to Lowe on the edge of the box, but his effort flew over.
Instead it was the Millers celebrating a late goal, as Smith crossed from the right and Mattock rose to head home at the far post.
Rotherham (4-2-3-1): Rodak; Emmanuel, Ajayi, Wood (c), Mattock; Forde (Taylor 83), Vaulks; Towell, Yates (Smith 65), Williams (Newell 65); Ball
Goals: Mattock 90+1
Booked: Emmanuel
Subs not used: Price, Ihiekwe, Cummings, Clarke-Harris
Pompey (4-2-3-1): McGee; Thompson, Burgess, Clarke, Deslandes (Ronan 66); Close, Donohue; Lowe, Naismith, Kennedy; Pitman (c) (Hawkins 66)
Subs not used: Bass, Widdrington, May, Bennett, Chaplin
Referee: Robert Jones
Attendance: 9,129 (1,471 Pompey fans)