Pompey finally saw off a stubborn Rotherham side to claim all three points at Fratton Park and register a fourth successive victory.
Ronan Curtis opened the scoring straight from kick-off and after Freddie Ladapo levelled, John Marquis restored their lead.
But Ladapo levelled again after the break – this time from the spot – and it took a neat finish from Ben Close to get the job done.
Kenny Jackett unsurprisingly stuck with the same side that won at Rochdale at the weekend, with Lee Brown once again wearing the armband in the absence of regular skipper Tom Naylor.
And the hosts came sprinting from the blocks, taking the lead with just 33 seconds on the clock – with no surprises about the identity of the goalscorer.
The in-form CURTIS continued his scoring spree with a fine shot from outside the box after receiving a cross-field pass from Ryan Williams.
Millers keeper Daniel Iversen then fumbled Brandon Haunstrup’s deep pass into the box and would have been relieved to see the ball land clear.
But Rotherham soon grew into the contest and when Kyle Vassell guided a shot through Oli Hawkins’ legs, Craig MacGillivray had to get down to make the save.
The unsighted keeper could only parry the same player’s free-kick moments later, though, and Ladapo gratefully converted the loose ball to level the scores.
Marquis sent a couple of half-chances off target, but United were looking threatening at the other end and MacGillivray did well to deny Ladapo.
It was soon the Blues’ turn to go on the attack and Close blazed wastefully over after racing into the box to meet Ellis Harrison’s header.
The hosts did restore their advantage moments later, however, as Harrison nodded Brown’s cross towards the far post, where MARQUIS was waiting to slam home.
There was an escape at the other end when the hosts made a mess of dealing with a cross, but Brown, Vassell and Carlton Morris all collided in the box and the ball drifted harmlessly wide.
Pompey then should have extended their lead in stoppage-time, with Curtis picking out Marquis in the box and the summer signing only able to send a tame header straight at Iversen.
Half Time: Pompey 2 Rotherham 1
The hosts did have the ball in the net again just after the restart, but Harrison forced Iversen into the net when the keeper collected Williams’ cross and a foul was awarded.
Both sides then collected yellow cards in quick succession, with substitute Chiedozie Ogebene punished for United and Harrison for the Blues.
Rotherham were seeing plenty of the ball around their opponents’ box, but unable to test MacGillivray.
And a quick Williams counter provided an opening for Pompey just before the hour mark, although Curtis’ eventual delivery was just behind Marquis.
But the Millers were given an excellent opportunity to level when Morris’ trickery inside the box drew a foul from Hawkins.
Referee Antony Coggins pointed straight to the spot and Ladapo confidently sent MacGillivray the wrong way to equalise for a second time.
But Pompey again managed to restore their advantage, this time taking just four minutes to go back ahead.
Rotherham made a hash of dealing with Curtis’ flag-kick and CLOSE chested the ball down on the edge of the box before volleying into the bottom corner.
The visitors were dealt a further blow when Billy Jones went down injured just seconds after they had made their final two changes.
He was forced to limp down the tunnel, leaving United to play out the rest of the contest with 10 men.
Pompey soon made a change of their own, with Williams leaving the field to a standing ovation and replaced by Marcus Harness.
They tried to make the most of their numerical advantage, but Curtis was unable to slip a shot inside Iversen’s near post following good work from Haunstrup.
It was certainly not all one-way traffic, though, and Jamie Lindsay’s effort was deflected narrowly past the post.
Harness then hit a rocket of a shot from the edge of the box just over the bar, as the Blues looked to wrap things up.
They were unable to score again, but saw out four minutes of stoppage-time to climb up into 10th spot in the League One standings.
Pompey (4-2-3-1): MacGillivray; Haunstrup, Hawkins, Burgess, Brown (c); Walkes, Close; Williams (Harness 78), Marquis, Curtis; Harrison
Goals: Curtis 1, Marquis 37, Close 66
Subs not used: Bass, Bolton, Downing, Cannon, Evans, Pitman
Rotherham (4-4-2): Iversen; Olosunde (Ogbene 46), Ihiekwe (c), Robertson, Jones; Morris, Wiles, Barlaser (Lindsay 73), Clarke; Ladapo, Vassell (Crooks 73)
Goals: Ladapo 15, 62 (pen)
Booked: Ogbene, Morris
Subs not used: Price, Thompson, Hastie, Smith
Referee: Antony Coggins
Attendance: 16,355 (204 away fans)