Pompey salvaged a dramatic point against Burton, courtesy of Brett Pitman's stoppage-time penalty.
Joe Sbarra and Kieran Wallace – the latter with the help of a huge deflection – struck for the visitors in the opening stages.
But the Brewers had to play much of the contest with just 10 men following two quick bookings for John-Joe O’Toole.
Ronan Curtis pulled a goal back before the interval and Pitman converted from the spot right at the death following a foul on Ellis Harrison.
Kenny Jackett made two changes from the side that drew with Blackpool in the last third tier outing at the end of August.
Pitman and summer signing Ryan Williams were both handed their first league starts, with Andy Cannon dropping to the bench and Marcus Harness out injured.
There was a minute’s applause before kick-off in memory of former Fratton forward Dennis Edwards, who passed away the previous week.
The hosts got off to a dreadful start and found themselves two goals down inside the opening six minutes.
Lucas Akins did well down the right and passed across the box to Wallace. His shot was blocked by Williams, but the ball fell for Sbarra to slot home.
And it soon got even worse for the Blues, with Wallace’s long-range effort taking a wicked deflection off Paul Downing’s knee to leave Craig MacGillivray wrong-footed.
Fratton Park was left stunned and the home side did not immediately look like getting back into the contest.
But there was a lifeline when O’Toole picked up two yellow cards in quick succession to leave Burton with just 10 men.
John Marquis felt it should have been a straight red after getting tugged to the turf by the defender, although referee Craig Hicks decided there was a covering defender.
Ross McCrorie then swung a cross into the box from the left and skipper Tom Naylor rose highest to head narrowly wide.
Marquis tried to wriggle his way through on goal moments later, but again O’Toole clumsily brought him down.
Hicks played the advantage and it almost paid dividends, with Pitman thundering a shot against the crossbar.
But the official then called O’Toole over to send him for an early bath, forcing Sbarra to be sacrificed for an extra defender.
The hosts tried to make their numerical advantage count and a stretching Curtis saw his header held under the crossbar by Albion keeper Kieran O’Hara.
Akins was still causing problems at the other end, though, and saw off both Naylor and MacGillivray from a long clearance, but there was nobody to meet his cut-back.
It had been a disjointed display from the Blues, but they found themselves back in the contest on 39 minutes.
Burgess’ cross caught a deflection and looped into the box, where Marquis’ header was saved by O’Hara.
But CURTIS was quickest to the loose ball and lashed it into the roof of the net to reduce Pompey’s deficit.
Half Time: Pompey 1 Burton 2
There was a change of system and personnel at the start of the second half, with Gareth Evans replacing Williams and the Blues playing with three at the back.
And the newcomer almost provided an equaliser on 50 minutes, sending in a cross that was met by Curtis, forcing O’Hara to tip it over the bar.
Ben Close then had an effort blocked by Wallace after Burgess’ delivery had been turned back into the centre by Curtis.
The visitors were briefly down to nine men just past the hour mark when substitute Colin Daniel received a head injury, but the defender returned with a bandage on.
Pompey, meanwhile, were being undone by too many sloppy passes that were halting their momentum in the final third.
And when Evans did whip in a dangerous delivery, Curtis and Pitman got in each other’s way as they both tried to attack the ball.
It had been a difficult evening for McCrorie and the on-loan midfielder made way for Cannon on 70 minutes.
Pompey did start to pick up the pace, however, and when O’Hara palmed away Curtis’ shot, Brandon Haunstrup’s fierce follow-up was blocked.
And the keeper made an even better stop moments later to keep out a goal-bound Curtis header from Haunstrup’s cross.
That was the left-back’s final contribution to the cause, though, as he was withdrawn for Harrison in an attacking move.
Cannon thought he should have won a penalty after going down under Akins’ challenge and Hicks took a moment before deciding there had been no foul.
Some patient build-up play then ended with Close slipping a pass through to Marquis, who spun and fired narrowly wide.
There were four minutes of stoppage-time for the Brewers to survive and they almost held on for the win.
But when Pitman lifted the ball into the box, Harrison was bundled to the turf by Jake Buxton and Hicks pointed to the spot.
It was PITMAN who stepped up to take the penalty and kept his cool, slamming the ball into the roof of the net to rescue a point.
Pompey (4-2-3-1): MacGillivray; Burgess, Downing, Naylor (c), Haunstrup (Harrison 77); Close, McCrorie (Cannon 70); Williams (Evans 46), Pitman, Curtis; Marquis
Goals: Curtis 39, Pitman 90+5 (pen)
Subs not used: Bass, Bolton, Walkes, Hawkins
Burton (4-2-3-1): O’Hara, Brayford (c), O’Toole, Nartey, Wallace; Edwards, Quinn; Sarkic (Boyce 77), Sbarra (Daniel 26), Broadhead (Buxton 83); Akins
Goals: Sbarra 3, Wallace 6
Sent off: O’Toole (two yellow cards)
Subs not used: Bywater, Hutchinson, Anderson, Fraser
Referee: Craig Hicks
Attendance: 16,610 (116 away fans)