Pompey scored and then conceded twice in quick succession to slip to defeat against Cheltenham at Whaddon Road.
They shook off a couple of refereeing decisions that went against them before the break to take the lead through Tom Bradbury’s own-goal.
But Liam Sercombe hit a three-minute double to stun the Blues and they could not turn the late pressure they applied into an equaliser.
John Mousinho made one enforced change from the side that beat Stevenage on home soil on New Year’s Day.
Alex Robertson missed out through injury, so Gavin Whyte played in the central attacking role behind Colby Bishop.
But there was a welcome return to the bench for Tom Lowery, who had been sidelined since the opening day of the season.
Pompey looked to make early inroads and when Sean Raggett got on the end of Marlon Pack’s deep cross, Luke Southwood had to get down to make the save.
But the visitors were also guilty of some sloppy play, with Raggett and Joe Morrell having to make crucial blocks after possession had been lost.
It was the Blues who were the more dangerous side going forward, however, and it took a fine Curtis Davies challenge to stop Bishop making the most of Paddy Lane’s centre.
A stretching Pack could then not quite make a clean connection with Abu Kamara’s low cross, allowing Southwood to save.
Pompey continued to apply pressure as the half continued and there were further half-chances for Morrell and Raggett.
There were then a couple of huge moments that ensured referee Simon Mather was not the most popular man with the visitors.
First, Kamara nipped in front of Tom Pett and was sent to the turf. Whether it should have been a penalty was unclear, but the yellow card shown to the forward for simulation was harsh.
A few minutes later, the rampaging Kamara was fouled by Elliot Bonds, with Mather immediately blowing his whistle.
The problem was that the ball had run straight to Lane, who lashed it into the net, leaving Mousinho fuming on the sidelines.
Half Time: Cheltenham 0 Pompey 0
The Blues were immediately on the attack following the restart, with Kamara controlling the ball and playing it through to Lane, whose fierce strike was saved by Southwood.
And they soon took the lead after their opponents had failed to deal with a throw launched into the box by Pack.
The ball was eventually steered towards Bishop by Raggett, only for Robins defender BRADBURY to slice it into his own net.
There was jubilation from the travelling fans, but that evaporated when Cheltenham hit back with two goals in quick succession.
This time it was Pompey’s turn to be undone by a long Will Ferry throw and Sercombe lashed the ball home.
And the same player then took advantage with a fierce finish after Conor Shaughnessy had been caught in possession by George Lloyd.
The visitors looked to hit back and a diving Raggett could not quite make a connection after Whyte’s teasing cross had been sent back across the face of goal by Joe Rafferty.
Bishop then felt he was being impeded as he tried to convert Shaughnessy’s flick from another Pack throw.
Pompey were applying plenty of pressure in search of an equaliser, but their opponents were doing a good job of protecting Southwood’s goal.
Mousinho made an attacking change on 77 minutes, with Jack Sparkes withdrawn and Christian Saydee on to replace him.
The newcomer soon picked out Bishop with a fine delivery into the box, but the striker was off balance and Southwood was eventually able to grab hold of the ball.
With time running out, Anthony Scully replaced Whyte to add some fresh legs to the visitors’ forward line.
Pompey poured forward in the closing stages and were left furious when Mather again decided that Pett’s challenge on Scully was actually a dive and brandished a yellow card.
And the Robins managed to hold on during the closing stages to boost their chances of beating the drop.
Cheltenham (3-1-4-2): Southwood; Smith, Davies, Bradbury; Pett; Long (c), Sercombe (Thompson 65), Bonds, Williams (Ferry 31); Lloyd, Keena (Butler-Oyedeji 80)
Goals: Sercombe 51, 54
Booked: Ferry, Pett, Bradbury, Long
Subs not used: Pardington, Horton, Liggett, King
Pompey (4-2-3-1): Norris; Rafferty, Raggett, Shaughnessy, Sparkes (Saydee 77); Pack (c), Morrell; Kamara, Whyte (Scully 84), Lane; Bishop
Goals: Bradbury 49 (og)
Booked: Kamara, Bishop, Rafferty, Lane, Scully
Subs not used: Schofield, Towler, Lowery, Devlin, Martin
Referee: Simon Mather
Attendance: 5,561 (1,517 Pompey fans)