A Ronan Curtis brace was not enough to keep Pompey in the Carabao Cup, as Newport mounted a second half comeback at Rodney Parade.
The visitors created an abundance of chances before the break, with Curtis netting either side of a Will Evans finish.
But Aaron Wildig drew the Exiles level soon after the restart and with a penalty shoot-out approaching, James Waite came off the bench to send the League Two side into round three.
Danny Cowley made eight changes from the side that beat Bristol Rovers in the league at the weekend.
Josh Griffiths, Connor Ogilvie and Curtis were the only players to keep their place in the starting line-up.
Skipper Clark Robertson made his first appearance of the season after recovering from a foot injury and was joined in defence by Zak Swanson and Haji Mnoga.
There were also returns for Reeco Hackett, Jay Mingi and Tom Lowery in midfield, while Joe Pigott partnered Dane Scarlett up front.
And with nine substitutes allowed to be named in the competition, academy duo Josh Dockerill and Adam Payce were included on the bench.
The first half was played at an electric pace, with the action quickly switching from one end to the other and no time to catch breath.
It was the hosts who started brighter and when Cameron Norman lifted the ball into the box, a header from Priestley Farquharson was saved by Griffiths.
But the deadlock was broken by the Blues on nine minutes, as a pass from Lowery sent CURTIS racing clear.
He sprinted from inside his own half to bear down on goal before stroking a cool shot past Joe Day and into the net.
They almost doubled the lead when Curtis fed a neat pass through to Scarlett, but Day was quickly off his line to make the save.
And Newport levelled just seconds later, as Evans played a one-two with Chanka Zimba inside the box and rifled past Griffiths.
Some sloppy play at the back from both sides was only serving to add to the excitement that was on display.
Pompey went back in front midway through the half, with a lovely move that started with a neat Mingi shimmy.
That provided the space to release Scarlett on the right and although his cross was just out of reach for Pigott, the ball landed for CURTIS to slam home his second of the night.
The attacks kept coming and Lowery combined with Scarlett before playing a one-two with Curtis, then laid a pass across for Pigott to smash a shot against the bar.
A change was forced upon Cowley on 33 minutes, with Swanson appearing to pick up a knock and being replaced by Kieron Freeman.
Scarlett was proving to be a particular menace to the Exiles defence and drew another good stop from Day after bamboozling Declan Drysdale.
Half Time: Newport 1 Pompey 2
Pompey had finished the first half in the ascendency, only to find their lead wiped out soon after the restart.
They lost possession in the middle of the pitch and Newport broke, with Adam Lewis sending in a cross that Wildig stooped to convert.
The Blues tried to hit back immediately and when Curtis cut in from the left, his low drive was well saved by the busy Day.
But they had gone a bit ragged and lost some of the rhythm that had made them such an attacking threat before the break.
Cowley freshened things up with a double switch just past the hour mark, as Lowery and Scarlett were replaced by Marlon Pack and Michael Jacobs.
And a further two changes followed soon after, with Owen Dale and Sean Raggett coming on for Hackett and Robertson.
With the hosts also switching things up, the contest inevitably became a bit disjointed as the half progressed.
But one of the Newport introductions gave them the lead on 74 minutes, as Waite collected a pass from Evans and found the bottom corner from the edge of the box.
Pompey tried their best to force an equaliser in the closing stages, but the Exiles held on to progress to the third round.
Newport (4-2-2-2): Day; Norman, Farquharson (Demetriou 61), Drysdale, Ad.Lewis; Wilmott (c), Bowen (Moriah-Welsh 70); Wildig (Waite 70), Collins; Zimba (Bogle 83), Evans
Goals: Evans 14, Wildig 49, Waite 74
Booked: Willmott, Evans, Drysdale
Subs not used: Townsend, Aa.Lewis, Stokes, Dolan, Nevers
Pompey (4-4-2): Griffiths; Swanson (Freeman 33), Mnoga, Robertson (c) (Raggett 68), Ogilvie; Hackett (Dale 68), Mingi, Lowery (Pack 63), Curtis; Scarlett (Jacobs 63), Pigott
Goals: Curtis 9, 25
Booked: Raggett, Mingi, Curtis
Subs not used: Oluwayemi, Rafferty, Dockerill, Payce
Referee: John Busby
Attendance: 3,866 (911 Pompey fans)