A late Anton Walkes own-goal saw Pompey crash out of the Carabao Cup in the first round for a third successive season.
It looked like they would progress when Christian Burgess nodded in following a first half that was short on entertainment.
But Joe Pigott headed in the equaliser and, with just a few minutes remaining, Walkes found his own net.
Kenny Jackett made five changes from the side that won at Blackpool in the league at the weekend.
Brett Pitman returned to skipper the team, while there was a first Pompey start for on-loan winger David Wheeler.
Luke McGee, Burgess and Brandon Haunstrup were also called upon, with Craig MacGillivray, Jack Whatmough, Lee Brown, Ronan Curtis and Oli Hawkins all dropping to the bench.
The first half was a forgettable affair, with little to entertain the fans packed into the Fratton End and South Stand.
It was the visitors who had the first chance when Mitchell Pinnock crossed from the left, but Scott Wagstaff’s header was off target.
The same player then screwed another effort well wide after Walkes had failed to properly deal with Pinnock’s free-kick.
And it was a hat-trick of chances missed when Wagstaff beat Haunstrup to a header and sent the ball looping towards goal, only for the left-back to recover and hook it clear.
But it was the Blues who almost broke the lead on 23 minutes when Wheeler met Pitman’s centre and sent his header wide.
Neither McGee or opposite number Tom King had been called into action and that did not change when Pinnock’s drive flashed past the post.
Jamal Lowe was not too far off target following a surging run through the centre, but there was still little to whet the appetite.
Half Time: Pompey 0 AFC Wimbledon 0
Pompey had struggled to create anything meaningful before the break, but they soon took the lead after it.
Pitman did well to bring the ball under control inside the box, only for his shot to be deflected behind.
And the deadlock was broken from the resulting corner, as a delivery from Evans was emphatically headed home by BURGESS at the back post.
AFC Wimbledon thought they should have won a penalty when McGee and Pinnock collided soon after, but referee Ollie Yates was not interested.
They then came close to a leveller, as Anthony Wordsworth unleashed a rocket form 25 yards that fizzed over the bar.
Jackett made a double change on 67 minutes, with Pitman and Evans withdrawn for Conor Chaplin and Hawkins.
It had been a quiet evening for both keepers, but McGee was certainly not caught napping by a fierce Wordsworth drive that was well saved.
He could do nothing about the Dons’ equaliser on 76 minutes, though, as Pigott found space to meet Wagstaff’s cross and head home from just a few yards out.
That unsurprisingly gave the visitors a boost and they set about trying to find a winner, with Curtis brought on for Pompey to try to offer more at the other end.
But it was Wimbledon looking more threatening and when a corner was not properly dealt with, McGee had to get down and push Wagstaff’s shot around the post.
The Blues then almost retook the lead in a frenetic end to the game, but Hawkins was denied as he tried to convert Lowe’s centre.
Instead it was the visitors who grabbed a late winner – although they had their opponents to thank for the goal.
Tom Soares’ cross found Walkes at the back post and the defender tried to clear it, but only succeeded in planting the ball in the back of his own net.
Pompey threw bodies forward to try to force a penalty shoot-out, but they were unable to find a second goal.
Pompey (4-2-3-1): McGee; Walkes, Burgess, Clarke, Haunstrup; Naylor, Close; Wheeler (Curtis 80), Evans (Hawkins 67) Lowe; Pitman (c) (Chaplin 67)
Goals: Burgess 49
Booked: Close, Naylor
Subs not used: MacGillivray, Whatmough, Brown, May
AFC Wimbledon (4-4-2): King; Watson (Sibbick 77), Oshilaja (c), McDonald, Garratt; Wagstaff, Wordsworth, Soares, Pinnock (Appiah 66); Hanson, Pigott (Hartigan 90)
Goals: Pigott 76, Walkes 88 (og)
Subs not used: McDonnell, Nightingale, Purrington, Ashley
Referee: Ollie Yates
Attendance: 6,588 (438 away fans)