Kal Naismith capitalised on a defensive howler to ensure Pompey saw off Plymouth at Fratton Park.
He snatched the ball from goalkeeper Remi Matthews and prodded home midway through the first half.
It was a deserved victory for the Blues, even if there were a few nervy moments towards the end of the contest.
Kenny Jackett made three changes from the side that were beaten at Peterborough earlier in the week.
Conor Chaplin replaced Curtis Main up front, while full-backs Nathan Thompson and Brandon Haunstrup came in for Matty Kennedy and the suspended Christian Burgess.
Evans continued to wear the captain’s armband in Brett Pitman's absence – rainbow coloured on this occasion to show the club’s support for LGBT inclusivity – but was pushed into a more advanced role.
There were plenty of familiar faces in the opposition line-up, with Sonny Bradley, Toumani Diagouraga and Jake Jervis all starting, while Nadir Ciftci was on the bench.
After a fairly uninspiring opening, it was the hosts who began to apply pressure as the first half progressed.
The first decent chance fell to Brandon Haunstrup after he was fed by Chaplin and the left-back’s fierce drive was just about palmed over by Matthews.
Matt Clarke – playing in a more familiar central role – then did well to burst forward and square the ball for Evans, whose shot was deflected behind.
Chaplin also had a snap-shot from long-range that was straight at Matthews, but fumbled nervously over by the stopper.
But the Blues were almost caught out at the other end when Yann Songo’o was left in space in the box to meet David Fox’s free-kick, only to guide his header wide.
Instead it was Pompey who broke the deadlock midway through the half – thanks to a huge error from Matthews.
The keeper came out of his area to try to collect Clarke’s long punt forward, but was tackled by the alert NAISMITH.
Matthews tried to tug him back, but the Blues forward kept his composure to poke the ball over the line.
Evans soon had a shot that swerved wide, but Plymouth began to pose some problems as the interval approached.
They were not really creating any scoring chances, although Luke McGee showed good timing to come off his line and dispossess Joel Grant.
Half Time: Pompey 1 Plymouth 0
There was an immediate chance for Pompey after the restart, but although Naismith’s shot had plenty of power, it was straight at Matthews.
And the keeper made amends for his earlier error with a stunning save to prevent Naismith from turning in Chaplin’s low cross.
Oliver Hawkins then had a shot deflected behind from the resulting corner, while Plymouth nervously dealt with another set-piece moments later.
But Jackett was forced into a switch just before the hour mark when Hawkins pulled up injured and had to limp off.
Kennedy was brought on to replace him, with Evans moving to right-back and Thompson switching to the centre.
Pompey gave away some sloppy free-kicks in the quest to protect their lead, as Argyle tried to get men forward.
They almost scored a second on 75 minutes, though, when Songo’o diverted Kennedy’s teasing delivery narrowly past his own post.
And there was another great opportunity when Chaplin stole possession and slipped Naismith through on goal.
Matthews came out of his box and the pair collided, with the ball rolling against the post before play was stopped so both players could receive treatment.
Naismith was initially cleared to continue, but soon had to depart and was replaced by Stuart O’Keefe.
There was a hint of nervousness creeping into Pompey’s performance as the seconds ticked down, but the visitors were unable to capitalise.
One throw did cause problems in the Blues back-line, although the danger passed when Diagouraga swiped at thin air.
The display of six additional minutes certainly caused some nerves among the Fratton faithful, but the hosts saw out the closing stages to secure the victory.
Pompey (4-2-3-1): McGee; Thompson, Hawkins (Kennedy 59), Clarke, Haunstrup; Close, Rose; Evans (c), Chaplin (Main 90+1), Lowe; Naismith (O’Keefe 83)
Goals: Naismith 24
Booked: Kennedy
Subs not used: Bass, Talbot, May, Bennett
Plymouth (4-1-4-1): Matthews; Sawyer (c), Songo’o, Bradley, Taylor-Sinclair; Fox; Carey (Lameiras 79), Sarcevic (Ainsworth 86), Diagouraga, Grant (Ciftci 71); Jervis
Booked: Sawyer, Diagouraga
Subs not used: Cooper, Miller, Wylde, Fletcher
Referee: Robert Madley
Attendance: 17,923 (1,870 away fans)