Pompey beat Altrincham at Fratton Park to book their place in Monday's draw for the third round of the Emirates FA Cup.
But it took a stoppage-time winner from Brett Pitman to seal a 2-1 victory against National League North opposition.
Ben Close had earlier opened the scoring for Kenny Jackett's side with a fine volley, only for Josh Hancock to level from the penalty spot.