Pompey scored late in extra-time to book their place in the second round of the Emirates FA Cup.
Sean Raggett bundled the ball over the line to ensure the Blues enjoyed a 3-2 victory over Ipswich at Portman Road.
Ronan Curtis and Tom Naylor had earlier given Kenny Jackett's side a commanding lead, but their League One hosts battled back through Jon Nolan and James Norwood.