It was a goal that set Pompey on their way to suffering a first defeat under new head coach Danny Cowley.
And Charlie Daniels was annoyed with the manner in which Burton were able to break the deadlock just before the interval at Fratton Park.
A long throw caused chaos on the box and there were three chances for the visitors to find the net, with Mike Fondop eventually getting the job done.
Joe Powell’s long-range finish then doubled their advantage early in the second half, ensuring a late header from Daniels was no more than a consolation.
The left-back said: “We’ve been on such a good run since the new manager’s come in, so of course it’s disappointing.
“The most frustrating thing is the goals that we’ve conceded and we knew that Burton would chuck in a lot of long throws.
"We’ve been on such a good run since the new manager’s come in, so of course it’s disappointing."
“When the ball bounces in your box, you know you’re in trouble. Nobody really attacked it and anything can happen – unfortunately for us, it went into the net.
“But I like the way the Cowleys go about their work and you can see from the past few games that they want attacking, possession-based football.
“It’s something that I’m used to and – with four wins out of four – it was something the lads have clearly taken to.
“We need to make sure that this result is nothing more than a blip and if we do that, then everything’s looking good.”
Daniels was brought on midway through the first half after Lee Brown limped off with a hamstring problem.
He said: “You have to be fully prepared when you’re on the bench and do what you can to try to make a difference.”