The Addicks edged it 2-1 courtesy of strikes in each half from Joe Aribo and Lyle Taylor at The Valley on Saturday.
That helped fifth-placed Charlton close the gap on place-above Pompey to just two points, while Kenny Jackett’s men now trail the top two by eight points with just 10 games to play.
Republic of Ireland international Ronan Curtis’ thunderous header from full-back Lee Brown’s corner in the dying seconds of the first half had cancelled out Aribo’s opener on 41 minutes.
But it was the Addicks who would come out on top, with Taylor firing home six minutes after the restart.
Although the Blues were far from their fluent best, winger Jamal Lowe side-footed wide late on then had a goalbound strike blocked in stoppage-time as Jackett’s troops went in search of a leveller.
And Pompey centre-back Burgess said they should remain positive despite the defeat.
He said: ‘There were a couple of good chances towards the end.
‘I think we showed a lot of character to carry on battling all the way till the end and we could have got something.
‘They made a good block and one just went wide. We had a goal ruled out, which I think was offside, but it’s not like it was a bad performance.
‘There are positives we will take from it and I don’t think we’ve lost any sort of belief.
‘We conceded a little bit too early on in the second half.
‘But if you look at it the other way, it gives you a lot of time to then get back into it.
‘I thought we came on strong as the second half developed and it was an unfortunate goal we conceded.
‘Clarkey (Matt Clarke) tried to clear and it was a ricochet that fell nicely for them.
‘It wasn’t what we were looking for but there was no disgrace in the result and it’s fine margins in football.’
The win ensured Charlton completed the double over Pompey for the first time since the 2005-06 Premier League season after they won 2-1 at Fratton Park earlier in the campaign.
With defender Burgess impressed by Lee Bowyer’s men in both games.
He added: ‘Fair play to Charlton, they played well.
‘I was impressed with them earlier on in the season and they did the same again against us.
‘It was a good game overall that just didn’t go our way.’