Jackett's almost four-year reign in the Fratton Park hot seat came to an end on Sunday.
Against the backdrop of the Blues picking up four points in seven League One games and falling out of the play-offs, Saturday's humbling defeat to Salford in the Papa John's Trophy final at Wembley proved the final straw.
Assistant-boss Joe Gallen took charge of Pompey's narrow 1-0 defeat at Peterborough last night.
Brown, who was brought to Pompey by Jackett from Bristol Rovers in the summer of 2018, revealed the dressing room has to shoulder a significant amount of the blame for the recent run of form.
But the vice-skipper admits the Blues now have to focus on the future with 12 games to go and promotion hanging in the balance.
Brown told The News: ‘Us as players have let him down. You can look at the manager but we have let him down. In fact, if any manager gets sacked then the players have got to look at themselves.
‘That's what players have done. Okay, he's gone but we have still got to look at ourselves in the changing room.
‘You saw a bit of a reaction last night in terms of the attitude. There was a bit of fight but we have still got to improve and start winning football matches.
‘Most players have spoken to the manager and said what they've had to say. There is no going back now, what has been done has been done.
‘The manager has been let go through a bit of everyone's fault. It's not just Kenny Jackett it's the players as well. It's everyone.
‘But it's about looking forward now. We can't be down in the dumps for too long as we are in and around the play-offs.’
Danny Cowley and Daniel Stendel are the current front-runners to succeed Jackett as Pompey boss.
Both have been offered short-term deals for the remainder of the season.
Regardless of who gets the job, Brown believes the players need to get themselves right before the new manager arrives.
He added: ‘We have got to get our own house in order first.
‘As players, you have got to look at yourself and get your own house in order regardless of the managerial situation.
‘Joe took charge last night so as players, we need to look after ourselves and make sure we're right for whoever comes in.’
A message from the Editor, Mark Waldron
The News has launched a subscription offer which gives you unlimited access to all of our Pompey coverage, starting at less than 14p a day.