Borough are facing a £40,000 hole in their budget following the withdrawal of sponsor Jim Fallon on the eve of the season.
The last thing they needed was to miss out on the financial boost of a good run in the FA Cup, too.
The defeat proved hard for manager Alex Pike to take as he sought a quick exit – leaving assistant manager Mick Catlin to deal with the media.
Catlin admitted the defeat was a big blow to the club.
Portsmouth transfer news: Blues receive bids for former Luton and Millwall midfielder
Transfer gossip: Portsmouth eye raids for ex-Chelsea man and Charlton Athletic's former Wigan Athletic midfielder
'Is proving one hell of a signing.... A real livewire' - Portsmouth match ratings from 2-0 win at Cheltenham
Former Portsmouth pair help Ipswich flex muscle as Norwich City man is slammed for conduct in Peterborough's Plymouth Argyle reverse
Cheltenham boss Wade Elliott praises experienced Portsmouth in 2-0 defeat at Whaddon Road
He said: ‘Our chairman has already stated there is a big hole in our budget.
‘Getting through this tie would have gone some way towards helping plug that.
‘Lots of clubs have found to their detriment not to rely too much on Cup money.
‘Hopefully that is not the case with us.’
The tie started well enough for Borough with Josh Carmichael giving them an 11th-minute lead.
The former Bournemouth youngster capitalised on a missed punch by the Terras goalkeeper to head Gosport in front.
Borough had further chances to extend their lead before Weymouth hit back with a 33rd-minute equaliser.
The visitors failed to deal with a looping cross from the left and Stuart Yetton headed home.
The goal came against the run of play but Yetton went close again on the stroke of half-time.
The striker found space inside the area and Gosport keeper Martin Rice had to turn his shot around the post.
‘Weymouth’s first goal seemed to knock the stuffing out of us,’ added Catlin.
‘We struggled to get any parts of our game going after that.
‘In the second half Weymouth came at us and scored twice in quick succession.
‘We just didn’t respond and when our second came it was far too little, too late.’
Borough’s second arrived in the final minute with Dan Wooden firing home after a square ball from Ollie Bailey.
Caitlin admitted Borough have big problems to solve if they are to avoid a troubled season.
Their away form is a dilemma, while they have shipped 10 goals in their last three games. Catlin knows that has to change.
‘Our biggest problem is the number of goals we are letting in,’ he said.
‘It doesn’t matter what competition you are in – if you concede goals like we have then you are not going to win games.
‘If we don’t sort it out then it will be a long hard season.
‘On the back of a very difficult week we are going to need strong characters.’