To compound their misery, the hosts also had defender Harry Medway sent off and missed a late penalty.
Borough remain rooted to the bottom of the Southern League premier division.
They are four points behind nearest rivals Dunstable Town, who were beaten by Weymouth.
In that respect, nothing has changed in the fight for survival.
Denver Hume: Portsmouth's expensive forgotten man and why former Sunderland defender really could be one for the future
'Real driving force... lion-hearted display... classy presence' - Neil Allen's Portsmouth ratings from 3-0 Carabao Cup win at Cardiff
Cardiff 0 Portsmouth 3: Neil Allen's verdict - From Fratton boos to 'Just like watching Brazil' as brilliant Blues produce remarkable cup shock
Portsmouth transfer news: The four Fratton Park departures Blues could still sanction before window shuts - including ex-Sheffield United defender and Luton midfielder
'We’ve got a real problem': Portsmouth boss reveals headache after career first at Cardiff
But Borough are running out of games in their bid to avoid the one relegation spot.
Vine, meanwhile, blasted the Borough defending that saw them go in at the break four goals behind Redditch on Saturday.
‘We didn’t defend properly and that was very disappointing,’ said the former Pompey striker, who is joint caretaker manager.
‘It gave us a mountain to climb in the second half.
‘We rallied, scored two goals and at that point felt quite confident.
‘Their fifth goal, however, took the wind out of our sails.
‘Having a player sent off and missing a penalty didn’t help.’
After nine minutes Borough paid the price for poor marking when visiting captain James Mutton was allowed far too much time and space to fire home.
It got progressively worse when they conceded another three goals before the break.
Borough were quick out of the blocks at the start of the second half – scoring twice after handing a debut to ex-Reading midfielder James Harper off the bench.
Ed Sanders began the fightback with a long-range strike and Ben Wright then thundered in a second – his first goal since returning to the club last week.
A goal from a 63rd-minute counter-attack dashed Borough’s hopes, though.
Then Medway was sent off for a late challenge which led to a fracas on the touchline.
‘Harry slid into their player late and suddenly he is surrounded by four or five of their lads,’ added Vine.
‘I am not sure how much contact their actually was.
‘It is disappointing because we face losing our captain for up to five games.
‘From our point of view we must patch ourselves up and be prepared to go again.
‘It is disappointing we have one less game in which to pick up the points we need.
‘This was one we felt we could get something from.
‘We must take the small positives from this game and that was the reaction we made in the second half.’