Alex Pike’s strugglers are now four points adrift of relegation rivals Dunstable Town with only one game in hand.
Borough handed a debut to Bristol Rovers loanee midfielder Charlie Ten-Grotenhuis and a second bow to striker Rowan Vine, who returned to the club last week.
And the home side achieved their first objective – getting through the opening 10 minutes unscathed against the leaders.
However, unlucky for Borough, they fell behind on 13 minutes.
A cross from the right evaded the home defence and Mike Symons sent a back-header bouncing past the diving Patrick O’Flaherty and into the net.
O’Flaherty then rescued Borough by saving with his legs after his side had lost possession outside their area.
Borough’s first effort on goal arrived on 24 minutes – Ryan Pennery forcing a low save from Martin Horsell.
But it was only a brief reprieve as twice in the matter of minutes O’Flaherty produced flying saves to keep his side in the game.
Hereford were piling on the pressure but Borough continued to get plenty of players behind the ball to keep them out before the break.
The hosts continued to show plenty of resolve at the start of the second period.
And when the visitors broke through O’Flaherty again made a brave save with his legs.
There was a let-off as Symons fired wide from point-blank range after the ball was cut back from the byline.
But the hosts’ resistance was finally broken in the 64th minute when Jennison Myrie-Williams latched on to a through ball and drilled his shot into the bottom corner.
Borough found themselves pinned back now as Hereford poured forward with wave after wave of attacks.
The pressure was bound to tell and on 71 minutes Dara O’Shea crashed in a low shot from just outside the area.
A minute later Symons headed in the visitors’ fourth.
Gosport Borough: O’Flaherty, Wakley, Medway, Ten-Grotenhuis, Flooks, Warren, Lea, Brooks, McAllister, Pennery, Vine. Subs: Bombelenga, Smith, Sanders, Mayes, Watts