The Privett Park side led at the break through a Ben Wright strike, before the visitors hit back early in the second period.
Substitute Pat Suraci restored Borough’s lead, but they we’re soon pegged back, with Whitehawk then snatching the win with a stoppage-time penalty.
The visitors started well, with Frankie Sutherland flashing a shot past the far post.
Borough were struggling to get anything constructive going – and they had Ed Sanders to thank after he rescued them just before the break by flicking the ball away at the far post.
Borough took the lead a minute before the break through WRIGHT.
The striker was fouled outside the area and his free-kick deflected off the defensive wall and into the far corner.
The home side made a double change at the break, sending on Suraci and coach Danny Thompson.
Whitehawk levelled five minutes into the second half when Danny MILLS fired home from eight yards.
SURACI restored Borough’s lead on 58 minutes with a cool finish.
Four minutes later Mike WEST equalised with a terrific 20-yard curling shot into the top far corner of the net.
SUTHERLAND then won the game for the visitors with a stoppage-time penalty.