Lee Bradbury was left ‘aggrieved’ as Whitehawk snatched a late double to sink the Hawks at Westleigh Park.
Goals from Brian Stock and Matt Paterson had seen the home side bounce back from going a goal behind in the National League South clash.
But Whitehawk turned the game on its head in the final seven minutes – netting twice to win 3-2.
Hawks boss Bradbury said: ‘I thought we did more than enough to win the game.
‘I was pleased by our enthusiasm and energy – partcularly in the second half.
‘We started the game well enough, only to find ourselves a goal behind.
‘We had good chances but then let them back into it through lapses of concentration.
‘I feel aggrieved that they nicked all three points with two late goals, which were poor from our point of view.
‘We also had a perfectly good goal ruled out early on when our lad was at least two yards onside.’
Bradbury made one change to his starting line-up – with Shamir Mullings replacing James Hayter up front.
Meanwhile, the hosts included two new faces on the bench – Romauld Boco and Lanre Azeez.
Boco signed for Pompey earlier in the day and was immediately sent out to the Hawks on a month-long loan.
Azeez, who has had spells at Maidenhead, Margate and Ebbsfleet, appeared as a non-contract player.
Mullings thought he had given the home side a 10th-minute lead when he thumped home a header only to be denied by an offside flag.
The visitors made the most of that escape and took the lead three minutes later.
Nick Arnold sent a superb ball forward to send Shamir Fenelon clear down the right. And when the Hawks failed to cut out his low cross, Danny Mills was left with the simple task of knocking the ball into the empty net.
It was all too easy for the visitors and as the half progressed it was largely one-way traffic.
But against the run of play, the Hawks levelled in the 38th minute.
Stock was the scorer as he fired the ball under the advancing goalkeeper from the edge of the area.
Pompey loanee Boco replaced Adrian Clifton for the second half.
Five minutes into the second period Ryan Young produced a remarkable one-handed save to turn Sam Deering’s wicked swerving shot against the bar.
Whitehawk were again looking the greater threat as they passed the ball around with confidence.
Again it was the Hawks, however, who broke out in the 64th minute to make it count.
Boco picked out Paterson with a raking crossfield ball and the striker linked with Stock before firing into the bottom far corner.
But it all began to unravel seven minutes from the full-time whistle.
Visiting substitute Jake Robinson equalised with a rising shot after the ball fell to him just inside the area.
And a minute into stoppage-time, Scott Neilson picked the ball up on the edge of the Hawks’ box and scored with a low shot into the corner.
The home side were devastated as Whitehawk turned the tables but it was difficult to begrudge the visitors their win after a thrilling clash.
Hawks: Young, Medway, Cummings, Stock, Strugnell, Monthe, Clifton, Donnelly, Paterson, Morgan, Mullings. Subs: Hayter, Blanchett, Azeez, Swallow, Boco