Two goals in the last four minutes condemned the Hawks to a 2-1 defeat against Bishop’s Stortford in the Skrill South.
Christian Nanetti’s first-half goal looked like sealing victory for Lee Bradbury’s men.
But headers from Sean Francis and George Sykes buried them at the death.
Gosport’s winless start to life in the Skrill South continued after they were beaten 2-0 by Maidenhead United.
Alex Pike’s side were powerless to stop the visitors deservedly extending their unbeaten start to the season.
Reece Lascaris and former Hawks striker Richard Pacquette got the goals.
Horndean sealed their place in the second qualifying round of the FA Vase with a late 2-1 victory against Team Solent.
Wessex League premier division Deans left it late against division one rivals Solent.
Tom Woodward’s double sealed their progress to the next round.
Phil Mitson had equalised for Solent with 10 minutes to play.