A fourth-minute goal by the visitors stunned the home side who still trailed at the break.
But second-half goals from Jason Prior and substitute James Hayter sent the Hawks through to the next round.
Gosport Borough crashed out of the FA Cup after losing 2-1 in their tie against Swindon Supermarine.
Tyrell Mitford equalised for the home team after the break but a goal from a set piece gave the visitors the win.
Bognor came from behind to proceed to the next round of the FA Cup after winning 2-1 against Weston-super-Mare.
The Rocks went behind in the first half, but an Aden Baldwin own goal on 49 minutes gave them the equaliser.
The visitors were reduced to 10 men with minutes to go, before James Crane scored the winner in injury-time.
In the Southern League east division, three first-half goals saw Moneyfields run out 3-2 winners at Thame United.
Ryan Pennery opened the scoring on 12 minutes, before the home side equalised.
Goals from Steve Hutchings and Sam Pearce handed the lead back to Moneys.
Thame scored at the start of the second half but were unable to find an equaliser.
In the Wessex League premier division, Horndean lost 6-0 at Blackfield & Langley.
Petersfield also suffered a 6-0 defeat at Team Solent.
Fareham Town lost 2-1 in their match at Amesbury Town.
Truro City held out to beat AFC Portchester 2-0 with two late goals.
Baffins Milton Rovers slipped to a 2-0 defeat against Lymington Town.