Having trailed 1-0 at half-time, Lee Bradbury's troops hit back to lead 3-1 through goals from Wes Fogden, Alfie Pavey and Theo Lewis.
However, a controversial penalty turned the tables back in favour of the hosts who soon levelled before Kevin Lokka netted a late winner.
The defeat meant the Hawks slipped to 22ndÂ in the table '“Â with just two points from their opening four fixtures at National League level.
On Saturday, the Westleigh Park outfit travel to Braintree, who are level on points and one place above in the standings.
Woman, 44, whose large Dobermann dog mauled girl, 3, on Leigh Park street released back into neighbourhood by police
WATCH: This is the moment a man with a grudge fire-bombed a family's garden in Leigh Park
Four people hospitalised after 'serious' multi-vehicle crash involving lorry on A27 near Chichester
Council refuses to close Gosport splash park despite it leaking 'thousands of litres of water'
Community in horror after little girl suffers ‘life changing injuries’ after being attacked by Dobermann dog in Leigh Park
Bradbury made two changes for the trip to Dover '“Â with Dan Strugnell andÂ Lewis replacing Bradley Tarbuck and Pavey, respectively.
And the Hawks began well, with a dangerous cross from Strugnell plucked out of the air by the home keeper following good work from Nicke Kabamba and Rory Williams.
Dover quickly got up to speed, though, asÂ Jamie Allen got the wrong side of Paul Robinson before forcing Ryan Young into a good save.
Three minutes later the striker shot inches wide from 20 yards as the hosts threatened again.
Shortly before the half-hour mark, Inih Effiong turned well to meet a right-wing cross but Young was equal to his shot.
And the Hawks keeper was on hand soon after to make another save from the former Ross County forward.
The visitors had a half-chance on 34 minutes when a Williams cross found Strugnell and his shot was only parried by the keeper.
However, Fogden was ruled offside as he followedÂ up.
And straight from the free-kick the Hawks fell behind.
The ball fell to Nortei Nortey and the former Chelsea youngster's 25-yard effortÂ curled over Young.
ThereÂ was still time for the Hawks to threaten before the break when Fogden was fouled on the left.
The free-kick was well worked for Lewis to fire in a shot and while the Dover keeper could only parry again, the Hawks were unable to take advantage.
Pavey was introduced at the break in place of Andreas Robinson as Bradbury hunted for a way back into the match.
The Hawks came out all guns blazing for the second half and it only took them two minutes to equalise.
Fogden's shot was saved but ballooned up and the former Pompey man gratefully headed home.
Just 60 seconds later the visitors were ahead when substitute Pavey nodded in a Williams cross.
Three minutes later Fogden won a corner, Strugnell picked up the ball out on the left to centre.
Kabamba pushed the ball back across goal and there was Lewis to head home to make it 3-1.
Dover pulled one back on 67 minutes from a controversial penalty award '“Â with Allen beating Young from the spot.
The assistant referee appeared to flag for a foul outside the box before the referee pointed to the spot shortly afterwards.
This spurred Dover on and after 70 minutes the Hawks were indebted to Kabamba for heading a corner off the line.
Another corner after 78 minutes saw the ball bouncing from head to head before it fell to Mitch Brundle, who shot home to level from 20 yards.
And as the Hawks were pushing forward, a counter-attack from the hosts forced another corner on 86 minutes.
The ball fell toÂ Lokka in the penalty area and he made no mistake inÂ poking the ball home to put Dover in front.
The visitors still pushed forward and in the final minute of injury time, Lewis fired a free-kickÂ inches over the crossbar.