Paul Doswell's side scored three goals in 11 second-half minutes – with two from Jonah Ayunga – to complete a remarkable comeback victory.
The visitors raced out of the blocks and caught the Beavers cold to take a second-minute lead.
Anthony Straker hoisted a long free-kick in from the right which the home defence failed to clear.
Predicted Portsmouth XI v Cheltenham: changes expected to side that beat Cardiff so handsomely - but attacking duo could keep places and defender set to return
League One latest: Oxford land former Pompey striker target from Swansea as Sheffield Wednesday bemoan exciting duo's move to Brentford and former Peterborough boss criticises club
Portsmouth boss Danny Cowley hails Spurs loanee Dane Scarlett as move already shows signs of putting Miguel Azeez's short stint from Arsenal in shade
Portsmouth boss Danny Cowley spells out why Blues can rival Sheffield Wednesday, Ipswich and Derby for promotion as he prepares for welcome return to Cheltenham
Former Portsmouth keeper Alex Bass suffers tough night on and off the pitch as he makes Sunderland debut following Fratton Park exit
And when the ball fell to Deacon off a defender he crashed his low shot into the net.
Within five minutes, however, the hosts hit back to equalise after hesitation in the Hawks defence.
Dean Beckwith and goalkeeper Ross Worner left a through ball for each other, allowing Tyrone Lewthwaite to nip in and score.
Midway through the first half it took a good block with his legs by home keeper Laurie Walker to deny Jonah Ayunga at the edge of the area.
Further questions needed to be asked of the visitor’s defence in the 32nd minute when they fell behind.
A simple long ball over the top released Danilo Orsi-Dadomo, who showed great composure to beat the defence and score.
The start of the second half was delayed after a Hawks fan collapsed behind the goal at the bottom end of the ground.
When the referee eventually gave the go ahead for the game to continue it was the home side that grabbed the initiative.
Two minutes into the second half, Orsi-Dadomo grabbed his second goal.
Once again the Hawks contributed to their own downfall by losing possession just outside their own penalty area.
When the ball was pulled back, the home striker finished well from close range.
Doswell’s men looked shell-shocked and the Beavers twice went close to increasing their lead.
Wes Fogden got the visitors back into the game on 62 minutes with a good finish from just inside the area.
The fightback was on and Ayunga brought the Hawks level 10 minutes later with a thumping close-range header.
Within a minute Ayunga scored a dramatic winner by firing into the far corner of the net after good work from Fogden.
Other results (Wessex League premier division): Baffins Milton Rovers 2-3 Fareham; Horndean 1-1 Hamble