The home side led 1-0 at the break courtesy of a Jonah Ayunga goal.
Early in the second half the Hawks had Danny Kedwell harshly sent off, however, which changed the whole complexion of the game.
Paul Doswell gave a debut to Millwall under-23 loan signing Junior Tiensa in place of the injured Anthony Straker.
The game opened in a quiet fashion, although the visitors threatened a couple of times on the break.
Denver Hume: Portsmouth's expensive forgotten man and why former Sunderland defender really could be one for the future
'Real driving force... lion-hearted display... classy presence' - Neil Allen's Portsmouth ratings from 3-0 Carabao Cup win at Cardiff
'We wanted to show him a bit of love' - Portsmouth-based Spurs fans' terrific gesture to ensure Dane Scarlett feels at home on the south-coast
League One latest: Portsmouth linked duo set for Oxford United and Lincoln moves, Southampton prospect joins Blues rivals as Ipswich land former Blackpool and Derby veteran
'We’ve got a real problem': Portsmouth boss reveals headache after career first at Cardiff
The Hawks remained patient and got the breakthrough they needed after 18 minutes after good work from Wes Fogden on the left.
The former Pompey midfielder chipped in a deep cross and AYUNGA headed in from close range after Danny Kedwell nodded back across goal.
In response Joe Iacfano sent a header wide of the upright when the hosts failed to clear an attack down the right.
Taking the lead had provided the boost the Hawks needed and Ayunga had another far-post header cleared from in front of goal.
Shortly into the second half, a neat flick from Kedwell sent Fogden racing away and Rutherford narrowly failed to meet his cross to the far post.
The Hawks’ task was made harder in the 53rd minute, however, when they were reduced to 10 men for the third successive home game.
Kedwell seemed to go up for a fair aerial challenge but the referee showed him a straight red card.
Fogden turned the ball narrowly past the near post as the Hawks continued to hold the upper hand.
Saints equalised on 65 minutes when substitute Sam MERSON rose above the defence at a corner to plant his header home.
The visitors grabbed the lead on 78 minutes when the Hawks were stretched by a quick break.
Iocafano sped through on the left and Zane BANTON easily turned the ball in to hand the hosts’ promotion aspirations a blow.