Dan Holman’s late goal saw National League club Boreham Wood knock League One side Blackpool out of the FA Cup.
Pompey target Bruno Andrade played the full match for the non-league side who came from behind to get the impressive victory.
Danny Philliskirk latched on to Nathan Delfouneso’s pass to fire the Seasiders into a second-half lead.
Substitute Blair Turgott converted Kane Smith’s cross from eight yards to make it 1-1 soon after.
Fellow substitute Holman struck from close range with just two minutes remaining as the hosts produced one of the biggest shocks of the first round.
The Seasiders, who won the FA Cup in 1953, wasted a number of chances before the break with Callum Cooke having a shot blocked by keeper Grant Smith as well as hitting the post.
Boreham Wood manager Luke Garrard would have taken a replay but he was thrilled with the outcome and is now hoping for more wins to create a memorable Cup run.
He said: ‘When it went 1-1 I looked at the clock and I thought if the referee blew the whistle then, I would be happy.
‘But we looked strong, we looked fit, so credit to the players.
‘It’s about the glory of beating a league side for the first time in our history. It puts a marker down, and looking at what Sutton and Lincoln did last year, why not?’
Woking earned an FA Cup first-round replay against managerless Bury after Jamie Philpot’s header secured a 1-1 draw.
Millwall loanee Philpot’s first-half header for the National League side keeps them in the hat.
League One outfit Bury, who sacked former boss Lee Clark last month, went ahead after just 42 seconds through ex-Blues striker Michael Smith but could not hang on at the Kingfield Stadium.
League Two side Accrington were also forced into a replay after a 0-0 draw at National League club Guiseley.
Former Pompey loanee Jayden Stockley’s second-half double saw Exeter beat Heybridge Swifts 3-1 at St James Park.
The Isthmian League division one north visitors held out until the 59th minute before two goals in four minutes from Stockley.
Sam Bantick made it 2-1 but Paul Tisdale’s Exeter made it through after Liam McAlinden added a third.
Wycombe also eased past Solihull Moors 2-0 thanks to goals from Nick Freeman and Craig Mackail-Smith.
Swindon avoided an upset at Dartford after racing into a 3-0 lead inside half an hour following Timi Elsnik’s double and Harry Smith’s drive.
Former Blues player Amine Linganzi added a fourth two minutes after the break and Paul Mullin made it 5-0 before Richard Sho Silva grabbed a consolation for Dartford.
Elsewhere, Jordan Ponticelli’s first-half double eased 1987 winners Coventry into the second round as they beat Maidenhead 2-0.
Charlton also ended Truro’s hopes of an FA Cup giant-killing as they won 3-1 at The Valley.
Ben Reeves’ double and Mark Marshall won it for the Addicks while Tyler Harvey netted for Southern League premier division side Truro.
Leatherhead and Billericay drew 1-1 as Billy Bricknell’s penalty earned a replay for Billericay, who included Paul Konchesky – a beaten finalist with West Ham in 2006 – in their line up.
Jack Midson’s header had given the home side the lead.
On Saturday Slough were convincing 6-0 winners at National League North side Gainsborough, with Matt Lench scoring a hat-trick.
Wigan and Blackburn had to come from behind to see off Crawley and Barnet and fellow League One club Peterborough drew 1-1 with Tranmere.