After establishing a comfortable 24-8 lead early in the second half the visitors were guilty of taking their foot off the gas.
The home side reduced the deficit to two points with 14 minutes left to play.
In the remaining time Havant had to show their defensive mettle to keep the hosts out.
Head coach Knight was relieved to come away with the win.
Most Popular
-
1
Portsmouth transfer news: Blues receive bids for former Luton and Millwall midfielder
-
2
'Is proving one hell of a signing.... A real livewire' - Portsmouth match ratings from 2-0 win at Cheltenham
-
3
Transfer gossip: Portsmouth eye raids for ex-Chelsea man and Charlton Athletic's former Wigan Athletic midfielder
-
4
Portsmouth boss Danny Cowley hails squad depth as tactical masterclass at Cheltenham pays off
-
5
14 splendid photos from Portsmouth’s 2-0 win against Cheltenham in sun-soaked League One encounter
He said: ‘We were nowhere near our best and to be honest it was a pretty dire game of rugby.
‘It was a workmanlike performance which got the job done, no more than that.
‘The game isn’t one we are particularly pleased about.
‘Sometimes you have to accept not playing well but coming away with the points.’
Havant led 12-8 at half-time with tries from James Wise and Scott Morris.
Further tries at the start of the second period from Joel Knight and Wise looked to have put the visitors into complete control.
Will Knight added: ‘At that stage I thought we would push on.
‘Whether the players thought the job was done I don't know.’
A depleted Southsea Nomads maintained their pressure on the top three in Hampshire two with a 24-17 win at Stoneham.
The visitors, badly hit by injuries and unavailability, fielded Chris Marsh and Ashley Emery, both for their first senior games in three seasons.
It turned out to be a painful return for Marsh who suffered a snapped Achilles’ tendon.
Nomads overcame their problems to secure an excellent win.
Captain Ben Horrod said: ‘It was always going to be a tough day when you have a bare 15 players for a match but all the lads played for the shirt and for each other.
‘Even when we were a man down Stoneham still couldn’t cope with our attacking play at times.
‘It was a battle for 80 minutes so it feels even better to come out on top.’
Tries were scored by Gavin Fewell and Gregory Stewart, who got two, with Horrod adding a penalty and two conversions.
Man of the match was awarded to Josh Soper for his constant attacking threat, tireless work-rate and his excellence at the breakdowns.