But the USP boss felt his side were a little unfortunate with two offside calls in their 4-1 defeat.
US Portsmouth had taken a surprise lead against Pat McManus' men after just 13 minutes thanks to Tom Dinsmore's superb solo effort.
But Grice admitted his side struggled to contain Brockenhurst throughout, who moved back above Horndean to the summit courtesy of the victory.
Although the USP boss had reason to believe both the home side's second and third goals could have been offside.
Grice said: ‘I think if you break the game down - we went into the lead with Tom Dinsmore’s brilliant solo effort - but we just couldn’t contain them. In the first 10-20 minutes they were everywhere.
‘Our guys were a little bit like a rabbit in the headlights in the sense of they weren’t sure what to do.
‘It took us 20 minutes to realise their movement, which we picked up on and got better, when they’re that fluid the communication needs to be so good as you’re passing people on and it just wasn’t to begin with.
‘I believe - I’ve got no reason to disbelieve any of my players - they were going absolutely mad at the linesman (for Brockenhurst’s second goal). I couldn’t see it, but they were appealing for offside.
‘This isn’t sour grapes, the third goal our lads thought was massively offside - so there’s two goals which weren’t 100 per cent.
‘Let’s not get away from the main facts of the game, they were better than us, they were fitter than us and they moved the ball better than us.’
Dinsmore's superb solo strike handed the visitors a shock 13th-minute advantage.
But Brockenhurst, determined to keep up their Wessex Premier title challenge, turned things around and eventually ran out 4-1 winners with Kieran Dodd bagging a double while the league's leading scorer, Silvano Obeng and Connor Ings completed the scoring.