The Bantams caretaker boss believes his side did not deserve to concede a penalty in their 5-1 loss at Fratton Park.
With the game goalless, Kenny Jackett’s side were awarded a spot-kick in the 22nd minute after Oli Hawkins was held back inside the box.
Gareth Evans stepped up to steer his effort into the top corner.
Pompey were rampant from thereon in, with a Ben Close double, along with goals from Tom Naylor and Jamal Lowe, ensuring the hosts picked up a first League One victory since New Year’s Day.
But Drury felt referee Oliver Langford’s decision to give the Blues a golden chance to break the deadlock was ‘soft.’
He told the Bradford Telegraph & Argus: ‘I thought we started reasonably well and contained Portsmouth.
‘I saw a little bit of grit and determination early on.
‘But when you’re low on confidence, the psychological side of the game is very important.
‘We got sucker-punched with the penalty, which I thought was soft. But you could see the confidence go from the players.
‘If we’d got to half-time at 1-0, we could have reassessed the situation and gathered our thoughts.
‘But we then conceded one of the softest and poorest goals from a set-piece.’
As things stand, relegation looms for Bradford.
They sit second bottom and are six points from safety.
Drury admits the Bantams group has a ‘soft underbelly’ and Pompey deserved to pick up all three points.
He added: ‘We’ve been saying it all season but there’s been a real soft underbelly to the group.
‘We’ve shown that we can create and score goals but we’re leaving ourselves too much to do.
‘I feel for the supporters. They are an unbelievable group and they deserve better.
‘It was a very difficult afternoon but I don’t think it was (down to) a lack of effort from the players.
‘I know it can look like that from the outside but they care.
‘It didn’t come down to effort but being beaten by a far better team.’