The captain found space in the box twice and netted each time.
His winner arrived in injury-time when it looked like a disappointing draw was on the cards.
For Kane it was an excellent start to his World Cup goal quest and a positive outcome for the team who play Panama next in group G on Sunday.
The Spurs striker said: ‘It’s massive. I’m so proud of the lads. It was tough.
‘I thought we played really well, especially in the first half.
‘We deserved to be ahead, could have scored a few more and then obviously I’ve not seen the penalty back but from my point of view it didn’t look like one.
‘But credit to the lads, we kept going and that’s what these games are about.
‘It’s the World Cup, you go until the last second and we’re absolutely buzzing.
‘We spoke about togetherness and we have got a great bond on the pitch so it’s nice to get the first game and see it working on the pitch as well.’
Gareth Southgate’s team started well and Kane tapped in the opening goal on 11 minutes after John Stones’ header was well saved.
But Tunisia equalised on 35 minutes when Kyle Walker was penalised for catching Syam Ben Youssef.
The penalty was scored by Ferjani Sassi.
Kane was denied a blatant penalty in reply.
After the break Tunisia looked increasingly comfortable.
They seemed to be coping well defensively and England ran out of energy and attacking drive.
But in injury-time Trippier’s corner was delivered for Harry Maguire to head on and Kane was in space to divert his header into the net.
Kane was particularly pleased England didn’t get ruffled by the decisions that went against them in the match.
They could have let their heads drop and had to settle for the draw.
But they fought on to the end in search of a winner.
That determination paid off with the Spurs striker’s late header.
He said: ‘We are delighted to get the win. It could have been a couple of penalties.
‘We had so much joy from corners, they were trying to stop us running.
‘It shows character to get on with it and that is what we have done.
‘We have done a lot of work on the training field but it’s tough, especially the first game.
‘We did really well and I’m so proud of them.’