Paul Mepham forced himself on his victim for 30 minutes, ignoring her desperate pleas to stop.
The 62-year-old kissed the woman before groping her breasts and trying to force his hand between her legs.
Portsmouth Crown Court heard the woman has been left feeling ‘dirty’ and was devastated by the attack.
Mepham had walked through her unlocked door and into her bedroom at night.
Speaking in court, the victim’s granddaughter said: ‘My nana told me that she believed he was going to rape her.’
She added: ‘My family are very concerned there may be other victims who are too unwell to report this to the police.’
Mepham, formerly of Waterlooville, was jailed for three years after admitting sexual assault.
He had claimed the pensioner was ‘encouraging him’, the granddaughter said in court.
‘Due to the fact that he waited until she was confused and incapacitated, we believe that he is a danger to society,’ the victim’s granddaughter added.
The victim, who has vowed never to return to her home, has told family she is scared of sleeping as she just sees Mepham walking towards her.
Prosecutor Matthew Lawson said Mepham had asked if could ‘stroke her to sleep’ before he eventually left her, leaving two cans of Guinness behind.
‘She then heard the door shut and thought to herself “thank God he’s gone”,’ Mr Lawson said.
Jailing Mepham for three years, judge Claudia Acker said the ordeal must have felt like a ‘lifetime’ for the victim.
The judge said: ‘This would have been a terrifying ordeal for her and it’s no wonder that at that point in time she thought she was going to be raped.’
Mitigating, Howard Barrington-Clark said: ‘I can’t explain the inexplicable.’
Mr Barrington-Clark added: ‘He did stop, he immediately said it was inappropriate and he has used that word all along.
‘I can only express the shame and remorse and the true, true regret for what he did.’
Speaking after the case, Detective Constable Amanda Layton said: ‘This was a disturbing, albeit rare, case which involved an assault on an elderly and vulnerable woman.
‘I hope that now Mepham has been jailed, the victim and her family have closure.’