With so much history, that leaves plenty of room for tales of haunted buildings, ghosts and spooky apparitions.
Here are eight of the places that are said to be most haunted in the city.
Have you ever dared to visit them?
1. The naval base
The Lady of the Harbour is a ghost that is said to haunt the area around the naval base. According to the legend, she is the ghost of a naval widow and makes whimpering noises. She is often seen in dark corners or down alleyways.
The Froddington Witch used to haunt this area and was said to make a high pitched whistle, which would mesmerise passers-by. So if you ever hear any whistling in Fratton, this could be why...
3. Victoria Park
It is said that the park is haunted by The Vanishing Man who appears when the light is low. He is dressed in a rain coat, carries a suitcase and is thin and tall - he approaches you before disappearing into thin air.
4. Fort Widley
This Victorian Fort is said to be haunted by the spirit of a little boy. It is said that he was a drummer who fell to his death down the spiral staircase. Rapid footsteps have been heard running along the tunnels.