The Londoner made 49 appearances for the Blues, scoring one goal, after moving to Fratton Park from Huddersfield in 2001.
Vincent joined Derby in 2004 – one of nine Football League clubs he turned out for during a career that spanned 19 years.
His last club was Didcot Town when he announced his retirement in 2012.
He played more than 500 senior games and had a combined transfer fee of £1.325m.
After starting his career at Palace, Vincent also played for Walsall, Millwall, Yeovil, Swindon and Aldershot.
Paying tribute to Vincent on Twitter, former Blues team-mate Kevin Harper wrote: ‘Such sad news, brilliant team mate my thoughts and prayers go out to his family and friends r.i.p’.
Pompey legend Alan Knight told the Pompey website: ‘Jamie was a really nice lad and a big part of the team spirit at the club.
‘He had a bit of a chequered time at the club, but always gave 100 per cent whenever he was picked.
‘I was goalkeeping coach at the time he was at Pompey and was able to get to know him well off the pitch.
‘He worked well with the young players, while he kept his head down and worked extremely hard. He was a model professional.
‘I’m just in shock because it’s no age at all. My thoughts go out to his family at what must be an incredibly difficult time.’