Portsmouth City Council called in support from a charity after being alerted to the mysterious deaths.
A team from Swan Support assessed the wellbeing of the remaining two swans and, concerned that they may also fall ill, decided to take them to a Berkshire rescue centre.
An RSPCA animal rescuer had previously been called out to a sick swan, who later died.
The vets couldn’t confirm what the problem was, said a spokesperson for the charity.
They added: ‘The swan was showing symptoms of poisoning, most likely environmental, such as botulism or blue-green algae or another source.’
The council has sent water samples to be analysed, while parks staff have increased their visits to the pond to check other wild birds.
One of the birds rehomed was a cygnet found to have a plastic ring from a bottle embedded in its beak.
Cllr Ben Dowling, the council's cabinet member for culture, leisure and economic development, said: ‘We're trying to find out what might have caused the swan deaths.
‘Sometimes you can tell there's a problem through signs of algae or a lack of oxygen in the pond but at the moment there's nothing obviously wrong.
‘I'm pleased they were able to help the cygnet by removing the plastic ring from its beak but it's terribly disappointing that we're still having to tell people don't drop litter.’
Visitors to Baffins Pond are advised to keep dogs on a lead to prevent any contact with the water.
Cllr Dowling added: ‘We're asking people not to feed any of the birds as this may not be helping their health. Bread isn't good for birds' diet anyway.
‘Animal welfare is our top priority here and while it's sad we don't have any swans at the pond now, it gives us an opportunity to look into the situation and I'm sure we'll have new swans back in the coming years.’
Baffins Pond’s wetlands area is overgrown and will be cut back as soon as possible, but legally has to wait until after the nesting season ends.
The RSPCA spokesperson said: ‘We’d ask anyone who sees any collapsed birds or any animal in distress to call the RSPCA’s emergency rescue line on 0300 1234 999.’