With trade growing through the port, bosses have decided it is time to upgrade the existing building that went up in 1976 and design a new one.
The first step has been to appoint Halcrow Group as a master planner to review its current passenger and freight facilities and to develop the plans for a new modern terminal to enhance the port's services.
The port said this would also meet anticipated passenger growth through to 2040.
Ferry port manager Phil Gadd said: 'A new terminal and the new services it will offer to our customers are essential to meet anticipated growth in the coming years.
'The master plan will enable us to ensure minimal disruption to our existing customers and our local environment throughout the design and build process.'
Halcrow is looking at the whole layout of the port in order to decide where a new terminal building should go before presenting this to staff and port customers such as Brittany Ferries and LD Lines prior to any designs of a new terminal building.
Construction will not start until the end of next year and would probably be funded by the city council, costing several millions.
The port is handling growing traffic both on the freight side and on the passenger side.
LD Lines is renting a second vessel for the Le Havre route and Gefco is importing thousands of new Citroen cars through the port.
Overall port manager Martin Putman said any port needed a terminal, and this was a one in 30-year opportunity to redesign the traffic layout of the ferry port.
He added: 'The terminal was temporary but was done extremely well.
'The outside of the terminal is dated by anyone's standards. The inside still gives a reasonable level of comfort for foot passengers, but it's time to move on and the city council fully supports that.'
Rachel Fowler, master planner at Halcrow, said: 'The key to this development is keeping the whole port operational while the new terminal is built, which is quite a challenge.
'That is why there is so much preparation and research to be done before the design and construction of the new terminal commences.'