Katrice disappeared from a supermarket near a British military base in Paderborn, Germany, on November 28 in 1981.
She went missing on her second birthday near the base where her 69-year-old father Richard Lee was based, together with her now Gosport-based mother Sharon and 45-year-old sister Natasha.
She was out shopping with her mother and aunt Wendy at the Naafi store close to where her retired sergeant major father was stationed.
Military police arrested a man in connection with her disappearance on Monday and began searching a terraced house in the Moredon area of Swindon, Wiltshire.
An Army spokesman said: 'A former serviceman was arrested by the Royal Military Police on Monday, September 23 in connection with the disappearance of Katrice Lee.
'He has now been released without charge. The search of an address in Swindon continues and the Royal Military Police are keeping the Lee family informed of developments.'
Gosport MP Caroline Dinenage, who has supported the family, said: ‘It is a potentially major development in what has effectively been a living nightmare for Katrice’s parents and sister for the last 37 years.
‘There have been so many ups and downs and false hopes in this tragic case over the years.
‘My one hope is that some answers can finally be found.’
Unmarked police vehicles and officers arrived on Abbey View Road at about midday on Monday.
One woman said: 'They searched the cars on the drive and took one of them away.
'This is a nice area - you don't expect anything like this.'
Four unmarked white vans were near the property on Wednesday, with officers in black uniforms seen entering and leaving through two doors at the front of the house.
There was police tape across the drive, where a red Suzuki Alto was parked.
A blue and white tent was erected in the garden on Wednesday morning.
As reported, Katrice's father, based in Hartlepool, told the Mirror that the arrest made it 'feel raw'.
He told the newspaper: 'As with parents in all cases of missing children, we want a happy ending but that might not be the case and we just hope we will get answers.'
In a post on Twitter, the Help Find Katrice account said: 'Following recent events in Swindon we will be making no comments until we know for sure what is happening.
'The family ask that people refrain from speculation at this time so as not to jeopardise any future proceedings. #WeLiveInHope #KatriceLee.'
Royal Military Police launched a re-investigation, named Operation Bute, into Katrice's disappearance in 2012.
Last year, a forensic search took place on the bank of the River Alma, near where Katrice was last seen, but nothing was found.
Katrice went missing while shopping for treats for a birthday party.
Her mother realised in the checkout queue that she had forgotten crisps and asked her sister to watch Katrice.
When she returned in what she estimated was less than a minute, Katrice was gone.
Her sister said Katrice had run after her and she believed they were together.