CCTV images of Mr Hancock and Ms Coladangelo kissing were published by The Sun on June 25.
A friend of Mr Hancock told MailOnline: “He is still very much together with Gina, but they are physically apart because of the intense media interest.”
His 15-year marriage to Martha has broken down in the wake of the scandal.
They have three children together.
Ms Coladangelo has three children with her multi-millionaire husband Oliver Tress who founded Oliver Bonas.
BBC radio presenter Jeremy Vine tweeted pictures of Mr Hancock boxing up his belongings.
He wrote: “This guy lost his family, his job, his home, his reputation, and may even lose his seat. There’s a life lesson here.”
Mr Hancock stepped down from his position as Health Secretary after the leak of CCTV images showing he had breached social distancing rules.
At the time, he said: “The last thing I would want is for my private life to distract attention from the single-minded focus that is leading us out of this crisis.
“I want to reiterate my apology for breaking the guidance, and apologise to my family and loved ones for putting them through this. I also need to be with my children at this time.
“We owe it to people who have sacrificed so much in this pandemic to be honest when we have let them down as I have done by breaching the guidance.”
Watch: Airline refunds - What are your rights as a consumer?'