Prince Andrew Speaks Out for the First Time Since Epstein Scandal to Mourn Father Prince Philip

STEVE PARSONS/POOL/AFP via Getty Prince Andrew speaks during a television interview outside the Royal Chapel of All Saints in Windsor on April 11, 2021, two days after the death of his father, Prince Philip

Prince Andrew joined other members of the royal family in mourning his father, Prince Philip, who died on Friday at the age of 99.

Speaking in public for the first time since he stepped away from public duties in the wake of his disastrous BBC interview in Nov. 2019 regarding his relationship with disgraced financier Jeffrey Epstein, Prince Andrew remembered his father as "the grandfather of the nation."

Listen below to the episode of our daily podcast PEOPLE Every Day for continuing coverage in the wake of Prince Philip's passing.

"I loved him as a father. He was so calm…He was always someone you could go to and he would always listen," Andrew, 61, told reporters at a special service at the Royal Chapel in Windsor Great Park on Sunday.

He added, "We've lost almost the grandfather of the nation."

RELATED: Princess Anne Pays Tribute to Her Father Prince Philip: He Was 'My Teacher, My Supporter and My Critic'

Prince Andrew also thought of his mother, Queen Elizabeth, who was married to Prince Philip for 73 years.

"I feel very sorry and supportive of my mother, who's feeling it, I think, probably more than everyone else," he said.

"The Queen, as you'd expect, is an incredibly stoic person. She described his passing as a miracle. She's contemplating, I think is the way that I would put it," Andrew continued. "She described it as having left a huge void in her life, but we, the family, the ones that are close to her, are rallying around to make sure we are there to support her."

Tim Graham/Getty Images Prince Philip and Queen Elizabeth

Prince Andrew's fall from the public face of the royal family began in Nov. 2019 when he was interviewed by the BBC about his friendship with Epstein, who died by suicide in jail in August 2019 while awaiting trial on multiple sex charges, and allegations that he had sex with Virginia Roberts (now Virginia Giuffre) when she was 17 years old, a claim Andrew denies. In the furor over the days that followed, the Queen's son issued a statement saying he was "stepping back" from royal duties.

"It has become clear to me over the last few days that the circumstances relating to my former association with Jeffrey Epstein has become a major disruption to my family's work and the valuable work going on in the many organisations and charities that I am proud to support," he said in the statement.

Prince Andrew joined his three siblings and other family members in speaking about Prince Philip's death over the weekend.

RELATED: Prince Philip's Final Moments Were 'Very Peaceful' and 'So Gentle,' Says Daughter-in-Law Sophie

Prince Charles, Queen Elizabeth and Prince Philip's eldest child, released a video message on Saturday.

"My dear Papa was a very special person who I think above all else would have been amazed by the reaction and the touching things that have been said about him and from that point of view we are, my family, deeply grateful for all that," the 72-year-old heir said. "It will sustain us in this particular loss and at this particularly sad time."

On Monday, Prince William shared a personal message and a never-before-seen family photo of his son Prince George with Prince Philip. The sweet photo, which shows the young prince reading a book as he sits beside his great-grandfather in his carriage, was taken by Kate Middleton in Norfolk in 2015.

"My grandfather was an extraordinary man and part of an extraordinary generation. Catherine and I will continue to do what he would have wanted and will support The Queen in the years ahead," William said. "I will miss my Grandpa, but I know he would want us to get on with the job."

Prince Harry, who has returned to the U.K. from California to attend Philip's funeral on Saturday, also paid tribute to his grandfather on Monday.

"To me, like many of you who have lost a loved one or grandparent over the pain of this past year, he was my grandpa: master of the barbecue, legend of banter, and cheeky right 'til the end," Harry said.

Phil Walter/Getty Prince Harry and Prince Philip

Can't get enough of PEOPLE's Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

"Grandpa, thank you for your service, your dedication to Granny, and for always being yourself," Harry also wrote. "You will be sorely missed, but always remembered—by the nation and the world. Meghan, Archie, and I (as well as your future great-granddaughter) will always hold a special place for you in our hearts."