"He has been warned to keep his head down. Charles does not want Andrew to be photographed every other day looking happy and waving as he is driven to the castle.
"Eventually Andrew will be made to leave Windsor and could get several million pounds from the royal family. But he has made it clear that he will hang on for dear life."
Charles reportedly feels like he was "misled" when Andrew said the scandal involving him, Jeffrey Epstein, and sex-trafficking victim Virginia Giuffre would go away.
Reports have not ruled out the possibility of Ferguson remarrying Andrew - they have both lived together at Royal Lodge since 2008 despite their divorce in 1996.
Andrew will reportedly give evidence under oath in March after a judge ruled the sexual assault lawsuit brought against him by Giuffre would go to trial.
Andrew has repeatedly denied all allegations against him and is preparing to fight the case in court.
He was also stripped of his royal title and honorary military titles by the Queen.
However, he will reportedly join Queen Elizabeth at Prince Philip's remembrance service.
The Duke of York - who no longer uses "His Royal Highness" in an official capacity and had his patronages and military titles withdrawn ahead of his civil case in the US - is expected to attend the event at London's Westminster Abbey on March 29 along with other members of the royal family.
A source confirmed to The Sun newspaper, "Prince Andrew is expected at the service of thanksgiving as he is a son of the Duke of Edinburgh".
- Additional reporting Bang! Showbiz
