- Video report by ITV News correspondent Paul Davies
Phil Dampier, author of Royally Suited: Harry and Meghan In Their Own Words, said the Windsors are concerned about the direction the couple are heading in.
"There’s no doubt the Queen and other royals are very worried about the direction Harry and Meghan are taking and it’s very serious."
Mr Dampier told the PA news agency he could foresee a situation in which Harry and Meghan would step back from royal duties and away from the royal family.
"A year ago I would have said no, but now I wouldn’t be surprised."
The couple are said to be planning a six-week holiday and should use it to take stock of their approach, Mr Dampier said.
"I think if they have a six-week holiday and have a break, hopefully they might come to their senses and realise it isn’t working for anyone," he said.
"It’s storing up long-term problems. There’s nothing to be gained unless they’re paving the way to leave."
Buckingham Palace declined to comment.
The couple opened up candidly about the pressure of their royal lives in the ITV documentary Harry & Meghan: An African Journey.
Meghan admitted feeling vulnerable, and spoke of the difficulty in coping with the intense tabloid interest, saying: "It’s not enough to just survive something, that’s not the point of life. You have got to thrive."
The duke, asked about an alleged rift with the Duke of Cambridge, said he and his brother are now "on different paths" and have "good days" and "bad days" in their relationship.
He also told of the pressure he felt trying to protect his family from unwanted media attention.
The couple’s high-profile overseas visit to Africa, carried out at the request of the Government, was overshadowed on the penultimate day when the duke launched a scathing attack on the British tabloid press, as news of Meghan’s lawsuit became public.
The duchess is suing the Mail on Sunday over an alleged breach of copyright and privacy after it published a private letter between her and her estranged father.
The Mail On Sunday said it stands by its story and will be "defending this case vigorously".
Harry later filed his own proceedings at the High Court against News Group Newspapers, which owns The Sun and the now defunct News of The World, and Reach plc, which owns the Daily Mirror, in relation to the alleged illegal interception of voicemail messages.