The family of David and Sally Abel, from Northamptonshire, said they have now been moved to a “prison”-like hospital.
The Abels were on the cruise for their 50th wedding anniversary when it was placed into quarantine.
Mr Abel has now been diagnosed with acute pneumonia, while Mrs Abel has mild pneumonia.
The news comes as the remaining Britons from the ship returned to the UK on Saturday to begin a further quarantine period - the group all tested negative for the virus before flying back.
Although the couple were originally in a hospital just 90 minutes from the coronavirus-stricken cruise ship, Mrs Abel called her son in the middle of the night to say the couple were suddenly being moved to a different "three-star" hospital.
It took seven hours for the family to track down which hospital they had been taken to.
Princess Cruises have sent the Abels a mobile phone that can make calls to their family.
Steve Abel, their son, posted video updates on the couple’s YouTube channel alongside his wife Roberta on Friday night and Saturday morning.
Speaking on Friday evening, he said: "They are in a really bad way.
"Dad is very weak. He can’t walk around - he’s in a wheelchair now. I haven’t seen him in a wheelchair, but mum tells me he has to be put in a wheelchair to go from the ambulance to the hospital.
"They are really distressed. His exact words to me were, 'this has to stop now, we can’t take any more of this. It’s like a prison.'"
He said the pensioners were left in a hospital room that has no shower or bathing facilities, and only provided with basic paper towels: "They haven’t eaten properly. Neither of them can eat the food."
"They don’t know what’s happening, they haven’t had any treatments.
"They have been told they won’t get any treatments, but have been told they will see if they test negative for the coronavirus in two or three days and if they test negative twice they can go - but that doesn’t treat the pneumonia, which is what we are so worried about."
On Saturday morning Steve Abel posted a second update and said: "They have spoken to a doctor who can liaise with them a bit better.
"That is really reassuring and has done a lot for their mental well-being.
"They are brighter today and seem a lot happier, but they are still in the same situation."
The family have said they are willing to fly out to Japan to bring the couple home.
Roberta Abel said: "We are in fight mode now, we just want them back."
She criticised the reaction of the Foreign Office, and said their response had been "next to useless".
She said, in an update on Saturday morning: "We do not believe they are in the best possible place like the Foreign Office are telling people."
A spokesman for the FCO said: "The welfare of all British nationals is of the highest priority to the UK government.
"We are working with the Japanese authorities to ensure those British nationals who are remaining in Japan for health reasons get the best care possible."