US Secretary of Defence Chuck Hagel said it was unfair to jump to conclusions about released captive Sgt Bowe Bergdahl until the circumstances around his capture by the Taliban had been reviewed.
Mr Hagel said:" "We don't do that in the United States. We rely on facts.
"It's not my place as a former sergeant in the Army to decide who's worthy of being a sergeant and who isn't."
Sgt Bergdahl had been held in Afghanistan for five years but was released on Saturday amid speculation that he might face desertion charges.
Mr Hagel said the army would review all the circumstances surrounding how he left his unit and was captured by the Taliban.
Pentagon spokesman John Kirby told NBC News there was "no reason to doubt the authenticity" of the clip purporting to show US soldier Sergeant Bowe Bergdahl, who has spent five years in Taliban captivity, being handed back into US custody.
He said: "US officials are currently reviewing the video, but our focus right now is in getting Bergdahl the care he needs."
A video shot by the Taliban appears to show US soldier Sergeant Bowe Bergdahl, who has spent five years in Taliban captivity, being handed back into US custody.
The footage shows Bergdahl sitting in a silver pick-up truck before getting out of the vehicle with a plastic bag and being frisked before being taken away in a helicopter.
Armed men can be seen on the hillside above him as one of his escorts waves a white flag.
The US Army may still pursue an investigation that could lead to desertion or other charges against Sergeant Bowe Bergdahl, who was freed from five years of Taliban captivity in a prisoner exchange last weekend, General Martin Dempsey, chairman of the Joint Chiefs of Staff said.
Speaking publicly for the first time about the case, Gen Dempsey said he does not want to prejudge the outcome of any investigation or say anything that might influence a commander's decision.
But he said US military leaders "have been accused of looking away from misconduct, and it's premature" to assume they would do so in Sgt Bergdahl's case, despite the soldier's five years as a Taliban prisoner.
Hillary Clinton has defended the Obama administration's controversial decision to swap five Guantanamo Bay detainees for a US soldier held hostage in Afghanistan, saying many of America's allies made similar deals.
The former US secretary of state, said she did not second-guess people who made such tough decisions, but the American tradition of caring for its citizens and soldiers was a "noble" one.
"This young man, whatever the circumstances, was an American citizen - is an American citizen - was serving in our military. The idea that you really care for your own citizens and particularly those in uniform, I think is a very noble one," Clinton said.
She also noted that countries like Israel made similar swaps, citing its decision to exchange more than 1,000 Palestinian prisoners for one of its soldiers in 2011.
It is surprising former prisoner of war Army Sergeant Bowe Bergdahl has not spoken to his family, despite being freed on Saturday, a former Gulf War hostage told Good Morning Britain.
Retired RAF navigator John Nichol, who was captured in Iraq after his plane was shot down in 1991, said he found it "quite curious" Bergdahl had not been able to speak to his parents.
"That was the first thing that we wanted to do - and I was only a prisoner for several weeks not for five years."
The family of US soldier freed from Afghanistan have said they have not talked to him yet, but said he is 'resilient'. Army Sergeant Bowe Bergdahl, needs time to "decompress" his family said.
The sole American prisoner of war held in Afghanistan was flown to a US military hospital in Germany.Army Sergeant Bowe Bergdahl had been held for nearly five years. He will be treated in Germany before being flown back to the United States.
US Defense Secretary Chuck Hagel said he hopes a prisoner-exchange deal with the Taliban that led to the release of Sergeant Bowe Bergdahl after five years might lead to breakthroughs in reconciliation with the Afghan militants.
Bergdahl, the last US prisoner of the Afghan war, was handed over to US special forces yesterday in an exchange swap for five Taliban detainees who were released from Guantanamo Bay prison and flown to Qatar.
Hagel said: "This was a prisoner of war exchange ... our focus was on the return of Sergeant Bergdahl and maybe this could provide some possible new bridge for new negotiations."
"The timing was right, the pieces came together," he added.