Two judges at the High Court in London have rejected Qatada's challenge against his continuing detention in a high-security Worcestershire prison.
The Special Immigration Appeals Commission (Siac) ruled two months ago that the detention of Qatada, who is fighting deportation to Jordan, was lawful.
Leaving him free during heightened security for the 2012 Olympics would be "exceptionally problematic", a judge said.
After the decision was announced, lawyers for Qatada indicated they would consider taking the case to the Court of Appeal once they have seen the court's reasons for its ruling.
A Home Office spokeswoman said the intention is to remove Qatada, referred to in court as Omar Mahmoud Othman, "as quickly as possible".
During the High Court hearing, his lawyers said the Islamic cleric had been detained for seven years, "the longest period of administrative detention, so far as we know, in modern English history."
"Our submission is that the detention has already gone on for so long as to be disproportionate and unlawful."
Qatada was described in one court judgment as "a truly dangerous individual" who was at the centre in the UK of terrorist activities associated with al Qaida.