Police and security chiefs "can never completely remove" the threat of disruption at London 2012 after the swimmer brought havoc to the Boat Race, British Olympic Association chairman Lord Moynihan has said.
In four months' time, crowds will line the route at a string of Olympic events including the open water swimming in Hyde Park as well as the marathon, triathlon and road cycling.
Lord Moynihan, a former Olympic rowing silver medallist, said "every conceivable scenario" was being reviewed to ensure the Games ran smoothly but conceded: "It just takes, and is likely to be, one idiot."
British Olympic boss Lord Moynihan says security in the wake of the boat race protest will be a"major challenge". @londontonight
"It is likely to be someone similar to the idiot yesterday who causes major disruption.
"That is why all the security measures need to be put in place to minimise the chance of that happening.
"You can never completely remove it but you can do everything possible to protect the interests of the athletes by minimising it."
He added: "It is so important to work closely with police and security agencies and to put in place every possible measure to protect the interests of all the athletes.
"In many respects that is the biggest ask of the Games: you are not just talking about the competitions, you are talking about the pre-Games training camps, athletes will come well in advance based around the country, you've got the torch relay coming up, the public need to be protected.
Activists have told me plans are afoot to disrupt Olympic events "in an annoying way".
"It is a major challenge and the Government have been aware of this since day one and work closely with Locog (the Olympic organising committee). Every conceivable scenario is being reviewed and I'm confident no more can be done.
"It's not impossible but it is a major challenge.
"You can never get it perfect unless you remove all the crowds and nobody is going to dream of doing anything like that."
Meanwhile, police have charged a man with a public order offence after "possibly the most dramatic Boat Race in history" was temporarily halted by a swimmer who appeared to deliberately cross the path of rowers.
Trenton Oldfield, 35, narrowly avoided the blade of an Oxford oar as he swam into the path of the vessels between the two and three-mile marker while the university crews were neck and neck on Saturday afternoon.
Oldfield, from east London, was held in custody at Chiswick police station before being released on bail later that night.