A fire service spokesman said the whole of the roof of the museum and library was on fire.
Station Manager Peter Critchell, at the scene, said:
"Flames are coming through the roof and there is thick grey smoke billowing up. Firefighters are still working hard at the scene to try and bring the blaze under control. We now have water jets attacking the fire from the four corners of the building. We evacuated 30 people from inside the museum and firefighters will be making efforts to salvage as many of the artefacts as possible."
More top news
The 34-year-old from Coulsdon in Surrey died during the private party in Headley in the early hours of Monday, Surrey Police said.
The Home Office made the arrests at various restaurants across London on July 4.
Mayor of Newham Sir Robin Wales criticised the International Olympic Committee on the fourth anniversary of the opening ceremony.