The new Rothschild's Bank building at New Court in London was also one of the six buildings on the Sterling Prize shortlist.
The ultimate winner of the Stirling architecture prize was the £82 million Sainsbury Laboratory, a new plant science research centre in Cambridge.
The judges described the laboratory as a timeless piece of architecture, a university building "taken to an extraordinary degree of sophistication and beauty".
When the shortlist was announced, just before the Games began in July, judges described the Olympic Stadium in Stratford as having "a spirit of fun".
They said the architects had "designed a space to create an amazing atmosphere, where every seat has a great view".
London's Olympic stadium has failed to secure its own gold medal, after missing out on the UK's most prestigious architecture prize.
The 80,000-capacity stadium was one of six buildings to be shortlisted for the Royal Institute of British Architects' (Riba) Stirling Prize. But in the end the gong went to a science laboratory in Cambridge.