Police said the fire may have been caused by sky lanterns launched to celebrate the new year.
Several witnesses reported that they had seen the cylindrical paper lanterns with little fires inside flying in the night sky shortly after midnight near Krefeld Zoo, said Gerd Hoppmann, the city’s head of criminal police.
“People reported seeing those sky lanterns flying at low altitude near the zoo and then it started burning,” Mr Hoppmann said.
Police and firefighters received the first emergency calls at 12.38am local time.
near the Dutch border said that the entire ape house burned down and more than 30 animals, including five orangutans, two gorillas, a chimpanzee and several monkeys, as well as fruit bats and birds, were killed.
Only two chimpanzees could be rescued from the flames by firefighters.
They suffered burns but are in a stable condition, zoo director Wolfgang Dressen said.
“It’s close to a miracle that Bally, a 40-year-old female chimpanzee, and Limbo, a younger male, survived this inferno,” Mr Dressen said, adding that many animal handlers were in shock at the devastation.
“We have to seriously work through the mourning process,” Mr Dressen said.
“This is an unfathomable tragedy.”
He also said that many of the dead animals were close to extinction in the wild.
The zoo said that in addition to the two rescued chimpanzees, the Gorilla Garden near to the Ape House did not go up in flames and that gorilla Kidogo and six other members of his family group are alive.
Mr Hoppmann said the use of sky lanterns is illegal in Krefeld and most other parts of Germany and asked that the people who launched them or people who witnessed anything should come forward to police.
Germans usually welcome in the new year with fireworks at midnight and people are allowed to buy and launch fireworks.
Sky lanterns, however, are both illegal and uncommon in Germany.
Sky lanterns, which are sometimes also called Chinese lanterns, are a sort of hot-air balloon made of paper.
They have been used in Asia for celebratory events for centuries.
Mr Hoppmann said investigators found some used lanterns on the ground that had not burned entirely.
They were 34cm (13.4in) long, made out of white paper with an opening at the bottom where a small fire would have been suspended.
The fire heats the air inside, making them fly and shine at night.
Mr Hoppmann said some of the lanterns had handwritten notes on them.
Krefeld Zoo was opened in 1975 and attracts some 400,000 visitors each year.