Many of the victims died as they jumped from the building to escape the flames, a police official said, while witnesses added the death toll could rise.
The cause of the blaze was not immediately clear.
Bangladesh has around 4,500 garment factories that make clothes for brands including Tesco, Wal-Mart, JC Penney, H&M, Marks & Spencer, Kohl's and Carrefour.
Ready made garments make up 80 percent of the country's $24 billion annual exports.
More top news
Police are investigating the death of a teenager who died shortly after an amateur boxing match held last night.
Labour leader says the UK "should not be afraid of debt or borrowing" as he urges the government to "stop dithering" and start investing.
A fundraising page for a Westminster terror attack victim has been taken over by JustGiving amid fears it was set up by a fraudster.