Many of the victims died as they jumped from the building to escape the flames, a police official said, while witnesses added the death toll could rise.
The cause of the blaze was not immediately clear.
Bangladesh has around 4,500 garment factories that make clothes for brands including Tesco, Wal-Mart, JC Penney, H&M, Marks & Spencer, Kohl's and Carrefour.
Ready made garments make up 80 percent of the country's $24 billion annual exports.
More top news
Manchester United want Renato Sanches and Everton slap £50m price tag on Ross Barkley with Spurs interested.
Shoppers who don't bring their own bag will need to buy a bag for life costing 8p or 10p.
Temperatures will reach around 28 Celsius (82F) at their highest.