David Seeckts, Head of Manufacturing Midlands at Premier Foods said the 511 jobs were going at Birmingham because the firm lost a significant bread contract for 2013 and because of general competition and the nature of the industry.
The Birmingham site opened in 1964. There are 323 production jobs and 188 in distribution.
The plan is for the closure to go ahead by the second quarter of next year.
More top news
Armed police have carried out a raid apparently linked to Wednesday's deadly terror attack in Westminster.
Six addresses have been searched as part of police investigations so far and seven arrests have been made.
A large part of Hagley Road in the south west of the city was cordoned off and later reopened.