Taunton's Liberal Democrat MP Jeremy Browne was sacked to give other members of the party a turn in government, Nick Clegg said.
In a letter explaining why he had removed the MP from his job as Home Office minister, the Deputy Prime Minister said he was keen to "provide the opportunity for as many in our ranks as possible to contribute their skills to Ministerial office".
The decision to kick him out of government, particularly from such a high profile role, was one of the surprise moves of the reshuffle.
Mr Clegg wrote: "It is always very difficult to move colleagues out of government but as you know, I have always been keen that we provide the opportunity for as many in our ranks as possible to contribute their skills to Ministerial office during this Parliament
so that, just as the Government has benefited from your contribution over the past three years, it can also gain from those of other colleagues in the remaining years of this parliament.
"I am immensely grateful to you for your commitment and support over the past few years. You have made a major contribution to this historic coalition government and as one of the very few ministers who have served in two departments,
I have no doubt there will be an opportunity for your experience to be deployed in government in the future."
Mr Browne, who also served as a minister in the Foreign Office, said he remained supportive of the Government but cautioned against "lapsing into transactional trade-offs and deferred decision making".
He added: "I supported your election as party leader and I admired your decision to take our party into government.
"Easy and permanent opposition may be alluring to some but you are right not to find it seductive."
More top news
Rugby World Cup winner Phil Vickery talks about his World Cupexperiences and what he hopes this year's tournament will do for the game.
Rugby fans are eagerly awaiting the start of the Rugby World Cup but it's not just sports enthusiasts who will benefit this autumn.
The last pilots to train on Lynx and Sea King helicopters in Somerset have taken part in a parade in front of some special guests.