Taunton's Liberal Democrat MP Jeremy Browne was sacked to give other members of the party a turn in government, Nick Clegg said.
In a letter explaining why he had removed the MP from his job as Home Office minister, the Deputy Prime Minister said he was keen to "provide the opportunity for as many in our ranks as possible to contribute their skills to Ministerial office".
The decision to kick him out of government, particularly from such a high profile role, was one of the surprise moves of the reshuffle.
Mr Clegg wrote: "It is always very difficult to move colleagues out of government but as you know, I have always been keen that we provide the opportunity for as many in our ranks as possible to contribute their skills to Ministerial office during this Parliament
so that, just as the Government has benefited from your contribution over the past three years, it can also gain from those of other colleagues in the remaining years of this parliament.
"I am immensely grateful to you for your commitment and support over the past few years. You have made a major contribution to this historic coalition government and as one of the very few ministers who have served in two departments,
I have no doubt there will be an opportunity for your experience to be deployed in government in the future."
Mr Browne, who also served as a minister in the Foreign Office, said he remained supportive of the Government but cautioned against "lapsing into transactional trade-offs and deferred decision making".
He added: "I supported your election as party leader and I admired your decision to take our party into government.
"Easy and permanent opposition may be alluring to some but you are right not to find it seductive."
More top news
Bob Crampton has some good news about the weather over the coming days - he even uses the words "unbroken sunshine".
Bath centre Sam Burgess has made it into the England squad for the Rugby World Cup just ten months after switching codes.
A former teacher in Bristol has been jailed after admitting making indecent images of scores of children using hidden cameras