Video report by ITV News Political Correspondent Paul Brand
Boris Johnson has said his brother Jo "does not agree with me about the European Union because it's an issue that obviously divides families and divides everybody".
It comes after the universities minister, who backed remain in the EU referendum, said he's been caught in an "unresolvable tension" and now wants others to take on his roles as an MP and minister.
A Number 10 spokesman said Jo Johnson had been "a brilliant, talented minister and a fantastic MP", and the Prime Minister "as both a politician and brother understands this will not have been an easy matter for Jo".
Jo Johnson told ITV News: "It's been an honour to be MP for Orpington and a minister under three governments but it's time to move on."
Speaking on Thursday afternoon, after delivering a speech to police officers in Wakefield, Boris Johnson said he would rather be "dead in a ditch" than go to Brussels to ask for a further delay to Brexit.
He added he won't be the second member of his family to resign from Government, adding he is "absolutely determined to deliver on the mandate of the people".
Former Tory MP David Gauke, who had the whip removed after voting against the government, tweeted: "Lots of MPs have had to wrestle with conflicting loyalties in recent weeks. None more so than Jo
"This is a big loss to Parliament, the Government and the Conservative Party."
What options does the Government now have? ITV News Political Correspondent Carl Dinnen has been looking at their chances of success
Mr Gauke was one of 21 MPs to be deselected by the Conservative Party after rebelling against in the Commons this week.
Another was Anne Milton, who told ITV News "when your own brother walks away from your premiership, then it says something".
She said it will have "required a lot of heart searching and soul searching for Jo," adding, "it is damaging" for the prime minister, "you can't get away from that".
Watch the moment a police officer, who appears unwell, has to sit down behind Boris Johnson during his speech
Following the rebellion Prime Minister Johnson attempted to call a general election under the Fixed Term Parliaments act, but was unable to win a two-thirds majority.
The government will try again for a general election on Monday, Commons Leader Jacob Rees-Mogg said.
It comes after former Labour MP Luciana Berger joined the Liberal Democrats, saying they are “the strongest party to stop Brexit”.
Business Secretary Andrea Leadsom tweeted: "It has been a pleasure to work with @JoJohnsonUK, both in Parliament for nine years and most recently as a Minister at BEIS; his expertise and knowledge of the area were a huge asset to the department. I wish him all the best."
Following Mr Johnson's resignation, Labour deputy leader Tom Watson tweeted: "I wonder how many Tory MPs are wishing @theresa--may was still Prime Minister this morning?"
Labour's shadow education secretary Angela Rayner said: "Boris Johnson poses such a threat that even his own brother doesn’t trust him."
Robert Peston explains why Jo Johnson's quitting is significant for the prime minister
In November last year, before his brother took over in Number 10, Mr Johnson resigned as transport minister in Theresa May's government over the handling of Brexit.
He quit over Mrs May's proposed deal, saying the situation at the time meant the UK was facing either "years" of economic uncertainty or a no-deal scenario.
He said at the time: "We are barrelling towards an incoherent Brexit that is going to leave us trapped in a subordinate relationship to the EU with no say over rules that will govern huge swathes of our economy."
He added: "So great is the gulf now between what was promised in the referendum campaign and what is on offer in the prime minister's proposed deal that I have had no choice but to submit my resignation."