Boris Johnson has been compared to Adolf Hitler by the chairman of the Conservative Muslim Forum.
Mohammed Amin said he would quit the party he has been a member of for 36 years if the former Mayor of London became the new prime minister.
Mr Amin told BBC Radio 4's Today programme: "I am not prepared to be a member of a party that chooses him as its leader.
I would resign after 36 years.”
When quizzed about Mr Johnson's popularity with grassroots Conservative party members, Mr Amin said: "There are many horrible people who have been popular. Popularity is not the test.
“The test is, is this person sufficiently moral to be prime minister, and I believe he fails that test.”
Mr Amin added: “A lot of Germans thought that Hitler was the right man for them.”
When confronted that it was a shocking comparison to compare the former foreign secretary to Hitler, Mr Amin said: "Yes.
“I am not saying Boris Johnson wants to send people to the gas chamber, clearly he doesn’t. He’s a buffoon.
“But he, as far as I’m concerned, has insufficient concern about the nature of truth for me to ever be a member of a party that he leads.”
He added: “We don’t expect our politicians, our prime ministers, to be saints.
“But we do require a basic level of morality and integrity.
“And of all the candidates in the Conservative Party leadership election, Boris Johnson is the only one that I believe fails that test.”
Mr Johnson's column in the Daily Telegraph last year where he compared women who wore burkas to "letter boxes and bank robbers" had put Muslim women at risk, Mr Amin said.
He said: “He knew exactly what effect it would have – it would lead to Muslim women who wear niqab and burka being verbally abused on the streets; in certain cases being physically assaulted, with people trying to tear it off.
“He chose to mock Muslim women who wear niqab and burka for his own purposes."
Conservative MPs who support Mr Johnson's bid to become prime minister have been vocal in their support.
Former Cabinet minister Priti Patel told the BBC: "That article was Boris’s very clear defence of women’s rights to wear whatever they like, and it was not written to mock in the way in which was asserted.”
Mr Johnson has also been accused of trying to ducking debates and managing his media appearance to avoid scrutiny by Conservative party members.
Fellow leadership hopeful Jeremy Hunt urged Mr Johnson to take part in the upcoming debates due to take place on Sunday.
Six of the seven Conservative MPs still in the running to become prime minister will take part in the Channel 4 debate - with Mr Johnson the only person not to agree so far.
Mr Hunt said: "What would Churchill say if someone who wants to be prime minister of the United Kingdom is hiding away from the media, not taking part in these big occasions?”