Gideon Bull said on Friday that now is not the best time to stand as a candidate after he was accused of making a reference to Shylock, the Jewish moneylender character in William Shakespeare's play The Merchant Of Venice, during a Haringey Council cabinet meeting in July.
He reportedly made the remark to fellow cabinet colleague, councillor Zena Brabazon, who then made a complaint.
Mr Bull, who is the London borough's cabinet member for local investment and economic growth, denies the accusation.
"The allegation that I called a Jewish cabinet member 'Shylock' is entirely false," he said in a statement on Friday.
He said he used an analogy when referring to a housing decision and was not referring to the councillor.
"When she politely informed me that this saying was offensive, I immediately apologised and explained that I did not know that Shylock was Jewish and I would never have mentioned Shylock if I had known this," he said.
"I grew up in a working class area in Ilford where this was a common saying, but I didn't know it was offensive.
"This was a genuine accident and I reiterate my sincere apology for this mistake."
The Campaign Against Antisemitism's head of political and government investigations Joe Glasman said Mr Bull's comment is "unacceptable".
"Gideon Bull's comment, in which he called a Jewish fellow councillor 'Shylock', after the villainous Shakespearean Jewish character, was unacceptable, and of course he had to withdraw his candidacy," Mr Glasman told the PA news agency.
"Not a day goes by without a revelation about a Labour candidate's troubling record in relation to Jews, exemplifying the institutional anti-Semitism of the party under the leadership of Jeremy Corbyn."
Mr Bull is the third Labour candidate to step down in one day.
Scottish candidate for Gordon Kate Ramsden quit after a controversial blog emerged in which she compared Israel to a child abuser.
Her resignation came at the same time as Labour lost another candidate - Frances Hoole in Edinburgh South West - following a controversial tweet.
The Labour Party and Ms Brabazon did not respond to requests for comments.
The Tories have also suffered difficulty over a controversial candidate.
Former BBC Radio Norfolk presenter Nick Conrad was selected on Wednesday to stand for Broadland despite the controversy over his comments in 2014 saying women should “keep your knickers on”.
However he was forced to drop out the race after details of the incident resurfaced.
Following criticism of his selection, Mr Conrad said he would be stepping aside as the media attention was becoming a "distraction".