Shadow home secretary Andy Burnham has said Labour should act quicker to deal with anti-semitic comments.
Ken Livingstone has been suspended by Labour for "bringing the party into disrepute" amid an anti-Semitism row.
"These allegations, when they are surfacing, have not been dealt with properly and quickly enough. They need to be dealt with much more speedily in the future," Mr Burnham told BBC1's Question Time.
Labour MP Wes Streeting, the vice chairman of the all-party parliamentary group against anti-Semitism, warned that inaction by the leadership was damaging the party's reputation.
"What has been most frustrating when it comes to dealing with individual examples of anti-Semitism in the Labour Party has been the flat-footed response of the Labour Party which has given the impression that we are apathetic to tackling anti-Semitism or indifferent," he said.
"This ostrich strategy cannot be allowed to continue. Our credentials as a party of tackling racism in all its forms is taking a battering."
Former shadow home secretary Yvette Cooper has said that suspensions are not enough and the Labour party needs an "action plan" to tackle alleged anti-Semitism within its ranks.
Speaking to Channel 4 news she said: "The Labour Party needs to do more than simply to have suspensions of people who have been saying anti-Semitic things.
"We've actually got to have an action plan to deal with this in order to have strong processes so that there is swift action because you can never tolerate that kind of discrimination in the party."
Labour leader Jeremy Corbyn has denied that Labour is in crisis after prominent party member Ken Livingstone was suspended for "bringing the party into disrepute" over comments about Hitler.
Speaking to the BBC he said: "It's not a crisis. There's no crisis. Where there is any racism in the party it will be dealt with and rooted out. I have been an anti-racist campaigner all my life.
"There is not a problem. We are totally opposed to anti-Semitism in any form within the party. The very small number of cases that have been brought to our attention have been dealt with swiftly and immediately, and they will be."
Tensions rose today as a row over anti-Semitism in the Labour party escalated following comments made by party member Ken Livingstone.
Labour faced calls to suspend Livingstone, which they subsequently did, after he said Adolf Hitler was "supporting Zionism" when he rose to power in the 1930s during an interview to defend fellow suspended MP Naz Shah.
ITV News Deputy Political Editor Chris Ship reports.
Anti-Semitism is a "very real poison" in our society and in British politics, according to the Holocaust Educational Trust.
Karen Pollock, the trust's chief executive, said: "If anyone was in doubt before, it is clear that anti-Semitism continues to be a very real poison in our society and sadly in British politics.
"Those who invoke the Holocaust to score political points should be loudly and roundly condemned - this is not the first time that Ken Livingstone has chucked the Holocaust around like political confetti and it will probably not be the last. But to be clear, the deliberate misuse of the history of the Holocaust is anti-Semitism - pure and simple."
Boris Johnson has claimed the Labour Party has become infected by a "virus of anti-Semitism" after party member Ken Livingstone was suspended following his comments that Adolf Hitler was a supporter of Zionism.
The London mayor told LBC radio: "There is plainly some sort of virus of anti-Semitism within the Labour Party that needs to be addressed.
"There is an ideological continuum between Ken Livingstone about Israel and the position of Jeremy Corbyn."
Ken Livingstone has been captured on video, coming out of his house to remove an Israeli flag that was tied to his front door.
Dressed in an 'old Labour' t-shirt and braces, the suspended MP, was heard saying "what's this?" as he came out to check if there was someone at his door and instead found a flag attached to the knocker.
A number of Jewish groups have called for Ken Livingstone's immediate expulsion from the Labour party amid an anti-Semitism row fuelled by his comments that Adolf Hitler was a supporter of Zionism.
Gideon Falter, chairman of the Campaign Against Antisemitism branded Livingstone "a hardened politician who has spent his political career accommodating anti-Semitic extremists and making anti-Semitic gaffes" and said he must go.
He said: "Jeremy Corbyn should understand that zero tolerance for racism is all or nothing, and it is time for Ken Livingstone to be banished or for Corbyn to stop pretending to oppose racism."
Jonathan Arkush, president of the Board of Deputies of British Jews, added: "Ken Livingstone's comments were abhorrent and beyond disgraceful. He denies anti-Semitism in Labour when the evidence is there for all to see.
"He lacks any sense of reality and decency. He must now be expelled from the Labour Party."
Labour Party leader Jeremy Corbyn has said that there are "grave concerns" over the language Ken Livingstone used when defending MP Naz Shah after her suspension for alleged anti-Semitic social media posts.
Livingstone said Adolf Hitler was "supporting Zionism" when he rose to power in the 1930s whilst publicly defending Shah in a radio interview.
Speaking after Livingstone was himself suspended today, Corbyn said: "There were very grave concerns about the language he used in an interview this morning. We had a discussion about it and decided we would suspend him and he would go through an investigation by the party."
Labour MP John Mann has been told it is "completely inappropriate" to be involved in public rows on television after his bust up with Ken Livingstone earlier today.
It comes after Mann angrily confronted Livingstone in Westminster, calling him a "disgusting racist" and a "Nazi apologist" over comments he made on anti-Semitism.