Around three quarters of Labour members believe the party's problem with anti-Semitism is being exaggerated to damage Jeremy Corbyn, according to polling.
Just 19% said it was a serious issue while 77% believe it is used to undermine the leader or stop criticism of Israel, a YouGov study for The Times found.
It comes despite a tumultuous week that has seen fresh recriminations within the party as well as a protest outside Parliament led by Jewish leaders.
A senior official forced to quit over her role in handling complaints claimed the row over anti-Semitism in the party was being "stirred up" to attack the Labour leader.
Christine Shawcroft stood down as disputes panel chairwoman after opposing the suspension of a council candidate accused of Holocaust denial.
A group of MPs and peers signed an open letter calling for Ms Shawcroft to be suspended from the National Executive Committee (NEC).
In a statement posted on Facebook, Ms Shawcroft said she was "not a Holocaust denier and I would not support a Holocaust denier".
She added: "This whole row is being stirred up to attack Jeremy, as we all know. That someone who has spent his whole life fighting racism in all its forms should find himself being accused of not doing enough to counter it, absolutely beggars belief."
In a later Facebook post, Ms Shawcroft issued a new statement removing the comments about the row being stirred up.
She said she was "deeply sorry" for the way she had handled the complaint about the local council candidate and knew "how much hurt it has caused".
Ms Shawcroft said she had already announced she would not seek re-election to the NEC so will be leaving in the summer.
The statement appeared to be removed from the site later on.
But a Jewish Labour peer said the party leader had "encouraged and endorsed" anti-Semites.
Lord Winston said hostility to Jews has "infected the Labour Party so it's become endemic".
YouGov interviewed 1,156 Labour members between Tuesday and Thursday.