David Cameron said it was "simply not the case" that people around him think that Conservative grassroots activists are "swivel-eyed loons", during an interview with BBC Radio 4's Today programme.
The Tory co-chairman Lord Feldman reportedly made the disparaging remarks - an allegation he strenuously denies.
David Cameron sent an email to members of his Conservative Party urging unity.
The Tory grandee Lord Howe has accused the Prime Minister of 'losing control' of his party as divisions over Europe come to the fore.
The Conservative Party has moved to head off a row with grassroots activists after a Tory ally reportedly described them as "loons".