David Cameron said it was "simply not the case" that people around him think that Conservative grassroots activists are "swivel-eyed loons", during an interview with BBC Radio 4's Today programme.
The Tory co-chairman Lord Feldman reportedly made the disparaging remarks - an allegation he strenuously denies.
There is a "growing gap" between David Cameron and the Conservative Party, according to one of the party's MPs.
Brian Binley MP said Mr Cameron was made leader because "we thought he was a winner" but had now "done a few things that the party in the country overall didn't want him to do".
Mr Binley called for an investigation into whether a close ally of the Prime Minister had dismissed grassroots activists as "mad, swivel-eyed loons", but his request was rejected by the party's board.
While welcoming Mr Cameron's conciliatory email to party members, Mr Binley told BBC Radio 4's Today programme: "He may believe in them but we are talking about the gap between himself and the party."
"I think it is a growing gap," he said, but added: "I think it can be put right and I think David Cameron is listening as proved by the fact that actions are now being taken."
A surge in support for the UK Independence Party has pushed it to within just two points of the Conservatives in a poll of voting intentions.
In a fresh headache for David Cameron, the survey by Survation showed a five-point slump for his party to a record low for the pollster of 24%.It was released as the Prime Minister wrote to Tory activists in a bid to smooth fraught relations with the grassroots over Europe and gay marriage.
Labour slipped one point to 35% but saw its lead over the Tories stretch to 11 points. The Liberal Democrats were down one point at 11%.
Survation's findings - based on 1,000 responses from an online panel - were calculated using a weighting system to reflect a number of factors including individuals' stated likelihood to vote.
Prime Minister David Cameron is sending a letter to members of his Conservative Party urging unity and stressing the importance of the "deep and lasting friendship" members enjoy. In it, he says:
I am proud to lead this party. I am proud of what you do. And I would never have around me those who sneered or thought otherwise. We are a team, from the parish council to the local association to Parliament, and I never forget it.
Does that mean we will agree on everything? Of course not. The Conservative Party has always been a broad church – one which contains different views and opinions – and we must remain so today.
He stresses the need for the party to celebrate the achievements of the party, and praises Conservative activists for their values of "duty, decency and civil pride." His personal note ends:
So to those reading this, here is my message: there will always be criticism from the sidelines. But we must remember what this Party has always been about: acting in the national interest. Our task today is to clear up Labour’s mess and make Britain stand tall again.
We have a job to do for our country – and we must do it together.
The first opinion poll has been released since allegations emerged that a senior Tory aide described grassroots activists as "swivel-eyed loons".
- Conservative: 24% (-5)
- Labour: 35% (-1)
- Liberal Democrat: 11% (-1)
- UKIP: 22% (+6)
- Other: 8% (nc)
The Conservative Party Board has rejected a suggestion that an investigation be held into alleged comments by chairman Lord Feldman, Tory MP Brian Binley said today.
Speaking amid the "swivel-eyed loon" row, the former Conservative deputy chairman and donor Lord Ashcroft has said the political priorities of party members do not always match those of the people the Tories need if the party is to win.
UK Independence Party leader Nigel Farage has moved to exploit unrest within the Conservative Party with a rallying call for disaffected Tory voters to join his ranks.
Mr Farage has taken out a full-page advert in The Daily Telegraph urging those unsettled by David Cameron's policies or insulted by a senior Tory figure's alleged description of activists as "mad, swivel-eyed loons" to defect to his party.
Some of the country's most prominent business leaders have accused Eurosceptic MPs of "putting politics before economics" by calling for Britain's withdrawal from the European Union.
The group, which includes the chairmen of BT, Deloitte and Lloyds along with Sir Martin Sorrell and Sir Richard Branson, said Britain's business interests and economy can only benefit from playing a central role in the EU.
In a letter to The Independent, they called for David Cameron to "strengthen and deepen" the European single market, adding: "The economic case to stay in the EU is overwhelming."