Lord Andrew Feldman, co-chairman of the Conservative Party, became embroiled in a row after a senior Conservative called Tory activists "mad, swivel-eyed loons" who were forcing MPs to take hardline positions on Europe and same-sex marriage.
An ICM poll for the Sunday Telegraph showed the public taking a more Eurosceptic line than Cameron, with 46 per cent wanting to leave the European Union and 44 per cent calling for an immediate "in/out" referendum. The survey also underlined the rise of Nigel Farage and the UK Independence Party.
Up to 200 Tory MPs prepared to vote against Cameron's plans to allow same-sex couples to marry. At least three Cabinet ministers are expected to join rebels in opposing at least some of the Government's proposals.
Two issues - Europe and same-sex marriage - have been seized on by grassroots activists to claim that Cameron and his inner circle are "out of touch" with the views of ordinary party members.
The chairman of Conservative Grassroots, a leading activists' pressure group, said the Prime Minister and his inner circle lived in a "Westminster bubble".
Relations between Cameron's circle and the party's grassroots worsened as Feldman put himself at the centre of the row over comments about activists, inevitably dubbed "swivelgate".
On Saturday, newspapers including the Daily Telegraph, the Times and the Daily Mirror quoted a senior figure who said Tory MPs were being forced into hardline views on Europe and same-sex marriage.
"There's really no problem. The MPs just have to do it because the associations tell them to, and the associations are all mad swivel-eyed loons," said a Conservative described as one of Cameron's closest allies.
Feldman, 47, who was appointed co-chairman of the party in 2010, denied making the remarks after rumours circulated on social networking sites that he was responsible.
He admitted speaking to journalists after a private dinner for Conservative Friends of Pakistan at the InterContinental Hotel in Westminster but said it was "completely untrue" that he had called activists "swivel-eyed loons".
Sources said the peer came out of a private room in the hotel and spoke to journalists who were at a table in a public area with a "fifth man", understood to be a senior civil servant.
The "fifth man" was said to have said he could not recollect the exact words used. The Daily Telegraph said it stood by its report. It and the other newspapers have not named the member of Cameron's circle behind the slur.
Feldman said: "There is speculation on the internet and on Twitter that the senior Conservative Party figure claimed to have made derogatory comments by the Times and the Telegraph is me.
"This is completely untrue. I would like to make it quite clear that I did not nor have ever described our associations in this way or in any similar manner. Nor do these alleged comments represent my view of our activists.
"On the contrary, in the last eight years of working for the party, I have found them to be hardworking, committed and reasonable people. They are without question the backbone of the party.
"I am very disappointed by the behaviour of the journalists involved, who have allowed rumour and innuendo to take hold by not putting these allegations to me before publication. I am taking legal advice."
A profile of Cameron in the Financial Times last year reported he had used a similar phrase, and "tells colleagues that anyone who wants to talk to him about the EU is 'swivel-eyed"'.
The row carries uncomfortable echoes of last year's saga which saw Andrew Mitchell, then the Conservative chief whip, accused of calling police officers at Downing Street "plebs" - claims which he strongly denies.
Feldman became a close friend of Cameron when the pair were at Oxford, in the 1980s.
Robert Woollard, a former chairman of Wycombe Conservative Association, said of the remarks linked to Lord Feldman: "These are arrogant and pompous comments that show the utter contempt some people in David Cameron's camp have for the grassroots of our party."
The timing of the "swivel-eyed loon" row is potentially damaging. This week, Cameron faces a new parliamentary showdown with his party, with up to 200 Tory backbenchers expected to vote against his plans to allow same-sex couples to marry.
Cabinet ministers are expected to be among the "rebels". Chris Grayling, the Justice Secretary, indicated that he would support amendments to protect people who spoke up for traditional marriage, while David Jones, the Welsh Secretary, and Owen Paterson, the Environment Secretary, will both vote against the bill at its third reading in the Commons.
Two Cabinet ministers have declared that they would vote to leave the EU if a referendum was held now. Michael Gove, the Education Secretary, and Philip Hammond, the Defence Secretary, made the comments last weekend. Both Gove and Hammond are suspected by MPs of being "on manoeuvres" to position themselves as future party leaders.
