Video report by ITV News Presenter Tom Bradby
In his first in-depth television interview about his time in office - which will be broadcast on ITV on Monday at 8pm - the referendum haunts him and he thinks about it every day, but insisted holding it was the "right thing to do".
He added the decision to suspend Parliament was a "mistake" and "probably counterproductive", but said it was for the courts to decide if it was legal.
He also said "taking the whip away from 21 incredibly hard-working, loyal Conservatives" was a "bad decision" and said if it isn’t reversed it will become a "disastrous decision".
In the interview, Mr Cameron was asked about domestic and foreign policy, the death of his son Ivan, and whether he would consider a return to frontline politics.
Mr Cameron claimed the referendum was "inevitable" but doesn’t accept "that everything that has followed had to follow".
He appeared to accept the suggestion that the political deadlock is a result of his decision to hold the referendum.
"Do I have regrets? Yes," he said. "Am I sorry about the state the country’s got into? Yes. Do I feel I have some responsibility for that? Yes. It was my referendum; my campaign; my decision to try and renegotiate."
"And I accept all of those things and people, including those watching this programme, will have to decide how much blame to put on me."
He claimed Mr Johnson’s support of Leave in the EU referendum was disingenuous, saying: "He thought that the Brexit vote would be lost but he didn’t want to give up the chance of being on the romantic, patriotic, nationalistic side."
The former prime minister said Mr Johnson texted him shortly before declaring his Brexit stance to the public, saying: "Brexit will be crushed like a toad under the harrow."
Mr Cameron added: "I can only conclude that - he’d never argued for it before; he thought it was going to lose and that’s why he made the choice."
He said he knew Mr Gove was a "very long-standing Eurosceptic" but seemed on the fence regarding the referendum.
"He seemed in two minds about it and I thought, if he’s in two minds, surely stick with the team and the programme and the government and the work that we’re doing together."
But he said after revealing which way he would campaign, Mr Gove "went from this liberal, modern, compassionate Conservative to something quite different".
The prime minister and Mr Gove declined to comment to the Tonight programme.
However, in a later statement, Mr Johnson told ITV News: "Nothing that David Cameron says in his memoirs will diminish the affection and respect in which I hold him."
On the division in the country around Brexit, Mr Cameron said: "I regret hugely where we have come to and I take my share of responsibility for that."
'Austerity was necessary'
Upon taking office as prime minister, Mr Cameron embarked on huge cutbacks in public spending.
Many blamed the rise in homelessness and food bank usage on the policy, but Mr Cameron said austerity was "necessary" in order to balance the country’s finances.
He said: "The cuts were very difficult to make and there were lots of very difficult decisions and I’m not sure we got all of them right, but I’ve never wavered in the belief that it was necessary to make difficult decisions."
'I blame myself over UK not joining Syria airstrikes'
In 2013 Mr Cameron became the first British prime minister to lose a vote on a major issue of foreign policy when the Commons decided the UK should not join US-led airstrikes on Syria.
After watching what appeared to be evidence of chemical weapons in Syria, Mr Cameron said "the sight of the children laid out in rows" made him think of his late son Ivan, and he thought "we’ve got to act".
But Mr Obama took four days to return Mr Cameron’s calls.
Mr Cameron said that was “far too long” but they eventually spoke and agreed a plan.
He then made a passionate speech to Parliament, arguing that military intervention was necessary but MPs voted not to back airstrikes.
He believes he lost the vote because MPs were "traumatised by what had happened in Iraq" and didn’t want to repeat history.
"I lost that vote," he said. "The first time a prime minister had lost a vote in the House of Commons on a major issue of foreign policy and you know, I have to - look I blame the people who voted against me, obviously, but I also blame myself."
The death of his son Ivan
Mr Cameron spoke about his son Ivan, who had a rare neurological disorder and died in 2009 aged six.
"I think the difficult thing was that he ... he suffered so much," he said.
"He sometimes would have, you know, 20, 30, 40 seizures in a day that when I think about it a lot, you do go back to the incredibly painful and difficult times that he had."
But Mr Cameron said, after months of mourning, happy memories of his son began to come back and now, when reflecting, he says he feels "incredibly blessed" to have been charged with "looking after someone so special".
Will he return to frontline politics?
In a short answer, the former PM does not see himself returning to frontline politics.
But he said he would always help with an issue if asked by a prime minister.
"I’ll always, you know, want to help. I love this country. I care passionately about what happens," he said.
"But I think the idea of going back to frontline politics is not going to happen, nor should it."