Lord Heseltine has told ITV News that the UK should get rid of what he described as 'slums', following the prime minister's pledge to regenerate around 100 of the UK's worst estates.
A total of £140 million will be made available to transform the run-down areas in a bid to tackle poverty, drug abuse and gang culture.
The multi-million pound redevelopment programme is to be overseen by Lord Heseltine, who helped to transform the Liverpool and London docks in the 1980s.
He said that it was his dream to get rid of the "slums".
Lord Heseltine added that it had been part of his background to "change the culture and opportunity of these places" but said that there were "still too many left".
David Cameron told the BBC's Andrew Marr Show it was time to "demolish the worst of these and rebuild houses people feel they can have a real future in".
"The odd thing about some of these high-rise blocks and the way that they've been structured is actually they don't provide a huge number of houses", he added.
Following Mr Cameron's interview, he appeared to be accused of "destroying the welfare state", by band Squeeze who performed at the end of the programme.
As the end credits rolled, front man Glenn Tilbrook changed the lyrics of the last verse of their song Cradle to the Grave.
In an apparent reference to the Government's announcement that sink council estates could be bulldozed to make way for better homes, he sang:
I grew up in council houses, part of what made Britain great. There are some here who are hell bent; on the destruction of the welfare state.
It was not clear whether Mr Cameron noticed the protest, as he continued to watch without reacting.
A BBC spokeswoman said producers had not been aware of Squeeze's protest beforehand.
Lord Heseltine's estate regeneration advisory panel has been told to produce a full blueprint by the time of the Chancellor's Autumn Statement.
Further details of the scheme will be set out in a keynote speech Mr Cameron will deliver on Monday.
In it the prime minister is also due to outline plans to double government funding for relationship counselling for troubled families and relaunch a coalition proposal to issue vouchers for parenting classes.
Writing in The Sunday Times, Mr Cameron also said: "Within these so-called sink estates, behind front doors, families build warm and welcoming homes.
"But step outside in the worst estates and you're confronted by concrete slabs dropped from on high, brutal high-rise towers and dark alleyways that are a gift to criminals and drug dealers."
The prime minister also made clear he sees the neglect of estates as behind the riots that swept Britain in 2011, writing: "The rioters came overwhelmingly from these postwar estates. That's not a coincidence."
Labour's John Healey, the shadow housing minister, said the scheme was "small-scale" and would fail to adequately tackle housing problems.
"Any extra to help councils build new homes is welcome but Conservative ministers have halved housing investment since 2010 and are doing too little to deal with the country's housing pressures," he said.
Possible housing development targets
The housing developments being targeted reportedly include:
The Winstanley estate in Wandsworth, south London
The Lower Falinge estate in Rochdale, Greater Manchester
Broadwater Farm in Tottenham, north London