Juliet Lyon, the director of the Prison Reform Trust, has told ITV News that the Government must ensure prisoners who are freed are better protected to prevent them from quickly re-offending, with so few entering employment upon leaving jail.
I asked the Prime Minister why Andrew Mitchell kept his job for so long.
"It's the easiest thing in the world if you're Prime Minister just to fire someone as soon as something goes wrong," he said.
"But that's not the right way to behave as Prime Minister. He did something that was wrong, he needed to apologise, he did apologise."
He said Mr Mitchell had resigned when it became clear that he would not be able to continue as an effective Chief Whip.
"The time it takes might be uncomfortable and difficult for politicians and governments...but in the end, government is about doing the right thing, making the right decision, not just making the easy decision," he said.
The Prime Minister insisted focussing on rehabilitation of criminals is "not wishy washy."
The Prime Minister said his government is launching a 'rehabilitation revolution', aiming for more people properly punished but fewer of them returning to prison.
It's not rocket science but has eluded previous governments, he said.
"We've tried just banging people up and it doesn't work," said David Cameron, outlining his plans for criminal justice reform.
The Prime Minister has refused to repeat his "hug a hoodie" phrase.
He refers to it as "three words, two of which begin with 'H'."
Visiting Wormwood Scrubs Prison to see how our "tough but intelligent" approach to crime is working.
David Cameron will later use his first major speech on crime since the coalition came to power in 2010 to distance himself from ever urging the public to "hug a hoodie".
Vicki Helyar-Cardwell, director of the Criminal Justice Alliance, praised the Tory leader's 2006 speech - from which the sometimes derided "hoodie phrase" emerged - for recognising that criminal and social justice were inextricably linked. But she said:
Several years on, the criminal justice system still often picks up the pieces when others have failed. The high levels of children in care who end up in our youth justice system are a telling example of this.
Looking ahead to the Prime Minister's speech, in which he will urge for a "tough, but intelligent" approach to justice, Ms Helyar-Cardwell warned of rising crime costs and said:
The danger is the intelligent bit gets lost at the expense of sounding tough. ... Making governors more accountable for what happens after prison is welcome, but this will backfire without efforts to curb huge prison overcrowding that blights much of the positive work in prisons.
Justice Secretary Chris Grayling, who was appointed by David Cameron to replace Ken Clarke in last month's reshuffle, has said fresh action is essential to cut the rates of reoffending.
– Chris Grayling, speaking on BBC Radio 4's Today programme
The reality at the moment, all too often, is that we release somebody on to the streets with £46 in their pocket - relatively little, if any, support.
We have to do this differently. We have got people coming back out onto the streets after prison who are as likely to reoffend again as not to reoffend. We have got to do things differently.
The benefit of a payment-by-results system is it forces the organisations working with you to look for what really does work because they don't get paid unless they do.
David Cameron's plans on crime and justice are undermined by a lack of money, The Guardian has reported.
The newspaper said aides "admitted there would be no additional money to pay for either more prisoners or for extending rehabilitation to more offenders."