Business Secretary Vince Cable said "certainly something bad happened here" after the Serious Fraud Office was urged to investigate G4S following its refusal to co-operate with the Government over the tagging contract scandal.
Mr Cable told BBC Radio 4's Today programme, "The Government has been looking very carefully at how it gets value for money and has judged that there is some overcharging taking place and we are trying to get down to competitive costs".
He said his Business Department was reviewing the contracts it has with private companies - including G4S and Serco - to deliver public services.
Asked whether the scandal casts doubt over the notion of contracting out state activities to private providers, Mr Cable said, "There are lots of success stories that you don't hear about, but where there is bad practice and lack of care then obviously we've got to tighten up".
More top news
Acclaimed author Michael Morpurgo has written this piece of prose exclusively for ITV News to mark 100 years since the Battle of the Somme.
The War Horse and Private Peaceful author joined ITV News in Northern France to reflect on the sacrifices made at the Battle of the Somme.
Detectives investigating an arson attack have released CCTV of a man igniting two petrol bombs moments before hurling them at the shop.