G4S has admitted it wrongly billed the Government for the provision of electronic monitoring, or tagging, services, and said it will pay back £24.1 million.
A criminal investigation has been opened into government electronic tagging contracts held with Serco, as well as G4S, the Serious Fraud Office (SFO) said.
An audit by accountancy firm PricewaterhouseCoopers, launched in May, alleged that overcharging began at least as far back as the start of the current contracts in 2005 but could have dated back to 1999.
Justice Secretary Chris Grayling told the two firms that an independent forensic audit was required to look at internal email trails between executives to establish what happened. While Serco agreed to take part, G4S refused.
Serco allowed a further forensic audit to take place but G4S refused to co-operate and were reported to the SFO.
A criminal investigation has been launched into a contract between security firm G4S and the Government for tagging criminals.
An inquiry has been opened by the Serious Fraud Office after it emerged G4S, along with Serco, had charged the Ministry of Justice for monitoring offenders who were dead, back in prison, had their tags removed, left the country or never been tagged in the first place.
An audit by accountancy firm PricewaterhouseCoopers, launched in May, alleged that overcharging began at least as far back as the start of the current contracts in 2005.
The Government reported G4S in July when the FTSE 100 firm refused to take part in an additional audit to rule out any dishonesty.
The Serious Fraud Office (SFO) has opened an investigation into the Government's contract with G4S for the provision of electronic monitoring, or tagging, services, the security firm has confirmed.
The security firm criticised for its handling of the London Olympics will bid to be involved in the Commonwealth Games in Glasgow next year. G4S will apply for one of the two aspects of event provision to be contracted out by Glasgow 2014- safety stewarding and security guarding.
Last year 3,500 military personnel were drafted in by the Government to assist with the security at London 2012 venues two weeks before the opening ceremony after G4S admitted it might not be able to provide enough guards.
Statt at a South African prison run by the British security company G4S have been accused of "shocking" abuses against inmates.
Prisoners claimed they were subjected to electric shock treatment, while video footage from the high security jail appeared to show inmates being given forced injections.
G4S has said it had no knowledge of abuse by any of its employees at Mangaung prison, which came under the spotlight in a year-long investigation by the Wits Justice Project.
The South African government recently took over running the jail, saying G4S had "lost control" of the facility.
The mother of one of Danny Fitzsimon's victims has said she believes G4S is to blame for deciding to hire, and for arming, her son's killer.
Both Fitzsimons and her son Paul McGuigan were working for the private security firm ArmourGroup, which is owned by G4S, at the time of the incident.
Corinne Boyd-Russell says that Fitzsimons should not have been hired given his history of post-traumatic stress syndrome and the fact that he was charged with a crime in the UK.
Private security firm G4S has pulled out of bidding for new contracts for tagging criminals following an overcharging scandal.
G4S had initially refused to withdraw from the process despite revelations that the firm, along with its rival Serco, had been overcharging the Government by millions of pounds for the electronic tagging of criminals.
Justice Secretary Chris Grayling said he was pleased that the company had decided to withdraw from the bidding process but revealed the company would still face a review into its deals with the Government:
"I am glad they have decided to withdraw now. We can now get on with awarding that contract, which will improve the monitoring of offenders and deliver savings for the taxpayer. Our concerns regarding the billing for the electronic monitoring contract still need to be addressed."
Business Secretary Vince Cable said "certainly something bad happened here" after the Serious Fraud Office was urged to investigate G4S following its refusal to co-operate with the Government over the tagging contract scandal.
Mr Cable told BBC Radio 4's Today programme, "The Government has been looking very carefully at how it gets value for money and has judged that there is some overcharging taking place and we are trying to get down to competitive costs".
He said his Business Department was reviewing the contracts it has with private companies - including G4S and Serco - to deliver public services.
Asked whether the scandal casts doubt over the notion of contracting out state activities to private providers, Mr Cable said, "There are lots of success stories that you don't hear about, but where there is bad practice and lack of care then obviously we've got to tighten up".
Security firms G4S and Serco have been accused of overcharging the government tens of millions of pounds for electronic tagging services.Read the full story ›