A criminal investigation has been opened into government electronic tagging contracts held with Serco, as well as G4S, the Serious Fraud Office (SFO) said.
An audit by accountancy firm PricewaterhouseCoopers, launched in May, alleged that overcharging began at least as far back as the start of the current contracts in 2005 but could have dated back to 1999.
Justice Secretary Chris Grayling told the two firms that an independent forensic audit was required to look at internal email trails between executives to establish what happened. While Serco agreed to take part, G4S refused.
Serco allowed a further forensic audit to take place but G4S refused to co-operate and were reported to the SFO.
More top news
Mr Osborne said he is "not the person to provide the unity my party needs".
Actress Lena Dunham has criticised Kanye West's "disturbing" new music video for depicting a host of naked celebrities in bed together.
David Cameron is to meet leaders of other EU states for the first time since the referendum vote for Brexit.