Cabinet Office minister Francis Maude said Bill Crothers, the Government's chief procurement officer, will lead the review into all Government contracts held with G4S and Serco, which include running immigration centres and the welfare-to-work scheme.
Mr Maude said: "The public rightly expects government suppliers to meet the highest standards, and for taxpayers' money to be spent properly and transparently."
Electronic tagging is used as a method of monitoring offenders in the community and can be used to make sure criminals stick to court-imposed curfews.
The equipment consists of a tag fitted to the offender's ankle and a monitoring unit in the home or other place of curfew.
All movements in and out of the home, or other activities such as removal of the tag or tampering with the equipment, are reported to a control centre.
Justice Secretary Chris Grayling has told the House of Commons he is "astonished" by the revelations the government has been overcharged by "tens of millions of pounds".
– Justice Secretary Chris Grayling
This is a wholly indefensible and unacceptable state of affairs.
The House will share my astonishment that two of the Government's biggest suppliers would seek to charge in this way.
The House will also be surprised and disappointed to learn that staff in the Ministry of Justice were aware of a potential problem and yet did not take adequate steps to address it.
The audit into the Government's electronic monitoring contracts with G4S and Serco found that overcharging began at least as far back as the start of the current electronic monitoring contracts in 2005 - but could have dated as far back as the previous contracts in 1999.
It also found that contract managers in the Ministry of Justice discovered some of the issues following a routine inspection in 2008 - but did nothing to tackle the problem, Justice Secretary Chris Grayling said.
Justice Secretary Chris Grayling has told the House of Commons that security firm G4S has refused to take part in an additional audit so the Government can rule out the possibility that dishonesty was involved in any of the contracts, while Serco has agreed to this demand.
Serco has agreed to withdraw from the current tender process for electronic monitoring, while the Justice Secretary is to instigate moves to exclude G4S as the company is still attempting to bid.
The Government says it has been overcharged "tens of millions of pounds" by G4S and another firm, as part of its electronic tagging contract.
Justice Secretary Chris Grayling told the House of Commons the Government had been charged in cases when Serco and G4S were not providing electronic tagging - and in a small number of cases when the offender was known to have died.
Justice Secretary Chris Grayling has told the House of Commons he is to ask the Serious Fraud Office to investigate security firm G4S over the handling of its electronic monitoring contract after the company refused to take part in an additional forensic audit.