The Government has rejected an offer from security firm G4S to repay the £24m it owes for overcharging in credit notes.
Officials turned down the settlement and say they will "pursue all possible avenues" to recoup more taxpayers' money.
A Ministry of Justice spokesman said it was still working with the firms and the independent auditors to work out "what the final sum will be".
"The Secretary of State has been clear: we are determined to secure a refund for the taxpayer.
"We have taken appropriate legal advice and will pursue all possible avenues."
More top news
Speaking to ITV News' Mark Austin on the anniversary of the London bombings, ex-PM says nations around the world will need to respond.
A Canadian man has been arrested after tying more than 100 balloons to a garden chair and flying over the city of Calgary.
A van bearing the faces of some of Britain's most wanted fugitives will be driven around parts of Spain popular with expats in a new push.