The Government has rejected an offer from security firm G4S to repay the £24m it owes for overcharging in credit notes.
Officials turned down the settlement and say they will "pursue all possible avenues" to recoup more taxpayers' money.
A Ministry of Justice spokesman said it was still working with the firms and the independent auditors to work out "what the final sum will be".
"The Secretary of State has been clear: we are determined to secure a refund for the taxpayer.
"We have taken appropriate legal advice and will pursue all possible avenues."
More top news
A man who has been on the run from the FBI for 10 years has been jailed for sexually assaulting a teenage girl in a Lancashire park.
Theresa May seems to have caved in to pressure from MPs and has agreed to publish a plan of her Brexit vision before triggering Article 50.
Theresa May agreed to outline the government's plans after tabling an amendment to a Labour motion calling for more details on 'Brexit'.