The Government has rejected an offer from security firm G4S to repay the £24m it owes for overcharging in credit notes.
Officials turned down the settlement and say they will "pursue all possible avenues" to recoup more taxpayers' money.
A Ministry of Justice spokesman said it was still working with the firms and the independent auditors to work out "what the final sum will be".
"The Secretary of State has been clear: we are determined to secure a refund for the taxpayer.
"We have taken appropriate legal advice and will pursue all possible avenues."
More top news
Justin Bieber is being sued for more than £50,000 after allegedly smashing the mobile phone of a man who was filming him.
President Barack Obama and First Lady Michelle Obama have celebrated Star Wars Day by getting down with R2-D2 and some storm troopers.
Police are appealing for information about two sisters aged 12 and five who went missing three days ago.