The Government has rejected an offer from security firm G4S to repay the £24m it owes for overcharging in credit notes.
Officials turned down the settlement and say they will "pursue all possible avenues" to recoup more taxpayers' money.
A Ministry of Justice spokesman said it was still working with the firms and the independent auditors to work out "what the final sum will be".
"The Secretary of State has been clear: we are determined to secure a refund for the taxpayer.
"We have taken appropriate legal advice and will pursue all possible avenues."
More top news
At least 14 people have been killed after an overcrowded bus slipped off a mountain road and fell 980ft in Nepal.
A man in his 70s has been arrested on suspicion of the murder of Elsie Frost more than 50 years ago.
A challenge to the decision not to take further action against Sir Cliff Richard over alleged sex offences has been dismissed.