The Government has rejected an offer from security firm G4S to repay the £24m it owes for overcharging in credit notes.
Officials turned down the settlement and say they will "pursue all possible avenues" to recoup more taxpayers' money.
A Ministry of Justice spokesman said it was still working with the firms and the independent auditors to work out "what the final sum will be".
"The Secretary of State has been clear: we are determined to secure a refund for the taxpayer.
"We have taken appropriate legal advice and will pursue all possible avenues."
More top news
The Filipino girl group lost out to Lauren Murray in the sing off after receiving the lowest number of votes.
Shoppers in the UK are expected to spend almost £1bn online tomorrow for Cyber Monday as a weekend of discounts comes to an end.
Fresh from his straight sets victory, Murray confirmed he would play in the opening tie against Japan next year.