Taxpayers forced to hang onto the phone while calling HM Revenue and Customs lost the equivalent of £97 million last year, a watchdog has found.
The overall cost incurred by customers who called the taxes helpline increased from £63 million in 2012-13 to £97 million in 2015-16, the National Audit Office reported.
It comes as the NAO concluded that the quality of service provided by HMRC to taxpayers "collapsed" amid staff cuts.
HMRC got its timing badly wrong when rolling out its digital strategy, which involves moving more personal taxpayers online and reducing demand for telephone and postal contact, which are more costly to handle, NAO found.
While HMRC maintained or improved its customer service up to 2013-14, it then misjudged the overall impact of its complex transition and shed too many customer service staff before completing service changes.
Within this estimate, customers paid £2 million less in call costs because HMRC reduced call charges by moving from higher-rate numbers to local-rate "03" telephone numbers in September 2013.
But an increase in the economic cost of time spent waiting for an answer or speaking to an adviser more than offset this saving, the NAO said.
An NAO survey of customers using HMRC services found 58% rated the service as good or excellent, 21% as average and one in five (21%) rated it as poor or terrible.
Between 2010-11 and 2014-15, HMRC cut staff numbers in personal tax from 26,000 to 15,000.
The reductions would be achieved in part by moving customers to less expensive contact through the expansion of HMRC's digital services.
Ruth Owen, HMRC's director general for customer services, said: "We recognise that early in 2015 we didn't provide the standard of service that people are entitled to expect and we apologised at the time.
"We have since fully recovered and are now offering our best service levels in years.
"Over the past six months we've consistently answered calls in an average of six minutes, and have launched new online tax accounts and webchat for everyone, enabling customers to manage their tax affairs wherever and whenever they want.
"There's never been a better or more convenient service for our customers".
Gillian Guy, chief executive of Citizens Advice, said waiting on the phone to HMRC put some people "at risk of debt", caused frustration and made people "miss important deadlines".