Mike Down, of accountancy firm Baker Tilly, said the HM Revenue & Customs (HMRC) should be checking tax returns to see whether individuals had already explained their circumstances.
He told the Daily Telegraph that The Revenue is "adopting a computer says yes approach, rather than simply checking the tax returns".
One case involved an elderly widow whose effective tax rate was low because she was giving more than half her income to charity.
More top news
Seven Ukrainian men have been arrested after a boat containing suspected illegal immigrants ran ashore in high winds off the Suffolk coast
Mixed weather this Bank Holiday Monday
Boxer told ITV news that as long as he is happy "with or without stardom and finance" then that was the main thing for him.