Government borrowing increased slightly in the last financial year. The Office for National Statistics revealed it rose by around £300m to £118.8 billion. But public sector borrowing for last month was lower than expected last month at £8.8 billion.
The Treasury said the Government is "taking tough decisions to deal with the deficit" following the release of public sector borrowing figures by the Office for National Statistics.
A spokesman said: "Today's data shows that borrowing is lower than was previously predicted.
"As the Chancellor said this week: the economy is healing; the deficit is down by a third; inflation is down by almost half from its peak; and more people are in private sector employment than ever before".
The public sector net borrowing figures for May were flattered by a £3.2 billion clawback of taxes from Swiss bank accounts under the UK-Switzerland tax treaty.
A £3.9 billion cash transfer from the Bank of England's Asset Purchase Facility also help May's deficit fall by £6.9 billion year-on-year to £8.8 billion, the Office for National Statistics (ONS) said.
Public sector net borrowing excluding distortions such as bank bailouts and quantitative easing cash dropped £3 billion from £15.6 billion a year earlier, the ONS said.
Chancellor George Osborne's plan to reduce the budget deficit will be boosted by the figures, although the ONS said that without the Swiss tax payment, May's deficit would have been broadly level with a year earlier.