Debt as a percentage of gross domestic product (GDP) - a broad measure for the total economy - hit 65% in May, up from 61.3% last year.
April's borrowing figures were flattered by a one-off £28 billion lift from the value of assets transferred from the Royal Mail pension plan.
But excluding this one-off impact, total borrowing for the current financial year stands at £28.4 billion.
Earlier this month, the Chancellor and Governor of the Bank of England Sir Mervyn King unveiled a multibillion-pound lending scheme to stimulate economic growth - but ministers insisted the move was not a "Plan B".
More top news
Team Sky's Chris Froome is set to win the 2016 Tour de France with an overall lead of four minutes 11 seconds after stage 20 to Morzine
The attempted kidnap of an RAF serviceman took place near RAF Marham, in Norfolk, on Wednesday.
Footballer Marc Crosas faces an investigation after pictures emerged of him appearing to have speared an endangered baby shark