The Treasury said it was "important that people pay the tax they owe on time" after plans to allow the taxman to seize money from personal bank accounts were criticised by a group of MPs.
A spokesman said: "Although the vast majority do this, there is still a minority that chooses not to pay, despite being able.
"The proposed powers will give HMRC [HM Revenue and Customs] another tool to collect tax debt owed.
"The current consultation includes a range of safeguards to ensure the power is tightly targeted."
The Commons Treasury Select Committee has highlighted the potential for fraud and error if the taxman was given direct access to millions of accounts.
"This policy is highly dependent on HMRC's [HM Revenue and Customs'] ability accurately to determine which taxpayers owe money and what amounts they owe, an ability not always demonstrated in the past," the MPs said.
"Incorrectly collecting money will result in serious detriment to taxpayers," the report continued.
"The Government must consider safeguards, in addition to those set out in the consultation document, to ensure that HMRC cannot act erroneously with impunity.
An influential group of MPs said giving the taxman the power to recover money directly from personal bank accounts without some form of prior independent oversight would be "wholly unacceptable".
The Commons Treasury Committee also dismissed George Osborne's argument that the Department for Work and Pensions (DWP) already had similar powers to collect child maintenance.
The parallel is not exact: in those cases, DWP is acting as an intermediary between two individuals.
HMRC [HM Revenue and Customs] would be acting not as an intermediary between two individuals but rather in pursuit of its own objective of bringing in revenue for the Exchequer.
Plans to allow the taxman to seize money directly from personal bank accounts have been criticised by an influential group of MPs.
The Commons Treasury Committee has expressed "considerable concern" over the Chancellor's debt collection proposals and called for further scrutiny.
In their report on this year's Budget, the MPs suggested the change could amount to a back-door reintroduction of the discredited Crown Preference rule - which gave HM Revenue & Customs (HMRC) priority access to assets when companies went bust.
Starbucks' decision to move its European headquarters from the Netherlands to London is a "ringing endorsement" of the capital's business environment, according to the chief executive of the London Chamber of Commerce and Industry, Colin Stanbridge.
This very positive move by Starbucks greatly reinforces London as a key global centre for business and a highly desirable location for firms to base their operations.
Creating the right environment for businesses to flourish is essential to London competing at an international level and we are delighted that Starbucks has given the capital a ringing endorsement.
Coffee chain Starbucks says it will "pay more tax in the UK" in the future after opting to move its European headquarters from the Netherlands to London.
The company said the move would make it "better able to oversee the UK market".
Starbucks has come under scrutiny over its tax affairs in the past, with the company telling a parliamentary committee in 2012 that it had not made a taxable profit for 14 of the 15 years it had been operating in the UK.
Chancellor George Osborne told ITV News the government is introducing a new criminal offence for anyone who fails to declare their offshore income.
"The message is very simple. If you are hiding your money offshore, we are coming to get you," the Chancellor said.
ITV News correspondent Lewis Vaughan Jones reports:
At the moment officials must prove that a person holding income offshore intended to avoid paying tax - the proposed changes would mean they would only have to show the money was taxable.
British workers and businesses will this week benefit from biggest tax cuts in a generation, George Osborne will claim.
Measures including the rise in the income tax personal allowance to £10,000 and cuts in business taxes will be brought in over the coming days.
Tomorrow the corporation tax rate will be cut by 1% to 21% and reforms to business rates will be introduced. The annual investment allowance for firms will be doubled to £500,000.
The £10,000 income tax personal allowance comes into effect on Sunday, along with the employment allowance which cuts employers' National Insurance contributions by up to £2,000.
Giving a speech in Essex later today, Mr Osborne will say that Sunday will be "the culmination of this week that sees the biggest reduction of business and personal tax in two decades".
Former Conservative Chancellor Lord Lamont has told ITV News George Osborne should raise the threshold at which workers pay 40% income tax.
He said that some the current threshold was meaning some "not very well off" workers such as teachers, nurses and train drivers are being "caught" by the higher rate.
Lord Lamont added that he hoped a pledge to raise the threshold would be included in the next Conservative Party manifesto.
George Osborne said the Government's latest clampdown on tax evasion`would make it very clear to people that if they do not pay their taxes then "we are coming after them".
The Chancellor said, "It is not fair for people to do that, it means that other people have to pay higher taxes and ultimately, as a society, we've all got to make a contribution."