Stephen Crabb, the new work and pensions secretary, has announced the government will abandon controversial cuts to the Personal Independence Payments.
"I can tell the House we will not be going ahead with the changes to PIP that had been put forward," he said.
He added that the government would not fill the savings gap left in the budget by the U-turn with further cuts to welfare.
The announcement had been widely anticipated following the resignation of Iain Duncan Smith, Mr Crabb's predecessor.
Mr Crabb paid "huge tribute" to Mr Duncan Smith, and appeared to try to win back the support of disabled people, saying he believed it was possible to do "so much better" for them.
Crabb: "we will not be going ahead" with changes to personal independence payments - & he won't have to fill £1.4bn hole in budget. Amazing
Jeremy Corbyn has lambasted the government's budget, saying it had "fallen apart" within days of its announcement and suggesting the chancellor should resign as a result.
The Labour leader accused David Cameron of "covering up his great friend" George Osborne by allowing the chancellor to avoid answering questions in the House of Commons on a budget that had "inequality at its core".
Mr Corbyn noted that Mr Osborne was not present, while "practically every other cabinet minister is here today" and said the chancellor should "consider his position and look for something else to do".
Mr Cameron responded that the chancellor would be in parliament on Tuesday "winding up the budget debate".
David Cameron has defended the budget and attempted to claw back some of his image as a social reformer amid a row over planned cuts to disability benefits.
The prime minister closed a statement to the House on the EU migrant crisis with a reference to the budget row that was sparked this weekend by the resignation of former work and pensions secretary Iain Duncan Smith.
Mr Cameron said the MP for Chingford and Woodford Green had much to be proud of in helping the government to reform welfare benefits.
He said "we must continue to cut the deficit, control the cost of welfare and live within our means", and highlighted government plans for the introduction of the national living wage and prison reform.
Prime Minister David Cameron is addressing the House of Commons amid the row over planned cuts to disability benefits that led to the shock resignation of Iain Duncan Smith as work and pensions secretary.
David Gauke, the financial secretary to the Treasury, has defended the government's budget, saying the government was "delivering for Britain".
"Jobs, growth - this is the record of this government. That is the record for the chancellor of the exchequer, and it's a record to be proud of," he said.
Mr Gauke was responding to accusations by Labour's John McDonnell that the government's budget was in "chaos" over the row concerning cuts to disability benefit.
John McDonnell called on the Chancellor George Osborne to "withdraw" his budget "and start again", saying the budget was in "absolute chaos".
"This is no way to deliver a budget and no way to manage an economy," he said, as part of an emergency question in parliament.
Labour's John McDonnell has called on Chancellor George Osborne to make a statement on changes to the budget.
"I ask it because the budget process is in absolute chaos," he said.
The shadow chancellor was granted the emergency question, allowing him to call on Mr Osborne to make a statement about changes to the budget.
The response came from David Gauke, the financial secretary to the Treasury Mr Osborne was not present in parliament.
Poorer working-age families will be "hit hard" by government tax and benefit changes while pensioners will be protected, a think-tank says.Read the full story ›
The new secretary of state for work and pensions Stephen Crabb has met with his staff on his first day in the role.
Chancellor George Osborne "absolutely" has the full confidence of the Prime Minister, a spokeswoman for David Cameron has said.