Citizens Advice has said an extra room "can be a lifeline" for a disabled person, calling it "as vital as a guide dog or a wheelchair".
Chief executive Gillian Guy said for many people a spare room "is essential to their wellbeing" as it is needed for dialysis equipment or room for a carer.
Ms Guy said: "Many sick and disabled people have bedrooms which have been adapted to help them to cope with their condition.
"Moving disabled people out of their houses is not only a potential risk to their health, but can cost the individuals affected and councils significant amounts of money if they need to start from scratch in a new home."
The head of homeless charity Crisis has said the Government risks forcing poor people out of their homes because of changes to housing benefit such as the 'Bedroom Tax'.
"The Government must listen to this verdict from MPs across the political spectrum and rethink its cuts to Housing Benefit, particularly for those who cannot change their circumstances by working or who are struggling in low paid jobs. Without prompt action, the Government risks driving homelessness still higher."
Shadow work and pensions secretary Rachel Reeves said it was "completely unfair" that many disabled people were hit by the "bedroom tax" and charged for the space used to store essential medical equipment.
"Britain can't afford another year of David Cameron and Nick Clegg's tax on bedrooms," Ms Reeves said.
"If they won't listen to the country and scrap their hated bedroom tax, then a Labour government will," she added.
The chair of the Work and Pensions Select Committee has said those being hit by the withdrawal of the spare room subsidy are often not those the policy was intended for.
Dame Anne Begg said: "Many of the people being hit by the so-called 'bedroom tax' are in vulnerable groups, very often people with disabilities who can't move house."
The Government has reformed the housing cost support system with the aim of reducing benefit expenditure and incentivising people to enter work.
But vulnerable groups who were not the intended targets of the reforms and are not able to respond by moving house or finding a job are suffering as a result.
The Government's reforms are causing severe financial hardship and distress to vulnerable groups, including disabled people.
Discretionary housing payments (DHPs), which local authorities can award to people facing hardship in paying their rent, are not a solution for many claimants.
Our reforms are necessary to restore fairness to the system and make a better use of social housing. Unreformed, the Housing Benefit bill would have grown to £26 billion in 2013/14.We have given councils £345 million since reforms came in last year to support vulnerable groups, especially disabled people.
The removal of the spare room subsidy means we still pay the majority of most claimants' rent. But we are saving the taxpayer £1 million a day which was being paid for extra bedrooms and are freeing up bigger homes for people forced to live in cramped, overcrowded accommodation.
The work and pensions select committee voted down a proposal from Labour MP Sheila Gilmore to call for the policy, officially known as the removal of the spare room subsidy, to be scrapped but it did urge ministers to exempt:
- anyone whose home has been adapted to help them with their disability
- any household containing a claimant receiving disability benefits at the higher level
- carers living with disabled people should be exempt from the benefits cap
It said that it was "particularly unjust" for homeless people to be subjected to the benefits cap, as they have no choice over the temporary accommodation in which they are placed, which may force them over the limit. The committee called for them to be exempted from the cap.
Disabled people are suffering "severe financial hardship and distress" as a result of the 'bedroom tax', a cross-party committee of MPs has said.
The decision to reduce housing benefit payments from social tenants deemed to have a larger home than they need has hit vulnerable people who were not the intended targets of the reform and have little hope of moving to a smaller property, the House of Commons Work and Pensions Committee found.
The Labour party is claiming more than 20,000 councils tenants have been wrongly paying the so-called "bedroom tax" because of a Government error.
Shadow work and pensions secretary Rachel Reeves says that 194 councils out of 346 who replied to a Freedom of Information request showed that 21,500 people have overpaid.
The coalition says it is working on fixing the loophole which is causing the errors.
Work and Pensions Secretary Iain Duncan Smith challenges Labour's figures, claiming the amount of tenants who have overpaid was likely to be around 5,000.