Lib Dem Care Minister Norman Lamb has said the evidence was "clear" that the controversial "spare-room" policy was not working.
He told BBC2's Newsnight that in his North Norfolk constituency he came across "many cases" where people may be willing to move, but the state of the housing market and the shortage of social housing just makes it impossible.
"So they are stuck in a situation," he said.
"It was also wrong for disabled people who needed an extra room for a carer to use or to be adapted in some way to be hit".
The Liberal Democrats called for a reform of the bedroom tax in a dramatic withdrawal of support for the controversial welfare reform.
More top news
Portman was due to receive the award in Israel in June and said that her reasons for skipping the ceremony were solely due to Mr Netanyahu.
Analysis of Department for Work and Pensions data found the average man receives £153.86 a week and the average woman receives £125.98.
The country says it is making the move to shift its national focus and improve its economy.