The chancellor cut duty on homes sold up to £300,000 in his Wednesday Budget, a boon to young people hoping to get onto the property ladder.
However, the OBR responded by predicting the move would increase house by around 0.3%, hindering first-time buyers while helping people that already owned a property.
Treasury Chief Secretary Liz Truss dismissed the OBR’s forecast as no more than a "minor increase."
She said the Government wanted to offer short-term help to people who had been struggling for years to buy a home.
"I think the main gainers are people who will now be able to buy their own home and get on the housing ladder," she told BBC2's Newsnight.
"Of course we need to increase the supply of homes. But we really felt it was necessary to do something now to help those who have struggled for a number of years. That is what this measure is about."
Treasury sources argued the measure, which will also apply to the first £300,000 of homes worth up to half a million pounds, would be a welcome boost to first-time buyers, with 95% seeing a cut in the amount of stamp duty paid and 80% paying none at all.
The OBR report said cutting stamp duty cut could result in as few as 3,500 additional home purchases a year, while costing the Treasury more than £3 billion by 2022/23.
"The Chancellor announced no measures to directly increase house-building and, without that, lifting stamp duty for some will only drive up prices and benefit sellers," said Labour Shadow Chancellor John McDonnell.
The OBR report also issued a downbeat economic forecasts, downgrading growth prediction for each of the next five years due to the UK's poor productivity performance, with OBR chairman Robert Chote noting it was now "unlikely" Hammond would achieve his target of eliminating the deficit in the public finances by the mid-2020s.
Despite this, Mr Hammond announced investments including:
An additional £15 billion for housing in a bid to reach the target of building 300,000 extra new homes a year by the mid-2020s
An extra £7.5 billion for the NHS over the next five years
Another £3 billion to prepare for the impact of Brexit
The extra cash for health includes £2.8 billion for day-to-day spending to help NHS England deal with pressures this winter, while Mr Hammond also promised to fund any increase in health staff pay resulting from current negotiations on improving efficiency.
NHS England chairman Sir Malcolm Grant said while the money would go "some way" towards filling the funding gap, the health service could "no longer avoid the difficult debate" on what it could provide on the funds it has.