The First Minister has called for an asylum seeker camp in Penally to close "as quickly as possible".
Mark Drakeford criticised the Home Office for behaving "unacceptably" and failing "to address serious issues regarding living conditions at Penally military camp".
The military training facility near Tenby is being used to house around 250 asylum seekers while their claims are being processed.
The use of the site, and communication of the decision to use it, has previously been criticised by people who live nearby and by Dyfed-Powys Police and Crime Commissioner.
In response to Mr Drakeford's comments, the Home Office said: "The Government takes the wellbeing of asylum seekers and the communities in which they live extremely seriously."
In a statement, Mr Drakeford said there were serious concerns about living conditions at the base. He added that it was "unacceptable" that these have "failed" to be addressed by the Home Office.
"The Welsh Government and local service providers have continually informed the Home Office of grave deficiencies in the standard of accommodation for asylum seeker", he said.
"Home Office has so far failed to act in any meaningful way.
"The welfare and safety of asylum seekers on site must not be compromised, and the wellbeing of the local community must be treated as priority by the Home Office."
The First Minister said the use of the site to house asylum seekers "must end as quickly as possible".
He said that he has raised "legitimate concerns" with the Home Office on numerous occasions but they have failed to act.
Mr Drakeford pledged to continue working "with local partners to minimise risks and maximise the welfare of all affected" until the Home Office does take action.
He added that "Wales is a Nation of Sanctuary. We take that seriously, and so should the Home Office".
A spokesperson for the Home Office responded: "The Government takes the wellbeing of asylum seekers and the communities in which they live extremely seriously.
"Our asylum system is broken and we are determined to introduce a new system that is firm and fair."
They also claimed that they have "taken every effort to ensure" the site is operated safely and they continue to work "collaboratively with stakeholders to address concerns".
The Home Office said it aims to eventually be able to house asylum seekers within the asylum estate without having to use temporary accommodation.
They added, where contingency housing is used, a "detailed assessment is carried out to ensure asylum seekers have the support services they need".
Dafydd Llywelyn, the Police and Crime Commissioner for Dyfed-Powys, previously described the move to house asylum seekers at the base as "totally unacceptable".
He said it showed a "lack of respect" to residents in Penally and the surrounding area, as well as local services.
Some people living nearby also expressed their concerns and staged a protest when the asylum seekers were moved in. On several weekends, dozens of people - both against and in support of the camp - have campaigned outside the facility.
The Home Office said an increase in Channel crossings, coupled with the need to adhere to coronavirus guidelines has led to less accommodation space being available.