New research for ITV News shows that 20% of consumers are now considering stockpiling in case of a "no-deal" Brexit - and 5% have already started.
The food industry is warning that gaps on supermarket shelves are possible if we leave the EU without a deal, but says it's unlikely that households hoarding supplies will help.
Our survey shows Remain voters are far more likely to be planning to stockpile and that there is also significant concern about the supply of medicines.
The new survey for ITV News spoke to just over 2000 people.
We found 36% of those who voted Remain are considering stockpiling, or are already doing so.
But far fewer - just 15% - who voted Leave are in that category.
Of people stockpiling or considering it , almost a third (27%) reckon they'll build up to a month's supply of the goods.
Tinned and dried foods like pasta are top of the list of goods being hoarded.
55% of those stockpiling or considering it are worried about medicines.
Source: ITV News/ComRes Poll
We spoke to Grant Humphries in Fort Augustus near Loch Ness, Scotland.
He said: "We have two large industrial-sized bags of pasta, about 20 cans of soups, freeze dried foods and pastas, some dried cereal, a couple of 2kg bags of oatmeal, 3 bags of milk powder, we have Band-Aids and antiseptic creams".
The family, with two young children, has spent around £200 stockpiling enough food for three months in case a "no deal" Brexit brings disruption to food imports.
The food industry warns that consumers who stockpile are probably hoarding the wrong goods, with any gaps on the shelves far more likely to be in the fresh fruit and vegetables aisles.
Retailers say they already have additional stocks of non-perishable groceries and other consumables such as toiletries.
Irene Hughes in Liverpool is stockpiling tins and dried foods, but that's not her main concern.
She told us: "My biggest worry is the medication. It is a tremendous worry because nobody has mentioned it at all.
"I don't know whether the pharmaceutical companies are 'just in time' with their medication or whether they do have stockpiles."
Irene cares for her husband, Graham, who has dementia.
"The medication he is on is medication he can't stop taking, you can't stockpile them," she said.
"You can only order your prescription the week before you need the prescription, and that gives you a month's supply of your tablets.
"It doesn't go into two months, it doesn't go into three months."
Responding to our new research, the government says it is aiming to avoid "no-deal".
The Department of Health added that it has asked many pharmaceutical companies to ensure that they will have a minimum of six weeks' supply of medicine - in addition to the stocks which they usually hold - by 29th March, to ensure that patients can continue to receive the medicines they need.