Disabled people say they are being discriminated against after measures were brought into pedestrianise roads to give shoppers more space to social distance.
East Street, one of the main shopping strips in the town has been shut to vehicles for weeks due to Covid-19. There are fears the pedestrianisation could end up being permanent.
It has led to the founding of campaign group, Taunton Disability Action Group, who say the changes are denying at least 30 parking spaces for people with disabilities.
Susan Glenn has MS and a condition called PMR - polymyalgia rheumatica. It means she has difficulties with walking and balance.
Once I've walked as far as my body says I can, my limbs stop working.
Susan used to be able to park right outside some of Taunton's main shops and banks on East Street but, because of the coronavirus pandemic, it has been shut to vehicles and the local councils are considering keeping it that way.
It is now too far for her to walk to from the parking spaces that are available.
Susan has formed the campaign group in the hope of uniting people like her.
She says, "I do my banking along East Street. My building society's on East Street. The shops I choose to use are on East Street. That has all been taken away.
"They're forcing us online but shopping, for some of us, is a social thing. It's where you get to interact with people and this is just making people more and more isolated."
Simon Clarke is also calling on the councils to think again. His daughter, Ella-Mae, can only walk short distances and he says the closure means she has less freedom.
He says, "What we're hoping to come of this is that more people will have more of an understanding of the every day difficulties that some disabilities can bring.
"Just walking into a shop we don't even think twice about, whereas for someone with some disabilities it can be a major challenge and that might be all they can manage to do for a whole day."
East Street is home to some of the main shops in the centre, including Wilko, Marks and Spencer, TK Maxx and many of the town's banks.
Plenty of shoppers are making the most of the pedestrianisation, probably unaware that disabled people say they are excluded.
Cllr Mike Rigby from Somerset West & Taunton Council says the council is listening to the campaigners. "We've arranged to meet with the disability group that have expressed concerns about how this operates to see if we can add more spaces.
"We've added another four disabled spaces, whereas previously there were none, specifically designated and we're talking to the County Council, our partner on this about maybe designating another four to six in the same street and perhaps in other streets as well.
"So we're very keen to make sure that no-one feels that they can't visit the town any more because of parking issues."
Susan Glenn says those extra spaces still make East Street inaccessible for her and is concerned that the ban is set to continue beyond Covid-19 in a bid to encourage walking and cycling in the town.
The measures were funded under the Department for Transport's £250m emergency active travel fund.
If moves to secure funding to keep the closure in place are successful, a public consultation would be undertaken.