People living in Watchet say business and tourism are being damaged because of the amount of mud in the town's harbour.
The harbour owner has told ITV West Country he wants the local council to help solve the problem and has called for a major dredge to fix it.
But Watchet Harbour Marina Ltd, who looks after the marina, says a new lock gate is the only answer to control the mud levels.
At low tide the mud is so high that boats are unable to float.
Mike and Cathy Yuill currently have a boat moored in Watchet but are considering leaving because of the severity of the issue.
As well as pleasure boaters, the harbour is home to several businesses.
Steve Yeandle takes people on sea angling trips and said the situation is making him "feel sick".
He said: "I just want a quiet life. I just want to run my trips and enjoy my life but instead I've got nothing but stress associated with doing the job and, quite frankly, it's not fair and it has gone on too long."
The Watchet Boat Owners Association believes the mud is putting people off from visiting the town which is now affecting tourism and trade.
Liz McGrath, from the organisation, said the shops, pubs, fish and chip shops and local makers "need more people" to help their businesses thrive.
She continued, "The greatest amenity, potentially - the harbour - isn’t doing that at the moment.”
Boat owners and locals want to see the harbour’s dredger on the water to prove it works. They also believe an expensive one is necessary.
According to Somerset West and Taunton Council this could cost between £400,000 and £1.2 million.
Both the inner and outer harbours in Watchet are owned by the council. Watchet Harbour Marina Ltd owns the inner harbour leasehold.
Tim Taylor, from the company, said he wants a sustainable and profitable business long-term which means improving the situation.
Councillor Alan Wedderkopp, Executive Member for Environmental Services at Somerset West and Taunton Council, said: “The council is keen to explore solutions and would support in principle the marina operator’s ambitions around a tidal gate."
He continued: "In the shorter term we are working with the Mud Working Group, a subsidiary of the Watchet Harbour Advisory Committee, to bring all interested parties together and produce a business case that looks not only at the immediate mud impacts but also the economic impacts.”
Read more: Large fire breaks out at Torquay harbour