Powers to fine litter louts have been used just a handful of times over the last three years by some councils as they spend millions on street cleaning.
But while one campaign group is calling on local authorities to use the £150 sanctions more - and increase the punishment - others say financial penalties will not bring about the cultural change needed to stop the problem.
In Tendring in Essex, not a single fine was issued between 2019 and 2021 while the council allocated £1.5m to its street sweeping budget every year during the same period.
Just one person was fined in that time in both Rutland and North Norfolk, while in North Hertfordshire, powers to fine litterers were exercised three times.
Campaign group Clean Up Britain has launched a national petition lobbying the government to make it mandatory for councils to enforce the law on littering while also increasing the fines to £1,000.
Founder John Reed said around 70% of councils in England are not enforcing the law and called littering a "crime against society".
He added: "We believe there is a strong appetite among the public for people to be fined a serious amount of money for littering - currently, it's only £150 although, perversely, people can be fined £1,000 for not cleaning up their dog 'litter' or mess."
Some councils are already using their powers frequently. Harlow Council went from issuing just 40 fines in 2019 to nearly 6,000 in 2021 while in Bedford nearly 7,500 fines were given out during those three years.
But the countryside charity Campaign to Protect Rural England (CPRE) believes fines should only be used as a last resort.
Jonathan Elvin, from CPRE Buckinghamshire, said: "Communication and education is where you should start.
"Look at how you can motivate people to change or improve their behaviour. The plastic bag charging has done really well in reducing the number of plastic bags we use."
The CPRE is calling for a deposit return scheme to be brought in for plastic bottles - where people would pay a small amount up front when buying a product in a bottle but get it back when they recycled it.
Mr Elvin said similar schemes in Norway and Germany had seen waste from glass and plastic bottles reduced by 90%.
For those councils not using their fining powers at the moment, the hope is that other measures are helping to change behaviour.
Councillor Michael Talbot, of Tendring District Council, said: "Issuing fines to people who litter is one tool in the box but we would far rather not have people litter in the first place."
He said the authority was not frightened to use its powers, but favoured education and simply asking people to pick up their litter first.
North Norfolk District council said it found providing plenty of bins encouraged visitors to be responsible and, despite an influx of tourists each summer, had relatively low levels of littering.
A spokesman added: "Historically in north Norfolk, the issuing of fines had little impact on the rates of littering."
North Herts Council said tight budgets and challenges around enforcing the law played a part in its low use of fines but insisted they could not solve the problem on their own.
As well as consulting on a deposit return scheme for bottles, the government banned the use of plastic straws and cotton buds and has carried out consultations into banning other single-use plastic.
Want a quick and expert briefing on the biggest news stories? Listen to our latest podcasts to find out What You Need To Know