An estimated 65 million seasonal garments are stashed away in Britain's closets, Hubbub said.
Research by the charity of 108 jumpers available this year from 11 different high street and online retailers found 95% of were made wholly or partly of plastic materials.
A further study by Plymouth University found that acrylic was responsible for releasing nearly 730,000 microfibres per wash, five times more than polyester-cotton blend fabric, and nearly 1.5 times as many as pure polyester.
Based on its research, Hubbub has asked people to swap jumpers with friends by organising a "Christmas jumper amnesty" in places of work and schools.
Sarah Divall, a project co-ordinator at Hubbub, said: "We don't want to stop people dressing up and having a great time at Christmas, but there are so many ways to do this without buying new.
"Fast fashion is a major threat to the natural world and Christmas jumpers are particularly problematic as so many contain plastic.
"We'd urge people to swap, buy second-hand or re-wear and remember a jumper is for life, not just for Christmas."