Watch Bob Cruwys' report
A group of people from Exmoor are threatening legal action against the Church of England in a bid to block it from selling a vicarage.
St Mary's Church lies in the centre of the small parish of Oare.
The area is well known to many because of its literary connections.
It provided the setting for RD Blackmore's famous novel Lorna Doone and the place where Samuel Taylor Coleridge wrote: “In Xanadu did Kubla Khan / A stately pleasure-dome decree.”
The church features in Lorna Doone and has a memorial to the author inside.
It still holds a weekly service, but there is no resident vicar after the previous one retired a few years ago.
The diocese now wants to sell it to raise money - but parishioners say that would mean they would be left without their own vicar.
The local congregation, along with the neighbouring parish of Culbone, now fear for the future of their churches.
"We live in a remote rural location, there are no shops, no pub no post office," Oare Culbone parish meeting chairman Jeremy Payne said.
"The church is absolutely key to the heart of the community. We have our parish meetings there. We have our choir practices there and there is nowhere else. And by selling the rectory, there will be nowhere else."
The rectory next to the church was built in the 1960s with funds raised by local families on land donated to the church.
It is currently being rented out to a private tenant. But if sold, the residents say it will be impossible to attract another vicar to their community.
Churchwarden Geraldine Woollacott told ITV News: "We always had a village hall just down the road, but that's been sold off now.
"So the church in the rectory has been our, you know, our community really and sort of everything's put up here on the noticeboard, what's happening. Really we're desperate to keep a rector here."
Under the new proposals, Oare and Culbone would be combined with a group already overseen by the vicar of Porlock.
However, residents say that could mean a reduction in the vital pastoral care required by such remote communities.
"If we are included in her patch, she will then have nine parishes to look after and it's just impossible," added churchwarden Graham Story.
"And again, you know, it is the pastoral care, from my point of view that is so important."
The diocese says the potential sale of the rectory is still dependent on the results of a consultation and insists there are no plans to change the pattern of worship.
In a statement, it said: "The Diocese of Bath and Wells is committed to rural ministry and the life of the church in Oare with Culbone.
"We are a largely rural diocese and are seeking to align our resources to support all our churches in flourishing. The consultation process with parishes on how the needs of the church in Oare with Culbone will be met is ongoing."