Conservative Cabinet minister Ken Clarke said he does not believe it is possible to return money to the Government after Iain Duncan Smith urged wealthy pensioners to hand their benefits back.
Mr Clarke, 72, refused to say whether he returned the universal benefits he is entitled to.
Speaking on Sky News' Murnaghan programme, Mr Clarke said, "It is certainly the case when it comes to a bus pass and when it comes to the winter fuel all taxpayers should decide and recipients should decide what to do with it themselves".
"You can't hand it back to the Government. I don't think it is a system for doing that. Every pensioner and retired person like myself has to make up their own mind about whether they really need it and whether they are going to give it to some worthwhile cause", he continued.
"No doubt most pensioners who are reasonably prosperous give quite a lot of money to charity and worthwhile causes in any event".
More top news
Five Palestinians were reportedly shot dead by Israeli soldiers after a fresh spate of clashes broke out in the conflict-ridden region.
This weekend: Patchy rain in the northwest, dry and bright elsewhere.
Former ITN newscaster Gordon Honeycombe has died aged 79 in Australia, his family has said.