Conservative Cabinet minister Ken Clarke said he does not believe it is possible to return money to the Government after Iain Duncan Smith urged wealthy pensioners to hand their benefits back.
Mr Clarke, 72, refused to say whether he returned the universal benefits he is entitled to.
Speaking on Sky News' Murnaghan programme, Mr Clarke said, "It is certainly the case when it comes to a bus pass and when it comes to the winter fuel all taxpayers should decide and recipients should decide what to do with it themselves".
"You can't hand it back to the Government. I don't think it is a system for doing that. Every pensioner and retired person like myself has to make up their own mind about whether they really need it and whether they are going to give it to some worthwhile cause", he continued.
"No doubt most pensioners who are reasonably prosperous give quite a lot of money to charity and worthwhile causes in any event".
More top news
A pensioner who is believed to be armed with a gun is in a stand-off with police.
One of the world's longest-running conflicts is set to come to an end.
Four people have been stabbed in a series of knife attacks at the Notting Hill Carnival.