Conservative Cabinet minister Ken Clarke said he does not believe it is possible to return money to the Government after Iain Duncan Smith urged wealthy pensioners to hand their benefits back.
Mr Clarke, 72, refused to say whether he returned the universal benefits he is entitled to.
Speaking on Sky News' Murnaghan programme, Mr Clarke said, "It is certainly the case when it comes to a bus pass and when it comes to the winter fuel all taxpayers should decide and recipients should decide what to do with it themselves".
"You can't hand it back to the Government. I don't think it is a system for doing that. Every pensioner and retired person like myself has to make up their own mind about whether they really need it and whether they are going to give it to some worthwhile cause", he continued.
"No doubt most pensioners who are reasonably prosperous give quite a lot of money to charity and worthwhile causes in any event".
More top news
Six people have begun living in a dome in Hawaii to simulate what life would be like on Mars.
The 12th series of X Factor kicks off on ITV this Saturday but which previous winners went on to make it big?
A mother and her four young children who are feared to be travelling to Syria are being sought by police.