Paul Flowers, the former banking boss of the Co-operative Bank, was convicted of gross indecency in 1981, reportedly over a sex act in a public toilet.
Mr Flowers, who was was caught allegedly buying and using illegal drugs in a newspaper sting at the weekend, led the Co-operative Bank for three years from 2010.
Today it also emerged that the minister resigned from Bradford Council in 2011 after "inappropriate" adult content was found on his computer.
At the time he citied personal reasons and increased responsibilities at the at the Co-op.
A church spokesman said that Mr Flowers was suspended after his conviction for gross indecency and that he went through the "usual procedures" before being allowed to continue in his role.