Defence Secretary Philip Hammond said he is "shocked" by the allegations about Nigel Evans and suggested it could be "very difficult" for the MP to continue as deputy speaker while the matter was being investigated.
"I know Nigel well, I have known him for years. I'm obviously as shocked as everybody else is," he told BBC1's The Andrew Marr Show.
"I stick rigidly to the view that we should treat people as innocent until they are proven guilty but it is quite difficult to carry out a sensitive and high profile role while being under this kind of scrutiny."
Evans today branded the allegations as "completely false", adding that they had been made by two people he had regarded as friends.
Deputy Commons Speaker Nigel Evans said allegations that he raped one man and sexually assaulted another are "completely false".
Deputy House of Commons Speaker and Conservative MP Nigel Evans has been arrested on suspicion of rape, sources said.