Lord Ahmed - who has resigned from the Labour Party - said he was "very disappointed" with the way he had been treated by Labour.
"I don't want to appear before any kangaroo court where the rules of justice have been denied. All I want is any evidence in front of me so I can look at it and defend myself. I can't defend myself if there is no evidence produced."
"I don't want my friends in the Labour to think that I have walked away but the rules of justice demand it. It's very unfortunate."
Lord Ahmed said he had "never been anti-semitic" and had "unreservedly apologised" for any offence caused to the Jewish community.
He added his "heart" is still with the Labour Party.
The Rotherham peer was due to face a hearing over allegations he made anti-Semitic comments in a TV interview.
More top news
Ken Cresswell and John Shaw, from Rotherham, were among four men to lose their lives after the collapse at the Didcot A plant a year ago.
An amber alert for gales will be in place across the North, Midlands and East today between 6am and 6pm in response to the conditions.
Jonathan Binns, 32, from Ings Crescent, was found in The Town in Thornhill with multiple injuries last Friday at around 8.40pm.