"I can only sincerely, and I mean heartfelt, apologise for all affected. All affected," he stressed at a press conference on Wednesday.
"As a father of two young women I found it deeply distressing and I can only apologise for all affected by this.
"I find it devastating. And I won't back off the challenge that we need to push this forward."
His words come after Charlotte Wathan, a former WRU general manager of women's rugby, claimed she faced offensive comments by a colleague in a BBC Investigation.
She said the comments left her 'in tears' and 'feeling sick'.
The WRU must now address the allegations amid claims that there is a 'toxic culture' at the organisation.
Evans, who has only recently taken on the role as Chair, revealed an external task force would be appointed to review the Union and make recommendations.
The scope of that review is yet to be determined.
"My ambition here and what I aim to do - as I said, have a mandate as chair - is to bring in some external expertise and establish a task force, for want of a better expression, to ensure we review everything and ensure that we establish, we develop the right culture," he said.
"We will require external expertise, I'm certain of that. We need that. This can't purely be an internal review. This has to be an external one.
"But just to reassure you, this will be done. It's my responsibility to ensure that happens.
"I want to ensure that our organisation is fit to be a modern outward looking organisation."
Last night, the WRU's CEO Steve Phillips - who yesterday insisted he didn't think it was 'necessary' for anyone to lose their jobs over the allegations - wrote to member clubs.
In it, he admitted the WRU's culture was not where it needed to be between the years of 2017 and 2019.
At that time, Phillips was on the Union's executive board in his previous role as Finance Director.
But Evans would not be drawn when asked if his position as the current CEO was tenable.
"This is not about individuals," said the Chair.
"This is a collective, culture is a collective. It's all of us. It's we, it's us. It's not an individual. It's more than that. This is a far more important subject than purely one individual.
"We need to embrace the challenge together. That's how we bring this to where we need to bring it."