Keith Vaz said the former South Yorkshire Police chief constable's explanation of his handling of child sex exploitation in Rotherham was "totally unconvincing".
Mr Vaz said Meredydd Hughes faced "serious questions" over his role.
"We find, on this committee, your evidence totally unconvincing in that there are still serious questions to be asked of the way in which you have conducted yourself," Vaz said.
"Your contrition today, I think, is welcome but, I think, it needs to go much further and certainly we will be referring the evidence you gave to us today to the bigger Rotherham inquiry that the Home Secretary has announced."
By Juliet Bremner: ITV News Correspondent
A researcher who looked into allegations of child sexual exploitation and grooming in Rotherham back in 2001 has told the committee of MPs that she was approached and warned off by two police officers.
She told the committee in private that she was told that the perpetrators knew were she lived and she said that she feared for her life.
MPs said they were deeply shocked and asked for guarantee that these allegations would be investigated.
Ex-South Yorkshire Police chief constable Meredydd Hughes says he "failed" the victims of child abuse in Rotherham.
"I have had a 32-year police career and yet on this issue I have singularly failed the victims of these criminals and it hurts," Mr Hughes told MPs.
Mr Hughes said he felt "sick" reading Professor Alexis Jay's report, but insisted he had "no idea" of the scale of the problem during his time in charge.
The report found at least 1,400 children were sexually exploited in Rotherham by gangs of men over a 16-year period.
The former chief constable of South Yorkshire Police insisted he had "no idea" of the scale of child exploitation in Rotherham.
"At the time I was both deputy and chief constable, I had no idea of the scale and scope of this type of organised crime," Meredydd Hughes told MPs.
The Home Affairs Committee chairman Keith Vaz said his committee found Mr Hughe's claim "impossible to believe".
Hughes said the crimes were "not something I would have turned a blind eye to nor wilfully ignored".
Twenty-five new victims have now come forward since the release of a report into child sex abuse in Rotherham, South Yorkshire Police chief constable David Crompton said.
Authorities in Rotherham caring for vulnerable young people "clearly had a culture" of ignoring girls who were groomed and sexually abused by gangs of Asian men, the MP for neighbouring Bassetlaw has said.
John Mann dismissed claims authorities did not intervene because they were worried about racial sensitivities and instead regarded the victims "as almost sub-human".
A "big minority" of British children are subjected to sexual abuse, with as many as one in four falling victim to "this dreadful social problem", the head of a children's charity told Good Morning Britain.
Founder of the National Association for People Abused in Childhood, Peter Saunders, said the convictions of perpetrators was "being driven by survivors and the media".
The mass sexual abuse which took place in Rotherham between 1997 and 2013 is "deeply troubling", according to the head of an influential group of MPs.
Home Affairs select committee chairman, Keith Vaz, said MPs wanted to know how this abuse "was allowed to go unchallenged for so long":
The revelations from the Independent Inquiry into Child Sexual Exploitation in Rotherham are deeply troubling. The Committee will be questioning those in positions of authority at the time that these offences were being committed on how the sexual exploitation of children on a horrifying scale was allowed to go unchallenged for so long.
We will also be hearing from Peter Wanless and Richard Whittam QC about progress in their ongoing review of how the Home Office handled historical allegations of child abuse.
Shaun Wright, the South Yorkshire PCC who was heavily criticised after mass sexual abuse was exposed in Rotherham, will appear in front of MPs alongside other high ranking officials.
Mr Wright will answer questions from the Home Affairs select committee after a report revealed 1,400 children in and around Rotherham had been sexually abused by gangs over a sixteen year period.
Despite numerous calls for him to step aside, Mr Wright, head of children and young people's services from 2005-10, only resigned from the Labour party.
He was accused of failing to act on numerous warnings of widespread sexual abuse in Rotherham, which last month's report revealed took place between 1997-2013.
The chief executive of Rotherham Council, Martin Kimber, stepped down yesterday.
Martin Kimber was the chief executive of Rotherham Council for five of the 16 years covered in the report into child sexual exploitation.
He was one of the officials interviewed by Alexis Jay OBE, although he is not mentioned by name in the report.
According to his LinkedIn account, he previously served as strategic director of Wigan Council for seven years before joining Rotherham as chief executive in October 2009.