Mr Vaz said it was "essential" trust between the federation and the public was restored.
However, he empathised with members of the public who felt let down by the police as he expressed some dismay that the "only people not to apologise" for "Plebgate" were the Police Federation.
The Police Federation faces a "very significant programme of reform" in order to become "a trusted voice for frontline police officers", said a former high-ranking civil servant investigating the body.
Sir David Normington, a former Home Office permanent secretary, revealed "91% of federation members think there needs to be change" and said the body needed to "rebuild trust" after a number of police scandals.
Sir David was speaking in a video released ahead of the publication of what is expected to be a damning report into the Police Federation.
The body representing policemen and women in England and Wales is expected to be told to make sweeping changes to the way it is run, governed and funded.
In a report published later today, Sir David Normington is expected to tell the Police Federation to reform after a number of scandals rocked public trust in forces across the country.
The report follows an interim one which warned the federation it risked becoming an "irrelevance" and had "turned in on itself".
The review has examined whether the federation still acts as a credible voice for officers, genuinely serves the public good and functions as an organisational democracy.
The Police Federation of England and Wales, which represents tens of thousands of officers, has said that the recommendations from the Home Affairs Select Committee could lead to greater confusion:
The chairman of the national Police Federation Paul McKeever died from a suspected embolism, the organisation has said:
The Metropolitan Police Federation has announced that the outgoing national chairman Paul McKeever has died:
The Police Federation of England and Wales has criticised the "half-hearted apology" from the Chief Whip Andrew Mitchell for allegedly abusing a Downing Street police officer, saying it is "especially disappointing during this tragic week " of the deaths of two police officers:
The man responsible for carrying out a review into police pay and conditions which led to mass protests by officers is set to become the next Chief Inspector of Constabulary (HMIC).
Tom Winsor has been named as favourite by the Home Secretary Theresa May to replace Sir Denis O'Connor when he retires at the end of July.
His recognition for the £200,000-a-year role has been met with criticism from among rank-and-file officers who have focused on his lack of policing experience.
Mr Winsor's reports were part of the most wide-ranging review of policing in more than 30 years and saw more than 30,000 officers taking to the streets in protest last month.
The report included recommendations for police constables' starting salaries to be cut by up to £4,500 and the retirement age raised to 60.