Sir Bob Neill, who chairs the Commons Justice Committee, added his name to a slowly-growing list of Tories urging Mr Johnson to quit on Friday.
“Trust is the most important commodity in politics, but these events have undermined trust in not just the office of the prime minister, but in the political process itself. To rebuild that trust and move on, a change in leadership is required,” he said
Stephen Hammond has also sent in a letter, saying in a statement he "cannot and will not defend the indefensible", adding he does not believe his party can regain public trust with Mr Johnson in charge.
John Baron attacked a "shameful pattern of misbehaviour" at the heart of government during the coronavirus pandemic and said "his repeated assurances in Parliament that there was no rule-breaking is simply not credible".
David Simmonds meanwhile said it was "clear" the public no longer supported Mr Johnson, adding: "It is time for him to step down so that new leadership can take forward the important work of the government".
They followed Julian Sturdy who was the first Tory MP to call on Mr Johnson to quit following the publishing of Ms Gray's report.
David Wood rounds up the latest on the backbench Tories calling for Boris Johnson to resign over Partygate
2019 entrant Alicia Kearns said the report demonstrated Mr Johnson had been “complicit in the holding of many goodbye parties for his staff” which the backbencher said “displayed a complete disregard” for Covid restrictions in place at the time.
The Rutland and Melton MP said she had reached the conclusion that the “Prime Minister’s account of events to Parliament was misleading” - she did not confirm she had submitted a letter of no confidence, but it's been report she has.
Tory MP Paul Holmes, meanwhile, has resigned as a ministerial aide after saying he was “shocked and angered” by the behaviour described in the report.
The Eastleigh representative wrote online: “As I have always made clear I, like most of you, was shocked and angered by the revelations when so many people across Eastleigh followed the rules and sacrificed many things in the need to stop the spread of the virus.”
Want a quick and expert briefing on the biggest news stories? Listen to our latest podcasts to find out What You Need To Know
He said a “deep mistrust” in the Tories has been created by the lockdown breaches as he tried to represent constituents with integrity.
“It is distressing to me that this work on your behalf has been tarnished by the toxic culture that seemed to have permeated Number 10,” he wrote.
“Over the last few weeks this distress has led me to conclude that I want to continue to focus solely on my efforts in being your Member of Parliament and the campaigns that are important to you. That is why I have now resigned from my governmental responsibilities as a Parliamentary Private Secretary at the Home Office.”
There are now at least 22 backbench Conservatives publicly urging him to go and it's likely others have submitted letters of no confidence without going public.
It appears the prime minister is struggling to move on from Ms Gray's critical report into Downing Street parties, which blamed the senior leadership in Number 10 for a party culture in government throughout various lockdowns.
Ms Gray said there had been a "failure of leadership" in No10 and Mr Johnson told MPs he takes "full responsibility" for the rule-breaking.
But, he said it was time to move on, telling a Downing Street press conference his priority is to now focus the cost-of-living crisis.
But shifting the focus to the cost of living does not appear to be enough to have satisfied Brexit-supporting Mr Baron, who said in a statement: "To knowingly mislead it can not be tolerated, no matter the issue. Whether or not the prime minister is an asset to the party or the country is of less importance.”
"Having always said I would consider all the available evidence before deciding, I’m afraid the prime minister no longer enjoys my support – I can no longer give him the benefit of the doubt.”
Mr Baron's call for the prime minister to resign is being viewed as significant as he is not a typical Tory rebel and has not been vocal in criticising government previously.
As the chancellor announces new measures to ease the cost of living crisis, has the Conservative party managed to turn the page on Partygate?
Mr Simmonds, a new MP who has represented Ruislip, Northwood and Pinner since 2019, said he'd been waiting to see the Sue Gray report before deciding whether he'd back the PM to continue leading the Tory party.
"Having listened to what he said [at PMQs and to backbench MPs] and the views of constituents and my Conservative association, it is clear that while the government and our policies enjoy the confidence of the public, the prime minister does not.
"Accordingly, it is time for him to step down so that new leadership can take forward the important work of the government in ensuring that our people and country prosper."
Another Tory MP, Mr Sturdy, called on the MP to quit after on Wednesday after snap poll from YouGov suggested three in five Britons want Mr Johnson to quit.
Boris Johnson has now been accused of watering down the rules for ministers following ‘partygate’, as Shehab Khan reports
Mr Sturdy, the York Outer MP, said: “It is in the public interest for him to resign”.
Former ministerial aide Angela Richardson said the scandal has eroded public trust in politicians and “reflects badly on us all”.
“I am clear that had this been a report about my leadership, I would resign,” the Tory MP wrote online.
Labour leader Sir Keir Starmer told the Commons the report “laid bare the rot” in No 10 and called on Tory MPs to tell Mr Johnson “the game is up” and that it is “time to pack his bags”.
Which Tories are publicly calling on Boris Johnson to quit?
Sir Bob Neil
Anne Marie Morris (who, after having the Tory whip restored, has resubmitted her letter)
What is the process for backbench Tories to remove their leader?
Tory MPs are able to force a vote of no confidence in their leader if they won't resign.
To do so requires 15% of the parliamentary party to submit letters of no confidence to the 1922 Committee, which is effectively a HR department for backbenchers.
It would take 54 letters of no confidence to trigger a secret ballot, with a simple majority required for either side to win.
If more than 50% of Tory MPs vote to remove him, he will lose his role of party leader and be barred from competing in the forthcoming leadership election.
If the leader wins over half the votes, then they remain party leader and are given a year's immunity from any further confidence votes.
If a party leader loses a confidence vote then MPs from across the party can be nominated as potential replacements.
The 1922 Committee will determine how many nominations an MP will require to appear on the ballot.
If more than two qualify then MPs will vote on their preference, leaving two final candidates who must then appeal to party members for votes before being elected leader.