Video report by ITV News Social Affairs Editor Penny Marshall
Dozens of people have come forward to make claims of abuse against charity workers since a scandal involving Oxfam staff in emergency zones first broke three weeks ago.
Regulator the Charity Commission says it is examining 80 safeguarding claims relating to 26 charities and aid groups received since February 12.
ITV News understands the allegations include claims of sexual abuse of children.
Penny Mordaunt, the International Development Secretary, warned there predators must be brought to justice.
"Let us ensure that there is no hiding place for those who wish to exploit the vulnerable in our sector," she said.
Let us ensure that the British public can take pride in everything that is done in their name."
Charity bosses at the summit discussed new measures to ensure the vulnerable under their care are protected.
They are discussing changes including better vetting of charity workers, greater information sharing about suspects within the sector and more involvement of the police where wrongdoing is suspected.
"All of us can do better, and we have to come together collectively," said Kevin Watkins, the CEO of Save the Children.
But Alexia Pepper de Caires, a whistleblower and former Save the Children employee, said she did not trust any gathering that did not include those who had reported concerns.
"The idea of a safeguarding summit sounded very serious," she said.
"However, my understanding of how we resolve matters is to bring in those who really know issues from the inside, and not just hear from the people at the top who oversaw those scenarios happening in the first place."