Chuka Umunna is among the big name Labour MPs who have left Brighton before Jeremy Corbyn makes his first conference speech as party leader.
ITV News' Chris Ship said Tristram Hunt and Mary Creagh were also among the prominent backbenchers who departed early.
Most Labour MPs already know who they want as leader, according to Tristram Hunt, who has just announced he would not be standing.
In a speech setting out his vision for the party, he admitted he hadn't been able to drum up enough support for his own bid
"Like other potential candidates in recent days I have made a lot of calls to potential supporters among my parliamentary colleagues. I found that the bulk of MPs are already committed to just a couple of candidates," he said.
"It is surprising that the nomination process to select a leader for at least the next five years appears to have been largely decided within at most five days of a devastating general election defeat."
The MPs currently still in the race to become leader are Liz Kendall, Yvette Cooper, Andy Burnham, and Mary Creagh.
Tristram Hunt has announced that he will not stand in the forthcoming Labour leadership election.
In a speech, the MP for Stoke said he would be backing Liz Kendall for the leader's role instead.
He also strongly criticised the party's strategy at the general election, claiming Labour's welfare policies "offended the British people's sense of fairness".
Labour's Tristram Hunt will appear on the final episode of this series of The Agenda tonight.
Mr Hunt will speak to Tom Bradby about how Labour can recover following last week's general election defeat.
The shadow education secretary may also discuss his own leadership ambitions as speculation over Ed Miliband's potential successor continues.
Shadow education secretary Tristram Hunt has admitted that a Labour government would rely on charitable and private childcare providers to put "mothballed" Sure Start centres back into use.
Speaking on BBC1's Andrew Marr Show, Mr Hunt said councils would be obliged by law to invite such providers to make use of Sure Start centres:
The headmaster of the shadow education secretary's former school has said Labour's plans to strip private schools of tax breaks worth hundreds of millions of pounds unless they do more to help the state sector could be deemed "offensive bigotry".
In a speech yesterday, Tristram Hunt told private schools to start doing more to help state pupils or risk losing £700 million-worth of tax breaks.
But Mark Beard, head of the independent University College School (UCS) in Hampstead, north-west London, said the shadow cabinet minister should be considering "new, helpful initiatives" to raise standards in state schools..
Shadow education secretary Tristram Hunt has told ITV News more "structured" play and learning in nursery schools "doesn't seem like a bad idea".
"I think we have to address this issue of disadvantaged children who aren't school ready by the time they turn up in reception year," Mr Hunt said.
Schools standards will only improve once the quality of teaching in classrooms is raised, shadow education secretary Tristram Hunt has said.
Michael Gove earlier put forward proposals to shake-up the education system which he hopes will end the "Berlin Wall" divide between state and private school sectors.
"Improving school standards starts with a qualified teacher in every classroom. Until Michael Gove commits to this, he is ruling himself out of any serious debate about how we raise standards in our schools," Mr Hunt, said.
"Whether on discipline, delivering extra-curricular activities or on improving learning outcomes, it all hinges on the quality of the teacher in the classroom.
"Raising the quality of teaching - that is where the focus needs to be and that is what Labour is concerned with. The Tories have lost sight of this and are undermining school standards as a result."