Around 1,000 EDL protesters then made their way along Whitehall before taking a position on the pavement across the road from Downing Street.
A smaller group of anti-fascist demonstrators gathered opposite them holding an "EDL Racists" banner.
Both sides traded insults but were kept apart by officers and barriers.
The EDL supporters chanted: "We hate Cameron."
The anti-fascist supporters chanted: "Racist scum off our streets."
More top news
Andy Murray has beaten Belgian David Goffin to win the Davis Cup for Great Britain for the first time since 1936
Mercedes' Nico Rosberg has won the season-ending Abu Dhabi Grand Prix, with team-mate Lewis Hamilton coming second
Celebrities and campaigners have joined a huge rally urging world leaders to agree tough measures to curb global warming