Around 1,000 EDL protesters then made their way along Whitehall before taking a position on the pavement across the road from Downing Street.
A smaller group of anti-fascist demonstrators gathered opposite them holding an "EDL Racists" banner.
Both sides traded insults but were kept apart by officers and barriers.
The EDL supporters chanted: "We hate Cameron."
The anti-fascist supporters chanted: "Racist scum off our streets."
More top news
The singer had been tweeting in support of migrants when she was bombarded with abusive messages.
Ministers are reportedly considering plans in a bid to lower inward migration.
In a tweet on Saturday, the US president didn't give a reason for his absence but said he wished"everyone well and have a great evening".