The owner of the house and white van which Emily Thornbury took a photo of before posting it on Twitter has said the MP is a "snob".
Dan Ware,told The Sun: "I've not got a clue who she is, but she's a snob".
He said the flags had been left up since being raised when England played in the football World Cup in May.
"We will continue to fly it," he added.
Mr Ware, a 37-year-old father of four, told the newspaper he could not remember when he last voted.
Labour revealed that Emily Thornberry had spoken to leader Ed Miliband a second time following her initial apology.
A party source said: "Ed and Emily had a second conversation.
"She thought the right thing to do was to resign. Ed agreed."
The owner of the house pictured in a tweet posted by Emily Thornberry has branded the Labour MP "a snob".
"Any hint that you're patronising your voters is very, very dangerous indeed," ITV News Political Editor Tom Bradby said from outside the Rochester home.
Each wooden panel, created by London artist Bambi, measures 6ft by 4ft. They vanished overnight on Upper Street in Islington.Read the full story ›
You've been giving us your thoughts on the Mayor's approval of plans to redevelop the Royal Mail site at Mount Pleasant - where affordable rents could be set at up to £2,800 per month for a family home.
Is he off his head !!! affordable to whom.Get real Boris.!!!
Stupid prices , I don't even earn that amount in a month , all he is doing is pushing the normal working class people out of London !!!! Disgraceful
I was born and bred in Islington, moved out 16 years ago couldn't afford to live there then!!
In what world is £2800 a month affordable. If people could afford that, they could afford to buy somewhere!
I live in Islington, and THAT is not an 'affordable' rent! What part of 'affordable does that excuse for a mayor not understand?? To call that an affordable rent beggars belief!!
London is in desperate need of truly affordable decent housing and Boris failed in his duty as Mayor today by approving this planning application.
Boris says he is committed to delivering affordable housing for Londoners, but he has no idea about what that actually means for people working in the capital.
It's an absolute travesty that postmen and women working just next door will be priced out of this luxury development at Mount Pleasant.
Mayor Boris Johnson has approved a plan to build 700 homes on the site of the former Royal Mail sorting office at Mount Pleasant in Islington.
The plans have come under fire from councillors after a report was leaked showing that affordable rent levels could be set at as much as £2,800 per month.
A leaked report on the redevelopment of the Royal Mail site at Mount Pleasant in Clerkenwell reveals that the Mayor plans to charge up to £2,800 per month for 'affordable' flats.
Boris Johnson is expected to grant consent for the development today.
The new site would contain 98 'affordable' rent apartments - with the report suggesting tenants could be charged £1,170 per month for a one bedroom flat, up to £1,690 for a two bedroom and up to £2,800 per month for a four bedroom family home.
The Guardian estimates that a family living in one of the four bedroom flats would need an income of £100,000 to make it affordable.
It's making a complete mockery of the term 'affordable'.
Boris Johnson doesn't really understand social housing in London - or he doesn't care - either way, Londoners are missing out on a great opportunity here for more affordable housing in the middle of London.
Boris Johnson is today expected to approve controversial plans to redevelop land at Royal Mail's Mount Pleasant sorting office in North London.
The proposals include hundreds of flats but have been criticised for their low proportion of affordable homes.
The mayor chose to take the final decision away from Islington and Camden councils after a request by Royal Mail.
The Met Police has said the woman who was threatened with a knife by a man at a house in Islington was unharmed. They believe that the man and the woman knew each other.
Neighbours saw a woman being escorted from the scene accompanied by a female police officer. The Met said she was treated for shock.