The London Mayor told BBC Breakfast on Thursday that wearing non-medical facial masks, such as a bandana, scarf or reusable mask, would add “another layer of protection” to the public.
Current UK guidance has emphasised the importance of masks for doctors and nurses but does not suggest widespread usage.
Transport Secretary Grant Shapps said it is “not the right moment” to encourage people to wear masks – adding that the Government needs to look at all of the evidence.
Mr Khan, who wrote to Mr Shapps about the issue, said he is lobbying for masks to be worn in circumstances where people cannot keep two metres apart, such as on public transport or while shopping.
He added: “I want a consistent approach across the country, we don’t want mixed messaging.
“So I’m lobbying the Government’s experts and the Government, (and) want them to change their advice and change their guidance so we can have this additional layer of protection.”
He said it is important that the UK is “no longer an outlier” as he referred to the Centre for Disease Control (CDC) in the US, which now recommends people wear cloth face masks when outside their homes.
Mr Khan added that everyone may need to wear a facial covering once the current lockdown measures begin to ease.
“What I’m lobbying for is, at the moment, when you can’t keep your distance, wear a non-medical facial covering,” he told the programme.
“But when it comes to exiting lockdown, we may need to have all of us wearing it as well.”
Dr Paul Cosford, Medical Director of Public Health England explains their stance on face masks
Dr Paul Cosford, Medical Director of Public Health England told ITV News: "Our guidance is clear that the most important time to use face masks is in the clinical care setting, that's for people who are looking after patients either with Covid-19 or with other diseases or of course for people who have Covid-19 themself and to help stop transmitting to other people."
"The most thing for us to remember about reducing the spread of infection is the social distancing measures that we have got."
Dr Cosford also urged everyone to follow the guidelines to stay at home and if you do go out for essential reasons then to stay two metres apart and adhere to social distancing.
He adds: "We do the hand-washing, we do the coughing or sneezing into a tissue and throwing that out and we absolutely do not go out when we are unwell."
He continued: "The added benefit to face masks, which is really quite marginal and is not something we recommend, it is important if you have got disease to stop spreading disease, but it is not our current recommendation."
But Mr Shapps said the Government needs to consider all of the evidence on wearing face masks before issuing any new guidance.
Speaking to LBC on Friday, he said: “So it is not the right moment to instruct people, as I saw the London Mayor do this morning, to wear them if we are not certain yet that they are going to be advantageous.
“In fact, he wrote to me about this and said in his letter he recognises that it could be counter-productive, so I don’t think we should be in that space right at this moment.”
At Thursday’s press briefing, England’s chief medical officer Professor Chris Whitty said the Government is continuing to keep advice on wearing masks under review.
He said it would be a “very bad thing” if demand from the public for masks led to shortages for healthcare staff.
He told the briefing: “The evidence is weak, but the evidence of a small effect is there under certain circumstances.
“What we are really trying to do is to work out under what circumstances, if any, should we extend the advice and under what circumstances should we not change that advice.”
Coronavirus: Everything you need to know: