Mr Javid, who tested positive for the coronavirus last weekend, said on Saturday he had made a “full recovery” and that his “symptoms were very mild, thanks to amazing vaccines”.
He continued to urge people to get their jabs, and tweeted: “Please, if you haven’t yet, get your jab, as we learn to live with, rather than cower from, this virus.”
The cabinet minister deleted the tweet on Sunday and tweeted: “I was expressing gratitude that the vaccines help us fight back as a society, but it was a poor choice of word and I sincerely apologise.
“Like many, I have lost loved ones to this awful virus and would never minimise its impact.”
Co-founder of Covid-19 Bereaved Families for Justice Jo Goodman said Mr Javid’s comments “are deeply insensitive on a number of levels”.
She said: “Not only are they hurtful to bereaved families, implying our loved ones were too cowardly to fight the virus, but they insult all those still doing their best to protect others from the devastation this horrific virus can bring.
“Words matter and the flippancy and carelessness of this comment has caused deep hurt and further muddied the waters of the Government’s dangerously mixed messaging.”
And, Safia Ngah, a spokesperson for the families group, told ITV News that the government and Mr Javid needed to do more to convince those who had lost loved ones they were doing more to arrest the recent surge in cases.
She also said his words had hurt a great many people - particularly those in the health service.
"Hundreds of health workers have died from this virus - they are clearly not cowering from it," she said.
Opposition MPs have also criticised Mr Javid's choice of words.
Shadow justice secretary David Lammy, whose comments were echoed by Labour deputy leader Angela Rayner, said: “129,000 Brits have died from Covid under your Government’s watch.
“Don’t denigrate people for trying to keep themselves and their families safe.”
Lib Dem health spokeswoman Munira Wilson said Mr Javid’s tweet was “outrageous” while thousands remain in hospital with Covid-19.
She said in a statement: “His careless words have insulted every man, woman and child who has followed the rules and stayed at home to protect others.
“He owes them all, especially the millions who are shielding, an apology.”
Public health expert Devi Sridhar said Mr Javid's words would be “painful to read for those who were severely ill” and those who lost loved ones to coronavirus.
The professor at the University of Edinburgh wrote: “It wasn’t because they were weak, just unnecessarily exposed to a virus.
“And wanting to avoid getting Covid isn’t ‘cowering’ – it’s being sensible & looking out for others.”
Mr Javid tested positive for coronavirus on July 17. Prime Minister Boris Johnson and Chancellor Rishi Sunak, who were close contacts had to go into isolation. They are due to finish their isolation on Monday.
The pair initially tried to avoid isolation by saying they were taking part in a testing pilot, but backed down after widespread public criticism.
Coronavirus: What you need to know - How can clinically vulnerable people be protected after Freedom Day?