Video report by ITV News correspondent Sejal Karia
The Government is urging GPs to roll out a seven-day service to help relieve pressure on crisis-hit A&E departments.
Ministers say patients that cannot get appointments outside of working hours are left with little option but to go to A&E, putting pressure on the NHS.
Almost half of GPs (46%) are closed during core hours and 18% are closed at or before 3pm on at least one weekday, figures from the National Audit Office reveal.
Prime Minister Theresa May has signalled her support for reforms and threatened to withhold the extra funding if GPs do not comply.
A Downing Street source said: "Most GPs do a fantastic job, and have their patients’ interests firmly at heart.
"However, it is increasingly clear that a large number of surgeries are not providing access that patients need – and that patients are suffering as a result because they are then forced to go to A&E to seek care.
"It’s also bad for hospitals, who then face additional pressure on their services."
The government said the NHS is already committed to bringing in 8am to 8pm services in general practice seven days a week, backed by an extra £528 million per year in funding by 2020/21.
But GPs have called it "unrealistic" and warned doctors are already "spread too thin" and quality of care will suffer.
They added waiting times for appointments would increase.
Labour leader Jeremy Corbyn accused Theresa May of being in denial about the crisis in the NHS.
"This is another example of a Prime Minister in denial. A Prime Minister who would much rather listen to spin doctors than real doctors," Mr Corbyn said.
"I suggest she goes to visit a surgery and asks doctors what it's like," he added.
The proposed reforms include:
GPs to meet the extended hours commitment of 8am to 8pm seven-days a week unless they can prove there isn’t the demand
Extended hours should be offered when patients want them including evening and weekend appointments
GPs accessing extended hours funding should expand their online services to free up consultation and treatment time in surgeries
Extra funding will be contingent on demonstrating that they offer and advertising flexible appointments
Practices could be asked to use a new appointments tool to submit data on the number of appointments offered to understand demand
The Government says that so far 17 million patients have benefited from extended access to appointments and that it is committed to increasing the numbers of doctors in general practice by around 5,000 by 2020.
The British Medical Association has angrily responded, saying ministers are trying to "scapegoat" doctors and not address the NHS funding crisis.
Dr Chaand Nagpaul, BMA chairman, said GPs already provide care 24 hours a day, seven days a week.
"Much of the pressure on A&E has nothing to do with general practice: it has to do with seriously ill patients for whom seeing a GP would not prevent a hospital admission," he said.
"This is not the time to deflect blame or scapegoat overstretched GP services, when the fundamental cause of this crisis is that funding is not keeping up with demand.
"The Government should take responsibility for a crisis of its own making and outline an emergency plan to get to grips with the underlying cause, which is the chronic under-resourcing of the NHS and social care."