The 2,013 babies born in the country yesterday will get a silver penny from the Royal Mint as a souvenir.
The occasion's also been marked with bell ringing at York Minster, and raised glasses at the only hotel in the country named after the future Queen Catherine.
Lindsay Eastwood reports.
The Duke of Cambridge took questions as he stood outside the hospital holding his new son.
"He's got a big pair of lungs on him, that's for sure," said Prince William.
"We're still working on name."
The Duke and Duchess of Cambridge have appeared outside the Lindo Wing at St Mary's Hospital with their new son.
Kate gave birth to the boy at 4.24pm yesterday.
The couple and the child will now travel home to Kensington Palace after Prince William makes some brief remarks to the gathered media.
The Duke and Duchess of Cambridge and their baby are expected to leave hospital later this evening, Kensington Palace officials said.
Prince Charles said his new grandson is "marvellous."
"You'll see him in a minute," he told reporters.
The bulging bank of photographers outside the Lindo Wing at St Mary's Hospital burst into action as Prince Charles stepped out of his car.
Hospital staff gathered behind barriers by the wing's doors also stretched to take pictures on their mobile phones.
Prince Charles has entered the Lindo Wing of St Mary's Hospital where his first grandchild was born yesterday.
His wife, Camilla, accompanied him.
"Have you had a long wait?" Charles asked reporters.
Princess Anne today said the arrival of a new Prince of Cambridge was "very good news."
"It's nothing to do with me," she told reporters during a visit to the National Mining Museum in Wakefield.
Deputy Prime Minister Nick Clegg has offered his congratulations to the Duke and Duchess of Cambridge after the birth of their son yesterday.
He said: "This is wonderful news. Miriam and I were absolutely delighted when we heard news and we want to send our heart-felt congratulations".
"It's news that will make the country smile", he added.