Deputy Leader of the Labour Party Harriet Harman said: "Jeremy Hunt should not be in his job. He took on responsibility for an impartial, quasi-judicial decision when he was not acting fairly, like a judge. He had already made up his mind."
Mr Stephens said he believed Mr Smith had been drawn into a "web of manipulation" in his dealings with News Corp.
Lord Justice Leveson said the episode was a calamity for the department and the special adviser.
You have an able, highly regarded young man who isn't in any sense mischievous, who is very keen to do the right thing but has got into a degree of contact, in the context of a small office. How has this happened?
Mr Stephens said:
How it happened I don't know, but the judgment I have made is that, sadly, Mr Smith personally, I believe against his will and intentions, was drawn into almost what seems to me to be a sort of web of manipulation and exaggeration, and was drawn inadvertently beyond what he intended to do or what he wanted to do. But, unfortunately, he was beyond it.
Jonathan Stephens, permanent secretary at the DCMS, said Mr Smith was considered before the emails emerged to be someone who "understood and would abide by his proper role".
He told the inquiry the better the special adviser, the more reliable a guide they were to a Secretary of State's view.
"I think he was well tuned in to the Secretary of State's thinking," he said.
He went on:
The first suggestion that contacts went beyond what was proper was April 24 with the release of emails from Fred Michel and this was the first occasion I recall mention of Michel by name.
The following morning I told the Secretary of State I thought the number, extent, depth and tone of contacts supported by those emails went beyond what was acceptable.
Adam Smith told Jeremy Hunt he would resign if the pressure became too great. But Hunt said "it won't come to that," says Smith.
Smith describes a "very pressured mood" in Jeremy Hunt's office after James Murdoch gave his evidence. But he says it was agreed I was just doing my job.
The mood then changed the next day and he was told "everyone here thinks you need to go".
Adam Smith says the tone and language used in e-mails with Fred Michel might have been too flippant and loose but he doesn't think the substance was inappropriate.
Lord Justice Leveson and Robert Jay QC are really struggling to get Adam Smith to say what his personal view was on the BskyB bid. Repeats he "was not that fussed"
The Prime Minister has said he does not regret giving the Culture Secretary Jeremy Hunt responsibility for the BSkyB takeover bid.
In an interview on ITV1's This Morning he said he was confident Hunt would "do the job" professionally and independently.