Former Paedophile Information Exchange (PIE) chairman Tom O'Carroll told ITV News he is "confident" Harriet Harman was aware the group was represented on a gay rights sub-committee in the late 1970s.
O'Carroll told UK Editor Lucy Manning he did not think Harman had any involvement with PIE "directly or indirectly", adding, "except through her knowledge of the fact that the organisation had a presence within the NCCL" sub-committee.
He said: "She presumably knew that we were affiliated with the NCCL and must have known, I think, that a representative of PIE - for a long time that was myself while she was there - was on the gay rights sub-committee.
"She must of known that, I am confident, because she had regular discussions with her staff, including the person who presided over the gay rights sub-committee, and that person would have talked to her about the proceedings of the committee from time-to-time I'm sure".
Harriet Harman's husband and Birmingham MP Jack Dromey said in a statement:
It is no surprise that a convicted paedophile, the like of whom I took action against during my time in the NCCL, should choose to smear me.
The record is clear. I took on PIE and, when I was elected chairman, defeated by a massive majority at the 1976 NCCL annual conference a loathsome motion calling on the NCCL to support the so-called rights of paedophiles.
Indeed my stand was denounced in a leaflet distributed by PIE to the delegates to the conference.
Quite simply, no one will believe a man like Tom O'Carroll.
Harriet Harman and Patricia Hewitt failed to take action to oust the Paedophile Information Exchange (PIE) because they "didn't want to rock the boat", its former chairman has claimed.
Tom O'Carroll said Labour's deputy leader and the former health secretary did not support PIE but "didn't even try" to cut its affiliation to the civil liberties organisation they worked at for the sake of their careers.
O'Carroll sat on an NCCL gay rights sub-committee in the late 1970s and said there had been no active attempt to remove him.
He told BBC Radio 4's Today programme: "Really they didn't do much to oppose PIE's presence in my view because there were these other liberal forces, or radical forces, within NCCL.
"The support didn't come from Harman and Co but it was there. The Gay Liberation Front was very radical and at that time Harman and Patricia Hewitt couldn't just kick out PIE.
"Well, they could both try but they didn't even try and the reason they didn't try is they didn't want to rock the boat because their careers within NCCL depended on them not rocking the boat too much."
Ms Harman has said she has nothing to apologise for over the involvement of the National Council for Civil Liberties with the paedophile rights campaign but expressed "regret" that there had ever been a link between the two.
The National Council for Civil Liberties was a very mainstream organisation at that time, the pre-cursor to Liberty, if you like.
Most people here actually think the idea that this horrible paedophile organisation was allowed to affiliate and that the NCCL had totally open affiliate policy is bonkers.
Quite lot of people in the Labour Party think it might have been better for Harriet Harman to just say "yes, that was crazy," to accept the deal and apologise and move on.
But as you see in my interview, she is clearly very upset about it.
She is upset because her point is: "I've spent my whole life campaigning for children, for women, for the dispossessed and here I am being accused of helping the very people I was fighting against all the time."
When it was question of Ed Miliband against The Daily Mail with the attack on Ed Miliband's father, there was emotion too, but there was an element of political calculation.
Ed Miliband wanted to be seen taking on the big newspaper organisations.
With this, I've got to say, it's pure emotion.
Political correspondent Carl Dinnen said Harriet Harman is not helped by the fact that Liberty, the organisation that formed from the NCCL, described links to the paedophile group as "a continuing source of horror and disgust."
Harriet Harman has repeatedly insisted that she had nothing to do with the NCCL's affiliation with PIE and that her work was never influenced by their campaigns.
"I have been in public life for 30 years and all of those years have been about protecting the vulnerable, protecting women and children," Harriet Harman insisted today.
"That's why I find it so unfair and offensive that the Daily Mail should put smear and innuendo on me as if somehow I supported those people that all my public life I've been fighting against.
"I think they are wrong to be doing that and that's why I'm speaking out."
"It's actually not me that should be apologising for something that I haven't done - it's the Mail that should be apologising for their smear and innuendo," Harriet Harman said today.
"I'm not going to apologise because I've got nothing to apologise for," Harriet Harman said today.
"I very much regret that this vile organisation, PIE, ever existed, and that is ever had anything to do with NCCL but it did not affect my work at NCCL - they had been pushed to the margins before I actually went to NCCL."
She went on: "To allege that I was involved in collusion with paedophilia or aplogising for paedophilia is quite wrong and is a smear."
Labour deputy leader Harriet Harman hit out at the Daily Mail newspaper today, after they attempted to "smear" her "by way of guilt of association" over the affiliation between a group she worked with in the 1970s and their links to a paedophile rights group. Writing on Twitter, she said:
When it comes to decency and sexualisation of children, would you take lessons from the Daily Mail?
The Camberwell and Peckham MP posted a picture of a Daily Mail article based around pictures of teen television personality Courtney Stodden photographed in her bikini as a 12-year-old child.
Harriet Harman has expressed "regret" over the links between a 1970s paedophile rights campaign and the civil rights organisation for which she once worked.
In a fresh statement this morning, a spokeswoman for Ms Harman said:
She regrets the existence of PIE and she regrets their involvement with NCCL before she joined, although it was immaterial to her work.
She stands by the statement from yesterday and she is certainly not going to apologise to the Daily Mail.