He says he met her on the online dating site, Match.com, and had no idea of her occupation - till the People newspaper started investigating the relationship.
According to one of his colleagues, he ended the relationship as soon as he learned of her occupation.
The question raised by the disclosure for campaigners for tougher press regulation is whether Mr Whittingdale as culture secretary has been softer in his treatment of the press, knowing that four newspapers - the People, the Sun, the Mail on Sunday and the Independent - had all been aware of the relationship for some time.
He denies his judgment as press regulator was affected.
Each of the four newspapers had looked at the story at different times, but none published.
Arguably Mr Whittingdale should have told the prime minister and the Cabinet Secretary Sir Jeremy Heywood about the press's knowledge of the relationship when he was appointed culture secretary after the general election last year.
There could be a perception that Mr Whittingdale's impartiality as the ultimate regulator of press and TV could be undermined by his awareness that the press was in possession of a salacious unpublished story about him.
But Downing Street has told me that "our view is that he is a single man who is entitled to a private life. He didn't know she was a prostitute and ended the relationship when he found out".
I asked Downing Street whether Mr Whittingdale's job is in jeopardy.
An official said "no".
The campaigning group Hacked Off, which represents victims of hacking, has been arguing that it is outrageous that Mr Whittingdale has put on hold - seemingly indefinitely - a new regulation voted by MPs in the last parliament that could land huge additional legal bills on newspapers that don't join an officially authorised regulator.
The newspapers' new regulator, IPSO, does not meet the test for its members to avoid those costs.
So newspapers are delighted by Mr Whittingdale's apparent shelving of the new regulation.
There is no suggestion Mr Whittingdale has in any way been influenced by newspapers' unpublished knowledge of his affair.
I've asked journalists at the relevant newspaper why they never published the story, which was being offered to them by another former sex worker - who has attracted publicity in the past.
They say they had no evidence that Mr Whittingdale paid for sex with the sex worker, who is a dominatrix - and Mr Whittingdale's colleague told me that he categorically didn't ever pay for sex.
So they regarded the story as "not very interesting", according to one, because it was about an MP who is not married but who had made a misjudgement when using online dating.
The revelations of press knowledge of Mr Whittingdale's relationship were made earlier this month by byline.com, a news website.
It may seem incongruous to some that the Mail and Sun failed to publish the Whittingdale story but have been making loud public statements of their fury that a judge has prevented them publishing a sex story about a British celebrity.