Senior Liberal Democrats have not entered the row which has seen David Cameron back Chancellor George Osborne's comments that the case prompted "wider questions about our welfare system."
Mr Alexander said: "The Philpott case is an individual tragedy. Children have died in that case.
"I think that is where we should let that case lie. I would not want to connect that to the much wider need to reform our welfare system."
The Prime Minister agreed with the Chancellor that the Philpott case prompted "wider questions about our welfare system".
The Prime Minister said the Chancellor's remarks about the Philpott case "were absolutely right".
He said: "I think what George Osborne said was absolutely right.
"He said that Mr Philpott was the one to blame for his crimes and he should be held responsible but what the Chancellor went on to say is we should ask some wider questions about our welfare system - how much it costs and the signals that it sends.
"We do want to make clear that welfare is there to help people who work hard, it shouldn't be there as a lifestyle choice and I think that's entirely legitimate."
Shadow Chancellor Ed Balls launched a scathing attack tonight on what he called the "cynical, nasty and divisive" way George Osborne linked the Philpott case with the broader issue of state benefits.
Mr Balls said the "desperate" Chancellor had offended millions of hard-working people and was playing politics with a tragic case for his own political gain.
The shadow chancellor Ed Balls has condemned the "nasty and divisive" comments by George Osborne in which he questions why taxpayers' money was being used to "subsidise lifestyles" like Mick Philpott:
I believe George Osborne's calculated decision to use the shocking and vile crimes of Mick Philpott to advance a political argument is the cynical act of a desperate Chancellor.
Our main thought at this time should be about the six children who tragically lost their lives, and the others in the family who have been left to mourn their loss.
We should have a proper debate about welfare reform.
And we should discuss what action needs to be taken to tackle the scourge of long-term unemployment including the need for a compulsory jobs guarantee so that people cannot languish on the dole for years and years on end.
But for the Chancellor to link this wider debate to this shocking crime is nasty and divisive and demeans his office.
It is wrong to link those acts with the debate about welfare and George Osborne should not be doing so, even implicitly.
Millions of people including pensioners and the disabled, people in work and out of work, receive benefits and tax credits. The Government needs to recognise that they are as shocked and disgusted by the callous killing of these children as anyone else in Britain.
Where the ultimate blame should lay for the deaths of six of Mick Philpott's children is at the centre of fierce debate.
Editor and founder of the Guido Fawkes blog Paul Staines and Independent columnist Owen Jones amplified that debate on ITV's This Morning.
Paul Staines levelled the blame at the welfare state while Owen Jones said that was like blaming Harold Shipman on the NHS.
Labour MP for Middlesbrough Andy McDonald has accused George Osborne on Twitter of "trying to make capital" out of the Philpott case:
Total disgrace that Osborne tries to make capital out of the appalling Philpott case. Typical Tory demonization of anyone on benefits.
Chancellor George Osborne has said that the case of the Derby house fire raises questions about whether the Government and taxpayers should be "subsidising lifestyles like that".
Chancellor George Osborne has questioned why the taxpayer should pay for benefit "lifestyles" such as those of child killer Mick Philpott.
When asked on a visit to Derby if the Philpotts were a product of Britain's benefit system, Mr Osborne said: "It's right we ask questions as a Government, a society and as taxpayers, why we are subsidising lifestyles like these. It does need to be handled."
He said Philpott "was responsible for horrendous crimes, crimes which have shocked the nation".