Understanding the new NHS Friends and Family Test stats

Patients on wards or in A&E are asked whether they would recommend the service to friends and family. Photo: PA Wire

With a fanfare and a tweet from Number 10, the curtain rises on the NHS Friends and Family rating system.

Now we will all know which are the most recommended hospitals - won't we? Well, perhaps not.

The idea of F and F (let's call it that) is brilliantly simple.

You ask patients on wards or in A&E: "would you recommend the service you've just received to your friends and family?"

Brilliant because you will probably get an honest answer and a simple, easily understood rating figure.

But hang on. It's not that simple.

For a start the score a hospital gets on F and F is NOT the percentage of patients who recommend it - as it would be in an opinion poll.

Sorry about an equation but, F and F score = (number of yes answers) - (number of nos).

OK not too difficult.

But then let's look at the response rate - what proportion of people actually gave ANY answer. It's low - for inpatients 27 per cent, for A&E patients around 10 per cent.

We don't know what the other 73 per cent think. Might be yes, might be no. Low response rates means the figures are less reliable.

What's more, the response rate varied from hospital to hospital making it difficult to compare them.

Lastly, hospitals collect these stats in different ways.

Sometimes its a face-to-face interview with a clip board. Sometimes by a text message. The point is, people may give different answers depending on how they're asked the question. For example, you may be more likely to say "yes" if asked face to face. This is a potentially serious bias.

OK so what can we do with the stats? Well I don't think you can say Hospital A with a score of 70 is better than Hospital B with a score of 60. Hospital A might have had 90 yes but 20 no answers to reach the 70 score. Hospital B might have had 60 yes and 0 no.

Comparing hospitals on these stats alone is like buying a car by comparing ONLY the carbon emissions.

But for individual hospitals, they're very useful. If your score has been 70 (around the average for inpatients), but then suddenly drops to 55, you know something is going wrong. If you change your caterers and the score leaps up to 75, you've put it right.

Food, you might think, is pretty unimportant in rating hospitals. In fact food is one of the two commonest causes of complaints about hospitals. The other? Parking.

More on this story