Elections provide the hardest of tests for political polling – we have a clear, unambiguous result against which to compare our pre-election polling.
2019 was the third general election since the partnership of ITV Wales, YouGov and Cardiff University combined to start conducting the Welsh Political Barometer polls in late 2013.
So how did the polls perform in 2019?
Plaid Cymru 10%
Brexit Party 5%
So how close were we to the final outcome? Now all the Welsh constituencies have declared, we know the final vote share outcomes, and this is what they were:
Plaid Cymru 9.9%
Brexit Party 5.4%
I think even the fiercest critics of polling would have to say that the match between our final Barometer poll and the actual election outcome in Wales is astonishingly close.
We should bear in mind that fieldwork for the final Barometer poll was completed three days before the election; we had no chance to do anything about any late swings that could have made our poll less accurate.
We should also recall that surveys can be expected to have some sampling error: the percentage figures we generate for party support (and on all our other questions) are estimates that we very much hope are close to the true figure, but cannot realistically be expected to be exactly correct.
In that context, to get every one of the six largest parties within less than a percentage point of their actual vote share is truly astonishing.
How does this performance compare with the past?
The simplest summary measure of the overall performance of a voting intention poll is the Mean Average Error, which pretty much does what it says on the tin: it provides a single measure of how far from the correct outcome the poll was, on average, across all the parties.
Here are the mean average errors for our final pre-election poll this year, as well as those for the final pre-election Barometer polls in 2017 and 2015, and the last pre-election poll YouGov did in Wales in 2010:
I would be happy with any of those performances, but that for 2019 is truly exceptional.
To average within less than half a single percentage point of the correct figure for all the parties is incredibly close.