How YouGov became the UK’s best but most controversial pollster

Last night at 10pm, the UK waited with bated breath the release of YouGov’s MRP election poll, which eventually forecast that the Conservatives would win 359 seats, a comfortable 68-seat majority. That trepidation was testimony of how much stock many Britons – and a big chunk of the chattering classes – put in the pollster’s predictions. But just two years ago, things looked very different.

On the evening of June 8, 2017 Stephan Shakespeare, the 62-year-old co-founder and CEO of British polling company YouGov, was nursing a lager in The Hairy Canary, a Brussels pub not far from the European Commission’s HQ. It was a tense moment for Shakespeare: the results of the 2017 UK general election were about to be declared, and YouGov had predicted that prime minister Theresa May would lose her majority and end up with a hung parliament. It was the only pollster with such gloomy an outlook for the Conservatives, at a time when the general consensus was that May would romp home to victory. Shakespeare normally drinks wine, but picked a long drink because he was expecting a long night.

“It was very hard to be out there with a number that was so different from everybody else’s,” Shakespeare said back in 2017. “We were very exposed: we were calling every single seat using a new methodology.”

That new methodology was called multilevel regression and poststratification, or MRP. This system works by training a model on a small set of data about people’s voting preferences and their demographics. Those bits of information – people’s interests, age, and voting history – are then used to try and predict how people may vote, on the grounds of shared features with other members of the electorate. For instance: if you have a Times-reading Labour-voter living in Barnsley who prefers brown sauce to ketchup, you can assume that Times-reading voters who live in Barnsley and prefer brown sauce may be more likely to vote Labour than any other party.

MRP had rarely been used for political polls in the UK, until YouGov had adopted it – with mixed-to-good results – to forecast the Brexit referendum in 2016. But now the technique had returned a result so out-of-whack from other polling firms’ forecast that Shakespeare felt that his fate, and that of the company he had built, hung by a thread. The political establishment was circumspect: when reporting on the YouGov’s findings, The Times newspaper had called its forecast a “shock poll”, perhaps in an attempt to distance itself from the prediction. At the time, Shakespeare was out on a limb.

But MRP turned out to be right. It gradually became clear that Theresa May’s gambit had failed, Labour had surged, and that parliament was stuck with no unequivocal majority. “Shock poll” was right – for everyone.

YouGov’s political forecasting prowess had nothing to do with politics at all. The company, which was founded by Shakespeare and Conservative MP Nadim Zahawi in 2000, is first and foremost in the business of commercial market research for private clients, quizzing the public about what washing powder they buy and which biscuits they prefer. The company regularly polls people about miscellanea like their fears of a zombie apocalypse, spiritual enlightenment, favourite pets and whether they’d like all-day breakfasts at McDonalds.

In order to get those responses, YouGov turns to online panels comprising a million people in the UK – and millions more internationally – whose members are constantly consulted for their opinions on everything from the news of the day to which brand of bread they buy. (When taking a poll on 26 November, I was asked everything from my opinion on the Chief Rabbi’s fulmination against Labour to what feelings I had about Chelsea FC, and whether I eat to live or live to eat.)

People who participate in YouGov panels are “paid” in points, usually receiving around 50 points for a daily poll. Once they reach 5,000 points, they can cash it out for £50.

Up until YouGov arrived on the scene, market researchers were dour-looking men and women, clipboard in hand, who would pelt busy shoppers with questions. The participants would be chosen at random, and would be complete strangers, proffering their opinions to the survey organisers for the first and possibly last time. Sometimes, as a sop to technology, researchers would call up people in their homes at random and ask them the same questions.

YouGov proposed an alternative method: remaining in contact with survey-takers through the internet, constantly questioning them about their preferences, in order to better track the changing tastes of consumers over time. This panel-based method was its way to gauge the mood of the nation. But in order to drum up business, YouGov needed a way to make its name.

It was with that goal in mind that, ahead of the 2010 general election, YouGov entered an exclusive contract to provide political polls to The Times, according to Laurence Janta-Lipinski, a freelance pollster who worked for YouGov at the time. Its entry into politics was a win-win for the company. “We were aiming at the corporate market research market, but the best way to make an impression is to use the medium we understood – politics – and to get [our] polls out there,” says Shakespeare. Since then, the company has been producing industrialised polls popping up almost weekly in the media – in The Times, Sky, CBS, The Economist, and The Guardian (via a partnership between YouGov and the University of Cambridge.)

“The way we do it is very structured,” says Shakespeare. “We send out surveys all the time,” whose results are fed into an infrastructure called The Cube. “We’re asking a whole load of attitudinal and political and social questions every day, and a lot of that is automated, creating a huge set of background data.” Computers do the heavy lifting, with YouGov’s entire political polling team consisting of just 15 people across 38 countries.

“If we’re talking about national polling, you could say that YouGov does have some advantages over other pollsters: mainly they have this big panel of people and they know a lot about their respondents,” says Pat Sturgis, professor of quantitative social science at the London School of Economics.

That trove of personal information – and the ability to link someone’s tastes in household items to their likelihood to vote a certain way – is meant to sort out the one big problem pollsters have: sampling.

Representative samples are difficult to achieve, particularly across all 650 constituencies in the UK. “The thing people get most het up about is sampling,” explains Eric Harrison, a senior research fellow at City University London’s sociology department. “Getting a representative sample has been increasingly difficult due to people’s reluctance to answer questions on telephone polls, and one can’t rely on voluntary panels to be representative,” says David Spiegelhalter, a statistician at the University of Cambridge.

Using MRP accepts that challenge, and deploys some statistical fixes to process the data after collection. The “M” and “R” section of MRP looks at more than simply voting intention to discern what people want; the “P” then is the statistical methods, looking at how many people who adhere to those characteristics live in an area and therefore, how many people are likely to vote for a political party in any given constituency.

Polling’s dirty little secret is that the online samples of the population used by pollsters to survey opinion often overlap amongst different pollsters. “Most of the pollsters, as I understand it, buy a sample [a list of self-selecting people who are willing to answer questions] from online providers, and where that sample has come from is really quite unclear,” says Sturgis. YouGov, by contrast, builds its own panels.

YouGov’s MRP is meant to counteract the fact that the people being polled are self-selected, by removing the likelihood that respondents are more politically-literate or skewed to one side of the political spectrum . “We have our MRP running now, and it keeps getting adjusted,” says Shakespeare. One judgement update they’re having to make this election is around the likelihood of tactical voting. “We collect something like 10,000 interviews every single day just for this.”

Some question whether daily polling is sustainable in terms of quality. The perception of polling has always been that you can do it cheaply, you can do it quickly, or you can do it well – and you can only do two of those. But YouGov, reckon those in the industry, ticks all three boxes. “If you can’t do beautiful, pure, random probability sampling, which if you had an unlimited budget you could do, YouGov is the best,” says Janta-Lipinski.

And yet people still snipe. Much of the criticism of polling – and of YouGov in particular – stems from one reason, says Harrison. “[Most people] are not statistically strong, even among social science graduates. And the problem is that a little knowledge is a dangerous thing,” he says. People assume that they can unweight polls in order to change the outcome of the survey without realising that there’s a reason the results are altered after collection for a reason. They bend reality to their beliefs.

As for Shakespeare, he expects to be abroad on the evening of December 12. “I’ll probably be in Brussels again, as it happens,” he says. But he’ll try not to spend the night staring at the TV screen.

Updated November 28, 2019 09:34BST: YouGov’s poll figures from November 27 have been updated

More great stories from WIRED

🐘 The illegal trade of Siberian mammoth tusks revealed

🙈 I ditched Google for DuckDuckGo. Here’s why you should too

📧 How to use psychology to get people to answer your emails

💸 The WIRED Recommends guide to the best Black Friday deals

Like this article?

Share on facebook
Share on Facebook
Share on twitter
Share on Twitter
Share on linkedin
Share on Linkdin
Share on pinterest
Share on Pinterest

Leave a comment