Forecasting the 2017 UK general election

Demonstrating a data-driven model to forecast election behaviour, reallocating votes between parties per constituency seat

---

So the UK has a snap general election coming up in 8 weeks. It’s another opportunity for polling to get it all wrong 🙂 I’ve created a data-driven model to explore what might happen.

TL:DR*; If 5% of UKIP votes switch to Conservative, Conservative wins outright with an increased majority. If the UKIP votes switch to Labour, there is no outright winner but Conservatives still win the majority of seats. Labour needs Conservatives and the SNP to lose a lot of votes to win the election outright. Liberal Democrats could prevent Conservatives from winning outright if they recover a chunk of the votes they lost in 2015. In short, most scenarios point to Conservatives still winning the majority of seats, but not necessarily enough to win outright.

Update, 26 April 2017: Wales scenario added in comments section

Update post election: The result was as expected from the model forecast. Conservatives still won the majority of seats, but lost their outright majority. The surprises from the election were a) much higher turnout than expected, and b) Liberal Democrats failed to make much of a recovery (not that surprising, given the poor performance by the party leader throughout the campaign). The Labour party was the beneficiary of the higher turnout but, as expected, it wasn’t enough for them to win a majority this time.

— original post —

What makes this election particularly interesting is that it comes in the midst of quite a bit of turmoil, and there is a large chunk of votes that could be up for grabs. To demonstrate, here are the approximate voting numbers for the parties of interest over the past three elections and the % change from the previous election in brackets, along with a spatial plot of the seats:

WNP = Plaid Cymru. SNP = Scottish National Party

Conservative and Labour typically secure two-thirds of the total vote and over 80 percent of available seats. In 2005 and 2010, the Liberal Democrats received over 20% of the votes. In 2015, their share collapsed to just 8%. UKIP had huge gains in 2015 but only converted that gain into one parliamentary seat. The SNP had less than half the votes of UKIP but won 56 seats compared to the 6 they won at the previous 2 elections. The image below shows how winning votes doesn’t necessarily win you seats…

There are three questions that will determine the 2017 election:

• Will Brexit affect Conservative voting patterns?
• Will Jeremy Corbyn’s leadership affect Labour voting patterns?
• Were the dramatic shifts in voting behaviour for UKIP, Lib Dems, Green and SNP a one-off trend in 2015?

For UKIP, many believe it is likely to have been a one-off. UKIP’s primary purpose was to push for leaving the EU. With that now happening, it is not unreasonable to assume many of those votes will go somewhere else.

For the Liberal Democrats, the question is whether or not votes will return. With Labour shifting away from the centre, there is a gap to be filled by a centrist party but their current leader isn’t the most inspirational.

For the SNP, it depends on how people feel about the prospect of a second Scottish referendum. This could be a big deal for Labour because of the 50 additional seats won by the SNP in 2015, most were at their expense. For the SNP, the only outcome is hold or lose. They hold all but 3 of the contestable seats.

For the Conservatives, the aim is to win by a bigger margin, and the polls suggest this will happen in the absence of a strong opposition. The other possibility is that people decide to vote anything-but-Conservative, particularly those who voted Remain in the EU Referendum and/or are concerned about a political shift to the right.

Labour is a bit of an unknown quantity. The current leader Jeremy Corbyn has taken the party further from Centre-Left to the Left, a move started by the previous leader after the 2010 election and one that is causing polarised opinions. Anti-Corbyn folk think the election will be a disaster for Labour. Pro-Corbyn fans think the election will be a massive success.

Forecasting based on data

Rather than leave the guess-work to opinion polls, let’s experiment with the data. I downloaded the election results from 2015 and created a forecasting model in Excel.

I created a simple taxonomy to classify seats, based on the 2015 results:

Status	Criteria	Number (C – L – O)
Safe	Would still win, even if 2nd and 3rd place combined	463  (247 – 158 – 58 )
Target	Won by more than 2%, but lose if 2nd and 3rd combined	150  (65 – 60 – 25)
Marginal	Won by less than 2%, and ditto	37  (18 – 16 – 3)

Number of seats won by Conservative (C), Labour (L), everyone else (O). Note: this table includes equivalent parties in Northern Ireland, with SDLP added to Labour, Democratic Unionist Party added to Liberal Democrats.

Yes, out of a total 650 constituency seats, 71% are classified as safe. Even if you combined all the votes from 2nd and 3rd place, the winner would still have won the seat. It will take a big push to displace the incumbent party.

Marginal movements

37 seats are marginal. Depending on the size of the constituency, this means as little as 27 votes separated first and second place. These seats will be influential for the main two parties. Not least because all but 3 of those seats are evenly divided between Conservative and Labour, with one winning and the other coming 2nd, and third place dominated by UKIP.  The image below shows a slice of the model, and how the outcome would change if 5% of votes in each constituency shifted from UKIP to another party. The first chart shows the results from 2015, the 2nd chart shows what would happen if 5% of UKIP votes when to Conservative, the 3rd chart shows what would happen if 5% of UKIP votes went to Labour. Just a 5% move from UKIP would see most of the seats switch to the party securing those votes.

If the Conservatives secure the marginal seats by taking 5% of UKIP’s votes, their outright majority in parliament increases from 5 to 20. If the Labour party secures the marginal seats, Conservatives lose their outright majority but are still the dominant party. Labour would have to form a coalition with the SNP to form a government.

Modelling voter preference

To explore all the seats, the full model includes a formula to adjust the number of votes for each of the top 3 parties in each constituency seat based on a change in voter preference from 2015 data. The slideshow embedded below walks through the results of various scenarios to demonstrate:

To summarise:

• If Conservatives gain 500,000 votes from UKIP in marginal seats, they win outright with an increased majority. If the votes go to Labour instead, nobody wins outright but Conservatives still win the majority of seats.
• If Conservatives lose 10% of their votes (approx. 1 million), and Liberal Democrats or Labour make equivalent gains, there will be no outright winner. Conservatives still win the majority of seats.
• Liberal Democrats could win back one third of the seats they lost in 2015 if they recovered just 600,000 of the 4 million voters they lost in 2015. No outright winner, Conservative majority likely.
• Labour needs to gain a lot of voters and needs the SNP and Conservatives to lose a lot of voters to gain a majority, let alone to win outright.

If you are interested in the model, please get in touch!

If you have a scenario you’d like to see modelled, leave a comment or send an email with your hypothesis.

Data source: Electoral commission

Featured image: Contact Juggling! kindly shared under Creative Commons on Flickr by Alex Abian

* Too long, didn’t read