How We Used Social Analytics to Help Predict the 2016 Elections

electionanalytics

Yesterday at about 4 p.m., Networked Insights’ Head of Analytics, Jaime Brugueras ran his final data model before the Presidential Election. We’ve been following the race for months, using our analytics platform Kairos and a proprietary model that accounts for emotional expressions in social data as a signal of intent to vote.

Our ultimate map, which was shared throughout the day by our election coverage partners at Wired Magazine, predicted a potential electoral tie of 269 to 269, which was a very wide departure from the trouncing by Clinton that other pollsters predicted. Huffington Post, for example, gave her a whopping 98% chance of winning.

As is abundantly clear now, our prediction was far more accurate than the vast majority of major polling sources.

Here it is:

screen-shot-2016-11-09-at-11-09-15-am

So what did we do differently?

While we did take into account survey data from Real Clear Politics to help properly weight the overabundance of social conversations around the election, our model relied primarily on the emotions people express voluntarily on social media as a powerful predictor of intent to vote.

Our analytics engine Kairos processed unstructured data from millions of sites, blogs, and social platforms like Twitter and Tumblr, analyzing and classifying billions of true consumer posts across 25,000 topics, emotions, and demographics. Using Kairos, we then built 4 metrics: Awareness, Positivity, Negativity and Intent, of which Negativity and Intent proved to be very valuable in predicting elections. Negativity and Intent are Natural Language Processing classifiers which take advantage of sentence structure as well as keyword matching.

Then we modeled the data against survey polls, primary results, and survey pools to obtain train the model and obtain the weights of influence for each of the social indices. Throughout the campaign we tracked the state level predictions and saw immediate swings after big events (e.g. debates, FBI announcements).

While we were able to identify these trends quickly using our analytics platform Kairos, they did not show up immediately in other national polls. Ultimately, the final last swing we detected 5 days before the elections after the Comey announcement may have proven to provide the momentum Trump needed to have such a surprising Election night.

So what are the takeaways?

We think these results validate some of what we’ve been saying for a while, and here’s why:

– We fundamentally believe the unstructured social data, when synthesized and classified appropriately using the right technology, can provide a deep and necessary layer of context and accuracy that simple polling data can no longer provide.

– We think that one of the many reasons why major pollsters were so far off on their models of this election is because they flat out ignored this data.

– We think this approach to gathering data is better, because it allows us to access much larger sample sizes than traditional polling, collecting unsolicited feedback which (if adjusted for bias appropriately by a data scientist) can open a deep new well of additional information.

– We’ve proven that analyzing social data can be incredibly useful for brands who want to make smarter media buys, predictively analyze campaign performance, or fuel their content strategies. We’ve proven that it can be useful for correctly guessing World Series Champions (Go Cubs!). We’ve proven it is useful in predicting the success of box office releases. And now we’ve shown how it can be useful in understanding elections.

– As we reported in our Wired coverage, the Top 3 issues for the general electorate in this election according to our data were jobs, national security and immigration. It is clear that Donald Trump focused his message squarely on those topics, which ultimately moved the needle in the election, while Clinton put split her focus on secondary or tertiary issues like climate change and gun control, which proved less important to voters.

For media inquiries, or to learn more about how Networked Insights’ analytics platform Kairos can help your organization make smarter business decisions, please contact richard.sharp@networkedinsights.com.