General

The Three Research Fundamentals 2016 Election Polling Missed

November 22, 2016

The results of the 2016 US Presidential election have been final for almost two weeks now, but Americans and the world at large remain stunned and highly skeptical of the age-old political mechanism used year after year to predict those results: the poll.

From the outset the poll results were accepted as collective fact. But in retrospect – regardless of individual stance on the election results – it’s clear the polls and the predictive models were, at their core, flawed. Not enough consideration and effort went towards disproving poll results with better, more accurate data, methodologies, and results.

At 360i, we are not political polling experts, but we do keep our ears to the ground and an eye towards the future when it comes to research. We study consumers: the way they think, how they behave, what they say, what they mean, and the environment around them. One of the fundamental factors contributing to this year’s polling inaccuracies is the evolving digital ecosystem, which has made traditional polls and predictive models outdated. We often talk about taking marketing fundamentals and adapting them for the digital age. The same is true for research fundamentals.

Understanding consumers today is complex, but the basic elements remain critical:

  1. Multiple data sources
  2. Data interpretation
  3. Sampling accuracy

——–

1. Multiple Data Sources

Because we live in a fragmented digital era, one data source will never show the full picture. In fact, you need multiple data sources from different disciplines of research. 360i has consistently advised a blend of traditional and emerging/digital research. To reveal the big picture, uncover legitimate insights, and yield more accurate results, we must access data from multiple sources. Blend what people claim they’re going to do (exit polls) with what people are thinking about (search data) and what people are talking about amongst friends (social data).

2. Data Interpretation

But data sources, no matter how varied, are merely an input. To add insight to the input requires Data interpretation. The leap from data to insights is regularly practiced at 360i and requires not just listening to what people are saying about a brand, person or topic, but also taking into account the general tenor of the overall conversation — are they optimistic, energized, pessimistic, sarcastic, hesitant? This is a manual process, but a necessary process as it helps add a layer of context to the equation, distinguishing trends from nuances in consumer perception.

In the case of the election results, the “data” (polls) were important but the lens of critical thinking and data interpretation (insights) was missing. Polls failed to understand and address the different variables at stake, such as any cultural shifts or a potential “embarrassment” factor – people feeling unease or hesitance to truthfully share which candidate deeply resonated with them. This type of insight only results from manual analysis. Exit polls are binary. Humans are not. And in 2016, the topic of politics and the environment in which it occurred were complex. Polls were not equipped to account for the variability.

3. Accurate Sampling

Despite the rigor put into analyzing data, it would be meaningless without accurate sampling. After the election, articles circulated on how sampling methods did not cover a true representation of the voting population or account for their behavioral changes in the digital ecosystem. 360i’s research and consumer insights discipline applies heightened scrutiny to coding the relevance of each person included in sample sets, and the sample set itself as it is uniquely defined by the objective of the research. Often, that does not always require the sample to mirror the overall population. If the galvanized voting population doesn’t mirror the overall demographic makeup of America, it is of little use to ensure your sample is representative of the total population.

 

At its core, research is meant to prove or disprove the norm. It’s deployed to shake things up or settle things down and in the case of this Presidential election, the research only misled. This election’s failed projections and predictions is cause for many to rethink how they approach research. When gathering consumer data, especially in a rapidly changing environment, research methods need to adapt. As they do, its important to remember these three fundamentals, and tailor the research accordingly.

 

Brian Vieira, Director of Insights & Planning at 360i, also contributed to this post.