This blog was originally published by Tim Wragg, CEO of Human8, as LinkedIn article.

After spending an estimated half a billion dollars surveying voters across the United States, with cutting-edge technologies and methodologies, the truth is staring us in the face yet again. We got it wrong. Pollsters, market researchers, forecasters and insights gurus all predicted things that never happened and were unaware of things that did. The disconnect between research and reality – something I hear from many business leaders – is as stark as ever.

Well, up to a point.

Yes, industry experts underestimated Donald Trump’s popularity across the board, especially with women voters, young males and Latinos. Online panels and phone surveys missed huge swings towards Trump throughout the country, particularly in once-staunch Democrat districts. And how can it be that pollsters confidently predicted Kamala Harris had a four-point lead in Iowa, only for Trump to win the state by a whopping 13 points with almost 56 per cent of the vote?

After misjudging the strength of Trump’s support in both the 2016 and 2020 elections, pollsters recalibrated their quantitative data models to, in theory, more realistically reflect his standing. Clearly, they didn’t recalibrate enough. No one spotted what became a Trump landslide. Was it wishful thinking? Poll convergence? Or did we just miss the signals because we rushed to judgement (feeding the media’s insatiable appetite for new polls with hastily-conducted surveys), instead of taking the time to understand people, talk to them, listen to them and observe their environments? Otherwise known as the qualitative side.

It’s why we at Human8 believe there needs to be a fundamental shift in how the entire industry operates. Technical adjustments only go so far. For too long, we have relied upon single-mode polls, usually online or on the phone that allow for inaccuracies and biases to play havoc with results. We have too much faith in quantitative data whilst quietly admitting that social norms affect the accuracy.

For instance, if you’re working double-shifts to make ends meet (a likely Trump voter), you’re not going to have as much time to fill in a survey at leisure, as younger people working from home might (typically, Harris voters). And considering the combustible nature of the election, it’s likely many people simply lied or stayed quiet about favouring Trump.

We have allowed our focus on quantitative measurement to take precedence over the qualitative. By that I mean the context in which these behaviours and beliefs exist. People’s everyday lives, their conversations with friends, their wavering emotions, the strength of their feelings and how they are affected by daily responsibilities, the issues that matter to them, what they love and loathe in their lives and to what extent, the context of people’s points of view. Their individual cultures. Unless we reach further inside these cultures, we’re never going to understand people as well as we think we can, no matter how sophisticated the technological bells and whistles at our disposal. Qualitative work, at scale, is key and it’s why we’re proud at Human8 to be recognised as one of the world’s most innovative businesses doing it.

Otherwise, we end up imprisoned in our cosy bubbles. And, yes, I’m as much a victim as anyone. When I visited clients in the US this year, I arrived convinced by media reports that Trump was a spent force. And then I drove through the country where I saw red MAGA caps everywhere, his face beaming from every front lawn, the car radio filled with Republican voters lauding their hero. Just a few days ago, working in Detroit, I sat in a room filled with mostly women voters, Democrat warm – the core of Harris’s vote – and each one solemnly said: ‘He’s going to win.’

On the ground, there was a strong sense that Trump had it in the bag. On the TV news later on, I watched journalists interview political wonks, commentators, other journalists and former politicians for their insight into voting behaviours and, as they predicted a slim Harris victory, wondered: ‘When was the last time these people spent quality time with voters outside of their bubbles?’

On election morning, I read polling expert Nate Silver, founder of Five Thirty Eight, boast that he had run 80,000 models and that the election was ‘literally closer than a coin flip’ but that Harris would probably edge it.

It doesn’t matter how many models you run, you can’t know people by asking them to tick a box and relying on highly reductive quantitative tools. Instead, you can only really understand people – whether it’s about the brands they buy or the votes they cast – by getting as close as you can to them.