Like many industries, insights and market research has undergone significant change in the last thirty years. As new technologies have come online, we’ve been able to fine tune processes, speed up data gathering and analysis, and ultimately, probe deeper to find more meaningful insights to empower organisations.
But with AI, automation and synthetic data entering the field in recent years, the pace of change has jumped to unprecedented levels. So, as the dust settles and ways of working get rewritten, it’s no surprise that many clients and industry professionals alike are wondering what’s changed in the insights field—and, of course, what’s next around the corner.
There’s probably no better way to see how far we’ve come than taking a trip down memory lane, which is, perhaps alarmingly, not so long ago! While we’ve largely moved on from paper and pencil and door-to-door surveys of the 90s, fieldwork has continued to evolve after its shift online.
Today, we’re seeing the emergence of synthetic data, which is allowing for fast scenario and hypothesis testing, almost as a pre-research stage to establish whether an idea has merit worth exploring in depth with proper research and experts. It's even helping researchers top up sample sizes, particularly for difficult to reach audiences, such as business owners and decision-makers.
“Synthetic data is transforming the speed at which we can gather insights,” says Rachael Weaver, Operations Director at Perceptive. “What once took days or even weeks of fieldwork can now be done in hours.”
Similarly, before AI and automation entered the scene, manual processes—such as data cleaning—often took days of work. Now, advanced tools are helping insights professionals detect survey fraud in the moment and screening the participant out before their data is collected.
“It’s the same story with dashboarding,” says Rachael. “Manually updating a dashboard used to take days but technology has helped us reduce this to a few hours.”
Saved from these repetitive, once-manual tasks now means researchers have more time to interrogate the data, engage with it far more critically, and spend more time helping clients make the most of the insights uncovered.
“It’s expertise and tech coming together to deliver insights with deeper relevance faster than we’ve ever been able to do,” says Rachael. “For us, it’s not about finding ways to remove humans from the process but rather find better tools and evolving our ways of working to speed that process up.”
Where some aspects of the insights industry are seeing major shifts in how the work is done, other aspects remain largely unchanged. This is particularly true for qualitative focus groups and interviews, which rely heavily on person-to-person interaction and observation.
While new technology, such as video conferencing, have given researchers new platforms and flexibility in how they run focus groups and interviews, the approach to the research itself has remained the same.
However, where we see change in the qualitative space is when technology adds a new layer of insight on top of traditional qualitative methods. For example, eye tracking and facial recognition that tests viewers’ emotional responses to ad creative are two technologies helping researchers to make more in-depth observations and gain clearer understanding about buyer behaviour and decision-making.
Read more: A case study on using eye tracking and heat mapping to optimise in-store point of sale
With all the advances in technology, AI and automation, it’s becoming clearer just how critical the role of human expertise is, particularly when it comes to detecting nuance and building strategy from insights.
AI and automation may be doing more of the heavy lifting in terms of automating and speeding up manual, time-consuming tasks, but it can’t prioritise or delve deeper into a particularly juicy insight without a human consultant.
This is where we see the industry heading: a marrying of AI speed with a human consultant’s deep thinking. This combination of human plus tools will allow insight experts to go further, faster, and provide even more tailored insights and advice to businesses.
“I see insights moving away from what has happened and into what’s going to happen,” says Rachael. “Rather than reactive results based on what happened months ago, insights will become continuous and actionable. We’ll be predicting instead of reacting.”
Want to unearth new insights to enhance your business? Book in a free consult with our insights experts.