Estimated reading time: 5 mins
Market Research data is complicated; a single question can have multiple answers which means the data cannot easily sit in a relational database. This complexity has made it difficult to move data between platforms and applications.
Market Research projects are complicated; to a large extent the client gets to define the input (a questionnaire) and the output (a report or a dashboard). Every project is different, even a tracking study changes every wave and this has made it extremely difficult to automate.
And of course, particularly as research increasingly moved online, clients’ expectations were that their insights would get delivered faster and cheaper each year. Smart agencies responded to these challenges through a series of connected strategies which are still in place today.
Consolidating on a single platform
Selecting a single platform for Market Research that could sample, collect, weight and process data and produce client-ready deliverables. This has multiple benefits:
However, these platforms are a ‘jack of all trades and a master of none’ there is no single platform that is best-in class at every stage of every project. For example, quality of output may be sacrificed because the platform has to be able to handle complex quotas or quick-turn ad-hoc Market Research projects may become slower and more expensive because the platform has to be able to handle tracking studies.
Standardising the projects
By reducing the amount of variation the client is able to introduce, agencies are able to automate more of the process thereby reducing costs and saving time. This standardisation also means that results can be compared across all studies of that type allowing the creation of Market Research benchmark database.
However, this standardisation means that clients are not able to ask the questions which are important to them. As soon as clients need a more bespoke service, the cost escalates rapidly as the ‘machine’ is not built to accommodate this type of service.
Offshoring and outsourcing
Increased standardisation and consolidating on a single platform makes offshoring easier to implement and this significantly reduced costs for Market Research brands and agencies, when they first moved work from the high-cost markets to be done in lower cost territories. Effectively, agencies replaced expensive humans with less expensive humans doing the same work just in a different country.
However, offshoring increases the complexity for project managers and researchers, it is harder to brief someone in another time zone and resolving queries is often a laborious process involving long email strings with armies of people copied. In many agencies, an unseen overhead of multiple layers exist for checking as the agency staff do not understand what the outsourcer is doing and how they are assuring quality. Finally, the initial cost advantage realised by using less expensive humans is now priced into clients’ expectations so is now just the new normal.
With the advent of artificial intelligence (AI) / machine learning (ML) and robotic process automation (RPA) some of these challenges are being addressed in a new way.
Deploying best in class platforms
The challenge of extracting data from one application, transforming it in some way, loading it into a different application and checking it can now be done using RPA rather than people. This is not just faster and cheaper but it is also better because RPA bots don’t make mistakes through boredom.
This also means that agencies can now move away from the single, generalist platform strategy to start using platforms and applications which are best-in-class for the specific job they have been designed to do. No more need to sacrifice functionality in any stage of the project lifecycle, just allow your RPA bots to handle the movement of data between your applications.
Delivering a bespoke solution
Advances in AI/ML are allowing automation with customisation as never before, allowing the client to receive a bespoke solution without incurring a rapid escalation of costs. It is already possible, given a sufficient set of training data, to translate a questionnaire in word into an online survey with minimal human involvement. Many stages of the project lifecycle can be automated or augmented in the same way, from questionnaire design to chart production. In the future, AI will be able to act as a virtual assistant to researchers, serving up the tools and data they need based on their upcoming meetings and tasks.
Onshoring and outsourcing
What does this mean for the offshored captive centre or outsourced provider? In the case of those providers who offer the same service as onshore just with less expensive humans, I’m afraid this is mostly bad news. RPA and ML will quickly replace people doing lower value, transactional tasks and the model of selling cheap labour will start to look like Henry Ford’s faster horse.
The outsource providers that will win in the new paradigm must have: