by Andy Waller, on Jun 10, 2020 1:14:02 PM
Estimated reading time: 5 mins
Market Research data is complicated; a single question can have multiple answers which means the data cannot easily sit in a relational database. This complexity has made it difficult to move data between platforms and applications.
Market Research projects are complicated; to a large extent the client gets to define the input (a questionnaire) and the output (a report or a dashboard). Every project is different, even a tracking study changes every wave and this has made it extremely difficult to automate.
And of course, particularly as research increasingly moved online, clients’ expectations were that their insights would get delivered faster and cheaper each year. Smart agencies responded to these challenges through a series of connected strategies which are still in place today.
Market Research right strategy…THEN
Consolidating on a single platform
Selecting a single platform for Market Research that could sample, collect, weight and process data and produce client-ready deliverables. This has multiple benefits:
- Data doesn’t have to be moved between platforms for scripting, processing and reporting, saving time and reducing errors
- Data from different countries can be easily integrated
- Investment in time and money can be focused on one platform rather than many
However, these platforms are a ‘jack of all trades and a master of none’ there is no single platform that is best-in class at every stage of every project. For example, quality of output may be sacrificed because the platform has to be able to handle complex quotas or quick-turn ad-hoc Market Research projects may become slower and more expensive because the platform has to be able to handle tracking studies.
Standardising the projects
By reducing the amount of variation the client is able to introduce, agencies are able to automate more of the process thereby reducing costs and saving time. This standardisation also means that results can be compared across all studies of that type allowing the creation of Market Research benchmark database.
However, this standardisation means that clients are not able to ask the questions which are important to them. As soon as clients need a more bespoke service, the cost escalates rapidly as the ‘machine’ is not built to accommodate this type of service.
Offshoring and outsourcing
Increased standardisation and consolidating on a single platform makes offshoring easier to implement and this significantly reduced costs for Market Research brands and agencies, when they first moved work from the high-cost markets to be done in lower cost territories. Effectively, agencies replaced expensive humans with less expensive humans doing the same work just in a different country.
However, offshoring increases the complexity for project managers and researchers, it is harder to brief someone in another time zone and resolving queries is often a laborious process involving long email strings with armies of people copied. In many agencies, an unseen overhead of multiple layers exist for checking as the agency staff do not understand what the outsourcer is doing and how they are assuring quality. Finally, the initial cost advantage realised by using less expensive humans is now priced into clients’ expectations so is now just the new normal.
Market Research right strategy…NOW
With the advent of artificial intelligence (AI) / machine learning (ML) and robotic process automation (RPA) some of these challenges are being addressed in a new way.
Deploying best in class platforms
The challenge of extracting data from one application, transforming it in some way, loading it into a different application and checking it can now be done using RPA rather than people. This is not just faster and cheaper but it is also better because RPA bots don’t make mistakes through boredom.
This also means that agencies can now move away from the single, generalist platform strategy to start using platforms and applications which are best-in-class for the specific job they have been designed to do. No more need to sacrifice functionality in any stage of the project lifecycle, just allow your RPA bots to handle the movement of data between your applications.
Delivering a bespoke solution
Advances in AI/ML are allowing automation with customisation as never before, allowing the client to receive a bespoke solution without incurring a rapid escalation of costs. It is already possible, given a sufficient set of training data, to translate a questionnaire in word into an online survey with minimal human involvement. Many stages of the project lifecycle can be automated or augmented in the same way, from questionnaire design to chart production. In the future, AI will be able to act as a virtual assistant to researchers, serving up the tools and data they need based on their upcoming meetings and tasks.
Onshoring and outsourcing
What does this mean for the offshored captive centre or outsourced provider? In the case of those providers who offer the same service as onshore just with less expensive humans, I’m afraid this is mostly bad news. RPA and ML will quickly replace people doing lower value, transactional tasks and the model of selling cheap labour will start to look like Henry Ford’s faster horse.
The outsource providers that will win in the new paradigm must have:
- Strong technology R&D particularly in RPA and ML and the willingness to deploy this technology internally, to optimise their own workforce, as well as externally with their clients.
- Scale to invest and a broad industry exposure to allow learning and best practice sharing across a wide range of clients and use cases.
- A learning culture and centres of excellence in future skills, helping their clients to replace data processors with data scientists, focus group moderators with community managers and coders with social media analysts.
In conclusion, three things
- This is a great time to be an insight client. You have probably stopped spending on legacy, tracking studies because of COVID-19 and you can use this budget to invest in the bespoke solution you need. You can get this delivered in the format you want through the platform you want and it shouldn’t cost significantly more than the standardised offering.
- This is a great time to work in an agency. Finally, you will be able to spend your time helping your client address their business issues by doing what is right for them and right for your agency.
- This is a great time to work for the right KPO company. You will be able to move up the value chain and use the skills you have to deliver new insights, operational efficiencies and revenue growth to your clients. If, however you want to spend your working life checking data tables now is the time to consider a career change!
