8 June 2015
This is part two in our series on the trends determining the future of mobile market research in Africa over the next 10 years, adapted from our Market Research in the Mobile World presentation in May – you can read part one here.
Now for the show…
In Africa, we believe we’re going to see more researchers partnering with multiple sample providers – because current sampling methodology is flawed, and using multiple respondent bases is one of the most important things we can do to mitigate the effects of these flaws. (For an in-depth look at what’s wrong with digital sampling right now, see this fascinating piece by Pondering Panda CEO Diane Gantz).
The fragmentation of researchers and sample providers
In the last few years, mobile has burst onto the research scene in Africa in a big way. Digital research techniques are lowering the barrier to entry to this market, and the number of small companies now offering mobile research has proliferated.
For example, there are about 30 companies in South Africa alone offering digital and mobile research, many of them founded to take advantage of an existing respondent base.
This, however, has resulted in a fragmented industry, and that’s a problem, both for research providers and for clients.
In many cases, these companies only have access to one digital or mobile respondent base for their research samples – and even large, established research houses are right now only using one or two respondent bases for their mobile research component.
This is bad for clients, because samples from respondent bases mostly based around a certain device type and/or content or social offering, contain an inherent coverage bias.
The result of this is that certain groups are often either underrepresented or overrepresented by the base. And while it is usually possible to target specific demographic groups within the total sample, the ubiquitous problem of non-response means that any sample you get is likely to be biased.
Here’s an example from our own research into multiple bases:
This is a correspondence analysis map highlighting sample providers in red, and demographic markers in orange.
We tested seven different sample providers, and the circled points you can see here are the four primary race groups in South Africa.
Each one of the seven bases we tested is biased, sometimes radically so.
For example, Provider A skews heavily white, while Providers E and F skew significantly black.
Let’s take a look at another example:
Here is another correspondence analysis map highlighting sample providers in red, and this time, language markers in orange.
The four circles highlight four of the most widely spoken languages in South Africa – isiZulu, isiXhosa, English and Afrikaans. Again, there is a dramatic bias among sample providers.
And don’t forget, language is often associated with a respondent’s wider culture, and how they might react to a new product, for example.
So what this shows is that if you deliver a random sample via a single mobile base, because of coverage bias and non-response, its validity is immediately in question.
So how do we mitigate the effects of this bias?
Luckily, there is something we can do about this, and it forms a key part of our research philosophy.
Our approach – and the approach we believe more and more researchers in Africa will adopt – is to to mitigate the impact of coverage bias by conducting research studies across multiple bases. And the more, the better.
The growing awareness of this within digital research will encourage researchers to start partnering with multiple sample providers in order to deliver more valid and more representative samples, and this will result in growing collaboration within the industry.
Which is going to look something like this:
Researchers partnering with multiple sample providers to deliver device agnostic research to a high number of widely dispersed respondents.
This approach can go a long way to reducing the skews inherent in respondent bases.
Pondering Panda, for example, can now conduct research across as many as nine respondent bases so far in South Africa and Africa, and this has proven both eye-opening and productive.
Got any questions? Get in touch or post them in the comments below.