Within the online panel and sampling industry, there has been a lot of debate and discussion on the topic of routers. But the fact is, the vast majority of market research buyers don’t know much, if anything, about routers, and don’t fully understand the impact different routing processes can have on their data. Unfortunately, too often they mistake filled quotas as representative sample, which is clearly folly given the results of the ARF Foundations of Quality research in 2009 and many other research on research initiatives examining respondent sources and routing.
Clearly, buyers expect online panel providers to deliver high quality data to support sound business decisions. However, they often don’t concern themselves with how that data is gathered and assume that that’s our job.
I’m not suggesting that MR buyers need to be fully versed on the myriad technical details and algorithms of routing processes. There are certain factors that directly impact the outcomes of their research projects, however, and buyers should equip themselves with enough knowledge to compare apples to apples when it comes to sample.
There are a couple of reasons this has grown in importance in recent years. First, the novelty of online surveys has worn off for consumers. There is more competition for their online time and the result is that it is harder than ever to recruit and retain panels. Second, and equally important, is the lack of two elements in our industry today: transparency and clear industry standards or guidelines. This has created a kind of “wild-west” scenario in which some vendors do whatever it takes to deliver the required completed interviews and quotas at the lowest price.
At GMI, our position is that routers, when used properly, are tremendously beneficial in reducing panelist screen-out and the disenchantment that can result. Of course, the key word there is “properly.” The flip side is that if used irresponsibly, this technology can introduce bias that substantively impacts data quality, and can abuse already overworked survey respondents.
So when evaluating the offers of online panel providers, market research buyers should know some basic facts about how respondents are being directed to their surveys. Among the questions they should ask are:
- What are the rules for matching respondents to surveys?
- How many surveys are put into the router?
- How many surveys will a respondent see in one sitting?
- What are the other surveys in the router?
- Which respondents are being routed?
- Is live web traffic being mixed into the sample?
- How is the router implemented?
The answers should persuade the buyer that the provider’s routing processes are well designed, used consistently and administered with expertise.
- See more at: http://www.ls-gmi.com/data-quality/dont-know-much-about-routers/#sthash.9pMM8JJ5.dpuf