Managing Survey Response Quality With Category And Method Exclusions

By Author: John Fitzpatrick
Total Articles: 2
Comment this article

Managing Survey Response Quality with Category and Method Exclusions



The TrueSample team is continually exploring the nuances of data quality that impact the results of online market research.  Our goal is to continue to ask questions and explore the possibilities as we guide the industry in assuring the highest possible data quality in quantitative market research.



Today, we’ll look at category and method exclusions – these are often discussed as good avenues for managing survey response quality.  In the context of these discussions, a “category” refers to a type of product or service (for example, skin care products) and a “method” refers to a type of study being conducted (like a concept test).



Many in the Market Research industry are interested in category and method exclusions because there is a belief that, in some cases, presenting the same respondents with several similar surveys in a ...
... short period of time may cause them to respond differently than they otherwise would.  This could be due to boredom, an increased ability to game the survey, or other reasons. But the results are the same—an inability to make valid apples-to-apples comparisons across respondents, and a lack of trust in the resulting data.  As if that weren’t bad enough, sending respondents too many surveys that seem too much alike may eventually stop them from participating in market research altogether.



So how do we avoid such problems?  First, we need to track the appropriate categories and methods for each survey.  Next, we need to be able to tie together that survey’s respondents and the date they entered the survey with the categories and methods we’re tracking.  That’s all the information that we need to make the exclusions – we would be able to answer a request like, “Give me a sample of women in their 20s, but exclude anyone who took a survey about makeup in the last 30 days.”



Unfortunately, the solutions that exist today are often quite fragmented and time consuming, requiring researchers to use multiple tools and multiple steps to make category and method exclusions.  In some cases, researchers are able to make these exclusions only with previous respondents to their own surveys.   To overcome these hurdles, companies with a commitment to data quality (like TrueSample) are working on developing technology solutions that will make this effort fast, simple and with less fragmentation across the industry.