Facebook Twitter YouTube Instagram LinkedIn Wikidata

Don't Be Fooled by Bad Data

tbe-datablog_0.jpg

We constantly rely on data to point our clients in the right direction when shaping their brands. The quality of the solutions we deliver can only be as good as the data our research provides. Recently, we learned an important lesson in how sample size can affect both research results and the conclusions we draw from it.

While engaging a new client, we conducted a very fast moving survey to examine their brand concept, appeal, opportunity, etc. Since we were under a tight schedule, we reported some preliminary results while waiting to obtain the full sample. We explained this limitation to the client, ensuring they understood our “quick hitter” initial objective. We weren’t reporting on all the measures, just taking a quick first glance based on a relative few.

The preliminary sample consisted of 120 respondents with a margin of error of +/-9 points. The final sample was 453 respondents with a margin of error of +/-4.6 points. Clearly, the final sample had a much smaller margin of error (almost half) and thus was much more likely to give trustworthy results.

Looking at more thorough measures than in the initial report, we asked our respondents to tell us how loyal they were to their current brand. What we found in our first, smaller sample was that 49% of respondents were advocates for their current brand. That decreased to 35% with the full sample, a difference of -14%. This was a crucial market share opportunity we would’ve overlooked based on the initial read from a small sample size. Consumers weren’t quite as loyal to their brand as we might’ve thought.

When we took a closer look at concept appeal we found another distinct difference. The smaller group of respondents showed that 29% would be interested in the concept. But with the full data of a larger sample size, this interest increased to 43% of all respondents, up 14%. Without taking the time to survey a larger group, we would have substantially underestimated our client’s opportunity in the marketplace. These percentages translated to over 6 MILLION additional consumer households.

What should we take away from this? Not all data reflects customer opinion accurately. We must be careful not to draw large conclusions from small samples. An adequate sample size is essential to conducting accurate and usable research. It ensures a trustworthy representation of the population under consideration and provides a confident direction for a brand. Sample size impacts the quality of the solutions we deliver to all of our clients, from insights and strategic recommendations to creative and interactive deliverables. If you have questions about consumer-based research alternatives, contact Trone Brand Energy for more information on finding and utilizing the best types of data to suit your needs.  

Add new comment