Facebook Twitter YouTube Instagram LinkedIn Wikidata
Category:

Don't Be Fooled by Bad Data

We constantly rely on data to point our clients in the right direction when shaping their brands.

The quality of the solutions we deliver can only be as good as the data our research provides. Recently, we learned an important lesson in how sample size can affect both research results and the conclusions we draw from it.

While engaging a new client, we conducted a very fast moving survey to examine their brand concept, appeal, opportunity, etc. Since we were under a tight schedule, we reported some preliminary results while waiting to obtain the full sample. We explained this limitation to the client, ensuring they understood our “quick hitter” initial objective. We weren’t reporting on all the measures, just taking a quick first glance based on a relative few.

The preliminary sample consisted of 120 respondents with a margin of error of +/-9 points. The final sample was 453 respondents with a margin of error of +/-4.6 points. Clearly, the final sample had a much smaller margin of error (almost half) and thus was much more likely to give trustworthy results.

Looking at more thorough measures than in the initial report, we asked our respondents to tell us how loyal they were to their current brand. What we found in our first, smaller sample was that 49% of respondents were advocates for their current brand. That decreased to 35% with the full sample, a difference of -14%. This was a crucial market share opportunity we would’ve overlooked based on the initial read from a small sample size. Consumers weren’t quite as loyal to their brand as we might’ve thought.

When we took a closer look at concept appeal we found another distinct difference. The smaller group of respondents showed that 29% would be interested in the concept. But with the full data of a larger sample size, this interest increased to 43% of all respondents, up 14%. Without taking the time to survey a larger group, we would have substantially underestimated our client’s opportunity in the marketplace. These percentages translated to over 6 MILLION additional consumer households.

What should we take away from this? Not all data reflects customer opinion accurately. We must be careful not to draw large conclusions from small samples. An adequate sample size is essential to conducting accurate and usable research. It ensures a trustworthy representation of the population under consideration and provides a confident direction for a brand. Sample size impacts the quality of the solutions we deliver to all of our clients, from insights and strategic recommendations to creative and interactive deliverables. If you have questions about consumer-based research alternatives, contact Trone Brand Energy for more information on finding and utilizing the best types of data to suit your needs.  

Related Articles

AAHA
AAHA | Insights | Pet | Pet care | pet health | veterinarian

AAHA: The League of Champions

Seven leaders of the veterinary profession came together to form one of the country's greatest organizations: the American Animal Hospital Association (AAHA). It was and remains the only institution that accredits companion veterinary hospitals in the U.S. and Canada.

We wanted to better understand the difference between how the subscribers of the biggest internet providers think compared to subscribers of the rest, including those who subscribe to smaller, regional internet providers.

A user’s experience with a website or app is made up of a variety of elements and moments in time. Every detail--from load time to aesthetics--has an impact. Luckily, Semantic Studios has created a big picture model of the components that influence user experience, also known as the UX Honeycomb.

Account planners at all levels should be familiar with the notion that brand planning is often an unexpected journey. For starters, you work within the scoped marketing budget to extract the best possible set of insights from primary research and existing data to get to a strategically-relevant brand story.

Looking for a little firefly magic for your brand?

High Point

1823 Eastchester Drive
High Point, NC 27265

336.886.1622
Durham

201 West Main Street
Durham, NC 27701

919.307.5270