Research mistakes are preventable

Tom Dougherty, CEO – Stealing Share

25 April 2017

Research mistakes. No excuses.

Making research mistakes is easier than ever. And these blunders can mangle and mislead brands. Especially brands that are looking for opportunities to rebrand. It’s best not to guess. It’s always better to know.

Research mistakesTimes have changed conventional wisdom

Thinking you know what you know is a bad place to begin. Research mistakes are plentiful when you don’t challenge your own thinking.

Here is an example of what I mean. For years, CATI studies (Computer Assisted Telephone Surveys) were the gold standard in market research. Nothing substituted for this methodology. But CATI research is expensive.

Not everything is what it seems

Today, CATI studies are harder to field. And not just because they are expensive. How many of you have a land-line telephone? That group of dinosaurs shrinks every day. We don’t need them any longer.

Many researchers tell you they compensate for that shrinking population by including cell phone numbers. A good idea in theory.

But, tell me, how often do you answer a call on your cell phone from an unknown caller? Worse still, an unknown caller from out of the area?

Research mistakes happen when we make assumptions

Methodology matters when it comes to projectable research. Using flawed methodology clouds results. Your research study can’t be self-selecting. The goal is to eliminate bias and get a fair representation of the target.

So, randomization equals great research. And being double-blinded. But it also relies on what researchers call incidence. Incidence is the percentage of randomly selected subjects that complete the study. The lower the incidence, the higher the margin of error.

“Research methodology is changing. It is keeping up with changes in demographics and privacy. What worked three years ago is outdated today.”

research mistakes and market share

This make sense. Common sense.

Think about this for a moment. If over half of the possible target audience refuses to complete the study, there may be a bias for those that do. Once a large percentage of respondents refuse to take the study your randomized sampling is anything but random.

To compensate for this, we pay many respondents an incentive to complete the study. This used to work well. The researcher tells you upfront that you are paid for your thoughts.  

It fails if you can’t get through

But you’re sunk if your target audience does not use a land-line or won’t answer a caller they don’t recognize. No amount of incentive can compensate for an inability to break through the barriers of availability.

On a recent CATI study, 75% of population were over the age of 65. Tell you anything?

Panel studies are now the GOLD Standard

You can predict the success and value in panel studies by the confidence you have in the panel itself. We have found them more accurate than CATI studies in many categories. And we are quickly finding them preferable in general population studies too.

It’s a new world in research. Research mistakes occur when we don’t challenge our own assumptions.

Read about market research here.

See more posts in the following related categories: CATI studies market research Panel Studies Quantitative research Research mistakes


Submit a Comment

Your email address will not be published. Required fields are marked *

Arya Stark had every right to get naked

  Arya Stark   Tom Dougherty, CEO - Stealing Share 24 April 2019 Arya Stark had every right to get naked (Game of Thrones spoiler alert! Especially when it comes to the beloved and now all grownup Arya Stark.)I was as shocked as everyone else this past...

5,994 and counting, retail closings at record numbers

  Retail closings   Tom Dougherty, CEO - Stealing Share 17 April 2019 5,994 and counting, retail closings at record numbers In 2018 alone, 5,864 retail locations closed. So far in 2019, retail closings stand at 5,994 and it is only April. If the pace stays...

Share This