Recently, I referenced  something quite obscure – the “sheep-goat effect”. The reason I say  “quite obscure”  is that no one had any idea what it was. And I spoke to literally scores of people. The “sheep-goat effect” was coined in ESP experiments whereby people who believed in ESP did significantly better than those who did not believe in ESP on ESP-type tests. The believers were called “sheep” and the non-believers, “goats”. The “sheep-goat effect” is therefore used (apparently by only a very small handful of people aside from me, if at all), to illustrate that the belief in something can have a causal effect on that event happening.

So why was I making this reference in the first place? Because the sheep-goat effect goes beyond ESP tests–it can be a very powerful concept in marketing.  This is especially true if your product or service is not widely accepted, is novel or is subject to public skepticism (like ESP).

Marketers love to gather feedback – by way of surveys.  Just Google how to create a good marketing survey and you’ll run across suggestions regarding target groups and sampling. Here are a few you’ll find regarding target group:

-Identify who should fill out your survey. Figure out the demographic you want to target. You can buy mailing lists from market-research companies.

-Decide whether your survey should target a specific group of people ora cross section. If you can’t easily control the respondents consider including questions/answers that will allow you to filter out respondents who don’t fit your target profile.

Sampling concepts include simple random surveys, systematic selection procedure sample, stratified sample, multistage sample, and cluster sample.  Cluster sampling “is considered a more practical approach to surveys because it samples by groups or clusters of elements rather than by individual elements” (12, Lee, Forthofer, and Lorimer, 1989). It also reduces interview costs.  But there is scant guidance on what types of people to include in this target group.

You really should take the concepts of target group and sampling one step further. Determine whether your target group or the sample you’ve chosen will include both sheep and goats. If it will, then either target or sample the sheep and goats separately. If you can’t do that, at least include some questions in the survey to allow you to determine whether the respondent is a sheep or a goat. For instance, if you are marketing an herbal weight loss supplement, consider including questions like:

– Do you think that current weight loss products address the real needs of dieters?

-Has the media done an adequate job of exposing questionable weight-loss programs?

Granted, bias in surveys has been discussed and is often considered. Some of these biases touch upon the sheep-goat effect but are somewhat off-target. You should keep the following biases in mind but also introduce the sheep-goat effect bias.

Hostility Bias

Some respondents may dislike the survey sponsor and provide all negative responses.

Sponsor Acceptance Bias

Some respondents provide answers to please the survey sponsor. Respondents interpret what they believe the sponsor wants to hear and their answers may be false.

Reference Bias (order bias)

Respondents develop a frame of reference from a previous question, discussion, activity, or thought.

They carry the reference to the next question, which biases answers. The sequence of topics, questions, and activities produce reference bias.

You can reduce reference bias by logically ordering questions, topics, and activities in qualitative research.

Sensitivity Bias

Questions may raise sensitive subjects, about which respondents would rather not talk. Respondents may give false answers to hide secrets.

You need to build trust here. People will talk to others they like and trust. Use projective techniques and indirect questions in qualitative marketing research.

Social Acceptance Bias

Respondents provide socially acceptable answers that may be false.

People say what is socially acceptable, even though they may feel or think something else. They may twist the truth, or offer half-truths.

For example, not many respondents tell you directly that they seek power, social status, or are envious because of their insecurities.

Most people want to conform to their group.

Challenge answers tactfully. Use projective techniques or indirect questions that deal with socially sensitive subjects.

Sponsor Bias

When respondents know who is sponsoring the research, their feelings and opinions about the sponsor may bias answers.

Purchase managers shift into negotiating mode when they know the sponsor.

Don’t reveal the name of the sponsor. Keep your studies blind as long as you can in qualitative research.

All of these biases, however, focus upon the respondent’s responses and do not adequately address the group to which the respondent may belong (i.e. are they a sheep or a goat)?

So, at your next meeting with the marketing folk at your company, feel free to say “Are you sure that is not the result of the sheep-goat effect?”  If they look at you blankly, take comfort that you are talking to a goat.

Further Reading:

• The Sheep-Goat Effect, per The Skeptic’s Dictionary
• Understanding Survey Bias
• Mineful’s post on Preventing Survey Bias

Follow

Related posts: