I posted to LinkedIn’s Answers forum, under Market Research and Definition. The full post and answers can be found here.

Here’s one example: recently I received an e-mail from a leading regional lifestyle magazine, inviting me to participate in an online reader survey. The subject line of the invite reads as follows:

“[Name of Publication] Readers: What is your foundation?”

This magazine covers topics such as natural health, personal growth, spirituality, environment, etc. so “foundation” in this context probably means something like “core values,” right? The body of the invite reinforces this:

“Please take our Reader Survey! Tell us who you are, what your life is all about (well, at least some of the foundation), and what you like. Upon completion of the survey you will be asked if you want to enter a drawing to win some pretty great gifts.”

And then I clicked into the actual survey. I got demographic questions, questions on readership, and questions on product usage to demonstrate to their advertisers. But what about my “foundation?” Or, “what my life is all about?” Nothing. Nada. Zip. All the magazine wanted from its readers was a fat, juicy reader profile to sell to their advertisers.

There’s nothing wrong with that purpose, or most purposes for conducting market research, as long as your selling argument to participants is consistent with your survey content. In the above example, the survey questions did not accurately reflect the subject line and body of the invite; instead, it felt like a classic bait-and-switch.

By not delivering on what was promised to survey participants (at best), or misleading the true intent (at worst), this magazine risks losing credibility and trust with its readers. And these are the readers who cared enough to respond! That doesn’t help their business, and it hurts researchers and users of market research because participants who are misled or feel their time is wasted become less and less willing to participate in future research efforts.

I have found that a key reason customers participate in surveys is because they want their voices to be heard by the executives responsible for serving their needs and empowered to make important improvements. This magazine missed a wonderful opportunity with its survey to truly listen to its readers and learn what their lives are all about, while still collecting information of interest to advertisers. That would have built a closer connection with readers, and in turn created even more value for the advertisers.

Stay tuned for another installment. And in the meantime, tell me what you think. I promise to listen. : )

Ranking questions ask the research participant to place several variables in priority order – #1, #2, #3, and so on. Each variable ranked is either higher or lower than the other variables – no two share the same ranking. It’s a good way to measure choices, such as a favorite color. Rating questions ask participants to assign a value (importance, performance, satisfaction, etc.) to each variable individually. Here two variables can share the same rating, and often do.

As to which is best for your research project, it depends on the study objectives.  Ratings are often best for separating “must haves” from “nice to have” features.  It’s the freedom to assign a value to each individual variable – even if the respondent believes 5 out of 15 variables are “most important”.

Rankings are generally best if you already know which features are “must haves” – then you can ask respondents to choose one variable over another (such as notebook price vs. battery life).

One point to keep in mind about rankings is they can get confusing for the respondent.  Most of us don’t think about the top ten features we want in a new rackmount server, and respondents can get bogged down thinking about which feature should be at which rank.  We recommend no more than 5-7 variables be ranked for online surveys and a maximum of 3-5 variables for telephone interviews.