Spending differences and boring questions

There are very few examples of exponential growth - poorly designed surveys might be one of the few examples that we all suffer through.

There are broadly two types of surveys:

(i) surveys where your goal is to measure a population and

(ii) surveys where your goal is commercial.

There are times when these overlap, but it's less often than you think.

Categorical questions (gender, age range, profession) make sense when you want to understand a population e.g. a Census. These make much less sense in commercial applications, which is why I'm surprised to see them as often as I do.

Let's say you work in retail and find that, on average, women spend 20% more than men. Great - how can you use this? Do you assume all women will spend 20% more? When a woman is at the checkout, and their spending is lower than you expect, do you go and tell them to buy more stuff? If a man is spending too much do you ask him to return some items? Of course you don't. Why did you collect and analyse this data if you can't use it?

I could present the same examples for age, profession and income. Most surveys are poorly designed, illicit little useful information and waste the time of everyone involved. If you're going to include categorical questions in your survey, justify how you will use these to serve the people you're surveying. If you can't just leave it out.

Once you cut out all the crap questions, you've got space to ask something interesting. What might be interesting in retail? Is there a difference between why genders shop? Does one gender tend to 'replace' what they already have and another gender buy the same item in a different style? If you knew this was the case, maybe you'd have better conversations with your customers.

Why is this hard to implement?

There are two aspects to asking better questions of your customers. The first aspect is technical. Knowing the types of questions and not asking categorical questions. You might not know the terminology, but you can use the heuristic I suggest to ask 'how could I use this?' If the answer is not clear, don't ask the question. Another heuristic is if the question identifies someone, don't include it. You turn people off and gain nothing in the process.

The second aspect is practical. You need to ask questions your customers can answer. Seems obvious? Too often I see companies asking the question they're interested in, blissfully unaware that no one can answer this. I rarely see what companies want to know translated into something someone can answer.

It's the same mistake people make when they sell something that people don't want to buy. I could spend the rest of my life trying to sell 'statistics' but what people want is confidence in decisions, or increased sales, or decreased costs. I believe that statistics is the answer to those questions - much more than storytelling, leadership development or executive coaching but I have to have discussions around the topics that interest people through the frame of my expertise. There is no point in the sales conversation where I'm talking about p-values, as much a I'd like to.

You might want to know about an esoteric topic that you believe your customers can provide useful information on; you might even want to do this rigorously. Congratulations. I applaud the idea; don't stuff up the implementation. Don't ask me questions you want answers to; ask me questions I can answer.

I recently wrote about how Harris Farm came up with the most stupid question I've ever seen. On first glance it seems reasonable until you realise that they've asked what they want to know, instead of what you can answer. I still believe that the entire premise is a waste of time and that you shouldn't be asking your customers about the accuracy of the transactions. You can read my explanation and suggested haiku here.