67% chose the blue pill*
We so often see summarised data encouraging us to buy or do something, that it is easy to forget to be critical. Critical thinking is essential to help avoid making poor decisions due to misleading ‘facts’.
The image above is an advert that appeared on LinkedIn this week encouraging people to vote in a poll. But the question is clearly biased. What if your favourite feature isn’t listed? An accurate response would be ‘Other’ or ‘None of the above’. But those options aren’t available which means you either a) make up an answer or b) don’t respond. This is a common fault with most surveys. Made up answers are more likely if an incentive is given such as free gift for responding to the survey. Otherwise most people would just go ‘Meh’ and not bother voting. But surveys that cover all options can be just as bad. Because people who are interested in the question are still more likely to respond (and respond accurately) than those who are not. Self-selection to participate in surveys will always lead to biased data and unreliable results. It’s the easiest way to make bad predictions.
So next time you see claims such as “84% of CEOs said that <Insert product name> helped increase sales”, check the underlying data for bias (and size! How does the sample size compare to the total population?)
It’s one reason why A|B testing is moving beyond web site design and being used in other industries where the practice can be applied – improving systems by leveraging feedback captured from actual uses rather than what people say they would do. An example is included in the recent presentation: How digital trends are compressing processes.
* The number was made up 🙂 Don’t let it influence a future choice
Featured image is from Flickr: Red pill or Blue pill by Paul L. Dineen
Joining Dots 