Misleading Pictures

Jon Udell has an excellent blog post showing why you should not assume that images are providing an accurate picture.

He mentions reading two of Edward Tufte’s books over Christmas. Coincidentally, I was also reading some of Tufte’s work during December: ‘The Visual Display of Quantitive Information‘ and ‘The Cognitive Style of PowerPoint‘. Both are excellent reads if you are interested in doing a better job of presenting information, as well as how to spot misleading visualisations.

Here is one snippet that should be observed by all those (mostly Microsofties, at the moment) who are swooning over the new visualisation features in PowerPoint 12 and Excel 12:

The number of variables depicted should not exceed the number of dimensions in the data. The use of 2 (or 3) varying dimensions to show one-dimensional data is a weak and inefficient technique, capable of handling only very small data sets, often with error in design and ambiguity in perception.

Anybody who uses 3d bar charts needs to consider this point carefully. Here’s a simple example:

The data presented in these two charts is identical, but it is much clearer and easier to analyse on the right. The shaded area in the 3D version on the left does not add any information and makes it harder to compare the data values.

On a related subject (I had a bit of a reading splurge during December) the book Freakonomics provides some great examples that demonstrate why we should not jump to conclusions and assume cause and effect when we see correlation between two data sets. Correlation only indicates a relationship between two elements, it does not prove that one causes the other. One of the most common abuses of statistics is to present indicators as causes.

As the technologies to store and analyse large quantities of information improve, it is important that we also improve our abilities to correctly interpret and present the information if we are to avoid poor decisions and the resulting consequences.

Related posts:

Dashboard Dangers

The concept of ‘dashboards’ – tools that present top level information collected from different data sources and presented in a single consolidated view – first appeared in the early days of portals (most v1 portals looked like dashboards). They are now making a comeback, often under the title ‘business scorecard’ with ‘key performance indicators’ (KPIs). Real-time communications and analysis technologies enable us to aggregate information, analyse, discuss, and decide quicker than was ever possible before, and dashboards are becoming the meeting point where it all happens – new richer web applications and development tools have reignited the trend in dashboard interfaces. In the world of application integration, ‘business activity monitoring’ is gaining ground, where processes are monitored and updated in real-time.

But there is a risk associated with relying too much on dashboards, particularly when making decisions that can affect business strategy. Having a single ‘dashboard’ can look nice, neat and tidy compared to switching between multiple applications on your desktop. But the price can be over-simplification of complex issues. The ease and convenience of viewing top-level patterns may stop us from drilling down into the messier, time-consuming details, and those details often hold the keys to real causes.

This concept has been noticed by others. The following quote is taken from an entry over on Steve Pavlina‘s web log. (The full article is here)

…In her book ‘Brain Building In Just 12 Weeks”, Marilyn vos Savant suggests that TV reduces your capacity for rational thought. One reason is that TV over simplifies reality. You are presented with subjects in a matter of minutes where everything is nicely wrapped up at the end. Reality is reduced to labels like good or bad, funny or serious, smart or dumb. This harms clear thinking by conditioning you to expect that most problems have a simple clear solution (and, if not, then it will be an overly dramatic solution). But real people and events defy labels. Real life weaves a much richer tapestry than TV. TV skews your map of reality…

Spot the similarities? There are roles where a dashboard adds huge value. Automated management of I.T. systems for starters, where a red alert flashing on a dashboard (and sent to your mobile phone) can warn you about a systems failure that needs immediate action. Business activity monitoring can provide an early warning system for processes that don’t produce the desired results and enable them to be changed before problems escalate. But sometimes (in fact, quite often), allowing time is the essential ingredient for success. Compare the following two charts:

The first chart shows data plotted for 1 year, between 1992 and 1993. The line appears linear – quantity has increased over time. It’s also quite flat – we might think the increase in quantity is not sufficient to warrant the investment we are making, and decide to change the system. But what if we had persevered? The second chart shows the same data but within 10 years of results. The line is clearly exponential, it just took longer than planned for the investment to pay off. We would not have been able to see or prove this growth curve until after it had happened.

It’s not just time that’s important. Often, particularly in innovation and idea generation, unexpected ‘failures’ create new solutions. We learn from our mistakes. Dashboards can be used to prevent us from making those mistakes.

Charles Handy describes this risk, in his book ‘The Elephant and the Flea‘.

…My first independent command was running Shell’s marketing company in Sarawak. There was no telephone line to the regional head office and my bosses in Singapore. We managed because we had to. And maybe it was better, because there was no real way they could judge me other than by results. Things had to be pretty worrying for anyone to spend two days coming to visit me in what was not the most luxurious of places… If I made a mistake I at least had the chance to correct it before anyone noticed. That might not be possible today without a lot of self-discipline by superiors. Fewer mistakes, maybe, but less learning, less responsibility.

Make sense? It does to me. The danger with dashboards is that they encourage us to over simplify and jump to early conclusions that may not be representative of the real situation. They can prevent us from making mistakes, when making mistakes can provide vital learning opportunities. They can discourage us from trusting people to take responsibility and find a solution. We are no longer satisfied with viewing the results after they happened, we want to use the dashboards to predict the future and enable us to change direction immediately. We can do that, and there will be times when that ability provides great results. But we also need to appreciate the risks involved in not giving people the time and trust to find solutions for themselves. Dashboards in the wrong hands can ruin an organisation…

  • This entry is filed under Analysis on the web site.
  • Related site topic: Growth – explaining the differences between linear and exponential growth

Update: broken link fixed