When Will Companies Figure Out that Tableau, Power BI and other Data Visualization Tools Have Been Massively Oversold?
Executive Summary
- The real impact of data visualization tools like Tableau has been far less than advertised.
- When do data visualization tools finally get called out for overstating their benefits?
Introduction
Companies like Tableau and other business intelligence vendors have presented a picture of analysis where analytics is at the center of a process that continually delivers amazing insights to these applications’ owners.
Tableau has a long-term pattern of aggressively overstating how easy their product is to use (how much time it takes to create analytics or reports). However, they also foster the idea that one does not require domain expertise to understand data. Other analytics vendors present exaggerated claims to a more or lesser degree.
Tableau, Better on a Resume Than in Real Life?
I have noticed for years how little relevance the visualization tools have had in companies. Tableau and other visualization tools are yet another application that the IT department is very interested in getting on its resume but is of minimal benefit to the business.
Tableau might be the worst offender, but the visualization software category as a whole massively oversells what their tools help companies accomplish. Tableau pushes the idea of a direct correlation between charts and better decisions. If they ever tried to prove this, I suspect that an honest analysis would have them unable to verify this claim.
Visualization Software: How Similar to the PowerPoint Claim?
The visualization claim is an extension of the PowerPoint claim. The concept that was sold was that if we could only have slides, we would make better decisions. Then PowerPoint became known for content-free and deceptive presentations. NASA partially blamed the over-reliance on PowerPoint for the decline of analysis quality inside of the organization.
It will take a little longer for visualization to earn a similar reputation – as visualization software is newer to the market.
As I wrote in the essay above, “Both the Columbia Accident Accident Investigation Board (2003) and the Return to Flight Task Group (2005) were filled with smart experienced people with spectacular credentials. These review boards examined what is probably the best evidence available on PP for technical work: hundreds of PP decks from a high-IQ government agency thorough practiced in PP. Both review boards concluded that (1) PP is an inappropriate tool for engineering reports, presentations, documentation;(emphasis added)and (2) the technical report is superior to PP. Matched up against alternative tools, PowerPoint loses.” – Edward Tufte
This critique goes on from a visiting professor to Microsoft.
Attempting to have slides serve both as projected visuals and as stand-alone handouts makes bad visuals and bad documentation. Yet, this is a typical, acceptable approach. PowerPoint (or Keynote) is a tool for displaying visual information, information that helps you tell your story, make your case, or prove your point. PowerPoint is a terrible tool for making written documents. That’s what word processors are for.
Why don’t conference organizers request that speakers instead send a written document that covers the main points of their presentation with appropriate detail and depth? A Word or PDF document that is written in a concise and readable fashion with a bibliography and links to even more detail, for those who are interested, would be far more effective. When I get back home from the conference, do organizers really think I’m going to “read” pages full of PowerPoint slides? One does not read a printout of someone’s two-month old PowerPoint slides, one guesses, decodes, and attempts to glean meaning from the series of low-resolution titles, bullets, charts, and clipart. At least they do that for a while…until they give up. With a written document, however, there is no reason for shallowness or ambiguity (assuming one writes well). – Elizabeth Lane Lawley
Companies that Use Tableau Use Tableau or Excel?
Even in companies that own and use Tableau, I keep seeing Excel-generated graphics. And even there, it is not like those “ah-ha” moments are particularly common.
Great Graphics — for False Measurements
I had one client that showed its forecast error as total units. If the forecast for total units was 25 million in one month, and the error was 23 million, then the error was recorded as 2/25 million. The graphic the demand planning department used to show this was pretty nice. However, the assertion and the presentation of this was of measuring forecast accuracy was completely misleading. It was deliberately designed to deceive the audience as the company in question did not sell a single item at a single location.
This was done to make the demand planning department look like it was doing a better job forecasting than it actually was. This type of falsification of forecast error is widespread in companies, and the existence of analytics software does not change or improve this fact.
The Rule of Supply Chain Forecast Error Measurement
There are many ways of measuring forecast error. For example, sales tend to measure the forecast error at a product group or geography. However, for supply chain forecast error measurement (which was the discussion area), the forecast error must be recorded at the product location combination. This is because the forecast error is used to drive the safety stock. And two different items cannot share the same safety stock. Two locations — unless they are extremely close, for the same item, also cannot share the same safety stock.
Conclusion
Analytics applications like Tableau provide little benefit to people companies that have them.
At some point, visualization applications will pick up the reputation for not being particularly beneficial. Furthermore, the degree of usage of applications like Tableau is greatly overstated by the vendors and IT departments.
https://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001yB