Why Gartner and Forrester Do Not Want TCO Calculated

Executive Summary

  • The last thing most IT analysts like Gartner want is an accurate TCO.
  • This article explains why Gartner prefer an underestimated TCO.

Why IT Analysts Don’t Want TCO Calculated

Gartner, the IT analyst company, was one of the first entities to market around the idea of TCO. However, Gartner never did much more than stay at the conceptual level. The likely reason for this is fairly simple: if Gartner had actually calculated the TCO for the applications it reviewed, it would have given all the wrong answers.

How Gartner Pushes Customers to the Most Expensive Solutions

Gartner also tends to push buyers toward more expensive solutions. This higher expense is not only for software but also for services. For instance, I work as an SAP consultant, and SAP consultants are some of the most expensive resources in IT. One aspect of the cost is the hourly billing rate, which is high. Another aspect is that SAP software is complicated to install, so SAP projects tend to be quite long.

Not only does Gartner not disclose its income from vendors for technology advisement services, but also does whatever it can to prevent buyers from knowing that it takes any income from software vendors. When someone who knows how the system works brings up this issue, Gartner points to the fact that they have an ombudsman, so none of the quite significant revenues they get from vendors has any influence on their ratings. However, this ombudsman follows none of the rules of a traditional ombudsman, and as the complaint process is completely hidden from view, the ombudsman cannot be considered an objective arbiter. This, in fact, was the exact charge in the lawsuit brought about by a software vendor named ZL Technologies, as explained in the following quotation.

Hypothetical Interest in TCO

Hypothetically, IT analyst firms should be good candidates for performing TCO analysis. Gartner, the largest IT analyst firm in the world, is actually credited with first introducing the concept of TCO to enterprise software. However, as has been discussed, most IT analyst firms have major conflicts of interest because they sell consulting services to vendors. They also take considerably more money from the large vendors in comparison to the smaller vendors—and not coincidentally larger vendors tend to charge more for their software, more for their consulting, and so on. The upshot is that the TCOs of large vendors will be higher.

Since the vast majority of IT analyst firms take the most money from vendors with the highest TCOs, they have little incentive to provide detailed TCO analyses, as doing so would probably cause them to lose some of their software vendor consulting revenues. Also, different IT analyst firms have different interpretations of TCO. Gartner, for instance, tends to be “pro-TCO,” while Forrester tends to be “anti-TCO,” proposing that most companies cannot aspire to TCO successfully and that simpler approaches should be used. Forrester also points out that, according to their surveys, only about twenty-two percent of purchasing companies use any type of TCO in their decision-making.

Is TCO Even Possible?

Several entities disagree on whether or not a full TCO analysis should be a goal. In this section, we will review the concerns leveled frequently at TCO. One of the
entities—which could be labeled as anti-TCO and which is influential in the area of enterprise software decision-making—is Forrester, the IT analyst firm.

“To really implement TCO-based analysis it takes a comprehensive and continuously updated catalog of asset inventory, in-service dates, agreedupon
operating cost rates for activities, and a scheme to divide shared costs among the constituent business processes that use them. For most firms, this is a pipe dream viewed either as a waste of resources in a futile quest for achievement or too intimidating to even begin. Forrester recommends a more expedient and realistic financial approach that can be just as effective but much simpler to calculate—relative cost of operations (RCO). RCO can be a middle-ground solution, moving far beyond acquisition-cost-only analysis, while being more achievable than a full-blown TCO.” — Forrester

I agree with Forrester’s assertion that companies rarely use TCO, but I do not see why performing a complete TCO analysis is beyond most companies if they are properly advised. We have performed a wide variety of TCO analyses for software categories and make these analyses available for a fee at our sub-site. Because we take no money from vendors, we can legitimately say that our TCO analyses do not have a bias. One could always propose other non-financial biases—and these are possible and also come down to whether you consider exposure to a topic to be a bias or to be knowledge (babies, after all, are completely unbiased and open-minded)—however, if one analyzes the major problem with respect to objectivity in analysis in not only the enterprise software arena, but also other areas such as financial advisement —consistently the problem is financial bias. This is true to such a degree that once financial bias is removed one has taken care of the vast majority of the problem. Forrester is not alone in their concerns regarding TCO. The white paper Rethinking TCO takes a similar view of TCO as Forrester. Rethinking TCO points out some of the limitations of performing a meaningful TCO analysis when they state:

The large amount of variability in product complexity and comprehensiveness from one enterprise vendor is an important limitation on the potential for using TCO in multi-vendor analysis.”

However, this type of calculation will be performed either implicitly or explicitly (with an actual TCO); therefore, it is difficult to see how the approach of not performing the analysis is better than performing a TCO, especially considering what is at stake. In fact, merely performing the analysis—imperfect as it may be—puts
a company in a more analytical frame of mind during their selection process.

Customized Versus Noncustom TCO

“…different companies within the same industry may have significantly different business processes expressed in the same software product, or may have extensively customized a standard software package in order to gain some degree of competitive advantage. Indeed, this variability in use often represents a key strategic value for the enterprise software package: by using a standard software product in a nonstandard or customized fashion, many customers hope to gain competitive advantage over other companies in their industries that may be using the same or similar software.”

That is all true. However, this is where the implementation experience of individuals performing the TCO comes into play. If they are sufficiently experienced, they
can increase or decrease the cost of a particular item through their knowledge of how much customization can be expected on the project. Furthermore, a noncustomized TCO value probably would not make a lot of sense. It appears that this is an argument against non-customized TCO calculations.

Implementation Costs as a Percentage of TCO

“The implementation process is another major factor that adds to the problems with using TCO as a vendor selection tool. Implementation costs figure as one of the largest single expenses in enterprise software, and yet they are neither standardized nor consistent from one vendor to the next or one implementer, or implementation, to the next.”

While true, this can easily be accounted for in the TCO estimation, so it is difficult to understand the exact concern. Yes, implementation times can be adjusted for TCO estimations. And that is not the end of it; maintenance is quite different from one software vendor to the next and we adjust the maintenance costs per software vendor.

We account for all of these differences in the TCO estimation. For example, if an implementation is from a large consulting company, we adjust the estimate accordingly. If the implementation involves a large brand-name software vendor, we again extend the implementation timeline. For example, when we develop estimates for software vendors like SAP (which is neither designed to be easily implemented or easily used), we impose higher costs than any other software in the enterprise market. SAP takes longer to install, which means implementation costs are high, as are maintenance costs. This is why it is so important to check that the software functionality is reliable and usable; not only is the application more effective, the cost of the implementation is reduced, thus reducing its TCO and increasing its ROI. This goes back to a central theme of this book: without quantification through something like a TCO process, reliability and usability may be presumed to be equivalent between various software applications, when, in fact, they are quite different. Strangely, no one seems to bring up the issues that arise if costs that are assumed to be equivalent end up being quite different, making it difficult to make good decisions.

Conclusion

The system is rigged in favor of vendors that pay the most and the vendors that can afford to pay Gartner the most are those with the highest license fees. Therefore, one of Gartner’s main roles is to guide buyers to both the most expensive TCO software solutions in the market, as well as to the most expensive consulting companies. Gartner could not perform an honest TCO analysis for any of the software categories that it covers without alienating its biggest vendor customers. Therefore, Gartner has no interest in TCO beyond discussing it as a general principle.

Financial Disclosure

Financial Bias Disclosure

Neither this article nor any other article on the Brightwork website is paid for by a software vendor, including Oracle, SAP or their competitors. As part of our commitment to publishing independent, unbiased research; no paid media placements, commissions or incentives of any nature are allowed.

Search Our Other Gartner Content

References

Gartner Book

Gartner and the Magic Quadrant: A Guide for Buyers, Vendors, and Investors

Gartner is the most influential IT analyst firm in the world. Their approval can make or break a vendor in an application category, or at the very least control their growth. Gartner has been behind most of the major IT trends for decades. However, many people read Gartner reports without understanding how Gartner works, how it comes to its information, its orientation, or even the details of the methods it uses for its analytical products. All of this and more is explained in this book.

Table of Contents

  • Chapter 1: Introduction
  • Chapter 2: An Overview of Gartner
  • Chapter 3: How Gartner Makes Money
  • Chapter 4: Comparing Gartner to the RAND Corporation, and Academic Research
  • Chapter 5: The Magic Quadrant
  • Chapter 6: Other Analytical Products Offered by Gartner
  • Chapter 7: Gartner’s Future and Cloud Computing
  • Chapter 8: Adjusting the Magic Quadrant
  • Chapter 9: Is Gartner Worth the Investment?
  • Chapter 10: Conclusion
  • Appendix a: How to Use Independent Consultants for Software Selection
  • Appendix b: What Does the History of Media Tell Us About This Topic
  • Appendix c: Disclosure Statements and Code of Ethics