How Accurate was Forrester’s TCO Study for SAP HANA?

Executive Summary

  • Forrester was paid off by SAP to create a false TCO study that is totally inaccurate.
  • SAP fed a passive Forrester TCO information on HANA.

Introduction to the TCO of HANA

In this article, we will review Forrester’s TCO study of HANA published in April of 2014. This study was funded by SAP.

At the time of this publication, less was known about HANA than is known today.

Forrester’s Explanation of HANA

Even Forrester’s beginning explanation of HANA appears curious.

“SAP HANA is a distributed in-memory data platform that enterprises can use to support real-time analytics, predictive and text analytics, and extreme transaction volumes. The next-generation data platform demands looking at these new technologies to help deliver the speed, agility, and new insights critical to helping your business grow. For decades, organizations have built the transactional, operational, and analytical layers to support various applications, operational reporting, and analytics. However, with the growing need to support real-time data sharing driven by mobile enterprise, separate transactional, operational, and analytical layers are creating an obstacle in supporting such an initiative. Distributed in-memory data platform offers a new approach to collapse the technology stack that can eliminate redundant hardware, software, and middleware components to save money and reduce complexity through automation and integrated systems that can help developers and DBAs become more productive.”

This sounds like it was provided by SAP.

SAP HANA Total Cost of Ownership Framework

Here is where Forrester lays out the foundation of the research.

“To better understand the costs and risks associated with a HANA implementation, Forrester interviewed several customers with multiple years of experience using HANA and conducted a survey of 25 additional HANA users. Drawing on the experiences of these customers, as well as Forrester expertise in this area, we constructed a financial model to represent the potential savings associated with using HANA as a replacement for a traditional database platform in several different ways.

Based On Financial Projections

The financial model Forrester created was entirely a projection. At the time of the publication, Forrester would not have had live HANA instances to draw upon.

While the customers had just begun to use HANA in this way, they were able to discuss the projected impact as they continued to expand their use of HANA as a replacement platform. In this financial analysis, we include the impact of using HANA in conjunction with the SAP Business Warehouse (BW) powered by SAP HANA, SAP Enterprise Resource Planning powered by SAP HANA, and a custom-developed application. For each case, we project the costs associated with using a traditional database platform, as well as with using HANA, and calculate the comparative TCO. We project a migration for BW in the first year of the analysis; in Year 2, the composite organization begins using HANA in conjunction with its current ERP deployment; in Year 3, the former ERP environment is retired and it is run on HANA; and, we project the costs and cost avoidance associated with developing a custom application on HANA (as compared with developing it on a traditional database platform) in the third year. The financial analysis is based on this scenario, as shown in Figure 1. The cost projections are based on information gathered through interviews and surveys, pricing information from SAP, and Forrester knowledge of database and application market rates.

Prior to HANA, customers were using traditional database platforms for their applications (both SAP and non-SAP), incurring costs associated with required hardware and software for processing and data storage, as well as the labor required for development and administration. By transitioning these applications to HANA, Forrester projects that customers are able to reduce the hardware and software required for these applications, as well as reduce the efforts for administration and development resources.”

This projection has not held up. The costs of HANA have turned out to be far higher than competing options — higher even than Oracle which has a very high maintenance cost.

Benefits of HANA?

This is an important part of the study. This is a projection. It is not based on what happened at these clients.

“This is due to the way that HANA eliminates or reduces the need for a variety of associated software and hardware, and adds efficiency to the development and system maintenance processes. Many of the customers we interviewed stressed that HANA also provided significant benefit to the business. However, for the purposes of this analysis, we focus entirely on the IT impact. Our interviews with four existing customers, survey of 25 additional customers, and subsequent financial analysis found that a composite organization based on these interviewed organizations could expect to experience the risk-adjusted costs and costs savings shown in Figures 2 and 3 when utilizing HANA for this purpose. 1 (See Appendix A for a description of the composite organization.) As demonstrated by the total cost savings with HANA, there were significant reductions in the hardware, software, and labor required for each application.”

Strange Statements from Forrester

That is strange. HANA has more hardware than other databases because it relies upon a number of components. As far as hardware, HANA has the highest hardware requirement of any database we track. This is because the database requires so much memory. The labor is also higher than competing offerings because HANA is still immature as a product. In fact, it was incredibly immature when this study was published.

At this time HANA could not be discounted and was the most expensive database one could purchase. HANA hardware is known to be very expensive. HANA labor was and is extremely expensive. HANA eventually became a discounted item, but that was several years later.

Incorrect Cost Savings Proposals

“Savings. Based on the composite total cost model, the sample organization experienced the following risk-adjusted benefits (that represent those experienced and projected by the interviewed companies): • Reduced hardware cost (simpler data footprint/simpler landscape/simpler setup). Moving to the HANA platform allows customers to reduce the volume of servers and storage required, due to the data compression, as well as efficiencies gained due to HANA’s functionality. • Lower software costs (simpler data footprint/simpler landscape/simpler setup).”

HANA has a very complex setup and has very high maintenance costs. This is due to several factors, one being the immaturity of the database. It simply has a very high TCO. Forrester’s proposals are the exact opposite of what was learned about HANA.

HANA as an ETL, Middleware & Development Platform?

“HANA also allows for a reduction and elimination in many of the software products such as ETL, data replication, management, and other middleware software. Additionally, the HANA platform includes advanced analytics capabilities such as text, geospatial, and predictive analytics as part of its platform offering without additional license. • Faster development time (simpler application development/simpler user experience). All of the interviewees cited a faster, more efficient application development process through automation and minimizing complexity, which allowed for a reduction in resource time. • Increased productivity for administrators (simpler processing and operations). Due to the simplification of the environment with HANA, customers are able to reduce the amount of administration time required to tune, optimize, and manage databases and servers.”

Are You Serious Forrester?

That assumes that HANA can be used as the application and reporting platform among other things. That is a big assumption and turned out no to be true.

The other parts of this quote sound like they came from SAP marketing. HANA never end up offering any ETL or middleware functionality. It offered no data replication functionality that was superior to any other database. HANA’s “analytics” never panned out. The HANA environment turned out not to be simplified, but instead became well known as the most complex database environment among the competing alternatives (Oracle, IBM, Microsoft). There is no particular automation that has ever appeared in HANA.

Did SAP Make Up Stories Case Studies About HANA from Whole Cloth?

Every single item listed here, supposedly listed in the interviews that SAP supposedly made with customers, and then passed on to Forrester appear to have been made up. There is no other explanation for how the “interviews” could be so much different from the reality that later came out about HANA.

“Based on the composite total cost model, the sample organization experienced the following risk-adjusted benefits (that represent those experienced and projected by the interviewed companies):

Reduced hardware cost (simpler data footprint/simpler landscape/simpler setup). Moving to the HANA platform allows customers to reduce the volume of servers and storage required, due to the data compression, as well as efficiencies gained due to HANA’s functionality.”

No HANA’s hardware is more not less expensive. We have this information from many HANA customers at this point. HANA does not compress anywhere near the predicted 98.5% proposed by SAP. This is covered in the article, How John Appleby Was so Wrong About HANA.

“Faster development time (simpler application development/simpler user experience). All of the interviewees cited a faster, more efficient application development process through automation and minimizing complexity, which allowed for a reduction in resource time.”

Why is any of that true?

HANA’s Environment is Simpler?

“Increased productivity for administrators (simpler processing and operations). Due to the simplification of the environment with HANA, customers are able to reduce the amount of administration time required to tune, optimize, and manage databases and servers.”

HANA’s environment is not simplified.

The sample organization experienced the following risk-adjusted costs:

“Software licensing costs. This includes the software cost associated with HANA, as well as the costs of the application software. We include initial costs as well as ongoing maintenance.

The majority of customers surveyed used full use licenses versus runtime licenses for their database software. This study has calculated savings based on runtime software license cost to better compare similar costs between HANA and potential alternatives. Using full use licenses will raise the software cost associated with the non-HANA scenarios for BW and ERP significantly; we encourage the reader to consider which type of license makes sense in the appropriate environment.”

The Logic of Using a Runtime License

Then why did Forrester used a runtime license to calculate the software cost? Runtime licenses are temporary, the full use license should have been used. The last part of Forrester’s statement is completely misleading. Customer need to use a full use license, it is well known that the runtime license is just for SAP to get their foot in the door.

“The study is commissioned by SAP and delivered by Forrester Consulting. It is not meant to be used as a competitive analysis.”

Yes, and it shows.

“Forrester makes no assumptions as to the potential cost savings that other organizations will receive. Forrester strongly advises that readers use their own estimates within the framework provided in the report to determine the appropriateness of an investment in SAP HANA.”

This is a dodge. SAP directly used this study to declare that “HANA reduces TCO.”

Conclusion

This study is paid for research and it turned out to be false on every claim. For the study to be so inaccurate versus what we later found out about HANA, SAP would have had to have falsified the customer examples that were given to Forrester. Forrester simply blindly accepted what SAP told them rather than performing analysis.

This study demonstrates very clearly that if you pay Forrester, you can get the result that you want.

Brightwork Disclosure

Financial Bias Disclosure

This article and no other article on the Brightwork website is paid for by a software vendor, including Oracle and SAP. Brightwork does offer competitive intelligence work to vendors as part of its business, but no published research or articles are written with any financial consideration. As part of Brightwork’s commitment to publishing independent, unbiased research, the company’s business model is driven by consulting services; no paid media placements are accepted.

HANA & S/4HANA Question Box

  • Have Questions About S/4HANA & HANA?

    It is difficult for most companies to make improvements in S/4HANA and HANA without outside advice. And it is close to impossible to get honest S/4HANA and HANA advice from large consulting companies. We offer remote unbiased multi-dimension S/4HANA and HANA support.

    This article is free, we do not answer questions for free. Filling out this form is for those that have a budget. If that describes you, just fill out the form below and we'll be in touch asap.

References

The Risk Estimation Book

 

Software RiskRethinking Enterprise Software Risk: Controlling the Main Risk Factors on IT Projects

Better Managing Software Risk

The software implementation is risky business and success is not a certainty. But you can reduce risk with the strategies in this book. Undertaking software selection and implementation without approximating the project’s risk is a poor way to make decisions about either projects or software. But that’s the way many companies do business, even though 50 percent of IT implementations are deemed failures.

Finding What Works and What Doesn’t

In this book, you will review the strategies commonly used by most companies for mitigating software project risk–and learn why these plans don’t work–and then acquire practical and realistic strategies that will help you to maximize success on your software implementation.

Chapters

Chapter 1: Introduction
Chapter 2: Enterprise Software Risk Management
Chapter 3: The Basics of Enterprise Software Risk Management
Chapter 4: Understanding the Enterprise Software Market
Chapter 5: Software Sell-ability versus Implementability
Chapter 6: Selecting the Right IT Consultant
Chapter 7: How to Use the Reports of Analysts Like Gartner
Chapter 8: How to Interpret Vendor-Provided Information to Reduce Project Risk
Chapter 9: Evaluating Implementation Preparedness
Chapter 10: Using TCO for Decision Making
Chapter 11: The Software Decisions’ Risk Component Model

Projected_Cost_Analysis_of_the_SAP_HANA_platform.pdf