How to Understand Forrester’s Fake S/4HANA TCO Study

 Executive Summary

  • Forrester created another TCO/ROI study that was paid for by SAP.
  • We review the accuracy of this Forrester SAP S4HANA study.

Introduction

Forrester has a history of taking money from SAP and then producing exactly the outcome results that SAP would have wanted. In this article, we will see how Forrester did with its analysis of S/4HANA.

Forrester Gets to Talk to Three Companies?

According to Bill McDermott S/4HANA is the fastest growing application in the history of SAP. So why did SAP only make 3 customers available to Forrester? It does not take very much time to interview customers, as SAP was paying for the study, one would think they would make an effort to bring more customers to the table.

Forrester found the following financial benefits.

Three Year Benefits

These look interesting until you see how these numbers were determined. The logic for these numbers do not make sense, as will see from the quotations. Forrester did not find any of these cost savings, they are all fake. 

Quotations from the Study

Let us look through some of the most impossible quotes from the article. These quotes were apparently stated to Forrester from executives in the companies provided to Forrester by SAP and Forrester did not bat an eye.

A potential reason for this is that a) Forrester is being paid here to look the other way and b) it’s very rare for anyone who works at Forrester to have any implementation experience.

But to a person with implementation experience, these quotes make little sense.

Complex Supply Chains Lead to Lack of Inventory Visibility?

“Our supply chain from extraction to market is exceedingly complicated. We have over 157 different avenues for how we go to market. By the numbers, we have 60,000 unique materials, 6,000 vendors, 4,000 customers, and an additional 10,000 partners.”

S/4HANA’s supply chain functionality still lags ECC by a substantial margin. Secondly, a complex supply chain should not impact inventory management. Each one of these locations, no matter how many, is simple setup as a plant material combination with a connection to another plant material combination. I have personally had many customers with the numbers listed above, so it is not at all surprising or more complicated than what many other companies deal with, and deal with using ECC. It is a simple goods issue and goods receipt in ECC. If this quotation were from 5 years ago, ECC would be put forward as what solved this issue. Pre-S/4HANA if you were to have asked SAP if they can handle

“157 different avenues for how we go to market. By the numbers, we have 60,000 unique materials, 6,000 vendors, 4,000 customers, and an additional 10,000 partners.”

Their answer would have been that ECC is running the supply chains of the largest companies in the world, so the scenario above is nothing they have not done. The study is not clear that the company moved from ECC, they may have moved from another ERP system. But this study is trying to make the burning platform to move to S/4HANA when almost all the large supply chains using SAP in the world presently are running ECC.

S/4HANA Provides Real-Time Inventories?

“The most important change is that we now know the real inventories at our mine locations and operating plants. This gives us visibility from the extraction point to the final shipping to buyers. We can now measure real-time inventory across the entire company.”

All ERP systems have immediate visibility to inventory and have had this for many years now. The term real-time inventory is used, however, systems stopped advertising “perpetual inventory” many years ago as all systems that deal with inventories have this. That is it ceased to be a differentiator, where at one time it was. In fact, perpetual or real-time inventory was one of the original selling points of purchasing MRP systems back in the 1980s. We cover this in the article Whatever Happened to the Perpetual Inventory.

Note to executives, please stop saying things that reflect information technology crazy mouth, like your new system provides real-time inventory. Part of the entire logic for why you are paid so much more than employees is based upon the concept that you know more than the average person. When you say things like this, it ruins the illusion. 

SAP and compliant customer executives have a habit of proposing that a new item newly solves a problem that was already supposed to have been solved with a previous version. If automobile manufacturers followed SAP’s lead in this regard, automobile ads might look something like the above. 

Sales People No Longer Losing Deals Due to Faulty ATP Checks?

“Our salespeople were losing deals because we could not promise when the products would be delivered. We just didn’t have visibility into our supply chain to be able to make commitments when our competitors could and did make promises. A seemingly small system problem became a competitive disadvantage.”

It is difficult to see how the situation with S/4HANA will be much different. S/4HANA does not have superior ATP functionality over ECC. is this company having a problem with ATP because of functionality or because of other reasons like their production schedule changes too often? There can be many reasons why the available quantity can be incorrect. However, this is too much nuance and Forrester has a preconceived conclusion they need to reach, so lets move on.

All Companies Benefited from Reduced Operating Costs and Inventory Carrying Costs?

“Each of the companies that Forrester interviewed reported significant improvements in managing operating costs, such as inventory carrying costs. Using SAP S/4HANA enabled the composite organization to combine cost data with inventory data in a way that enabled employees to optimize flow and cost of inventory.”

This might be a good time for Forrester to re-emphasize that SAP did not allow Forrester to speak to any customers who bombed on their S/4HANA implementations, or even to the average S/4HANA customer. But our research supports the fact that most S/4HANA implementations (as covered in the Implementation Study of S/4HANA) have been unsuccessful. That is the failure rate is far higher than 50%. This failure rate is currently covered up by both SAP and the large SAP consulting firms. In fact, SAP also uses an NDA for its customers that implement S/4HANA, which means that no matter bad the experience, customers can’t talk about it. Forrester did not see fit to report on the S/4HANA NDA.

This is the problem with Forrester posing as doing research while its entire sample is unrepresentative of the population. What is the value of a completely rigged sample designed to lead to a predetermined outcome? The answer is zero…or perhaps negative.

The Company Reduced the Need for 600 Employees….And No One Was Fired?

“The financial model is based on a composite organization that was able to reduce the need for 600 employees. Although these staff will produce additional revenue in the future, the immediate benefit is measured by the reduction in employees required to do the same work. The future value is discussed in the Flexibility section.”

This is the equivalent to having your cake and eating it too.

Cost Savings…..But With No Cost Savings!

It is highly unlikely that a company would make 600 employees redundant, but keep them on staff. Companies today are simply too bottom-line oriented. And every single one of these people is a good fit to be moved to a new job?

Hmmmmmm…… Some of those employees, if not most of them would have been fired. However, this is a way that a company can claim cost savings, while not admitting that they fired people. This is political language that circumvents the discussion of what really happens to employees that are supposedly reduced by systems. However, this is also quite overstated. None of the projections around the labor savings of ERP systems turned out to be true — what did happen is companies were required to staff up IT.

Notice the following quotation as to the financial benefit of the moved employees.

Massive Revenue Unrealized Because of Lack of Employees

“By centralizing planning, production, and tracking functions, we eliminated the need for 600 employees. We redeployed those people into new locations to open new mines and refineries. It’s still too early to claim any revenue, but within a couple of years, we expect that these same employees will drive growth that will be hundreds of millions of dollars.”

Let us assume a future revenue of $250 million (they don’t specify how many hundreds of millions, only that it is plural, so $250 million would be conservative). Divided by 600 employees this is $416,000 per employee. if this type of opportunities were available to the company, they now look incompetent to have not hired these people before the S/4HANA implementation freed up 600 people. Let us review what we know about this mining company.

  • They don’t understand that their inventory system has been real time for decades.
  • They allow investment opportunities to wither on the vine, instead of hiring people to drive hundreds of millions in revenue.

How badly run is this mining company?

S/4HANA Took Nine Months to Implement?

“The composite organization no longer needed to pay the annual subscription costs or maintenance fees for older applications. The company executives told Forrester that it took an average of nine months to implement SAP S/4HANA.”

Naturally, the implementation times reported are far less than the implementation for ECC, which is a one-year minimum, but often has implementations stretch to 2 or 3 years or more. This is true even though S/4HANA is far less mature than ECC, and immature products take longer than mature products to implement. Every single SAP reported implementation duration is shockingly fast.

However, when information that SAP does not control is reported, the timeline is quite long. In the case of the Lidl failure, it was a total of 7 years, with S/4HANA being a subcomponent of that as they transitioned to S/4HANA after beginning with ECC. Haribo, another failure was also 7 years. Yet the companies interviewed by Forrester averaged 9 months? If we include the implementation duration of Lidl and Haribo, what happens to the average of Forrester’s sample? Forrester did not even have to interview these companies as the duration and cost was published due to the projects failing.

Real Implementation Data Points Keep Rolling In that Contradict Forrester

It was later learned that the Schweizer Post S/4HANA implementation rose to $84 million. 

Wait a second, why did this implementation cost even more than $1 million? According to Forrester, a typical S/4HANA implementation is around $877,000 and runs around 9 months with only two consultants who stay on the project for 6 months. Forrester’s study of SAP provided sources showed almost no variance in this.

Now to hear of a $83 million implementation is very distressing. The 12X cost overrun seems odd, as this amounts to $83M/.87M 95X the common cost of and S/4HANA implementation according to Forrester. Actually, this cost will rise in the future. If we include just this implementation into the Forrester “study” we end up with (3 * $.877M + $84M) or $21M. Could there be some reason that the Schweizer Post case study was not provided to Forrester by SAP? I mean isn’t SAP interested in providing the full context of data points to Forrester? Its a study right???

It is curious. We keep teaching statistics and talking about the importance of math, but we don’t seem to have research entities where you can be employed to use this math honestly. Forrester can publish a study with 3 rigged case studies, and it goes with little notice. Math + lying does not lead anywhere.

Sales Orders are Entered in Manually?

“We design complex systems for our customers, but then the information must be converted into orders and placed into SAP. Today this is a manual process, but we are looking into the idea of having the information convert directly over, which will reduce delays and avoid errors.”

This company has not yet figured out how to upload externally generated sales orders to S/4HANA? Interesting.

These implementation costs are ridiculous. ECC and S/4HANA implementations take many months of consulting support. In this case, the consultants rolled off three months before the go-live (or they were part-time).

Immature ERP System Implemented with Two “Swiss Army Knife” Consultants

S/4HANA is a risky implementation because so much of it is still immature. If only two consultants were used, how would that have worked? A minimum is to have a consultant for materials, finance/accounting, and sales. (it is unclear if production was implemented).

Which of the two consultants covered two areas?

And further, who covered HANA? S/4HANA must be implemented with HANA, a high overhead database. S/4HANA consultants are not database consultants.  So one consultant implemented S/4HANA and the other consultant implemented HANA?

It also means that there could not have been any variability in the number of consultants. Unless one project had one S/4HANA/HANA consultant, and another project had three.

  • This means that one consultant implemented HANA while the other implemented S/4HANA? Or HANA took 3 months of consulting time to implement and S/4HANA took 12 months of consulting time (summing to 9 * 2 = 18 total months of consulting)(that is for some of the time there were 2 S/4HANA consultants and sometimes not?)
  • So those two consultants are “Swiss Armied” to the extreme degree. And, they were able to run this lean on three different projects with three different sets of consultants?

Unrealistic Implementation Durations…..from Forrester

Every report by Forrester highly understates the implementation costs to customers. There are some clients with 70 to 100 SAP consultants on site (for multiple products), and with immense monthly billings, but every client Forrester seems to interview has a tiny amount of billings. However, if you approach Deloitte and ask them to help you implement S/4HANA with two consultants for six months, they will laugh at you.

For this claim, Forrester wins our highly coveted Golden Pinocchio Award. However as the lie was funded by SAP, and SAP knowingly provided the faulty information and would have manipulated the customers into providing the false information, the award is co-shared with SAP. 

This estimate is even less believable because it is an average, which means that these minimum values were the average of the three companies. That is this is not from a single company that had some very small scope.

We can say with high confidence that this did not happen.

Either the 3 companies were instructed to lie to Forrester or Forrester changed these values as directed by SAP.

Inverted Implementation Versus License Ratio?

Notice the license/subscriptions costs that serve as the major part of the costs for S/4HANA. Something that should be a major red flag is that the license costs are far below the implementation costs.

In fact the ratio of implementation costs to license costs is $833,750 / $3,000,000  or 27%. That is never the case, the implementation expense on SAP projects is a multiple of the license cost, not a fraction of the license revenue. Even an SAP sales rep with a few months under the belt knows this (it comes up during discussions around implementation estimates for customers). And the fraction here is far less than 1. If multiple of 2 were used, that would mean that all three customers in the study 2/.27 = .135 or 13.5% of a typical implementation. Two is the number that many consulting companies use in the sales process, but it is far too low. Readers can see for themselves at the Brightwork ECC TCO Calculator (this is for less than 800 users, see the included link for more than 800 users. But even a more simple ECC implementation for say 500 users will run up more than $20 million in consulting fees.

However, SAP makes most of its money from licenses and support, so the unrealistic consulting costs are not SAP’s problem, they are Deloitte and Accenture, etc…’s expectations to manage.

Forrester is Still Learning About Implementation Versus License Revenues

Is Forrester serious unaware of the implementation/license multiple that is demonstrated through decades of SAP implementation experience? Let us look at the Lidl example again, that was 500 million Euro. ECC implementations are routinely $100 million and often much more. SAP even has multi-billion dollar failures on its resume. The Oracle ERP “legacy replacement” (it is well known that Oracle ERP is less in implementation costs), with the Airforce, cost $500 million before the program was stopped. Sub million dollar ERP implementations from commercial vendors in general, but particularly with SAP do not exist. Forrester seems unconcerned that other people will also read this study and see how impossibly inaccurate it is.

And the errors get perhaps (even more?) or just as extreme with the next calculation for license costs.

Where are the Costs for HANA?

Forrester is quite unclear as to what version of S/4HANA was implemented. SAP is completely misleading about how different S/4HANA Cloud is from the on-premises version. S/4HANA Cloud can be implemented quickly, but its scope is very small and given the licenses, which show a $3,000,000 on premises charge, these were on premises S/4HANA versions.

And this brings up a related question. Where is the license charge for HANA? HANA is a very expensive database, and S/4HANA can only run on HANA, so it appears that Forrester entirely left out the cost for HANA.

These previous two points must be taken together. Not only is the implementation versus license multiple completely deflated, but the license numbers are missing the most expensive component of the overall BOM, which is the HANA database.

Forrester has absolutely no fear in making these types of errors (or deliberate underestimations) knowing full well that at least some people will read the entire study.

Conclusion

This TCO/ROI study gets off on bad footing by having SAP completely control the sample. The sample is rather ridiculously small (3 companies) for what is supposed to be a product that many companies have implemented (according to SAP). Apparently, SAP was not willing to bring very many customers to the table, which of course is a major red flag.

Forrester’s study gets worse with nonsensical quotations and then completely falls apart showing its seams with the ridiculous average implementation time and the implementation costs that will never be experienced by any company implementing on-premises S/4HANA. And Forrester tops it off by excluding the cost of HANA, which would have added very significantly to the cost for each of the three customers that Forrester spoke to.

This study is a way for SAP and Forrester to say to readers “we think you are fools.”

For vendors, the study illustrates that Forrester is “open for business.” If you want to say your applications has an infinite ROI, or can be implemented without a database, Forrester is your go-to source to falsify a TCO/ROI study.

Something else to note, while Forrester was paid to make this fake study, Brightwork was not paid to analyze it and show how false it was. There is no vendor or consulting firm that will pay another entity to fact check their statements and lampoon them in public. Following a profit-maximizing model, one would never write an article like this. This is why most of the information providers that operate in enterprise software don’t do it. And it is why Forrester is so comfortable writing such a blatantly false article. Any experienced SAP consultant that reads this will see it is fake, but on LinkedIn, they will stay quiet as a mouse. Most of them work for organizations that also publish false information, and the only information they are allowed to release is information that is promotional of SAP.

And it is not only Forrester that is open for business it is ComputerWeekly, Forbes etc…, The enterprise software market operates in a way such that the entities that provide the most inaccurate information receive the bulk of the income available in the market.

References

https://www.sap.com/documents/2018/07/7eb7617f-117d-0010-87a3-c30de2ffd8ff.html

https://www.techrepublic.com/article/how-a-troubled-sap-s4hana-migration-caused-a-gummy-bear-shortage-in-germany/

Brightwork Disclosure

Financial Bias Disclosure

This article and no other article on the Brightwork website is paid for by a software vendor, including Oracle and SAP. Brightwork does offer competitive intelligence work to vendors as part of its business, but no published research or articles are written with any financial consideration. As part of Brightwork’s commitment to publishing independent, unbiased research, the company’s business model is driven by consulting services; no paid media placements are accepted.

HANA & S/4HANA Question Box

  • Have Questions About S/4HANA & HANA?

    It is difficult for most companies to make improvements in S/4HANA and HANA without outside advice. And it is close to impossible to get honest S/4HANA and HANA advice from large consulting companies. We offer remote unbiased multi-dimension S/4HANA and HANA support.

    Just fill out the form below and we'll be in touch.

TCO Book

 

TCO3

Enterprise Software TCO: Calculating and Using Total Cost of Ownership for Decision Making

Getting to the Detail of TCO

One aspect of making a software purchasing decision is to compare the Total Cost of Ownership, or TCO, of the applications under consideration: what will the software cost you over its lifespan? But most companies don’t understand what dollar amounts to include in the TCO analysis or where to source these figures, or, if using TCO studies produced by consulting and IT analyst firms, how the TCO amounts were calculated and how to compare TCO across applications.

The Mechanics of TCO

Not only will this book help you appreciate the mechanics of TCO, but you will also gain insight as to the importance of TCO and understand how to strip away the biases and outside influences to make a real TCO comparison between applications.
By reading this book you will:
  • Understand why you need to look at TCO and not just ROI when making your purchasing decision.
  • Discover how an application, which at first glance may seem inexpensive when compared to its competition, could end up being more costly in the long run.
  • Gain an in-depth understanding of the cost, categories to include in an accurate and complete TCO analysis.
  • Learn why ERP systems are not a significant investment, based on their TCO.
  • Find out how to recognize and avoid superficial, incomplete or incorrect TCO analyses that could negatively impact your software purchase decision.
  • Appreciate the importance and cost-effectiveness of a TCO audit.
  • Learn how SCM Focus can provide you with unbiased and well-researched TCO analyses to assist you in your software selection.
Chapters
  • Chapter 1:  Introduction
  • Chapter 2:  The Basics of TCO
  • Chapter 3:  The State of Enterprise TCO
  • Chapter 4:  ERP: The Multi-Billion Dollar TCO Analysis Failure
  • Chapter 5:  The TCO Method Used by Software Decisions
  • Chapter 6:  Using TCO for Better Decision Making