- Forrester created another TCO/ROI study that was paid for by SAP.
- We review the accuracy of this Forrester SAP S4HANA study.
Forrester has a history of taking money from SAP and then producing exactly the outcome results that SAP would have wanted. In this article, we will see how Forrester did with its analysis of S/4HANA.
Forrester Gets to Talk to Three Companies?
According to Bill McDermott S/4HANA is the fastest growing application in the history of SAP. So why did SAP only make 3 customers available to Forrester? It does not take very much time to interview customers, as SAP was paying for the study, one would think they would make an effort to bring more customers to the table.
Forrester found the following financial benefits.
Three Year Benefits
These look interesting until you see how these numbers were determined. The logic for these numbers do not make sense, as will see from the quotations. Forrester did not find any of these cost savings, they are all fake.
Quotations from the Study
Let us look through some of the most impossible quotes from the article. These quotes were apparently stated to Forrester from executives in the companies provided to Forrester by SAP and Forrester did not bat an eye.
A potential reason for this is that a) Forrester is being paid here to look the other way and b) it’s very rare for anyone who works at Forrester to have any implementation experience.
But to a person with implementation experience, these quotes make little sense.
Complex Supply Chains Lead to Lack of Inventory Visibility?
“Our supply chain from extraction to market is exceedingly complicated. We have over 157 different avenues for how we go to market. By the numbers, we have 60,000 unique materials, 6,000 vendors, 4,000 customers, and an additional 10,000 partners.”
S/4HANA’s supply chain functionality still lags ECC by a substantial margin. Secondly, a complex supply chain should not impact inventory management. Each one of these locations, no matter how many, is simple setup as a plant material combination with a connection to another plant material combination. I have personally had many customers with the numbers listed above, so it is not at all surprising or more complicated than what many other companies deal with, and deal with using ECC. It is a simple goods issue and goods receipt in ECC. If this quotation were from 5 years ago, ECC would be put forward as what solved this issue. Pre-S/4HANA if you were to have asked SAP if they can handle
“157 different avenues for how we go to market. By the numbers, we have 60,000 unique materials, 6,000 vendors, 4,000 customers, and an additional 10,000 partners.”
Their answer would have been that ECC is running the supply chains of the largest companies in the world, so the scenario above is nothing they have not done. The study is not clear that the company moved from ECC, they may have moved from another ERP system. But this study is trying to make the burning platform to move to S/4HANA when almost all the large supply chains using SAP in the world presently are running ECC.
S/4HANA Provides Real-Time Inventories?
“The most important change is that we now know the real inventories at our mine locations and operating plants. This gives us visibility from the extraction point to the final shipping to buyers. We can now measure real-time inventory across the entire company.”
All ERP systems have immediate visibility to inventory and have had this for many years now. The term real-time inventory is used, however, systems stopped advertising “perpetual inventory” many years ago as all systems that deal with inventories have this. That is it ceased to be a differentiator, where at one time it was. In fact, perpetual or real-time inventory was one of the original selling points of purchasing MRP systems back in the 1980s. We cover this in the article Whatever Happened to the Perpetual Inventory.
Note to executives, please stop saying things that reflect information technology crazy mouth, like your new system provides real-time inventory. Part of the entire logic for why you are paid so much more than employees is based upon the concept that you know more than the average person. When you say things like this, it ruins the illusion.
SAP and compliant customer executives have a habit of proposing that a new item newly solves a problem that was already supposed to have been solved with a previous version. If automobile manufacturers followed SAP’s lead in this regard, automobile ads might look something like the above.
Sales People No Longer Losing Deals Due to Faulty ATP Checks?
“Our salespeople were losing deals because we could not promise when the products would be delivered. We just didn’t have visibility into our supply chain to be able to make commitments when our competitors could and did make promises. A seemingly small system problem became a competitive disadvantage.”
It is difficult to see how the situation with S/4HANA will be much different. S/4HANA does not have superior ATP functionality over ECC. is this company having a problem with ATP because of functionality or because of other reasons like their production schedule changes too often? There can be many reasons why the available quantity can be incorrect. However, this is too much nuance and Forrester has a preconceived conclusion they need to reach, so lets move on.
All Companies Benefited from Reduced Operating Costs and Inventory Carrying Costs?
“Each of the companies that Forrester interviewed reported significant improvements in managing operating costs, such as inventory carrying costs. Using SAP S/4HANA enabled the composite organization to combine cost data with inventory data in a way that enabled employees to optimize flow and cost of inventory.”
This might be a good time for Forrester to re-emphasize that SAP did not allow Forrester to speak to any customers who bombed on their S/4HANA implementations, or even to the average S/4HANA customer. But our research supports the fact that most S/4HANA implementations (as covered in the Implementation Study of S/4HANA) have been unsuccessful. That is the failure rate is far higher than 50%. This failure rate is currently covered up by both SAP and the large SAP consulting firms. In fact, SAP also uses an NDA for its customers that implement S/4HANA, which means that no matter bad the experience, customers can’t talk about it. Forrester did not see fit to report on the S/4HANA NDA.
This is the problem with Forrester posing as doing research while its entire sample is unrepresentative of the population. What is the value of a completely rigged sample designed to lead to a predetermined outcome? The answer is zero…or perhaps negative.
The Company Reduced the Need for 600 Employees….And No One Was Fired?
“The financial model is based on a composite organization that was able to reduce the need for 600 employees. Although these staff will produce additional revenue in the future, the immediate benefit is measured by the reduction in employees required to do the same work. The future value is discussed in the Flexibility section.”
This is the equivalent to having your cake and eating it too.
Cost Savings…..But With No Cost Savings!
It is highly unlikely that a company would make 600 employees redundant, but keep them on staff. Companies today are simply too bottom-line oriented. And every single one of these people is a good fit to be moved to a new job?
Hmmmmmm…… Some of those employees, if not most of them would have been fired. However, this is a way that a company can claim cost savings, while not admitting that they fired people. This is political language that circumvents the discussion of what really happens to employees that are supposedly reduced by systems. However, this is also quite overstated. None of the projections around the labor savings of ERP systems turned out to be true — what did happen is companies were required to staff up IT.
Notice the following quotation as to the financial benefit of the moved employees.
Massive Revenue Unrealized Because of Lack of Employees
“By centralizing planning, production, and tracking functions, we eliminated the need for 600 employees. We redeployed those people into new locations to open new mines and refineries. It’s still too early to claim any revenue, but within a couple of years, we expect that these same employees will drive growth that will be hundreds of millions of dollars.”
Let us assume a future revenue of $250 million (they don’t specify how many hundreds of millions, only that it is plural, so $250 million would be conservative). Divided by 600 employees this is $416,000 per employee. if this type of opportunities were available to the company, they now look incompetent to have not hired these people before the S/4HANA implementation freed up 600 people. Let us review what we know about this mining company.
- They don’t understand that their inventory system has been real time for decades.
- They allow investment opportunities to wither on the vine, instead of hiring people to drive hundreds of millions in revenue.
How badly run is this mining company?
S/4HANA Took Nine Months to Implement?
“The composite organization no longer needed to pay the annual subscription costs or maintenance fees for older applications. The company executives told Forrester that it took an average of nine months to implement SAP S/4HANA.”
Naturally, the implementation times reported are far less than the implementation for ECC, which is a one-year minimum, but often has implementations stretch to 2 or 3 years or more. This is true even though S/4HANA is far less mature than ECC, and immature products take longer than mature products to implement. Every single SAP reported implementation duration is shockingly fast.
However, when information that SAP does not control is reported, the timeline is quite long. In the case of the Lidl failure, it was a total of 7 years, with S/4HANA being a subcomponent of that as they transitioned to S/4HANA after beginning with ECC. Haribo, another failure was also 7 years. Yet the companies interviewed by Forrester averaged 9 months? If we include the implementation duration of Lidl and Haribo, what happens to the average of Forrester’s sample? Forrester did not even have to interview these companies as the duration and cost was published due to the projects failing.
Real Implementation Data Points Keep Rolling In that Contradict Forrester
It was later learned that the Schweizer Post S/4HANA implementation rose to $84 million.
Wait a second, why did this implementation cost even more than $1 million? According to Forrester, a typical S/4HANA implementation is around $877,000 and runs around 9 months with only two consultants who stay on the project for 6 months. Forrester’s study of SAP provided sources showed almost no variance in this.
Now to hear of a $83 million implementation is very distressing. The 12X cost overrun seems odd, as this amounts to $83M/.87M 95X the common cost of and S/4HANA implementation according to Forrester. Actually, this cost will rise in the future. If we include just this implementation into the Forrester “study” we end up with (3 * $.877M + $84M) or $21M. Could there be some reason that the Schweizer Post case study was not provided to Forrester by SAP? I mean isn’t SAP interested in providing the full context of data points to Forrester? Its a study right???
It is curious. We keep teaching statistics and talking about the importance of math, but we don’t seem to have research entities where you can be employed to use this math honestly. Forrester can publish a study with 3 rigged case studies, and it goes with little notice. Math + lying does not lead anywhere.
Sales Orders are Entered in Manually?
“We design complex systems for our customers, but then the information must be converted into orders and placed into SAP. Today this is a manual process, but we are looking into the idea of having the information convert directly over, which will reduce delays and avoid errors.”
This company has not yet figured out how to upload externally generated sales orders to S/4HANA? Interesting.
These implementation costs are ridiculous. ECC and S/4HANA implementations take many months of consulting support. In this case, the consultants rolled off three months before the go-live (or they were part-time).
Immature ERP System Implemented with Two “Swiss Army Knife” Consultants
S/4HANA is a risky implementation because so much of it is still immature. If only two consultants were used, how would that have worked? A minimum is to have a consultant for materials, finance/accounting, and sales. (it is unclear if production was implemented).
Which of the two consultants covered two areas?
And further, who covered HANA? S/4HANA must be implemented with HANA, a high overhead database. S/4HANA consultants are not database consultants. So one consultant implemented S/4HANA and the other consultant implemented HANA?
It also means that there could not have been any variability in the number of consultants. Unless one project had one S/4HANA/HANA consultant, and another project had three.
- This means that one consultant implemented HANA while the other implemented S/4HANA? Or HANA took 3 months of consulting time to implement and S/4HANA took 12 months of consulting time (summing to 9 * 2 = 18 total months of consulting)(that is for some of the time there were 2 S/4HANA consultants and sometimes not?)
- So those two consultants are “Swiss Armied” to the extreme degree. And, they were able to run this lean on three different projects with three different sets of consultants?
Unrealistic Implementation Durations…..from Forrester
Every report by Forrester highly understates the implementation costs to customers. There are some clients with 70 to 100 SAP consultants on site (for multiple products), and with immense monthly billings, but every client Forrester seems to interview has a tiny amount of billings. However, if you approach Deloitte and ask them to help you implement S/4HANA with two consultants for six months, they will laugh at you.
For this claim, Forrester wins our highly coveted Golden Pinocchio Award. However as the lie was funded by SAP, and SAP knowingly provided the faulty information and would have manipulated the customers into providing the false information, the award is co-shared with SAP.
This estimate is even less believable because it is an average, which means that these minimum values were the average of the three companies. That is this is not from a single company that had some very small scope.
We can say with high confidence that this did not happen.
Either the 3 companies were instructed to lie to Forrester or Forrester changed these values as directed by SAP.
Inverted Implementation Versus License Ratio?
Notice the license/subscriptions costs that serve as the major part of the costs for S/4HANA. Something that should be a major red flag is that the license costs are far below the implementation costs.
In fact the ratio of implementation costs to license costs is $833,750 / $3,000,000 or 27%. That is never the case, the implementation expense on SAP projects is a multiple of the license cost, not a fraction of the license revenue. Even an SAP sales rep with a few months under the belt knows this (it comes up during discussions around implementation estimates for customers). And the fraction here is far less than 1. If multiple of 2 were used, that would mean that all three customers in the study 2/.27 = .135 or 13.5% of a typical implementation. Two is the number that many consulting companies use in the sales process, but it is far too low. Readers can see for themselves at the Brightwork ECC TCO Calculator (this is for less than 800 users, see the included link for more than 800 users. But even a more simple ECC implementation for say 500 users will run up more than $20 million in consulting fees.
However, SAP makes most of its money from licenses and support, so the unrealistic consulting costs are not SAP’s problem, they are Deloitte and Accenture, etc…’s expectations to manage.
Forrester is Still Learning About Implementation Versus License Revenues
Is Forrester serious unaware of the implementation/license multiple that is demonstrated through decades of SAP implementation experience? Let us look at the Lidl example again, that was 500 million Euro. ECC implementations are routinely $100 million and often much more. SAP even has multi-billion dollar failures on its resume. The Oracle ERP “legacy replacement” (it is well known that Oracle ERP is less in implementation costs), with the Airforce, cost $500 million before the program was stopped. Sub million dollar ERP implementations from commercial vendors in general, but particularly with SAP do not exist. Forrester seems unconcerned that other people will also read this study and see how impossibly inaccurate it is.
And the errors get perhaps (even more?) or just as extreme with the next calculation for license costs.
Where are the Costs for HANA?
Forrester is quite unclear as to what version of S/4HANA was implemented. SAP is completely misleading about how different S/4HANA Cloud is from the on-premises version. S/4HANA Cloud can be implemented quickly, but its scope is very small and given the licenses, which show a $3,000,000 on premises charge, these were on premises S/4HANA versions.
And this brings up a related question. Where is the license charge for HANA? HANA is a very expensive database, and S/4HANA can only run on HANA, so it appears that Forrester entirely left out the cost for HANA.
These previous two points must be taken together. Not only is the implementation versus license multiple completely deflated, but the license numbers are missing the most expensive component of the overall BOM, which is the HANA database.
Forrester has absolutely no fear in making these types of errors (or deliberate underestimations) knowing full well that at least some people will read the entire study.
Analyzing The Vendor or Vendor Funded TCO Estimates
Vendors view TCO discussions within the context of making a software sale. Some vendors still produce TCO calculations and we have analyzed a number of these TCO calculations. In fact, we have used some of these estimations for our own estimations and have found some good and some bad in how they were put together.
Several of the most obvious issues are listed below:
No Complete or Full TCO
We have yet to come across a true “TCO” produced by a vendor or of course any vendor-sponsored study (which we view as the one and the same). While a few have been accurate in several respects, they did not estimate the client’s cost of either working on the implementation or maintaining the application. However, the TCO is literally that; it must be inclusive of the entirety of costs to bring the application into the company and to keep it running, and even to decommission the application. I have certainly appreciated the work put in by software vendors to estimate some of the costs, but it is confusing to see “TCO” in a publication’s title and then read the following
“…does—NOT—include analyst costs, IT costs, local data management costs, etc.”
One could imagine the following riddle:
“When is a TCO not a TCO? When it does not include all costs.”
Incorrect Duration Estimation
Vendors have a strong tendency to underestimate the duration of the software implementation. For example in the published S/4HANA case studies, the numbers published by SAP were far shorter than the known duration of ECC implementations, even though ECC is far more mature than S/4HANA. Therefore, part of underestimating TCO is underestimating the time it takes to go live.
It may certainly be true that the application could be installed as quickly as estimated in a laboratory environment, but companies are not laboratories. Some vendors (other than those like SAP and Oracle, both of which have business models of long implementations due to the needs of the consulting companies to which they outsource their consulting), will complain that the client is really what holds up the implementation. This is a bit like a doctor complaining that if only patients had medical degrees, he or she could see many more patients and get through the day faster. It’s simply not relevant to the question of duration estimation. The following statement is included on the TCO questionnaire that we give to vendors:
“Please use historical timelines rather than ‘optimal scenario’ timelines.”
Therefore, the knowledge level of the client and the time required for knowledge transfer must be estimated, along with all of the other realities of projects. Another example of this is integration. IT departments are slow to respond with extract creation.
Software Vendors Faking Enthusiasm for TCO
While most software vendors hide their pricing information, all software vendors still like to talk about TCO in the abstract. Being in favor of TCO is like saying you are in favor of the American flag or apple pie; you’re in favor of goodness. You can’t actually find anyone who opposes TCO in principle. But the devil is in the details. As long as nothing is quantified or the TCO studies are rigged, all vendors can say they have the lowest TCO. Most software vendors are not only against publishing TCO estimations but are even against publishing pricing information for their applications on their websites.
It is certainly not difficult to find examples of software vendors creating false financial estimates for their clients in order to promote sales. I used to work for one of these companies and witnessed this firsthand (although I am happy to say I never participated in or supported this). The software vendor i2 Technologies offered something called a Strategic Opportunity Assessment (SOW), which estimated the financial implications of using its software.
“Another company, whose i2 deployment was two years behind schedule because of problems with i2 software, told Nucleus the following I2 did an ROI calculation for us during the sales process it was very positive. But based upon our experience now, we will [independently] evaluate ROI in the future.” – Nucleus Research
How any company could actually believe a TCO or ROI study developed by a software vendor and which is part of a vendor’s sales process, is well, strange. In a surreal turn, this company also priced their software based upon this value estimation. Talk about a conflict of interest! The following quote from Nucleus Research is educational:
“One customer, that is still cited on the i2 Web site (2003) as a success story but stopped using i2 more than a year ago, told Nucleus:
‘What killed the ROI was the fact that i2 priced their modules based on an ROI calculation that they themselves estimated during the sales process. (i2 stated) that this would lead to $40 million in benefits. (emphasis added) There are definitely benefis to the solution, but their pricing is too high. Sometime, I should take the time to contact i2 and say you really need to look at your pricing.’”
I worked with people out of this group at i2 which was called “Strategic Services.” Strategic Services would falsify any data in order to obtain a sale. Strategic Services was filled with MBA from the top schools. They were not technologists, did not know the software except at a superficial level. Their job was to perform financial estimation for prospects that would lead them into purchasing i2 applications. This was called an SOA, or strategic opportunity assessment.
This is the value of TCO or ROI estimations from vendors or vendor-sponsored studies — zero or below zero if you include the damage of following the advice from such studies.
This TCO/ROI study gets off on bad footing by having SAP completely control the sample. The sample is rather ridiculously small (3 companies) for what is supposed to be a product that many companies have implemented (according to SAP). Apparently, SAP was not willing to bring very many customers to the table, which of course is a major red flag.
Forrester’s study gets worse with nonsensical quotations and then completely falls apart showing its seams with the ridiculous average implementation time and the implementation costs that will never be experienced by any company implementing on-premises S/4HANA. And Forrester tops it off by excluding the cost of HANA, which would have added very significantly to the cost for each of the three customers that Forrester spoke to.
This study is a way for SAP and Forrester to say to readers “we think you are fools.”
For vendors, the study illustrates that Forrester is “open for business.” If you want to say your applications has an infinite ROI, or can be implemented without a database, Forrester is your go-to source to falsify a TCO/ROI study.
Something else to note, while Forrester was paid to make this fake study, Brightwork was not paid to analyze it and show how false it was. There is no vendor or consulting firm that will pay another entity to fact check their statements and lampoon them in public. Following a profit-maximizing model, one would never write an article like this. This is why most of the information providers that operate in enterprise software don’t do it. And it is why Forrester is so comfortable writing such a blatantly false article. Any experienced SAP consultant that reads this will see it is fake, but on LinkedIn, they will stay quiet as a mouse. Most of them work for organizations that also publish false information, and the only information they are allowed to release is information that is promotional of SAP.
And it is not only Forrester that is open for business it is ComputerWeekly, Forbes etc…, The enterprise software market operates in a way such that the entities that provide the most inaccurate information receive the bulk of the income available in the market.
The Problem: A Lack of Fact-Checking of S/4HANA
There are two fundamental problems around S/4HANA. The first is the exaggeration of S/4HANA, which means that companies that purchased S/4HANA end up getting far less than they were promised. The second is that the SAP consulting companies simply repeat whatever SAP says. This means that on virtually all accounts there is no independent entity that can contradict statements by SAP.
Being Part of the Solution: What to Do About S/4HANA
We can provide feedback from multiple HANA accounts that provide realistic information around S/4HANA — and this reduces the dependence on biased entities like SAP and all of the large SAP consulting firms that parrot what SAP says. We offer fact-checking services that are entirely research-based and that can stop inaccurate information dead in its tracks. SAP and the consulting firms rely on providing information without any fact-checking entity to contradict the information they provide. This is how companies end up paying for a database which is exorbitantly priced, exorbitantly expensive to implement and exorbitantly expensive to maintain. When SAP or their consulting firm are asked to explain these discrepancies, we have found that they further lie to the customer/client and often turn the issue around on the account, as we covered in the article How SAP Will Gaslight You When Their Software Does Not Work as Promised.
If you need independent advice and fact-checking that is outside of the SAP and SAP consulting system, reach out to us with the form below or with the messenger to the bottom right of the page.
Inaccurate Messaging on HANA as Communicated in SAP Consulting Firm Videos
For those interested in the accuracy level of information communicated by consulting firms on HANA, see our analysis of the following video by IBM. SAP consulting firms are unreliable sources of information about SAP and primarily serve to simply repeat what SAP says, without any concern for accuracy. The lying in this video is brazen and shows that as a matter of normal course, the consulting firms are happy to provide false information around SAP.
SAP Video Accuracy Measurement
|SAP's Statement||Link to Analysis Article|
|HANA runs more "in-memory" than other databases.||HANA uses a lot of memory, but the entire database is not loaded into memory.||How to Understand the In-Memory Myth|
|HANA is orders of magnitude faster than other databases.||Our research shows that not only can competing databases do more than HANA, but they are also a better fit for ERP systems.||How to Understand the Mismatch Between HANA and S/4HANA and ECC.|
|HANA runs faster because it does not use disks like other databases.||Other databases also use SSDs in addition to disk.||Why Did SAP Pivot the Explanation of HANA In Memory?|
|HANA holds "business data" and "UX data" and "mobile data" and "machine learning data" and "IoT data."||HANA is not a unifying database. HANA is only a database that supports a particular application, it is not for supporting data lakes.|
|SRM and CRM are part of S/4HANA.||SRM and CRM are not part of S/4HANA. They are separate and separately sold applications. SAP C/4HANA is not yet ready for sale.||How Accurate Was Bluefin Solutions on C-4HANA?|
|Netweaver is critical as a platform and is related to HANA.||Netweaver is not relevant for this discussion. Secondly Netweaver is not an efficient environment from which to develop.|
|HANA works with Business Objects||It is very rare to even hear about HANA and Business Objects. There are few Buisness Objects implementations that use HANA.||SAP Business Objects Rating|
|Leonardo is an important application on SAP accounts.||Leonardo is dead, therefore its discussion here is both misleading and irrelevant.||Our 2019 Observation: SAP Leonardo is Dead|
|IBM Watson is an important application on SAP accounts.||Watson is dead, therefore its discussion here is both misleading and irrelevant.||How IBM is Distracting from the Watson Failure to Sell More AI and Machine Learning|
|Digital Boardroom is an important application on SAP accounts.||SAP Digital Boardroom is another SAP item that has never been implemented many places.|
Financial Bias Disclosure
Neither this article nor any other article on the Brightwork website is paid for by a software vendor, including Oracle, SAP or their competitors. As part of our commitment to publishing independent, unbiased research; no paid media placements, commissions or incentives of any nature are allowed.
S/4HANA Implementation Research
We offer the most accurate and detailed research into S/4HANA and its implementation history. It is information not available anywhere else and is critical correctly interpreting S/4HANA, as well as moderating against massive amounts of inaccurate information pushed by SAP and their financially biased consulting ecosystem.
Select the description that best matches you.
Option #1: Do You Work in Sales for a Vendor?
See this link for an explanation to sales teams.
Option #2: Do You Work for an Investment Entity that Covers SAP?
See this link for an explanation for investment entities.
Option #3: Are You a Buyer Evaluating S/4HANA?
For companies evaluating S/4HANA for purchase. See this link for an explanation to software buyers.
Search Our Other S/4HANA Content
Getting to the Detail of TCO
The Mechanics of TCO
- Understand why you need to look at TCO and not just ROI when making your purchasing decision.
- Discover how an application, which at first glance may seem inexpensive when compared to its competition, could end up being more costly in the long run.
- Gain an in-depth understanding of the cost, categories to include in an accurate and complete TCO analysis.
- Learn why ERP systems are not a significant investment, based on their TCO.
- Find out how to recognize and avoid superficial, incomplete or incorrect TCO analyses that could negatively impact your software purchase decision.
- Appreciate the importance and cost-effectiveness of a TCO audit.
- Learn how SCM Focus can provide you with unbiased and well-researched TCO analyses to assist you in your software selection.
- Chapter 1: Introduction
- Chapter 2: The Basics of TCO
- Chapter 3: The State of Enterprise TCO
- Chapter 4: ERP: The Multi-Billion Dollar TCO Analysis Failure
- Chapter 5: The TCO Method Used by Software Decisions
- Chapter 6: Using TCO for Better Decision Making