Is It True That SAP Performs Tests Before Selling Software?

Executive Summary

  • SAP makes the claim that they support testing to prevent customers from buying software that is not a good fit.
  • We verify this claim for accuracy.

Introduction

SAP has a long history of forcing ill-fitting applications into their customers. This is enabled by SAP’s large compliant ecosystem that normally repeats whatever SAP says, and denigrates all other applications. However, Luca Massasso of BizBrain Technologies (and ex-SAP employee), a consulting firm focused on APO and IBP, states that this is not the case.

Setup

These comments we made on an article by Lora Cecere titled Where There is Smoke There is Fire. Luca stated that SAP was adopting Lora Cecere’s focus on testing before making purchases.

Luca’s Initial Comment on Lora’s article.

Hi Lora, I was part of the team that developed SAP Supply Network Collaboration and since 2013 I dedicate my time to the implementation of SAP IBP (before “S&OP on HANA”). One thing I can confirm to you: the vast majority of customers has changed habit and they are fully following your advice: “Test technology and ensure business results. And, in the process, trust but verify.” In these recent years we have been involved in more than 40 evaluations and sales cycles and pretty much all of them required a Proof-of-concept or a small pilot. This happened for customers of almost any size. Perhaps additionally, it would make sense to mention that in the test you recommend, not much emphasis goes into the aspect of modeling the integration of Supply Chain Planning and Supply Chain Execution. On this topic, I have seen but also heard (2nd hand) many horror stories of allegedly better solutions than SAP, where the integration of planning into SAP ECC was a failure. I think your point captures what we have so fare experienced in the IBP market, but I would additionally humbly recommend to look at the business process as a whole and understand that Operations team need a working end-to-end solution. Thanks for your articles, always a pleasure to read.

My Response

“This is an interesting attempt to embrace Lora’s message and perform damage control. Contrary to your statement, SAP entirely opposes Lora’s advice on testing. SAP habitually and as a strategy circumvents testing. If APO had ever been testing, almost no one would have purchased it. This is why SAP can’t really move to the cloud, the cloud is about testing, and SAP is about making based upon giving out goodies to executives, getting SAP consulting partners to lie about the SAP applications and false promises by SAP sales. It is the exact opposite of a testing approach. If you like testing, we have a large number of articles that test functionality at brightworkresearch.com/sapplanning/.

But you probably won’t want to read the test results. If the testing of APO were a movie, it would fall into the horror genre.  If SAP sees a deal moving away from them based upon testing, they will put pressure at the tippy top to get the executives to ignore the test results, to ignore the desire of the users. Unless you get to see how the sausage is made, it is difficult to appreciate how corrupt SAP is. SAP is all about the money, they will jam the least appropriate solutions into customers by any means necessary.”

Luca’s Response

“Your statements oversimplify the reality. I am in this business almost three decades and money is pervasive in so many dimensions including the ones that claim independence by statute. I respect different viewpoints but again reality had been and is that in my direct experience every company evaluating sap ibp has worked exercises to evaluate the software and put it to the test. For confidentiality reasons i cannot name them on a social platform, but they are among the largest sap customers and the most important companies in the world. The exercise led to conversation with SAP development on roadmaps where gaps were identified. In some cases IBP has been successfully deployed (microsoft is a public reference for example or Prestige Brands or Nature’s Way on smaller size organizations). Where i saw problems with the delivery was when the consulting force was not proficient on what the tool can’t or can do. There are plenty of examples where previous apo customers selected (or did not select) ibp based on poc/prototypes and pilots. We can look back at apo, but ignoring this and writing that sap sales cycles are not subject to testing is in my experience factually inaccurate and depending on statistics probably wrong.”

My Response

I only want to address the last part of your answer. So I want to distinguish what you consider testing and what I consider testing. Earlier you commented about POCs. This gives me the impression that you think a POC is a test. POCs do not meet my definition of a test. The test must be performed by an entity that does not care what the results of the test are. That is, they are indifferent. I have worked on pre-sales engagements and received heavy pressure from sales to rig the results to make the SAP software look it was a perfect fit for the requirements when it wasn’t. When we would get a POC, it was good because the client would pay, but sales entirely controlled the process. The pressure is to make the sale, not to show anything realistic. I don’t think other vendors work much differently in this regard, but SAP, because of its enormous influence and partner network leads all other vendors in being able to push their applications and databases into situations for which they are a poor fit.

SAP nor can any SAP consulting partner perform any test or POC. The reason is apparent, a test must have no bias for the test to succeed or to fail. So, under your definition of a test, tests are sometimes performed during the sales process (although no often). SAP and most other vendors will oppose any independent party being included in the process. I can communicate that I have extensive experience in testing solutions, and I run a research entity and I am constantly evaluating software and recording accuracy of things like SAP’s statements to clients. I would provide an entirely unbiased test result. I can guarantee you; I will never receive a call from SAP (or from Oracle or from IBM) to ever perform tests where the client pays. SAP, Oracle or IBM would actively fight my entry or the entry of any testing entity that is not controlled by them. Also, I can promise, any test I perform will not match with the statements made about sales to the prospect.

Conclusion

The following comment is Luca’s strongest point.

“We can look back at apo, but ignoring this and writing that sap sales cycles are not subject to testing is in my experience factually inaccurate and depending on statistics probably wrong.”

However, again, a POC run by SAP is not a test. Secondly the fact that APO does not win every single competition is not evidence that testing is performed prior to APO being purchased. And this generalizes far beyond APO. SAP applications are routinely purchased without an independent entity performing the tests. Biased SAP consultants and SAP like to stated that a POC that they control is a test, when it isn’t.

Financial Disclosure

Financial Bias Disclosure

Neither this article nor any other article on the Brightwork website is paid for by a software vendor, including Oracle, SAP or their competitors. As part of our commitment to publishing independent, unbiased research; no paid media placements, commissions or incentives of any nature are allowed.

Search Our Other SAP Content

SAP Contact Form

  • Interested in Our SAP Research?

    The software space is controlled by vendors, consulting firms and IT analysts who often provide self-serving and incorrect advice at the top rates.

    • We have a better track record of being correct than any of the well-known brands.
    • If this type of accuracy interests you, contact us and we will be in touch.

References

https://www.linkedin.com/feed/update/urn:li:article:8658798029287428681/?commentUrn=urn%3Ali%3Acomment%3A(article%3A8658798029287428681%2C6543793631919251456)&replyUrn=urn%3Ali%3Acomment%3A(article%3A8658798029287428681%2C6544519168807448577)