- SAP proposed that the SD benchmark is no longer applicable because the way that customers used SD has changed.
- We review the accuracy of this claim.
SAP’s stance on SD benchmarks is curious. In this article, we will review the history and accuracy of SAP’s claim regarding benchmark applicability.
See our references for this article and related articles at this link.
The Position of SAP on its SD Benchmarks
“Hey guys keep your calm. let’s look at the benchmark issue first.
the current sd application of the sap erp suites reads tables without projection (didn’t matter in the past), maintains multiple indices (some via db, some as redundant tables), maintains materialized aggregates to achieve a decent response time for oltp reporting and still has some joining of tables trough loops in abap. all this is bad for a columnar in memory db. the current select ‘single’ is 1,4 times slower for a normal projection
(equal for a projection with one attribute, significantly slower for a projection of all attributes of a table with hundreds of attributes. the oltp applications have a large amount of supposedly high speed queries and transactional reporting. some of this had to be moved in the past to the bw for performance reasons. also planning activities should be part of the transactional scope, just think about the daily delivery planning.” – Hasso Plattner
Hasso’s English is a bit rough, and while he is endorsing a particular statement here, the quotation from John Appleby is more understandable. (Appleby did not work for SAP, at the time this article was published, but he faithfully represented SAP’s position as Appleby lead a HANA consulting group at Bluefin and was a shill for SAP.)
“I’ve not run the benchmark but I believe it’s because:
1) SD doesn’t run well on HANA
2) SD doesn’t accurately represent how customers actually use systems in 2014
3) HANA does run well for how customers really use systems in 2014
SAP are in the process of creating a new benchmark which I understand will include mixed-workload OLTAP queries.
The BW-EML benchmark was designed to take into account the changing direction of data warehouses – a move towards more real-time data, and ad-hoc reporting capabilities.”– John Appleby
This quote is analyzed in more detail in the article How Accurate Was John Appleby on SAP BW-EML Benchmark?
The Very Serious Logical Problems with SAP’s Logic
Even though the vast majority of ERP instances are not on HANA (SAP claims 2000 live customers on S/4HANA, which, even if true, would still be a tiny percentage of the total ECC live customers), the SD benchmarks are no longer representative of how companies use SD. This statement was made as far back as Nov 2013 by Hasso Plattner, as we covered in the article John Appleby, Beaten by Chris Eaton in Debate and Required Saving by Hasso Plattner.
However, when we check the SD benchmarks, there is not a single HANA benchmark for SD, and we are now up to 2019.
Also, according to SAP, the appropriate benchmark for SD is the BW-EML/BWAML, which is an analytics benchmark. So, somewhere around when HANA was introduced, SD switched from being primarily a transaction processing module to being an analytics module? That is curious how a module can change its basic processing type retroactively. It’s almost like this entire proposal was made up out of thin air.
This is true even if SD on ECC has no analytics other than opening tables in SE16(n)?
Talk about a story that does not hold together.
SAP’s contention around the inapplicability of the SD benchmarks is false. It was done very obviously to create an excuse to keep HANA from competing with other databases and to hide the fact that HANA does not perform well for transaction processing, as we covered in the article HANA as a Mismatch for S/4HANA and ERP.