SAP HANA False Claims

The Results of the Brightwork SAP HANA Survey

Executive Summary

  • This is the publication of the results of our survey on SAP HANA.
  • These results are quite contrary to what both SAP and SAP’s consulting partners said about HANA.

Introduction

We ran a poll on the Brightwork website, where we asked detailed questions of those that had experiences with HANA. We are publishing this poll now because we have 100 responses to the survey.

See our references for this article and related articles at this link.

The Maintenance Overhead of HANA Versus Oracle or DB2?

Consistent with our research into HANA, the maintenance overhead is rated as higher by roughly 1/2 the respondents. But we did not expect any respondents to say it was lower than Oracle or DB2, primarily because it is indisputable that HANA is far less mature than either Oracle or DB2. 

HANA and Transaction Processing Performance

The result of 35% of the respondents replying that HANA is faster than Oracle or DB2 is not possible. This is covered in the article Why HANA Has Problems with Transaction Processing, John Appleby on HANA Not Working Fast for OLTP, and Issues with SD HANA Benchmarks. This is because SAP only added row oriented tables (which are better for TP) to HANA after it was first designed.

SAP, of course, promised enormous speed improvements, but it is not clear from the responses.

But then again, 35% of the respondents that stated it runs faster are incorrect. We know this because the design of HANA and because of SAP’s unwillingness to allow for any competitive benchmarking, and the fact that they created a new benchmark, the BW EML benchmark, as we cover in the article John Appleby on SAP BW EML Benchmark after HANA failed to perform on the previous transaction processing benchmark (called the SD benchmark).

This makes me wonder how many respondents are themselves HANA consultants trying to promote HANA (hence the problem with reviews on G2Crowd, which is overflowing with consultant reviews.)

HANA Meeting Promises

This answer is amusing. How could HANA have possibly met the promises of SAP that 41% of the respondents stated that it did? Bill McDermott said that HANA ran 100,000 times faster than any competing technology. Steve Lucas stated that HANA would improve worker productivity by an average of 5,000 times! 

I have reviewed the promises made about HANA to many customers at this point, and the promises are so absurd, there is no way for them to be met.

Of the many promises made, there is one promise that could be met. If HANA was used for a pure analytics application, it could outperform previous versions of Oracle or DB2 on old hardware. If that were the only promise that 41% of the respondents replied “Yes” measured, then it would be possible.

Fifty-nine of the 100 respondents stated that HANA did not meet the promises.

Warning Companies About HANA?

Only 13% of the respondents recall if HANA not meeting its claims was ever discussed! However, 59% reported it did not meet its claims after the implementation.

This is quite standard in the SAP space, where both SAP and the SAP consulting partners mislead SAP customers in a unified manner. Gartner and Forrester would also be useless in warning customers, as they are both paid by SAP. Gartner published false information about HANA as we cover in the article How Gartner Got HANA So Wrong.

Will Your Company Expand the Use of HANA?

The standard modality of HANA implementation is that we have tracked is that it tends to slow quickly after the first implementation. However, 1/2 of the respondents state that HANA will be expanded in use.

Again, as at least 35 of the respondents either deliberately falsified their answer on question one (or answered without knowing), and are therefore unreliable, therefore this 50% value is quite overstated.

Did SAP Pressure You Fill Out the Survey?

No one felt pressured by SAP to fill out the survey. Again, we are trying to get realistic and not rigged answers to the survey. 

Conclusion

The survey results were interesting. But an impossible answer to one of the questions shows the bias in the survey results. For those that work in HANA, they want to promote HANA and would like it if the survey were inaccurate and positively biased.

There is no doubt that this survey has been positively biased in favor of HANA. Survey bias is a common issue. Political polls, for example, are biased by the method used, which often results in a non-representative sample being used. Confirming identity is more natural in person, but far more expensive. Online polls are easier to administer, but the very individuals that visit the site (and therefore take the survey) means that there is a bias in the sample. A very desirable online method is to send the poll to specific people. But only a small fraction of people sent a survey will fill out the questionnaire, unless there is a strong relationship with the sender or sending organization.

Overall, there has been close to little attempt to get accurate information about HANA to customers or the market. IBM belatedly pushed back on HANA’s claims as we cover in the article IBM Finally Fighting Back Against HANA, and individually John Appleby, Beaten by Chris Eaton in Debate and Required Saving by Hasso Plattner. But competitors have been restricted in critiquing HANA because each of them is partners with SAP. Overall, SAP’s powerful marketing machine has pushed inaccurate information about HANA into the market.