How Accurate is Seeking Alpha on the Relative Decline of SQL?

Table of Contents: Select a Link to be Taken to That Section

Executive Summary

Seeking Alpha proposed interesting things regarding SQL vs. NoSQL.
The article covered commercial database growth and questioned whether commercial SQL databases had hit a wall of diminishing utility.
Seeking Alpha’s prediction for NoSQL is, in part, due to the advancements in hardware that allow data to be stored at a lower level of normalization.
The article also points out the marginalization of mission-critical application databases, asks the question of the costs of Oracle and DB2.

Introduction to SQL Versus NoSQL

Significant changes are occurring in the database market. This article will cover an exciting proposal made in an article regarding these changes from Seeking Alpha.

SaaS Vendors and Open Source Databases

“The vast majority of SaaS providers today either use an open-source database, or, as is the case with SaaS HCM vendor Workday (NYSE:WDAY), develop their own. Every user of an on-premises enterprise application, including one of the five core client-server application categories: ERP (Enterprise Resource Planning), CRM (Customer Relationship Management), HCM (Human Capital Management), SCM (Supply Chain Management), and BI (Business Intelligence) applications, which moves to SaaS therefore eliminates a commercial database seat, and with it, the maintenance/support and future upgrade revenue it would have generated. Even an enterprise seat that moves to Salesforce.com will generate far less revenue for Oracle than that seat did when it was deployed on-premises.”

This is a fantastic point. If you use SaaS, the database question is taken out of the equation. A true multi-tenant database means that there are fewer databases used. Therefore, even if Oracle is used (and Oracle 12c has added much multi-tenant functionality), fewer instances of Oracle 12c are required.

Commercial Database Growth?

“Hence, we believe the enterprise migration to SaaS/cloud is highly deflationary for commercial-database revenue. Further, we are still early in the migration from client-server to SaaS/cloud. Depending on which research one believes, this migration is only 10% to 25% complete and is only just beginning to impact more mission-critical applications – such as ERP – that are more likely to be running on higher-end SQL databases (including those from Oracle and IBM, and to a lesser extent, Microsoft).”

True, as SaaS picks up speed, this issue becomes a stopping point in growth for commercial databases. This hits Oracle and IBM the hardest.

Commercial SQL Databases Hitting Diminishing Utility?

“Commercial SQL Databases are Not Well-Suited to Handle the Most Compelling Emerging Use Cases. After SQL was standardized in 1987, it went unchallenged for over 20 years in defining how data was organized, searched, and sorted. However, in the mid-2000s, colossal tech companies began running into the limitations of SQL databases in handling both the sheer volume and varying structure (or lack thereof) of the data that they wanted to preserve, navigate, analyze, and serve up on-demand to their users/customers.”

This is true. The market share for SQL databases is declining as a percentage of all databases used.

“Amazon (NASDAQ:AMZN), Google (NASDAQ:GOOG) (NASDAQ:GOOGL), LinkedIn (LNKD), and Facebook (NASDAQ:FB) each solved its own scaling problems by developing and implementing its own database software that broke with the constraints of the SQL standard.”

Hard to argue with that.

The Decline of SQL and the Rise of NoSQL Due in Part to Hardware Upsizing

“NoSQL Databases and Schema-on-Read Processes Are on the Right Side of Moore’s Law; SQL and Schema-on-Write Are on the Wrong Side. In general, NoSQL databases are substantially more processor, memory, and storage intensive than SQL databases, even after considering the fact that they relax many of the constraints that SQL places on how data is organized. Though NoSQL database architectures existed well before 2005 – at least, in theory – there was simply not enough processing power, memory capacity, and storage space for them to be put to practical use outside of academia.”

I had not considered this before, but SQL databases require a lot of effort to organize them. But with more hardware, one can reduce this “energy input” for many applications of the database.

Hardware Restrictions Meant More SQL DBs?

“During the comparatively processor/memory/bandwidth-starved 1980s and 1990s, adherence to the more-rigid SQL standard was effectively compulsory to ensure the performance and reliability that enterprises needed in order to migrate their applications, especially those more mission-critical applications, from mainframes and minis to the more-distributed and network-dependent client-server architecture.”

This logically follows nicely.

The price paid for SQL’s acceptable performance and reliability was driven by:

“The overall inflexibility of the data structure called a schema;

The onerous requirement to define that structure prior to the deployment of the database and its associated applications;

The difficulty in changing that structure over time in order to capture different kinds of data, and to better reflect changes in the structure of the data and in the organization of the enterprise.

The requirement for a schema-on-write process, which fits data into the schema as it is inputted, as opposed to schema-on-read, which is simply dumping data into a big “container” and organizing it later into a schema.”

So basically a lot of overhead and work. This is the work of the database administrator.

No SQL Flexible Schema Benefits

“Over time, however, Moore’s Law-driven improvements in processing power, memory capacity and speed, storage capacity and speed, and network throughput have made the rigidity of the SQL standard and the requirement for schema-on-write processes increasingly less necessary and will continue to do so. Hence, with every Moore’s Law cycle, SQL and its resource-efficient schema-on-write process loses its competitive advantage, while NoSQL, with its relatively resource-inefficient but much more flexible schema-on-write process, becomes increasingly free from the constraints that prevented it from being adopted in the first place.”

Right, resource inefficiency becomes less of an issue with more computational resources, meaning human resources can decline.

“In-Mem Possibilities and the Elimination of HDD Compromises. When SQL was standardized, hard disk drives (HDDs) were the only cost-effective storage medium that could be accessed in real time. Therefore, much of the fundamental code written into SQL database software is designed to accommodate the shortcomings of HDDs, such as the slow speed with which they can read and transmit requested data into memory and their relatively higher failure rates – at least, compared to the major solid state components of a system such as the CPU, memory, and network throughput. Now that SSDs are rapidly becoming a cost-effective replacement for HDDs, much of the design and certainly much of the code of legacy SQL database software that made such compromises in order to accommodate comparatively much slower HDDs is now rendered unnecessary.”

This is a constant feature of computer systems. Early design decisions are made to fit with the hardware of the time. When hardware improves, the earlier design decisions no longer make sense.

Leveraging SSDs

“In contrast, many NoSQL databases are written so as to take maximum advantage of SSD storage and will likely be updated to take advantage of even faster-emerging non-volatile memory technologies such as Intel (NASDAQ:INTC)/Micron’s (NASDAQ:MU) 3D xPoint, which is just now becoming commercially available. We believe that, given the strong requirement to maintain compliance with the SQL standard and maintain their own backward compatibility, SQL database vendors will not be able to optimize their code for SSDs nearly as fast as many of the open source projects will be able to, putting them further at a competitive disadvantage – a classic example of Clayton Christensen’s “The innovator’s dilemma.”

This is a true insight. It means that SQL vendors are at a disadvantage in making the transition due to legacy designs. That is, designs developed for a different hardware era.

Marginalizing Mission Critical Application DBs

“Increasing Availability of “SQL Methadone” Solutions for Weaning Enterprises Off Expensive Commercial Databases. We are seeing an increasing number of software tools and services designed to help enterprises migrate from commercial SQL databases. Even the most mission-critical applications running on an Oracle or IBM database can be increasingly surrounded, “quarantined”, and dismantled over time, much like what happened to many of the legacy mainframe applications during the shift to the client-server architecture in the 1990s.”

This is a fascinating observation. This brings up many interesting questions as to how complicated or easy this process is. Also, for what uses is this currently being performed? What are the larger implications?

“While SQL databases from the likes of Oracle, IBM, and MSFT will survive in some enterprises for many more years – again, much like mainframes – they will be increasingly marginalized and cost-reduced as much as possible. We see many of these tools maturing already within the Hadoop ecosystem, which already has multiple ways to integrate with SQL databases and/or put SQL interfaces and querying ability on NoSQL databases. In our opinion, this is a very similar development to the early-1990s emergence of multiple ways to integrate mainframe applications with PCs and client-server applications.”

This brings up the question of whether Big Data applications and typical applications, such as production planning, accounting, etc. applications can also transition to NoSQL databases.

What is the Cost of Oracle or DB2?

“The main benefit of NoSQL is they are usually open source and either completely free, or relatively low cost since some support is usually purchased. Most people don’t realize how expensive Oracle and DB2 are, often costing a license fee of $1 million per server, plus $200,000 per year for ongoing support after that. Most enterprise applications need more than one server for fault tolerance, staging, QA, testing, development, etc (although licensing costs for non-production installations are often less than the production server). – From Mark A commenting on the article The Death of the Commerical Database Oracle’s Dilemma.”

This is, in part, driving the move away from NoSQL.

Future of Oracle Database Growth?

Seeking Alpha calls out what the shift in the market means for the biggest player in that market.

“We see the $29.6b commercial database market contracting 20-30% by 2021, and do not believe Oracle (NYSE:ORCL) can transition its revenue streams (from legacy commercial database to cloud-based subscription offerings) fast enough to offset the decline of this market, which represents a major legacy core of its revenue.”

That makes sense. Oracle has been proposing that its cloud revenues will more than make-up for any losses. Seeking Alpha also points out that IBM will also lose market share, but that IBM is far less dependent upon its database business for its overall revenues than Oracle.

Checking the Database Growth Figures with DB-Engines

This is an excellent article by Seeking Alpha. This author has an impressive insight into the topic of the material. Now the question is checking to see DB-Engines see if this is occurring in the database growth rates.

Of the top 30 databases, as ranked by DB-Engine, the following applies:

The Rate of Growth for SQL Versus NoSQL?

To test the hypothesis presented by Seeking Alpha, we took these top 30 databases (where most of growing movement happens in the database market, and we averaged the growth rate for the 18 SQL databases, and then the 12 NoSQL databases (which includes all of the database types listed in the previous graphic that are not called RDBMSs)

We came up with the following growth rates from July 2016 to 2017.

SQL Database Growth Rate: +3.4%
NoSQL Database Growth Rate: +8.6%

So NoSQL databases are indeed growing faster. But SQL databases are not declining. But what is happening is something a bit different. We then averaged the growth of the proprietary databases in the top 30 list and open-source. As of July 2017, there were 11 proprietary databases in 19 open source. The growth rates broke down as follows:

Proprietary Growth Rate: -1.2%
Open Source Growth Rate: +8.4%

Therefore, the growth is mostly attributed to open source databases, rather than to NoSQL databases. The NoSQL databases have a stronger tendency to be open source. And, of course, open-source databases cost less than proprietary.

Go to top

Conclusion

This is an excellent article by Seeking Alpha, with many great insights.

However, the central hypothesis that SQL databases are losing market share to NoSQL does not seem to bear out in the numbers from DB-Engine. What seems to be the case is that there is a shift to open source databases, and most of the NoSQL databases are open source.

References

https://seekingalpha.com/article/4044813-death-commercial-database-oracles-dilemma