The DB-Engines Knowledge Base: A Handy Resource for Developers and Site-Building Teams Working with Database Systems

TL; DR: DB-Engines, a division of the consultancy company Solid IT, has published statistics and trends in database usage since 2012. The site, which reflects the growing diversity in the database landscape, provides regularly updated and meticulously calculated information. Developers and site-building teams can find everything from popularity trends to encyclopedic knowledge on the site, which is rounded out with a blog that puts facts into perspective.

When Dr. Paul Andlinger and his team started the consultancy company Solid IT in Vienna, Austria, they were well prepared to use their in-depth knowledge and years of expertise to help clients navigate the landscape of database technology.

But there was one question the group couldn’t answer confidently.

“Clients would frequently ask us which databases were the most popular, which ones would gain popularity in the future, and which ones would become obsolete,” Paul said. “We tried to answer such questions using surveys and analysis, but we couldn’t find any definitive information.”

Rather than basing responses on conjecture, Paul and his team set up an intricate system to accurately gauge the popularity of various database systems. The result was DB-Engines, a comprehensive knowledge base of relational and NoSQL database management systems that quickly gained prominence in its own right.

“While the intention was to use DB-Engines for computing consultancy, it ultimately became a standalone business,” Paul said.

Today, DB-Engines provides regularly updated information on database usage and concepts with an emphasis on the quickly expanding NoSQL space. The site also offers several industry resources, from an encyclopedia of database terms and concepts to an educational blog.

DB-Engines still features meticulously calculated popularity rankings via its DB-Engines Ranking list, which is updated monthly. The most important properties of various systems are illustrated in the overview of database management systems section, where viewers can compare metrics side by side.

When combined, these free resources allow developers and site-building teams working closely with databases to keep a finger on the pulse of the industry.

DB-Engines was founded in 2012 and continues to publish updated, high-quality information to this day. The company’s innovative popularity-based rankings serve as a standout feature among competitors.

“There are a lot of platforms online today that describe different database technologies and also some that list various database management systems,” Paul said. “But I don’t know of any comparable platform that provides popularity data like the DB-Engines Ranking.”

That section of the site is by far the most frequented, according to Paul’s data. “Most people come to us after Googling ‘database popularity,’” he said.

The team determines its ranking scores using an in-depth calculation method that takes into account several important sources of information.

These include frequency of searches for a system in Google Trends, frequency of mentions in technical discussions on Stack Overflow and DBA Stack Exchange, and offers on job search engines in which the system is mentioned. Other metrics include profiles in professional networks in which the system is noted and references on social networking sites.

“We use the Google and Bing search engines to measure the number of mentions of each system, such as Oracle or Microsoft SQL Server, on websites (measured as the number of results in search engine queries),” Paul told us. “Another ranking source is the jobs category. We look to Indeed and Simply Hired for jobs in which database skills for specific systems are requested.”

DB-Engines searches for a system name in conjunction with the term database — e.g., Oracle and database — to ensure only relevant results are collected. The team also measures the number of LinkedIn profiles in which a system is mentioned.

“Twitter is another important ranking source; we measure how many tweets refer to a specific system,” Paul said. “We count all the figures we get from these different ranking sources at the end of the month, apply some mathematics (which are more complicated than one might think), and come up with one score for the ranking.”

The stringent math-based guides DB-Engines uses to calculate factors like the popularity value of a system boost both the credibility and reliability of the data.

For instance, the group calculated popularity score by standardizing and averaging individual parameters. This preserves the statistical distance of the individual systems.

“That means, when System A has twice as large a value in the DB-Engines Ranking as System B, it is twice as popular when averaged over the individual evaluation criteria,” the DB-Engines site explains.

It’s also important to note that the DB-Engines Ranking does not measure the number or use of installations of particular systems. Instead, the team uses trends in increased popularity — as measured by discussions or job offers — which typically occur before the installation process. That way, the site can more accurately measure emerging trends among developers and site-building teams.

After tracking database-related information for nearly a decade, the DB-Engines team is also able to identify trends from a zoomed-out perspective.

“One trend we’ve observed is the shift between open-source and commercially licensed systems,” Paul said. “As we discussed in a recent blog, this is the first time since the start of DB-Engines in 2012 that the sum of the popularity of the open-source systems is greater than the sum of the popularity of commercially licensed systems.”

According to the blog, open-source database systems now account for 50.1% of total popularity scores on the site, representing an increase of 35.5% in the past eight years.

The company has also noticed a significant increase in the popularity of cloud-based systems, which DB-Engines defines as “systems which are available only as a cloud service and cannot be installed on-premises or rented servers.”

According to a December 2019 blog on the site, cloud-based database management systems accounted for about 3.7% of the overall popularity in 2019, versus just 1.6% three years prior, indicating a constant upward trend.

DB-Engines also nominates a database management system of the year on an annual basis. In 2020, the runner-up behind PostgreSQL was Microsoft Azure SQL Database, a fully managed database-as-a-service product optimized for use in the cloud.

Moving into 2021, Paul told us there’s another trend that developers, IT groups, and site-builders working with databases should look out for: new database models.

“A couple of years ago, there was a lengthy debate with two opinions on whether to use the relational data model or rather go with NoSQL,” he said. “We have very important and popular systems in both groups on our site. But what we see is that the large systems are now trying to cover both worlds.”

By expanding their reach, these hybrid database management systems can serve a broader purpose and customer base.

“We’re seeing relational data models where you would have, for example, a relational database for JSON documents,” Paul said. “We expect more growth in this area in the future.”