Scalable, Efficient, and Distributed: How MinIO’s Cloud-Native Object Server Helps Modernize Infrastructure

TL; DR: MinIO is a distributed, software-defined object storage system created to handle the demands of unstructured data growth via enterprise-grade features. From the start, the MinIO, Inc., team designed the open-source solution to become the industry standard in private cloud object storage. Because the object storage server was built from the ground up six years ago, the technology is native to the latest technologies and architectures of the cloud, including containerization and orchestration through Kubernetes.

Like it or not, Amazon has spread its tentacles across global industries, offering everything from online shopping and grocery delivery to cloud-based compute and storage solutions via Amazon Web Services (AWS).

In 2020, it was ranked by Statista as the most valuable brand in the world. After seemingly dominating the Earth, Jeff Bezos has also set his sights on the moon, vowing to build infrastructure for colonies to come.

From a customer service perspective, however, the ability to be a provider of all things can come at the expense of specialized expertise. That’s why MinIO is countering Amazon S3, the conglomerate’s object storage service solution, with an open-source alternative. While Amazon’s S3 API has become a de facto standard in object storage for the public cloud, MinIO aims to be the standard for S3-compatible cloud object storage in the private cloud.

“When we started the company, we knew data was growing exponentially, and enterprises need simple and scalable solutions,” said Garima Kapoor, COO & Co-Founder at MinIO. “AWS had convinced the world that object storage was the right solution for scaling that much data. We knew AWS could not possibly be the entire world’s datacenter, so we aimed to become the object storage provider of choice for anyone outside of AWS.”

By focusing on object storage for unstructured data at scale, MinIO has become the most frequently downloaded object storage server in the industry. The purpose-built, software-defined solution leverages a single-layer architecture to serve objects only, providing a lightweight, scalable, and performant solution.

Over the past decade, the prevalence of data-producing smartphones and Internet of Things (IoT) devices have caused a big data boom that will continue to grow exponentially in the near future.
So much data is being created that traditional storage strategies no longer suffice. That’s where object storage comes in.

Object storage is a data storage technique that leverages distinct units, called objects, to section off data. Unlike conventional file or block storage, the strategy lacks a hierarchical structure or directory tree. This flat structure enables the system to grow through the seamless addition of nodes.

MinIO, built from scratch in November 2014 by Anand Babu Periasamy, Garima Kapoor, and Harshavardhana, stands out in that its object storage solution was created explicitly for the private cloud.

Of course, it excels in traditional object storage use cases, such as archiving, secondary storage, and disaster recovery. But it can also facilitate the use of private cloud in analytics, machine learning, and cloud-native application workloads.

In fact, as a cloud-native object storage solution, MinIO was designed with containerization, orchestration with Kubernetes, microservices, and multitenancy in mind.

“Our goal from the beginning has been to address the demands of unstructured data growth through standardized object storage outside of AWS,” Garima said. “From an adoption point of view, in terms of the traction we are getting within the industry and the open-source community, we have followed through with what we wanted to accomplish.”

The Amazon S3-compatible object storage system also includes enterprise-grade features, including inline erasure coding, bit-rot detection, state-of-the-art encryption, active-active replication, object locking, lifecycle management, and identity and access management.

“Our focus is being Kubernetes-native and being the platform of choice when it comes to enterprise IT teams that are looking to modernize the infrastructure by moving to a private or hybrid cloud environment,” Garima said. “Kubernetes for computing storage is becoming standard, and more and more applications are using object storage platforms for their primary storage requirements.”

Garima told us that VMware’s entrance into the Kubernetes market further solidifies that trend. Of course, the ongoing pandemic has served to accelerate digital transformation as a whole. Some of the enterprises the MinIO team interacts with are further along than others, but she said there’s “absolutely no doubt” that enterprises will move to an entirely software-defined infrastructure in the future.

“Everything’s going to be software-defined, especially considering how elastic the infrastructure needs to be with Kubernetes coming into prominence in the market and with the applications moving to object storage platforms,” Garima said. “The events of last year, especially with COVID-19, have brought that point to the forefront.”

MinIO, Inc. is ahead of the curve in the digital transformation arena. The company was one of the first players in the private and hybrid cloud server space to offer subscription-based services.

“You can just hit the subscribe button, enter your credit card information, and get started the same way you would with a public cloud,” Garima said. “It’s a very similar, frictionless onboarding process.”

Some customers choose to use a public cloud with MinIO.

“We have customers using MinIO inside Azure, inside Google cloud, just because they need that standardization from an API perspective,” Garima said. “They are also looking for that ease of deploying a storage software product.”

Garima told us that many MinIO users who have used a public cloud service like AWS in the past are now looking to provide a consistent service on a private cloud. In some cases, their internal customers are accustomed to AWS, and they are under a lot of pressure to deliver a similar user experience. And that’s where MinIO shines.

“Every company has its own object story right now,” Garima said. “Everyone wants to be in this space, but when an IT group is looking to rearchitect or modernize their infrastructure and deliver a service that looks and feels like a public cloud, there is no other option except to partner with MinIO.”

MinIO offers a wide range of features, including erasure coding, bit-rot protection, encryption, continuous replication, and identity management. But connecting all of these services is a philosophy of simplicity.

“MinIO is certainly feature-rich, but if we introduce something today, a competitor will catch up tomorrow,” Garima said. “So I think the main tenet that differentiates MinIO is the simplicity of everything. That’s why we are able to deliver such high performance and scale to hundreds of petabytes. Everything starts from the simplicity of the architecture.”

In the past, object storage required the expertise of sysadmins. But Garima and her colleagues designed the project in a way that resonates with development and administration teams alike.

“Storage has traditionally been one of those things that have always been under the purview of sysadmins — you needed that specific knowledge to spin up an interim storage environment. But through the simplicity of MinIO, we see that developers have taken it upon themselves to spin up such environments.”

Garima described the MinIO, Inc. team as humble, passionate, and willing to learn and adapt to new concepts.

“When it comes to the development of the product itself, we have been very focused from day one on becoming AWS for the rest of the world,” she said. “It is as simple as that. We know that we can be far ahead of others when we are only doing one thing and being the best at it.”

MinIO’s road map is also aligned with that goal. The company will continue to foster an AWS-like consumer experience on the private cloud. It also plans to release features in MinIO that will make Kubernetes consumption easier.

“Kubernetes is very nice with automation, but anyone who has played with it knows how difficult it is to run a Kubernetes environment, especially when you have to orchestrate nodes from a storage standpoint,” she said. “So we are coming up with a Kubernetes console. It’s already there for the community and customers to play around with on GitHub, but we are going to put out an official release soon.”