Storj’s Open-Source Platform Reimagines the Cloud with Decentralized, Distributed Cloud Storage Powered by Blockchain

TL; DR: As the first decentralized, encrypted cloud storage platform leveraging blockchain technology and cryptography to secure files, Storj has flipped the switch on traditional data storage. Through a passion for decentralization and open-source software, the company has created a system that’s fast, cheap, and secure, yet independent of untrustworthy corporations and vulnerable datacenters. John Gleeson, VP of Operations at Storj, recently updated us on the company and how its community benefits from a secure network of computers around the globe.

Sit down for a cup of coffee with folks in the banking industry and spark up a conversation around blockchain. Chances are, the reactions you’ll receive will range from skepticism to fear — and maybe even a little confusion.

That’s because blockchain — a decentralized, distributed public ledger originally developed as an accounting mechanism for digital currencies such as Bitcoin or Ether — is poised to disrupt traditional banking in a major way. The concept is notoriously hard to understand, but in a nutshell, cryptocurrencies are created, exchanged, and stored in the blockchain using encryption techniques that control the generation of currency units and verify fund transfers. Each “block” on the chain represents the most recent transaction, in chronological order.

Studies show the technology’s influence on financial services may even mirror the internet’s detrimental effect on print newspapers. Since blockchain’s distributed ledger allows transactions to be shared across a peer-to-peer network of computers, or nodes, digital currency transactions can be tracked and verified by the community in a decentralized fashion. This eliminates the need for central recordkeeping institutions and, potentially, intermediaries such as bankers.

But blockchain’s whirlwind effect isn’t limited to the financial services sector. While it’s typically used to verify digital currency transactions, virtually any type of document can be stored within the blockchain — a concept that Shawn Wilkinson, Co-Founder of Storj, began toying with in 2014.

At the time, Shawn was working on a project that required heavy data storage. Separately, he was trying his hand at Bitcoin mining — a process in which “miners” use specialized software to solve Bitcoin algorithms in exchange for certain amounts of the cryptocurrency.

Suddenly, it clicked.

“One of the thoughts he had was, if you can use the compute to problem-solve for the proof of work for blockchain related to Bitcoin, and you could store that blockchain data in the decentralized distributed ledger, what else can you store that way?” said John Gleeson, VP of Operations at Storj.

After some fundraising, Shawn devised his plan to deliver distributed cloud storage with blockchain-based payments.

“We started working on how to build that decentralized storage component but, at the same time, make it secure, make it perform well, and also do it in a way that’s even more cost-effective than the centralized cloud,” John said.

To generate additional funding, Storj held a sale for its Ethereum-based application token, STORJ, which the company uses as a secure payment system for network users. Participants of the Token Sale received STORJ tokens that could be exchanged for cloud-based storage space. Just one week after launching the sale, Storj had already met its $30 million goal.

“That gave us the money to go to market and really put a world-class team together to take this to industry,” John said.

Since emerging from beta, the Storj network has grown from just a handful of nodes to 100,000-plus computers running the open-source Storj Share application and storing more than 100 petabytes of data.

The company strives to create decentralized storage that’s private by design and makes up for what centralized cloud storage lacks in terms of being open to a single point of attack.

“People are sick of hearing the stories of data stored in centralized repositories getting hacked, compromised, downloaded, and sold,” John said.

Storj solves this problem through erasure coding, which shreds encrypted files into “shards” before distributing them in a network of computers around the globe.

“The analogy we use is sprinkling encrypted sand on an encrypted beach — taking those shards and just throwing them out over 90K nodes in a way that can’t be compromised,” John said.

Of course, data owners can retrieve files through a simple process.

“You spread out the files into so many pieces — but you don’t need all the pieces back to recover the files,” John said. “You only need 51% of the shards to actually reconstitute the file, so it’s extremely efficient and extremely secure.”

In the event of a regional power outage, a datacenter going down, or an ISP issue that might effect a number of nodes, the algorithms used to distribute data ensure that the data is sufficiently spread out to maintain high availability.

“It’s security for that kind of thing because we’re not dependent on any one region, any one datacenter, or any one building,” John said. “We’re looking at nodes that are distributed and have nothing to do with each other.”

Between artificial intelligence, machine learning, and the Internet of Things, we’re accumulating data at an unprecedented pace.

“According to analysts, we’re approaching 44 zettabytes by 2020, and only half of that data is going to be in traditional centralized cloud storage,” John said.

But John told us centralized cloud storage isn’t necessarily cost-effective for large datasets.

“We provide a highly secure but lower-cost and high-performance solution,” he said. “It becomes a way to match the right storage method for the right types of data.”

Through the Storj Share application, users can even rent out excess hard drive space in exchange for STORJ, a cryptocurrency similar to Ether. The process, called Drive Farming, is secure in that no “farmer” holds an entire file, and all shards are encrypted. Since farmers compete to win the business of users, rates remain competitive.

As demand for decentralized storage solutions increases, Storj continues to experience significant growth. To help keep up the momentum, Ben Golub, former CEO and co-founder of Docker, joined Storj as Executive Chairman and Interim CEO this March.

“We’ve done an awful lot of work in terms of internal alignment around our vision, mission, values, strategy, and direction — and a lot of it has been led by our new CEO,” John said. “He’s someone who really knows open-source and is very familiar with storage and turning an open-source project into an enterprise-ready platform.”

And John told us he’s backed by a team of relentlessly focused employees.

“It’s all people who are passionate about decentralization,” he said. “If you’re passionate about decentralization, that brings a similar passion for things like transparency and open-source.”

The Storj staff spans the globe, from Georgia, California, Utah, and Michigan in the U.S. to Germany and Japan overseas.

“Our company is pretty much just as distributed as our product,” John said.

John noted that, as Storj expands, so must the technological foundation supporting it.

“One of the things we noticed when we hit somewhere between 30 and 50 petabytes was there were aspects of the platform that didn’t scale as easily as we wanted them to,” he said.

In response, the company kicked off plans for a network update. John said the idea was to take certain components that were centralized and had scale limitations and fully decentralize them. The result of those efforts, V3, will be launched as early as September.

“Network V3 will give us S3-level performance at half the price and a number of different capabilities that will really help us in terms of being able to target geographic locations for storage,” John said.

As John told us, it will be jam-packed with features based on feedback from the Storj community.

“The next version of the network is the key to a lot of different capabilities — everything from high-performance general storage all the way through hyper-local CDN,” John said. “I’m really excited to see it launch in the fall.”