From Locked Out to Lift-Off: 90% of Orgs Embrace Neoclouds to Escape GPU Gridlock

Imagine waiting in an endless airport security line. Then you realize something — it’s mainly airline and security workers cutting the line and using scanners, holding up paying customers. This is what it’s like with supply bottlenecks for cloud computing power, particularly when hyperscaler cloud providers (Amazon, Google, etc.) hoard massive collections of GPUs (graphics processing units).

As a result, you’ll wait in line while Amazon, Microsoft, and Google commit most GPUs to their own AI projects. You’re a secondary business to them, even as a customer willing to pay.

Here’s where it gets interesting, though. There’s a new category of cloud providers with one purpose: rent GPUs without competing with their own customers. These are called neoclouds.

Whether you need GPUs for language learning models, computer vision, or 3D rendering, you’re stuck in bureaucratic limbo while hyperscalers build their own billion-dollar LLMs (language learning models).

A neocloud provider, however, fills a market gap by only specializing in high-performance computing, offering GPUs and cutting-edge hardware without the capacity issues and wait times seen with hyperscalers like Microsoft Azure, Google Cloud, and AWS.

Some examples of neocloud providers include CoreWeave, Lambda Labs, Crusoe and Nebius. You can think of them as boutique airlines with more specialized services and routes, ready and waiting to help customers who got bumped from their overbooked major carrier flights. They provide exactly what customers need, but without the complexities involved in doing business with the hyperscalers.

Our survey of tech directors and senior managers showed incredible numbers. They found a seismic shift to a new market category for AI compute, called neoclouds, away from behemoths like Microsoft Azure and Amazon Web Services.

Here’s what shocked me most: 9 out of 10 survey respondents are either already using neoclouds, currently migrating to a neocloud, or planning to migrate soon.

Here are the results:

• Already using neoclouds extensively: 25% of respondents
• In the process of testing or piloting neoclouds: 34% of respondents
• Have plans to adopt neocloud within the coming six months: 21% of respondents
• No current plans to adopt neoclouds: 10% of respondents
• Planning to adopt within 6-12 months: 7% of respondents
• Planning to adopt within 1-2 years: 3% of respondents

This is not a slow trickle of adoption; it’s a waterfall. I’ve tracked shifts in cloud trends for years, and I’ve never experienced such a drastic shift, especially when it involves running away from companies like Google and Microsoft.

Our survey asked what respondents saw as the main reasons driving the adoption of neoclouds. Most responses revolved around GPU scarcity, primarily wait times for GPUs.

In fact, 31% of survey respondents suggested wait times as their main reason for moving to neocloud providers. The numbers make sense, too, since 34% of businesses wait around two to four weeks to access the usual A100 and H100 GPUs from hyperscaler cloud providers like AWS and Google Cloud. Even worse, 20% wait over three months.

Here’s the complete list of reasons businesses jump ship:

• Reduced GPU wait times with neoclouds: 31% of respondents
• Better AI/ML performance and speed: 28.6% of respondents
• Cost: 13% of respondents
• Access to latest GPUs: 8.6% of respondents
• Not considering neoclouds: 7.4% of respondents
• Superior customer support: 7.1% of respondents
• Avoiding vendor lock-in with bigger providers: 4.3% of respondents

Although cost ended up as the least mentioned reason for switching to neocloud, you’ll see below that the savings are actually quite impressive.

We’re not hearing about marginal savings after organizations switch from traditional cloud providers to neocloud offerings. Organizations are experiencing 10%, 20% or even north of 50% savings with neoclouds.

Here’s a look at the study’s findings:

• 10-24% savings: 36% of respondents
• 25-49% savings: 33.5% of respondents
• 50% savings or more: 15.6% of respondents
• 5-9% savings: 7.7% of respondents
• No notable difference in savings: 4.2% of respondents
• 5-15% cost increase: 3% of respondents

When looking at those numbers, it makes me realize these organizations aren’t simply optimizing their AI infrastructure budgets; they’re transforming them. Keep in mind that these savings may level out eventually.

It’s more than likely a classic case of market arbitrage; the hyperscalers created artificial scarcity, so other businesses could exploit the market inefficiency by sourcing and deploying GPU resources more efficiently.

Cutting your budget means absolutely nothing if cloud performance suffers. However, our survey showed positive performance data from respondents. Here are some highlights:

• Performance improved significantly (25-49% faster): 42% of respondents
• Performance improved dramatically (50%+ faster): 14.6% of respondents
• No difference in performance: 2% of respondents
• Worse performance: 0% of respondents

I’m astounded by such results because ‌neocloud providers truly found a gap in the market — a gap created by hyperscalers putting customers behind their own projects. As a result, neocloud providers can offer both better performance and a lower cost.

While many organizations still rely on hyperscalers, our survey shows that a growing number are turning to neoclouds as their main AI infrastructure solution. Rather than fully abandoning traditional providers, businesses are blending hyperscalers and neoclouds, and, in a surprising shift, a third now identify neoclouds as their primary AI infrastructure provider.

The results continued to surprise me:

• Consider neocloud their primary AI infrastructure provider: 33% of respondents
• Use neocloud for specific workloads and specialized tools: 26% of respondents
• View neocloud providers as secondary/backup to hyperscalers: 14% of respondents
• Testing/experimentation platform only: 9.4% of respondents
• Part of a multicloud strategy to avoid vendor lock-in: 7.3% of respondents
• Neoclouds are not part of their strategy at all: 7% of respondents
• Temporary solution until hyperscaler capacity improves: 2.6% of respondents

I’m a bit stunned by these results, considering traditional cloud providers such as AWS, Azure, and Google Cloud spend billions in attempts to dominate the entire AI computing market. And yet 59% of respondents use neoclouds as either their primary or specialized infrastructure.

Right when I thought spending patterns might shift away from hyperscalers and favor neoclouds, our survey showed me otherwise. We asked industry decision makers about their spending plans for AI infrastructure over the next 12 months, and here’s what they said:

• Expect to spend significantly more on traditional hyperscalers: 93% of respondents
• Expect to spend significantly more on neoclouds: 92% of respondents
• Expect to spend significantly more on on-premises infrastructure: 82% of respondents

We’re seeing growth everywhere. Everyone is doubling down on every type of infrastructure. Hyperscalers. Neoclouds. On-premises infrastructure. It appears that neocloud isn’t entirely eating away at hyperscaler spending. The market at large is just booming.

Google Cloud, AWS, and Azure have serious pricing and capacity issues they’ve failed to solve, or perhaps don’t want to solve because they don’t have to. But neoclouds have sniffed out this market inefficiency and exploited it perfectly.

I’m not just noticing another cloud computing trend, but real market arbitrage fueled by GPU scarcity. And because of that, organizations are receiving significantly lower wait times, boosted performance, and greater savings from neocloud providers.

I suspect, too, that other advantages come along with such a neocloud revolution. Improved customer support, flexible billing models, custom hardware configurations… the potential is endless. Either way, we’re in the early stages of a restructuring of AI infrastructure. I’m excited to watch it progress.