Logz.io: Delivering an Observability and SIEM Platform for Engineers Hosting Apps on Public and Private Clouds

TL; DR: Logz.io provides a suite of enterprise-grade software tools that monitor, manage, and analyze log data in open-source DevOps environments. Logz.io’s cloud-native observability solutions, which based on the popular ELK Stack open-source log analysis platform, integrate with other established open-source log analytics stacks. Specific task functionality includes alerting, log parsing, application performance monitoring, and IT operations analytics. The company was founded in 2014 by Check Point alums Tomer Levy and Asaf Yigal, and now serves businesses worldwide.

Some of the world’s biggest brands, including ZipRecruiter, Unity, Siemens, Turner, and eBay, show no qualms about relying on open-source development. And that endorsement has driven a significant market for providers of open-source tools and software services that enable DevOps engineers to deliver on schedule and on budget.

MarketWatch expects the value of the open-source services market to reach more than $13.1 billion by 2026 at a compound annual growth rate of 18.9%. That growth is fueled by the fact that open-source is also the environment of choice for many SMB software providers. But those businesses often find challenges when scaling up DevOps resources as they expand their operations.

Logz.io knows all about scaling those environments, and ZipRecruiter, Unity, Siemens, Turner, and eBay have all used its open-source services to grow their market share.

Enterprise efficiency is Logz.io’s specialty, and it offers a variety of capabilities across three key areas: log management, infrastructure monitoring, and security information and event management (SIEM). The company also supports customers by providing a unified SaaS solution.

Logz.io’s integrated security, compliance, and other helpful features allow clients to use the same tools and datasets to monitor and troubleshoot their open-source development processes.

Its portfolio also leverages machine learning to automatically scan through logs and other data to identify critical errors and exceptions. Logz.io customers can investigate and resolve production issues faster with actionable and context-rich insights.

Logz.io believes that the community-based nature of open-source software creates new business opportunities that can revolutionize competitive dynamics. In that model, SaaS providers enable open-source engineers to optimize tools they are familiar with and prefer. The company doesn’t sell proprietary solutions; instead, it fosters open-source development.

“We believe in open-source and in the open-source community,” said Jonah Kowall, Logz.io CTO. “As engineers who live and breathe DevOps, we understand what it takes to keep everything ticking in today’s complicated hosting environments. We help open-source engineers achieve what they need with the tools they have already come to love.”

The widely-used ELK Stack — or the Elastic Stack — is a group of open-source products from Elastic that comprises Elasticsearch (a search engine), Logstash (a log parsing engine), Kibana (analytics and visualization tool) and Beats (data shippers). The stack enables users to take data from any source or format and search, analyze, and visualize it in real time.

With Grafana, the analytics and visualization application that creates interactive monitoring dashboards, ELK Stack is core to Logz.io’s SaaS product suite. Many enterprises around the world also leverage ELK Stack.

“Other market-leading, non-open-source tools out there have a combined user base of about 30,000 companies. Around 700,000 organizations use open-source ELK Stack for some very intensive development projects,” Jonah said. “As organizations grow and their IT demands expand, it puts pressure on in-house resources when it comes to managing their own information.”

Many customers have invested months of time and effort in self-management, and consider the shift to a managed service daunting, and that is where Logz.io’s value proposition kicks in.

“We make it seamless for someone to move from an open-source environment to Logz.io,” Jonah said.

The Logz.io founders set out to create a cloud-based, open-source experience that is the same as the self-styled environment its customers typically work with. But the team delivers that experience as a seamless, managed service.

The Logz.io platform is multi-tenanted and can host on both Amazon Web Services (AWS) and Microsoft Azure. That means all security assurances and value-added features those cloud service providers (CSPs) offer come standard. Logz.io believes providing a cloud host choice is essential to replicate the user experience customers knew when self-managed.

“We work with teams of all sizes,” Jonah said. “From hyper-growth startups to large enterprises.”

However, the company understands that, among larger enterprise customers, many are wary of hosting all of their IT requirements in the cloud.

That is likely less about data sovereignty concerns than a wariness toward possible increases in cloud hosting pricing. And organizations certainly don’t want to be shifting critical data sets between CSPs every year to get a better deal.

“If Logz.io were to pitch to a traditional enterprise and advise them to send their log data to the cloud, half of them will say, ‘I don’t know if we’re ready to do that,’” Jonah said. “But that’s changing. To an extent, it depends on where a customer is in the IT maturity cycle.”

Jonah said that when Logz.io works with businesses that have launched in the last five years — and they are already on AWS — they don’t want to run their own datacenter. Most of them have already decided that the future of their business is in the cloud.

“In that situation, a service like Logz.io’s makes a lot of sense, and they are ready to work with us,” Jonahf told us.

Summer 2020 has been a busy period for Logz.io. In July, the company was recognized as the top solutions provider in the Momentum Grid Report for Log Analysis. The top ranking reflects Logz.io’s emergent leadership and innovation in the cloud monitoring and observability space.

Logz.io is developing a new tracing capability for its solutions suite, based on the Jaeger distributed tracing system from Uber Technologies. Jaeger monitors and troubleshoots microservices-based distributed systems, including root cause analysis and performance/latency optimization. Logz.io’s solution-in-progress is currently in beta testing and due to be released before the end of 2020.

“Logging is helpful in debugging problems — if you have good log data,” Jonah said. “Tracing will show you exactly where a request is going and what it’s touching, so it provides great isolation of which component is causing the slowdown, or throwing errors, in a distributed system. That means tracing has become a critical piece of the puzzle. ”

Logz.io aims to have the capability to “surface interesting things” that its systems observe in traces, in the same way as it surfaces them in the logs.

“To us, they’re basically stored in the same database and considered the same kind of data structure,” Jonah said.

As it serves global enterprises, Logz.io also has an eye on the future of open-source innovation. That’s why the company is paying close attention to the OpenTelemetry project from the Cloud Native Computing Foundation (CNCF).

The project is a merger of the OpenTracing and OpenCensus standards, and recently entered beta testing. Many expect it to become a standard for instrumentation and collection of observability data.

The OpenTelemetry project provides a single set of APIs, libraries, agents, collector services, and other components needed to capture distributed traces and metrics from an application.

“We are very excited about what’s happening around this project,” said Jonah. “The open-source community puts less emphasis on the agents and the technology that is proprietary, and more on what you can do in your service to analyze the data to create customer value there.”

Besides focusing on open-source, Logz.io’s future will be about how soon it can innovate what’s most needed by its customers.

“All of us are now dealing with information overload, so getting quickly to actionable insights contained within our data is what’s going to differentiate our tools,” Jonah said.