How Shodan Helps Enterprises, Investors, and Security Professionals Track Devices, Measure Risk, and Identify Opportunities

TL; DR: Built as a search engine for everything, Shodan uncovers internet-connected devices for hardware manufacturers, enterprise organizations, and anyone looking for insights into where various technologies are being used. The company’s data goes beyond market intelligence to inform security companies, risk assessors, and hedge funds about a product’s popularity and threats. Founder John Matherly shared with us how this information can help weed out malware and other attacks relying on botnets and unprotected Internet-of-Things devices.

While Google and other search engines index the web, Shodan finds everything else on the internet — webcams, routers, traffic lights, refrigerators, and even control systems for the likes of medical devices, wind turbines, and nuclear power reactors.

The transparency and exposure are frightening to consumers and tech neophytes who fear the data will only be used to exploit vulnerable systems — but Shodan’s mission is quite the opposite, according to Founder John Matherly, who calls himself an internet cartographer.

“We think of Shodan in terms of tracking your assets and inventory management,” he said. “Especially now, when you have things in the cloud and all over the place, it’s more difficult to know what you have.”

For instance, companies like Cisco can learn where customers use their equipment and how often they update the firmware. Online businesses can gauge transactional security based on how customers connect to their marketplace. Instead of exposing vulnerable systems that lack basic security measures, Shodan enables organizations to gain greater visibility into their networks and devices.

“You probably don’t want a person checking out from your store if they connect from this compromised device,” John said. “You don’t often see refrigerators buying stuff on the internet. We can see things like that and help users protect themselves and prevent future exploits.”

Moving From Market Intelligence to Cyber Insurance

John drew inspiration from Netcraft, a service popular in the mid- to late-1990s for providing market-share analysis of web hosting providers, web servers, and operating systems.

“They did all these breakdowns, and I knew companies like Microsoft would buy the data to figure out who is using their products and things like that,” he said. “I wanted to basically create a Netcraft for everything. That was the initial tagline for Shodan.”

As a result, Shodan initially focused on market intelligence by helping vendors identify where and how customers used their products and those of their competitors. From there, however, clients began using the company’s data to identify network and device weaknesses.

“We don’t try to call ourselves a vulnerability scanner because we’re not actively checking for issues,” John said. “We can tell users, ‘This person is connecting over Tor, or over a VPN, or connecting from a compromised IoT device. Based on the type of device that is running as an IP, they can make various judgments.”

In addition to the technology companies themselves, Shodan has proven useful to ancillary industries like insurance and risk management.

“They want to know, before providing a policy, how exposed a company might be,” John said. “If they have all these vulnerable services running on the internet, they’ll probably be more expensive to insure.”

Hedge Funds to Malware Hunting: Shodan Finds New Ways to Use Data

When Shodan launched in 2009, John never envisioned his data would be used by cyber insurers, investors, and security professionals.

“I had never worked in insurance, and it’s definitely not an area I’m knowledgeable in,” he said. “Our customers were, though, and some people saw the opportunity and helped us get a foot in the door.”

More recently, John has noticed Shodan gaining traction with large hedge funds looking for more information about the popularity of emerging companies, technologies, and products before much is known about them.

“They’re trying to get ahead before earnings reports come out to figure out how well the product sold,” he said. “Basically, we’re tracking a lot more of the edge products, like how popular Kubernetes is compared with Apache Mesos, and those kinds of questions.”

With each new use case for the company’s data, John and the Shodan team uncover new challenges to making information more accessible to people with varying skillsets and competencies.

“Everybody at our team is very technical, and a lot of the people that will consume this data are in something like finance,” he said. “They’re not as technical in the same areas. It’s a learning experience for both of us to figure out how to translate what we collect into something more meaningful.”

Among perhaps most noteworthy applications of Shodan’s port scanning applications is the company’s one-of-a-kind malware hunter that seeks out command and control servers of global botnets — by pretending to be infected.

Without access to malware, Shodan asked customers to send samples of malware programs and trojans so programmers could develop signatures to emulate infected clients. The specialized crawler sends requests to every IP on the internet; when one responds, Shodan knows its a command and control (C2) server.

“It’s a very different dataset,” John said. “We can very often identify pieces of the C2s while they’re still in testing or development.”

Fostering a Two-Way Relationship With Customers

As evidenced by the company’s diversifying group of clients, Shodan perpetually looks to dive deeper into data to provide meaningful insights into network and device management. Where those efforts are specifically concentrated, however, largely depends on customer feedback and interactions, John said.

For instance, Shodan started off by crawling HTTPs before a customer wanted to know which sites used the more secure TLS 1.3 protocol. The company introduced version testing and now can show exactly which encryption is being used.

“A lot of these things we add are more in-depth and based on what our customers are doing before there’s the data,” John said. “Actually, we very often get customers sharing some of their processes and data with us so we can do it on our end.”

Recently, a client created machine learning models to find a customer that was exposing control systems over a virtual networking computing or remote desktop connection. After finding success with it, the client brought the solution back to Shodan to integrate into the larger product.

“It helps them because they don’t have to run pipelines locally, and it’s, of course, a huge win for us because we have machine learning functionality and these cool features,” John said.

Next up for Shodan programmers is working to make the platform more accessible and easier for a wider group of clients to use. The company’s website only serves as a demo of Shodan’s functionality, according to John, with products and services fueled by APIs.

“We spent a lot of time building out the platform and collecting the data, and I feel like we’re very strong in that,” he said. “We’re not very good about making it easy to consume. We do a lot more than what our website shows. If you’re not comfortable with using APIs, then you’re not going to get as much value as you could out of Shodan.”

Shodan’s Focused Philosophy on Providing Internet Intelligence

Ideally, according to John, customers can go to a website, enter a network range and get notified via Slack whenever Shodan finds something that meets the user’s criteria. The functionality already exists in the API, but clients currently need to build a script for it.

Until then, Shodan continues to operate as a documentation-first company. Employees will guide enterprises through the platform during onboarding, but the company invests in engineering resources instead of a sales team.

“I think what has made Shodan very successful is that we have adopted a Unix philosophy of “Do one thing and do it well,” John said. “That drives a lot of what we do. We don’t try to diversify too much.”

John’s childhood in Switzerland informs his approach to business, which differs from the traditional U.S. mentality. To this day, Shodan has been entirely bootstrapped and operates entirely with remote workers.

“The company’s growth has always been focused on making a good product and, hopefully, the rest will follow,” he said. “We were the first people to do this, to show there’s an actual market that people want to pay for. That wasn’t a known thing at the time.”

While customers find varying uses for Shodan’s data, John said the company’s only role is to provide the intelligence and data that enables others to make interpretations.

“When I first launched Shodan, it was more for devices and desktops and servers,” he said. “Now it’s used for teapots and all these crazy things. I have no vision of being a multibillion-dollar business. We do one thing, we do it well, we have a lot of very happy customers, and that makes us happy.”