Experts Warn: AI Vulnerability Testing Lacks Clear Rules (But There Is a Solution)

Every provider is, without a doubt, integrating AI into their platforms. Whether it’s via onboarding, chat support, building and design, or extra features and add-ons, AI tools are everywhere. And yet, HackerOne says AI-related issues jumped more than 200% last year alone.

That’s the nudge that pushed the bug bounty and cybersecurity platform to expand its regulatory framework to include a new Good Faith AI Research Safe Harbor, which specifically addresses the “legal gray zones” that AI security testing keeps running into.

“While the original framework [the Gold Standard Safe Harbor] was designed for traditional software security testing, the AI Safe Harbor goes further to address potential ambiguity by explicitly protecting testing of AI behaviors that don’t fit neatly into legacy vulnerability categories,” Michael Woolslayer, Policy Counsel at HackerOne, told us.

So, under the program, companies agree not to enforce Terms of Service or Acceptable Use restrictions against approved good-faith AI research, won’t pursue legal action over that testing, and will support researchers if third parties challenge their work.

OK — so what? Most hosting providers have already sat through the AI planning meetings. Should you add a builder? SEO tools? Generative copy? Automated security? Those are the easy questions; the fun ones. The harder ones are…less sexy.

Hosting providers typically rely on vendors to vet the AI tools they integrate. But when a third-party feature causes an issue for customers, clients will look to their host.

But adopting the AI Research Safe Harbor can also apply directly to hosting providers, not just their vendors. By adopting the program, Woolslayer says they can authorize and support research into prompt injections, hallucinations, and other AI-specific risks much easier.

Woolslayer explained to us that, unlike traditional software bugs, AI vulnerabilities don’t yet quite fit into existing vulnerability categories. For example, most bug bounty programs aren’t designed to recognize them (yet).

But HackerOne’s own report found that AI tools are not only expanding attack surfaces, but prompt injections are the most common entry point. Prompt injections can happen through any AI feature that accepts user input and passes it to a model. It’s that simple.

“Traditional bug bounty rules were built around code — clear, predictable logic with well-defined security boundaries,” Woolslayer added. “AI systems behave in dynamic and sometimes unexpected ways, and they’re vulnerable to misuse that isn’t always captured by conventional vulnerability frameworks.”

Under the safe harbor framework, researchers can test AI systems you or your vendors own or control without fear of legal retaliation. You might be surprised to learn just how frequent these researchers are punished, too.

Take a look at Hewlett-Packard, which threatened a lawsuit against a security firm after a vulnerability exploit was publicly released. Or another case, which tells the story of a security researcher who was fined by a German court for reporting vulnerabilities they found in a company’s systems.

That’s what the AI Research Safe Harbor is trying to resolve, Woolslayer said. Even when researchers uncover real flaws, violating a platform’s Terms of Use is still violating the Terms of Use. No company thanks someone for finding a critical issue if the way they discovered it violated the rules.

And while no hosting provider or vendor or developer is required to adopt the framework, its existence is the reminder that if AI runs on your platform, whether you built it, licensed it, or bundled it, the risk will fall onto the provider that’s deploying it to their customers.