OpenAI and Anthropic's AI Guardrails Are Blocking the Defenders Who Need Them Most

How AI Safety Measures Are Hampering Legitimate Security Researchers

Artificial intelligence has long been framed as a double-edged sword for cybersecurity—capable of accelerating attacks and automating defense. To keep the sharpest edge out of criminals' hands, leading labs like OpenAI and Anthropic have built strict guardrails into their models, preventing them from writing malware, crafting phishing emails, or helping to develop exploits. Those guardrails, however, are now drawing heavy criticism from the very people who protect networks: offensive security researchers. Their job is to find unknown vulnerabilities and develop proofs of concept before malicious hackers do. But when the AI tools they rely on refuse to answer basic questions, they say the safety measures are backfiring.

The friction became highly visible when U.S. export controls were slapped on Anthropic's advanced models Mythos and Fable after a report showed their guardrails could be bypassed. Though those controls were later lifted, the episode underscored the gatekeeping mentality. Both OpenAI and Anthropic offer vetted programs that grant approved cybersecurity researchers access to less restricted versions of their models, but many in the industry view the process as arbitrary and infantilizing. "It's not really comfortable to me that these random large companies are making arbitrary decisions about what is safe in security and what's not," said John Dowd, who sells zero-day exploits to Western governments.

Other researchers detailed their experiences to TechCrunch. Chris Anley, chief scientist at NCC Group, pointed out that asking a model to exploit a bug is often the fastest way to confirm it's real and worth fixing. When guardrails block that step, defenders lose a critical capability. Paolo Stagno, CTO of Crowdfense, said his team avoids feeding sensitive vulnerability data into cloud models entirely, preferring open-source alternatives that run locally without restrictions. An anonymous researcher at a smartphone-component manufacturer said that without Anthropic's CVP clearance, the models are "barely useful" because any security-adjacent prompt causes them to shut down. Chris Thompson, CEO of RemoteThreat, added that even inside vetted programs, the guardrails are inconsistent, forcing researchers to waste time negotiating with the model instead of analyzing vulnerabilities.

The cumulative effect, several experts warned, is that responsible researchers are being pushed toward ungoverned open-source models—including Chinese models like GLM—that carry no safety restrictions whatsoever. "You have these responsible researchers that are being pushed away from U.S.-governed systems to foreign-owned systems," Thompson said. "I think it's more harmful than good to have these guardrails in place."

The Offensive-Defensive Conundrum: When AI Guardrails Hurt Defenders

The Blurred Line Between Offense and Defense

At the heart of the tension is an irreducible fact: the same prompt that helps identify a critical vulnerability is also a roadmap for exploiting it. Anley compared the situation to a hammer—essential to build a house but impossible to separate from its potential as a weapon. When an AI model refuses to assist with any step that could be considered offensive, it also rejects tasks that are fundamental to defensive work. This paradox makes one-size-fits-all guardrails inherently brittle. Researchers may simply move to unrestricted tools, defeating the safety objective.

Why the Vetted Programs Aren't Fixing the Problem

Anthropic's CVP and OpenAI's equivalent programs are designed to give trusted researchers fewer restrictions, but they have significant limitations. Access is controlled, creating a barrier for smaller firms or independent researchers. Moreover, Thompson notes that even inside the programs, guardrail behavior can change daily, leading to inconsistent results that erode trust. The gatekeeping also sends a message that offensive security is dangerous by default—an idea that many in the industry reject, since their work is precisely what keeps systems secure. "These AI companies essentially treat customers like children who need babysitting," Stagno said.

The Open-Source Escape Hatch and Geopolitical Risk

Frustrated researchers are turning to fully open-source models that can be run locally with no vetting. This shift carries its own risks: sensitive prompts may be absorbed into future training runs if a model is cloud-hosted, but local execution addresses that. More concerning is the movement toward Chinese open-source models like GLM, which are freely downloadable and impose no usage restrictions. Security experts worry that pushing Western talent onto foreign-controlled AI ecosystems could compromise sensitive research and cede influence over the tools used in advanced vulnerability discovery. Thompson called it a "big storm coming" of attacks at scale, and warned that the companies stifling legitimate research will leave defenders unprepared.

Where the AI Labs Stand

OpenAI and Anthropic have not publicly detailed how they balance safety mandates with the needs of offensive researchers, though both maintain that vetted access strikes the right balance. The export control episode on Mythos and Fable shows that governments are also watching—and sometimes acting preemptively. For the labs, the challenge is acute: any loosening of guardrails could invite regulatory backlash and reputational damage, while keeping them tight risks alienating the core community that protects the digital ecosystem.

What Vulnerability Researchers and Cybersecurity Teams Can Do Now

• Use locally run open-source models for vulnerability discovery and exploit development. Researchers like Stagno already avoid cloud models for sensitive tasks to prevent data leakage. Models run locally (e.g., LLaMA or GLM) offer no guardrails and keep data off external servers, though they require careful handling of the resulting research products.
• Press AI labs for transparent, behaviorally consistent vetted access. Thompson's experience shows that even approved researchers face unpredictable guardrails. Organizations that rely on these tools should document and report inconsistencies to the labs, building a case for more reliable, auditable access programs rather than blanket restrictions.
• Separate offensive and defensive tooling in training data strategy. Anley noted that offensive and defensive coding capabilities are deeply intertwined. Security teams can advocate for training data curation that acknowledges this overlap, avoiding brute-force keyword filtering that cripples both activities. They can push for models that understand context—distinguishing legitimate research from malicious use—rather than blocking all security-related output.