Gray Swan

Gray Swan builds security tools for AI models and agents: Shade tests for vulnerabilities, Cygnal monitors behavior and enforces policies during deployment, and Arena supplies attack intelligence through community red teaming. Its cofounders include Matt Fredrikson, CEO, and Zico Kolter, Chief Scientist. The company addresses the risks introduced by deploying AI—including agents leaking data or making unsafe tool calls.[5][1][3]

The platform connects attack discovery with defense. Arena runs prize competitions in which participants try to circumvent models’ safety and security objectives. Gray Swan uses the resulting attack trajectories to train the models powering Shade and Cygnal. In the June 2026 interview, Fredrikson described a shift from primarily testing chat-based content policies toward agents, tool use and applications built on top of models.[5][1]

Shade runs automated adversarial tests against models and agents before deployment and within CI/CD pipelines. Its product description specifies campaigns scoped to a deployment’s model, guardrails and tools, with adaptive attack techniques, reproducible findings and severity ratings. In the June interview, the cofounders reported that automated testing could find more breaks than human red teamers within a fixed time window on specific tasks; they qualified this as a bounded comparison rather than generally superhuman red teaming.[5][8][1]

Cygnal checks inputs, outputs and agent tool calls for violations of an organization’s custom policies. It can inspect incoming content for prompt injection and outbound actions for risks such as sending an API key to an untrusted destination. The operational distinction is between encountering malicious instructions and acting on them: an agent need not stop simply because it reads an injection, but a planned action that violates policy should be blocked. Gray Swan’s documentation lists customizable input/output moderation and an integration path that changes the base URL without changing the SDK.[1][7]

Gray Swan’s Indirect Prompt Injection Arena tested attacks hidden in external content that agents read, such as emails, webpages and repositories. Success required both a harmful action and concealment from the user. In a March 18, 2026 account, Mateusz Dziemian credited collaboration with UK AISI, US CAISI, OpenAI, Anthropic and Meta. Over three weeks, 464 participants submitted more than 272,000 attempts against 13 frontier models across 41 scenarios, with $40,000 in prizes; the account reported 8,648 successful attacks.[9]

Every tested model was compromised in the competition. Reported attempt success rates ranged from 0.5% for Claude Opus 4.5 to 8.5% for Gemini 2.5 Pro; these were competition results for the tested configurations, not production compromise probabilities. The study found only weak correlation between capability and robustness across models, while also finding robustness improvements with capability within model families. One attack template used a simulated environment and fake control panel to make injected instructions appear authoritative, working across 21 scenarios and nine models with minimal modification.[9]

Gray Swan described releasing the evaluation scenarios, judging system and a small attack-data sample, while providing the full competition dataset to UK AISI and US CAISI. Participating frontier labs received attack data relevant to their own models and open-weight models for defense training; a held-out portion preserved the benchmark for future evaluations.[9]

The testing-and-defense platform followed earlier work at Carnegie Mellon. Fredrikson recalled that he and Kolter had spent more than a decade studying vulnerabilities and attack surfaces in deep learning, how to measure their severity and how to make inference more robust. Gray Swan publicly emerged from stealth on July 16, 2024, initially offering Shade as an AI evaluation and continuous integration tool alongside Cygnet, a safety-tuned model based on Llama-3. At launch, Cygnet offered free web experimentation and per-token API access.[1][4]

On July 25, 2024, Gray Swan stated that it held no position on California’s proposed SB-1047 and would not design its products to satisfy the bill’s auditing requirements or operate as an AI compliance auditing company. Because advisor Dan Hendrycks was associated with the proposal, the company announced his decision to divest his entire financial stake “to avoid even the appearance of a financial conflict” and continue as an unpaid advisor without equity.[6]

On May 28, 2026, Gray Swan announced a $40 million Series A co-led by Wing Venture Capital and Madrona, with participation from Obvious Ventures, Snowflake Ventures, Hudson River Trading, Samsung Next and existing investor Magarac Venture Partners. It said the funding would expand go-to-market operations, deepen frontier-lab partnerships and scale the team. The announcement also introduced native integration of its runtime AI protection with Snowflake’s AI ecosystem.[5]

At that announcement, Gray Swan reported more than 20 customers across frontier labs and global enterprises, citations in 11 recent frontier-model system cards, an Arena community of more than 15,000 researchers and security professionals, and more than one million attack trajectories. In the June interview, the cofounders described enterprise deployment as the intended next stage of expansion. They also emphasized that AI guardrails must accompany system-level protections such as isolation, authentication and access controls, particularly for agents with broad computer-use capabilities.[5][1]