Guardrails AI

Guardrails AI develops tools for validating language-model outputs and testing conversational agents. Its open-source framework checks outputs against application-specific requirements; Snowglobe simulates user conversations before deployment. In September 2026, Harvey announced its acquisition of the San Francisco company, saying co-founders Shreya Rajpal and Zayd Simjee and their team would join its product and engineering organization.[1][5][6]

Harvey said the team would begin work immediately on its Applied AI initiatives, bringing expertise in specifying, simulating, and evaluating agent behavior to its legal-work platform. The announcement reported more than 250,000 framework downloads a month—a download rate rather than a count of unique users or paying customers.[6]

The framework predates Snowglobe and the acquisition. Its early design explains how Guardrails approached reliability at the level of individual model calls.[1][5][6]

In a May 2023 interview, Rajpal traced the project to applications she built in late 2022. Prompt instructions gave her too little control over whether outputs met her requirements. By the interview, she had left her job and was working full-time on Guardrails and its open-source package.[1]

The early framework used RAIL, a reliable AI markup language, to specify output structure and types, prompt templates, and scripts for validation and correction. A guard wrapped an LLM call, generated prompts from the specification, and checked the returned output. When validation failed, configured corrective actions could reask the model, apply programmatic fixes, supply defaults, or filter invalid values. Reasking was one option rather than a required response to every failure.[1]

Rajpal said she initially tried Python classes but found them too verbose. XML let her separate a field’s properties—descriptions and validity criteria—from its nested children, while attaching error handling to individual fields. HTML-style event handlers and embedded scripts informed the design. In May 2023, collaborators were developing a code-first RAIL interface through an RFC and community input.[1]

The bug-free SQL validator demonstrated a check that went beyond output formatting. It built a sandbox SQL environment from a connection string or schema file, executed a generated query there, and checked whether it was executable. With reasking enabled, errors went back to the model for correction. This checked executability, which did not by itself establish that the query answered the user’s intended question.[1]

An October 2023 comparison with Cube contrasted Guardrails’ execution-and-error-feedback approach with a semantic layer that required predefined metrics. The article argued that Cube’s approach involved more upfront work but produced more reliable and predictable outputs; it presented an architectural comparison rather than benchmark results.[2]

Guardrails accepted string-input, string-output callables and integrated with LangChain and GPT Index/Llama, allowing developers to bring their preferred model-call components. Its logs recorded prompts, raw model outputs, validation results, and corrective actions. Rajpal qualified the framework’s proposed “SLAs”: they concerned content and output constraints, not guarantees about how long external API calls would take.[1]

Community applications included AutoPR, which Rajpal credited to an unnamed contributor. The GitHub Action used Guardrails to generate a pull request from an appropriately labeled issue, with the issue text supplying instructions.[1]

Guardrails AI launched Guardrails Hub on February 15, 2024, in an announcement signed by Rajpal as CEO. The open-source platform let developers find, build, and share validators, then combine them into application-specific guards. One example composed checks for competitor mentions and toxic language. At launch, installation used the Hub CLI and API-key configuration.[4]

Guardrails AI also announced a $7.5 million seed round, reported by GeekWire. Harvey’s later acquisition announcement named Zetta Venture Partners, Bloomberg Beta, Pear VC, Factory, and Microsoft among the company’s investors.[7][6]

Guardrails introduced Snowglobe on August 14, 2025, extending its products to testing conversational agents before production. Snowglobe models personas, orchestrates stateful conversations with multiple exchanges, and grounds scenarios in an agent’s context. Its scenarios include ordinary users interacting in varied ways, without assuming every interaction is adversarial. Generated scenarios can be exported as datasets to Hugging Face or evaluation and tracing tools.[5]

The company linked Snowglobe’s design to simulation used in self-driving development, where testing must cover risky situations beyond those encountered on real roads. At launch, it reported that customers were generating tens of thousands of simulated conversations before production. The announcement supplied no customer count or aggregation period.[5]

In its security advisory, Guardrails reported that an attacker published malicious version guardrails-ai==0.10.1 to PyPI on May 11, 2026, at approximately 6 p.m. Pacific. The company described a compromised employee GitHub personal access token being used to trigger Actions across 30 repositories, producing artifacts containing repository secrets. Extracted deployment tokens enabled the malicious release. Researchers identified the package within approximately two hours, and PyPI quarantined the repository.[9]

Guardrails described rotating organization and repository tokens, resetting the employee’s accounts and devices, taking its Ray inference cluster and validator Hub offline, and restoring infrastructure with rotated credentials. It scheduled proactive Snowglobe and Hub API-key rotation and tightened token and signed-commit policies. The company reported no evidence of user-data exfiltration through its systems, while warning that machines which installed the malicious package could be compromised.[9]

A maintainer notice opened in July 2026 announced a move from Hub installation and a private validator registry to standard public-PyPI packages, alongside discontinuation of company-hosted remote inference. The stated reasons were installation friction with pip and uv and the high cost of inference offered free since 2024. Migration instructions retained registered validator names while changing package and import paths; model-backed validators could run locally or against a developer’s own endpoint.[8]

Guardrails announced retirement of Hub distribution and company-hosted inference with an August 25, 2026 cutoff, although the notice also gave August 6 for remote inference. The announced retirement concerned those services; Guardrails would retain the open-source framework.[8]