Agent engineering with Pydantic: types, graphs, and observable execution

A string arrives where an integer belongs

An application declares an integer field and receives "123". [Pydantic](https://pydantic.dev/) normally converts that compatible string into 123; strict mode can disable this coercion. Calling it a validation library understates its job: Pydantic uses Python type hints to define the expected data, converts acceptable inputs, and supports serialization. The difficult part is specifying exactly which conversions are acceptable. The original recording was published on February 6, 2025; product status and plans here reflect that conversation.

Samuel Colvin started Pydantic in 2017, when using type hints to define schemas still attracted objections. He credits Pydantic and FastAPI with helping normalize the approach. Its later adoption for AI was unplanned: the same machinery proved useful for defining and validating structured model outputs.

The bridge was JSON Schema, a representation of the expected data structure that Pydantic can generate. Sebastián Ramírez contributed that support before V1 for OpenAPI use; Colvin remembers roughly fifty contribution-related emails arriving over a weekend. Pydantic could then provide a single source of truth for structured outputs and tools: generate their schema and validate their data. Colvin was not involved in OpenAI’s decision to adopt JSON Schema and does not know why it was selected.

The cost of crossing into Python

Pydantic V2 combined a Rust rewrite with repairs to longstanding design problems. Colvin began the rewrite in early 2022 and worked on it full time for a year and a half, with others joining after the company formed. He considers it an investment a conventional startup would have struggled to justify: extensive engineering on a free library, undertaken before its eventual commercial value was clear.

Colvin reports that an unnamed foundation-model company reduced its internal time-to-first-token metric by 20% after upgrading from V1 to V2. The company’s workload and measurement method were unspecified. This is a deployment anecdote, not a general speedup estimate or a separate measurement of CPU savings.

A proposed next optimization would keep validated data in Rust types. Applications that validate and immediately serialize could then avoid constructing Python values altogether. Colvin estimated a further three-to-fivefold speedup for that path—an expectation, not a measured result. Other prospective work included easier support for specialized types such as NumPy arrays, extending the JSON parser’s processor-specific optimizations beyond AMD64, and possibly a binary serialization format such as CBOR for database round trips. He anticipated a less disruptive V3 without giving a release date.

The team’s move toward AI followed demand for these general-purpose tools. Colvin says the preceding year and a half had focused on improving Pydantic and building observability in Logfire. AI applications pulled both products toward a market where he saw substantial room to improve the surrounding software, independently of the models themselves.

Agents built around Python contracts

Colvin’s motivation for [Pydantic AI](https://ai.pydantic.dev/) centers on engineering quality. He argues that agent frameworks too often discard established Python practices, while acknowledging that early projects such as LangChain were still discovering the problem space. Pydantic AI accepts a learning cost from extensive generics in exchange for static type checking in larger applications. Its documentation examples are executed in tests, including checks on printed output; coverage, linting, and type checking are treated as baseline maintenance.

The fundamental unit is an agent containing a system prompt, tools, and an optional structured return type. Several units can compose into the larger application a user might call an agent. More complex workflows can use graphs: explicit nodes and transitions that expose the possible execution paths.

Pydantic’s graph design derives transitions from Python types. Nodes are normally dataclasses, and each node returns the next node to execute. Its return annotation declares the possible successors, including alternatives expressed as a union. The framework inspects those annotations to construct the graph, supporting static checking and generation of a Mermaid diagram. Execution follows a small loop: 1. Execute the current node. 2. Use its returned node as the next node to execute. 3. Repeat until a node returns an end value.

The argument for graphs is maintainability. Ordinary Python control flow can express these workflows, but complex branching can become difficult to follow. Type-derived graphs retain familiar Python while making transitions explicit; developers must still understand that annotations help construct the workflow. A change reportedly merged on the recording morning rebuilt Pydantic AI’s agent implementation on its graph library without changing the public API. The agent abstraction thus supplied one common graph, with lower-level graphs available when its assumptions did not fit.

A graph edge can mean six days

An e-commerce returns workflow exposes the difference between a diagram and an operational system. One arrow means an immediate function call. Another means waiting six days for a customer to print a label and mail a package. The second transition requires the application to continue later from the correct point.

Passing a starting node into the graph provided a way to resume. The available example was a question-and-answer command-line application: invoking it again instantiated a node and continued from there. Database persistence between nodes was still planned, so resuming across long delays and distributing execution across machines remained prospective capabilities rather than a demonstrated durable execution system.

Separate containers offered a possible isolation boundary for each unit of work. The proposed mechanism required serializing calls to subsequent nodes and giving every container the same code. Python on Cloudflare Workers was a candidate compute layer, but exploration awaited private-beta access. At recording time, Pydantic AI remained a Python library without a full orchestration platform’s deployment infrastructure.

That proposal also depended on handling binary libraries. Cloudflare’s Python support used Pyodide, which also enables Python execution in browsers. Pyodide provided builds of Pydantic Core and NumPy; for Pydantic Core, Rust was compiled to WebAssembly and called from Python. Sharing memory across Pydantic installations mattered when thousands of workers could occupy one physical machine, and remained a challenge under discussion.

Control points—and the limits of orchestration

Pydantic AI offered a rough progression from single agents through delegation and programmatic handoff to graph-based control flow. Its built-in agent repeatedly chooses a tool, calls it, returns to a central decision point, and eventually exits. Graph support was only a couple of weeks old, so there was no established catalog of community patterns to recommend.

Smaller steps create opportunities to intervene. Observability can reveal what an agent did, but an application without intervention points may have only a blunt response, such as raising an error after a limit is exceeded. Pydantic’s graph execution can advance with next inside the caller’s loop, letting the application inspect progress and break out using its own control flow. This addresses the risk of discovering a bad trajectory only after substantial time or money has been spent.

The enthusiasm was qualified. One participant warned that developers can overapply graphs and learn unnecessary domain-specific languages; the advocate acknowledged having put none into production personally. A separate challenge asked whether stronger trained models would absorb manually arranged reflection and reasoning loops. The response accepted that more capable models should need less prescribed structure, comparing scripted customer-service staff with trusted relationship bankers given greater autonomy.

Provider adapters, tests, and evaluation

Provider integration raised a choice about where abstractions should live: - A dedicated normalization layer, such as LiteLLM or Portkey, could centralize model support and reduce duplicated work across frameworks. - Framework-owned adapters could preserve Pydantic AI’s required type safety, with its advocate arguing that unifying providers was manageable work. Concerns about routing requests through a proxy were accompanied by an acknowledgment of limited investigation into LiteLLM. A claim that Pydantic AI lacked DeepSeek support was immediately corrected and withdrawn.

OpenAI-compatible interfaces in DeepSeek, Groq, and Ollama supported the argument that providers were converging on a common request format, making the OpenAI SDK an approximate common client. Authentication remained a separate concern: normalizing model requests does not resolve secret management or access across hosted models.

Testing covered both local substitutes and live services. A test model produces responses through Python without calling a real model; a function model lets users customize outputs. Separately, Pydantic AI reportedly ran a handful of live-model tests on at least every commit to main. One speaker estimated that Google’s Generative Language API returned HTTP 503 errors around 20% of the time and recalled repeated test failures in the preceding week. That estimate had no request count or measurement window, and the speaker explicitly distinguished it from the more reliable Vertex service.

Evaluation was less settled, with better support planned for both Pydantic AI and [Logfire](https://pydantic.dev/logfire). The suggestion that thirty examples might provide much of the statistical value of two hundred was back-of-the-envelope intuition, not a sampling prescription. Logfire exposed observability data through SQL and an API so users could develop their own analyses. The team used that same SQL access to explore capabilities, accepting that any built-in evaluation framework would leave gaps.

Telemetry for sensitive, unfinished work

[OpenTelemetry’s conventions for generative AI](https://opentelemetry.io/docs/specs/semconv/gen-ai/gen-ai-metrics/) aimed to standardize the structure of observability data sent by model SDKs and agent frameworks. Shared fields could make comparisons across libraries and models easier. At recording time, most of that work concerned individual large language model calls; conventions for the surrounding agent remained largely undecided.

Conversation capture changes the sensitivity of telemetry. In the hospital example, instrumentation can retain a SQL query’s structure while omitting patient identifiers and drug values from its parameters. A patient’s model conversation mixes those details into natural language. Logfire could suppress values under recognizable keys such as password, but the speaker considered that inadequate for sensitive conversational content. Self-hosting was planned for customers who needed it and could afford it.

Long operations also need visibility before completion. A span records an operation with a beginning and an end; Logfire sends information at both points so ongoing work can appear before it finishes. The examples were a model request lasting around twenty seconds and a batch job lasting thirty minutes. Completion-only reporting creates a consequential delay at those durations. Logfire did not yet correlate slow model calls across customers to identify provider-wide incidents.

The team expected instrumentation for major model SDKs eventually to live in OpenTelemetry’s repositories and was avoiding a separate effort it lacked capacity to sustain. Owning an agent framework nevertheless let it implement telemetry directly, rather than retrofit someone else’s framework. Exactly which framework-level data should preserve agent context remained unresolved.

The conventions themselves were changing: message capture was moving from attributes into events, while events were moving toward a separate top-level API. One participant worried that prompt/completion token accounting already lagged newer reasoning-token accounting. OpenTelemetry allowed additional attributes, but the exchange exposed a recurring maintenance tradeoff: a shared standard can delay support for new capabilities, while maintaining integrations internally can also fall behind competing priorities.

SQL as a product interface

Logfire’s database development moved from ClickHouse to Timescale and then to a custom database built around DataFusion. The migrations were costly, but the requirements extended beyond storing telemetry. Because users were expected to write SQL themselves, query ergonomics were part of the product interface.

Flexible JSON attributes were one pressure point. During the earlier evaluation, ClickHouse’s map-based storage made JSON-style lookups awkward; by the conversation, the speaker had been told its JSON support had improved. Duration comparisons supplied another historical complaint: comparing interval numbers without their units reportedly made two nanoseconds appear longer than one second. The speaker described a subsequent change to reject mismatched units and tentatively reported that conversion support followed.

The desired architecture stored binary data in object storage and queried it through nearby caching. The speaker said the relevant ClickHouse and Timescale implementations were available only through closed-source hosted offerings, adding another vendor’s margin. DataFusion offered an open-source toolbox for building a database, with Rust internals the team could modify. One speaker reported rewriting a slow string-comparison kernel and adding needed JSON support in a weekend by reusing Pydantic Core’s parser.

That control came at the cost of reaching customers sooner; staying with ClickHouse would have been faster. DataFusion was a long-term bet. Logfire’s intended differentiation was general-purpose observability with first-class AI support: its advocate argued that model traces lose value when separated from the rest of an application, making Datadog a more formidable competitor than AI-specific platforms. Easier onboarding was another priority, with Sentry held up as a positive example.

Runnable documentation becomes onboarding

Pydantic and Logfire belonged to one company, with an explicit commercial boundary. Pydantic and Pydantic AI were MIT licensed; Logfire was closed source, with no immediate licensing change planned. Pydantic.run was an open-source browser sandbox for demonstrating Logfire and Pydantic AI, without plans to make it a separate commercial product.

The onboarding problem was concrete: people signed up for Logfire but never created a project. The working hypothesis was that opening a development environment and configuring a project interrupted their momentum. Browser execution could provide an immediate experience. Plans included a model-provider proxy with daily spending limits and a Run button on every Pydantic AI example, potentially connected automatically to Logfire. The proxy was not yet implemented, and there was no measured conversion improvement.

Documentation testing had already supplied the foundation. Repeated complaints about stale examples led to pytest examples, which extracts Python from docstrings and Markdown, executes it, and runs linting; type checking was planned. This produced hundreds of examples that were self-contained or explicitly imported preceding examples. Keeping documentation executable had made it suitable for reuse in an interactive environment.

The team considered Marimo, whose notebooks also run in the browser through Pyodide, but chose a more basic, terminal-like interface to avoid suggesting that Logfire was only for notebooks. Marimo’s advocate also disclosed an angel investment. Participants favored plain-text notebook formats and criticized Jupyter’s dependence on cell execution order and the awkwardness of .ipynb files in Git and model inputs. One participant remained uncomfortable with annotation-driven behavior that felt too implicit.

Learning what to build—and preserving time to build it

The closing discussion welcomed the UK’s growing treatment of AI as an opportunity, while acknowledging the pull of the US ecosystem. One participant preferred writing code as a way to attract attention but saw value in spending more time in the US. The failure mode was familiar: talking about software can consume the time needed to produce it.

The case for conferences was more specific than networking. They help technical work reach a focused audience through well-produced content, while in-person conversations reveal workplace problems in depth. Repeated requests across the first ten or twenty conversations can guide priorities. Short social-media posts offer a weaker signal: experienced production builders and beginners can sound equally authoritative.

Pydantic AI remained intentionally incomplete at the time of the conversation, with embeddings, storage, and tool sets among its missing capabilities. The stated priority was deliberate implementation over immediate feature completeness, and users were encouraged to revisit the framework over the following months. Its success was described as existential for the company.

The company was nevertheless not hiring. It wanted more Logfire revenue and commercial traction before expanding a team already stretched across work resembling several startups. Preserving years of runway took precedence over immediately adding headcount.