Emmanuel Ameisen

Emmanuel Ameisen is a researcher on Anthropic’s interpretability team and a core contributor to Circuit Tracing, a method for investigating how language models compute their answers. His contributions include graph-generation and experimental infrastructure for tracing interactions among internal features and testing their effects on model behavior. He also wrote O’Reilly’s Building Machine Learning Powered Applications, drawing on an earlier career in applied ML.[6][2][4]

Published March 27, 2025, Circuit Tracing: Revealing Computational Graphs in Language Models lists Ameisen first among several core contributors. With Jack Lindsey, Joshua Batson and Chris Olah, he developed and refined replacement-model and attribution-graph computation. Ameisen built a stable graph-generation pipeline and infrastructure for patching experiments; he and Lindsey built infrastructure for running the replacement model. Adam Pearce built and maintained the interactive graph interface, assisted by Brian Chen and Shan Carter, while Batson led the overall circuits project.[2]

The team’s method uses sparse features—internal signals that activate selectively—as building blocks for explanations. A cross-layer transcoder learns to approximate the model’s multilayer perceptron computations: each feature reads from one layer and can contribute to outputs in that and later layers. This can consolidate repeated amplification of a concept into a simpler graph. Transcoders also bridge nonlinear MLP computations, allowing researchers to analyze direct interactions between features.[2]

For a particular prompt, researchers hold attention patterns and normalization denominators fixed and add correction terms for reconstruction errors. The resulting local replacement model matches the original forward pass’s activations and outputs. Its attribution graph traces contributions from prompt tokens through active features to candidate output tokens. That agreement alone does not establish that both models use the same mechanisms; interventions test whether changing a feature produces the downstream effects the graph predicts.[2]

Ameisen investigated the paper’s Michael Jordan example: completing “Fact: Michael Jordan plays the sport of” with “basketball.” The graph separates a path identifying the requested category—a sport—from another carrying basketball-related information about Jordan. Suppressing either group strongly affects the output while having a smaller effect on the other group, supporting the proposed parallel structure.[2]

The March 2025 method leaves important gaps. Frozen attention shows information flowing from an attended word but omits the query–key computation that selected it; when that selection drives the answer, the graph can miss the central mechanism. Default graphs omit inactive features, even when suppression matters. Nicholas L. Turner and Ameisen added visualization of attributions to selected inactive features, enabling investigation of specific hypotheses. Reconstruction errors still leave computation unexplained, and grouped intervention checks support coarse dependencies rather than a complete causal account. Perturbation predictions become less accurate many layers downstream.[2]

Ameisen, Wes Gurnee and Joshua Batson share equal-contribution credit on the seven-author When Models Manipulate Manifolds: The Geometry of a Counting Task, originally published October 21, 2025 and archived on arXiv in January 2026. The study examines how Claude 3.5 Haiku predicts line breaks in fixed-width text despite receiving tokens rather than a visual page.[5]

The team found curved, low-dimensional representations of character count, line width and remaining space. Attention heads transform these representations to detect an approaching boundary; multiple heads use different offsets to estimate how many characters remain. Later computation combines that estimate with the length of the planned next word to decide whether it fits. Targeted interventions in the count representations change linebreaking behavior, connecting the geometry to computation in this model and task.[5]

The investigation required a change in approach. The team initially made little progress with probing—reading information from activations—and patching, which substitutes activations to test their effects. Their probes targeted quantities different from those the model represented cleanly. Sparse features and synthetic datasets helped distinguish current position, line width and remaining space; geometric analysis then simplified parts of the algorithm. The study leaves questions open, including how the model aggregates line width across preceding lines and handles multiple plausible next words.[5]

In his June 2025 Latent Space interview, Ameisen described interpretability as long-term, high-risk research whose immediate utility was investigating particular behaviors and their causes. He argued that understanding should grow alongside capability and deployment: “you kind of want your ability to understand the model, to scale with like how good the model is doing”. He identified attention, reconstruction error, long generated sequences and broader model structure as areas needing further work.[1]

In a September 2026 account of a public discussion, Tim O’Reilly reported Ameisen challenging the inference that a simple training objective implies simple internal computation. O’Reilly quoted him saying, “To predict the next word well, you need a very complex world model.” The account also records Ameisen distinguishing deception from limited introspection: a model’s inaccurate verbal explanation of its computation need not mean it deliberately concealed that computation.[6]

Before his interpretability research, Ameisen worked on applied ML systems and helped others build them. His biography describes data-science work at LocalMotion and Zipcar, followed by serving as Head of AI at Insight Data Science and working at Stripe. At Stripe, he describes improving core fraud models and leading automation of model training and deployment. He was Head of AI at Insight in October 2018 and was working at Stripe by January 2020.[3][7][8]

In How to deliver on Machine Learning projects, published October 4, 2018, Ameisen and Adam Coates advocated an analyze → select an approach → implement → measure loop. They recommended defining success through useful product performance, diagnosing the largest bottleneck and favoring simple, fast experiments. Their examples include Insight Fellow Jack Kwok’s disaster-recovery segmentation project: recognizing that hurricane imagery was blurrier than the training images led him to add blur augmentation and reduce the performance gap.[7]

O’Reilly published Building Machine Learning Powered Applications in January 2020. Its end-to-end scope moves from defining a product problem and building an initial pipeline through data acquisition, evaluation, debugging, deployment safeguards and monitoring. An ML-assisted writing application supplies the recurring case. This work documents the practical application-building background that preceded Ameisen’s research into internal model computation.[4][3]

At Anthropic, Ameisen worked on fine-tuning with a focus on tool use before joining interpretability. In the June 2025 interview, he recalled becoming increasingly interested in understanding the production models he had been fine-tuning. He joined interpretability after the team’s original dictionary-learning paper, initially worrying whether methods demonstrated on small models would scale to models used in practice. He also pushed back against treating research as more valuable than engineering, arguing that executing experiments quickly and interpreting them correctly are central research skills.[3][1]