Andreas Stuhlmüller

Andreas Stuhlmüller is the cofounder and CEO of Elicit, an AI research assistant for searching, summarizing, and understanding scientific literature. His contributions include research on decomposing language-model reasoning into inspectable steps and detecting hallucinations in academic-paper summaries. With cofounder Jungwon Byun, he advocates supervising the reasoning process, especially when the quality of a decision cannot be judged through immediate outcome feedback.[2][4][7][6]

In their April 2022 essay “Supervise Process, not Outcomes”, Stuhlmüller and Byun distinguished systems built around human-understandable task decompositions from systems optimized end to end through feedback on final results. Their central problem was how to reduce long-horizon work into independently meaningful short-horizon tasks. They argued that this could make AI useful for forecasting, policy decisions, and theoretical research, where outcome feedback may be unavailable, while reducing incentives to game evaluation measures.[4]

The coauthors treated process and outcome supervision as a spectrum, acknowledging that practical systems would likely combine them. They also recognized competitive pressure toward end-to-end optimization. Their essay credited Paul Christiano and Jon Uesato primarily for any new ideas; its contribution was to frame the case for developing process-based systems before increasingly capable AI made that choice harder to reverse.[4]

Stuhlmüller coauthored the January 2023 paper “Iterated Decomposition” with Justin Reppert, Ben Rachbach, Charlie George, Luke Stebbing, Jungwon Byun, and Maggie Appleton. The team described a human-in-the-loop workflow for inspecting failing components of compositional language-model programs and refining them through further decomposition, additional context, or chain-of-thought reasoning. They developed ICE, an open-source tool for visualizing program execution traces. In their case studies, the authors reported accuracy gains over less compositional baselines: 25% to 65% for describing trial placebos, 53% to 70% for evaluating participant adherence, and 38% to 69% for Qasper question answering.[7]

With Charlie George, Stuhlmüller also coauthored “Factored Verification”, submitted in October 2023. They evaluated automated hallucination detection in academic-paper summaries, reporting 76.2% accuracy on HaluEval’s summarization hallucination-detection task. Asking models to self-correct with Factored Critiques reduced reported mean hallucinations per multi-paper summary from 0.62 to 0.49 for ChatGPT 16k, 0.84 to 0.46 for GPT-4, and 1.55 to 0.95 for Claude 2. The authors cautioned that the hallucinations were often subtle.[6]

Byun attributed Elicit’s first literature-review version to a weekend prototype Stuhlmüller built. In their April 2024 Latent Space interview, he recalled that a question produced a list of claims, but both the claims and supporting papers were often irrelevant. The team printed poor results and put them on the wall to keep the shortcomings visible and motivate improvements. He tentatively recalled using Semantic Scholar’s API or something similar initially; the team later built its own search engine.[1]

Stuhlmüller explicitly credited Byun as a cofounder, explaining that both Elicit and its predecessor Ought had become very different from what he originally started. He recalled evaluating their prospective partnership through approximately 50 pages of questions and exchanges about values and decisions, using questions they devised themselves. Byun said she also shared her past performance reviews.[1]

Elicit became a public benefit corporation through a spinout announced in September 2023, with most Ought staff transferring to the new organization. The decision to sell Ought’s intellectual property was made by independent directors Owain Evans, Paul Christiano, and Owen Cotton-Barratt, with legal advice and an independent valuation. In 2024, Stuhlmüller described Elicit’s technical approach as building scaffolding through which powerful models could be deployed transparently and controllably.[9][1]

Stuhlmüller announced Elicit Notebooks in March 2024, bringing multiple searches, uploaded papers, summaries, and systematic filtering onto one page. Byun described the interface as the team’s fourth attempt to make composable research workflows work. Stuhlmüller explained the distinction from chat with a short formulation: “with notebooks, you can define a process.” His illustrative goal was to debug an analysis on a few papers, then rerun it over 10,000 papers.[8][1]

He envisioned searches, extractions, and lists of concepts as reusable notebook building blocks, with detailed model reasoning available beneath them. Initially, the human would act as the agent by choosing the next step. Increasingly reliable predictions of those choices could support gradual automation, while templates could package frequently used action sequences.[1]

Stuhlmüller emphasized internal benchmarks and vetted training datasets that reflected users’ queries for summarization, paper-grounded question answering, and ranking. He credited Charlie with a constitutional summarization project: specifying concise, abstract-faithful summaries and using constitution-based model feedback with RLHF to fine-tune an open-source summarizer. For engineering hires, he emphasized orchestration, useful abstractions, and “making reliable components out of unreliable pieces,” rather than requiring specialized AI expertise.[1]

In his May 2026 essay “Planning is unsolved”, Stuhlmüller distinguished models executing longer-horizon tasks from models helping people develop better plans. He argued that larger models alone should not be relied upon to close that gap. His proposed tools would decompose goals, identify uncertainties, explore possible futures, and check whether the reasoning behind a plan considered relevant evidence and failure modes.[5]

One example involved sampling possible clinical-trial trajectories using enrollment rates, dropout, signal strength, and competitor readouts, then using another language model to find weaknesses in a proposed plan. These illustrations were labeled Internal Elicit Beta. His broader proposal was to evaluate the process that generated a plan, with domain experts judging whether the tools actually helped people reason better.[5]

Before Ought and Elicit, Stuhlmüller researched probabilistic programming and cognitive science at MIT with Josh Tenenbaum and at Stanford with Noah Goodman. He describes that earlier work on uncertainty, inference, and models of cognition as continuing to inform his approach to AI reasoning.[2]

Stuhlmüller’s 2015 MIT PhD thesis explored probabilistic programs as representations of concepts and social reasoning, alongside algorithms for inference. With Noah Goodman, he coauthored the 2014 book “The Design and Implementation of Probabilistic Programming Languages,” which designed and implemented WebPPL, a small probabilistic programming language embedded in JavaScript. His 2013 “Learning Stochastic Inverses,” with Jessica Taylor and Goodman, investigated amortized Bayesian inference: investing computation upfront to accelerate later queries by learning inverse factorizations of a model.[2]

In the 2024 interview, Stuhlmüller recalled weighing an academic career and a startup after his Stanford postdoc. He found academic work poorly suited to building tools with strong engineering teams, but believed useful machine-reasoning products were still premature. He therefore chose a nonprofit research lab, Ought, to investigate how machines could support reasoning. Ought explored factored cognition and process supervision as approaches to scalable AI alignment before Elicit became the product through which he and his collaborators pursued that work.[1][2]