Erik Schluntz

Erik Schluntz optimized Anthropic’s SWE-bench coding agent and co-authored practical guidance on building effective agents with Barry Zhang. In November 2024, he was a member of Anthropic’s technical staff working on tool use, computer use, and SWE-bench. His work paired model-directed execution with carefully engineered tool interfaces; his public recommendations also emphasized making an agent’s results understandable and auditable.[3][4][5][1]

Schluntz’s SWE-bench work involved implementing the evaluation and optimizing a reference agent that developers could build upon. Anthropic’s engineering report credits him with agent optimization and report authorship, and Simon Biggs, Dawn Drain, and Eric Christiansen with helping implement the benchmark. Model training was a separate collaborative effort. Anthropic reported 49% on SWE-bench Verified for upgraded Claude 3.5 Sonnet with the scaffold, compared with the report’s previous state-of-the-art figure of 45%. The evaluation covered a human-reviewed subset of 500 GitHub issues from open-source Python repositories.[1][3]

The agent used SWE-Agent as its code foundation, with a prompt, a Bash tool, and an Edit tool for viewing and changing files and directories. Claude chose its own steps and continued until it decided it had finished or exceeded its 200,000-token context limit. Instead of requiring a fixed search pipeline, the implementation let the model explore directories and files. Schluntz noted that SWE-bench bug reports often supplied stack traces pointing toward the relevant code; finding files for broader feature requests could be harder.[3][1]

Schluntz and his colleagues spent more effort optimizing tools than the overall prompt. The Edit tool used string replacement: the model supplied an existing string and its replacement, and the edit proceeded only if the original matched exactly once. Ambiguous or missing matches produced an error so the model could retry. Requiring absolute paths prevented confusion after directory changes. In the interview, Schluntz said regenerating whole files had been more accurate, but could become prohibitively expensive in tokens for large files.[3][4][1]

The report illustrated the loop with a scikit-learn issue: Claude reproduced an unexpected-keyword error, changed RidgeClassifierCV to accept and forward store_cv_values, and reran its reproduction script. Anthropic reported that the task’s tests passed after 12 steps. Longer runs could consume more than 100,000 tokens. Hidden grading tests sometimes exposed fixes made at the wrong level of abstraction, while environment setup and grading errors complicated evaluation. The agent also lacked an integration for viewing images, despite the underlying model’s vision capabilities, which hindered tasks involving Matplotlib.[3]

Schluntz and Barry Zhang co-authored Building effective agents, which distinguishes workflows governed by predefined code paths from agents that dynamically direct their own processes and tool use. They recommend starting with the simplest adequate solution and adding complexity when it demonstrably improves outcomes. Their evaluator-optimizer pattern uses one LLM call to generate a response and another to evaluate it and provide feedback in a loop. An official cookbook supplies minimal implementations of these patterns.[4][5]

In November 2024, Schluntz cautioned that frameworks could make unnecessary complexity easy to add and obscure the prompts and responses needed for debugging. He favored clarifying questions and human feedback on an implementation plan before lengthy coding work. His concern extended beyond benchmark performance: an agent returning after hours of work needed to explain its result and how it reached it. He described the requirement as doing work in a “trustable, auditable way”.[1]

Schluntz saw computer use as a way to reduce the engineering effort of integrating software systems. He suggested prototyping through a graphical interface, then converting an adopted feature into an API integration for greater speed and lower token use. Discussing the beta capability in 2024, he also defended a sandboxed default: it limited what the model could affect and gave it a separate screen so the user could continue working.[1]

Before Anthropic, Schluntz was Cobalt Robotics’ co-founder and CTO alongside Travis Deyle. His earlier work on security and inspection robots supplied concrete examples for his later discussion of computer-use integration and reliability.[2][1]

At Cobalt, Schluntz recalled that elevator API integrations delayed robot deployments: working with a new elevator company could take six months, with further delays to install interface hardware. He added a small, vision-guided arm that pressed elevator buttons, allowing deployment in a day. The arm was slower and less reliable than the API and sometimes needed another attempt. He used this tradeoff to explain computer use’s appeal: acting through an existing human interface could reduce integration time while sacrificing operational efficiency.[1]

Another Cobalt recollection concerned intermittent thermal-camera failures. After investigating software and hardware changes, Schluntz learned that procurement had switched USB-cable suppliers to save a dollar per cable. He attributed the failures to electrical differences in the replacement cables despite their matching specifications. In his November 2024 assessment of robotics, he identified reliability as a major obstacle and questioned the economics of expensive robots replacing labor. He explicitly presented these as personal opinions and qualified them by acknowledging burnout from robotics.[1]

Schluntz said using coding assistants such as Copilot increased his excitement about AI. After leaving Cobalt, he took a sabbatical, studied language models, and joined Anthropic. He cited trusted colleagues, cultural fit, and concern about AI safety as reasons for choosing the lab. His initial work included function calling and tool use, followed by coding-agent work as it became more relevant.[1]

Schluntz’s personal biography places his earlier engineering and entrepreneurial work before Cobalt: he left Harvard to start Posmetrics, which entered Y Combinator’s winter 2013 class and was later acquired; prototyped low-power electronics for Google[x]’s Smart Contact Lens project in 2014; and tested a new Falcon 9 fuel system on SpaceX’s Flight Software Team in 2015. He graduated from Harvard in 2015 with bachelor’s and master’s degrees in electrical engineering, cum laude. He describes his interdisciplinary outlook as requiring an understanding of technology, people, and business together.[2]