Language Agents: From Reasoning to Reliable Action

Before fighting the monster, find the sword

A text-game agent encounters a monster. Defeating it requires a sword; getting the sword requires reaching another room. The next move becomes clearer when the agent can express those dependencies before acting. This is the motivating example behind ReAct: pair reasoning about intermediate goals with actions in an environment.

Shunyu Yao began working seriously with language models when he started his PhD with Karthik Narasimhan in 2019, when GPT-2 was available. He came from undergraduate computer vision research, without a background in natural language processing (NLP). His initial proposal was to revisit his advisor’s game-related work using language models.

At the time, agent research was strongly associated with reinforcement learning in environments such as Atari and Go, while NLP centered on separate tasks such as summarization and question answering. Language agents offered a common interaction method across a broader range of environments. Zero-gradient agents assemble that behavior through prompting, tools, and chained large language model (LLM) calls, without gradient-based training of the agent.

The ReAct prototype is recalled as dating to roughly November 2021, but its early text-adventure experiments worked poorly. These games remained difficult even for GPT-4 at the time of the Latent Space conversation. The approach showed more promise during a Google internship, when it was applied to Wikipedia and the simpler text-game environment ALFWorld. A useful idea still needed a tractable environment in which to demonstrate it.

ReAct separates reasoning from the tool-call loop

Harrison Chase identifies ReAct’s appeal as its simple combination of reasoning and interaction with information outside the model. Developers already wanted models to call APIs; reliability was the problem. Early LangChain demonstrations combined search and calculator tools and made custom tools easy to plug in. Production builders reportedly tended toward their own application-specific tools, supplemented by browsing, search, and code execution.

Three meanings can hide inside the ReAct label: - A particular prompt format: the original prompting strategy. - Reasoning paired with action: explicit thinking helps determine what to do next. - An action loop: repeated tool invocations, whether or not they include explicit reasoning. The boundary of the name remains open. The enduring ideas are a general method for interacting with environments and reasoning alongside tool use. A web agent and an agent calling Python APIs can share that method despite using different tools.

Function calling does not by itself supply explicit reasoning. The argument is that an inner monologue can remain useful, especially when a tool differs substantially from those represented in the model’s training. One participant recalls better results from placing a thought field before the required tool arguments, so the model generates reasoning first. This is an anecdotal technique, without a specified benchmark or improvement magnitude.

Reflection needs a signal worth learning from

[Reflexion](https://arxiv.org/abs/2303.11366) adds self-reflection and memory to the ReAct discussion. The collaboration account credits Noah with most of the work. Its intuitive starting point is a manager’s review: language feedback can explain what went wrong and what went well, conveying more than a single numerical reward.

Reflection is presented as a language-based analogue of backpropagation: 1. Evaluate the downstream result. Obtain feedback about the completed work. 2. Trace the problem to earlier decisions. Reason about what should have changed. 3. Revise the instructions and retry. Update prompts for the relevant steps and rerun the pipeline. Textual reasoning supplies the update mechanism in this analogy, rather than gradient descent.

The evaluator is the bottleneck. In difficult mathematics without external tools, judging a chain of thought—a sequence of explicit reasoning steps—may be as hard as solving the original problem. Coding offers a more concrete opportunity: an execution error supplies something specific to diagnose and repair. Reflection can therefore be general in principle while remaining difficult to package as a universally useful loop.

Latency adds another condition. A user-facing application may not tolerate repeated reflection or extended Tree of Thoughts search. One proposal moves that computation into training-data generation: collect outputs from the more expensive methods, then train a model that uses a simpler ReAct loop when serving users. The intended improvement is a faster response path; it is a proposal here, not a measured result.

Separate what memory stores from how it is used

Agent memory involves two independent choices. First, what information should persist? - Semantic memory: knowledge about the world. - Episodic memory: trajectories or behaviors—records of what happened. - Procedural memory: reusable skills, illustrated by Voyager’s executable code snippets. Second, how should the agent use that information? It might retrieve it into context or use it for fine-tuning. The content category does not prescribe the implementation.

Two projects illustrate deriving memory from accumulated events: - LangMem’s proof of concept accepts messages into threads and processes them in the background. It stores information and extracts user-defined facts or other material of interest. - Generative Agents, also called Smallville, periodically synthesizes raw events into higher-level memories. Participants tentatively disagree over whether such reflections belong to semantic or procedural memory. Those labels help organize the design space, but do not determine how the system behaves.

A participant who tried LangMem describes it as a materialized view over a stream of logs: a stored, derived representation that compresses history into useful context. That representation is also inspectable. When a memory is wrong, a user can correct it and continue, making memory an editable component of the system.

Match reasoning cost to the task’s success criterion

The discussion of Tree of Thoughts and extended search turns on two different kinds of success: - Search tasks need a valid solution. A mathematical proof or coding solution may justify many attempts if one successful result is enough and latency is tolerable. - Reactive tasks need consistent service. Booking tickets or handling customer requests requires fast, reliable execution across many interactions. Finding one solution in a hundred attempts and succeeding on ninety-nine out of a hundred requests are illustrative objectives, not reported measurements. They demand different engineering choices.

A practical method can be assessed by implementation effort, compute requirements, the tasks it helps, the size of those improvements, and its likely relevance to future models. ReAct is described as easier to implement and less compute-intensive than Tree of Thoughts, without a comprehensive comparison of task-specific gains. Research has a separate criterion: an idea can matter for the direction it opens even when its implementation is inconvenient for an application.

Start with the simplest approach that works. Add a tool when the task requires it, or reflection when useful feedback creates an opportunity to improve. Before adding another model call, try clarifying the original instructions.

The preference for ordinary, coworker-like instructions meets a historical counterpoint: GPT-3 could be highly sensitive to exact phrasing. A proposed explanation for greater robustness is that reasoning and tool use became explicit training targets as their value became clear. Brittle prompt tricks may lose relevance, while clear communication remains a skill—including the work of editing prompts.

Better methods need better benchmarks

Method complexity should match task complexity. The criticism is that increasingly elaborate reasoning methods can outgrow the simple environments used to demonstrate them. Building better coding, web, and other agent benchmarks is presented as at least as consequential as inventing another method.

Benchmark design must balance automatic grading, practical relevance, and scalability: - Text adventures have authored rewards and authentic language, but expert game construction is expensive, limiting scale. - WebShop is described as scalable and automatically gradable, while its synthetic tasks limit practical grounding. - SWE-Bench is presented as a stronger balance because it draws on GitHub work engineers actually perform. Making evaluation easier can reduce realism; making tasks realistic can make them harder to scale.

Building such a benchmark also requires a product mindset: explain why others should adopt it, what useful difficulty it exposes, and why it matters. Academic incentives have favored method papers, but a widely used benchmark can have more practical reach than a complicated method that few people can reproduce or adopt.

Coding agents need usable environments

The coding work progresses from Intercode’s interactive execution, through SWE-Bench’s real repository tasks, to [SWE-Agent’s interface design](https://github.com/princeton-nlp/SWE-agent). Intercode starts with a familiar observation: programming involves executing code, inspecting results, using tools, and editing. Requiring a model to emit an entire program without those opportunities removes much of that workflow. Code and APIs can also expose actions beyond software development, including browsing and robot control.

SWE-Bench raises task realism by drawing from GitHub issues. The account credits Carlos and John with the idea and emphasizes that infrastructure and filtering were harder than the initial concept. A typo-only pull request illustrates why simply collecting repository changes is insufficient. Initial filtering was imperfect; subsequent work improved task solvability.

SWE-Agent exposes another bottleneck: the tool interface can fail before the planner does. An ordinary text terminal supplied inadequate feedback after edits, leaving the agent confused about what had happened. SWE-Agent’s modified terminal reports when an edit introduces a syntax error. Argument formatting and single-line versus multiline edits are also design choices that affect model behavior.

This is agent-computer interface design (ACI): treat the agent as a user whose tools should fit its capabilities and task. SWE-Agent’s initial development loop was to try an interface, inspect failures, and fix them. More systematic behavioral experiments and A/B tests could both improve usability and reveal how agents behave. The practical claim is that reliable tools can substantially simplify agent design; elaborate planning cannot compensate for poor tools.

Design inputs, outputs, and recovery together

Function calling offers a common interface for people and agents: developers get a concrete contract, while models draw on coding capabilities. The whole contract matters: - Predictable inputs: functions do one thing consistently, with types supplying additional context. - Legible outputs: compact text and clear fields make results easier to use than an undifferentiated JSON blob. - Recoverable errors: messages explain enough for the model to correct its mistake. An error message can therefore act as recovery instructions, potentially warranting more detail than a human-facing interface would normally provide.

Even field names can communicate a workflow. One participant reports better structured-output results after changing topics to candidate topics, signaling provisional ideas, and summaries to topic summaries, connecting the output to the preceding field. Participants differ over whether this anecdote illustrates distinct model-oriented design or simply more explicit communication.

Search suggests a larger divergence. A model may benefit from receiving a group of plausible results together and selecting relevant material itself. This can trade a larger context for fewer sequential interactions, compared with a human interface that reveals smaller pieces over more steps. Cost savings are proposed, not measured. Human interfaces, agent interfaces, and interfaces serving both remain worthwhile directions to explore.

Knowledge caches work; trajectories teach how it happened

Human cognition offers a reference point for agent design: a working example of intelligence to compare against, without requiring exact imitation. The comparison leads to a practical disagreement about separating knowledge from reasoning ability. One participant wants a model that interprets supplied information without relying on memorized facts, because believing it already knows can prevent a needed tool call. Another doubts a complete separation is possible: learning what things are and learning how to do things develop together.

Knowledge can be understood as a cache of intelligence. Instructions for tying shoelaces preserve the outcome of earlier problem-solving for reuse. Reflection performs a related operation: store experience, retrieve it later, and use it to improve the next attempt.

Improving the model itself raises a data problem. Internet content often preserves finished answers while omitting the searches, tool calls, revisions, and failed attempts that produced them. The priority proposed is better agent data, rather than architecture changes for which there was no clearly better alternative at recording time.

FireAct trains on a mixture of successful agent behaviors. Its procedure is straightforward: 1. Run diverse tasks using methods such as ReAct, chain of thought, and Reflexion. 2. Filter for correct solutions. 3. Train a model on the resulting mixture. The intended benefit is adaptive strategy selection: simpler methods for easier tasks and more elaborate methods for harder ones. That benefit is an expectation in this account; the stated bottleneck is a shortage of diverse, high-quality tasks.

Human demonstrations are difficult to collect, too. Some thinking may not occur in language, and expertise can hide steps inside practiced behavior. Proposals range from publishing worked university problem sets to recording people’s thoughts and computer actions. Training on successful human trajectories is floated as a possible route to general intelligence, explicitly as a thought experiment. A participant’s disagreement rests on a spiritual belief in a soul, rather than a technical demonstration.

Few-shot prompting—supplying examples of desired behavior—offers a nearer-term use for demonstrations. Examples can convey complex expectations more easily than instructions, but elaborate agent workflows are costly to demonstrate manually. Automatically collecting feedback and retrieving useful examples could support personalization. The unresolved problem is credit assignment across calls: a user judges the final outcome, while the system must determine which underlying model calls should change.

CoALA separates memory, actions, and decisions

[CoALA—Cognitive Architectures for Language Agents](https://arxiv.org/abs/2309.02427) organizes complex systems around three questions: - Where is information stored? Working memory covers shorter-lived task context. Persistent information includes the neural network, the code using it, and optional external stores. - What actions are available? External actions affect the environment. Internal actions include thinking, retrieving memories, and writing them. - How is the next action selected? A decision procedure coordinates planning and execution in an interactive loop. These dimensions provide a vocabulary for understanding agents without requiring a single implementation.

[LangGraph](https://www.langchain.com/langgraph) encodes part of the decision procedure. If an action must always be followed by another action and a check, developers can put those requirements directly in code. This makes mandatory transitions explicit instead of relying on the model to remember every dependency from a prompt. More complex workflows were reportedly driving more of this logic into code.

Memory design remains application-specific. Successful action trajectories teach how to perform tasks; remembered preferences support personalization. Either might be represented in several ways, from an external store to a persistent instruction list. The choice of tool is also unsettled: - Let agents learn to choose memory tools: offer several useful stores and allow learning to determine how they are used. - Keep developers in control: select tools for the application, much as someone might use Linear for software planning but not for a grocery list. A participant acknowledges that their team’s memory service had not found product-market fit, underscoring how immature these design choices remained.

At recording time, LangGraph persisted developer-defined state within a thread. A planned extension would persist state across threads, scoped to a user, assistant, or organization, without imposing a memory schema. The aim was to preserve developer control while useful memory structures were still being discovered.

Even if much stronger models internalized more reasoning, CoALA could remain useful as an interpretive framework. Its categories might guide investigation of memory and decision-making inside those models, rather than only describe the external system assembled around them.

TauBench tests service across many customers

[TauBench](https://arxiv.org/abs/2406.12045) evaluates agents interacting with LLM-simulated users. Its motivation is reliable customer service across many requests, complementing benchmarks focused on coding and web tasks. Handling roughly ninety-nine percent of simple interactions is presented as an aspiration, not an achieved result.

Simulating a request does not require solving it. A customer can ask for the second-cheapest flight satisfying several constraints without knowing which flight that is. The service agent has access to APIs and internal policies unavailable to the customer and must perform the reasoning. Simulator failures remain possible, but when the user clearly expresses a valid need, satisfying it is the agent’s responsibility.

Asked which applications were working, Harrison distinguishes their maturity: - Customer support: clear success through service providers and systems built inside larger enterprises. - Coding: substantial excitement, though he stops short of calling the category an established success at that point. - Research-style agents: promising work in company-data enrichment and legal applications. The research category here centers on repetitive information gathering; PhD-level research is described as a more elusive goal.

Interfaces connect autonomous work with human control

Harrison highlights a spreadsheet interface for company research: rows are companies, columns are requested attributes, and each cell represents an agent’s work. A person can try a few cases before launching a batch. Another participant suggests using the first completed rows as few-shot examples for later work. The same interface supports testing, demonstrations, and larger runs.

Background agents raise a different design question: when should an agent interrupt the person? An email-assistant example triages messages, attempts responses, and asks for decisions such as whether to accept a podcast invitation. Its described stack uses LangGraph for orchestration and LangChain for model integrations. The central interaction is the handoff: background work needs a way to request human judgment, including decisions about when, how, and how often to ask.

LangGraph Studio makes a coded agent available for visual inspection and testing. Described as an integrated development environment for agents, it takes graph-defining code, displays the graph, and lets users interact with it. Persistence supports time-travel functionality. LangGraph itself is explicitly described as not low-code: developer control remains central to reliability.

The proposed team workflow divides responsibility: - Engineers define the decision procedure: the cognitive architecture stays in code, alongside initial prompts. - Collaborators adjust prompts and configuration: the visual interface helps them understand behavior and control those parts of the system. Interfaces for building agents and interfaces for using them may overlap. The closing opportunity is to discover those interactions while conventions remain immature—an interest also reflected in an informal AI user-experience meetup for people experimenting with interfaces.