Why agents need reasoning, inspectability, and better abstractions

An interested candidate is only part of the job

A hiring manager opens their inbox and finds a candidate who is already interested. Sorceress automated the path to that introduction, identifying suitable people and helping contact them through a combination of human writers and automated email customization. Kanjun says the product worked well, but the team never fully made it something people could trust and rely on.

Two obstacles shaped the business and its successor: - Success could end demand. Customers who filled their roles could stop hiring and stop needing the service—a dynamic Kanjun compares to dating businesses. - Users needed reasons to trust the selection. Customers wanted confidence that candidates had been vetted and chosen for a reason. For agents pursuing longer goals, the corresponding question is whether the system has thought through the situation.

That trust problem carried into Imbue’s work on reasoning, inspectable execution, and reliable agent construction. In the October 2023 Latent Space conversation, these are complementary requirements for computers that can take on larger tasks.

Kanjun traces the ambition through three companies with Josh. He recruited her from Dropbox to Ember, where they built a laser raster projector virtual-reality headset before deciding that VR was not their strongest interest. Sorceress became a practical way to learn what deploying AI required. By late 2019 and early 2020, they were seriously considering the venture that became Imbue. Progress in scaling language models and in self-supervised learning across text and images suggested that computers could learn more independently.

The motivation is human agency. Kanjun describes helping friends and teammates identify what blocks them and building systems to improve her own effectiveness. Her computing vision extends that impulse: a machine that could handle bills, travel arrangements, and detailed work while its user spends more time creating and collaborating. The aspiration is less micromanagement of the computer.

Reasoning means revising the work—and sometimes the goal

Imbue’s central hypothesis is that reasoning is the largest blocker to agents accomplishing larger goals. Essay writing makes the requirement concrete: draft, inspect gaps, research further, reconsider the outline, and rewrite. New information may even change what the essay should accomplish. A plausible first answer is only one operation within that process.

Reasoning also determines when to interrupt the user. An agent about to book a flight should consider its uncertainty and the risk of acting on a mistaken assumption. Ask when risk and uncertainty warrant intervention is the proposed rule. Anticipating scenarios, choosing goals, making plans, and changing those plans all belong to the capability Imbue wants to train.

Imbue optimizes its underlying models for reasoning during pre-training, arguing that explicit reasoning data is relatively scarce on the internet. It also builds coding agents for daily internal use and experiments with their interfaces. Research and production serve different purposes: first make the technology usable, then determine how to productize it.

The organization works around projects rather than fixed teams, shaping responsibilities around people’s strengths and interests. Projects include data collection and engineering, pre-training and some fine-tuning, and agents built with both internal and external models.

Evaluation separates three questions that are easy to conflate: - Is the model improving? Assess the capabilities being trained. - Does a better model produce a better agent? Check whether improvements transfer to the system doing the work. - Is the agent reliable? Assess whether its behavior is dependable enough to use.

Strong reasoning still leaves substantial systems work because behavior varies between runs and remains sensitive to prompting. Imbue’s internal interface lets users inspect execution, fork it, try alternatives, and change a prompt or plan. Forking creates an alternative continuation from an intermediate state, making intervention part of the workflow.

Targeted reasoning data and Avalon’s curriculum

Imbue generates training data with its own models and other models. An interviewer questions whether this merely resamples a model distribution that already differs from human behavior. The response makes the objective explicit: concentrating training data around useful reasoning is intentional. Internet text is itself a skewed sample, and reproducing it is not necessarily the goal. Imbue generates targeted reasoning traces—explicit sequences of reasoning steps—rather than sampling indiscriminately. Its assessment that generated data approaches human-labeled quality is qualitative, without a task-specific benchmark comparison.

[Avalon](https://imbue.com/research/avalon/) is a research environment for agents learning many tasks. The team wanted to study planning and curiosity without handcrafting a separate reward for each task. Avalon uses a single reward across tasks and can generate easier or harder environments. This adjustable difficulty supplies a curriculum: a way to vary the challenges an agent encounters while learning.

In the reported Avalon experiments, reinforcement learning (RL)—learning through rewards from the environment—failed without a curriculum. The team’s intuition was that easier environments create more successes and rewards, giving the agent more learning signal. That explanation remained a hypothesis, but it suggested a useful experimental question: how can the environment provide more signal?

Agents learned hand-over-hand climbing and door-opening sequences involving multiple switches and a lever. Yet they struggled with higher-level tasks and could not consistently execute even those lower-level skills. The experience led the team to judge pure end-to-end RL inadequate for its planning and reasoning needs.

[SayCan’s use of natural-language reasoning for robot tasks](https://arxiv.org/pdf/2204.01691.pdf) helped inspire a different direction. Language mattered especially as an interface for human inspection and intervention. Even if a general end-to-end agent eventually worked, a user would still need to redirect it while it decided what to do.

A test-generation agent needs a debugger

Internal coding tools exposed the control problem in a familiar setting. A test-generation agent frequently went off track: why had it chosen a particular test, and had it considered a call to another function? Unexpected formatter behavior also needed investigation. Inspectability serves development as well as user control, giving the supervising programmer somewhere to intervene.

At the time of the conversation, the reported internal successes were limited to very small, specific tasks, such as changing a website button’s color. Several approaches could extend the models: - Critique: have models review their own work or one another’s work. - More computation during execution: spend additional inference time exploring or improving an answer. - Better context and tools: use debugging tools, context manipulation, and retrieval-augmented generation to supply relevant information. These approaches had not removed the ceiling the team encountered: limitations in both reasoning and the tools and abstractions surrounding it.

That left an unpredictable gap between building an agent and making it robust enough to deploy. Interfaces could make failures less frustrating, but users still demanded dependable behavior. The remaining work could not yet be reliably scheduled.

Imbue’s tentative product hypothesis was an operating system for agents: some combination of debugging tools, techniques applied after generation, user interfaces, models, and model-to-model interactions that other builders could use. The aim was to make reliable agents faster to construct, much as higher-level programming languages reduce work at lower levels. The product’s composition remained unsettled.

Explicit reasoning, code, and interfaces beyond chat

Imbue’s data strategy favors teaching explicit reasoning strategies. Handling confusion provides a concrete sequence: 1. Notice that something is confusing. 2. Recover the original claim. 3. Identify the evidence offered for it. 4. Check whether that evidence supports the claim.

Debugging tools can make a reasoning trace an object of experimentation. A user can change an earlier step and inspect how the conclusion changes, exploring the model’s assumptions. This interaction requires openness in both directions: the user needs ways to challenge the model and a willingness to be persuaded by it.

Code offers another form of explicit structure. Variables, their uses, and function behavior expose relationships that ordinary language can leave implicit. Imbue treats code as a possible curriculum for reasoning—an intuition rather than a measured training result in this discussion—and uses it in three ways: - A diagnostic workload: coding agents reveal where agent behavior breaks down. - Reasoning material: programs supply explicit structure for training. - A means of action: generated code lets a model act on a user’s computer.

This leads to a vision of agents as a natural-language programming language, with interfaces serving as integrated development environments for users. People could customize their computers by describing desired behavior, then inspecting state, modifying the agent, and forking its execution. The response endorses a characterization attributed to Linus Lee: chat is skeuomorphic, borrowing the familiar act of texting to make a new system approachable.

The deeper problem is a leaky abstraction: unpredictable text generation forces people building on it to keep handling its underlying variability. Imbue hopes better representations and tools can contain that variability well enough to support larger systems. Two computing analogies distinguish the work: - Analog to digital: develop abstractions that contain errors, without assuming the agent solution will take the same form as digital computing. - Xerox PARC: develop interfaces that make underlying capabilities usable by people.

A protocol allowing agents to communicate and call one another could come later. The response considers standardization potentially premature while reliable agent construction remains poorly understood. For now, multi-role critique and other reasoning-prompt techniques can be combined as modules. Combining techniques and simplifying their construction are complementary tasks: more techniques may improve reliability, while tangled prompts create a need for cleaner abstractions.

Evaluate the agent; distinguish retrieval from durable learning

Imbue uses public and internal evaluations for different purposes. Public benchmarks track particular reasoning capabilities; internal evaluations address agent behaviors for which the team finds few suitable benchmarks. The environment should likewise fit the question. Avalon supplies control over specific experiments, while Minecraft remains useful for broader agent exploration.

Voyager’s agent-created tools and curriculum earn particular praise. Letting an agent build tools and then use them offers a useful direction even though Imbue needed a different environment for its own research questions.

Memory introduces a distinction between retrieving information and incorporating experience into durable knowledge: - Preferences and situational guidance: retrieval-augmented generation (RAG), which supplies retrieved information to a model, is considered sufficient for some tasks of this kind. - Scientific expertise: an envisioned AI scientist would accumulate many small observations, often with few repetitions, and learn from them. The response does not consider RAG sufficient for that workload.

How an agent should store experience and reliably change its future behavior remains open. Imbue expects simpler agents to precede difficult, long-running ones, which may require memory breakthroughs. Its platform strategy prioritizes reasoning, inspectability, and dependable abstractions, with daily internal use checking assumptions about agent quality.

The release criterion is usefulness to other builders. Models and tools were possibilities, with their form undecided. Imbue was also exploring co-building evaluations with agent developers, who repeatedly identified evaluation as their biggest issue, and invited interested builders to get in touch.

Curiosity and goal-directed work can share a starting point

Forking suggests a way to combine curiosity with goal orientation: branch from the same execution state into different modes. One branch could pursue the assigned goal while another explores alternatives under a different prompt. This is a proposed use of forking, not a reported experimental result.

The research organization alternates those modes over time. Its stated philosophy remained largely unchanged after a $20 million Series A in late 2021: develop intermediate capabilities that can eventually enable more useful agents, mixing exploration with focused delivery.

[CARBS](https://imbue.com/research/carbs/), a cost-aware optimizer for experimental settings, grew out of Abe’s effort to automate his research process. He drew on ideas from plasma physics for its local search. Imbue describes CARBS as central to carrying results from smaller-model experiments to larger models and deriving scaling laws for multiple hyperparameters. Avalon illustrates the more goal-directed mode: a team release sprint recalled as six to eight weeks.

Returning to permanent learning, storing a lesson does not establish that a model can reliably incorporate it. The response suggests larger models may fine-tune faster and learn from fewer examples, perhaps because they already contain partial representations or memorize more effectively. No model sizes or measurements accompany that suggestion; reliably learning new abstractions remains unresolved.

Communities can make ambitious experiments feel less risky

The discussion broadens to the communities around the builders. The early community referred to as SPC is remembered as an informal space for people deciding what to do next: friends spending time together without organized programming. The Archive’s shared-living community combined selective membership with personal growth and psychological safety, choosing people others would be excited to spend time with.

Residents exchanged ways to organize their lives, understand their psychology, and navigate founder problems. Shared interests in AI also supported technical discussion. The account credits those relationships and habits with making more ambitious projects feel less risky, and describes applying similar principles within the team.

Scenius means an environment whose social interactions generate ideas and enable achievement. It captures the community-building ambition. Xerox PARC and Bell Labs serve as possible examples of that mechanism, rather than proof of what caused particular breakthroughs.

Investing adds another connection to practical agent development. Ali is credited with handling Outset Capital’s work, including events and finding potential investments. Conversations with agent founders exposed Imbue to experiments and potential customers, while switching between investor and founder perspectives helped inform decisions.

Dependable components, neglected maintenance, and careful deployment

The closing questions return to a surprising source of progress: how quickly researchers discover new capabilities in existing models and build techniques around them. The platform remains the priority, with a central unsolved problem: turn stochastic systems into components people can reliably use and build upon.

The potential payoff includes intellectual effort for neglected maintenance. Some problems worsen because people do not consider them worth the attention required. More capable computers could make it worthwhile to address those problems, allocating thought to work that otherwise remains unattended.

Safety brings the argument back to engineering controls: - Inspect and stop execution. Imbue favors systems people can examine and halt when unexpected behavior appears, with natural-language reasoning and debugging tools supporting that control. - Understand how training changes behavior. Studying what happens during pre-training and fine-tuning on individual examples complements agent debugging. - Target regulatory gaps. The stated policy position is that existing rules already cover many proposed dangers, so additional work should address specific missing protections.

Policy analysis also supplied an agent application. The team reports using agents to help analyze 20,000 pages of submissions to a Department of Commerce request for AI policy proposals, extracting problems and proposed solutions and producing a summary analysis. The department reportedly expressed interest in the tool; adoption and accuracy were not established.

The closing ambition pairs stronger reasoning with abstractions that make agents robust enough for larger tasks. Careful deployment and protective regulation belong to the same undertaking: making a more capable computer useful to the people who depend on it.