The Agent Reasoning Interface: From Shared Documents to Delegated Work

A workplace assistant needs room to work

Early Claude in Slack could summarize a thread or suggest ideas when someone tagged it. But Slack’s interface constrained the features its builders wanted to add. Karina Nguyen recalls wanting richer interactions without becoming dependent on the platform. Making the assistant more useful meant reconsidering where people worked with it.

In this Latent Space conversation, Nguyen describes her OpenAI team’s work as combining human-computer interaction with model training. Canvas, her first project as a tech lead, applies that combination to writing and coding. Improving the experience involves work from reinforcement learning and synthetic-data generation through deployment and product design.

Nguyen’s route into AI began with computer vision for investigative journalism at Berkeley, working with the Human Rights Center, journalists, and Berkeley AI Research professors. At The New York Times, she built mobile storytelling prototypes. Her early experiments also included fashion recommendation search using CLIP. After an initial rejection when Anthropic was not hiring for the roles she wanted, she applied to a frontend opening and joined around August 2022, combining engineering with product design.

After ChatGPT launched, her manager challenged her to reproduce a similar interface in two weeks. Nguyen identifies that challenge as the origin of Claude.ai and says she wrote roughly its first 50,000 lines of code. Her retrospective lesson concerns the estimate: she should have requested twice as much time. Two weeks was the proposed deadline, not a confirmed delivery time.

The document that writes back

Claude Workspace, explored in 2023, offered another interaction: a person and Claude working inside one document. Its inspiration was the responsive diary in Harry Potter—a person writes, and the document answers. The exploration focused on writing; Canvas would also make coding a central use case.

Its relationship to later Artifacts and Projects remains uncertain. After the contributor moved from product engineering into research, the growing product team developed its own ideas. The retrospective explanation for why shared workspaces did not arrive sooner is tentative: priorities were unclear, the AI product landscape was young, and relatively few designers or researchers were exploring these interactions.

Post-training is behavioral debugging

Post-training—the further training used here to refine a model’s behavior—creates its own debugging workload. A Claude 3 contributor recalls a team of roughly 10–12 people, with their own work concentrated on Haiku, evaluations, and the model card. One informal recollection puts the candidate count around 70, each with different defects to investigate.

Dataset interactions make those defects difficult to predict. Contradictory training datasets can produce unexpected side effects; understanding their interactions remains an open research problem. The practical response borrows from software engineering: detect failures, debug them, apply targeted interventions, and iterate quickly.

An interviewer questions how that care can coexist with speculative training experiments, dubbed “YOLO runs.” The response treats them as a compute-allocation choice. A well-resourced team might afford two or three; a constrained team must prioritize its most important experiments. Research management means choosing bets, testing them quickly enough to build conviction, and committing more resources when they work.

Evaluation results depend on the setup. A speaker recalls averaging five runs of a benchmark because its results varied substantially. Comparing published scores also required checking model-card appendices for matching settings. A shared evaluation harness—the software that prompts models and scores their answers—does not remove every difference: extra newlines can break parsing, while requiring XML may penalize formatting ability alongside the capability being tested. Adapting the format for each model changes the comparison conditions.

Reasoning needs constraints—and verification

For o1, the practical prompting advice is to state hard constraints explicitly. The respondent finds that clear requirements help the model select candidates that satisfy them. Protein properties and chemical binding requirements are offered as possible applications, rather than demonstrated scientific results.

A participant describes using o1 for AI News workflows with many prompts, steps, and criteria, finding that it checks through them systematically. That experience motivates a proposed evaluation for multistep instruction following. External experimentation matters here: the respondent learns from users tackling difficult coding tasks and personally explores model-generated, or synthetic, data with o1, while acknowledging that they have not exhausted its uses.

This leaves room for engineers outside model labs to discover useful behavior that the creators did not explicitly train for or fully understand. But stronger answers also raise a harder question: can the user verify them? Without relevant expertise, a person may be unable to judge a difficult scientific or technical answer. Helping people check complex reasoning outputs is framed as an alignment problem, with external feedback offering a possible learning signal.

Turn values into specific behavior

Behavioral design specifies how a model should act in context: its tone, persona, and use of follow-up questions. A model collaborating on a document needs different behavior from a conversational assistant. It must decide whether to ask for clarification, leave a comment, or change the work directly.

The starting point resembles designing a video-game character: establish core principles. Claude’s helpful, honest, and harmless goals provide that foundation, but the goals can conflict. Avoiding harmful responses should not lead to routine refusals of innocuous creative-writing requests. Training data must demonstrate how to balance those values in concrete situations.

The connection to Constitutional AI is principle-guided generation of training responses: create completions that embody chosen values, then express those values across varied scenarios. Honesty needs many concrete forms if the behavior is to generalize. Dataset construction is described as partly a craft, even when its effects can be studied empirically.

Deliberate design can also begin with an accidental strength. Users praised Claude 2 for its writing and personality before the team had a clear method for making those qualities dependable. That feedback prompted more intentional investment in Claude 3’s behavior, alongside a goal of improving factuality for chat and enterprise use.

Canvas: what belongs in the model?

The Canvas team came together around a July 4 company break, with Thomas Dimson credited for the initial engineering prototype. Designers, engineers, product managers, and researchers worked together from the start. Their shared responsibility included deciding which failures called for model training and which belonged in application code.

Development began with a prompted baseline: an existing ChatGPT model instructed to treat Canvas as a tool. Its failures exposed decisions the team believed required post-training: - When to open Canvas. - When to comment instead of edit. - When to rewrite the whole document. - When to change only the requested section. The team retrained 4o with Canvas-specific work and shipped a separate beta model to gather feedback while broader integration proceeded. The recalled interval from beta to general availability was roughly three months.

Writing quality had two tracks: improve established nonfiction tasks such as emails, blog posts, and cover letters through better data; and pursue longer-term research into creative thinking and novel writing forms. Internal human evaluations used prompts representing the tasks the team cared about. Model writers and designers developed task-specific rubrics—what makes an email good, for example—and judged outputs against those criteria.

Version differences complicated users’ impressions. One participant preferred Canvas to the 4o API for transcript-based podcast write-ups, finding the API wording formulaic, but had tried the comparison only about three times. The respondent was uncertain about exact parity between Canvas and core 4o and personally found the original Canvas beta more creative than the version available during the conversation. These were experience-based judgments, not controlled rankings.

The shared document also changes how users recover from disappointing output. A participant describes drafting conference copy, making direct edits, and telling the model what they dislike. For a humorous blog post, they supplied examples, rejected the generated continuations, and asked for self-critique and another attempt inside the document. Their interpretation of Canvas reverses Google Docs with Gemini: the conversation is central, with the document beside it, rather than a document that occasionally calls in AI assistance.

Further priorities included easier formatting and editing, plus executing code inside Canvas. The broader ambition was an interface that adapts to intent, becoming a writing environment or coding environment as needed. That ambition makes understanding the request part of interface design itself.

Code diffs, which describe changes rather than reproduce a whole file, proved unexpectedly difficult. The team weighed improving diff generation against full-document rewrites, described as more accurate without a numerical comparison. Combining tools introduced further cases: selecting a passage and requesting search, using Canvas with DALL-E, or choosing between Canvas and Python-based advanced data analysis. Testing did not cover every ChatGPT feature; deciding which experience the user expected was itself a difficult model behavior.

Tasks repeats the product-research loop

Tasks addressed an unmet expectation: users were already asking ChatGPT for reminders when reminders did not work. A resident led the project, while the participant recounting its development supervised and delegated. The team reused Canvas’s process, bringing designers, engineers, and researchers together for evaluations and testing. Development took roughly two months in the recollection.

The proposed value was scheduling combined with a general model’s abilities: recurring assistance could involve search, writing, or Python puzzles. These examples expressed the direction, rather than confirming every integration. Tasks initially shipped as a separate beta model because that made deployment and iteration easier; integration into the core model was still underway.

The development procedure was repeatable: 1. Build a prototype with a prompted baseline. 2. Create evaluations that capture desired behavior and expose meaningful baseline failures. 3. Iterate on training to address those failures. 4. Run broader evaluations to check for overfitting and regressions. Better feature scores are insufficient if other capabilities deteriorate. Comparing evaluation datasets against a baseline helps, though an interviewer points out the difficulty of catching regressions in areas whose data the feature team does not own.

At the time, requests containing three tasks remained difficult, and recurring actions required explicit instructions. The proposed next step was to notice recurring behavior and offer help—for example, suggesting a daily article selection based on someone’s Hacker News interests. Organizational possibilities included processing user feedback and simple data analysis, returning to the idea of an assistant working inside a company.

Collaboration before delegation

A tweet read during the interview proposes a progression from one-off actions to collaboration, then trustworthy long-horizon delegation: handing over work that unfolds across an extended sequence of actions. The respondent treats this as an intuition, not a formal definition of agents. As with human colleagues, collaboration reveals working styles, communication habits, and implicit preferences. Consistent performance builds the trust needed to delegate more.

Missing context gets more costly as work gets longer. The failure scenario is an agent spending five minutes in the background and returning an unwanted answer because it failed to clarify the request. An agent should recognize missing information, ask a follow-up, and use the answer before proceeding. Knowing when it has enough context to succeed remains an open research question.

The computer-use discussion took place just before Operator launched, with the interviewer introducing OpenAI’s work as a rumor rather than a disclosure from the respondent. Operating a computer is presented as a core capability for delegated actions such as ordering a book or booking a flight. A related proposal would extend Canvas into asynchronous work: ask a model to develop a feature and test it in its own virtual environment.

One participant questions flight booking as the flagship example, preferring a bot that scans Facebook Marketplace for free items and arranges to get them. Their broader skepticism concerns speed, cost, and accuracy. Full operating-system interaction goes beyond a coding sandbox or browser: the agent must target controls precisely and understand the active interface. The response proposes pairing computer-use agents with increasingly capable small models, including o3-mini, o1-mini, Claude 3 Haiku, and Claude 1.2 Instant, to reduce latency. That benefit remained a prediction.

A monthly expense report makes the challenge concrete. Expensing lunch requires checking a calendar for attendees, finding and uploading a receipt, and tagging the right person in the expense system. Other entries require Uber or receipt photos in the camera roll. Each action is simple; completing the workflow requires access and coordination across applications. The participants regarded it as tractable but not yet achieved, hoping it would become possible by the end of 2025.

Demonstrations could make these workflows personal: show an agent how you prefer to book a flight or complete a task, then let it learn that procedure. Useful personal assistance would also need access to context such as calendars and email. Both ideas extend the collaboration argument: learning how someone works is part of learning what to do for them.

Choose the interface as well as the answer

The closing product vision is a task-oriented generative operating system: a model chooses an interface suited to the request, potentially generating a React app instead of prose. An Apple stock query returning a chart illustrates the shift. Adapting visualizations, colors, and other presentation preferences could take it further—from a personal computer toward a personal model. Arbitrary personalized interfaces remain an ambition in this discussion.

The organizational comparison separates product strategy from research practice. OpenAI is described as more willing to explore product bets, Anthropic as perhaps more enterprise-focused, and daily experiment execution as more similar than different. A lesson credited to Mira is to connect product, research, and safety through a systems perspective. Another is to change direction and reallocate resources when empirical results warrant it, rather than spend years persisting with an unsuccessful idea.

The hiring invitation at the time sought research engineers and scientists who could train models and wanted to deploy them into products. The broader closing challenge was for designers to experiment directly with models. Human creativity is presented as a bottleneck in rethinking software: exploring an interaction can reveal a useful product behavior, and product-oriented research can train the model to perform it.