Harness Engineering, Part 2 of 2: Evidence, Context, and Control

A pull request should arrive with evidence

A coding agent opens a pull request. What should it provide so a reviewer can decide whether the code is mergeable? In the harness-engineering conversation, a Symphony practitioner proposes a demonstration video attached to the change. The goal is to compress the execution trajectory into legible evidence: show why the result is good without making someone watch the entire coding session.

That expectation supports a broader workflow: let the agent execute independently, then require evidence that earns trust in its output. The practitioner reports a further fivefold productivity improvement using Symphony, though no baseline or measurement method accompanies the figure.

Codex using FFmpeg is offered as a practical way to produce these review artifacts. Early computer-control experiments made the delegation tangible: windows opened, captures happened, and files appeared on the desktop while the human occasionally moved the mouse to keep the machine awake.

Match model speed to the task

Spark is described as a fast, distinct model, with a different architecture and no image support at the time of the conversation. Its speed had not made it a substitute for extra-high reasoning in the practitioner’s workflow: on tasks normally assigned to that reasoning setting, Spark reportedly went through three context compactions before writing any code.

Another participant highlights 5.4’s million-token context window, arguing that running longer before compaction helps agents perform better. No benchmark accompanies that assessment. The practitioner remains undecided about Spark’s broader deployment but identifies narrower uses for its speed: - Quick exploration: prototype an idea or try an approach. - Documentation updates: make focused changes rapidly. - Turn feedback into lint rules: in a codebase with existing lint infrastructure, encode a correction as an automated check.

Missing requirements keep humans in the loop

Two kinds of work still demand substantial steering in the reported workflow: - New products without existing screens: translating an idea or mockup into an interactive prototype. - Difficult refactorings: making changes that require frequent interruptions and tooling to help decompose monoliths.

Another participant distinguishes understanding an existing codebase from scaffolding a new product. They describe Codex as strong at the former, while Lovable, Bolt, and Replit address idea-to-product workflows. For the practitioner, starting from scratch exposes a requirements problem: some missing decisions become apparent only while the agent works. Live interaction supplies context that was absent at the start.

A harness—the tools and workflow surrounding the model—can narrow that uncertainty in several ways: - Elicit requirements: help the human articulate missing context. - Constrain framework choices: reduce the number of implementation decisions left open. - Supply templates: give the model a starting point that carries additional requirements. These interventions provide nonfunctional requirements, such as framework constraints, and reduce the range of possible outcomes. The practitioner expects models to handle increasingly complex work; the surrounding workflow should remain useful as human attention moves to the next bottleneck.

Frontier: deploying agents with organizational control

Frontier is presented as a platform for deploying observable, controlled, identifiable workplace agents. Its stated integration goals include companies’ existing security and workspace tools. The full roadmap remains undisclosed; the emphasis is on giving organizations the infrastructure to deploy agents successfully.

Agents SDK is described as a core building block: a software development kit intended to provide a working default harness for startup and enterprise builders. It brings together shell tools, the Codex harness, file attachments, and containers, with an emphasis on safe composition.

Useful defaults need company-specific controls. The GPT OSS safeguard model is offered as an example: it supports a safety specification that can encode an enterprise’s particular concerns, including internal code names and risks of company data leaving authorized boundaries. The platform goal is to expose customization hooks without requiring every customer to assemble everything from scratch.

The agent-management concept includes a dashboard for inspecting individual agent instances. Frontier serves two distinct audiences through different surfaces: - Employees using agents: people working through the available interfaces and connectors. - Teams overseeing deployment: IT, governance and compliance functions, AI innovation offices, and security teams use the dashboard to support oversight tied to regulatory requirements and customer attestations. Inspecting a trajectory—the course of an agent’s work—supports both security and skill improvement. The same activity record can help an operator investigate behavior and a developer improve how the agent works.

Agents need the business meaning behind the data

An internal data agent reportedly uses Frontier technology to expose the organization’s data ontology: context for understanding what its warehouse contains. Access to that context still leaves an organizational problem. People disagree about definitions of revenue and active users; marketing and sales attribution can add up to more than 100% when teams count differently.

Data closes the product feedback loop only when agents can understand the relevant business context. The discussion connects work beyond coding to definitions of revenue, customer segments, and product lines. Those facts also inform software decisions, so they belong in the agent’s working context.

The team’s corebeliefs.md makes that context explicit. It reportedly records team membership, the product being built, end customers, pilot customers, and the vision for the next 12 months. The rationale is straightforward: give agents the business context that guides their human teammates.

Team context extends to humor. The team maintains skills for generating deep-fried memes and uses Codex with the Slack ChatGPT app to post on a team member’s behalf. The practitioner suggests a related experiment: ask Codex to review your coding-agent sessions and roast you.

Shared oversight and knowledge beyond one tool

Returning to deployment, a participant argues for one shared view of long-running company processes, using Temporal as an analogy. Such oversight could help stakeholders approve agents at scale. The dashboard discussed might be a demo rather than a production screenshot; the participant is explicitly unsure. The broader proposal also anticipates substantial customization because enterprises need different controls.

The responding practitioner describes the building blocks used to assemble agents as products themselves. Frontier is said to expose controls to steer agents and revoke authorization if a model becomes misaligned. Further stakeholder-facing capabilities are planned to support wider deployment.

The team also collects and distills Codex trajectories into a knowledge base, currently reflected back into its codebase. The ambition is to share that knowledge beyond Codex so ChatGPT understands the product and the team’s working context. Distilling trajectories is an existing practice; sharing that context through Frontier remains an aspiration.

Invest in checks that survive better models

Harness engineering faces a continuing tradeoff: add external support, or train the model to handle more by default. The practitioner’s success criterion is better engineering judgment from the model, with surrounding mechanisms that do not degrade its performance.

That yields a practical distinction: - Tests and guardrails expressed in code work with the output Codex already produces and support reliable software development. - A separate scaffold that restricts output is more likely, in the practitioner’s view, to need replacement as models improve.

Another participant compares this to on-policy versus off-policy reinforcement learning. The intended harness-design analogy is to start with behavior the model already produces and modify from there. The practitioner agrees; this is a design intuition, not an experimental result.

The release of 5.3, Spark, and 5.4 over roughly a month makes that investment question immediate. The participants praise the pace without committing to a future monthly schedule. Their closing technical ambition is broader knowledge work: once a coding harness works, use it to expand what people can accomplish.

Closing: adoption, hiring, and applied AI

The closing invitation is to try the Codex app and consider joining Frontier. At recording time, a participant reports that Codex had passed 2 million weekly active users, growing 25% week over week. Neither the measurement method nor the duration of that growth rate is specified. Frontier was hiring, with substantial work remaining to serve enterprise customers.

The Bellevue office had held its grand opening the previous day. A participant describing themselves as one of OpenAI’s first Seattle engineering hires points to durable products and business lines developed there alongside new-product work, and notes a substantial engineering presence in New York. Their account of applied AI centers on finding where models can be deployed successfully.