Harness Engineering: Delegating the Whole Development Loop — Part 1 of 2

When the agent cannot assemble the feature

Early versions of Codex CLI, running Codex Mini, could not reliably assemble the product features Ryan Lopopolo wanted. His self-imposed constraint was to write none of the code himself. When a task failed, he had to break it into smaller pieces the agent could build and recombine. He estimates that the first month and a half was ten times slower than doing the work himself.

That investment anchors this Latent Space conversation: delegation requires an environment in which the agent can finish the job. Lopopolo attributes the later gains to building tools for the entire software development lifecycle. His engineering loop starts with two questions: where does the agent make mistakes, and where does a human repeatedly intervene? The next step is to automate that work and build confidence in the result.

At the interview, Lopopolo described his team as Frontier Product Exploration, developing enterprise products using OpenAI models. He reported having no internal rate limits. He also affirmed a host’s summary of five months of work, more than a million lines of code, no human-written code, and an estimated tenfold speedup over doing the work by hand. That speedup is a counterfactual estimate without a stated measurement method; the initial slowdown and unusually generous model access are part of its context.

Keep builds within a one-minute maintenance budget

The environment had to adapt as models and harnesses changed. Lopopolo reports that the Codex harness used with GPT-5.2 lacked background shells, allowing the team to rely on blocking scripts for long-running work. Background shells let the agent continue working while a command runs—reviewing code during an expensive build, for example. With GPT-5.3 and that capability, he found the agent less willing to wait.

The response was a build-system overhaul. Over one week, the team moved from a bespoke Makefile build to Bazel, then Turbo, then Nx, stopping when builds were fast enough. Agent productivity drove the choice; this was not a general comparison of the tools.

One minute was a maintenance budget, not a timeout. Exceeding it signaled that the team should decompose the build graph and recover the fast development loop. The number was convenient and achievable; crossing it did not kill a command. Lopopolo argues that parallel model work makes continuous maintenance practical: preserving properties such as build speed reduces variation in the code and development process, giving subsequent work more dependable foundations.

Give the agent runtime evidence and durable guidance

The scarce resource is synchronous human attention. Early generated code needed close supervision because the agent lacked building blocks for modular, reliable software and a functioning frontend. Observability—access to runtime evidence such as logs, metrics, and traces—gave it a way to investigate its work without requiring someone to remain at a terminal.

Adding the local observability stack reportedly took roughly half an afternoon of human time, using Victoria Stack binaries written in Go and a little Python glue. The consequential design choice was to start Codex, then let it boot the supporting environment. Skills and scripts explain how to launch the stack and configure the application to use it. This gives the reasoning model options and enough context to choose how to proceed.

Repository guidance uses a roughly 100-line table of contents pointing to smaller documents. A Markdown-table tracker prompts Codex to compare business logic with documented guardrails and propose follow-up work. Initially, those tasks were Markdown notes that scheduled agents could work through.

A missing network timeout shows how a local fix can become durable guidance. Alongside fixing the incident, a developer can mention Codex in the Slack page and ask it to document a requirement for network-call timeouts. The same rule can guide code authorship, generated tests, and automated review. A host raises the corresponding risk: a rule adopted after one failure may overlook legitimate exceptions and later require revision or rollback.

Make review converge; keep release authority explicit

Automated review introduced another failure mode. Local Codex wrote a change and pushed a pull request; updating the PR triggered a review agent. The authoring agent had to acknowledge and respond to feedback, but excessive deference sometimes produced a revision cycle that never converged.

The team changed instructions on both sides: - Reviewers should favor merging. Their prompts encouraged bringing the change to completion. - Authors may challenge or defer feedback. Scope-expanding suggestions can become backlog work instead of prerequisites for the current PR. The prompts made a human convention explicit: responding to feedback does not require implementing every suggestion immediately.

Lopopolo says most human code review had moved after merge. Agents worked across product code, tests, continuous integration (CI), release tooling, documentation, evaluation harnesses, repository scripts, and dashboard definitions. Autonomous merging still had human release gates: the native application did not use continuous deployment, a human cut the release branch, and distribution required a human-approved smoke test.

These practices were developed in a completely new repository for a customer-bound application, not infrastructure with demanding uptime targets. The team explicitly cautions against assuming they apply generally.

Own the diagnostic loop and delegate the landing

Codex authors and publishes Grafana dashboard JSON and responds to pages. Keeping dashboard definitions, alerts, and application logging together lets it trace an alert to the exact log that triggered it. If an outage produces no page, it can inspect the existing coverage and repair both the dashboard and underlying instrumentation.

Human oversight is compared to being a group tech lead for a 500-person organization: sample merged code, identify recurring struggles, and improve shared foundations. One such foundation is a command base class that supplies tracing and metrics to repeatable business logic. Ensuring that code uses this primitive matters more than prescribing each function’s internal structure. The application also maintains deliberate boundaries between Electron’s main and renderer processes.

The discussion broadens this approach to AI products: if a user journey can be expressed as code, the Codex harness—the machinery connecting the model to its working tools—can supply the wiring. Prompts direct the work, and scripts expose actions an engineer might otherwise perform. A participant describes this as starting with a coding agent and extending it into non-coding knowledge work; the responding speaker agrees.

Pull-request landing makes complete delegation concrete. The team uses worktrees heavily and argues that merge conflicts matter less when an agent can resolve them without synchronous human involvement. Its `$land` skill carries a change through to `main`: 1. Push the PR and wait for human and agent reviewers. 2. Wait for passing CI and address flaky failures. 3. Merge upstream changes as needed to resolve conflicts, then wait for checks again. 4. Enter the merge queue and keep addressing flakes until the change reaches main. The described local workflow still requires keeping the laptop open.

This gives harness engineering a practical definition: encoding non-functional requirements—reliability and quality standards beyond the requested feature—into documentation, tests, review agents, and lints that reach the coding agent. A lint error should explain how to correct the problem. Agent mistakes become signals to extract the tacit knowledge an engineer would normally teach a new hire.

Reconsider dependencies and human debugging interfaces

Bringing dependency code into the application’s own repository extends the same ownership model. The discussion identifies both its appeal and its limits: - Smaller, directly editable implementations. One participant estimates that a dependency of a few thousand lines could be internalized in an afternoon, with unused generic functionality removed. They limit current feasibility to low-to-medium complexity. - Less upstream coordination. Codex can review and change internalized code directly, avoiding the wait for upstream patches, releases, and compatibility checks. - Confidence must be rebuilt. Mature dependencies carry scale testing, security scrutiny, and lessons from past failures. Participants agree that internalizing one can mean relearning mistakes already addressed elsewhere.

A debugging anecdote raises a related question about interfaces. After the app reached its first dozen internal users, performance problems prompted users to export trace tarballs. An on-call engineer spent an afternoon with Codex building a local Next.js app that visualized the trace when someone dragged in the archive.

Before building a human debugging interface, ask who will interpret the evidence. The speaker argues that Codex could have inspected the raw tarball directly, avoiding a human handoff before the agent wrote the fix. The suggested five-minute investigation is an estimate, not a measured comparison, but the workflow question remains: does the visualization help the investigator, or merely preserve a familiar way of working?

Symphony: reconstruct the system from a specification

Symphony coordinates agents working on tasks and drives them toward completion. It was distributed as a specification detailed enough for a coding agent to reconstruct a local implementation—a form of distribution the discussion calls a ghost library.

The specification itself was developed through a reconstruction loop: 1. Have Codex use the proprietary repository as a reference to write a specification in a new repository. 2. Start a disconnected Codex instance in tmux to implement it. 3. Have another instance compare the implementation with the original and revise the specification to reduce divergence. 4. Repeat toward a specification that reproduces the system with high fidelity.

An unresolved evaluation question follows: can an agent describe a system it cannot implement? One participant judges that work that is both hard and new still needs human direction, while other combinations of difficulty and novelty are largely manageable with suitable scaffolding. The human work they want to preserve includes deep refactorings where the right interfaces are still unknown. This is a capability judgment, not a benchmark result.

The model reportedly chose Elixir for Symphony’s implementation because its native process supervision suited the orchestration problem: run a small daemon for each active task and drive it to completion. That choice became especially useful in the coordination layer, which the team found difficult to get right.

Symphony’s origin was another human bottleneck. The team reports moving from about 3.5 PRs per engineer per day at the end of December to 5–10 after the holidays with the model referred to as “5.2,” without other repository work. These are reported throughput figures; PR size and quality were not specified. Engineers became exhausted switching among active tmux panes to keep agents moving. Experiments with dev boxes and automatic agent startup aimed to remove that supervision. Opening Linear twice a day to approve or reject work remained an aspiration.

Rework starts a fresh attempt. When Symphony escalates a PR for human review, the intended decision is simply whether it is mergeable. Moving it to rework causes the Elixir service to discard the worktree and PR. Before returning the ticket to progress, the human should identify what led the agent astray and correct the guidance. Delegating authorship makes discarding the implementation less emotionally costly.

Faster execution creates a shared-understanding problem

The team consulted with the Codex team on the app, skills, and automations. Product support for those capabilities let it retire some custom infrastructure and hand-built control loops. Rapid experimentation also made it harder for humans to keep track of the codebase.

The Playwright integration illustrates the tradeoff. Its original tool connection injected definitions for many more calls than the workflow needed. One speaker reports that this consumed context and interacted poorly with automatic compaction, sometimes leaving the agent forgetting tool usage. Someone replaced the connection with a local daemon and a small command-line interface (CLI). The speaker did not notice the implementation change because Codex continued to drive the Electron app.

Less terminal supervision did not eliminate human coordination. The team reports a 45-minute daily stand-up to share the current state. Its roughly 500 npm packages for seven people reflect an attempt to partition concurrent work and maintain clear interfaces. The comparison to each person directing 10–50 people’s work explains the architectural choice; it is not a measured productivity multiplier.

The team has not found a general solution for collaboration among multiple humans and agents. Its current approach combines two forms of consistency: - Consistent packages reduce exploration. Shared structures, languages, and internal patterns let an agent understand its task without repeatedly investigating other directories. - Shared skills pool engineering preferences. The repository has about six skills. When part of the development loop is missing, the first attempt is to incorporate it into an existing skill.

Session logs feed process improvements back into the repository. Codex can inspect its own sessions and suggest better tool use. The team collects logs into blob storage and runs daily agent loops over them so improvements can benefit everyone. PR comments and failed builds serve a similar purpose: the speaker treats them as clues that relevant context was missing and should be available to future runs.

Provide complete access and compact feedback

An agent with ticketing access can file its own follow-up work when instructed. One speaker advocates full access within the agent’s domain; another describes a bounded environment containing everything it needs. Their common requirement is context, tools, and access to the results of its own actions. Running supporting services locally helps complete that feedback loop while reducing cloud dependencies.

Command-line interfaces are favored because models can work with shell commands and text. A related suggestion for interface work is to supply a layout rendered as text characters alongside the image, giving the agent another representation of the object it manipulates. This is a proposed technique, without a measured advantage.

Technology choices should reduce what the agent has to implement. Elixir’s process supervision fits coordination; shared frontend and backend TypeScript types are offered as a way to reduce full-stack complexity. When the agent authors the implementation, the human’s ability to personally write Elixir need not determine the language choice.

For policy, one proposed simplification is to give the agent the GitHub CLI and an instruction that CI must pass. The speaker argues that this can replace substantial custom waiting logic and make the system easier to maintain.

The complementary interface rule is show the failure that requires action: - Build logs: extract the actual exception from the wall of output, as a developer productivity team would for a human. - Formatting checks: suppress routine messages about already formatted files and expose whether correction is needed. - Test runners: the team wrapped recursive pnpm execution to emit failing portions instead of the large volume of passing-suite output.

Make the blueprint specific, then let it evolve

Symphony specifies Linear and GitHub, but those integrations are replaceable; Jira and Bitbucket are offered as alternatives. Concrete identifier formats and agent execution loops provide a complete starting system. Specific instructions make the blueprint buildable; they need not make it permanent. The intention is to get a working system running and then adapt it. The speaker reports that strict success criteria improve deployment success, without giving a rate or comparison.

The closing exchange returns to the temptation to supervise every run. One participant describes a deep research request about “LLM” that was interpreted as a legal topic, producing an off-target report roughly an hour later. They advocate tolerating discarded attempts and sometimes launching several alternatives. Another speaker emphasizes the durable repair: encode the discovered ambiguity in the task’s guardrails, so subsequent execution receives better instructions.

Part 1 ends with a question about customer sentiment. The response is that the speaker thinks Symphony has no customers at that point: it is something they have put into the world, with customer feedback still to come.