The AI Coding Factory: From Code Completion to Delegated Engineering

A Python loop looking for the right problem

An early experiment put a large language model in a loop: write Python, execute it, and use feedback to continue. Built for finance, it seemed more promising as a software-development tool. Meanwhile, Hugging Face customers were asking one of Factory’s future founders for models fine-tuned on their own codebases. These experiences suggested an opportunity to build systems that could work with an organization’s software and check the results of their actions.

[Factory](https://factory.ai/) builds Droids for delegating software engineering, from implementation to incident investigation. In this May 2025 conversation, co-founders Eno Reyes and Matan Grinberg explain how they arrived at that scope. Both attended Princeton, but their first substantial conversation came at a LangChain hackathon in 2023. They describe spending the next 72 hours building together; within eight days, one had left his PhD and Eno had quit his job.

One founder had moved from theoretical physics and string theory into AI research, drawn by the belief that coding capability was fundamental to broader model intelligence. Code’s verifiability offered a concrete reason to pursue it: a model could generate output, execute or validate it, and receive feedback grounded in what that output actually did.

The founders did not think the available model—recalled only as “3.5”—could support fully autonomous engineering. Their bet was on the harness, the surrounding software that supplies integrations, information, execution, and feedback. Better reasoning and larger context windows would bring more tasks within reach. They also expected good models to become cheaper, while the best models might retain their price and become more capable.

Delegation changes the product constraints

Factory’s target was the full software development lifecycle in enterprises, including work on old, interconnected codebases. The founders offer a COBOL migration as an example: potentially valuable to developers, even if it makes an unremarkable demo. Their interest was in the engineering work surrounding code generation as well as the implementation itself.

Collaboration and delegation impose different constraints. An assistant inside an integrated development environment, or IDE, must respond quickly enough to stay useful during editing. Free or roughly $20 monthly plans also limit the computation a product can spend on each outcome. Delegation permits longer execution and more computation while the human does other work. The founders describe handing off whole tasks as their enterprise ambition.

The company began as the San Francisco Droid Company, before legal advice prompted a rename. “Factory” came from wordplay around an actor-critic research paper: substituting “actor” into F(A) = Y suggested the name. Droids remained, partly because “agent” then evoked unreliable systems taking actions in endless loops without guidance.

Early Droids were asynchronous, event-triggered workflows with specific purposes and partly predetermined execution paths. By the interview, the team described more open-ended agents guided by planning, decisions, and feedback from their environment. That broader autonomy did not imply replacing a whole person. The founders expected the outer loop—planning, talking with colleagues, and deciding what matters—to remain human-driven, while the inner loop of writing code and documents became increasingly delegated.

The demo begins with context and questions

Factory’s dashboard presented a general-purpose Droid alongside three recurring use cases: - Knowledge and technical writing: investigate with search and other tools, produce a document or answer, then refine it conversationally. - Coding: delegate an implementation task. - Reliability: supply an incident description, gather evidence, and produce a root-cause analysis. The team says the popularity of incident response surprised it: gathering evidence during an overnight outage was work users wanted to hand off.

The live coding task begins with a pasted ticket and an @ reference to Factory’s codebase. The Droid searches the repository, summarizes its findings, proposes an initial plan, and asks clarifying questions. During execution, the interface emphasizes an activity log and a changing context panel; code review becomes the focus once the work is complete.

The presenter’s answers mix explicit constraints with delegated decisions: imitate existing interface examples; show previews only after a button click; choose the displayed fields yourself. This addresses a common failure mode in delegation. An agent fills in missing assumptions, then disappoints a user whose actual constraints were never communicated. Factory wants the system to recognize ambiguity and ask useful questions without requiring carefully engineered prompts.

Those questions sit alongside context gathered from outside the repository. Factory reports integrations with Linear, Jira, Slack, GitHub, Sentry, and PagerDuty, plus memory and web search. In the demonstrated configuration, only search is enabled initially; the Droid may request additional tools as it works. The founders compare code-only access to onboarding an engineer without team conversations, documentation, issue history, or operational information.

Factory also generates a project overview during codebase indexing: development-environment setup, repository structure, and connections between important modules. It generates finer-grained insights throughout the repository to reduce the setup information users must supply. Memory and .droid.yaml hold additional guidelines. The system also ingests other providers’ rules files, filtering advice it considers redundant.

Evaluate behavior as well as code

A model upgrade can improve capability while disrupting a familiar workflow. Factory says users noticed behavioral changes after its move from Sonnet 3.5 to Sonnet 3.7. The application therefore faces an unresolved choice: absorb those differences to preserve familiar behavior, or teach users a new interaction pattern when the model warrants it.

Factory evaluates both execution and interaction. Task-based evaluations, including an internal extension of Aider’s benchmark, test code editing and file generation. A behavioral specification covers the top-level agent loop: high-level principles become tasks with grades and rubrics. One test is whether the agent asks questions when a request is ambiguous. Results also guide prompt optimization.

Public benchmarks also attract customers. A founder says benchmark charts brought customers to Factory, while acknowledging the incentive to tune specifically for those charts. By this conversation, Factory no longer prioritized competing on SWE-Bench, arguing that its enterprise tasks differed substantially. Internal benchmarks and daily use supplied complementary feedback: improve concrete task performance while maintaining reasonably consistent agent behavior.

Much of perceived quality depends on details below the top-level loop: accurate edits, correct tool parameters, and interfaces the model can use effectively. Factory describes Sonnet 3.7 and Codex as appearing to favor command-line tools, sometimes choosing grep despite having a search tool the team considered more suitable. The speaker attributes that preference to agentic post-training—an interpretation of observed behavior, rather than a confirmed training recipe.

Tool adoption becomes an evaluation problem: supplying a better tool does not ensure the model will use it. Factory favored continued iteration on surrounding systems over reinforcement fine-tuning that would fix a model at a particular capability level and dataset. The team viewed entrenched tool preferences as a generalization problem.

While the agent works, the human plans

The Droid continues working during the interview. The intended workflow includes executing and iterating on code, editing multiple files, and running pre-commit hooks and tests toward a mergeable pull request. Separate browser tabs can run additional tasks. The founders envision starting several tasks, then spending human attention on priorities and harder reasoning.

That expectation explains the browser-based interface. Factory argues that delegating engineering needs a different interaction pattern from writing each line in an IDE. In its proposed test-driven delegation, developers spend more time understanding, planning, and defining tests that establish completion criteria. Sufficiently strong tests could allow merging without inspecting every implementation detail. Another participant adds a necessary qualification: changing functionality sometimes requires breaking existing tests. Passing the old suite cannot define every new task.

A participant describes a Next.js conference website running through Factory’s built-in browser. In that participant’s attempt with ChatGPT Codex, the project had not worked out of the box and no built-in browser was available. This was a particular trial, not a systematic comparison. Near the demonstration’s end, a presenter reports changes across roughly 12 edited or created files. Creating a pull request through the GitHub tool remains the next instruction.

Retrieval, token costs, and useful outcomes

The demo’s context indicator shows 43% used in Factory’s large monorepo, a repository containing multiple parts of its system. Without a matched comparison, that figure does not establish relative efficiency. The design principle is clearer: retrieve what each model call needs. Even a hypothetical billion-token context window would leave a reason to select relevant material, because supplying the whole repository would still incur cost.

At the time, pricing combined small team-access and additional-user fees with predominantly usage charges. Factory expressed consumption in standard tokens, benchmarked against its standard models, and preferred that unit to opaque credits. Reporting connected consumption to deliverables such as pull requests and merged code.

Enterprise reporting also included code churn, changes to recently merged lines. The speaker uses a three-week window as a possible signal that code needed fixing, while acknowledging that improvements and new requirements can cause the same pattern. The time cutoff makes the distinction from refactoring somewhat arbitrary. Factory considered churn more useful for established enterprises than for startups with rapidly changing requirements.

After investing heavily in productivity metrics, the team found that dashboards rarely changed customers’ existing opinions. It came to favor asking whether developers felt more productive and whether the organization delivered earlier or expanded a quarter’s scope. One founder reports an unnamed public company completing a migration in about three and a half days with no downtime, against an expected four months. The baseline was an estimate and the scope unspecified, so the anecdote illustrates customer value without establishing a general migration speedup.

A migration is a sequence of delegations

A hypothetical modernization of a twenty-year-old hospital codebase to Java 21 makes the coordination problem concrete. The work includes dependency analysis, documentation, migration strategy, schedules, tickets, incremental implementation, and handoff. In the conventional team described, four to ten people handle interdependent tasks; delays in one part can affect the rest. Code editing occupies only part of that process.

The proposed Factory workflow spans multiple sessions: 1. Analyze modules and document the current system. 2. Combine those documents with actual code and relevant system documentation to develop a migration plan. 3. Create epics and tickets through Linear or Jira, identifying dependencies and parallel work. 4. Launch separate engineering sessions for independent tasks, then have a human review changes, check that CI passed, and merge incrementally. Factory says its largest time savings appear in planning. Throughput increasingly depends on how quickly a human can delegate appropriately.

Learning this workflow takes support. Factory describes working alongside two customer engineers in the migration engagement discussed earlier; those engineers then taught colleagues. Delegation requires earning trust and accommodating established developer habits, rather than assuming that access to the product will change behavior.

The remaining limits: duration, cost, and intent

Factory expects faster inference to improve everyday use and make delegation easier to adopt. In established enterprise migrations, organizational bureaucracy may still dominate elapsed time. Lower costs could also make parallel candidate generation practical: the team says it had techniques for generating many candidates but avoided that degree of parallelism because the quality gain did not justify the expense.

A major model request was sustained goal-directed work. Factory wanted post-training on general agentic trajectories—extended sequences of work—lasting one to three hours, so models could remain focused throughout a hard problem. It was building benchmarks with possible post-training uses in mind, while leaving open how much training it would undertake itself.

The tooling gap was semantic observability: understanding what a user wanted and whether a session satisfied that intent. Factory reports using LangSmith without LangChain and being happy with it. Collecting execution traces was not its main difficulty. Classifying good and bad experiences was harder, especially when dissatisfaction involved subjective code style and enterprise privacy prevented inspecting the customer’s code.

The proposed direction puts language models inside the observability system to interpret messy natural-language goals. A speaker sees product analytics and feature-flag companies such as Amplitude and Statsig as promising places for this capability to develop, without claiming they already provide it.

Teaching delegation and preserving judgment

Factory says it spent roughly its first year and a half refining the enterprise interaction pattern, with customer acquisition relying heavily on word of mouth. At the time of the conversation, expansion into large enterprises made a particular hiring profile valuable: people who could speak with chief information officers and engineering executives, then sit alongside developers and work through concrete use cases.

Design provided another way to make expectations explicit. Cal, described as a founder’s older brother, created the original logo and worked across brand, marketing, and product design. The team valued a non-engineer’s perspective in a mostly engineering organization: developers building for themselves can overlook design principles.

Design systems and written principles also guide agents. Factory says its Droids can imitate the brand voice and style Cal established, while product engineers still judge what reaches users. Its playful slogan, “the machine that builds the machine,” refers to software writing software; the founders explicitly distinguish that from Factory building itself.

The conversation closes by pairing openness to AI-native working habits with skepticism about hype. That balance fits the workflow described throughout: people communicate intent, decide what to delegate, and judge the result, even as agents take on more execution.