Dust: Building Workplace Agents from Tasks That Already Work

A capable model can still be the wrong tool

Before Dust, Stanislas Polu tried using transformers to find software bugs. Working with a friend, he experimented with fuzzing: mutating a library’s inputs to discover bugs. They used feedback about how far execution propagated through the binary to guide reinforcement learning. The approach failed because the transformers were too slow relative to evolutionary algorithms.

The experiment followed an exploration of autonomous RC cars and a nearly launched self-driving truck company in Europe. Operational demands, uncertainty about the team, and the possibility of fatal software errors persuaded Polu to stop. Atari-playing systems had impressed him, but he wanted applications with direct consequences in the world. Mathematics offered another route: less immediately tangible than transporting people, but foundational.

The question of what models can usefully do runs through the Dust conversation. Dust lets employees—including nondevelopers—build assistants connected to company information in Slack, Notion, GitHub, and other systems. It evolved from developer tooling toward bounded workplace tasks, supported by the integrations and instructions needed to make them work.

Creative proposals, checkable proofs

Polu began exploring AI before leaving Stripe, after moving back to Paris and finding the nine-hour separation from his team difficult. His work on mathematical problems overlapped with OpenAI’s emerging reasoning team. Conversations with Greg Brockman and Ilya Sutskever led to interviews and eventually a contractor arrangement that let him remain in Paris. He entered without a PhD or prior research experience.

At OpenAI, he focused on mathematical reasoning in formal systems. Language models could propose creative solutions but made mistakes; formal systems could verify proofs, while their conventional proof tactics were comparatively mechanical. The attraction was to combine those capabilities: the model proposes; the formal system verifies.

Polu describes a formal proof as a program checked by a sufficiently expressive type system. Successful checking establishes correctness within that system. This separates two kinds of work: - Proof search: tactics may perform substantial computation to find a solution and expand it into an actual proof. - Proof verification: the resulting low-level proof is checked. Polu calls this effectively instantaneous, as a qualitative description rather than a measured runtime.

The early targets sat at the easier end of the math benchmark under discussion. Looking back on three years of work, the researcher recalls solving individual problems while an Olympiad medal still seemed distant. Asked about DeepMind’s reported silver-level result, the researcher offers a provisional explanation: reinforcement learning combined with larger-scale data generation through autoformalization. The approach had not yet been published in full, and the conversation does not unpack autoformalization’s mechanism.

Compute allocation as research management

In the researcher’s recollection of OpenAI before the Anthropic split, most compute went toward GPT training, principally GPT-3, while smaller teams explored particular problems and algorithms. Researchers had considerable freedom, but compute allocation communicated organizational priorities. Mathematical reasoning aligned with the mission and generally received the resources needed; choosing a question did not guarantee resources to pursue it.

Further allocations depended on incremental progress, often within a few weeks. A strong negative result counted as progress, and could be harder to establish or more informative than a positive result. The process was imperfect: existing priorities favored some projects, internal enthusiasm could expand budgets, and a change of direction could bring an allocation back near zero.

Ilya’s contribution combined technical coaching with organizational coordination. The reasoning team had its own manager, while he spent substantial time communicating a shared technical vision and aligning teams around scaling compute, including compute for reasoning. The researcher recalls a strong belief around the time of GPT-2 that transformers were general enough to learn broadly with sufficient scale—a belief preceding the scaling-laws work discussed in the interview.

Sam Altman supplied a different leadership lesson: within roughly one to two years, he learned enough about the research goals, current work, and recent results to hold productive discussions without knowing every implementation detail. The founder values that ability to move between company-wide priorities and operational detail, while noting that the distance was smaller in their own companies: the first reached ten people, and Dust had twenty-five at the time.

The account of the Anthropic split is more tentative. The speaker understood the disagreement to involve commercialization, particularly making models available through an API, but was not closely involved and does not claim to know the core cause. What surprised them was OpenAI’s quick recovery, which they attribute to a clear mission, research focus, and continued access to compute.

Turn available capabilities into useful products

The founder describes OpenAI as an informal version of the PhD they had wanted. They enjoyed research, but its long stretches of uncertain progress made returning to product work appealing. Seeing GPT-4 internally before leaving in September 2022 supplied a business thesis: capabilities seemed substantially ahead of deployment, leaving room for products that could unlock usage. A second, conditional motivation added urgency: if broadly capable AI arrived soon, the window for starting a company might be short.

Dust initially served developers building on language models. Its workflow interface addressed a concrete debugging problem. The founder believed orchestration should be developed against roughly a dozen examples; running them through multistep workflows encouraged parallel execution, turning console output into interleaved token streams. Separate views of each model interaction made parallel runs inspectable. The founder nevertheless acknowledges that LangChain substantially outperformed Dust’s original offering, while emphasizing that the products differed.

Dust was open source originally and remained entirely open source at the time of the recording. Publishing source and using open source to acquire customers are separate choices; Dust pursued the former without building its sales strategy around the latter. The founder sees the team’s future work, product, and sales organization as more valuable than a snapshot of the code.

Source availability had practical consequences: - Customer transparency: security teams could inspect implementation, and technical users could follow issues through pull requests to deployment. Outside contributions were possible but uncommon. - Security exposure: publishing code could help attackers, while also giving bug bounty researchers better tools. - Limits to openness: the founder’s low concern about cloning applied to product code; proprietary model-inference kernels could justify a different decision.

The browser experiment XP1 made the product principle explicit: choose tasks the current model can already handle. Extracting information from LinkedIn pages and summarizing newspaper articles were useful within its capabilities. Better, faster, cheaper models could improve the experience later. The founder does not recall the precise model name.

English instructions supply the orchestration

Dust’s subsequent horizontal strategy let curious employees build assistants for their own operational work. That required both maintained connections to company systems and a product employees could use. The founder compares conversational AI products to Pong: a useful beginning, with richer interaction patterns still to invent.

The starting point was deliberately constrained. In the founder’s assessment, open-ended AutoGPT-style agents still got lost. Users could instead prescribe a workflow in English: 1. Retrieve a particular page. 2. Run a specified search. 3. Fetch the relevant document. 4. Perform the requested work and return it in a defined format. The user specifies how the tools work together. Commands such as X and Y can select different workflows without a drawn graph, provided the necessary tools are available.

Available tools included semantic search, structured database queries, and web search. External actions—issuing a Stripe refund, sending email, or operating an administrative control—were only beginning to be explored. Dust maintained most integrations because customers expected them to work without extra effort, while allowing customers to build unsupported connections and push data into the system.

Salesforce illustrated why integration was more than authentication. Each company could have a different data model, and Dust did not yet support it natively. A proposed approach would ask customers to supply queries in Salesforce’s query language, SOQL, then ingest the results as context.

Use APIs where the customer’s systems provide them was the founder’s preference. Most internal integrations used those programmatic interfaces; browser automation served public-web cases. The preference reflected Dust’s target market: technology scale-ups with roughly 500–5,000 employees whose software services mostly had APIs.

For websites without APIs, the desired browser interface went beyond feeding extracted page text into a model. A compact page representation would preserve selectors—the references identifying where to act—and expose available actions. The model’s selection could then map back onto the original page. Preserve actionable structure while removing page noise was the goal.

Build the small assistants before the manager

Precise, scripted instructions reportedly made tool selection reliable, while broad goals paired with sixteen tools still produced mistakes. This was a qualitative observation, but it motivated a concrete direction: make small agents work well before asking a larger agent to coordinate them.

Dust’s weekly meeting supplied the example. One assistant retrieved a week of incident and shipped-work posts from Slack and turned them into a table, replacing manual preparation. Separate assistants generated financial and progress graphs. The founder reported that these narrowly defined assistants worked well.

A proposed meeting assistant would invoke those specialists and prepare a Notion page without further prompting. In this hierarchy, reliable agents become actions for higher-level agents. The estimated saving of roughly an hour of company time per meeting was prospective: Dust had not yet explored these coordinating agents or established practices for their communication. Another participant proposed shareable dependency graphs so builders could reuse smaller assistants.

Small assistants were also easier to explain to operational teams without AI expertise. Asking employees to build useful tools for their jobs made sense to them. Dust deliberately used “instructions” instead of “system prompt,” crediting designer Ed with pushing for less intimidating language and branding.

At the time, Dust did no research, trained no models, and ran no GPUs. That product focus did not eliminate the evaluation problem: a conversation trace may not reveal whether the user succeeded. A departing user might be satisfied or frustrated, and even a human reviewer could struggle to tell. The immediate improvement loop was to collect end-user feedback so builders could revise assistants. Model-based evaluation was a possible source of quantitative feedback, not an established solution.

The founder left open whether a larger product company might eventually need post-training: further training to align an existing model with its product. Model behavior sits close to the user experience, which could make an internal adjustment loop valuable without requiring pretraining a foundation model. Software vendors might then help supply simulation or data-generation environments, but they would face a competing incentive to retain human users paying for seats.

Useful context matters more than marginal model gains

Dust let users select a model per assistant, keeping the choice in advanced settings and moving the default to a newer preferred model. Function calling—having a model invoke a tool with parameters—was a priority, because incorrect parameters could derail the entire interaction.

The model discussion distinguished impressions from evaluations. One participant suspected GPT-4 Turbo still exceeded GPT-4o at function calling, explicitly without proof, and praised a reasoning step observed with Claude 3.5 Sonnet during tool use. Another reported that the then-new Berkeley Function Calling Leaderboard V3 placed Turbo above GPT-4o, Salesforce’s xLAM third, and o1 Mini eleventh. No scores or evaluation settings accompanied those historical rankings.

For Dust, broader product improvements still took priority over fine-grained model comparisons. It reported roughly 60–70% weekly active employee penetration among its then-current enterprise customers, with a highest customer figure of 88% daily active penetration. These figures concerned companies Dust successfully activated; no sample size or measurement window was supplied. The argument that product changes offered much larger gains than model selection was a prioritization judgment, not an experiment.

The immediate bottleneck was connecting users to all the information their tasks required. Moving data is not enough to make it useful to a model. In the founder’s Notion example, Airbyte exported structured block details, but useful model interactions required additional processing.

Notion ingestion required three distinctions: - Preserve page structure when chunking: splitting information into smaller pieces should respect the page’s organization. - Treat text-heavy databases as text: a database container does not make its contents quantitative. - Recognize genuinely tabular data: quantitative databases require different handling from collections of prose. The founder had not found a way to achieve the desired interaction quality without owning the connection end to end.

Access to a continuous flow of company information also suggested products beyond chat. A proposed documentation-repair assistant would watch Slack messages and new Notion pages for information contradicting a document. It would email the document’s owner with the relevant message and paragraph, leaving them to update it or contact the person involved. This was a product opportunity, not a reported deployment: its value would come from reviewing more information than an individual could routinely read.

Own the data handling; buy the orchestrator

Dust used Temporal to coordinate asynchronous ingestion of Slack, Notion, and GitHub updates in semi-real time. Incoming information could also trigger workflows that ran agents and alerted users. The founder praised Temporal while acknowledging rough edges, including versioning.

Dust paid for Temporal Cloud despite considering it expensive. Buying general infrastructure saved engineering time for the core product. Owning application-specific ingestion did not require building a replacement orchestrator.

The founder proposes that agents could change the calculation for other software services. If an AI interface takes actions on users’ behalf, they may need less of the service’s existing interface, making it look increasingly like an underlying database. The resulting buy-versus-build decision would depend on the company: - Mature technology companies under cost pressure might have the engineers and incentive to build substitutes. - Companies without that technical capacity would have fewer options to replace purchased software. - Companies prioritizing rapid growth would still have reasons to buy for speed.

Klarna’s reported removal of Zendesk and Salesforce enters as an example of changing incentives, but the participants question how well it will work and offer only a tentative account of its architecture. A broader idea is software as a labor interface: users ask a person or agent to accomplish a task instead of operating the application themselves. This is the direction discussed as “Services as Software”.

Other stack choices reflected Dust’s history. It owned its unified interface to model providers because that layer already existed in the original orchestration product. The founder’s concrete implementation regret was starting in plain JavaScript, recommending TypeScript even when moving quickly. Dust used Next.js, with its internal service written in Rust.

Small teams still need editorial judgment

Revisiting a prediction about a one-person billion-dollar company, the founder clarifies that its already having started was a hypothesis. Such a company should grow quickly enough to become visible, and the tools needed to operate broadly through models were not yet sufficiently available. A more convincing prospect was a billion-dollar company with an engineering team of about twenty: a small, focused group assisted by machines, with less management overhead. This remained an aspiration.

Another participant emphasizes work organizations currently leave undone because they lack staff. AI could let companies accomplish more without first reaching the headcounts those tasks previously required. Creator businesses offer a possible setting for this division of labor: agents handle production while a human supplies brand identity and curation.

One participant estimates that their daily newsletter is 99% AI-generated, while retaining responsibility for commentary and choosing among four options. The human decides what enters the finished product. Another concrete example is a video-editing agent the participant’s team already pays for; wrapping FFmpeg in a GPT is suggested as a direction for serving creators, rather than a complete implementation.

The horizontal bet is company-specific work

The closing disagreement concerns how to build a business around these capabilities: - Dust’s horizontal approach: serve people across a company and expand into many operational tasks, accepting a harder sale than a narrowly specified solution. - The product-first alternative: establish a strong product and brand in one niche, let that experience guide infrastructure development, and potentially expose the underlying capabilities later.

The product-first argument points to Perplexity’s search product and Perplexity Labs, and to Amazon and Amazon Web Services, while acknowledging that such successes are few. The counterargument is that a specialized identity can constrain expansion: a product known for one function may struggle to serve another, with Zendesk serving engineers offered as the example.

Dust’s final clarification makes its bet more specific. “Platform” means a product people throughout a company can use. Sales and support have sufficiently standardized processes to sustain powerful vertical products. Many other operations are peculiar to the company performing them. Company-specific work is the opening for a general assistant product—one that lets the people who understand those tasks build their own assistance.