From Computer Use to Agents in Production — 2024 Year in Review, Part 3 of 3

Computer Use: attention outpaces adoption

Computer Use drew exceptional attention on Hacker News, yet one speaker saw comparatively little everyday use. That gap opens Part 3 of the 2024 year-end conversation: giving an agent control of a computer makes a compelling demonstration, but precision, latency, and cost still limit routine use. The speaker treats it as an emerging capability whose practical performance has yet to catch up with its appeal.

Delegating control also raises a separate question of comfort. One speaker suggests that many users feel uneasy about letting an agent operate their personal computer; the other is personally unconcerned. Adequate performance and willingness to grant control are distinct adoption hurdles.

In the November recap, the speakers describe DeepSeek’s R1 as a surprising open-source competitor to o1. One considers it and a model from the Qwen team leading contenders in that category—a judgment about the competitive field, without a benchmark comparison.

Agent infrastructure: separate authority, reconsider interfaces

Stripe Agent Toolkit, described as a developer-experience wrapper around Stripe’s existing software development kit (SDK), prompts a debate about what agents actually need. One speaker interprets Stripe’s investment as a signal of demand and institutional belief from outside the model business. The other questions why agents need separate SDKs at all.

The clearest proposed requirement is separate authorization for delegated actions. An agent acting for a user should be distinguishable from the user, with infrastructure that can: - Identify the actor: distinguish a person’s actions from an agent’s actions on their behalf. - Attribute activity: reveal the source of unexpected API calls. - Restrict access quickly: make delegated activity easier to lock down.

An ElevenLabs API-key compromise makes the attribution problem concrete. One speaker recalls unexpected calls and difficulty determining whether they were their own, mentioning a key committed to GitHub. The incident motivates a requirement that applies directly to agents: knowing which credential was used should also help establish where the activity came from.

Beyond authorization, two positions emerge: - Reuse existing tooling: one speaker sees reinventing every SDK for agents as a failure of developer tooling and AI. - Reconsider human defaults: the other argues that APIs contain implicit assumptions about human users. An agent with much larger memory and context might benefit from different interface choices.

Query design supplies a concrete case. One speaker previously preferred building a REST endpoint for a known need. Agents’ less predictable queries make GraphQL more appealing to them: the possibility of many different requests weakens the assumption that developers can anticipate the endpoints they will need.

Search is changing product assumptions too. Alongside SearchGPT and Perplexity, the speakers highlight Dropbox Dash’s Google Drive integration. Its strategic significance is that Dropbox can help users find information without hosting their files. Search gives the company a reason to reach beyond its own storage.

The 2025 bet: agents change the economics of work

As the recap reaches December, the speakers trace their interest in autonomy back to “The Anatomy of Autonomy” in April 2023. One argues that model capabilities advanced substantially in the final two months under discussion, making a longstanding ambition more practical.

DeepMind’s Gemini 2.0 announcements included Deep Research, Project Mariner, a browser agent with computer-use functionality, and Jules, a coding agent. One speaker also anticipates an OpenAI agent project called Operator in 2025. They speculate that it could command $2,000 if it actually replaced a junior employee; this is neither a confirmed price nor a price with a specified billing interval.

The consequential prediction concerns the skill floor—the minimum economically viable performance in a role. One speaker expects AI to begin setting that floor for some jobs in 2025. If systems such as Devin and customer-support agents work as hoped, a human doing the same job will need to outperform the available agent for the economics to make sense. They expect software engineering to face similar pressure.

Security enters through a prediction relayed from conversations with researchers at NeurIPS: a foreign spy will be caught at a major AI lab. The speaker presents this as a forecast, connecting it to their view that labs hold highly sensitive information with inadequate protection.

That concern extends to secure cloud inference, the infrastructure for running language models in the cloud. One speaker argues that existing security is insufficient for systems attracting state-level interest and expresses interest in investing in stronger infrastructure available beyond a single provider. They also cite OpenAI’s hiring of a chief information security officer during 2024 as a sign of growing attention to security.

Lasting lessons: concentrated compute, careful evaluation, useful surveys

The retrospective turns to the podcast’s own work. The Substack popularity list under review begins with “Why Google failed to make GPT-3,” “How NotebookLM was made,” and “The ultimate guide to prompting.” One speaker describes the editorial goal as finding insights worth referencing long after an interview, beyond a guest’s current project.

The first episode’s lasting lesson is to concentrate compute into large bets. Connecting an earlier guest’s account of a compute marketplace with their reading of Ilya’s emails, one speaker argues that a large training run can be much more valuable than a hundred smaller runs using equivalent aggregate resources. This is a qualitative strategic argument, not a measured performance multiplier.

The NotebookLM interview benefited from timing: the product was already top of mind, and the episode circulated on social media. Its memorable technical detail was subjective group evaluation—people assessing outputs together—rather than formal evaluations. One speaker praises the attention devoted to training and ventures that comparable care could make any model state of the art. That broader claim remains an opinion, without a training procedure or benchmark comparison in the exchange.

The prompting episode shows the value of organizing scattered techniques into a usable survey. The speakers favor a comprehensive overview over many interviews about papers whose main contribution is another prompt. Crediting Sander’s extensive preparation, they consider two follow-ups: - Broader guides: apply the comprehensive format to other subjects. - Deeper prompting coverage: invite Sander back in 2025 for a “prompting 201” episode.

A disagreement about the next episode reveals that popularity depends on the metric. One speaker is using Substack’s list, described as including likes; the other is using downloads. Audience growth can also give newer episodes more viewers. Separately, one reports that listeners appreciated the not-safe-for-work (NSFW) episode for addressing a subject that usually receives little coverage.

The technical retrospective closes on simulation as an environment for model training and agent learning. One speaker’s interest extends beyond the immediate use cases of WebSim and WorldSim to that broader research direction. The speakers recommend their then-new seven-hour episode on simulative environments for a deeper treatment.

Annual progress, funding, and the production test

Both speakers reject the idea that 2024 was an AI winter. One distinguishes weaker sentiment from continuing technical progress; the other acknowledges the debate over scaling limits without treating it as proof of stagnation. Their proposed check is to compare with a year earlier across benchmarks, frontier models, market share, and capabilities now taken for granted.

The discussion reports 133 AI funding rounds above $100 million in 2024, including Databricks’ $10 billion round. Its reported $60 billion valuation also changes the theoretical value of its mostly stock acquisition of Mosaic: the stock consideration appreciates with the acquirer’s valuation. The speakers estimate that consideration at roughly $2 billion or more, recalling earlier doubts about Databricks’ stale private valuation while Snowflake had declined.

The test for 2025 returns to the opening adoption problem: agents need to work in production. One speaker predicts that they will. The other is less certain, arguing that deployment needs to happen and that failure would bring an AI winter. They agree on the importance of the next step while differing on the likelihood of reaching it.

A growing community and the next hundred episodes

The closing reflections celebrate a Discord community approaching 5,000 members roughly four years after its founding, with particular appreciation for the Paper Club. The speakers feel that growth has preserved its atmosphere. The norms are straightforward: post news, discuss it in threads, and limit self-promotion. They say members mostly follow those rules and bans are rarely necessary.

Having completed 100 podcast episodes, they now find it easier to envision reaching 200 than they once found reaching the first hundred. Their closing request is to subscribe on YouTube. Video production takes substantial work and expense, with an uncertain payoff but hopes of better discovery; one speaker expresses greater confidence in YouTube as a podcasting platform than in Spotify.