Writeups
Source-grounded explanations from our conversations and reporting.
What METR’s Time Horizon Measures—and What It Leaves Out
Human task duration makes AI capability easier to interpret. Productivity, autonomous research and catastrophic risk depend on more than the work a benchmark captures.
Where AI Progress Could Come From: January–February 2024 Recap, Part 1 of 3
Part 1 follows five research bets—from longer inference to online models—then examines Sora's physical consistency, Groq's utilization economics, and Gemini's million-token context.
Why agents need reasoning, inspectability, and better abstractions
Imbue’s approach starts with a recruiting product’s trust problem and asks what it takes to make models dependable enough to delegate work to.
Why AlphaFold Didn’t Solve Protein Biology
Pushmeet Kohli and Sal Candido on protein structure, biological data, and the work required to make models useful for science and medicine.
Why OpenPipe Bet on RL: From Cheaper Tokens to More Reliable Agents
Falling model prices weakened OpenPipe’s distillation business. Reinforcement learning offered a new opportunity: improve agents through experience, provided their training environments behave like production.
Why SWE-Bench Verified Stopped Measuring Coding Progress
Narrow tests and leaked repository knowledge make higher scores harder to trust. The next coding evaluations need harder work and credible judgments of quality.
Windsurf: Building an AI IDE Around Developer Intent
Why Codeium built its own editor—and how developer actions, code retrieval, reversible execution, and enterprise requirements shape the product.
Wondercraft: turning speech generation into a production workflow
Converting a blog into a podcast requires more than a convincing voice: worthwhile source material, controllable delivery, publishing tools, and a way to fix the result.
WorldSim and WebSim: When the Model Becomes the Interface
A fictional terminal and a browser for imaginary websites turn prompting into exploration—and reveal the gap between plausible state, working software, and understanding.
Write benchmarks for the work you actually do
Useful imperfect models, evaluations drawn from real tasks, and security research that changes deployed systems—from the conversation published August 29, 2024.
Xaira’s X-Cell: learning what happens when you turn a gene down
Large-scale perturbation experiments give X-Cell data about interventions. The test is whether it can use that data to predict responses in unfamiliar biological contexts.
Public beta. Model source and teaching reviews passed; human editorial review is pending. Original articles and recordings remain separate.