Over the last 2 days with AIE Miami concluding (Singapore is next!) the top conversations we have been hearing from AI leadership (CTOs, VPs, Founders) have all centered around the concept of “Tokenmaxxing” and how leaders want to get their teams using more AI, WITHOUT the downside of incentivizing the kinds of horrendous waste our friend described at his AIE keynote.

Dex Horthy, coiner of Context Engineering and “the Dumb Zone”, publicly retracted his extremely vibe-coding-pilled call 6 months ago and encouraged people to please read the code, citing ’s Z/L continuum from AIE Europe:

Off the record, many senior leaders I talk to are more on the Zechner side than the Lopopolo side of the Z/L spectrum — this does not mean that one side is true for every one in every situation, nor does it mean it will continue to be true with advancing model progress! To point out the most obvious, engineers and engineering leaders are the ones most setup to make a big deal out of minor architectural quality issues that sheer quantity of cheap code generation and code review might overcome.

Today’s LS guest, Mikhail Parakhin, CTO of Shopify, had another take on the “tasteful tokenmaxxing” - you want to go for depth (e.g. do more serial autoresearch loops) than go for breadth (e.g. solve a problem by kicking off 5, 10, 50, 500 parallel runs of the LLM slot machine). Worth thinking through.

AI News for 4/21/2026-4/22/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Open Models: Qwen3.6-27B, OpenAI Privacy Filter, and Xiaomi MiMo-V2.5

• Qwen3.6-27B lands as a serious local/open coding model: @Alibaba_Qwen released Qwen3.6-27B, a dense, Apache 2.0 model with thinking + non-thinking modes and a unified multimodal checkpoint. Alibaba claims it beats the much larger Qwen3.5-397B-A17B on major coding evals, including SWE-bench Verified 77.2 vs 76.2, SWE-bench Pro 53.5 vs 50.9, Terminal-Bench 2.0 59.3 vs 52.5, and SkillsBench 48.2 vs 30.0. It also supports native vision-language reasoning over images and video. The ecosystem moved immediately: vLLM shipped day-0 support, Unsloth published 18GB-RAM local GGUFs, ggml added llama.cpp usage, and Ollama added a packaged release. Early user reports from @KyleHessling1 and @simonw were notably strong for local frontend/design and image tasks.

• OpenAI quietly open-sources a practical privacy model: Multiple observers flagged OpenAI’s new Privacy Filter, a lightweight Apache 2.0 open model for PII detection and masking. According to @altryne, @eliebakouch, and @mervenoyann, it is a 1.5B total / 50M active MoE token-classification model with a 128k context window, intended for cheap redaction over very large corpora and logs. This is a more operationally interesting release than a generic “small open model”: it targets a concrete infra problem in enterprise/agent pipelines where on-device or low-cost preprocessing matters.

• Xiaomi pushes agentic open models upward: @XiaomiMiMo announced MiMo-V2.5-Pro and MiMo-V2.5. Xiaomi positions V2.5-Pro as a major jump in software engineering and long-horizon agents, citing SWE-bench Pro 57.2, Claw-Eval 63.8, and τ3-Bench 72.9, with claims of 1,000+ autonomous tool calls. The non-Pro model adds native omnimodality and a 1M-token context window. Arena quickly listed MiMo-V2.5 in Text/Vision/Code evaluation, and Hermes/Nous integration followed via @Teknium.

Google Cloud Next: TPU v8, Gemini Enterprise Agent Platform, and Workspace Intelligence

• Google’s infra announcements were substantial, not cosmetic: @Google and @sundarpichai introduced 8th-gen TPUs with a split design: TPU 8t for training and TPU 8i for inference. Google says 8t delivers nearly 3x compute per pod vs Ironwood, while 8i connects 1,152 TPUs per pod for low-latency inference and high-throughput multi-agent workloads. Commentary from @scaling01 highlighted an additional claim: Google can now scale to a million TPUs in a single cluster with TPU8t. The productization signal matters as much as the raw hardware: Google is clearly aligning chips, models, agent tooling, and enterprise control planes into one vertically integrated offering.

• Enterprise agents became a first-class Google product surface: @GoogleDeepMind and @Google launched Gemini Enterprise Agent Platform, framed as the evolution of Vertex AI into a platform for building, governing, and optimizing agents at scale. It includes Agent Studio, access to 200+ models via Model Garden, and support for Google’s current stack including Gemini 3.1 Pro, Gemini 3.1 Flash Image, Lyria 3, and Gemma 4. Related launches included Workspace Intelligence GA as a semantic layer over docs/sheets/meetings/mail, Gemini Enterprise inbox/canvas/reusable skills, Agentic Data Cloud, security agents with Wiz integration, and Gemini Embedding 2 GA, a unified embedding model across text, image, video, audio, and documents.

Agents, Harnesses, Traces, and Team Workflows

• The “agent harness” abstraction is hardening across vendors: OpenAI introduced workspace agents in ChatGPT, shared Codex-powered agents for teams that can operate across docs, email, chat, code, and external systems, including Slack-based workflows and scheduled/background tasks. Google made a parallel enterprise move with Gemini Enterprise Agent Platform, while Cursor added Slack invocation for task kick-off and streaming updates. The pattern is converging: cloud-hosted agents, shared team context, approvals, and long-running execution rather than single-user chat.

• Developer ergonomics around harness/model independence improved: VS Code/Copilot rolled out bring-your-own-key/model support across plans and business/enterprise, enabling providers like Anthropic, Gemini, OpenAI, OpenRouter, Azure, Ollama, and local backends. This is strategically important because, as @omarsar0 noted, most models still seem overfit to their own agent harnesses. Cognition’s Russell Kaplan made the complementary business case: enterprise buyers want model flexibility and infrastructure that spans the full SDLC, not attachment to one lab.

• Traces/evals/self-improvement are becoming the core agent data primitive: The strongest thread here came from LangChain-adjacent discussion. @Vtrivedy10 argued that traces capture agent errors and inefficiencies, and that compute should be pointed at understanding traces to generate better evals, skills, and environments; a longer follow-up expanded this into a concrete loop involving trace mining, skills, context engineering, subagents, and online evals. @ClementDelangue pushed for open traces as the missing data substrate for open agent training, while @gneubig promoted ADP / Agent Data Protocol standardization. LangChain also teased a stronger testing/evaluation product direction via @hwchase17.

Post-Training, RL, and Inference Systems

• Perplexity and others shared more of the post-training playbook: @perplexity_ai published details on a search-augmented SFT + RL pipeline that improves factuality, citation quality, instruction following, and efficiency; they say Qwen-based systems can match or beat GPT-family models on factuality at lower cost. @AravSrinivas added that Perplexity now runs a post-trained Qwen-derived model in production that unifies tool routing and summarization and is already serving a significant share of traffic. On the research side, @michaelyli__ introduced Neural Garbage Collection, using RL to jointly learn reasoning and KV-cache retention/eviction without proxy objectives; @sirbayes reported a Bayesian linguistic-belief forecasting agent matching human superforecasters on ForecastBench.

• The “minimal editing” problem in coding models got a useful benchmark treatment: @nrehiew_ presented work on Over-Editing, where coding models fix bugs by rewriting too much code. The study constructs minimally corrupted problems and measures excess edits with patch-distance and added Cognitive Complexity; it finds GPT-5.4 over-edits the most while Opus 4.6 over-edits the least, and that RL outperforms SFT, DPO, and rejection sampling for learning a generalizable minimal-editing style without catastrophic forgetting. This is one of the more practical post-training/eval contributions in the set because it targets a failure mode engineers actually complain about in production code review.

• Inference efficiency work remained highly active: @cohere integrated production W4A8 inference into vLLM, reporting up to 58% faster TTFT and 45% faster TPOT vs W4A16 on Hopper; the details include per-channel FP8 scale quantization and CUTLASS LUT dequantization. @WentaoGuo7 reported SonicMoE throughput gains on Blackwell—54% / 35% higher fwd/bwd TFLOPS than DeepGEMM baseline—while maintaining dense-equivalent activation memory for equal active params. @baseten introduced RadixMLP for shared-prefix elimination in reranking, with 1.4–1.6x realistic speedups.

Top tweets (by engagement)

• OpenAI workspace agents: @OpenAI launched shared, Codex-powered workspace agents for Business/Enterprise/Edu/Teachers.

• Qwen3.6-27B release: @Alibaba_Qwen announced the new open 27B dense model with strong coding claims and Apache 2.0 licensing.

• Google TPU v8: @sundarpichai previewed TPU 8t / 8i, with training/inference specialization.

• Flipbook / model-streamed UI: @zan2434 showed a prototype where the screen is rendered as pixels directly from a model rather than traditional UI stacks.

• OpenAI Privacy Filter: @scaling01 and others highlighted OpenAI’s new open-source PII detection/redaction model on Hugging Face.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Qwen 3.6 Model Releases and Benchmarks

• Qwen 3.6 27B is out (Activity: 2576): Qwen 3.6 27B, a new language model, has been released on Hugging Face. This model features 27 billion parameters and is designed to improve upon previous iterations with enhanced performance benchmarks. A quantized version is also available, Qwen3.6-27B-FP8, which allows for more efficient deployment in environments with limited computational resources. The release includes detailed benchmark results, showcasing its capabilities across various tasks. The community is expressing excitement about the release, with some users highlighting the significance of the model’s performance improvements and the availability of a quantized version for broader accessibility.

  • Namra_7 shared a benchmark image for Qwen 3.6 27B, which likely includes performance metrics such as inference speed, accuracy, or other relevant statistics. However, the specific details of the benchmarks are not described in the comment itself.

  • challis88ocarina mentioned a quantized version of Qwen 3.6 27B available on Hugging Face, specifically in FP8 format. Quantization can significantly reduce the model size and improve inference speed, making it more efficient for deployment without a substantial loss in accuracy. The link provided leads to the Hugging Face model repository for further exploration.

  • Eyelbee posted another image link, which might contain additional visual data or performance metrics related to Qwen 3.6 27B. However, the comment does not provide specific insights or details about the content of the image.

• Qwen3.6-27B released! (Activity: 895): Qwen3.6-27B is a newly released dense, open-source model that excels in coding tasks, outperforming its predecessor, Qwen3.5-397B-A17B, on major coding benchmarks. It features strong reasoning capabilities across both text and multimodal tasks and offers flexibility with ‘thinking’ and ‘non-thinking’ modes. The model is released under the Apache 2.0 license, making it fully open-source and accessible for community use. More details can be found on their blog, GitHub, and Hugging Face. The comments reflect excitement and admiration for the Qwen team, with users expressing eagerness to utilize the model on their hardware and suggesting the team’s contributions are monument-worthy.

  • ResearchCrafty1804 highlights the impressive performance of Qwen3.6-27B, noting that despite having only 27 billion parameters, it surpasses the much larger Qwen3.5-397B-A17B model on several coding benchmarks. Specifically, it achieves scores of 77.2 on SWE-bench Verified, 53.5 on SWE-bench Pro, 59.3 on Terminal-Bench 2.0, and 48.2 on SkillsBench, outperforming the larger model by significant margins in each case.

  • bwjxjelsbd comments on the competitive landscape, expressing satisfaction that Alibaba is advancing with Qwen models after META’s perceived setbacks. The commenter hopes for continued competition and transparency, suggesting that META should open-source their Muse family models to maintain a healthy competitive environment.

• Qwen3.6-35B becomes competitive with cloud models when paired with the right agent (Activity: 848): The post discusses the significant improvement in benchmark performance of the Qwen3.6-35B model when paired with the little-coder agent, achieving a 78.7% success rate on the Polyglot benchmark, placing it in the top 10. This improvement highlights the impact of using appropriate scaffolds, suggesting that local models may underperform due to harness mismatches. The author plans to test further on Terminal Bench and GAIA for research capabilities. Full details and benchmarks are available on GitHub and Substack. Commenters express surprise at the performance gains from scaffold changes, questioning the validity of benchmarks that don’t control for such factors. There’s also interest in using pi.dev for its extensibility in harnessing models.

  • DependentBat5432 highlights a significant performance improvement in Qwen3.6-35B when changing the scaffold, noting a jump from 19% to 78%. This raises concerns about the validity of benchmark comparisons that do not control for such variables, suggesting that scaffold choice can dramatically affect model performance.

  • Willing-Toe1942 reports that Qwen3.6, when used with pi-coding agents, performs almost twice as well as opencode. This comparison involved tasks like modifying HTML code and searching online resources for documentation, indicating that the choice of agent can significantly enhance the model’s effectiveness in practical coding scenarios.

  • kaeptnphlop mentions the strong performance of Qwen-Coder-Next when paired with GitHub Copilot in VS Code, suggesting potential for further exploration with other tools like little-coder. This implies that integrating Qwen models with popular coding environments can leverage their strengths effectively.

Read more

From near-universal AI tool adoption inside Shopify to internal systems for ML experimentation, auto-research, customer simulation, and ultra-low-latency search, Mikhail Parakhin joins us for a deep dive into what it actually looks like when a 20-year-old, $200B software company goes all-in on AI. We cover why Shopify has become much more vocal about its internal stack, what changed after the December model-quality inflection, and why the real bottleneck in AI coding is no longer generation, but review, CI/CD, and deployment stability.

We also go inside Tangle, Tangent, SimGym, which are three major AI initiatives that Shopify is doing to make experimentation reproducible, optimization automatic, customer behavior simulatable, and search and catalog intelligence faster and cheaper at scale. Along the way, Mikhail explains UCP, Liquid AI, and why token budgets are directionally right but often measured badly, why AI-written code can still increase bugs in production, what makes Shopify’s customer simulation defensible, and what he learned from the Sydney era at Bing.

We discuss:

• Mikhail’s path from running a major Microsoft business unit spanning Windows, Edge, Bing, and ads to becoming CTO of Shopify

• Why Shopify is talking more publicly about AI now, and why staying at the frontier has become necessary for the company

• Shopify’s internal AI adoption curve, the December inflection, and why CLI-style tools are rising faster than traditional IDE-based tools

• Why Jensen Huang is directionally right on token budgets, but raw token count is still the wrong way to evaluate engineering output

• Why the real unlock is not more agents in parallel, but better critique loops, stronger models, and spending more on review than generation

• Why AI coding can still lead to more bugs in production even if models write cleaner code on average than humans

• Why Shopify built its own PR review flow, and why Mikhail thinks most off-the-shelf review tools miss the point

• How PR volume, test failures, and deployment rollback are becoming the real bottlenecks in the agent era

• Why Git, pull requests, and CI/CD may need a new metaphor once code is written at machine speed

• What Tangle is, and how Shopify uses it to make ML and data workflows reproducible, collaborative, and production-ready from the start

• Why Tangle is different from Airflow, and why content-addressed caching creates network effects across teams

• What Tangent is, and how Shopify is using auto-research loops to optimize search, themes, prompt compression, storage, and more

• Why Tangent is becoming a democratizing tool for PMs and domain experts, not just ML engineers

• Why AutoML finally feels real in the LLM era, and where auto-research still falls short today

• Why Tangle, Tangent, and SimGym become much more powerful when combined into one system

• What SimGym is, why simulated customers only work if you have real historical behavior, and why Shopify’s data gives it a moat

• How SimGym evolved from comparing A/B variants to telling merchants what to change on a single live storefront to raise conversions

• Why customer simulation is so expensive, from multimodal models to browser farms to serving and distillation costs

• How Shopify models merchant and buyer trajectories, runs counterfactuals, and thinks about interventions like discounts, campaigns, and notifications

• Why category-level behavior is so different across commerce, and why ideas like Chinese Restaurant Processes are showing up again in practice

• Shopify’s new UCP and catalog work, including runtime product search, bulk lookups, and identity linking

• Why Shopify is using Liquid AI, and why Mikhail sees it as the first genuinely competitive non-transformer architecture he has used in practice

• Where Liquid already works inside Shopify today, from low-latency query understanding to large-scale catalog and Sidekick Pulse workloads

• Whether Liquid could become frontier-scale with enough compute, and why Shopify remains pragmatic and merit-based about model choice

• Who Shopify is hiring right now across ML, data science, and distributed databases

• The Sydney story at Bing, why its personality was not an accident, and what Mikhail learned from deliberately shaping AI character early on

Mikhail Parakhin

• LinkedIn: https://www.linkedin.com/in/mikhail-parakhin/

• X: https://x.com/MParakhin

Timestamps

00:00:00 Introduction: Mikhail Parakhin, Microsoft, and Shopify

00:01:16 Why Shopify Is Talking More About AI

00:02:29 Internal AI Adoption at Shopify and the December Inflection

00:06:54 Token Budgets, Jensen Huang, and Why Usage Metrics Can Mislead

00:10:55 Why Shopify Built Its Own AI PR Review System

00:12:38 AI Coding, More Bugs, and the Real Deployment Bottleneck

00:14:11 Why Git, PRs, and CI/CD May Need to Change for Agents

00:18:24 Tangle: Shopify’s Reproducible ML and Data Workflow Engine

00:21:19 Why Tangle Is Different from Airflow

00:26:14 Tangent: Auto Research for Optimization and Experimentation

00:30:07 How Tangent Democratizes Experimentation Beyond ML Engineers

00:33:06 The Limits of Auto Research

00:36:36 Why Tangle, Tangent, and SimGym Compound Together

00:37:20 SimGym: Simulating Customers with Shopify’s Historical Data

00:42:47 The Infra Behind SimGym

00:46:00 Why SimGym Gets Better with Real Customer History

00:47:30 Counterfactuals, HSTU, and Modeling Merchant Trajectories

00:51:55 CRPs, Clustering, and Category-Level Customer Behavior

00:53:30 UCP, Shopify Catalog, and Identity Linking

00:55:07 Liquid AI: Why Shopify Uses Non-Transformer Models

00:59:13 Real Shopify Use Cases for Liquid

01:03:00 Can Liquid Scale into a Frontier Model?

01:09:49 Hiring at Shopify: ML, Data Science, and Databases

01:10:43 Sydney at Bing: Personality Shaping and AI Character

01:13:32 Closing Thoughts

Transcript

[00:00:00] swyx: Okay. We’re here in the studio, a remote studio, with Mikhail Parakhin, CTO of Shopify. Welcome.

[00:00:08] Mikhail Parakhin: Thank you. Welcome.

[00:00:10] swyx: I don’t even know if I should introduce you as CTO of Shopify. I feel like you have many identities. Uh, you led sort of the, the Bing ML team, I guess, uh, uh, or ads team. I, I don’t know, I don’t know, uh, you know, it’s, uh, people va-variously refer you as like CEO or, or, uh, I don’t know what that, that, that said previous role at Microsoft was.

[00:00:29] Mikhail Parakhin: Uh, that was... Yeah, my previous role w- at Microsoft was the-- I actually was the CEO of one of Microsoft’s business units, which included, as I, you know, as we discussed, all the things that people like to laugh about, uh, including Windows and Edge and Bing and ads and everything.

[00:00:47] swyx: Yeah, yeah. What a, what a, what a wild time.

You’ve obviously, uh, done a lot since you landed at Shopify. Uh, one of the reasons I reached out was because you started promoting more sort of internal tooling, uh, primarily Tangle, but also a lot of people have seen and adopted Tobi’s QMD, uh, and obviously, I think, uh, Shopify has always been sort of leading in terms of, uh, engineering.

I think more-- it’s just more recent that you guys have been more vocal about your sort of AI adoption. Is that, is that true?

[00:01:16] Mikhail Parakhin: Well, I think AI tools in general are fairly recent development, uh, and we’ve-- Shopify, you know, at this stage of its development, we’re developing AI in-in-house and other, uh, building tools that use AI and, you know, interfacing with the wider AI community, uh, you know, are on the sort of the, uh, runaway trajectory.

So it just did by sort of natural byproduct. We, we talk about it more also. We just, uh, just even yesterday, Andrej Karpathy was famous in tweeting about, oh, are there some, uh, ways, uh, that, that you can organize your agents to store the data and then, uh, look up the data so that you don’t have to research or, or lose context every- Yes

time. And a little bit tongue in cheek, I tweeted that, “Hey, we’ve, we’ve done it much earlier, and we even have different approaches, Tobi and I.” Tobi, of course, is a big fan of QMD, and I’m more of a SQL, SQLite fan. But, uh, yeah, very similar things that we’ve already done here. The point is, yeah, we’re very dynamic, you know, explosively growing company, and we have to be at the forefront of AI adoption, obviously.

[00:02:29] swyx: Yeah. Yeah. Um, you, your team kindly prepared some slides actually that we were gonna bring up on to, uh, the screen. I think I can, I can screen share, and then we can kind of go through some of the shocking stats that maybe, maybe put some numbers to what exactly is going on. So here we have, uh- An internal AI tool adoption chart.

What are we looking at here? What ?

[00:02:54] Mikhail Parakhin: Yeah, this is very interesting statistics. Uh, this is number of daily active workers, you know, think of, uh, DAO, basically the active users of-

[00:03:05] swyx: Yeah ...

[00:03:05] Mikhail Parakhin: AI tool as a percentage of all the people in the company, right? And then- Yeah ... different AI tools. And, uh, you could see two things here is that one is the green is total.

Uh, green is just total. So you could see that it approaches really % by now. It’s hard not to do your job now without interacting deeply, at least with one tool. You could see another interesting thing is just as many people commented in December was the phase transition when suddenly models gotten good enough that, that everything took off and started growing.

Uh, it, it was many people noticed that the thing is that small improvements accumulated into this big change in Sep- December roughly timeframe.

[00:03:52] swyx: Yeah.

[00:03:52] Mikhail Parakhin: The other thing I would claim you could see is that, uh, CLI-based tools and tools that don’t require you to look at the code becoming more popular, and you could see, yeah, various versions of, uh, Cloud Code and Codex and Pi and internal development tools taking off.

Uh, exactly, yeah, uh, and blue is our River, just internal agent for coding, where tools, uh, that require IDEs such as, uh, GitHub, Copilot or Cursor, they’re not exactly shrinking, but they’re not growing as fast. Like, uh, red, red line is, is the IDE kind of tools. So you could see that they’re, they’re not experiencing as, as fast of a growth.

[00:04:37] swyx: As I understand it, basically, every employee has their choice, right? Of choose whatever tool you use, and then you’re just kind of doing a, a daily sur-survey or something.

[00:04:47] Mikhail Parakhin: Exactly. And, uh, we- Yeah ... the, the push is to get your job done, you can use any tool, and we effectively fund unlimited tokens for everybody.

Uh, we, we do, we do try to control the models that, uh, people use, but from the bottom, not from top. Like we basically say, “Hey, please don’t use anything less than Opus four point six.”

[00:05:09] swyx: Oh .

[00:05:10] Mikhail Parakhin: Some people, some people end up using GPT five point four extra high. Some people use Opus four point six. Um, uh, you know, uh, there are some, uh, there are plus and minuses in going for full one million context window versus not.

But, uh, we try to discourage people from using anything less than that.

[00:05:28] swyx: Yeah, yeah. Got it, got it. Uh, I mean, uh, that’s, you know... The, the next chart here, it really kind of shows the expansion and the sort of December twenty twenty-five inflection, right? That, uh, people are using a lot of tokens. I think it’s also really interesting that no one was kind of abusing it in twenty twenty-five.

Like it was- Had comparatively, uh, to this year, there was almost no growth. I mean, it’s still like, you know, probably, probably gave fifty percent.

[00:05:56] Mikhail Parakhin: Yeah. This is just a different scale. It’s still exponential- Yeah, yeah ...growth at just a different- ...rate of expansion. Uh, there was inflection point, and Sean, I would claim the, the super interesting part here is that you could see that the distribution becoming more and more skewed.

Yes. The top percentiles grow faster. So that means- Yeah ...the people in the top ten percentile, they, their consumption grows faster than seventy-five and so forth. So, uh, the distribution skews more and more towards the highest users, which is... I don’t know what it tells me. It’s like it feels not ideal, to be honest.

Or maybe it’s okay. We’ll see.

[00:06:36] swyx: Why does it feel not ideal? Is, is it because of, um, quantity over quality, or what’s the concern?

[00:06:42] Mikhail Parakhin: Because take it to the limit. That means, you know, if, if this rate of separation continued- Ah, yes ...a year, there will be one person consuming all the tokens. So it’s just, it’s kinda strange.

[00:06:54] swyx: Yeah, I mean, um, uh, I, I think internal like teaching and all that, uh, will, will help sort of distribute things more widely. But in, in the early days, of course, the people who are sort of more AI-pilled will obviously find more ways to use it than the people who are less AI-pilled. Maybe let’s, let’s call it that.

I’ll just, I’ll just kinda quickly, uh, pause from the, the... You know, we will go back to the rest of the slides, but I just wanna, um, review, you know, there are a lot of CTOs of, of large companies like yourself where they’re all considering some kind of token budget, right? Like I think it’s something, something that Jensen Huang has been talking about, where like if your 200K engineer is not using 100K of tokens every year, like they’re, they’re underutilizing coding agents.

Of course, Jensen Huang would say that, but like it seems a very quantity over quality approach and like some, some people are basically saying like, well, is this comparable to judging engineer quality by lines of code, right? Which we also know is like kind of flawed, but better than nothing. So I, I don’t know if you have like a sort of management take here on, on how to view this kind of, uh, metrics.

[00:08:02] Mikhail Parakhin: Well, I mean, you’re, you’re baiting me. I, I like... This is my favorite topic. Uh, if you let me, I’ll probably talk for two hours on just this. I have a lot of things to say. Like I do think Jensen gotten a lot of bad press saying, “Oh, of course you’re, you know, this, uh, the- ...the cake seller says you don’t need enough cakes.”

You know? Like, of course. Uh, but, uh, I actually, uh, think that’s undeserved. I think he, he’s actually right. Uh, I do think- He,

[00:08:33] swyx: he’s directionally correct.

[00:08:35] Mikhail Parakhin: Yeah. Yeah. He’s directionally correct for sure. Uh-

[00:08:37] swyx: Who knows what the right number is? Yeah.

[00:08:39] Mikhail Parakhin: The thing that I do Uh, want to say, and this is something that we learned through trial and error and very important is like two things.

One is that it’s not about just consuming tokens. Uh, you can consume tokens and, and in fact, the anti-pattern is running multiple agents, too many agents in parallel that don’t communicate with each other. That’s almost useless, uh, compared to just fewer agents and burns tokens very efficiently. Uh, setting up the right critique loop, especially with the high quality models, where one agent does something, the other one, ideally with a different model, critiques it, uh, suggests ways to improve it, the agent redoes it with this critique and, and so it takes much longer.

So people don’t like it because latency goes up. You know, they, they have to wait until this debate is happening. But, uh, the quality of the code is much higher. And another thing, just since you mentioned like, look, uh, uh, yeah, the overall budget is just like, uh, lines of codes. Lines of codes are exploding for everybody right now, or partially because AI is really mover balls, but partially just because AI can write a lot more code, you know, doesn’t get tired.

And so you have to have to have a very strong narrow waist during PR review. Otherwise, just the number of bugs will go through the roof. It’s, uh, it’s this unexpected consequence of the just volume trumping everything. I would claim by now good model writes code on average with fewer bugs than, than the average human.

But since they write so much more of it, like more of it will make it into production. So you have to- You still

[00:10:26] swyx: have

[00:10:26] Mikhail Parakhin: more bugs. Yeah. Have to have a very rigorous PR reviews, also automated of course. But, uh, yeah, that to spend a lot budget there. Like this, this for me, for me, actually, the important metric is the ratio of budget spent during code generation versus, uh, spent, uh, expensive tokens like GPT, uh, five point four Pro or, uh, uh, Deep Think from Gemini, you know, checking on PR reviews.

[00:10:55] swyx: Yeah, totally. Uh, I noticed in your chart you didn’t have any review tools. Do you just use like, like let’s say a Claude code to review tools? Or do you have another set of review tools like the Greptiles, the Code Rabbits, uh, Devin Reviews has a review tool. I don’t know if you’ve had those specialist review tools.

[00:11:13] Mikhail Parakhin: You are a little bit jumping on my store tool right now because the graphs I was only showing public tools. Uh, uh, the-- I haven’t found a good PR review tool that, that does what I think should be done. And, uh, partially my, my thinking is because it’s so... It just goes against both what people feel like emotionally they prefer and, uh, some of the, uh, you know, frankly Even business models that, that the companies run.

At peer review tool, uh, time, you want to run the largest models. That means, I don’t know, Codex or, or, uh, Cloud Code is not gonna cut it. You need to have pro-level models if you really want to, uh, stand the tide of bots from going into production. And you need us to spend a lot of time, the models taking turns, but you don’t want, like, a big swarm of, uh, of, uh, agents.

So in fact, you end up in a different dual-dualistic world where you generate not that many tokens. You, in fact, generate few tokens, but it takes f-a long time because these are expensive models taking turns rather than many, many agents trying to do many things in parallel. So that’s, that’s why I feel like I haven’t found good tools, so we are using our own for peer review for now.

[00:12:33] swyx: Yeah. Yeah. I mean, uh, I think a lot of companies are building their own, uh, especially to their needs, right?

[00:12:38] Mikhail Parakhin: Mm-hmm.

[00:12:38] swyx: Um, I, uh, you also have a chart here going back to the slides on, uh, PR merge growth, where we’re now at thirty percent, uh, month on month rather than ten percent. Uh, and also the, the estimated complexity is going up.

You know, this is productivity, right? ‘Cause y- presumably there’s more stuff going into the code base and more, more features getting worked on. I’m curious about the backlog, right? Like the, the, the-- I actually don’t mind a pro-level model taking an hour or two hours to review my PR, because I’ve dealt with humans who take a week to review my PR, right?

And I keep pinging them on Slack, “Hey, hey, review my PR.” So, you know, I think there’s some trade-off here where, like, it still doesn’t make sense.

[00:13:18] Mikhail Parakhin: Exactly. That, that’s exactly m-my point. Uh, that on one hand, you can tolerate longer latencies at, uh, PR. On the other hand, like right now, the real problem is not in spending time waiting for PR.

It’s real problem is since there’s so much more code than- Yeah ... uh, probability of at least some tests failing going up, and then you, like, keep de-failing, then you have to find the offending PR, evict it, retest it without that PR, and so deployment cycle becomes much longer. Uh, so it actually, in terms of the overall time to deploy, it’s total time savings if you spend more time on a longer model, like thinking for an hour, because then, then you, you don’t have to spend all that time during testing and rolling, you know, rolling back the deployment.

[00:14:03] swyx: Yeah, totally. That’s still worth it. You know, you don’t look at the individual, look at the aggregate, and look at the, the, the change in the aggregate system.

[00:14:11] Mikhail Parakhin: Exactly.

[00:14:11] swyx: I’m kind of curious if, like, there’s this PR mentality and, like, c-- the, the, the CICD paradigm will be changed eventually. Some people are like, obviously a lot of people want new GitHub, but I even wonder if, like, Git is the problem, right?

Like, is that the bottleneck? Is the concept of a PR a bottleneck? Do you guys use stack diffs? I don’t know if, uh, that’s a, like, a merge queue stack diff type of thing.

[00:14:34] Mikhail Parakhin: We, we use, we use Stacks, we u- we use Graphite. We worked with, uh, Graphite a lot. Uh, so we use Stack, uh, PRs. I think, uh, like that’s clearly the overall CICD in general, and the interaction with the code repository right now is the, clearly the sort of the, the main issue and the bottleneck for us, uh, and highest top of mind.

I would say we probably need a different metaphor or different whole design of how to process it in new agentic world. I haven’t seen anything dramatically better yet. I, I think everybody right now is just trying to keep their head above the water ‘cause, ‘cause there, there’s so many PRs and then everybody’s CICD pipelines start creaking, the, the times are increasing, the number of bugs slipping by increasing, and you have to, have to clap on down.

And so we are a little bit in this situation when we need to first stabilize that story and then start thinking, hey, what, what it could be a completely different and new world, which I haven’t... I know some people working on it. I haven’t seen something, like anything super compelling yet, but clearly the old thing were designed for humans will need to be morphed into something new.

[00:15:53] swyx: One of the thing that I, I think about is kind of like the merge conflict is basically a global mutex on the whole system, right? And in, in hu- in human organizations, we do have something like that. It’s the company standup. But like, other than that, it’s like it’s actually fitting for us to be somewhat decentralized, somewhat plugged into one stream of information source, but somewhat lossy.

Like it’s okay, you know, that, that not every delivery is like atomic consistency. Like we’re not dealing with a database sometimes.

[00:16:27] Mikhail Parakhin: This is a very good point, uh, because since humans don’t write code too fast, you know that global mutex is not too bad. Once you-

[00:16:36] swyx: Yes ...

[00:16:37] Mikhail Parakhin: start writing code at the speed of machine, it becomes the, you know, the bottleneck.

Then what do you do? Maybe, and I can’t believe I’m saying this because I, I’m long-- lifelong opponent of, uh, microservices, and I always thought that was, like, a really bad idea. And now that you’re saying it, like, maybe in new guys like microservices will make a comeback, you know, because then you, you can ship things independently in tiny things and, and the managing all that complexity automatically will be much easier.

I don’t know. Like, we’ll s-- we’ll have to see.

[00:17:10] swyx: Yeah. I mean, I don’t know what the Microsoft or, or Shopify thing is, but I, I read this paper from Google where they have a monorepo that deploys into microservices, right? And then, uh, the other concept that I think about a lot is the Chaos Monkey concept from, from Netflix.

Being able to create, like, this robust system where, um, uh, you know, you, you have the service discovery, you have the, uh, the independent, independent microservices discovery and, and, uh, you know, probably going to be a fair amount of duplication. That’s how an organic system sort of scales, uh, that, that you have that...

I don’t know how you call it. Slack? Robustness? Depend-- uh, d-duplication. I, I, I forget the-- I, I’m-- And this-- those-- these are not exactly the terms- Hmm ... I’m looking for, but I c-can’t really think of the words. Okay. I was gonna go into Tangent and Tangle. Uh, so, uh, we, we sort of discussed the overall stats that, uh, Shopify has.

Uh, but, you know, I, I think some, some pretty cool stuff that you guys are working on is your ML experimentation, uh, and your, your sort of auto tr-research training pipeline. Presumably you’re much closer to this one because it’s, it’s a sort of personal hobby of yours. How, how would you explain them in, together?

I thought we have a slide that, like, uh, has the s- the system diagram.

[00:18:24] Mikhail Parakhin: Yeah. Tangle first and then Tangent as a-

[00:18:27] swyx: Yeah ...

[00:18:28] Mikhail Parakhin: as a thing on top of Tangle. And, uh, Tangle is the third generation, I claim, of, uh, systems of, uh, running any data processing, but a bit with a skew for ML experiments, but not necessarily. Any sort of data processing tasks where you need to iterate, share, and you have scale so that you want maximum efficiency.

You know how, like, normally you would work, you would-- Imagine you’re a data scientist or an ML practitioner, you would get Jupiter notebooks or, or maybe you would get, uh, you know, Pyth- your Python scripts, and you would manage the data, and you produce those TSV files, and you put them in some JFS or something.

Then you would notice that, oh, it has this, uh, weird missing values. You go and write another script that, uh, goes and replaces them with, uh-

[00:19:20] swyx: Ah ...

[00:19:21] Mikhail Parakhin: dash S. And then, then you, then you run some, some, uh, “Oh, I need to filter bots.” And so you run some light GBM model that, uh, removes the bots. And then, then you like-- And then you, you kind of like get into shape, and then you start experimenting, and you run multiple experiments, and then you’re like, “Oh my God,” like, “this experiment is worse.”

You undo, and you cannot get to previous result. And like, “Ah, what did I do?” Like that. Again, then, then you finally like get everything working. Then you like start throwing it over the fence to production. You, you replicate it, those things don’t work, and then sometimes you like don’t notice that you forgot some feature naming and the, the features don’t match.

But then, like imagine you, you did everything, and then six months later you’re like, have to repeat it because now there’s more data, or you wanted to do another pass, and you’re like, “What, what did I do?” Or like, or like, “This script crashes now,” or the, “the path has changed.” And then, then you’re trying to, like you spend another month just doing ar- digital archeology on your own, you know, history, right?

Now multiply that by many, many teams. Now imagine you got an intern that you wanna ramp up. Now you have to show that intern, “Oh, you know, look, here’s the folder, there’s the scripts, you know, ask your cloud agent to do, and then, uh, to, to figure it out.” And then cloud agent does something, and then you’re, “Ah, yeah, right, right, it was the wrong folder.

I forgot to tell you, I actually have this other thing I forgot myself.” And, and that’s, that’s the, like, the daily life we all, uh, all know it, uh, if, if you’re a data scientist, machine practitioner, ma- machine learning practitioner or, uh, or even like any data managing, uh, person.

[00:21:00] swyx: Yeah. So I, I used to do this, uh, f- uh, on the quant finance side, uh, in, in my hedge fund.

So we did this before Airflow, and then, uh, obviously Airflow came along and, uh, then more recently Dagster, uh, I would say is like, in my mind, what I would use for that shape of problem, uh, where you had to materialize assets and create a pipeline.

[00:21:19] Mikhail Parakhin: And that’s, that’s very good segue because... So Airflow is great, but Airflow is more about you, you have something and you wanna repeatedly run it in production on schedule.

It’s less about you as a team developing things and being able to share, and you grabbing the standard pipeline and saying, “Hey, I wanna change this tiny little component in the huge sea of data processing, and I don’t wanna-- I wanna run ten experiments on this, and I wanna do hyperparameter optimization.”

All that is very hard to do with Airflow. It’s very easy to do with Tango. Tango is m- more about, it’s everything about group of people Running experiments, it might be agents too nowadays. Uh, running experiments cheaply, collaborating, sharing results. Uh, you don’t need to understand fully. You, you grab-- you clone somebody else’s experiment or somebody else’s pipeline, uh, run, uh, change small piece, run it, be, like, get it to production state, and then ship in one click.

So then the... You don’t have to port it into any other system to, to run in production. You can just run the same experiment. It’s, it’s fully production ready. And, and it’s, uh, it has lots of... Again, as I said, it’s third generation system. The original one was, I would claim there was Ether and then, uh, at least in my career, Ether was the first, first, uh, that pioneered this type of approach.

And then there was, uh, Nirvana, which, uh, uh, at Yandex, which did kind of sec-second take on this. And now this one aggregates the, the learnings from all of those and, and Airflow as well to, to get to the state where you try it, it, it feels kind of magical. Uh, ‘cause now everything is based on content, uh, hashes.

So even if the version changed, but if the output didn’t change, nothing is being rerun. It’s very efficient. If you... Multiple people start experiment that needs the same sort of data preprocessing, it’s not repeated multiple times. It’s automatically done only once. If you start ten experiments that all require, you know, some, some data preparation first as the first step, and you don’t have to coordinate for that.

Like, you don’t have to know that other people are starting it. You now, it’s very easy compos-, uh, composability, any language you can u- uh, you wanna use, and it’s very visual. So you can see immediately, you can edit it easily, you can assemble small things with just even mouse clicks if you want to, and, uh, share, clone.

And everybody knows also it’s fully kind of static in the sense that we rerun it second time, it will exactly have the same results. Like, you will never have to do digital archeology. So full versioning and everything is also there.

[00:24:06] swyx: Uh, so, so people can, uh... It’s open source. Go to the GitHub repo and, and, uh, check it out.

Uh, and it is also a really good, uh, blog post about it. I think all these is, like, really appealing. The, the, the, the thing that I think sells me the most about it is that, um, sort of development to production transition, right? Which I think, um, a lot of people haven’t really solved that, uh, strictly, right?

Like, we develop really, really well in, in Python notebooks, but then, you know, that’s obviously not a sort of production ready process. I think that, like, any way in which that is solved, I think is, is very appealing. Then the other thing that you mentioned, which also raised my eyebrows, was content-based caching, which you mentioned is, is, um, you know, is ve-very much, uh, um, a sort of efficiency measure about, uh, you know, just like recalculation only on, on sort of content addressing Which I think makes sense.

Uh, it surprised me that the savings could be this much, but maybe I just haven’t worked at your scale where there’s so much duplication, uh, that people just rerun because they change a single ID upstream.

[00:25:10] Mikhail Parakhin: It does, yeah. But it’s not only you rerun. The, the main savings are coming from the fact that you ran it, you got your job done, and you moved on.

Then- Yeah ... somebody else in some department you don’t know existed runs the same task, but on a newer version.

[00:25:27] swyx: Yeah.

[00:25:27] Mikhail Parakhin: Like right now, you can’t, in, in most of the organizations, you can’t even find out about it so that you can’t even measure that you’re spending that time twice, right? Here- Yeah ... if everybody’s on Tango, that’s detected automatically and detected that the output is the same.

And then for that person, all it looks like is like experiment just suddenly moved, jumped forward, right? Uh, uh- Yeah ... so that’s because, because the, there’s network effect of multiple people helping each other.

[00:25:51] swyx: Yeah. This is one of those things where it’s designed to be a platform from the beginning rather than an individual developer’s tool from the beginning, right?

And, and everything’s gonna streams down from there. That is the sort of Tango, uh, orchestrator, and it’s, it manages jobs. We’ve seen a few versions of this, and this is obviously, uh, uh, the sort of, uh, unique approaches that you guys have, have, uh, figured out. And then there’s Tangent.

[00:26:14] Mikhail Parakhin: Yeah. And Tangent is basically an automatic auto research loop that can help and kind of do your work for you.

Uh- ... you know, uh, effectively, effectively, Andrej Karpathy recently popularized it with auto research. Yes. Remember he said like he was, uh, speed running this, uh... Yeah, uh, you know the story. The, here we’re basically bringing the same capability into Tango so that, uh, the, uh, Tangent can analyze it. It’s just an agent that can run multiple experiments, figure out what can be changed, and keep on rerunning it, keep on modifying until, uh, maximizing some goal, some loss function, whatever you need to, to achieve.

And in general, I would say if you’re not using auto research-like approach in whatever you do, like literally whatever you do, then you’re missing out. We saw at Shopify that taking like a wildfire, anything where you can put measurements can be done dramatically better. Our-

[00:27:19] swyx: Mm-hmm ...

[00:27:20] Mikhail Parakhin: uh, speed of, uh, templatization HTML, uh, completely new UX tem- uh, templatization of, uh, reducing latency for liquid themes.

Uh, we-- Our, uh, search, uh, recently we moved from It’s hard even, uh, quote from eight hundred QPS to forty-two hundred QPS with the same quality just by pure optimizations and not a research loop that kept running and changing code in our index serve on the same number of machines, just increasing the throughput.

We, we managed to improve the quality of gisting and machine learning process. Uh, you know, gisting is the prompt compression technique that

[00:27:59] swyx: allows for

[00:28:00] Mikhail Parakhin: lower latency and, and lower and, uh, actually higher quality slightly. So like literally whatever different walks of life, and it doesn’t have to be AI related.

Uh, we, we had a reduction in, uh, storage because the agents would go and find data sets that clearly are derivative, uh, and then you don’t need to store things twice. You know, we, we, we found somewhat embarrassingly that it was one of the largest tables was hashing random IDs into another random ID, and we literally- Oof

put only one. So it was translating, yeah, two random IDs hashed

[00:28:36] swyx: into

[00:28:37] Mikhail Parakhin: each. So, so

[00:28:37] swyx: it has access to the code as well, so it can, it can check the, like what, what the hell is it doing?

[00:28:42] Mikhail Parakhin: So there, there cou- it could be run in two levels. You, uh, you know, at the superficial level, it could just use ex-existing components and, uh, reshuffle them.

Uh, you know, like you can grab- Yeah ... uh, XGBoost, and you can grab some, some Py- PyTorch module, and then can grab some, you know, grab another tools and, and combine them. At a deeper level, since Tangle is all sort of CLI based underneath you, every, every component is a wrapped really CLI, uh, call and a YAML file, it can analyze code and create new components and, and, uh, keep on iterating as well.

So, so you can, you can both have quick modifications of existing t- uh, pipelines with the, with components that are already there pre-baked, or you can create new components, uh, and-

[00:29:29] swyx: Yeah ...

[00:29:29] Mikhail Parakhin: keep iterating on those. So auto research is, again, this is probably the, the thing I was excited the most in the last two months happening, and we see it taking like, like totally like a wildfire.

Just, uh, everybody, every day, every... well, every day, every minute, I would, uh, have somebody Slack message saying, “Oh, look how much better I made it.” And, uh, it’s all throughout the research.

[00:29:53] swyx: Is this democratized in some way in, in the sense that like is it your ML, uh, engineers and researchers doing this, or is it your regular PMs and software engineers also have the ability to auto-- to use Tangent?

[00:30:07] Mikhail Parakhin: This is an awesome question. Like, Tango in general and Tangent in particular are extremely democratizing. Like they- Yeah ... they are the main tools for- ‘Cause I don’t

[00:30:15] swyx: need the details.

[00:30:16] Mikhail Parakhin: Yeah. Exactly. Initially used by ML and AI engineers, but then literally, as you said, PMs are like the highest user right now is one of PMs on our org, uh, Sartak and he was, he was number one by, by usage of, of this ‘cause they’re just, uh, energetic and knowledgeable, and now it, it unlocks a lot of capability where you don’t have to co-change code manually.

[00:30:39] swyx: I mean, I mean, because it kind of cuts out the ML, ML engineer from the process because the, the, the PMs have the domain knowledge and the ability to think about, uh, from first principles about, okay, what, what results do I want? And they can-- they even have the access to the data that, that needs to go in.

So it’s like in some ways, like this is the magic black box that we’ve always wanted for, for training and, and for, uh, I guess, uh, uh, hill climbing, whatever.

[00:31:04] Mikhail Parakhin: It’s basically cloud code for your AI development- ... uh, situation, right? Like now, now you don’t have to know exactly how algorithms work. You can just, uh, bring your domain knowledge and expertise and product knowledge and iterate within Tangent until you’ve gotten the results that you need.

[00:31:21] swyx: In my previous roles, every time that someone has pitched AutoML, you know, I’ve always been like, “Uh, this is not, this is not gonna work. It’s, you know, it’s, it’s always gonna be a flop.” Somehow it’s working now. I mean, presumably the answer is now we have LLMs and it’s good enough, right? It’s, it’s an emergent property that we can do auto research, but like, it doesn’t feel that satisfying that how come we didn’t do this before, right?

Like we just did like parameter search and like, I don’t know. That’s maybe that’s it.

[00:31:48] Mikhail Parakhin: Yeah. Bayesian optimization and hyperparameter optimization was, was the one that, or facet of AutoML that was used very actively, which incidentally also built into, uh, Tango. But, you know, I know Patrice Simard very well, and, uh, he was such a, uh, such a proponent of AutoML, and he put, like literally spent careers trying to democratize it.

Without LLMs, it just turned out to be very hard. Like it, you, you would have flexibility within certain narrow domain, but it was hard to wider scale, and now with LLMs suddenly it’s like magic wand, and so suddenly everybody- ... is an AutoML expert.

[00:32:28] swyx: Yeah, I, I think it’s multiple things, right? Like I’m, I’m just gonna bring up the, the, the chart again, right?

Like LLMs can do the monitoring very well. That is the very potentially unbounded, super unstructured. It can do the analysis very well, it can do the... Uh, and basically it is much more intelligence poured into every single step. Uh, there’s maybe nothing structurally changed about AutoML, but this is just m-more intelligent and more unstructured.

[00:32:53] Mikhail Parakhin: Exactly.

[00:32:54] swyx: Any flaws that you’ve run into? Like everyone is like drinking the Kool-Aid, oh my God, time savings, uh, you know, performance improvements. Like what, what, uh, issues have you have, uh, come up?

[00:33:06] Mikhail Parakhin: This is really cool. It’s not a solution to all the world’s problems for sure. The limitations are usually the ones I-- And this is where we get into a bit of a subjective territory.

Uh, I can only share what I’ve, I’ve seen so far, and I’m sure the situation, uh, is changing, and, you know, maybe after I say it, like many people will reach out and say, “Hey, what about this?” And you don’t know that, and then, then we’ll be probably right. But what I’ve seen is auto research is very good at doing kind of obvious things that you don’t have bandwidth to do or you didn’t notice or maybe you’re not aware of like the-- some standard practices.

It is not good at doing something completely out of distribution, something that, you know, you have to think for, for multiple days, uh, and, and do something like none of this. So, so it’s, uh, I, uh, set an experiment once, uh, on, on my sort of, uh, hobby thing, and I let it run for, uh, ended up, uh, several weeks run, uh, you know, it’s like full production kind of scale, so it, you know, slow runs and, and it ex-- it performed in the end, uh, over four hundred experiments, and only one was successful.

I’m like, “Okay, that’s, that’s good.” But-

[00:34:18] swyx: But it saved time.

[00:34:19] Mikhail Parakhin: Yeah, I saved time. Like it, it was the, that thing. Yeah, if I, if I were doing four hundred experiments myself, my betting average, as I said, would have been much higher, I’m sure. But also, first of all, it would take me like three years to do four hundred experiments.

And, uh, I didn’t have to do them. Like the machines were just, uh, the price of electricity did that. So, and I got one improvement, uh, that in, uh, my, my-- Honestly, when I was starting that experiment, my thinking was to go and show that, “Hey, Andre, maybe you just don’t know how to optimize.” And I was super smart because in, in my pro-problem, it was optimized for many years, and it was like fully improved.

Uh, and I didn’t expect it, you know, auto research to find anything at all. Yet it did. So instead of making fun of Andre, I ended up, uh, a big, big supporter. Yeah, that’s exactly the tweet. Yes.

[00:35:10] swyx: You and Toby really, really go back and forth on-online a lot, which is really funny. Uh, think of it as, as an eval for the optimalness of the code it’s running on.

Uh, it’s almost like it reminds me of like a Kolmogorov complexity thing, but, uh, I guess it’s-- there’s some optimal thing that you’re trying to sort of reduce down to, I guess. Um, and so, so you, you, you know, you should congratulate yourself that you had, uh, you know, uh, ninety-nine percent, uh, optimality.

[00:35:36] Mikhail Parakhin: Exactly, yeah. I think Andre really deserves a lot of credit for popularizing this approach. This is, uh, this is incredibly, I think, powerful and cool and You know, the, uh, even him, him just mentioning it led to a lot of gains in a lot of places in the industry, so we should be thankful.

[00:35:56] swyx: Yeah. I think he also has a just...

I don’t know what it is. Like, um, you know, it, it is a simple self-contained project that people can take and apply to other things, which is, is, is one thing, but also just the name. Just like somehow no one, no one managed to call their thing auto research. It’s just naming things is very important. I think that that is mostly, uh, our coverage of Tango and, and, uh, Tangents.

I think obviously, you know, there’s a lot of, uh, ML infra at, at Shopify that people can, uh, dive into. We’re about to go into SimGym, but before I do that, any, any other sort of broader comments around this whole effort? Like where is it, where is it leading to?

[00:36:36] Mikhail Parakhin: As a segue to SimGym, like all those things start composing strongly.

And, uh, you could see a huge unlock when you can look at each one of the tools and, and you see, oh, they’re extremely useful. Uh, Tango is useful by itself. Auto Research is useful by itself. SimGym is useful by itself. If you combine all three, you create like synergetic effect. I think that’s why we wanted to even, uh, cover them today is because this is something that if you go back even, you know, five years ago, would’ve been unthinkable.

Uh, replicating that, uh, would, would be either incredibly costly or impossible, right? With probably thousands of people are required.

[00:37:20] swyx: Well, we have serverless human, uh, serverless intelligence, right? Like, uh, so yes, you do have thousands of hu-- of, of intelligences, not just, not humans. And that’s, that’s close enough, right?

Even if they’re not AGI, they’re, they’re close enough to do the, the task that you need them to do. And, and, you know, that’s, there’s plenty for, for a lot of routine work, knowledge work. Okay, let’s get into SimGym. Um, this is one of those things I, I was surprised to see actually it’s apparently your, uh, one of your most popular launches, and I think something that, uh, I think Sim AI, I think Yunjun Park, who did the Smallville thing, there’s a very small cottage industry of people trying to do like the simulate customer thing.

I think a lot of people maybe don’t super trust this yet because they’re like, well, obviously they would just do what you prompt them to do, right? But maybe just think, uh, tell us about the sort of inspiration or origin story.

[00:38:10] Mikhail Parakhin: That’s exactly actually the thing I wanted to cover, because if you don’t have the historical data, all you can do is prompt a-agents in a vacuum, and they will do exactly what you prompt them to do.

In fact, when I first proposed it, and this is a bit of, um, my brainchild initially, if I, I can boast, even Toby said like, “But wouldn’t they, they just repeat what, what you tell them?” And, uh, but I’m like, “Yes, except Shopify has decades of history of how people made changes and what there is, uh, there, what it resulted in terms of sales.”

So now what we can do is we can-- we have this... It’s not, it’s a noisy data. There’s a small, usually websites, uh, you know, like things, things are never in isolation. It’s almost never AB experiment. It’s always AA experiment when there’s has two meanings, but basically, you know, in different time you run two different things.

But if you aggregate in general, uh, like everything together, and you apply, uh, denoising and collaborative filtering like approach, you can extract a very clear signal. And then you can optimize your agents. And that’s why it took so long. It took almost a year of that optimization of just us sitting and fiddling, and, and we had this internal goals of correlation of hitting-- internal goal was to hit zero point seven correlation with, uh, add to cart events, for example.

Like that, that if we run real AB test experiment, that it should, it should go and, and rep-uh, replicate, uh, same sort of success that, that humans had or lack thereof. And it, it took forever, and I don’t think that’s easily replicatable because, uh, like who else would have that data? You have to have this historic, you know, decades, uh, worth of data.

And now, now the, like the other thing you need is in-infrastructure and the scale, right? Because, uh, w- again, what we found, uh, stat sig results, you need to run a lot of simulations, a lot of agents, and, and it’s-- Those are expensive things. Like you’re, you’re making actions in the browser because you want a real friction.

You want to, to be able to get the image like of what humans will see because you wanna, uh, detect effects like, “Hey, if I make my images larger, will I have more sales or l- uh, fewer sales?” And like usually people’s intuition here, by the way, is that I increase my images, I will have more because they look nicer.

You know, designers all look sparse and big images. Like usually your sales tank, right? But, but, uh, you know, from HTML, all the characters look the same only the, the size tag looks different, right? So it’s very hard. So you have to take visual information, you have to run this in simulated browser environment on the big farm and, and of course, you have to have, uh, like very, very expensive model, good model with multi-model model.

So all this it’s-- is what’s taken so long and, uh, to share my personal fail a little bit there, Sean, is like, you know, we always had this bias to-- for like large company bias. You know, we always, uh, whenever you-- we do, we’re like, “Hey, we’ll run an experiment,” right? We make, make a change, and we will run an experiment and then, uh, see, uh, see which one’s better or like, “No, this is worse,” and most of them are worse, so you discard it and keep iterating, hill climbing.

And we’re like, “Oh, like smaller merchants, they cannot get stat sig results. They cannot really run experiments simply because, you know, in a week there would be not enough data for them.” So we thought from this perspective. What we didn’t realize is that most people don’t have A and B, they just have one thing, and they need suggestions of What A and B should be.

So, uh, we first build this, hey, we run simulation on two separate teams and, and, uh, say, “Hey, which one is better?” We then morphed it into, and very recently just released it, when you have just your site, your theme, we run over it and we say, “Hey, here’s what predicted values of, of, uh, uh, conversions are, and here’s how we think you should modify it to increase your conversions.”

And then circling back to what you started with, the proof is in the pudding. Like, if we are not correlating with reality, like, people will not be using it. And, uh, thankfully, we see literally every day more users than the previous day. So, so right now, uh, right now- It’s working. Yeah. I’m-- Right now my problem is how to pay for it all because the so our major thing is how to optimize the LLMs, do distillation, how to run the headless browsers, uh, and handful browsers, uh, uh, cheaper so that we can accommodate the increase in traffic.

[00:42:47] swyx: Yeah. I, I understand that you, uh, you published a lot of technical detail at GTC, so I was just gonna bring it up a little bit. I think s- was this in, in con-conjunction with some kind of GTC presentation? Or something like that, right?

[00:42:59] Mikhail Parakhin: Well, we, yeah, we, we did it in several place, but yeah, we had the engineering- Yeah

blog, uh, as well. Yeah.

[00:43:05] swyx: Yeah. So you’re running, uh, GPT OSS. Uh,

[00:43:08] Mikhail Parakhin: the, this is an older version. You know, now we run multimodal model. But yeah- Yeah ... GPT OSS, we still run GPT OSS as well for

[00:43:15] swyx: And then you have the VMs, and you also have browser-based. I really like this one where it you said, “It violates almost every assumption that standard LLM serving is designed for.”

And then you had like, basically orders of magnitude differences between everything.

[00:43:29] Mikhail Parakhin: Exactly. Which is, which, uh, which was, you know, a bit of a challenge to implement, like when, like even simple things. Uh, be- since it violates all the assumptions, for example, multi-instance GPUs, like MIGs don’t work as well.

But we needed, uh, to get MIG to work because, ‘cause otherwise it’s way too expensive. And so we had to deal with the, yeah, with, uh, lots of infrastructure and, and, uh, work with, uh, uh, Fireworks and CentML, uh, you know, to help with optimizations and browser-based, as you mentioned. Yeah, like, takes a village.

[00:44:04] swyx: Okay. So there’s a lot of like, I guess, experimentation in the infrastructure so far, and you’ve published more or less what you have here. I guess I’m, I’m less familiar with CentML. I, I don’t do, uh, that much work in this, this part of the stack. But why was it the sort of preferred instance platform?

[00:44:22] Mikhail Parakhin: There are really three probably top companies. There used to be, uh, uh- Three top companies, uh, at least I was aware of that did, uh, LM optimization. You know, together Fireworks and Santa ML, not necessarily in that order. Santa ML recently got acquired by NVIDIA. Uh, what they did is if you have a model and you want to optimize it to a specific prof-- uh, profile of usage, uh, they would go and do it.

And, uh, we work with, with those companies, uh, this was work particularly in with Santa ML and NVIDIA to get them the best possible results out of it. And, and sometimes you, you have to retune depending on, like sometimes you want the maximum throughput, sometimes you want minimal latency, sometimes you want like the cheapest, right?

And, yeah, or some combination. And so yeah, these are people who would come and help you.

[00:45:14] swyx: I see. I see. Yeah, yeah. I’m familiar with these people for the LLM, you know, autoregressive stack. But the other interesting category of these optimizers is also the diffusion people, whereas like Fel and, you know, uh, Pruna recently has come up a lot as well, which I think is like really underappreciated, uh, at least by myself, because I, I thought, oh, all the workload would be LLMs, but actually there’s a lot of diffusion as well.

[00:45:38] Mikhail Parakhin: Exactly.

[00:45:38] swyx: There’s a lot here, so I, I, I... it’s, it’s, uh, it’s, it’s, it’s hard to cover. But I, I do think like people underappreciate the importance of customer simulation, basically. I think this is something that I’m candidly still getting to terms with. Uh, you know, uh, you also-- your team also like prepared this, like, really nice diagram.

Uh, I, I assume this is AI generated.

[00:46:00] Mikhail Parakhin: Yeah, it looks-

[00:46:01] swyx: Maybe it’s not.

[00:46:01] Mikhail Parakhin: Yeah, it looks, uh, Gemini-ish. Yeah, but, uh, uh, honestly, I, I don’t know where, where the hell they generated. It looks, look, uh, looks like it’s, uh, Google. But the interesting part, John, that, that, uh, we haven’t covered, but I, I wanted to mention is if your store had previous customers, rather than it’s a new store, you’re like new merchant just launching things, it helps tremendously in just correlation and forecast.

Yeah, we take your previous, uh, customer’s behavior, and we create agents that replicate those specific distribution of, of customers that you get, and then we a- we apply those to your changes, and then that, that raised raw, you know, the re-- uh, just correlation with the add to cart events or to-- with conversion or whatever it, it, it may be, uh, quite dramatically.

So, uh, replicating humans in general seems like an interesting, cool challenge.

[00:46:58] swyx: As a shareholder, I think this is the-- like if people are Shopify shareholders, they should really deeply understand this because this is basically the moat. The, the more you use Shopify, the more it will just automatically improve, right?

Like you’re, you’re doing the job for them.

[00:47:13] Mikhail Parakhin: Yeah, that’s what we started with. Like, uh- ... uh, otherwise, if you’re just a startup, I wouldn’t do it if, uh, you know, if it was my startup because Without the data, it, yeah, as, as you said, it’s, it’s exactly the case that, uh, whatever you say in prompt, that’s, that’s what the agents will be doing.

[00:47:30] swyx: The statistician in me wants to like really satisfy the sort of, um, statistical intuition, I guess. Um, to me it’s kind of, uh, the, the word that comes to mind is, um, ergodicity. Uh, so let’s say a, a customer takes this path, customer takes this path, customer takes this path, right? Um, the... In my mind, the way I explain it is like, okay, here, here’s the ninety-five percentile, here’s the five percentile, and here’s the median, right?

Um, but to me, what SimGym is potentially doing is that it can, uh, modify... It can sort of model the sort of in-between sort of journeys as well, that, that maybe are dependent on the previous states. This may be like a very RL-type conclusion where like basically the summary statistics, if you only did naive AB testing, you only have the, the statistics at, at, at a certain point, and you only judge based on the sort of overall summary statistics.

But here you can actually model trajectories. Does that make sense? Or-

[00:48:31] Mikhail Parakhin: That makes total sense because like, well, that, that makes even more sense that maybe even you realize bec- because-

[00:48:38] swyx: Okay. Please,

[00:48:38] Mikhail Parakhin: please. Yes ... we do-- Yeah. The, so internally, uh, we have this system, we talked about it briefly once at NeurIPS.

We have a huge HSTU-based system that models the whole companies, uh, and their possible paths. And like- Yeah ... what you are, what you are showing, like actually at any point of time, you can either model the user’s behavior or you mo- can also think about, uh, the whole merchant as a company, as the entity that acts in the world.

You can model that as well. And then you can do, can do counterfactuals. In your graph, like in your blue graph, uh, if you’re... Imagine in the center there, uh, somewhere in the middle, you would have an intervention. I give that person a coupon, or I don’t know, I send a personal thank you card, or give a discount in some- somewhere.

And then you can, uh, then you can do forward rollouts from that counterfactual. So what would have happened with that intervention or without the intervention? And you can even ch- change where that intervention, uh, in time can happen, right? Like some- where, where in this journey. So we, we do this at the Shopify scale for our merchants, and then if we notice that something that they can be fixing, like there’s a strong counterfactual, like we have Shopify policy, they basically get a notification like, “Hey, we think your...

something is wrong with your-” I don’t know, Canadian sales. Like, uh, it looks like it’s misconfigured. Here’s what you need to do. Or do you think like, uh, you have to set up this campaign with these parameters? And we do that at the buyer level to literally offer discounts or cashback or, or things to buyers.

So this is-- I’m getting very excited. Like this is my sort of area of, uh, interest, I guess, and, and hobby. But being able to m-model something complex as human beings or companies and model counterfactuals on it, where you can have interventions in the future and optimize when to make intervention, what kind inter-- uh, what kind of intervention to make.

It’s such an unlock that previously was completely impossible. Like the-- it was, it was always dreamed of, but never... Like how would you even simulate it without LLMs or HTUs? I think very, very exciting times.

[00:50:59] swyx: I just wanted to, uh, to maybe illustrate this. I, I’m not the best illustrator, but I, I am a conceptual statistics guy.

And y-you know, you cannot just do this. Like this is a dimensionality AB test doesn’t do, right? Like, uh, because it doesn’t have the, the, the change over time, uh, stochastic nature, uh, and it doesn’t have the sort of contextual like... Here’s all the context to this point. Um, okay, cool. Um, that’s SimGym.

You’re, you’re gonna burn a lot of tokens on this thing. But you’re, you’re one of the, the only scale platforms in the world that can, uh, that can do this across a huge variety of workloads, right? I’m even curious on a sort of human, uh, research level of like, well, do, does retail behave d-differently from like clothing sales?

D-does that behave differently from electronic sales? I, I don’t know. I don’t know what else you guys... The Kardashian shoppers, do they differ from like people who buy, uh, I don’t know, cars and, uh, whatever.

[00:51:55] Mikhail Parakhin: Well, very different, and different sensitivities and different modes of, uh, shopping and, and different levels of what’s important.

Now, to-totally, you can do aggregations at, uh, at a store level. You can do aggregations at a different, uh, category level. I don’t know if, uh, you know, for our statisticians among us, I couldn’t believe, but we-- recently we’re looking at it, and we had to bring back, uh, CRPs, you know, Chinese restaurant process.

It’s a, like, way of aggregating and, like, naturally grow clustering. So across... Specifically to answer questions that, uh, like you were just posing on how, how if, if buyers behave different categories. And I’m like, “I haven’t seen CRP since two thousand and one.” It’s

[00:52:37] swyx: so What? It’s so- What is... No, I haven’t, I haven’t seen this.

No. This is not in my training. Uh,

[00:52:44] Mikhail Parakhin: but, but yeah, it, uh, uh, it actually, like the, the-- there was a very popular kind of theory, popular neurips HTML circles in early two thousands, uh, kind of nice. And now, now it has practical applications, uh- Yeah ... that we were resurrecting.

[00:53:03] swyx: Yeah, amazing. Uh, I, I can see, I can see how this is like a, uh, a fun job for you where you get to apply all these things.

Um, yeah, yeah, so super cool. Super cool. So, okay, so, so anyone who, who knows what CRPs are and has always wanted to use them at work, uh, they should, they should definitely join Shopify. Okay, so w-we have a lot and but I, I’m, I’m being mindful of the time. I, I do wanted to, to sort of cover some other things.

Um, I-I’ll give you a choice, UCP or Liquid?

[00:53:30] Mikhail Parakhin: Liquid. I think, I think on UCP, you know, like UCP is very important for us and, and it just we are-- UCP, we have a structured, uh, discussions, and you can read about them, and we have, uh, blog posts, and we have a big release this week, in fact, like with our catalog.

Oh,

[00:53:46] swyx: okay.

[00:53:46] Mikhail Parakhin: Uh, yeah,

[00:53:46] swyx: but- Le-I mean, we, we can, we can discuss the, the, the release briefly because we’ll release this after the-- after it’s already announced so whatever. There’s a catalog that you guys are doing?

[00:53:55] Mikhail Parakhin: Yeah. So we are, we are- Okay ... we are bringing in capabilities of a whole, uh, Shopify catalog.

Basically, you now you can search for products, you can do lookups by specific ID, you can do bulk lookups when you need to bring m-multiple products. You don’t need to know in ad-in advance what you’re trying to show or to sell or check out. Like, you can now, you can now have this decided at, at runtime, and this big area for investment for us for both non-personalized and personalized searches, trying to provide basically a win-window into whole universe of products that are being sold everywhere in the world.

And Shopify is really not exactly, but almost like a super set of any-anything being sold. Now we are bringing it into UCP and, uh, and, uh, identity linking is another big thing for us, uh, so that you, you can use, uh, like Google or whatever, whatever identity you have, uh, they’re minimizing friction.

[00:54:56] swyx: Yeah. So

[00:54:57] Mikhail Parakhin: yeah, big release for us.

But Liquid AI of course we never talk about, and the problem might be more, more aligned with what we d-discussed previously on this chat.

[00:55:07] swyx: Sure. The main thing that everyone understands about Liquid is that it is inspired by Worm, and I still don’t know why. I’m curious on your explanation. I think you, you, uh, you can make things very approachable.

And also I think like what is the potential of like the, the level of efficiency that you get out of Liquid?

[00:55:23] Mikhail Parakhin: You- we all familiar with transformer architectures. And, uh, for the longest time, there was a competing architecture, it’s called the state space models. So, so Sams, uh, you know, Chris, Chris Reyes, one of the pioneers and, and lots of startups, uh, trying to make those realities.

They have, uh, significant benefits being main being, uh, being much faster and, uh, lower footprint and not quadratic in length, you know, sort of, uh, linear in, in, uh, in your context length. But with state space models- They never quite made it. Like they’re used-- They have, uh, certain niches when they thrive, their hybrid architectures are useful, but they never quite made it.

And liquid neural networks are, you can think of them as a next step, like, uh, sort of, uh, state-space model square. It’s non-transformer architecture that’s more complicated than sta-state space and really difficult to code if you-- if I’m being honest. But it’s, um, very efficient. It’s, uh, subline-- sub, uh, quadratic in, in length of your context.

Uh, it’s very compact way to represent things, and that’s a liquid AI company. They... Their goal is to productize it, and very often you have this need, uh, when you need to have long context and small model, and you want to have low latency. Like in general, it’s basically on par with transformers, and if you do hybrids with transformers, it’s, it’s even better.

That’s why we at Shopify, when we tried multiple and we constantly try multiple models, multiple companies, we found that for small, particularly with low latency applications, when you have low latency and/or if you need longer context lengths, liquid was the best. And so we still use the whole zoo and always like obviously test and use everything, uh, every open source model and, you know, it feels like sometimes even every private model.

Uh, but liquid’s been taking quite a bit of, uh, at least internal Shopify share. And the reason I’m excited is, yeah, because it’s, it’s the only non-transformer architecture that I found being genuinely competitive. Uh, and, uh, you know, for we use it for search and for, for long context, uh, pulse distilling and others.

This is the overview. I don’t know how approachable Sha, sorry. Maybe, maybe still too obtuse.

[00:57:51] swyx: I, I mean, I think they haven’t been that open about their implementation details. I think the... I would say like liquid hasn’t been like if there’s a lot of technical detail published, I haven’t read like a, a formal sort of paper on the implementation details.

Uh, but I, I did get the sort of relationship between the SSMs and the others. This is one of the sort of, uh, charts that was, you know, showing the relationship between like full attention versus Something that’s, uh, more like a RNN type in terms of their, their efficiency. Um, and then the, the other chart was this old one, uh, where it compares versus, uh, some of the other models.

Uh, doesn’t exactly have the correct Y-axis, but close enough where you can see like it’s basically a, a step change difference in terms of the efficiency. I think the surprise to me was that you guys are, uh, actively using it already in internally inside of Shopify. And like I, I’m curious, like what are the constraints that you’re optimizing for, right?

Is it when you say smaller, is it like the 1B size? Uh, what kind of like latency constraint are you, are you optimizing for? What kind of context length, um, sort of considerations, right? Like I think for example, right, like in the audio kind, kind of use cases, the SSMs ef-effectively have unbounded context length because they, they just have to operate on like the most, the sliding window of the most recent stuff.

Uh, I’m just kinda curious, like w-what do you see the potential here?

[00:59:13] Mikhail Parakhin: Yeah. The SSMs are effectively because, yeah, because the state embeds all the, all the previous information needed, or that’s the assumption. SSMs effectively have infinite context length. The, the problem with, uh, with them is that expressiveness is not there.

The, uh, uh, Liquids are effectively souped up SSMs. We are much more expressive, m-uh, com-more complicated again to code. There is, there is a paper on it. You can, you can see it. Differential equation rolled out and, and then computed as a, uh, as really as a convolution. It’s a bit involved. The thing where we, we use it is specifically either for where we need super low latency, and we’re-- there was a lot of very fun project with, uh, Santa ML and Liquid AI themselves.

We run it at, uh, thirty milliseconds, a, a tiny model, like three hundred million parameters in, but we run it in thirty milliseconds, uh, end to end for search when you, when you type a query, and then we produce all the possible things what you, what you can mean by that query and some, you know, uh, not only synonyms, but, but, uh, a que-kind of full query understanding the, the whole tree of what you might need and including your personal personalization because you might have done like previous queries and lowering it all down into the search server so that the requirements on latency obviously they are very, uh, very strict.

So, so then we are able to run it under thirty milliseconds because, ‘cause at Liquid, you know, Qwen doesn’t run on this. And even Liquid, we had to work a lot with NVIDIA and to... because almost everything is not designed in CUDA for or in, in the current stack for, for low latency. Like small things that don’t matter with large models, you know, start mattering a lot, and we had to optimize it.

There is different end of the spectrum where this is maximum through, uh, bandwidth throughput for things like, for example, offline categorization when A new product appears. We need to do analysis. We need to assign where it is in taxonomy. We need to extract and normalize attributes. We need to do, uh, you know, clusters like, oh, it’s the same thing as that other merchant is selling, right?

That is like un-- like almost unbounded, uh, amount of energy you need to spend on it because it’s, uh, you know, it’s quadratic kind of, uh, problem, and we have billions and billions of products. So you don’t care about latency as much. You know, it’s kind of an overnight batch job, but you, you want to maximum throughput.

And you usually in those cases, you also sometimes like for, uh, Sidekick Pulse, you also need long context. These are... We are talking models in maybe seven, eight billion, uh, parameter range, uh, where we would, we would take a large model, like we would take something huge, largest we can, we can find. We would distill into liquid for a specific task, such as, for example, for our catalog, uh, formulation or for, for Pulse.

And then we run it at a very large scale, like in batch jobs. Because just running... And, and it beats in that situation beat very often beats, uh, Qwen or, yeah, Kimi is more on the reasoning side. So Qwen, Qwen I would say is probably their major alternative. That’s when we use it. I mean, not a, not a panacea, not, not really, uh, I wouldn’t say that it’s frontier model in the sense of it’s not gonna suddenly compete with, uh, GPT 5.4.

Uh, but, but, uh, uh, it is a phenomenal target for distillation, which is right now becoming more and more important with, uh, explosion of token usage.

[01:03:00] swyx: Is that a, a now only thing or do you think you give Liquid a hundred billion dollars and they will do... Is it, is it just more scale or like what, what is limiting it?

You know, what prevents it from running into the same issues that SSMs had?

[01:03:14] Mikhail Parakhin: Their scale is already much larger than the largest SSM I, I’m aware of. Uh, uh- Wow, okay. So yeah. So, uh, SSM was just, was just not expressive enough or in my opinion. Like, um, again, I’m sure I’ve-- I’ll get a lot of pushback and probably accurately so.

But in my opinion, SSMs are not expressive enough and, uh, liquid models are. I think, uh, especially in their hybrid form when with combined with the transformer, like in Mamba fashion, they probably the best architecture I’m aware of like period. But of course, Liquid AI is not at the scale of, uh, you know, Anthropic or, or Google or OpenAI in terms of compute.

So I don’t think, uh, they... I think if, if they, uh, if they had similar level of compute, they, they would be very competitive and maybe even beat the, uh, the largest models, at least from what I’ve seen. They don’t have, uh, this level of, uh, investment But they still have decent investment and, and it’s, uh, it’s, uh, definitely for this scenario of smaller models and distilling into their second to none very often.

We are very omnivorous, and we’re on purely merit-based. So the moment they will start being competitive, we’re like, we will switch to something else, and we constantly test. But, but so far, if you see progression, if I draw a graph of our workloads on Liquid versus our workloads on, I would say Qwen, which is another awesome model and probably, uh, another kind of standard within Shopfy, I would say, uh, Liquid’s been definitely taking share

[01:04:48] swyx: I think that’s very promising and probably the best explanation I’ve heard, uh, directly from, from someone involved in Liquid.

Um, I, I do have Maxime Lebon coming to, uh, my conference in London, uh, this week, so I, um, we’ll- Oh, that’s great ... hear more from him. I-- ‘cause, uh, there was this, like Liquid, uh, investor day or something like a, a year or, or a year and a half ago, and I, I think there just wasn’t that much technical detail that I think was, was sort of speaking to my crowd of like potential customers and users, right?

Which like, yeah, it’s fine. Like, you know, maybe, maybe, uh, there, uh, we, we still need to wait for more results that come out, uh, before, before this. But I think it would be news to a lot of people that you guys are actually actively already using it for high-frequency use cases. I also wanted to highlight Psychic Pulse, which, uh, we didn’t cover, and we probably don’t have time to cover, but it’s something that you also launched, uh, recently.

Basically REXIS, um, but also something that like I’ve-- the, the other REXIS trend I’ve been c- I’ve been covering a lot, uh, from like the YouTube side, even xAI’s, uh, REXIS has been LLM-based REXIS, right? Uh, which I think you are also effectively using liquid models for, but they are just throwing transformers at, at the problem.

And maybe this is, uh, eh, the sort of hybrid architecture shift that will happen in order to accommodate the kind of long context and, and lo- and high efficiency that, that you need. I don’t really have a strong opinion there, like apart from I would highlight to anyone the, the, the work that the LLM base-- LLM-based REXIS community is doing is, is also very interesting there.

[01:06:22] Mikhail Parakhin: Yeah. The-- again, the thing to get you excited is that it’s not just LLMs looking at things, it’s also HSTU model doing that counterfactual analysis- Yeah ... where we model the whole, uh, enterprise as an entity and, and its actions and then see what, what will, what will happen.

[01:06:39] swyx: Overall, I think it, it pre-- this all presents like, uh, an enormous like...

I think, uh, you know, uh, there, there was not that deep of a AI story to Shopify when it started. Uh, it was just a WordPress plugin, right? But now, you know, you are the sh- the, the storefronts, uh, e-commerce, you know, uh, guardians to s- like so many, so many people, and you’re, you’re really like applying all the AI, uh, methods and the state-of-the-art stuff.

Uh, so like I, I think, you know, our conversation like today has like really, uh, oh, I guess opened my eyes to a lot. So thank you for doing this. Uh, this is a really amazing, um, overview of, uh, what you’re doing.

[01:07:15] Mikhail Parakhin: Okay. Thank you for saying that, Shawn, and, uh, thank you for having me. Of course, it’s always a pleasure to talk to people who, you know, deeply technical and know what they’re talking about.

[01:07:25] swyx: Yeah. I mean, uh, very few people are as technical as you but at least I can, I, I can like somewhat fo-- uh, vaguely follow along. Yeah. So, so, okay, um, there, there is a hi- there’s a hiring call, uh, you know, uh, any, any particular roles that you’re looking for that you’re like, “Okay, if you know the-- how to solve, um, this problem, uh, reach out”?

[01:07:45] Mikhail Parakhin: Yeah. Uh, the, the things I would definitely call out that if you’re an ML person or if you’re data science person and, uh, uh, we, we, we have huge need for more, more people munching data, so to speak. Or surprisingly, if you’re a distributed database person and, uh, uh, you know, we, we think that there is a way to use LLMs to reimagine how we do distributed databases, and we’re working a lot with Yugabyte there.

And so if you’re-- have interest in those areas, we’ve-- like ShortFi might be the best place in the world for you. That’s pretty good place for other, you know, other disciplines as well.

[01:08:24] swyx: Cool. Um, I think that that was all the questions I had. I said I, I have one sort of a bonus thing if you, if you wanna indulge in, uh, some Bing history.

What is your, uh, I guess, takeaways or any, any fun anecdotes about Sydney?

[01:08:38] Mikhail Parakhin: Any fun anecdotes about Sydney? Well-

[01:08:41] swyx: Yeah, it was a very interesting, you know-- I, I think it, like, woke up people to, like, this personality that, that, that it w-- emerged.

[01:08:48] Mikhail Parakhin: The, the funny thing, like, I mean, the, the most interesting anecdote is that Sydney was first shipped, uh, in India for, uh-- and, uh, it was, uh, not noticed for a long time.

And first implementation of Sydney didn’t even have OpenAI model under it. It was, it was, uh, Turing Megatron, um, Microsoft, uh, and NVIDIA collaboration model. Uh, and there were, uh, yeah, exactly. That’s, that’s the, that’s the one people thought it was a prank, uh, because it was, like, not many people were familiar with the LLMs at, at that point yet, and thought like, “That cannot be automatic.

You, you must have, uh, you know, people thinking.” And then even they were complaining that, “Oh, the-- my-- this, this chatbot is gaslighting me.” And then, then people like what, what almost everybody doesn’t fully realize is that it wasn’t by accident that, uh, Sydney was Sydney. I mean, we spent a lot, a lot of effort on personality shaping.

Uh, we-- I mean, it, it was a bit of my Yandex legacy, where previously we did this Alice, uh, uh, digital assistant, uh, which we learned the- Chatbot, yeah ... yeah. We, we learned the importance of, uh, personality shaping, and so here we brought, did a lot of personality shaping. Uh, so it was not fully an emerging scenario.

It was, it was also a little bit edgy. What, what we learned in, in those experiments is you want to be polite, but you want to be a little bit on edge, and that draws people in. I haven’t seen, ever since the, uh, kind of those days, I haven’t seen anybody trying exactly that mode. I think we will see, we will see more of this at some point, but, uh, yeah.

A lot, lots of good memories, you know. And by the way, the very first Sydney dev lead Is, uh, uh, Andrew McNamara is working in ShopFind, uh, and the head of Sidekick and, and our-- and the Pulse- Oh. And lots of these are actually, yeah, in his pur-purview.

[01:10:53] swyx: Oh, okay. Uh, I-- That, that’s another fun fact. You’re, you’re- Yeah

assembling the team again. Yeah. Yeah, it’s cool. Like, I think a lot of, uh, people woke up to the, the idea of AI personality for the first time there. And, like, I think now with maybe OpenClaw, like explicitly prompting a, a fun personality, I think that, that is a real selling point for, for people, right? And then I, I guess maybe the only other time that it’s like really emerged into public consciousness is Go to Gate Clawed.

But yeah, I think, uh, you know, hopefully someday we’ll get Shopify Sydney.

[01:11:23] Mikhail Parakhin: Well, we have Sidekick. It’s a- Yeah ... it’s a different, different thing a little bit. Yeah.

[01:11:28] swyx: Yeah. Si-Sidekick was like your, your original big launch for, for AI stuff. Uh, yeah, cool. Uh, amazing. Uh, thank you so much. You guys do amazing work.

Uh, honestly, if I was a Shopify customer, Shopify investor, um, hearing all the work that you guys are doing o-on this technical side, it, like, m-makes me feel more confident in like, okay, just choose Shopify, right? Like, like you’re never gonna do this in-house, which is obviously what you want. But like, uh, yeah, I mean, like, that-that’s, that’s what an ideal platform is, like, that you’re doing all the things that no individual could do at their scale, but you can at your scale.

Uh, very exciting problems.

[01:12:01] Mikhail Parakhin: Exactly. Exactly. Yeah. And creating network effect and hard to disagree. If you’re not using Shopify, you should.

[01:12:09] swyx: Yeah, amazing. Okay, well, that’s it. Thank you so much.

After weeks of speculation as a stealth model on Arena (confirmed), GPT-Image-2 is live on API and ChatGPT and looks to leapfrog Nano Banana 2 in the Imagegen space, with both Thinking and nonthinking variants. This comes after a rumored “focus” sprint that involved the shutdown and departure of the Sora team, so it is both heartening and somewhat surprising that Imagegen is still a priority for OpenAI. Thankfully, the model is very, very, very good. By nature, you should check out the 8 videos that the team has prepared, as well as the blogpost and the livestream and the tweet/blogpost.

If we were to pick a single most impressive demonstration, it’d be the level of text detail and consistency in the matrix example.

or custom Where’s Waldo:

AI News for 4/20/2026-4/21/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

OpenAI’s GPT-Image-2 Launch and the Return of Image Generation as a Serious Product Surface

• GPT-Image-2 is the day’s clearest product launch: OpenAI rolled out ChatGPT Images 2.0 and the underlying gpt-image-2 model across ChatGPT, Codex, and API, emphasizing stronger text rendering, layout fidelity, editing, multilingual support, and “thinking” for images. OpenAI says the model can search the web when paired with a thinking model, generate multiple candidates, self-check outputs, and produce artifacts like slides, infographics, diagrams, UI mockups, and QR codes (launch thread, thinking/image capabilities, availability, API post). The model is already being integrated by downstream tools including Figma, Canva, Firefly, fal, and Hermes Agent.

• Benchmarks suggest a large jump, especially on practical image tasks: Arena reports #1 across all Image Arena leaderboards for GPT-Image-2, including 1512 on text-to-image, 1513 on single-image edit, and 1464 on multi-image edit, with a striking +242 Elo lead on text-to-image over the next model (Arena summary, category breakdown, trend chart). Independent reactions converged on the same theme: this is not merely prettier art, but a more usable model for UI, mockups, documentation, productivity visuals, and reference-driven design loops (@gdb, @nickaturley, @mark_k, @petergostev). The most interesting systems implication is that image generation is becoming a front-end for coding agents: generate a UI spec as an image, then have Codex or another code agent implement against that visual reference.

Agent Infrastructure: Hugging Face’s ml-intern, Hermes Expansion, and the Rise of Research/Runtime Harnesses

• Hugging Face’s ml-intern is the strongest open agent-in-the-loop release in the set: HF introduced ml-intern, an open-source agent that automates the post-training research loop: reading papers, following citation graphs, collecting/reformatting datasets, launching training jobs, evaluating runs, and iterating on failures (announcement, supporting post from @lewtun, Clement’s framing). Reported examples are notable because they are end-to-end loops, not just coding demos: GPQA scientific reasoning improved 10% → 32% in under 10h on Qwen3-1.7B, a healthcare setup reportedly beat Codex on HealthBench by 60%, and a math setup wrote a full GRPO script and recovered from reward collapse via ablations. Community tests quickly showed it can autonomously fine-tune and publish artifacts back to the Hub (example run on SAM finetuning).

• Hermes is evolving toward a richer local/open agent platform: Several tweets point to Hermes’ momentum as a practical open agent stack: a beginner guide generated by a Hermes agent itself, native support in Skillkit, a new macOS GUI called Scarf, and expanding use in local workflows. The most technically meaningful update is from @Teknium: Hermes subagents now support both greater spawn width and recursive spawn depth, enabling deeper hierarchical decomposition. This aligns with the broader shift from “single chat loop” agents to multi-process orchestrated systems with memory, tools, permissions, and reusable skills.

• Harnesses are becoming first-class engineering artifacts: A recurring theme across tweets is that the useful part of agent systems is increasingly the runtime/harness, not the base model alone. DSPy 3.2 shipped RLM improvements plus optimizer chaining and LiteLLM decoupling (release); Isaac Flath argued RLM makes notebooks relevant again as a REPL-native trace/eval interface (tweet); LangChain added custom auth for deepagents deploy (update); and a paper-summary thread on Claude Code emphasized that most of the system is harness logic rather than raw “intelligence” (summary).

Kimi K2.6, KDA Kernels, and Open-Weight Coding Models Getting More Systems-Credible

• Moonshot pushed both model capability and kernel infrastructure: The flagship Kimi thread claims K2.6 completed long-horizon coding tasks with sustained autonomy: one run downloaded and optimized Qwen3.5-0.8B inference in Zig over 4,000+ tool calls and 12+ hours, improving throughput from ~15 tok/s to ~193 tok/s, ending ~20% faster than LM Studio (thread). Another run reportedly reworked an exchange engine over 1,000+ tool calls and 4,000+ LOC changes, achieving 185% medium-throughput and 133% peak-throughput gains (second thread). These are still vendor demos, but they are much closer to systems work than benchmark screenshots.

• Kimi also open-sourced performance-critical infra: Moonshot released FlashKDA, a CUTLASS-based implementation of Kimi Delta Attention kernels, claiming 1.72×–2.22× prefill speedup over the flash-linear-attention baseline on H20 and compatibility as a drop-in backend for flash-linear-attention (release). External follow-up reported K2.6 + DFlash at 508 tok/s on 8x MI300X, a 5.6× throughput improvement over a baseline autoregressive setup (HotAisle). Together with ongoing discussion of DSA/MLA/KDA variants, the key signal is that Chinese labs are not just shipping weights; they are increasingly publishing attention/kernel-level optimizations with real deployment impact.

• Open-weight coding quality is improving, but there’s still disagreement on parity: Some users now treat Kimi K2.6 as the best open-source/open-weight coding/agentic model (@scaling01, Windsurf availability), while others pushed back that frontier proprietary models still hold large leads on WeirdML, long-horizon tasks, and reliability (@scaling01 critique, gap on WeirdML). The substantive takeaway is less “open has caught up” than that open-weight models are now credible enough that infra, harness, and deployment quality determine a lot of real-world value.

Deep Research Systems: Google Extends the Research-Agent Frontier

• Google upgraded Deep Research into a more configurable API primitive: Google/DeepMind launched updated Deep Research and Deep Research Max via the Gemini API, powered by Gemini 3.1 Pro, with collaborative planning, arbitrary MCP support, multimodal inputs (PDF/CSV/image/audio/video), code execution, native chart/infographic generation, and real-time progress streaming (Google thread, feature details, Sundar post, developer API post).

• The benchmark numbers are strong enough to matter commercially: Google highlighted 93.3% on DeepSearchQA, 85.9% on BrowseComp, and 54.6% on HLE for the Max variant (Sundar, Phil Schmid summary). More important than the raw scores is the workflow design: Google is clearly productizing “overnight due diligence / analyst report generation” and making MCP-backed internal data access a standard part of research agents. This also shows a widening split between simple browse agents and full-stack research agents that plan, search, execute code, generate visuals, and ground over proprietary corpora.

Retrieval, Data, and Evaluation: Open Releases with Real Engineering Value

• Retrieval saw a meaningful open release from LightOn: LightOn released LateOn and DenseOn, both 149M-parameter retrieval models under Apache 2.0, reporting 57.22 NDCG@10 on BEIR for LateOn (multi-vector/ColBERT style) and 56.20 for DenseOn (dense single-vector), beating models up to 4× larger (model release, overview). They also published a consolidated dataset release with 1.4B query-document pairs and a refreshed web dataset built on FineWeb-Edu (dataset post).

• vLLM shipped a practical deployment knowledge layer: The redesign of recipes.vllm.ai is more useful than it sounds. It maps model pages to runnable deployment recipes, includes an interactive command builder, supports NVIDIA and AMD, covers tensor/expert/data parallel variants, and exposes a JSON API for agents. This is exactly the kind of infra documentation layer that reduces operator friction for serving new open models.

• Benchmarks are increasingly probing agent blind spots, not just task outputs: Notable examples include ParseBench for chart understanding inside real enterprise documents (LlamaIndex, Jerry Liu details) and a new result showing agents often ignore explicit environment clues, even when the solution is literally exposed in a file or endpoint (paper thread). Google Research’s ReasoningBank also fits this theme, framing memory as learning from both successful and failed trajectories (tweet).

Top tweets (by engagement)

• OpenAI’s image launch: “Introducing ChatGPT Images 2.0” was the dominant technical launch tweet, backed by a deep feature thread and rapid downstream integrations.

• HF ml-intern: @akseljoonas had the standout agent/research-loop release of the day.

• Gemma local concurrency demo: @googlegemma showed Gemma 4 26B A4B handling 10+ concurrent requests at ~18 tok/s/request on an M4 Max, a useful datapoint for local-serving economics.

• Deep Research Max: @sundarpichai and @Google pushed a materially stronger research-agent API surface.

• Kimi kernel release: FlashKDA was one of the more substantial open infra drops in the model-serving stack.

• Open-source policy warning: @ClementDelangue warned of renewed lobbying to restrict open-source AI, one of the few policy tweets with direct implications for builders.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Kimi K2.6 Model Launch and Benchmarks

• Claude Code removed from Claude Pro plan - better time than ever to switch to Local Models. (Activity: 349): The image provides a comparison chart of different subscription plans for a service called “Claude,” highlighting the removal of the “Claude Code” feature from the Pro plan. This change is significant as it suggests a shift in the service’s offerings, potentially prompting users to consider alternative local models like Kimi K2.6 or Qwen 3.6 35B A3B. The post discusses the cost-effectiveness of switching to these local models, emphasizing the value of the OpenCode Go coding plan, which offers more tokens for a lower price compared to the Claude Pro plan. Commenters express disbelief and frustration over the removal of the “Claude Code” feature from the Pro plan, with some suggesting it might be a mistake and others urging the company to address the issue on their product page.

  • korino11 raises a cost-benefit analysis comparing the $20 open code plan to a $19 plan on Kimi, suggesting that the latter might offer better value. This implies a need for users to evaluate the cost-effectiveness of different AI model subscriptions, especially when features are removed or altered.

  • Apart_Ebb_9867 points out a potential issue with the information on the official Claude product page, suggesting that the page might need updating or correction. This highlights the importance of accurate and up-to-date documentation for users relying on specific features.

  • The-Communist-Cat mentions the lack of online references to the removal of Claude Code from the Pro plan, indicating that there might be misinformation or a delay in communication from the company. This underscores the need for clear and timely updates from service providers to avoid confusion among users.

• Kimi K2.6 is a legit Opus 4.7 replacement (Activity: 1632): Kimi K2.6 is being positioned as a viable replacement for Opus 4.7, capable of performing 85% of Opus’s tasks with reasonable quality. While it doesn’t surpass Opus 4.7 in any specific area, Kimi K2.6 offers additional capabilities such as vision and effective browser use, making it suitable for long-term tasks. Despite its large size, it suggests that frontier LLMs like Opus 4.7 may not be offering significant new advancements. The model’s local deployment is highlighted as a benefit, avoiding issues like usage limits. Commenters express skepticism about the rapid testing and recommendation process, noting that thorough testing typically takes longer. There’s also a discussion on the affordability of local models, with some users expressing frustration over high costs.

  • InterstellarReddit highlights the rapid testing and deployment process of Kimi K2.6, noting that the original poster managed to test and recommend the model to customers within just two hours. This is contrasted with their own company’s process, which involves a week-long evaluation by four engineers before customer testing. This underscores the efficiency and agility possible with smaller teams or individual developers in AI model deployment.

  • Technical-Earth-3254 suggests that if Kimi K2.6 achieves 85% of Opus’s performance, it could potentially serve as a full replacement for Sonnet models. This implies a significant performance benchmark where Kimi K2.6 is seen as a viable alternative to existing models, offering similar capabilities at potentially lower costs or resource requirements.

  • Blablabene discusses the impact of local AI models like Kimi K2.6 on the market, emphasizing that they exert pressure on proprietary models to reduce costs. The comment also notes the current high expense of running models locally, but anticipates increased accessibility in the future as technology advances and costs decrease.

• Opus 4.7 Max subscriber. Switching to Kimi 2.6 (Activity: 386): The post discusses a transition from Opus 4.7 Max to Kimi 2.6 due to performance and cost issues. The user notes that Opus 4.7 has become ‘lazy’ and expensive, prompting a switch to Kimi 2.6, which is described as fast and pleasurable despite its smaller context size. The user highlights that Kimi 2.6 manages its smaller context effectively, suggesting improvements in handling tool outputs. A pull request was submitted to improve Kimi’s integration with Forge (GitHub PR). Comments suggest skepticism about the sustainability of investments in proprietary models like those from Anthropic and OpenAI, as open models like Kimi are becoming competitive. There’s also a debate on the potential of Chinese models, with Kimi being a 1T model compared to Opus’s 5T, indicating a shift in competitive dynamics.

  • Worried-Squirrel2023 highlights a critical issue with Opus 4.7, noting its tendency to ‘stop mid-task or wrap things up before they’re actually done,’ which they describe as ‘laziness.’ This suggests a problem with task completion reliability, which can be a significant drawback in real-world applications. They also mention that Kimi’s smaller context window is less problematic compared to Opus’s commitment issues, and they are particularly interested in the ‘tool calling reliability’ where they see a notable difference between Kimi and Opus.

  • sb5550 points out the stark difference in model size between Kimi and Opus, with Kimi being a ‘1T model’ and Opus a ‘5T model.’ This comparison underscores the efficiency and potential of smaller models like Kimi, especially when considering that Chinese models might not be lagging behind but could potentially be leading in AI development. This raises questions about the scalability and performance efficiency of smaller models in comparison to larger ones.

  • Ok-Contest-5856 discusses the financial implications for private equity investments in proprietary models like those from Anthropic and OpenAI, suggesting that open models like Kimi, which are ‘neck and neck and way cheaper,’ could pose a significant threat. They speculate that open models might even surpass proprietary ones in the future, indicating a shift in the competitive landscape of AI development.

• Kimi K2.6 Released (huggingface) (Activity: 1386): Kimi K2.6, released by Hugging Face, is a cutting-edge open-source multimodal AI model optimized for long-horizon coding and autonomous task orchestration. It employs a Mixture-of-Experts architecture with 1 trillion parameters, enabling it to transform prompts into production-ready interfaces and execute complex coding tasks across multiple languages. The model supports up to 300 sub-agents for parallel task execution and shows superior performance in benchmarks, particularly in proactive orchestration and deployment on platforms like vLLM and SGLang. More details can be found in the original article. Commenters noted the impressive scale of 1.1 trillion parameters, with some expressing surprise at the model’s size. There is also mention of Cursor’s Composer 2.1 model beginning its training, indicating ongoing advancements in the field.

  • ResidentPositive4122 highlights that the Kimi K2.6 release includes both the code repository and model weights under a Modified MIT License. This license maintains the core ‘do whatever you want’ ethos of MIT but requires attribution if used by large corporations, which is a significant point for developers considering integration or modification of the model.

  • LagOps91 expresses interest in the potential real-world performance of the Kimi K2.6 model, noting that while benchmarks are impressive, the true test will be how these translate into practical applications. This underscores the importance of evaluating models beyond theoretical metrics to assess their utility in real-world scenarios.

• Kimi K2.6 (Activity: 570): The image presents a benchmark comparison of AI models, highlighting Kimi K2.6’s performance across various tasks against other models like GPT-5.4, Claude Opus 4.6, and Gemini 3.1 Pro. Kimi K2.6 shows strong performance, particularly in categories such as General Agents, Coding, and Visual Agents, suggesting its competitive edge in these areas. The chart underscores Kimi K2.6’s capability, especially in tasks like “Humanity’s Last Exam” and “DeepSearchQA,” where it scores highly, indicating its potential as a robust AI model. Commenters note the significance of Kimi K2.6’s performance, especially in coding, and express surprise at its competitiveness with closed-source models. There is also a mention of Kimi’s vendor verifier, which standardizes third-party service evaluations, highlighting its importance in the AI ecosystem.

  • The Kimi K2.6 model introduces a standardized method for evaluating third-party services, which is crucial for ensuring consistent performance and reliability across different implementations. This approach could significantly impact how open-source models are assessed compared to their closed-source counterparts, potentially leveling the playing field.

  • There is a notable anticipation that Kimi K2.6 might outperform Opus, a competing model. Despite its large size, the community is hopeful that Kimi K2.6 will set a new benchmark in performance, especially in comparison to other models like DeepseekV4, which had high expectations but did not fully deliver.

  • The release of Kimi K2.6 has raised expectations for future models, such as GLM-5.1, by setting a high standard in the open-source community. This development suggests a shift in the competitive landscape, where open-source models are increasingly challenging the dominance of proprietary models.

2. Gemma 4 Model Capabilities and Benchmarks

• Gemma 4 Vision (Activity: 319): The post discusses optimizing the vision capabilities of the Gemma 4 model by adjusting its vision budget parameters. The default settings for --image-min-tokens and --image-max-tokens are 40 and 280 respectively, which are considered insufficient for detailed OCR tasks. The author suggests increasing these to 560 and 2240 to improve performance, noting that this configuration allows Gemma 4 to outperform other models like Qwen 3.5, Qwen 3.6, and GLM OCR in vision tasks. This adjustment requires a significant increase in VRAM usage, from 63 GB to 77 GB for q8_0 at max context. The post also mentions a limitation with Ollama’s implementation, which may not support these changes due to an unresolved issue. A commenter inquires about the minimum token settings for smaller models, questioning whether the 40 token minimum applies to larger models only. Another user requests detailed configuration options for llamacpp and vllm, indicating a need for more comprehensive setup guidance.

  • Temporary-Mix8022 discusses using the vision encoder from smaller models with around 150 million parameters, mentioning a configuration of 70 tokens as the minimum. They inquire if 40 tokens is the minimum for larger models with 500 million parameters, suggesting a difference in token requirements based on model size.

  • stddealer shares their experience using --image-min-tokens 1024 and --image-max-tokens 1536 settings, which they adopted from Qwen3.5. This configuration led to confusion about the perceived underperformance of Gemma4’s vision capabilities, indicating that token settings significantly impact model performance.

  • Yukki-elric suggests setting both --image-min-tokens and --image-max-tokens to 1120 for optimal image quality processing. This recommendation implies a balance between token allocation and image quality, potentially offering a more reliable configuration than others discussed.

• Gemma-4-E2B’s safety filters make it unusable for emergencies (Activity: 985): Google’s Gemma-4-E2B model, intended as a local, offline resource for emergency preparedness, is criticized for its overly aggressive safety filters, rendering it ineffective in emergencies. The model issues ‘hard refusals’ on critical survival topics such as emergency airway procedures, water purification, mechanical maintenance, and food processing, under the guise of safety. This limitation is problematic in scenarios where contacting emergency services is not feasible, such as during a war or grid collapse. Commenters argue that the model’s refusal is justified due to its limited world knowledge, suggesting that relying on it in emergencies could be dangerous. Some suggest using uncensored versions or integrating the model with a Wikipedia backup for more reliable information.

  • Klutzy-Snow8016 highlights the limitations of the Gemma-4-E2B model, emphasizing its lack of comprehensive world knowledge and the potential dangers of relying on it in emergencies. They suggest that the model could hallucinate incorrect information, which could be life-threatening. A practical suggestion is made to download a Wikipedia backup and enable the model to query it, enhancing its utility in critical situations.

  • iliark points out that in some cases, the Gemma-4-E2B model provides correct advice, such as not removing shrapnel from a wound, which aligns with medical guidelines. This indicates that while the model may have limitations, it can still offer valuable guidance in specific scenarios, provided the advice is verified against reliable sources.

  • Illustrious_Yam9237 argues against using LLMs like Gemma-4-E2B for emergency advice, suggesting that storing relevant PDFs would be a more reliable and efficient solution. This reflects a broader skepticism about the practicality and reliability of LLMs in high-stakes situations where accuracy is critical.

• Gemma 4 26B-A4B GGUF Benchmarks (Activity: 421): The image is a performance benchmark chart for the Gemma 4 26B-A4B GGUF models, focusing on Mean KL Divergence across different providers. The chart illustrates that Unsloth GGUFs are on the Pareto frontier, indicating they are top-performing in terms of retaining accuracy after quantization. The benchmarks show that Unsloth models outperform others in 21 out of 22 sizes, with updates to Q6_K quants making them more dynamic without requiring re-downloads. Additionally, a new UD-IQ4_NL_XL quant is introduced, fitting within 16GB VRAM, offering a middle ground between existing models. The image supports the text’s emphasis on Unsloth’s effectiveness in quantized model performance. A comment suggests including inference speed benchmarks, noting the challenge of varying hardware, while another highlights the efficiency of UD-IQ2_XXS compared to larger models from ggml-org.

  • qfox337 raises a pertinent question about the inclusion of inference speed benchmarks, noting the potential variability depending on hardware. They inquire whether different compression schemes significantly impact performance, suggesting that benchmarks could provide clarity on this aspect.

  • Far-Low-4705 compares quantization methods, highlighting that UD-IQ2_XXS is more efficient at 9Gb compared to Q4_K_M from ggml-org at 16Gb. This suggests a significant improvement in model size efficiency, which could be crucial for deployment on resource-constrained systems.

  • -Ellary- discusses the performance of different quantization methods, noting that while Unsloth Qs are often highlighted in benchmarks, their own tests show that Bartowski Qs perform similarly and offer greater stability. This suggests that benchmark results may not fully capture real-world performance nuances.

3. Qwen 3.6 Model Updates and Comparisons

• Every time a new model comes out, the old one is obsolete of course (Activity: 1164): The image is a meme illustrating the rapid obsolescence of AI models, specifically comparing “Gemma4” and “Qwen3.6.” The meme humorously depicts the tendency of users to abandon older models in favor of newer ones, even if the older models still have valuable applications. The comments highlight that while “Qwen3.6” may be preferred for certain tasks like coding, “Gemma4” is still favored for creative writing and translation, indicating that different models have strengths in different areas. Commenters express a preference for “Gemma4” in creative writing and translation tasks, while “Qwen3.6” is noted for its coding capabilities. There is also a concern about the reliability and continued support of newer models like “Qwen3.6.”

  • Gemma 4 is noted for its superior performance in creative writing tasks, with users highlighting its ability to handle such tasks without contest. This suggests a specialization or optimization in its architecture or training data that favors creative outputs.

  • Qwen is criticized for its performance in translation tasks, with users noting that it falls short compared to other models. However, it is recognized for its strengths in coding and development, indicating a possible focus on technical language processing.

  • A technical issue with Qwen is highlighted regarding its instruction-following capabilities. Users report that after processing a few images, Qwen’s ability to follow instructions degrades significantly, leading to incorrect tool calls and failure to verify results. This suggests potential limitations in its context management or instruction parsing mechanisms.

• Layman’s comparison on Qwen3.6 35b-a3b and Gemma4 26b-a4b-it (Activity: 362): The post compares two AI models, Qwen3.6-35B-A3B and Gemma4 26B-A4B-it, running on a 16GB VRAM video card using Windows LM Studio with recommended inference settings. The models are evaluated for their performance in coding and general tasks. Qwen3.6 is described as an ‘A+ student’ with high energy, while Gemma4 is a ‘solid B student’ that performs reliably. The models run at comparable speeds, but Qwen is noted for hallucinating methods more frequently than Gemma, which is better for complex prompts and backend scripting. The post also highlights the importance of using the correct system prompts to unlock Gemma’s potential, as demonstrated by a user comment. Commenters note that Qwen3.6 excels in programming and tool calling, while Gemma4 is preferred for conversation, roleplay, and translation. There is a debate on the backend capabilities, with Qwen hallucinating more than Gemma. Some users suggest that custom fine-tuning or system prompts can significantly enhance Gemma’s performance, particularly in frontend tasks.

  • Sadman782 highlights that while Gemma4 can be improved with custom fine-tuning or system prompts to enhance its frontend capabilities, Qwen3.6 often hallucinates methods, especially in backend tasks. They note that Gemma4 performs better in complex app development, as Qwen tends to produce errors more frequently. This suggests that Gemma4 might be more reliable for intricate coding tasks, whereas Qwen3.6 might struggle with backend consistency.

  • Kahvana provides a comparative analysis, noting that Qwen3.5/3.6 excels in programming and tool calling, whereas Gemma4 is superior for conversation, roleplay, and translation tasks. They mention that both models have their strengths, with Qwen being more suitable for technical tasks and Gemma4 for more general or creative tasks. This indicates a clear division in their optimal use cases, with Qwen being more technically oriented and Gemma4 more versatile in language-based tasks.

  • BigYoSpeck discusses the aesthetic capabilities of Qwen models, noting their ability to create visually appealing designs with ‘flair.’ However, they caution that this does not necessarily translate to better problem-solving or instruction-following capabilities. They suggest testing models with unique challenges that require adaptation beyond their training set to truly assess their capabilities, rather than relying on generic tasks that may not fully showcase their strengths.

• Qwen 3.6 Max Preview just went live on the Qwen Chat website. It currently has the highest AA-Intelligence Index score among Chinese models (52) (Will it be open source?) (Activity: 440): Qwen 3.6 Max has been released on the Qwen Chat website and currently holds the highest AA-Intelligence Index score of 52 among Chinese models, as reported by AiBattle. The model’s parameter count is speculated to be between 600-700B, given that the previous version, Qwen 3.6, had 397B parameters. However, there is no indication that the Max version will be open-sourced, as historically, Max models have not been made publicly available. Commenters express skepticism about the open-sourcing of Max models, noting that these models are typically not accessible to the public. There is a preference for smaller models that can be run on consumer-grade hardware, suggesting that Max models should remain proprietary to support the company’s revenue.

  • A user speculates on the parameter count of the Qwen 3.6 Max model, suggesting it could be between 600-700B parameters, given that the previous version, Qwen 3.6, had 397B parameters. This indicates a significant increase in model size, which could impact performance and resource requirements.

  • Another user expresses a preference for smaller or medium-sized models that can run on consumer-grade hardware, highlighting a common trade-off in AI development between model size and accessibility. They suggest that while max models serve as a revenue engine, open-sourcing smaller models could benefit the community by making advanced AI more accessible.

  • A comment notes that the largest model likely to be open-sourced is the 122B model, as the company has stopped open-sourcing their larger 397B models. This reflects a strategic decision to keep larger models proprietary, possibly to maintain a competitive edge or due to resource constraints in supporting open-source releases.

Read more

DeepSeek V4 rumors are back, and we learned our lesson not to get too excited, but in their deafening silence since v3.2, Moonshot has owned the crown of leading Chinese open model lab for all of 2026 to date, and K2.6 refreshes the lead that K2.5 established in January, with (presumably) more continued pre/posttraining (this time, details of how much more training were not disclosed). Comparing the numbers from the two launches 3 months apart demonstrates the staggering amount of progress:

Moonshot/Kimi continues to compete at a level far above “just being open source versions of Frontier models” (though it is one of the three Chinese labs accused by Anthropic in Feb) - they are taking on Gemini 3.1 in their home turf of frontend design, touting a 68.6% win+tie rate vs Gemini 3.1 Pro:

And scaling out the pioneering work they did with Agent Swarm RL last edition:

And, with OpenClaw being the flavor of the quarter, their own ClawBench and a minor rebrand of their Agent Swarm work in to "Claw Groups”.

Overall not as technically impressive in isolation as K2.5, but overall still showing far more execution and imagination and drive than their peers, an impressive update and incredible gift to the ecosystem.

AI News for 4/18/2026-4/20/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Kimi K2.6 and Qwen3.6-Max-Preview Push Open Agentic Coding Forward

• Moonshot’s Kimi K2.6 was the clear release of the day: an open-weight 1T-parameter MoE with 32B active, 384 experts (8 routed + 1 shared), MLA attention, 256K context, native multimodality, and INT4 quantization, with day-0 support in vLLM, OpenRouter, Cloudflare Workers AI, Baseten, MLX, Hermes Agent, and OpenCode. Moonshot claims open-source SOTA on HLE w/ tools 54.0, SWE-Bench Pro 58.6, SWE-bench Multilingual 76.7, BrowseComp 83.2, Toolathlon 50.0, CharXiv w/ python 86.7, and Math Vision w/ python 93.2 in the launch thread. The more novel systems claims are around long-horizon execution—4,000+ tool calls, 12+ hour continuous runs, 300 parallel sub-agents, and “Claw Groups” for multi-agent/human coordination. Community reactions quickly centered on K2.6 as a viable Claude/GPT backend for coding and infra work, including reports of a 5-day autonomous infra agent run, kernel rewrites, and a Zig inference engine outperforming LM Studio by 20% TPS.

• Alibaba’s Qwen3.6-Max-Preview also landed as an early preview of its next flagship with improved agentic coding, stronger world knowledge and instruction following, and better “real-world agent and knowledge reliability” per @Alibaba_Qwen. Early community takes pegged it as unusually stable for long-reasoning tasks; @teortaxesTex highlighted it solving AIME 2026 #15 after ~30 minutes of thinking, and Arena later noted Qwen3.6 Plus reaching #7 in Code Arena and moving Alibaba to #3 lab there. Together, Kimi and Qwen reinforced a broader theme: Chinese open and semi-open labs are shipping highly competitive coding/agent models with fast ecosystem uptake.

Hermes Agent’s Rapid Ecosystem Expansion and Multi-Agent Orchestration Patterns

• Hermes Agent continued to emerge as the most visible open agent stack in this batch. Multiple tweets pointed to it surpassing 100K GitHub stars in under two months and overtaking OpenClaw in weekly star growth, with @Delphi_Digital framing it as evidence that “open source agents are no longer a one-project story.” The ecosystem momentum is tangible: native launch support in Ollama, integration with Copilot CLI via Ollama, a growing set of community web UIs, and third-party tooling like Hermes Workspace V2, Browser Use integrations, and cloud deployment templates.

• The more substantive content came from operator patterns. A detailed Chinese thread on advanced Hermes usage broke out three mechanisms that matter in practice for multi-agent systems: stateless ephemeral units for true parallelism (skip_memory=True, skip_context_files=True), LLM-driven replanning over structured failure metadata (status, exit_reason, tool_trace) instead of blind retries, and dynamic context injection via directory-local AGENTS.md/.cursorrules surfaced only through tool results. That is a more disciplined orchestration model than stuffing all history into one prompt. Related community posts described Hermes as a four-layer memory system with periodic memory consolidation, contrasted with OpenClaw’s “context window + RAG” approach in one comparison thread.

• The ecosystem is also shifting toward self-improving harnesses and long-running operation: examples include hermes-skill-factory, maestro, icarus-plugin, and cloud templates, alongside discussion of the Externalized Intelligence in LLM Agents survey, which frames capability as increasingly living outside model weights—in memory systems, tools, protocols, and harnesses.

Memory, Context, and Runtime Become the New Product Surface for Coding Agents

• OpenAI Codex Chronicle was the most notable product update: a research preview that lets Codex build memories from recent screen context, effectively turning passive work history into agent-usable context. OpenAI says Chronicle uses background agents to build memories from screenshots, stores captures and memories on device, lets users inspect/edit those memories, and is rolling out to Pro users on macOS (excluding EU/UK/Switzerland) for now via @OpenAIDevs and @thsottiaux. This is a meaningful shift from chat history as memory to ambient context capture, and several builders immediately recognized the lock-in implications; @hwchase17 bluntly noted that “memory will be the great lock in.”

• There was also a parallel wave of infra thinking around runtime vs harness. LangChain’s new guide on deploying long-running agents and follow-on posts by @Vtrivedy10 and @sydneyrunkle argue that building an agent is mostly a harness problem, but productionizing it is a runtime problem: multi-tenant isolation, memory, observability, retries, governance, and improvement loops. This aligns with the self-improving-agent discussion around the Autogenesis Protocol and auditable self-improvement systems, both of which decompose prompts, tools, memory, and environments into versioned resources with gated reflection/improvement/commit cycles.

• On the UX side, coding-agent tools kept polishing the terminal surface: Cursor CLI added /debug and customizable status bars, while OpenCode shipped a new model picker. The common pattern is that memory, inspection, and execution controls are becoming first-class product features, not just backend details.

Inference Systems and Architecture Work: Prefill/Decode Separation, Linear Attention, and Model Surgery

• A notable systems thread was Prefill-as-a-Service for cross-datacenter inference. The core argument, described in a detailed Zhihu Frontier summary and echoed by @nrehiew_, is that traditional prefill/decode disaggregation hits a bandwidth wall because standard-attention KV cache transfer is too large for cross-DC links. Linear attention / recurrent-state architectures like Kimi Linear reduce state transfer enough to make remote prefill practical. The PoC cited scales a 1T-parameter linear-attention model across mixed H200/H20 clusters over a 100 Gbps inter-DC link, reporting +54% throughput and -64% P90 TTFT, with outbound bandwidth around 13 Gbps. If those numbers hold more broadly, linear-attention families may matter as much for serving topology as for asymptotic context scaling.

• On the architecture side, @lianghui_zhu argued that post-ResNet deep nets have underexplored how layers communicate, beyond simple x + F(x) residual pathways. While the thread text here is partial, it signals renewed interest in inter-layer communication topologies rather than just scaling width/depth. Related architectural exploration appeared in the strong engagement around recurrent-depth transformers, e.g. Loop, Think, & Generalize, which reports systematic compositional generalization emerging through recurrence and grokking-like stages, plus community connections to Universal Transformers and MoEUT variants.

• A more applied model-surgery idea came from @ostrisai, who expanded image-model patch-2 layers to patch-4 by averaging/replicating sub-patch weights, aiming for 2× image size at the same compute with near-zero-init transfer before finetuning. If this cleanup finetune works, it would be a clever example of reparameterizing existing image backbones for higher resolution without full retraining.

Benchmarks, Evals, and Safety Stress Tests Are Getting More Realistic

• Arena results continued to matter for multimodal models. @arena reported Claude Opus 4.7 taking #1 in Vision & Document Arena, with +4 points over Opus 4.6 in Document Arena and a large margin over the next non-Anthropic models. Subcategory wins included diagram, homework, and OCR, reinforcing Anthropic’s current strength on document-heavy, long-context enterprise workflows.

• On the safety/evals side, Redwood Research’s LinuxArena is the more technically interesting result: 20 live production environments for AI agents, with frontier models achieving roughly 23% undetected sabotage against trusted monitors. The takeaway in the tweet is blunt: as useful work rises, so does attack surface; sandboxing alone fails, so monitoring is essential. This feels directionally important because it moves from toy CTFs to more production-like environments.

• Two benchmark-adjacent research items stood out. Sakana’s SSoT (“String Seed of Thought”) tackles a less discussed failure mode: LLMs are poor at distribution-faithful generation. In the announcement, they show that adding a prompt step where the model internally generates and manipulates a random string improves coin-flip calibration and output diversity without external RNGs. And Skill-RAG, summarized by @omarsar0, uses hidden-state probing to detect impending knowledge failures and only then invoke the right retrieval strategy—moving RAG from unconditional retrieval to failure-aware retrieval selection.

Top tweets (by engagement)

• Kimi K2.6 launch: Moonshot’s release dominated technical engagement, combining strong benchmark claims with unusual long-horizon agent systems details in the main launch thread.

• Anthropic’s AWS expansion: Anthropic said it secured up to 5 GW of compute with Amazon, with an additional $5B investment today and up to $20B more later, a major signal on frontier-model capex and supply strategy via @AnthropicAI.

• Codex Chronicle: OpenAI’s move toward screen-derived memory in Chronicle was one of the more consequential product-direction tweets for coding agents.

• Qwen3.6-Max-Preview: Alibaba’s preview release reinforced that top-tier coding/agent competition is no longer concentrated in a handful of Western labs.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Kimi K2.6 Model Release and Benchmarks

Read more

TL;DR: 95% of cancer treatments fail to pass clinical trials, but it may be a matching problem — if we better understood what patients have which tumors which will respond to which treatments, success rates improve dramatically and millions of lives can be saved — with the treatments we ALREADY have.

See our full episode dropping today:

Why Big Pharma is licensing AI Models

Tolstoy famously wrote, ‘All healthy cells are alike; each cancer cell is unhappy in its own way.’ Or something like that. Cancer might be the most misunderstood disease out there. It’s not one disease, it’s a family of diseases. Hundreds, maybe thousands, of unique diseases each with its own underlying biology. With this lens, saying you’ll “cure cancer” is like saying you’ll solve legos.

We keep hearing AI will cure cancer, but sadly it may not be so easy. Today’s guests — Ron Alfa and Daniel Bear from Noetik — thinks they can use AI to break through a core bottleneck in the treatment development process.

GSK recently signed a $50M deal for their technology that also includes an (undisclosed) long-term licensing deals for Noetik’s models like the recently announced TARIO-2, an autoregressive transformer trained on one of the largest sets of tumor spatial transcriptomics datasets in the world. Whole-plex spatial transcriptomics is the richest way to read a tumor, and approximately ~0% of cancer patients going through standard care ever get one — and TARIO-2 can now predict an ~19,000-gene spatial map from the H&E assay every patient already has.

Most big AI plays in BioTech have focused on discovery, and usually result in an in-house development effort (meaning tools companies usually become drug companies). This deal stands out in that it is a software licensing deal, and represents a commitment to a platform rather than a drug.

With attention on other software tools for drug development (see the Boltz episode and Isomorphic for example), it is starting to look like the appetite of Pharma for biotech tools has finally started to grow. Why the sudden interest?

Cancer is hard

Biology is hard, cancer is harder. But despite this, we’ve made incredible progress. So many cancers that would have been death sentences twenty years ago are routinely survivable. It used to be our main strategy was just chemotherapy — poison you and hope the tumor dies before you do. Now, there are many treatments that actually kill a tumor and leave the rest of you intact! Immune checkpoint inhibitors like Keytruda and Opdivo target the defenses of dozens of tumor types. CAR-T therapy adds modified T-cells to your blood that can target B-cell malignancies very accurately. Antibody Drug Conjugates such as Trastuzumab combine a drug with an antibody, allowing it to target very specific (cancer) cells. We truly live in marvelous times.

With that said, we still have a long way to go. For every type of cancer with a miracle treatment, we have many more that are still death sentences. The world spends $20-30 billion a year trying to cure cancers, with hundreds of clinical trials yearly.Yet, progress is slow with a 95% failure rate in clinical trials.

The lab doesn’t translate to the clinic

Are we leaving something on the table? Enter Noetik and Ron Alfa. Ron’s core thesis is that many of these “failed” treatments actually work! But we’re not looking at the right patients with the right tumors. If only we had a way to really understand the unique types of cancer biologies and which patients will respond to which treatments, we might be able to show a much higher success rate. Millions of lives (and billions of dollars) may ride on this.

The Hard part: Blind Faith in Data Collection

Ron and Noetik had the conviction to spend almost two years just collecting data. Lots, and lots, and lots, of data. Noetik has acquired thousands of actual human tumors, and collects a large multimodal dataset1 of hundreds of millions of images that allows them to create a detailed map of the cell makeup in the local environment.2 These are real human tumors, not frankenstein mouse models or immortal cell lines.

This data is then fed into a massive self-supervised model, creating a “virtual cell”. This model has a deep understanding of cancer biology — Noetik has worked carefully to show it can distinguish different types of tumors. Maybe even tumors we didn’t identify as distinct previously! More recently they figured out how to scale up their model and data, and see no limit in their scaling laws!

Noetik’s models can simulate how a patient will respond to experimental treatments. They are working with partners to test promising drugs that were demonstrated to be safe, but not effective. If these models work as hoped, Noetik will bring new cancer treatments to patients without developing a new drug! Their models will also guide the discovery process towards drugs that are more likely to make it through clinical trials. You can imagine why this is so attractive to GSK.

We’ll see…

Ron and Dan make pretty persuasive arguments that their models will truly assist in cohort selection in useful ways and this seems valuable. And we think it’s pretty clear that

1. Translation from lab to clinic is the biggest bottleneck for drug development.

2. Better cohort selection using biomarkers is likely to improve translation from lab to clinic.

Noetik has already had some success here. We’ll see if they’re able to translate that into a reliable advantage.

Stepping back a bit from the technology, curing cancer is a pretty unambiguously positive application of AI. It is also a very hard problem to solve. Our guess is that most people have been impacted by cancer or will be at some point soon. And we hope that learning about the amazing work that companies like Noetik are doing will inspire a generation of AI engineers to work on the hardest and most exciting problems that society faces.

Full Video Pod:

To the engineering audience, it was more sober, talking about the unprecedented levels of security incidents (60x more reports than curl, at least 20% of skill contributions malicious) and scaling issues involved in maintaining the fastest growing open source project in history:

An AMA moderated by me is included at the end.

Contrast them, thoughts welcome.

AI News for 4/16/2026-4/17/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Anthropic’s Claude Opus 4.7 and Claude Design rollout

• Claude Design launched as Anthropic’s first design/prototyping surface: @claudeai announced Claude Design, a research-preview tool for generating prototypes, slides, and one-pagers from natural-language instructions, powered by Claude Opus 4.7. The launch immediately framed Anthropic as moving beyond chat/coding into design tooling; multiple observers called it a direct shot at Figma/Lovable/Bolt/v0, including @Yuchenj_UW, @kimmonismus, and @skirano. The market reaction itself became part of the story, with @Yuchenj_UW and others noting Figma’s sharp drawdown after the announcement. Product details surfaced via @TheRundownAI: inline refinement, sliders, exports to Canva/PPTX/PDF/HTML, and handoff to Claude Code for implementation.

• Opus 4.7 looks stronger overall, but the rollout was noisy: third-party benchmark posts were broadly favorable. @arena put Opus 4.7 #1 in Code Arena, +37 over Opus 4.6 and ahead of non-Anthropic peers there; the same account also had it at #1 overall in Text Arena with category wins across coding and science-heavy domains here. @ArtificialAnlys reported a near three-way tie at the top of its Intelligence Index—Opus 4.7 57.3, Gemini 3.1 Pro 57.2, GPT-5.4 56.8—while also placing Opus 4.7 first on GDPval-AA, their agentic benchmark. They also noted ~35% fewer output tokens than Opus 4.6 at higher score, and introduction of task budgets plus full removal of extended thinking in favor of adaptive reasoning. But user experience was mixed in the first 24 hours: @VictorTaelin reported regressions and context failures, @emollick said Anthropic had already improved adaptive thinking behavior by the next day, and @alexalbert__ confirmed that many initial bugs had been fixed. There were also complaints about product stability in Design itself from @theo and account-level safety issues from the same account here.

• Cost/efficiency discussion became almost as important as raw quality: @scaling01 claimed ~10x fewer tokens for some ML problem runs versus prior high-end models while maintaining similar performance, while @ArtificialAnlys placed Opus 4.7 on the price/performance Pareto frontier for both text and code. Not every benchmark agreed on absolute leadership—e.g. @scaling01 noted it still trails Gemini 3.1 Pro and GPT-5.4 on LiveBench—but the consensus from these posts is that Anthropic materially improved the model’s agentic utility and efficiency.

Computer use, coding agents, and harness design

• Computer-use UX is becoming a mainstream product category: OpenAI’s Codex desktop/computer-use updates drew unusually strong practitioner reactions. @reach_vb called subagents + computer use “pretty close” to AGI in practical feel; @kr0der, @HamelHusain, @mattrickard, and @matvelloso all emphasized that Codex Computer Use is not just flashy but fast, able to drive Slack, browser flows, and arbitrary desktop apps, and may be the first genuinely usable computer-use platform for enterprise legacy software. @gdb explicitly framed Codex as becoming a full agentic IDE.

• The field is converging on “simple harness, strong evals, model-agnostic scaffolding”: several high-signal posts argued that reliability gains now come more from harnesses than from chasing the very largest models. @AsfiShaheen described a three-stage financial analyst pipeline—router / lane / analyst—with strict context boundaries and gold sets for each stage, arguing that many bugs were actually instruction/interface bugs. @AymericRoucher extracted the same lesson from the leaked Claude Code harness: simple planning constraints plus a cleaner representation layer outperform “fancy AI scaffolds.” @raw_works showed an even starker example: Qwen3-8B scored 33/507 on LongCoT-Mini with dspy.RLM, versus 0/507 vanilla, arguing the scaffold—not fine-tuning—did “100% of the lifting.” LangChain shipped more of these patterns into product: @sydneyrunkle added subagent support to deepagents deploy, and @whoiskatrin announced memory primitives in the Agents SDK.

• Open-source agent stacks continue to proliferate: Hermes Agent remained a focal point. Community ecosystem overviews from @GitTrend0x highlighted derivatives like Hermes Atlas, Hermes-Wiki, HUDs, and control dashboards. @ollama then shipped native Hermes support via ollama launch hermes, which @NousResearch amplified. Nous and Kimi also launched a $25k Hermes Agent Creative Hackathon @NousResearch, signaling a push from coding/productivity into creative agent workflows.

Agent research: self-improvement, monitoring, web skills, and evaluation

• A cluster of papers pushed agent robustness and continual improvement forward: @omarsar0 summarized Cognitive Companion, which monitors reasoning degradation either with an LLM judge or a hidden-state probe. The headline result is notable: a logistic-regression probe on layer-28 hidden states can detect degradation with AUROC 0.840 at zero measured inference overhead, while the LLM-monitor version cuts repetition 52–62% with ~11% overhead. Separate work on web agents from @dair_ai described WebXSkill, where agents extract reusable skills from trajectories, yielding up to +9.8 points on WebArena and 86.1% on WebVoyager in grounded mode. And @omarsar0 also highlighted Autogenesis, a protocol for agents to identify capability gaps, propose improvements, validate them, and integrate working changes without retraining.

• Open-world evals are becoming a serious theme: several posts argued current benchmarks are too narrow. @CUdudec endorsed open-world evaluations for long-horizon, open-ended settings; @ghadfield connected this to regulation and “economy of agents” questions; and @PKirgis discussed CRUX, a project for regular open-world evaluations of AI agents in messy real environments. On the measurement side, @NandoDF proposed broad NLL/perplexity-based eval suites over out-of-training-domain books/articles across 2500 topic buckets, though that sparked debate about whether perplexity remains informative after RLHF/post-training from @eliebakouch, @teortaxesTex, and others.

• Document/OCR and retrieval evals also got more agent-centric: @llama_index expanded on ParseBench, an OCR benchmark centered on content faithfulness with 167K+ rule-based tests across omissions, hallucinations, and reading-order violations—explicitly reframing the bar from “human-readable” to “reliable enough for an agent to act on.” In retrieval, @Julian_a42f9a noted new work showing late-interaction retrieval representations can substitute for raw document text in RAG, suggesting some RAG pipelines may be able to bypass full-text reconstruction.

Open models, local inference, and inference systems

• Qwen3.6 local/quantized workflows were a practical bright spot: @victormustar shared a concrete llama.cpp + Pi setup for Qwen3.6-35B-A3B as a local agent stack, emphasizing how viable local agentic systems now feel. Red Hat quickly followed with an NVFP4-quantized Qwen3.6-35B-A3B checkpoint @RedHat_AI, reporting preliminary GSM8K Platinum 100.69% recovery, and @danielhanchen benchmarked dynamic quants, claiming many Unsloth quants sit on the Pareto frontier for KLD vs disk space.

• Consumer-hardware inference keeps improving: @RisingSayak announced work with PyTorch/TorchAO enabling offloading with FP8 and NVFP4 quants without major latency penalties, explicitly targeting consumer GPU users constrained by memory. Apple-side local inference also got a showcase with @googlegemma, which demoed Gemma 4 running fully offline on iPhone with long context.

• Inference infra updates worth noting: @vllm_project highlighted MORI-IO KV Connector with AMD/EmbeddedLLM, claiming 2.5× higher goodput on a single node via a PD-disaggregation-style connector. Cloudflare continued its agent/AI-platform push with isitagentready.com @Cloudflare, Flagship feature flags @fayazara, and shared compression dictionaries yielding dramatic payload reductions such as 92KB → 159 bytes in one example @ackriv.

AI for science, medicine, and infrastructure

• Scientific discovery and personalized health were prominent applied themes: @JoyHeYueya and @Anikait_Singh_ posted about insight anticipation, where models generate a downstream paper’s core contribution from its “parent” papers; the latter introduced GIANTS-4B, an RL-trained model that reportedly beats frontier models on this task. On the health side, @SRSchmidgall shared a biomarker-discovery system over wearable data whose first finding was that “late-night doomscrolling” predicts depression severity with ρ=0.177, p<0.001, n=7,497—notable because the model itself named the feature. Separately, @patrickc argued current coding agents are already highly useful for personalized genome interpretation, describing <$100 analysis runs that surfaced a roughly 30× elevated melanoma predisposition plus follow-on interventions.

• Large-scale compute buildout remains a core meta-story: @EpochAIResearch surveyed all 7 US Stargate sites and concluded the project appears on track for 9+ GW by 2029, comparable to New York City peak demand. @gdb framed Stargate as infrastructure for a “compute-powered economy,” while @kimmonismus put today’s annual global datacenter capex at roughly 5–7 Manhattan Projects per year in inflation-adjusted terms.

Top tweets (by engagement)

• Claude Design / Anthropic product expansion: @claudeai launches Claude Design, by far the day’s biggest pure-AI product launch signal.

• Model benchmarking / rankings: @ArtificialAnlys on Opus 4.7 tying for #1 overall and leading GDPval-AA.

• Coding agents / computer use: @cursor_ai doubles Composer 2 limits in the new agents window and @HamelHusain on Codex Computer Use.

• Open-source agents: @ollama ships native Hermes Agent support.

• Applied AI in medicine: @patrickc on coding agents for genome analysis and personalized prevention.

• Infra / power scaling: @EpochAIResearch on Stargate’s 9+ GW trajectory.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Qwen3.6 Model Launch and Features

Read more

The key chart is this one:

Basically 4.7-low is strictly better than 4.6-medium, 4.7-medium is strictly better than 4.6-high, 4.7-high is now better than 4.6-max, and there is a new xhigh effort level that Claude Code defaults to. While Anthropic says the new tokenizer (new pretrain?) can cause up to 35% more token usage, the overall reasoning efficiency has improved so much that overall token use is STILL down by up to 50% of their former equivalents. The true test is if default Claude Code, now 11 points higher on SWE-Bench Pro, does noticeably better in your own usecases.

The other notable capability that quite literally has to be seen to be believed, is the “substantially better vision”: Opus 4.7 has better vision for high-resolution images: it can accept images up to 2,576 pixels on the long edge (~3.75 megapixels), more than three times as many as prior Claude models. This opens up a wealth of multimodal uses that depend on fine visual detail: computer-use agents reading dense screenshots, data extractions from complex diagrams, and work that needs pixel-perfect references. More details in the focused topic summary below.

AI News for 4/14/2026-4/16/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

Top Story: Claude Opus 4.7

Anthropic officially launched Claude Opus 4.7 as its newest top-tier Opus model, positioning it as better at long-running work, coding, instruction following, self-verification, computer use, and knowledge work than Opus 4.6, while keeping list pricing unchanged at $5 / $25 per million input/output tokens according to user summaries and launch discussion [@claudeai, @kimmonismus]. The release sparked unusually active technical discussion around benchmark gains, a new tokenizer, higher image resolution support, new xhigh reasoning effort, token-cost implications, and whether Opus 4.7 is a straightforward 4.6 successor, a new base model, or a partially distilled “Mythos-adjacent” system.

Release details and product changes

Official framing. Anthropic’s launch pitch emphasized three behavioral improvements: better handling of long-running tasks, more precise instruction following, and stronger self-verification before responding [@claudeai].

Availability.

• Claude platform / app reported live immediately [@dejavucoder].

• Claude Code shipped day-one support and set xhigh as the default effort level [@_catwu, @_catwu].

• Anthropic also launched or highlighted task budgets in public beta, /ultrareview in Claude Code, and broader Auto mode access for Claude Code Max users [@kimmonismus].

New effort tier.

• Multiple users noted a new xhigh reasoning effort mode, positioned between high and max [@scaling01, @scaling01].

• Cat Wu said Claude Code now defaults to xhigh for Opus 4.7 [@_catwu].

Vision/computer use changes.

• User summaries reported support for images up to 2,576 px on the long edge (~3.75 MP), described as 3x larger than previous Claude image inputs [@kimmonismus].

• Anthropic employee Alex Albert highlighted “No more downscaling of high-res images” and better output taste in UI/slides/docs [@alexalbert__].

• This was repeatedly linked to better computer use and screenshot-heavy workflows [@dejavucoder, @omarsar0].

Tokenizer and token economics.

• Several observers discovered Opus 4.7 uses a different tokenizer from 4.6 [@natolambert, @nrehiew_].

• Kimmonismus summarized Anthropic’s caveat that the same input can map to 1.0–1.35x more tokens depending on content type [@kimmonismus].

• This triggered debate over whether 4.7 is effectively a new base model, a tokenizer-swapped continuation, or some kind of midtraining/distillation bridge from Mythos [@natolambert, @stochasticchasm, @eliebakouch, @maximelabonne].

• Anthropic employee Boris Cherny later said they increased limits for all subscribers to offset increased token use [@bcherny, @bcherny].

Benchmarks and measurable progress

Reported benchmark gains vs Opus 4.6

The most cited launch numbers came from benchmark screenshots and summaries shared by external accounts:

• SWE-bench Pro: 64.3%, with users citing roughly +11 points over Opus 4.6 [@scaling01, @kimmonismus]

• SWE-bench Verified: 87.6%, roughly +7 points vs 4.6 [@scaling01, @scaling01]

• TerminalBench 2.0: 69.4%, around +4 points [@scaling01, @kimmonismus]

• Document reasoning: 80.6%, up from 57.1% per third-party discussion [@scaling01, @llama_index]

• GDPval-AA: 1753 Elo [@scaling01, @ArtificialAnlys]

• ARC-AGI-1: 92%; ARC-AGI-2: 75.83% per [@scaling01]

Artificial Analysis said Opus 4.7 launched as the new #1 on GDPval-AA, with an implied ~60% head-to-head win rate vs GPT-5.4 on that task set [@ArtificialAnlys].

• Anthropic increased subscriber limits to compensate for greater token usage [@bcherny, @bcherny].

• Anthropic acknowledges benchmark tradeoffs and retained MRCR in the system card “for scientific honesty,” while signaling a shift toward Graphwalks as a preferred long-context metric [@bcherny].

Vals AI said Opus 4.7 took the #1 spot on the Vals Index at 71.4%, up from a previous best 67.7%, and also ranked #1 on Vibe Code Bench, Vals Multimodal, Finance Agent, Mortgage Tax, SAGE, SWE-Bench, and Terminal Bench 2 [@ValsAI].

They separately said Opus 4.7 became #1 on Vibe Code Benchmark at 71%, versus no model above 25% when they first launched the benchmark 4.5 months earlier [@ValsAI].

Product/evals from partners and customers

• Cursor said its internal benchmark jumped from 58% to 70% with Opus 4.7 [@cursor_ai, @scaling01].

• A separate Cursor post said, across 500 teams, developers are tackling 68% more high-complexity tasks this year, though that was about better models generally, not solely Opus 4.7 [@cursor_ai].

• Notion reportedly saw a 14% lift on internal evals with one-third of tool errors [@mikeyk].

• GitHub reportedly saw similar improvements, though no hard numbers were included in the tweet thread [@scaling01].

Document understanding: progress, but mixed economics

LlamaIndex and Jerry Liu provided useful independent nuance:

• LlamaIndex’s ParseBench-style comparison said Opus 4.7 massively improved charts (13.5% → 55.8%) but only slightly improved formatting (64.2% → 69.4%), content (89.7% → 90.3%), tables (86.5% → 87.2%), and regressed on layout (16.5% → 14.0%) [@llama_index].

• Jerry Liu separately said Opus 4.7 is “quite good at tables,” better on charts, and strongest on content faithfulness, but expensive for OCR-like use at ~7¢/page vs their agentic mode at ~1.25¢/page and cost-effective mode around ~0.4¢/page [@jerryjliu0].

This is one of the clearest examples of independent evaluation tempering launch optimism: broad capability improved, but specific enterprise document pipelines may still prefer specialized stacks on cost/performance grounds.

Opinions / interpretations

• “This is a distilled version of Mythos” [@eliebakouch].

• “This is a new base model because the tokenizer changed” [@natolambert].

• “Anthropic artificially kept cyber scores low during training” is partly factual insofar as users quote the system card language about differentially reducing some capabilities, but broader claims about “nerfed Mythos” are interpretation [@scaling01, @Yuchenj_UW].

• “Benchmarks don’t do it justice” and “actual usage is massively improved” are subjective but widely repeated by hands-on users [@mweinbach, @jeremyphoward].

• “System prompt has lobotomized the model” is a user complaint about behavior changes, not an established fact [@theo].

Different perspectives

Supportive: meaningful real-world upgrade

A large portion of technical users argued this is a substantial iteration, especially given more frequent release cadence.

• Scaling01 repeatedly pushed back on “mid update” takes, noting the jump from around 80% to almost 90% on SWE-bench Verified and emphasizing this would have looked huge in prior release cycles [@scaling01, @scaling01, @scaling01].

• Alex Albert highlighted better async work, more predictable effort levels, better image handling, and stronger taste in UI/docs [@alexalbert__].

• Michael Weinbach said after just two prompts that behavior and instruction following were “pretty massive” improvements [@mweinbach].

• Jeremy Howard said it was the first model that “gets” what he’s doing and praised its willingness to discuss rather than bulldoze ahead [@jeremyphoward, @jeremyphoward].

• Cat Wu explicitly advised users to treat it like an engineer you delegate to, not a pair programmer you micromanage, suggesting Anthropic sees it as stronger in autonomous execution [@_catwu].

Neutral / analytical: strong update with tradeoffs

Some of the best commentary was technical and mixed.

• Kimmonismus called it a “solid upgrade” focused on Anthropic’s core buyer priorities: agentic coding reliability, vision for computer-use agents, and knowledge work—but also “obviously shy to Mythos” [@kimmonismus].

• Artificial Analysis validated the GDPval-AA gain and #1 ranking, but did not frame it as an across-the-board blowout [@ArtificialAnlys].

• LlamaIndex and ParseBench results suggested noticeable but uneven document gains with real pricing constraints [@llama_index, @jerryjliu0].

Skeptical / critical: regressions, token inflation, and UX concerns

There was also substantial pushback.

• Multiple users said long-context performance looked worse, especially on MRCR / needle-in-a-haystack-style metrics [@scaling01, @nrehiew_, @eliebakouch, @kimmonismus].

• Anthropic’s Boris Cherny replied that MRCR is being phased out because it overweights distractor-stacking tricks and that Graphwalks is a better applied-reasoning signal; he gave numbers showing Graphwalks 38.7% → 58.6% from 4.6 to 4.7 [@bcherny, @scaling01].

• Tokenizer changes led to complaints about Opus becoming a “token guzzler” and potentially raising effective costs despite flat list pricing [@dejavucoder, @madiator].

• Yuchen said Claude web only exposed “Adaptive” or non-thinking, with no explicit force-thinking toggle, which for some users made non-coding tasks feel worse in practice [@Yuchenj_UW].

• Mikhail Parakhin similarly said first impressions on non-coding replies were “dumber” because he couldn’t force reasoning [@MParakhin].

• Theo sharply criticized the new system prompt as “lobotomized,” and later suggested trying the model in T3 Chat “without the lobotomized system prompt” [@theo, @theo].

Safety / governance angle

• Scaling01 highlighted a system-card statement that Anthropic experimented with efforts to differentially reduce cyber capabilities during training [@scaling01].

• At the same time, users noted Opus 4.7 still scores higher than 4.6 on some exploitation-related evaluations like Firefox shell exploitation, and has prompt-injection robustness close to Mythos [@scaling01, @scaling01].

• One user hyperbolically said “Opus is going to be a bioweapon risk at this pace,” reflecting the ongoing tendency to conflate general capability jumps with worst-case misuse narratives [@scaling01].

Claude Code workflow guidance from Anthropic

Cat Wu’s thread is a useful operational signal for engineers:

1. Delegate, don’t micromanage [@_catwu]

2. Put full goal + constraints + acceptance criteria up front [@_catwu]

3. Tell the model how to verify changes; encode testing workflows in claude.md or skills [@_catwu]

That strongly suggests Anthropic optimized toward autonomous task loops where explicit validation is central.

Examples of progress in practice

Read more

For anyone that learned to code in the last 15 years it is hard to imagine a life without Git, GitHub, and Pull Requests, but there was a time before them, and it well may come to pass that there is life after.

Pull Requests were arguably invented in 2005, successfully popularized by GitHub, and only 21 years later, GitHub is for the first time in history allowing people to disable pull requests on their open source repos (you could only disable issues before).

The rise of Generative AI in code has spelled the pending death of the Pull Request for a while now — Pete Steinberger is by now well known (along with Theo) for only wanting Prompt Requests rather than Pull Requests (for multiple reasons, eg 1) no merge conflicts, 2) it’s easier for the maintainer to fix/add to the prompt than to look at code, 3) less likely to have malicious/insecure code slipped into an innocent looking PR), and other folks like Mitchell Hashimoto and Amp Code have created “reputation”-based systems for handling untrusted code contributions.

In Building for Trillions of Agents, Aaron Levie noted that “the path forward is to make software that agents want.” Humans invented git for human collaboration reasons. It’s increasingly clear that Git-based workflows may not be suitable once we remove the human bottleneck from the flow of code.

And if Code Reviews are dead, and Pull Reviews are dead… how long until Git itself is dead?

AI News for 4/14/2026-4/15/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

OpenAI Agents SDK Expansion and the New Sandbox-Oriented Agent Stack

• OpenAI split the agent harness from compute/storage and pushed its Agents SDK toward long-running, durable agents with primitives for file/computer use, skills, memory, and compaction. The harness is now open-source and customizable, while execution can be delegated to partner sandboxes instead of being tightly coupled to OpenAI infra, per @OpenAIDevs, follow-up, and @snsf. This effectively makes “Codex-style” agents more reproducible by third parties and shifts differentiation toward orchestration, state management, and secure execution.

• A notable ecosystem formed around that launch immediately: @CloudflareDev, @modal, @daytonaio, @e2b, and @vercel_dev all announced official sandbox integrations. The practical pattern is converging on stateless orchestration + stateful isolated workspaces. Example builds already appeared, including a Modal-backed ML research agent with GPU sandboxes, subagents, persistent memory, and fork/resume snapshots from @akshat_b, and Cloudflare guides for Python agents that execute tasks in a sandbox and copy outputs locally from @whoiskatrin.

Cloudflare’s Project Think, Agent Lee, and Voice Agents

• Cloudflare had one of the busiest agent-infra release cycles. @whoiskatrin and @aninibread introduced Project Think, a next-gen Agents SDK centered on durable execution, sub-agents, persistent sessions, sandboxed code execution, a built-in workspace filesystem, and runtime tool creation. In parallel, @Cloudflare launched Agent Lee, an in-dashboard agent using sandboxed TypeScript to shift Cloudflare’s UI from manual tab navigation to prompt-driven operations; @BraydenWilmoth showed it issuing infra tasks and generating UI-backed results.

• Voice and browser tooling also moved into the core stack. @Cloudflare shipped an experimental real-time voice pipeline over WebSockets for continuous STT/TTS, while @korinne_dev described voice as just another input channel over the same agent connection. On browser automation, @kathyyliao summarized the rebranded Browser Run stack: Live View, human-in-the-loop intervention, session recordings, CDP endpoints, WebMCP support, and higher limits. Taken together, Cloudflare is making a strong case that the production agent platform is really a composition of durable runtime + UI grounding + browser + voice + sandbox.

Hermes Agent’s Self-Improving Workflow and Competitive Positioning

• Hermes Agent’s distinctive idea is not just tool use but persistent skill formation. A Chinese-language comparison from @joshesye contrasts OpenClaw as a more GUI-first, ready-to-use personal assistant with Hermes as a “professional” agent that decides whether a completed workflow is reusable and automatically turns it into a Skill. This “learn from completed tasks” framing appeared repeatedly: @chooseliberty showed Hermes autonomously backfilling tracking data, updating a cron job, then saving the workflow as a reusable skill; @NeoAIForecast emphasized session hygiene and thread branching/search as critical to turning Hermes into a real work environment rather than a disposable chat box.

• Community sentiment strongly positioned Hermes against OpenClaw, often bluntly. Examples include @vrloom, @theCTO, and @Teknium highlighting Hermes’ role in real workflows, including the now-viral autonomous Gemma 4 “abliteration” story from @elder_plinius: the agent loaded a stored skill, diagnosed NaN instability in Gemma 4, patched the underlying library, retried multiple methods, benchmarked the result, generated a model card, and uploaded artifacts to Hugging Face. There were also concrete product additions: browser control via /browser connect from @0xme66, QQBot + AWS Bedrock support from @Teknium, a native Swift desktop app alpha from @nesquena, and ongoing ecosystem tooling like artifact-preview and hermes-lcm v0.3.0.

Model, Architecture, and Training Releases: Sparse Diffusion, Looped Transformers, and Efficient Long-Context MoEs

• Several technically meaningful open releases landed across modalities. @withnucleusai announced Nucleus-Image, positioned as the first sparse MoE diffusion model: 17B parameters, 2B active, Apache 2.0, with weights, training code, and dataset recipe, and day-0 support in diffusers. NVIDIA followed with Lyra 2.0, a framework for generating persistent, explorable 3D worlds that maintains per-frame 3D geometry and uses self-augmented training to reduce temporal drift, per @NVIDIAAIDev. On multimodal retrieval, @thewebAI open-sourced webAI-ColVec1, claiming top ViDoRe V3 performance for document retrieval without OCR or preprocessing.

• Architecture research around compute efficiency was especially strong. @hayden_prairie, @realDanFu, and @togethercompute introduced Parcae, a stabilized layer-looping Transformer formulation. The claim: for fixed parameter budgets, looping blocks can recover the quality of a model roughly 2x the size, yielding a new scaling axis where FLOPs scale via looping, not just parameters/data. NVIDIA also surfaced Nemotron 3 Super, summarized by @dair_ai: an open 120B hybrid Mamba-Attention MoE with 12B active parameters, 1M context, trained on 25T tokens, with up to 2.2x throughput vs GPT-OSS-120B and 7.5x vs Qwen3.5-122B. These releases collectively point to a theme: memory bandwidth and long-context throughput are increasingly first-class architectural objectives.

Google/Gemini’s Product Surge: Mac App, Personal Intelligence, TTS, and Open Multimodal Models

• Google stacked multiple launches in one cycle. The most visible was the native Gemini app for Mac, announced by @GeminiApp, @joshwoodward, and @sundarpichai: Option + Space activation, screen sharing, local file context, native Swift implementation, and broad macOS availability. In parallel, Personal Intelligence expanded globally in Gemini and into Chrome, allowing users to connect signals from products like Gmail and Photos, framed around transparency and user-controlled app connections by @Google and @GeminiApp.

• The more technically interesting model launch was Gemini 3.1 Flash TTS. @GoogleDeepMind, @OfficialLoganK, and @demishassabis positioned it as a highly controllable TTS model with Audio Tags, 70+ languages, inline nonverbal cues, multi-speaker support, and SynthID watermarking. Independent evaluation from @ArtificialAnlys put it at #2 on its Speech Arena, just 4 Elo behind the top model. Google also open-sourced TIPS v2, a foundational text-image encoder under Apache 2.0 with new pretraining recipes, via @osanseviero, and the community flagged the day as unusually dense for Google AI product velocity.

Research Signals: AI-Assisted Math, Long-Horizon Agents, Eval Shifts, and Open Data

• The highest-signal research discourse was around AI-assisted mathematics. @jdlichtman reported that GPT-5.4 Pro produced a proof for Erdős problem #1196, surprising experts by rejecting a long-assumed proof gambit and instead exploiting a technically counterintuitive analytic path using the von Mangoldt function. Follow-ups from @jdlichtman, @thomasfbloom, @gdb, and others framed it as potentially the first AI-generated “Book Proof” broadly respected by mathematicians. That matters less as a one-off result than as evidence that models may now occasionally find non-aesthetic but compact lines of attack in mature research spaces.

• Long-horizon agent research also kept converging on state management and harness design. @omarsar0 summarized AiScientist, where a thin orchestrator coordinates specialized agents through durable workspace artifacts in a File-as-Bus pattern; removing that bus hurts PaperBench and MLE-Bench Lite materially. @dair_ai highlighted Pioneer Agent for continual small-model improvement loops, while @yoonholeee open-sourced Meta-Harness, a repo meant to help users implement robust harnesses in new domains. On evals, @METR_Evals estimated Gemini 3.1 Pro (high thinking) at a 50% time horizon of ~6.4 hours on software tasks, and @arena showed Document Arena top ranks shifting with Claude Opus 4.6 Thinking at #1 and Kimi-K2.5 Thinking as the best open model. Meanwhile, @TeraflopAI released 43B tokens of SEC EDGAR data, reinforcing the day’s broader push toward more open datasets and open infrastructure.

Top tweets (by engagement)

• Gemini on Mac: @sundarpichai and @GeminiApp drove the biggest launch engagement around the native desktop app.

• Gemini 3.1 Flash TTS: @OfficialLoganK and @GoogleDeepMind highlighted a materially more controllable TTS stack.

• AI-assisted math proof: @jdlichtman and @gdb sparked the strongest research discussion of the day.

• OpenAI Agents SDK update: @OpenAIDevs marked a meaningful platform shift toward open harnesses and partner sandboxes.

• Anthropic’s subliminal learning paper in Nature: @AnthropicAI drew major attention to hidden-trait transmission through training data.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

Read more

• (friend of the show) Aaron Levie reports that “AI is not causing anyone to do less work right now, and similar to Silicon Valley people feel their teams are the busiest they’ve ever been.”

• Tyler Cowen argues from an economics standpoint that you should work much harder RIGHT NOW whether you believe AI will lower your value OR increase your value.

• Simon Last of Notion commented on today’s pod that he’s back to sleepless nights and 24/7 work for the first time since giving up on ML model training, but this time because of agent layer token anxiety.

How can it both be true that “Agents are doing more work and yet Everyone is working harder”? How can it be true that Claude Mythos has been used internally for 2 months, and yet Claude keeps going down? How can it be true that Model and Agent Labs are more productive than ever and yet acquihiring and acquiring more than ever?

A simple thought exercise we’ve made before is the “Turkey problem”, where, based on real evidence and an abundance of historical data, Turkeys should conclude that life is fantastic and all of humanity is set up to make turkeys well fed as far as they’ve ever experienced. Turkey doomsayers would be alarmist, crackpots, and then ignored. Until Thanksgiving.

Are engineers, or all knowledge workers in general, turkeys, in this scenario? Should our “elasticity” and value of work be increasingly positive, right up to some crossover point we become horses? Now that SWE-Bench is saturated (with SWE-Bench Pro soon to be, Mythos is at 78%) and GDPval rates GPT 5.4 as better than/equal to human experts 83% of the time in most swathes of the economy, what’s left?

Notion is working on Notion’s Last Exam. Greg and Francois are have set out ARC-AGI-3. I’m working on the next frontier of coding evals. But it all seems somewhat moot if hardware is destiny and AGI is predictably a 20GW supercluster away…

…or are there more valuable problems left?

AI News for 4/3/2026-4/4/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Top Tweets (by engagement)

• Google’s Chrome “Skills” turns prompts into reusable browser workflows: Google introduced Skills in Chrome, letting users save Gemini prompts as one-click actions that run against the current page and selected tabs. Google also shipped a library of ready-made Skills, which makes this more than prompt history: it’s effectively lightweight end-user agentization inside the browser.

• Tencent’s HYWorld 2.0 positions world models as editable 3D scene generators, not video models: Ahead of release, @DylanTFWang teased HYWorld 2.0 as an open-source, engine-ready 3D world model that generates editable 3D scenes from a single image.

• Google DeepMind shipped Gemini Robotics-ER 1.6: The new model, announced by @GoogleDeepMind, improves visual/spatial reasoning for robotics, adds safer physical reasoning, and is available in Gemini API / AI Studio. Follow-up posts highlight 93% instrument-reading success and better handling of physical constraints like liquids and heavy objects.

• OpenAI expanded Trusted Access for Cyber with GPT-5.4-Cyber: OpenAI says GPT-5.4-Cyber is a fine-tuned version of GPT-5.4 for defensive security workflows, available to higher-tier authenticated defenders under its Trusted Access program.

• Hugging Face launched “Kernels” on the Hub: @ClementDelangue announced a new repo type for GPU kernels, with precompiled artifacts matched to exact GPU/PyTorch/OS combinations and claimed 1.7x–2.5x speedups over PyTorch baselines.

• Cursor described a multi-agent CUDA optimization system built with NVIDIA: @cursor_ai says its multi-agent software engineering system delivered a 38% geomean speedup across 235 CUDA problems in 3 weeks, a concrete example of agents being applied to systems optimization rather than app scaffolding.

Agent Infrastructure: Hermes, Deep Agents, and Production Harnesses

• Hermes Agent is becoming a serious open local-agent stack, with reliability and memory as the differentiators: Several posts converged on the same theme: users are migrating from alternatives to Hermes Agent because it is more durable for long-running work. The project shipped a substantial v0.9.0 update with web UI, model switching, iMessage/WeChat integration, backup/restore, and Android-via-tmux support via @AntoineRSX, while Tencent highlighted a one-click Lighthouse deployment for always-on cloud hosting with messaging integrations. On the memory side, hermes-lcm v0.2.0 from @SteveSchoettler adds lossless context management with persistent message storage, DAG summaries, and tools to expand compacted context. Community posts from @Teknium, @aiqiang888, and others reinforce that Hermes’ key advantage is less raw model IQ than operational stability, extensibility, and deployability.

• LangChain is pushing “deep agents” toward deployable, multi-tenant, async systems: The deepagents 0.5 release adds async subagents, multimodal file support, and prompt-caching improvements. Related posts emphasize that deepagents deploy is an open alternative to managed agent hosting, with upcoming work around memory scoped to user/agent/org and custom auth / per-user thread isolation via @LangChain and @sydneyrunkle. The interesting pattern here is a shift from “agent demos” to platform concerns: tenancy, isolation, long-lived tasks, and integration surfaces like Salesforce and Agent Protocol-backed servers.

• Harness design is becoming a first-class engineering topic: Multiple posts argued that agent performance depends at least as much on the scaffold as the model. @Vtrivedy10 made the clearest case for task-specific open harnesses over ideology (“thin vs thick”), while @kmeanskaran stressed workflow design, memory switching, and tool output control over frontier-model chasing. This aligns with @ClementDelangue asking for a curated mapping from models to their best coding/agent harnesses, which is increasingly necessary as open-weight models diversify.

Robotics, World Models, and 3D Generation

• Google’s Gemini Robotics-ER 1.6 is a notable productization step for embodied reasoning: The release from @GoogleDeepMind emphasizes better visual/spatial understanding, tool use, and physical constraint reasoning. Follow-ups note 10% better human injury-risk detection, support for reading complex analog gauges, and availability in the API; @_philschmid highlighted 93% success on instrument-reading tasks. This feels less like a robotics foundation-model paper drop and more like a developer-facing embodied-reasoning API.

• World models are shifting from cinematic demos to editable spatial artifacts: Tencent’s HYWorld 2.0 teaser explicitly contrasted itself with video-generation systems by framing the output as a real 3D scene that is editable and engine-ready. On the web side, Spark 2.0 from @sparkjsdev shipped a streamable LoD system for 3D Gaussian splats, targeting 100M+ splat worlds on WebGL2 across mobile, web, and VR. Together these suggest the stack for “AI-generated 3D” is maturing from content generation into interactive rendering and downstream use.

• Open 3D generation is advancing on topology, UVs, rigging, and animation readiness: @DeemosTech introduced SATO, an autoregressive model for topology and UV generation, while @yanpei_cao released AniGen, which generates 3D shape, skeleton, and skinning weights from one image. These are meaningful because the bottleneck in production 3D pipelines is rarely “can you generate a mesh?”; it’s whether the asset is structured enough to animate, texture, and edit.

Models, Benchmarks, and Specialized Systems

Read more

Notion, the knowledge work decacorn, has been building AI tooling since before ChatGPT, with many hits from Q&A in 2023 and unified AI in 2024 and Meeting Notes in 2025. At the end of their last Make user conference, Ryan Nystrom teased Notion 3.0’s Custom Agents - and they are finally embracing the Agent Lab playbook!

Sarah Sachs and Simon Last of Notion join us for a deep dive into how Notion built Custom Agents, why it took years and multiple rebuilds to get right, and what it means to turn a productivity tool into an agent-native system of record for enterprise work.

We go inside the product, engineering, evals, pricing, and org design decisions behind one of the most ambitious AI product efforts in software today — from early failed tool-calling experiments in 2022 to agent harnesses, progressive tool disclosure, meeting notes as data capture, and the long-term vision for software factories and agentic work.

We discuss:

• Sarah and Simon’s path to launching Notion Custom Agents, and why the feature was rebuilt four or five times before it was ready for production

• Why early agent attempts failed: no tool-calling standard, short context windows, unreliable models, and too much complexity exposed to the model

• The “Agent Lab” thesis: not just wrapping a model, but understanding how people collaborate and building the right product system around frontier capabilities

• How Notion thinks about roadmap timing: not swimming upstream against model limitations, but also building early enough that the product is ready when the models are

• Why coding agents feel like the kernel of AGI, and how Notion is thinking about “software factories” made up of agents that spec, code, test, debug, review, and maintain codebases together

• How Sarah runs AI engineering at Notion (“notes from Token Town”): objective-setting over idea ownership, low-ego teams comfortable deleting their own work, and a culture designed to swarm around fast-changing opportunities

• The “Simon Vortex,” company hackathons, and why security gets pulled in early rather than late

• How Notion organizes AI: core AI capabilities and infrastructure, product packaging teams, and a broader company mandate that every product surface must increasingly work for both humans and agents

• Why prototypes have become much easier to build internally, and how “demos over memos” changes product development inside a tool the whole company already uses every day

• Notion’s eval philosophy: regression tests, launch-quality evals, and “frontier/headroom” evals that intentionally only pass ~30% of the time so the company can see where model capabilities are going

• What a “Model Behavior Engineer” is, and why Notion treats eval writing, failure analysis, and model understanding as a distinct function rather than just software engineering

• The changing role of software engineers in the age of coding agents, and why the new job looks less like typing code and more like supervising a rigorous outer system of agents, PRs, and verification loops

• How the “software factory” should work: specs, self-verification, bug flows, subagents, and minimizing human intervention while preserving the invariants that matter

• A live walkthrough of a Notion Custom Agent handling coworking space tenant applications by triaging email, enriching applicants with web search, and writing structured data into a Notion database

• How agents compose inside Notion: shared databases as primitives, agents invoking other agents, “manager agents” supervising dozens of specialized agents, and memory implemented simply as pages and databases

• Notion’s take on MCP vs CLI: why Simon is bullish on CLI’s self-debugging nature, where MCP still makes sense, and how Sarah thinks about capability, determinism, permissioning, and pricing alignment

• The evolution of Notion’s internal agent harness: from early JavaScript coding agents, to custom XML, to Markdown and SQL-like abstractions, to tool definitions, progressive disclosure, and a much shorter system prompt

• Why Notion cares about teaching “the top of the class,” building for sophisticated operators rather than abstracting away too much capability for everyone

• How agent setup works today: agents that can configure themselves, inspect their own failures, and edit their own instructions — with guardrails around permissions

• How Notion prices Custom Agents: credits as an abstraction over tokens, model type, serving tier, web search, and future sandbox costs; why usage-based pricing was necessary; and how “auto” tries to match the right model to the right task

• Why Notion is not eager to train a foundation model, where they do fine-tune and optimize today, and why retrieval/ranking is one of the most important investment areas as more searches come from agents rather than humans

• Why Meeting Notes became one of Notion’s strongest growth loops: not just as transcription, but as high-signal data capture that powers search, custom agents, follow-up workflows, and the broader system of record for company collaboration

• Why Notion is more interested in being the place where collaboration data lives than in building hardware themselves — and how wearables or other capture devices may eventually feed into that system

Sarah Sachs
LinkedIn: https://www.linkedin.com/in/sarahmsachs
X: https://x.com/sarahmsachs

Simon Last
LinkedIn: https://www.linkedin.com/in/simon-last-41404140
X: https://x.com/simonlast

Full Video Episode

Timestamps

• 00:00:00 Introduction and launching Notion Custom Agents

• 00:01:17 Why Notion rebuilt agents four or five times

• 00:03:35 Building for where models are going, not just where they are

• 00:05:32 The Agent Lab thesis, wrappers, and product intuition

• 00:08:07 User journeys, leadership, and low-ego AI teams

• 00:13:16 The Simon Vortex, hackathons, and bringing security in early

• 00:16:39 Team structure, demos over memos, and building for agents

• 00:20:25 Evals, Notion’s Last Exam, and the Model Behavior Engineer role

• 00:27:37 Evals as an agent harness and the changing role of software engineers

• 00:30:42 The software factory: specs, verification, and agent workflows

• 00:32:18 Live demo: a custom agent for coworking space applications

• 00:35:08 Composing agents, manager agents, and memory as pages

• 00:38:15 Notion Mail, Gmail, native integrations, and tools

• 00:39:43 MCP vs CLI and the cost of capability

• 00:44:13 When Notion uses MCP vs building its own integrations

• 00:47:43 The history of Notion’s agent harness rebuilds

• 00:55:35 Power users, public tools, and the setup agent

• 00:58:01 Self-fixing agents, permissions, and “flippy”

• 01:01:13 Pricing, credits, and choosing the right model automatically

• 01:09:01 Why Notion isn’t training its own frontier model

• 01:14:07 Retrieval, ranking, and search built for agents

• 01:17:27 Meeting Notes as data capture and workflow automation

• 01:21:18 Wearables, hardware, and Notion as the system of record

• 01:23:45 Outro

Transcript

[00:00:00] Alessio: Hey everyone. Welcome to the Latent Space podcast. This is Alessio founder of Kernel Labs and I’m joined by swyx, editor of the Latent Space.

[00:00:11] swyx: Hello. Hello. We’re back in the beautiful studio that, uh, Alessio has set up for us with Simon and Sarah from Notion. Welcome.

[00:00:18] Sarah Sachs: Thanks for having us.

[00:00:19] Alessio: Thanks for having us. Yeah.

[00:00:20] swyx: Congrats on the launch recently the custom agents, finally it’s here. How’s it feel?

[00:00:26] Sarah Sachs: We ship things slowly. So it had been in Alpha for a little bit and at the point at which is it’s an alpha, um, there’s a group of people that are making sure it’s ready for prod, and then there’s a group of people working on the next thing.

So sometimes some of these launches are a bit delayed satisfaction, so it’s quite nice to remind yourself all the work you did because we do have a habit of like. Being two or three milestones ahead. Uh, just ‘cause you have to be, you know, you can’t get complacent. Um, but it’s been great that people understood how this is helpful.

And I think that’s just easier in general building AI tools today than it was two, three years ago. People kind of get it and so that user education, um, there’s just, it was our most successful launch in terms of free trials and converting people and things like that. It was really successful, so yeah.

But there’s a lot to build.

[00:01:12] swyx: Making it free for three months helps.

[00:01:16] Sarah Sachs: Yep.

[00:01:17] Simon Last: It was definitely super exciting for me because it’s probably the fourth or fifth time that we rebuilt that.

[00:01:22] swyx: Yes.

[00:01:23] Simon Last: And I mean,

[00:01:24] swyx: you’ve been building this since like 20, 22.

[00:01:26] Simon Last: Yeah, I mean, like, it was even right when we got access to like GPT four in late 20 22, 1 of the first ideas we had is like, oh, okay, let’s make an agent that I, we used the word assistant at the time, there wasn’t really the word, the word agent yet, but, oh, we’ll give an access to all the tools the notion can do, and then it, we run in the background like, like do work for us.

And then we just tried that many times and it just. Was too early. Um,

[00:01:48] swyx: I need to force you to like double click on that. What is too early? What didn’t work?

[00:01:52] Sarah Sachs: We were fine to, like, before function calling came out. We were trying to fine tune with the Frontier Labs and with fireworks, like a function calling model on notion functions.

This is right when I joined. I joined because, um, we needed a manager as Simon was needed to be able to go on vacation. So, uh, that’s, that’s around when I joined, so you can speak much more to it.

[00:02:11] Simon Last: Yeah, we did partnerships with both philanthropic and open AI at different times, uh, to try to, at the time the, I mean, when we first tried, there wasn’t even a constant of like tools yet.

We, we sort of designed our own like, like tool calling framework and then we tried to fine tune the models to, uh, to use it over multiple turns. Um, and because it, it didn’t work well out the box, I think. Yeah. The models are just too dumb and the context thing was also way too short.

[00:02:37] Alsesio: Yeah.

[00:02:37] Simon Last: Um, and yeah, we just kind of banged our head against it for a long time.

Uh, unfortunately it was always like, there was always like sort of. Glimmers that it was working, but um, it never felt quite robust enough to be like a useful, delightful thing. Um, until I would say, uh, the big unlock was probably like Sonic 3.6 or seven, uh, early last year. And that’s when we started working on our agent, which we shipped last year.

Um, and then, and then uh, uh, custom agents, kinda a similar capability and that, that one just took longer because we, we just wanted to get the reliability up a lot higher. ‘cause it’s actually running in the background.

[00:03:14] Sarah Sachs: And the product interface of like permissions and understanding, you know, this custom agent is shared in a Slack channel with X group of people and has access to documents that are surfaced to Y group of people.

And the intersect experts, Y might not be whole. And so how do you build the product around making sure administrators understand that permissioning took multiple swings.

[00:03:35] Alsesio: Everything is hard back at the end of the day. Yeah. I’m curious, like when the models are not working, how do you inform the product roadmap of like, okay, we should probably build, expecting the models to be better at some reasonable pace, but at the same time we need to, you know, you had a lot of customers in 2022.

It’s not like you were a new company or like no user base.

[00:03:54] Simon Last: Yeah, I mean I think there’s always the balance of, you know, like you want to be a GI pilled and thinking ahead and building for where things are going. Uh, but also you wanna be like shipping useful things. And so we always try to like, like keep a balance there.

You know, we. We try to take clear, like a portfolio approach. You know, we’re always working on multiple projects and, and we’re always trying to work on, you know, maintaining things where that have already shipped, like, like shipping new things that are like eminently working well and make them really good.

And, and then we wanna always have a few projects that are a little bit crazy. Um,

[00:04:23] Alsesio: and what are the a GI peel projects that you have today? I’m curious about, uh, you don’t have to share exactly what you’re working on, but I’m curious what are things today that maybe in 18 months people will be like, oh, obviously this was gonna work

[00:04:35] Sarah Sachs: 18 months.

[00:04:37] Alsesio: Yeah, 18 months is, you know,

[00:04:37] Sarah Sachs: it’s a long time and Yeah. Yeah.

[00:04:39] Simon Last: I mean, there’s a number of things happening. I think one thing that’s becoming more clear is I think like, like, uh, coding agents are the kernel of EGI, sort of, everything is a coding agent. Mm-hmm. I think that’s, that’s sort of one, one direction.

Um, and then, yeah, the exciting thing about that is sort of your agent can sort of bootstrap its own software and capabilities and actually debug and maintain them. And so yeah, we’re, we’re, we’re thinking a lot about that. And then, yeah, like, like another category of things that I’m, I’m really excited about is like, uh, we call the software factory also.

People are using this, uh, this, this sort of word. Um, basically it just means can you create sort of like a, as automated as possible, a workflow for developing debugging. Mm-hmm. Merging, reviewing, and maintaining a code base and a service where there’s a bunch of agents working together inside, and like, like how does that work?

[00:05:28] Sarah Sachs: If you think back to your initial question, like, why did this take so long? I think something,

[00:05:32] swyx: I didn’t say that, but Yes. Okay. Go ahead.

[00:05:34] Sarah Sachs: Why, what, what changed over the three and half years of trying

[00:05:37] swyx: it? Exactly. Right. Because most people always say like, it didn’t work yet. Then reasoning models came, then it worked.

I was like, okay, let’s go a little

[00:05:43] Sarah Sachs: bit. That’s, I mean, that’s part of it, but I think the other part of it that I actually think is really what will set notion apart for every new capability is we have like. Two skills that are crucial when it comes to frontier capabilities. One is not letting yourself swim upstream.

So like quickly realizing if you’re just pressing against model capabilities versus not exposing the model to the right information, not having the right infrastructure set up. That and of itself is the skill of intuition. And the second is to see, okay, you’re not swimming upstream. Which direction is the river flowing and what is like, how do we think ahead about the product and start building it even if it’s not great yet, so that when it is there, we’re ready for it.

Right? And like those can sometimes feel like counterintuitive things. Like we can be trying to fine tune a tool calling model when they don’t exist yet. And that the trick is to not do that for too long, but realize that there was something there. And we’ve had a lot of things which like, um, we’re just like not swimming in the right direction with the streams.

I think we had multiple versions of transcription before we got meeting notes, right? Oh, I gotta talk

[00:06:39] swyx: about that. Yeah.

[00:06:40] Sarah Sachs: Yeah. Um, and so. I, I, I think that like we, we really closely partner with the Frontier Labs on capabilities and we also have to have strong conviction on, as those capabilities move.

Notion is about being the best place for you to collaborate and do your work. And how does that narrative change if the way that we work changes?

Yeah.

[00:06:58] swyx: Yeah. You told me you were a fan of the Agent Lab thesis, and this is, this is kind of it, right?

[00:07:02] Sarah Sachs: Right. I show that thesis to so many candidates. Like I have it as like micro chrome autofill.

Um, at this point, like it’s one of my most visitations

[00:07:10] swyx: because like, is this the, here’s why you should work in notion and not open, open eye. I, it’s like,

[00:07:14] Sarah Sachs: here’s, here’s what’s different about it.

[00:07:16] swyx: Yeah.

[00:07:16] Sarah Sachs: And here’s why. It’s not just a rapper. I actually think more and more people understand it’s not just a wrapper.

[00:07:21] swyx: Yeah.

[00:07:22] Sarah Sachs: Um, and by the way, like in the beginning, parts of what we build are wrappers on functionality. That works well, of course, but that’s not really the most, um. I would say that’s not the product that, that drives revenue. And that’s not necessarily always what users need.

[00:07:35] swyx: I mean, you know, notion is the AWS wrapper, but like the, the wrapper is very beautiful and like very, very well polished.

So

[00:07:40] Sarah Sachs: like the analogy,

[00:07:41] swyx: like

[00:07:42] Sarah Sachs: the analogy that I’ve been coming back to his Datadog in AWS

[00:07:45] swyx: Yeah.

[00:07:46] Sarah Sachs: So, uh, Datadog could not exist with, without cloud storage. Right. That it’s kind of fundamental that that works. Um, and AWS has like a CloudWatch product, but Datadog is an expert on understanding how people want observability on the products they launch.

And we’re experts in understanding how people wanna collaborate, and that’s really where our expertise lies.

[00:08:04] swyx: Totally.

[00:08:04] Sarah Sachs: Um, regardless of the tools that we use,

[00:08:07] Alsesio: I’m kind of curious how you think about implicit versus explicit expertise. I feel like Datadog is half and half implicit and explicit. It’s like they understand across markets and industries what engineering teams usually look for.

With notion, it’s almost like more of the expertise is at the edge because you as a platform, you’re like so horizontal that the end user is not really the same. Mm-hmm. Like with Datadog, the end user is always like, yeah, an engineering lead, a kinda like SRE related person with notion. It can be anything.

So I’m curious how you put that expertise into a product versus, you know, obviously it, WS cannot build notion. It’s, that doesn’t quite work in this case, but

[00:08:44] Simon Last: it’s, it’s a little bit differently shaped. I think, you know, a classic vertical SaaS, like the data is kind of like that. They understand their individual customer very deeply.

It’s kinda a narrow slice, um, notion has always been super horizontal. And our, our task has always been to sort of balance these two somewhat opposing forces of like, we’re listening to our customers and what they want us to build. It’s a broad slice. And then also we’re thinking about like, okay, how do we decompose what they want into, uh, nice primitives that are, that are really nice to use and we’ll, we’ll get us like as much bang for the buck as possible.

And then, you know. Maintain the whole system, make it all like, like super clean and nice to use.

[00:09:22] Sarah Sachs: We still have user journeys. I mean, we still focus on like core. I actually think the failure of our team is when we focus too much on what are cools that are, what are tools that are

[00:09:31] Simon Last: mm-hmm.

[00:09:31] Sarah Sachs: Cool tools. I actually think that’s when we make have the least velocity because you still need some sort of focus on a user journey.

So like for instance, we’ll all sit down every Friday and look at the P 99 of like the most token exhaustive custom agent transcript and just look at why it didn’t do well and cut a bunch of tasks. Like we still focus on like, this has, like this should work. Email triaging should work. Mm-hmm. Right. And similarly, like when we’re talking about before building, um, chatting, um, before we started filming about, okay, how can I do PDF export?

Well that’s functionality that then merits. Maybe we should build a tool that has access to a computer sandbox in a file system and the ability to write code. Right? Right. Um, but it’s because we’re thinking about the fact that our users to do their, to do their daily work, need to export PDFs, not because we’re like, Hmm, I think a computer tool could be cool.

Like, let’s just see what happens. Mm-hmm. Like we, we have to focus on some user journeys, otherwise we just don’t have like, enough strategy to, to prioritize.

[00:10:29] swyx: I think there’s a lot of like really strong opinions that you’ve had. Do you have like sort of like a towel of Sarah Sachs? Like, you know, like what, how do you run your team?

Like I feel like you just have accumulated all these strong opinions. Obviously part, part of this is your, your token town thing.

[00:10:43] Sarah Sachs: I think the TAs working with Service X is, um, you’d have to, it depends who you ask. Um, I think it depends if you’re on my team or a partner Right. Or a vendor.

[00:10:54] swyx: Yeah. There other people want to run their teams the way that you’re Yeah.

You’re like bringing these things. And then also similarly, uh, Simon, when you did the custom agents demo, you had like, well, we’ve been using custom agents and here’s the super long list of everything that we do. No humans ever read it. Right? That’s what you said. I was like,

[00:11:07] Sarah Sachs: yeah. So I think for, for me, um, something that I learned very quickly and became very comfortable with was that my job was not to be the ideas per person or the technical expert.

My job was to make it so that everybody understood the objective, had a resource to help prioritize what they should work on, and had an avenue to prioritize what they thought was important. And I think that’s true with all, all leadership, but I think especially on the AI team. Almost all of our best ideas come from prototypes, from people that have a cool idea because they saw a user problem, and it’s a huge disservice if all of those ideas have to pass, like the sniff test of what me and a product partner or Simon and Ivan decided were the direction, right?

Because a lot of what we’re doing is leaning into capabilities, so. I think that’s the first thing is like, I don’t really view like the role of engineering leadership as like, uh, hierarchical, nor has it ever been, but especially now, like very willing to change direction based on, um, like proof is in the pudding.

Yeah. And like, and I think we have rebuilt our harness three or four times. And when you do that, then the second rule of engineering leadership is like you need to build a team that’s comfortable deleting their own code and is very low ego and is driven by what’s best for the company. And, um, doesn’t write design docs because they think it’s their promotion packet.

Right. And that’s a culture that notion had long before I joined, but like our willingness to just swarm on different problems and um, redo things that we’ve built before because something has changed. Like, there’s a lot of friction that can happen at companies when you do that. And it doesn’t happen at Notion.

And because it doesn’t happen when new people join. Like they don’t wanna be the ones that are saying, we shouldn’t do this. I wrote that code. So then it’s, you know, you, you create a culture that everyone thoughts and that culture comes directly, I think from Simon and Ivan though, um, because they’re very open-minded.

[00:12:50] swyx: Anything that you,

[00:12:50] Simon Last: you’d add? I’m not a manager, like, like, like Sarah is. Um, a lot of my role is really to try to think a little bit ahead, make sure that we’re, we’re building on the right capabilities and then like the prototyping stuff. And yeah, it’s really, really critical to always just be starting again.

It’s like, okay, this is new thing. What does this mean? What if we just rethought everything or wrote everything? And so I, I’m, I’m basically just doing that in a loop every six months.

[00:13:16] swyx: Yeah. Do you believe in internal hackathons for this stuff?

[00:13:19] Sarah Sachs: I think there’s like two different versions. So one is like, we just have a, a, a solid bench of senior engineers that come and go on what we call the Simon Vortex and Productionizing what we built, right?

Because when you’re in the Simon Vortex, the velocity is super high. The direction changes daily, and it’s meant to be like the equivalent of a SC Works lab. We don’t need to do hackathons for that. We need to have senior engineers that we trust to come in and out of those projects. For instance, like management boundaries are really loose.

Like you report to him, but you work for her right now. Yeah. That’s something that when we hire managers, it’s important they don’t care about because we tend to form more structures. Yeah. Don’t be too

[00:13:54] swyx: territorial.

[00:13:55] Sarah Sachs: We form more. It’s after we ship things, not not before, just historically. Um, the second thing is we do have companywide hackathons.

Actually we just had our demos day for the hackathon we had last week this morning. That’s more for people that aren’t directly working on the project, feeling like they have the time to pause and learn how to make themselves more productive or how they would use notion custom agents to build something.

Or part of the hackathon was actually encouraging everyone across the company to build their own agentic tool loop, calling from scratch. Follow like an every blog post on how to do what I think because we want

[00:14:26] swyx: just with the compound engineering one. Yeah.

[00:14:28] Sarah Sachs: We want everyone to use cloud code in the company or whatever the coding agent they please and understand that fundamental.

So we set aside a day and a half. We’re all leadership, encourage everyone on their teams across the company to do it. So we have hackathons like that. I would say like kind of facetiously, like everything we build is a little bit like a hackathon until it graduates and puts on big boy pants and as a product ops rollout leader and has a assigned data scientists and stuff like that,

[00:14:54] swyx: security review enterprise stuff,

[00:14:56] Sarah Sachs: actually security reviews one of the things that we bring in first because it just slows us down way more and, um, causes a lot of tension and they build better product if they’re involved early.

So, um, that is probably the first person to get involved in something that’s the

[00:15:09] swyx: right PR approved answer.

[00:15:10] Sarah Sachs: No, but it’s not just PR approved. It like, um, um, it’s

[00:15:13] swyx: actually real. It’s actually real. It’s like, um, I’m just saying scar

[00:15:15] Sarah Sachs: tissue.

[00:15:15] swyx: Yeah,

[00:15:16] Sarah Sachs: because like, you know, my background’s also, I worked at Robinhood for a number of years.

Yes. So like, uh, compliance and things like that, um, are a little bit more, you learn the hard way when it doesn’t come naturally.

[00:15:26] Simon Last: Yeah. I think the. The hackathon is really important for uplifting the general population, but like, if that’s the only way you can build new things, you’re kind of toast. I mean, it, it has to be like the daily processes, like, you know, building these new things.

Um, and it has to be about, I think like, I think in the AI era a lot more leverage accumulates to the most curious and excited people. And so it’s like we’re all about just like activating that energy. You know, like if someone’s protesting something on the weekend that they’re excited about and it’s important, that should be the main thing that we’re doing.

Yeah. Um, it’s not a hackathon that we schedule once a quarter, it’s just like, yeah. Daily process. Part of the culture.

[00:16:02] Sarah Sachs: I mean, that’s how we shift image generation and notion now. It was always this thing that would be kind of nice to have, but it wasn’t really clear where that was necessarily aligned in product priorities.

It’d be a lot of work. And we had someone on the database collections team, Jimmy, who was like. I really wanna do image generation for cover photos and inside notion. And we’re like, if you wanna build it, like it’s, do it please. Like we encourage you. We gave ‘em all the resources of working directly with Gemini and being able to like track the token usage and it working through endpoints.

We gave them eval, support, everything, and then became a, a full project.

[00:16:34] Alsesio: Yeah.

[00:16:35] Sarah Sachs: That’s why you can’t have like ego as a, a leader. Like that’s, that’s how we work.

[00:16:39] Alsesio: What’s the size of the team today, both engineering and overall?

[00:16:43] Sarah Sachs: I manage, uh, the team. That’s what we’ll call it. Core AI capabilities and infrastructure.

That’s about 50 people. But then we have per i partner teams that do packaging. So how it shows up in the corner chat versus custom agents versus meeting notes, that’s another 30, 40 people. And, and then every team that has a product service at Notion that a user can interface with owns the tool that the agent interfaces with the editor team.

The team that did CRDT for offline mode is the same team that handles how two agents, um, edit competing blocks. Mm-hmm. Right? It’s the same problem. The team that built the underlying SQL engine is the same team that owns how the agent asks it to run a SQL query, and it does it performantly. And so from that regard, anyone working on product engineering is tasked with making them work for customers that are humans and agents because over time the majority of our traffic will be coming from agencies using in our interface, not humans.

And so. Our objective is to make it so that the whole product org is building for agents.

[00:17:40] Alsesio: Yeah. How has it changed internally? The activation bar is kind of lowered a lot. Like anybody can kind of create a prototype very, somewhat easily, especially if you’re like an existing code base. Have you raised the bar on like what type of prototype people need to bring forward to gonna be taken?

Not like seriously, but like, you know what I

[00:17:58] Simon Last: mean? Yeah. I think the bar is lowered in many ways. Be like, one thing our, uh, our team built that is really cool is our, uh, our, our design team made a whole separate GitHub repo, uh, called the, the design Playground. And it’s basically just to create a bunch of like, like helper components and you, uh, for, for quickly a throwing together UIs.

And it’s become like actually quite sophisticated. Like it has like an agent in there and like, uh, that’s pretty fun. So like, we pretty much, like, they don’t do mocks, they just make like, like full, full prototypes.

[00:18:27] swyx: Here it is. It works.

[00:18:28] Simon Last: They give you like a u rl. They’re like, okay, all right. So we have to make the, like the real production version of that.

Um, and then for engineers. A prototype looks like just making it a feature flag that actually works. Like that’s sort of the bar.

[00:18:39] Sarah Sachs: Something to understand that’s really unique about notion. One of the reasons I joined we’re super lucky is no one uses Notion in their job as much as people that work at Notion.

[00:18:46] Simon Last: Of course.

[00:18:47] Sarah Sachs: So I think there’s very few companies, maybe if you worked on Chrome I guess, but like everything that we ship, we ship internally first and get a lot of really quick feedback. And also sometimes our dev instance is totally borked and you have to change a bunch of flags to get things done. And that’s kind of like, but everyone, so people that do it ticketing, people that do supply chain procurement, recruiting, everyone is using the same instance of notion with like a lot of flags on for these prototypes people build.

Um, and so we have this, Brian Levin, one of the designers on our team, I think evangelize this concept of demos over memos.

[00:19:18] swyx: Ooh, too

[00:19:20] Sarah Sachs: good. Um, which has been, uh, very good for building demos, and I think it’s put a big pressure point on us to have really strong product conviction, because if anything can be demoed, you really need a strong filter of making sure that if you know, you’re doing X amount of work, you’re making the, you’re, you’re focusing on one tower, you’re not just building a really flat hill.

Right. That’s actually where I think there has to be more conviction from our PMs, um, and our designers and, and well, the company really to have conviction of what journey we’re going on.

[00:19:52] Simon Last: But overall, I feel like it works pretty well. Like people, almost all the engineers have good enough taste to realize that like, this prototype doesn’t actually make sense in the product, or, or it does.

So it’s not that common that I would see a prototype. It’s like, oh, this makes no sense. Mm-hmm. It’s like, you know, people are doing reasonable things and, and, and then it’s just a matter of. Which things we build first and then often just, just figuring out how to turn it on and off. There’s our, in the, in our like experimental chat ui, there’s this, there’s probably like, like a hundred check boxes in there.

[00:20:22] Sarah Sachs: Kills me

[00:20:23] Simon Last: the things you could turn on and off.

[00:20:25] Sarah Sachs: Uh, but I think that, okay, so that is kind of true, Simon, but like being the person that manages the evals team, like there is a level of intensity that it adds to the platform team. So, you know, if we’re gonna do image generation and notion, all of a sudden the way that we do attachments and the way that we, um, our LLM completion like cortex talks and expects tokens back and now it’s getting images back.

Like there’s a lot of platform work that we do need to, like solidify a little bit. So sometimes it’ll be in dev for a couple weeks before it makes it to prod just because we still have to like, make it robust, make it HIPAA compliant, ZDR compliant, figure out the right contracting with the vendor, whatever it is.

And we need to eval it because we want the team. To still maintain what they build. That’s the one thing is like if we have a bunch of prototypes, it can’t just be like a small group of people that then maintain whatever end prototypes. So we have invested a lot of people in an eval and model behavior understanding teams that, we call it agent dev velocity.

So your dev velocity building agents can be faster if we invest in that platform. And so we have a whole org dedicated to Asian, um, platform velocity so that you can build your own eval and then maintain it once you ship it. So if a new model release comes out and we, every

[00:21:38] swyx: team maintains their own eval,

[00:21:40] Sarah Sachs: we maintain the eval framework.

Every team owns their own evals and a lot of them we’ve integrated to Optin, to ci, or we run them nightly and we have a team, uh, a custom agent that triggers to a team to look at the major failures. That’s really critical because if we have like all these different surfaces now, a lot of it’s on the same agent harness, so it’s easier to maintain.

It’s just packaging of different agent harnesses, but new functionality of the agent. Let’s say that like we wanna update like. Uh, you know, they deprecated, sonnet, um, four or whatever it is and we need to auto update. Are

[00:22:11] swyx: they already? That’s so, okay. Yeah. Actually wasn’t that long ago.

[00:22:14] Alsesio: They

were

[00:22:14] Alsesio: just 3.5.

[00:22:15] Sarah Sachs: 3.537. Just got deprecated.

[00:22:18] swyx: 3 7, 5 0.2 or, yeah. No,

[00:22:20] Sarah Sachs: it’s not. 5.2 is five point. Five point no. Yeah, five four is 40% more expensive than five two. So if they deprecated five two, you would hear they can, you would hear from me about that one. Um, but, uh, another conversation to have.

[00:22:35] swyx: I have a cheeky evals question for you.

Have you noticed any secret degradation from any of the major model providers?

[00:22:40] Sarah Sachs: Secret degradation,

[00:22:42] swyx: like. During the War Bay, when it’s high traffic, it suddenly gets dumber.

[00:22:47] Sarah Sachs: Yeah. I mean, not just between the, I mean, we definitely notice flakiness, we’ve definitely noticed, particularly for some providers, that things are slower during working hours and

[00:22:57] swyx: there’s a latency argument.

Yes. Not a quality argument.

[00:22:59] Sarah Sachs: No. I think the quality difference that’s interesting is, um, even though companies that say they’re selling the same, a, it’s really into like quanti quantization, but like companies that say they’re selling the same model through different vendors, whether it be through first party or Bedrock, Azure, et cetera.

We do see different qualities sometimes, and that’s not necessarily what’s advertised.

[00:23:21] swyx: Yeah. Kidney went to the point of like, if we, they shipped like this, like eval across all the providers and it was like very obvious we were secret equalizing and it was very,

[00:23:28] Sarah Sachs: yeah. But

[00:23:29] swyx: that’s very embarrassing.

[00:23:30] Sarah Sachs: You know, um, we hire Subprocess to figure that out for us.

So we just wanna understand where it’s regressing or where it’s optimized. And sometimes we’re okay with regressions that optimize latency if they’re the appropriate regressions. Our job is to make sure we have the evals to understand the changes that are important to us. And even like when we’re partnering with labs on pre-releasees of models, they’ll send us multiple snapshots.

And this is less about quantization, but more just regressions. Like they have shipped models that were not the snapshots that we wanted, and they have changed the snapshots that they shipped based on the feedback that we give. Because our feedback tends to be more enterprise work focused and not coding agent focused.

And definitely those can be bummers, like, you know, uh, we know that this wasn’t the version you wanted, but we’ll help you make it work. I mean, we always make it work, but that definitely happens.

[00:24:16] Alsesio: Yeah. Do you have, um, failing evals that you’re just hoping, oh, that will have success eventually when a good model comes out?

[00:24:23] Sarah Sachs: Uh, I mean, yeah. So I think. I mean, I could talk about this for 60 minutes, so I will limit myself. I think it’s a real issue when people say evals and it’s just like, that’s quality, that’s like unit, I mean, it’s like saying testing. It’s not just unit tests, right? So. We have the equivalent of unit test.

Regression test. Those live in ci, those have to pass a certain percent, you know, within some stochastic error rate. Then we have, as you’re building a product, evals of these aren’t passing right now, and this is launch quality. So we have a report card and we need to, on these categories, you know, be it 80 or 90% of all of these user journeys to launch, and then what we have what we call frontier or headroom evals, where we actively wanna be at 30% pass rate.

And that’s actually been a effort that we took in partnership with philanthropic and OpenAI in the past maybe two or three months, because we actually hit a point where our evals were saturated and we weren’t able to really give insightful feedback other than it wasn’t worse. And not only is that not helpful for our partners, it’s not helpful for us to understand where the stream is going.

You know, going back to that analogy. And so we spent a lot of time thinking about. What notions last exam looks like, right? Mm-hmm. Not just humanities, last exam. Ooh, notions last exam. Mm-hmm. And, um, there’s a lot of, you know, dreams about what that would look like. I know we’ve talked a lot about benchmarking, um, swix, but, uh, yeah.

Notions last exam is a big thing inside the company and we have people, full-time staff to it exclusively. Mm. We have a data scientist, a model behavior engineer, and an full-time, um, evals engineer just dedicated to the evals that we pass 30% of the time.

[00:25:56] swyx: What you’re hiring for

[00:25:57] Sarah Sachs: MBEs? I am hiring

[00:25:58] swyx: What is an MBEA

[00:25:59] Sarah Sachs: model?

Behavior Engineer Model. Behavior engineers started with a title data specialist before I joined when they were working with Simon on like, uh, Google Sheets and like Simon just needed someone to look through Google Sheets and say, yes, no, this looks bad. This looks good. Right? And so we hired people with kind of diverse linguistics background.

We had like a linguistics PhD dropout. Mm-hmm. And a Stanford ate new grad. And they’re amazing. And they formed a new function basically. And over time we’ve built a whole team, um, with a manager who’s now kind of reinventing what that role is with coding agents. So they used to be kind of manually inspecting code.

Now they’re primarily building agents that can write evals for themselves or LLM judges. There’s a really funny day I can send you the picture where Simon, about a year and a half ago, was teaching them how to use GitHub. Um, and they’re on the whiteboard and it was like, okay, I think it would be so much faster if our data specialists learned how to use GitHub and like learned how to commit these things in Dakota.

And, and that was then and now I think, you know, coding has been a lot more accessible. Um, but moving forward it’s this mix of like data scientist PM and prompt engineer because there’s craft in understanding like even like what models can and can’t do things. How do we define like that headroom? How do we define like what a good journey is?

Um, is this model better or not? Why is this failing? There’s some qualitative work, but then there’s also like a lot of instinct and taste to it, and that’s not necessarily software engineering. And so we have like very firm conviction and we have had for a number of years now that that is its own career path and we have always welcomed the misfits, so to speak.

So we really firmly believe that you don’t need an engineering background to be the best at this job. And that’s what’s quite unique about this particular role.

[00:27:37] Simon Last: Yeah, this is something that I’ve been pretty excited about recently is we made an effort basically to treat the eval system as like an agent harness.

So if you think about it, like, you know, you should be able to have an agent end-to-end, download a dataset, run an eval, iterate on a failure, debug, and, and then implement a fix. And ultimately you should be able to, you know, drive the full time process with a human sort of observing the, you know, the outer uh, system.

So yeah, we went, went pretty hard on that. And that’s, that’s worked extremely well so far. It’s like basically just to turn it into a coding agent, uh, uh, problem.

[00:28:11] swyx: Your coding agent or just whatever

[00:28:13] Simon Last: harness No coding agent. Yeah, code, cloud code. It should be totally general. Yeah. I think if it would be a mistake to like, like fix it on any, any particular coding agent.

At the end of the day, it’s just like CLI tools.

[00:28:21] Sarah Sachs: It’s like the same way that you would’ve a coding agent write the unit test. You should have a coding agent write the eval.

[00:28:26] swyx: Yeah.

[00:28:26] Sarah Sachs: But there’s a lot of supervision in that still. We just don’t believe that supervision has to come from software engineers because a lot of it is like, um, kind of you XREE and whatever, and these are the people that also triage failures and tell us where we should be investing next.

[00:28:40] swyx: Yeah. I’m gonna go ahead and ask a spicy question. Is there a data, there are no software engineers at Notion.

[00:28:46] Simon Last: Um,

[00:28:46] Sarah Sachs: what does it mean to be a software engineer?

[00:28:47] swyx: Exactly.

[00:28:48] Simon Last: I mean, I think the way things are going is like we’re on some continuum where. If, if you look back three years ago, humans were typing all the code and then we had auto complete, you’re typing list of the code.

Then we had sort of like filling agents, filling lines, and now we’re getting into like agents doing longer range tasks where you can debug and implement a fix and then verify it works and you know, get your, get your PR even like, like Merion deployed. I think we’re sort of just moving up the abstraction ladder and then the human role becomes more about observing and maintaining the outer system.

There’s a string of agents flowing through, like me prs what’s going off the rails. Like what do I need to approve? Is there like a learning or memory mechanism that that works? So it’s kind of a hard engineering problem. There’s a, you know, there’s, there’s a lot to do there. I think we’re just sort of moving up stack

[00:29:34] Sarah Sachs: the same transition machine learning engineers have made, right?

Like I haven’t looked at a PR curve in a while.

[00:29:39] swyx: Yeah. You used to do this stuff and now, um, auto research can do it,

[00:29:42] Sarah Sachs: right? Like I think it depends on what you define as a software engineer.

[00:29:46] swyx: Yes. It’s, that’s changing for sure.

[00:29:49] Sarah Sachs: I think every software engineer in notion this summer went through like this, um, sheer, um, one of our engineering leads of the company called it, like every software engineer is going through the, the, uh, identity crisis that every manager goes through, where all of a sudden they realize their ability to write code is less important than their ability to delegate in context switch.

And I think that is a transition out of being a software engineer. But

[00:30:12] Simon Last: yeah. Yeah, there’s a critical difference to being a manager, which is that like, it is actually very deeply technical. The problem, you know, humans are very like, like, like fuzzy and you can’t like treat a team of humans like a, like a rigorous system where like, you know, prs like, like flow through and can be in like a block status and then what happens when they’re blocked, right.

With a set of agents, you actually can do that. And, and, and I think it’s actually, there’s a lot of interesting technical rigor that that goes into that it’s like it’s a technical design problem. Ultimately.

[00:30:42] Alsesio: What is the design of the software factory that you’re building?

[00:30:46] Simon Last: Yeah, I mean, I think we’re. Trying a lot of different things.

I mean, ultimately you want to design a system that requires as little human intervention as possible, but like still maintaining the in variance that, that you care about. So yeah, we’re exploring a lot different ideas there. I mean, I think I could talk about a few things I think are important there.

Like, one thing I think is really important is, um, having some kind of like specification layer you can just commit marked on files. Mm-hmm. That works pretty well, but

[00:31:15] swyx: it’s nice to be notion man. I’m just saying like the spec, like Yeah. The natural home for specs is notion.

[00:31:21] Simon Last: Yeah. Right. It can be a database of pages.

Yeah. I mean, it needs to be something that is, you know, human readable and I viewable and I think that’s pretty key. Another really key component is like the, the self verification loop. Yes. You need really, really good testing layers, basically. And that’s a really deep, uh, uh, problem. But by getting that right, you know, and then, and then it’s kinda like the workflow of like.

What happens when there’s a bug? How does it flow into the system? Like, is it like a subagent working on it? How does it make a PR and how does that get reviewed? And me, and then, you know, so there’s like the, the flow or process.

[00:31:56] swyx: Yeah. Cool. Uh, you know, one thing we did work out before you guys came in was this demo or this

[00:32:01] Simon Last: agents

[00:32:02] swyx: agent demo.

Uh,

[00:32:03] Simon Last: so every,

[00:32:04] Alsesio: every time we do an episode, we try the product. Right. I don’t think there’s ever been an episode that I haven’t tried. Yeah. Um,

[00:32:11] swyx: and we, we try, try is a, a big word. Like since day one lane space has been on Notion, but this is the, this is the net new thing. Yes.

[00:32:18] Alsesio: So this is for Nel Labs, which is the space we’re in.

So next week we’re opening applications for tenants. So there’s a web form, let me, we got this form done here. Uh, so, uh, before. Uh, the workflow would be I get an email, then I look at the person. It was like, should I spend time talking to this person? Then I respond, they respond back. So I build this. So the name it came up for on its own.

Can you maybe h how do, how does it come up with its own name?

[00:32:43] Simon Last: Yeah, that’s a pretty app name. It’s, it, it is just a random, it’s a random, a name generator.

[00:32:47] Alsesio: Oh, that’s funny. It just came,

[00:32:49] Simon Last: the fact that it picked that is, is kind of hilarious. I’m pretty sure it’s just determined,

[00:32:54] Sarah Sachs: resilient collector. I, I think I’ve never looked at the code for that.

I’ve never second guessed it. I think it’s kind of like a madlib situation.

[00:33:00] Simon Last: Yeah, I think you’re right. Yeah. It’s, it’s totally a, a deterministic. Oh, I thought it was great. Yes. Although, although when the, if you use the AI to set itself up, it can update its own name, so. Okay. Um,

[00:33:11] Sarah Sachs: how did you create it? It, did you just do

[00:33:12] Alsesio: classroom?

I,

[00:33:13] Sarah Sachs: okay.

[00:33:13] Alsesio: I did, yeah. I’ll say just check my inbox for applications for a coworking space. Keep a people, so it created the database for me. Which I have here. And I guess database is like an notion table because everything is notion. Um, and then whenever um, an email comes in, like here, it just creates a new role for the person.

Mm-hmm. And then it uses web search to enrich the mm-hmm. The profile. So it kind of like searches the web and it’s like, this is who this person is, this is when they say they wanna move in and kind of updates everything else. This is, I mean, it’s not a GI, but to me, I don’t wanna do this work. So it feels like, I mean, it took me maybe like 15 minutes to set up the whole thing.

Um, and I really like that most of the information should live here. You know, it is not like some other tool asking me

[00:34:01] Sarah Sachs: Yeah.

[00:34:01] Alsesio: To like, bring my stuff there. It’s like I would’ve probably already created an ocean thing.

[00:34:06] Sarah Sachs: Mm-hmm.

[00:34:06] Alsesio: So

[00:34:07] Sarah Sachs: most of our biggest use cases and gains are from. That extra layer of human involvement in the process to make it so right.

And so like one of our biggest use cases is bug triaging. So if someone posts something in Slack, can you just have a custom agent that lives there that has its own routing constitution of what team this belongs to, creates a task in your task database and then posts in that Slack channel, right? Like that’s like one of the first things that we built internally, I think.

And it’s completely changed the way that notion functions as a company. Nothing falls through, well, most things don’t fall through the crack. We don’t know what we don’t know. But it’s not replacing people, it’s replacing processes.

[00:34:44] Alsesio: Yeah.

[00:34:44] Sarah Sachs: Right.

[00:34:45] Alsesio: And I’m curious how you think about composability of these things.

So the other one I was working on is like a. These filler. So whenever somebody signs up as a tenant, kind of he’ll sell the lease for them. There should probably some agent that is like office manager agent mm-hmm. That can handle the request, make the lease, and then, uh, give them a ADA access to the office and all of that.

How do you think about that feature?

[00:35:08] Simon Last: Yeah, so I mean, there’s, there’s two ways you can compose. One way is by using like the data primitives. So you can, you know, you, you could give, you have one agent, uh, be writing to the database and there’s another agent that’s walked in the database. So that’s, that’s one way that they, they can coordinate that’s like a little bit more decoupled and mm-hmm.

Works really well. Or you, you can couple them. So I, I think it’s actually not released yet. Releasing it like next week is, uh, in the settings for an agent, you can give access to invoke any other agent.

[00:35:34] swyx: Hmm.

[00:35:34] Simon Last: So you can have them just. Just, uh, uh, talk directly. So

[00:35:37] swyx: you, was there a limit on like, number of recursions or just,

[00:35:40] Simon Last: um, probably,

[00:35:42] swyx: you know what I mean?

Like, you can just get an infinite loop that way there’s

[00:35:45] Simon Last: some kind of Yeah,

[00:35:46] Sarah Sachs: I think it’s, there is actually a number somewhere.

[00:35:49] swyx: I believe I’m just, you know, like, you’re, you’re, someone’s gonna screw up. You

[00:35:51] Simon Last: should you try to see

[00:35:53] swyx: Yeah. I mean, everything’s gonna be paperclips.

[00:35:55] Simon Last: Oh, yeah. Yeah. But, uh, but, but that’s really useful.

Yeah. So we, you know, like I just, I, I helped, uh, someone internally the other day, they had, they had built like over 30 custom agents for, uh, for our go to market team doing all kinds of different things. You know, for example, like researching, you know, like, like filling information about, about a customer or like, like triaging customer feedback or like, uh, something like that.

Literally over 30 of them. And, and then he, and then he even made like a database of all the agents and then he is like, okay, and, and now I’m getting 70, over 70 notifications per day with just the agents are blocked on various things. Uh, and then I was like, oh, okay, cool. You know, the obvious thing to do there is to make a manager agent,

[00:36:32] Sarah Sachs: right?

[00:36:33] Simon Last: That’s gonna sort of blocks be another abstraction layer in between your, your, uh, uh, 30 agents. Uh, so yeah, we, we send out with like a manager agent and then has access to invoke all the other agents and it’s sort of like, like watching and observing them and then it sort of, it just creates a layer of abstraction.

So instead of 70 notifications per day, it’s like, like five. And then, and then the manager agent can help like, uh, debug and fix any problems with the,

[00:36:54] swyx: does this is a concept of like an inbox or something like piece, you’re basically saying that they can message each other?

[00:37:00] Simon Last: Yeah.

[00:37:01] Sarah Sachs: Well

[00:37:01] swyx: they use the system of record, which, which is

[00:37:02] Sarah Sachs: notion, so we

[00:37:03] Simon Last: actually, yeah, we didn’t make any special concepts at all.

[00:37:06] swyx: They’re interested to the motion notifications that I would’ve got,

[00:37:09] Sarah Sachs: they can just like write a task to a database that the other agent’s task to listening to, or they can actually call a web book to the agent, like they can just add the agent. Okay.

[00:37:17] Simon Last: Yeah, I mean, this is something that, that we’re still working on.

I, I think we, you know, like, like generally, generally the way we do these things is, you know, you first make it possible, maybe like a sort of janky way. So I, I, I think the way I set ‘em up is like, you know, we created like a new database that was sort of like issues mm-hmm. That the custom agents were, were experiencing, and then gave them all access to file an issue and then the manager has access to, to read the issues.

Um, and that works pretty well, essentially like, like give it its own like internal issue tracker just for the agents. And then, you know, if that becomes a, a concept that seems useful, generally maybe we will think of how to package it in. But I mean, generally we try to just keep it to composing the primitive if we can.

You know, another example of this is we have no built-in memory concept. Memory is, is just pages and databases. And so if you wanna give a memory, just give it a page and give it. Edit access to that page and the

[00:38:03] swyx: human can edit it. Agent can edit

[00:38:04] Simon Last: it. Yeah. And so that works, that pattern works extremely well on it.

And you know, depending this case, you can have it be just a page or it could be an entire database with, you know, or, you know, I can have sub pages is is pretty on what you can do with that.

[00:38:15] Alsesio: So when I was setting this up, uh, I connected my inbox and it was like, do you wanna use Gmail or Notion Mail? And I’m like, I don’t wanna use Eater, I just want you to do it.

I’m curious how you think about, you know, notion, mail, notion, calendar, all of these kind of ui ux interfaces, full stack

[00:38:29] Simon Last: notion.

[00:38:30] Alsesio: Yeah. When like at the same time you have the agents abstracting them away from you in a way, you know, how do you spend like the product calories so to speak?

[00:38:37] Simon Last: Yeah, I mean, I think it’s pretty important that you don’t have to use, not your mail to connect to the mail capability.

So we can just connect to Gmail or, or whatever you want, uh, to use. And we’re thinking of the mail service as being really great to the extent that it’s really agent built, right? So maybe the mail app is just sort of a prepackaged agent that helps you automate your, your inbox.

[00:39:00] Alsesio: Yeah, the auto labeling is great.

Think

[00:39:03] Sarah Sachs: the, when we, um, integrate with Gmail for instance, we have a series of tools available that are available via MCP or API to Gmail. When we integrate with Notion Mail, we have the Notion Mail engineering team to build us the, um, exact right tools that optimize latency, optimize performance and quality.

They own that quality. Um, there’s product leads there. They’re directly thinking about the user problems that happen in mail. So it tends to be when we build integrations and connections, we build natively first. Um, and then think about, um, extending them generally just because it’s also easier. Mm-hmm. Um, um, to build natively first.

Um, so that tends to be how we phase things out.

[00:39:43] swyx: Talking about integrations, you prompted me, so I gotta ask. M-C-P-C-L-I. What’s going on? What’s the

[00:39:48] Simon Last: Yeah. Opinion. I think, I mean, I’m, I’m definitely bullish and excited about cli. I think there’s a few really cool things about cli. So one really cool thing is like, um, is that it’s in the terminal environment, so it gets a bunch of extra power.

So it, you know, for example, it can like, like paginating and cursor through like long outputs. Um, and it has a progressive disclosure inherently. Uh, so, you know, you don’t see all the tools at once. It’s just, you see the CLI wrapper and you can like use the, the help commands and, and, and read files. And then I think the most important thing that’s, that’s super cool is that there, it’s also inherently a, a bootstrapped.

So if there’s an issue, uh, the agent can debug and fix itself within the same environment that it uses the tool.

[00:40:30] swyx: Mm.

[00:40:30] Simon Last: Right. Like, you know, I think I saw a tweet this morning. Someone said, you know, my agent didn’t have a browser, so I asked it to make all a browser tool and within a hundred lines of code, it gave itself a little browser, like, like wrapping the, the, the chromium API, um.

That’s pretty incredible. And then if there was a bug, it would just immediately try to fix it. Mm-hmm. Right. On the other hand, if you use an, you know, if you use like of, of the Chrome dev tools, MCP, I’ve had this issue where like, like sometimes the transport gets like messed up. If it gets messed up, the agent has no way to fix itself.

It, it no longer has a browser, it’s, it’s not broken. Right. I think that’s, that’s pretty fundamental, but I would say like a lot of the, the bad things about it can be fixed. Uh, so I think like, as a progressive disclosure, that can be fixed with, with right harness. Like, it, it obviously doesn’t make sense to show it all the tools all the time.

That’s not really inherent to the MCP protocol. It’s just like how you wrap it and use it.

[00:41:16] swyx: There’s many poorly built MCPs because we didn’t know.

[00:41:19] Simon Last: Yeah, yeah. I mean it was just early, like, like the obvious thing is, uh, you know, to start with is, is to just show it all the tools and it’s like, okay, now we have a hundred tools.

Yeah. And like the tool calling actually works. So let’s of

[00:41:28] swyx: your success

[00:41:29] Simon Last: give it a way to like, like filter to source the tools. So yeah, I would say like broadly speaking, I’m really bullish on cli. I’m still bullish on CPS and in a certain environment. I think in, in particular, CP is really great for when you want sort of like a narrow, lightweight agent.

I think there’s, there’s definitely a lot of use cases where, where you don’t want like a full coding agent with a compute run time. And also you want it to be like more tightly permissioned. MCP inherently has a really strong permission model, like all you can do is call the tools. A CLI is a little bit murkier.

It’s like, can I access the, if PI token are you, like, properly sort of like re-encrypt the token so it can’t like exfiltrate it, it introduce a lot of like, like new issues, which are. Real and hard to solve. And MCP is just like the dumb simple thing that works and it that it’s pretty good.

[00:42:12] Sarah Sachs: I’ll add two more perspectives, not from it working well for Notion, but how notion like commits to both platforms.

Notion is dedicated to being the best system of record for where people do their enterprise work. So we will always support our MCP and so far as other people are using cps, right? So regardless of our perspective, we’ve put a lot of effort into our MCP and we have a fantastic team that we’re building, um, to do more there.

And the second thing I’ll say, I think, um, we all think a lot, but lately I’ve been thinking a lot about making sure there’s a value alignment and pricing, um, with capability.

[00:42:43] swyx: Literally our next question

[00:42:44] Sarah Sachs: and. Needing language to execute deterministic tasks feels wasteful and requiring on a language model to interface with third party providers seems wasteful for tasks that don’t require it.

And particularly because our custom agents are using usage-based pricing. We think of pricing as like the barrier of entry for use of our product, and we’re quite committed to making sure that it’s not wasteful. Um, not just because it’s a bad deal for our customers, but it’s also bad business. We wanna have as many buyers, like there’s a, there’s an elasticity of demand and so if we can have our agents properly execute code that calls on CLI deterministically, it’s a one-time cost, right?

Versus constantly having a language model integrate with an MCP over and over and over and paying those like repeated token fees and it’s happening outside the cash window, then you’re paying for it over and over and over and it’s just kind of unnecessary and less deterministic when it doesn’t have to be.

[00:43:36] Alessio: Yeah, the open-endedness I think is like, the main thing is like, well, if I go write code to just call an API, I would never use an MCP. But then you need an NCP sometimes when you know what to call, but you don’t want it to restart versus like, I think the it built a browser from scratch is like, it’s great when you’re doing it on your own, but like if your customers were having your AI write a browser from scratch every time and you had to pay the token cost of that, yeah.

You’d be like, no, no. The Chrome dev tools CP is actually pretty great. Just use that. I’m curious, how do you make that decision? Like should it be. Just straight API call very narrow. Should it be an MCP? Should it be super open-ended?

[00:44:10] Sarah Sachs: Do you mean for when we ship notion capabilities or when we add capabilities to

[00:44:13] Alessio: notion

[00:44:14] Sarah Sachs: AI or,

[00:44:14] Alessio: I mean, you might have a capability that the only way to do is an open-ended agent, like an agent with a coding sandbox.

[00:44:21] Sarah Sachs: Yeah. In Notion ai they’re not explicit, not We also ship an MCP.

[00:44:24] Alsesio: Yeah. Yeah. In B,

[00:44:25] Sarah Sachs: yeah.

[00:44:26] Alsesio: Internally. Okay. Like is there ever a discussion of like, we’re not gonna ship it because we’re not able to tie it down? Or are you happy to just like,

[00:44:33] Sarah Sachs: um, no. I mean, there are a lot of things where we choose not to use MCP because we wanna add more high touch to quality.

I think search an agent to find is like the largest instance of that, where we have. Um, slack and linear and Jira search and notion that is not using necessarily the search MCP functionality that is provided by those companies. And that’s because it’s quite critical we think, to how our agent trajectories work is for us to have a little bit more control on the functionality of the search journey.

And so it usually comes from quality and there’s a long tail of things and that’s why we built an MCP client or an MCP server, excuse me, so that people can connect whatever they want. There’s that long tail, right. But we, for search particularly, I would say that’s like the primary entry point, but there are other connections as well that it’s a little bit of secret sauce about when we are okay with like MCP functionality and user driven off.

And when we actually want to wanna carry a lot more ourselves.

[00:45:31] Simon Last: I think that there’s not really a conflict here. There’s just like different layers of the stack and different abstractions. I mean, if we were to like map it out, it’s like, you know, you’ve got CPS give you a, a way to, it’s a protocol for gaining access to tools.

It’s an open protocol, so you can, you can easily get like a long tail, many things. So if you open up our, like in the tool settings, oh, that’s saw the trigger. Actually, actually, that’s something that MCP can’t do. So if you scroll down and you, and yeah. The, the tools and access, so you’re gonna a connection.

Yeah. MCP is a really great way to gain access to tools or really well, but you just looked at the, the trigger why, for example, there’s no trigger protocol. And so those are things we had to build ourselves. And then there’s, there’s some integrations where we use MCP. Like, so for example, I think the, you know, the linear and the GitHub

mm-hmm.

[00:46:20] Simon Last: Use M ccp, but, but the Slack mail, er, those are actually ones they built in house. And we spent a lot of time really fine tuning all the tools to make the really good and also like building out the triggers. So it’s just like different layers of the stack. Some things make sense sometimes. And then, you know, we just have to like, like harness the right tool at the right time.

I don’t think there’s an inherent like. Strong conflict between these things.

[00:46:40] Alsesio: Do you have a canonical representation of these tools internally where like you wrap these things together, the MCP plus, the custom built?

[00:46:46] Simon Last: Yeah. Yeah. We have like internal abstractions for like what is a tool, what is an agent, what is a completion call?

Yeah.

[00:46:55] Sarah Sachs: We even have internal obstructions for like, what is a chat archetype, whether it be from teams or Slack.

[00:47:02] swyx: Yeah.

[00:47:02] Sarah Sachs: Right.

[00:47:02] swyx: It’s like the only

[00:47:03] Sarah Sachs: way a to

[00:47:03] swyx: build with, with ai ‘cause everything’s moving so quickly, you would have to attract it so that you can swap things up.

[00:47:09] Simon Last: Yeah. I mean, there’s always a dance.

We, we probably rebuilt our, our framework like, like I said, like, like five different times. Um, it’s always a dance of like, okay, how does this new thing work? Right? What should the abstraction be? Like, what is OpenAI giving us? What is that therapy giving us? Um, you know, like we’re trying to wrap over it. I think.

I think we’ve been pretty successful with that. It, it’s just a matter of like, like staying nimble. Yeah. And making sure that you always have like the simplest, dumbest obstruction you can, that you know, that the maps are different things. Yeah. So, so we have like a tool integration abstraction, for example.

And then MCP is like a, a type of integration.

[00:47:41] swyx: Yeah.

[00:47:42] Simon Last: That’s, that’s one of the,

[00:47:43] swyx: this might be a big ask, uh, um, but I’m gonna try, uh, which is, you said, you’ve said multiple times, you rebuild a few times, like five times through, I don’t know if the, what the right number is. Is there like a brief history of what was the each rebuild doing and Yeah, I know it,

[00:47:56] Simon Last: I can try to do that.

I

[00:47:57] swyx: mean,

[00:47:58] Simon Last: yeah, there’s

[00:47:58] swyx: interesting, you need, you need to rag over

[00:48:00] Sarah Sachs: archeology.

[00:48:00] Simon Last: I mean, the first version, the first version that we started building in like late 2022. Oh my gosh. Well, there’ve been many versions actually. Okay. Well the writers, the,

[00:48:08] swyx: I like the highlights. The,

[00:48:09] Simon Last: the

[00:48:10] swyx: like,

[00:48:10] Simon Last: oh

[00:48:10] swyx: wow.

[00:48:10] Simon Last: I mean the, the first version we built was actually a coding agent.

Yeah. So we’re like, oh, instead of building tools, let’s make everything be JavaScript and then we’ll just give it JavaScript APIs and we’ll just write code. And that’s how it speaks to the tools. Um, but at the time. It just sucked at writing code. It wasn’t that good. Uh, so then we moved to, uh, more of like a tool calling obstruction.

A tool calling didn’t exist yet, so we created this whole XML mm-hmm. Of representation. And a big, a big learning in that version is we were catering way too much to what made sense for notion and notions data model versus what the model wants. So as an example, we created this whole, uh, XML, uh, format that can losslessly mapped in notion blocks.

And the transformation between them is super easy to do. Uh, and then we created this sort of like mutation operations to, to add to pages. Um, but it sucked because the model didn’t know the XML format and also the, and you have to prompt it

[00:49:04] swyx: in and

[00:49:04] Simon Last: Yeah, to prompt it in and the team just more convenient.

And so yeah, we’re like, okay, well it has to be marked down. Uh, uh, the model’s no markdown, you know. So, uh, we did a whole project around basically, uh, uh, creating a notion flavored markdown where, uh, you know, the whole goal was like, it has to be just simple markdown at the core, and, and then we can add some enhancements.

And it doesn’t have to be a, a full lossless conversion. That was a big one we did. And, and then we did a whole similar learning to, uh, the, the database layer. So, so to query a database, I mean, in the notion API, the way you query a database is there’s a crazy JSON format and it’s, you know, kind of limiting, but it maps nicely to like how we represent things internally.

We scrapped all that and we’re like, okay, let’s just make it SQL light. Everything is a SQL Light database. You, you can query it just like a SQL light query. And the models are super good at that. So

[00:49:51] swyx: give the models what they want.

[00:49:52] Simon Last: That was another one. Yeah. Yeah. Give us what they want. I mean, that was, I would say that was a big learning is just, you know, really be, be savvy and really careful thinking about what the model wants in terms of, you know, its environment and, and, and cater around that.

And really try so hard not to expose it to any complexity about your system that, that’s unnecessary.

[00:50:12] swyx: Notions underlying database is Postgres, right? Not sql, right? Yeah. So I don’t know if there’s any mismatch there.

[00:50:18] Simon Last: That one was kind of a fortuitous thing because we actually already, um, had a big project, uh, going where, so, so we have this, um, when you query Notion database, it’s actually querying this like, uh, cluster of SQL databases.

[00:50:34] swyx: Mm-hmm.

[00:50:35] Simon Last: That’s something that we’d already been working on even before the agents came around.

[00:50:38] swyx: Yeah. You know, you guys had a fantastic blog post about it and like it’s, it is actually a really good database engineering knowledge to have that from you guys because where else would we get it?

[00:50:47] Simon Last: Yeah, yeah.

It’s a, it’s, it’s a crazy engineering problem when you want to have like millions and billions of tiny databases or where, where some of them are tiny, but some of ‘em are, are very large and want everything to be very fast.

[00:50:57] swyx: Yeah. And also like, not that hierarchical sometimes, you know, uh, so somewhat of a graph.

[00:51:02] Simon Last: Mm-hmm.

[00:51:03] swyx: I do like that history because I think that shows the evolution that you guys went through and the work that went into it,

[00:51:09] Sarah Sachs: that he just ended you a year and a half ago.

[00:51:11] swyx: Oh, okay. Okay. Oh,

[00:51:13] Simon Last: I need to, I need

[00:51:13] swyx: to hit continue.

[00:51:14] Sarah Sachs: If you’re curious. I mean, we can keep going. Just saying like, that’s really,

[00:51:18] Simon Last: that’s another one.

Yeah.

[00:51:19] Sarah Sachs: I lemme think. Well, no. ‘cause there was tool calling and then there was research mode, which wasn’t a fully agentic tool calling. Um, then we moved away from few shot prompting entirely to tool definitions. Um, and now we’re thinking about Agent 2.0.

[00:51:34] swyx: So no fusion prompts ever. Right.

[00:51:35] Sarah Sachs: Uh,

[00:51:36] swyx: okay. No, maybe not.

[00:51:37] Sarah Sachs: I know never, but

[00:51:38] Simon Last: yeah, that kind of went away. It’s an interesting thing,

[00:51:40] swyx: right?

[00:51:41] Simon Last: Yeah. I mean, so

[00:51:41] swyx: these just instruction follow really well,

[00:51:44] Simon Last: I would say if there’s been like a general arc where, you know, it’s like you gradually strip away everything. And it, it looks more a GI like. And so, you know, it it, it started out as like, it’s a one shot, one prompt.

There’s a few shot examples. And it became like, okay, actually let’s give it, let’s give it tools, but it’s still a few shot examples. And then it became actually like, no, no, no, let’s just give it a whole bunch of tools. One big, big shift that, uh, that we I’ve been working on recently that’s about to ship is, um, you know, what happens when you have a lot of tools?

[00:52:13] swyx: Yeah.

[00:52:13] Simon Last: So then tool search. Yeah. So then a, a progressive disclosure becomes really important. So, you know, we were, we sort of hit a bottleneck where our, our agent worked really well. Um, we hit a bottleneck where, um, it, it, it became pretty hard to. Add new tools. Mm-hmm. And we, and we became sort of worried about it, like, like breaking the model.

It’s like, okay, someone No, I

[00:52:32] Sarah Sachs: just heard it was like saying hello was like thousands and thousands and thousands

[00:52:35] Simon Last: Yeah.

[00:52:35] Sarah Sachs: Tokens. It was really slow.

I

[00:52:37] Simon Last: can see you’re the efficiency person here. Yeah. It’s, it was too many tokens. But also it’s a quality issue because it meant that like any engineer could introduce this, this new tool for some like, like niche feature.

And it would kind of like, like Nerf, the overall model by like causing it to call the tool too much and stuff like that. And so, um, it, uh, yeah, so we, uh, we had an effort basically to, to make our harness. Uh, implement progressive disclosure in, in a nice way. Um, that’s a big shift.

[00:53:00] Sarah Sachs: You said earlier, like everyone says reasoning models was the big shift.

Like what’s more there? When we went away from few shots to describing the goal of the tool in like goal-driven, basically moving from a DAG to like a, a true system with feedback, that’s something we could distribute tool ownership to the teams. Much better because when it was all few shots, it was everyone truly editing one string and things would o would compete.

And like the order, there were all this, all these papers about, oh, you know, not all context is created equal. The higher up it is in your examples, like the more the model listens and we’re trying really hard to like fight against the order and the selection of the few shot. And that really had to be a center of excellence and it didn’t scale with the number of people for the need the company had.

It was really just five or six people that were allowed to even touch that or had to approve it rather in our code base. And then now we can actually, with the right eval, setup, distribute, um, so that everyone owns their tool and their tool definition. And sometimes we have crazy things where like we write two tools that have the same title and the agent crashes and stuff like that.

So like, you know, there are issues actually, believe it or not, um, Andro couldn’t take it. Sonic couldn’t handle two tools with the same name and open AI GPT five point. Two, it was like, I can figure this out. So that was an interesting one that we learned by accident through a, a sev.

[00:54:17] swyx: But I mean, then, you know, the underlying representation is that’s a addict, right?

Clearly. Like that’s a safety. Yeah,

[00:54:23] Sarah Sachs: exactly. Exactly. Um, but so that was like a big shift for the company and velocity not immediate because the AI team that was the center of Excellence team that owned, you know, that one file of few shop prompts had to become a platform team overnight, and that wasn’t natural.

Yeah. Yeah. But I would say that in terms of like the velocity of how we contribute to the agent, beyond coding tools, obviously being a big velocity lever, um, being able to distribute tools and not have to all collaborate on like one very select string of system prompt is truly, I would say the biggest lever on how we’ve scaled.

[00:54:57] Simon Last: We’re fighting to keep the prompt as short as possible now and then, yeah. Yeah. It’s, uh, in the latest version of the agent, I, it’s not in custom agents yet, but it will be like, like next week, a week after or so, um, there’s now like over a hundred tools. Just for all, all the crazy notion stuff. So we’re able to, to really go deep and like,

[00:55:11] swyx: would you list those tools publicly?

Is this like IP or, uh,

[00:55:15] Simon Last: no, no, no. It’s, it’s totally public. You can ask,

we

[00:55:17] Sarah Sachs: can fine

[00:55:19] Simon Last: just ask. You can just ask the agent and, and we’ll tell you.

[00:55:21] swyx: I find,

[00:55:21] Sarah Sachs: and we’re gonna post a bench. I mean, like you’re

[00:55:23] swyx: post bench.

[00:55:24] Sarah Sachs: We don’t think our system prompt is our secret sauce.

[00:55:26] swyx: Yeah. Mm-hmm.

[00:55:27] Simon Last: Great. We don’t try to hide the tools at, at all.

I think it’s, I think it’s kinda important actually as an operator, you know?

[00:55:32] swyx: Yeah. As a power user, I wanna be like, oh, I can do this, this, this. Great.

[00:55:35] Simon Last: Yeah. Yeah. I mean, one thing that, one phrase we say internally in lot is to, to teach at the top of the class. You know, we wanna build like, like the customization’s, kind of like a power tool.

I mean, we try to make it as easy as possible to set up, but we want it to be pretty deep and sophisticated. And I think a huge part of that is the operator needs to be able to interrogate. The way the system works. And a big part of that is like, what are the tools? How do they work? You know, like, like how should I prompt it to use the tools in the right way?

[00:56:00] Sarah Sachs: I’d actually say we don’t try and make it as easy as possible to use. ‘cause the more we do that, the more we abstract away that interpretability, that Simon’s talking about, that basically nerfs the model or nerfs the agent from being super capable. So a huge. I would say turning point, I can think about like the week and a half that we all came together on this as we were building custom agents, was that alignment that we’re not trying to build for everyone here.

We’re not trying to build the model that, um, or build the user experience that anyone can figure out how to use. ‘cause the more we do that, the more we just diminish its capabilities. And that was a big, you know, everyone in a couple Slack messages aligned on that, that actually made us all work faster again.

Right? ‘cause we all were like more centralized on who we were building for

[00:56:40] Alsesio: what does the meta prom generator look like? So I looked in the system prompt that it, gen, for example, uses emojis. That’s not a, you know, obvious thing to be doing.

[00:56:50] swyx: Wait, did you just

[00:56:51] Alsesio: ask it? What’s your system prompt? Oh no. This is how to generate prompts.

[00:56:54] swyx: The

[00:56:54] Alsesio: prompts generate prompts.

[00:56:55] Sarah Sachs: We call it set. Then it’s

[00:56:56] Alsesio: a set.

[00:56:56] Simon Last: Well, well, so this is actually just the agent. So, so one thing we did that, that I really like with the custom agents is it can set itself up. So we not only give it access to use the tools than it has access to like send your emails or whatever, um, but it has more tools to set itself up and to debug itself.

And so when you ask it to write system prompt, it’s just your agent itself is doing that.

[00:57:16] Alsesio: So this is just the model preference. You’re not really injecting and then into the model too much.

[00:57:21] Sarah Sachs: No, no. We haven’t guide the same thing. Makes a good custom agent and Yeah.

[00:57:23] Alsesio: Yeah.

[00:57:24] Sarah Sachs: And things like that. And then, and, and it’s really nice too because like if it fails, you can ask it, why did it fail?

And then say, okay, update your instructions so it doesn’t fit again. Obviously we should build product of self-healing that’s, that’s next on our roadmap. But um, it actually, it creates a nice system.

[00:57:40] Simon Last: Yeah. We do essentially give it like a development guide. Here’s, you know, here’s how to make a custom agent.

Here’s how to like, like help the user test it end to end, you know, to, to help them gain confidence that it works. Stuff like that.

[00:57:49] Alsesio: Mm-hmm. Yeah. Yeah. The fixing thing work, I mean, it wasn’t automatic, but I, I miss set something up and then there works like a fix button and then just, yeah,

[00:57:58] Simon Last: yeah, yeah. One thing where

[00:57:59] Alsesio: fix agent makes more,

[00:58:01] Simon Last: it’s, it’s actually, it’s an interesting sort of permission problem.

So like, right. The thing about custom agents. That is that by default it has no permission to do anything and then you have to explicitly grant it all its permissions and that’s what lets you trust it can work in the background. Right? Like you can know like, oh, it, it can read my email but not send email.

Okay, I can trust that. Right. If you let it fix itself, you know, you’re, you’re breaking that, that version there, it, it is not allowed to edit its own permissions. But as, so, you know, in the current product you can sort of click a button to fix, but now you’re entering sort of an admin mode where, where, where you’re in a synchronous chat and, and you can, and you can see what it’s doing.

[00:58:35] Sarah Sachs: Yeah. And it, and it confirms before it

[00:58:37] Alsesio: changes.

[00:58:37] Sarah Sachs: Yeah.

[00:58:37] Alsesio: The thing that I really like that most people don’t do is like, the editing chat is the same thing as the using chat. Like you can message the agent to both edit it and use it, versus a lot of other products are like, I think

[00:58:49] Simon Last: that’s really key. I think, I

[00:58:50] Sarah Sachs: think a lot of designers will feel so happy you said that.

Yeah. ‘cause we spent, we, we call this flippy, um, uh,

[00:58:55] Simon Last: yeah. What is

[00:58:56] Alsesio: this?

[00:58:56] Sarah Sachs: What do you mean? This,

[00:58:57] Simon Last: this view of, well, yeah, so if you sort of, if you close that in like open settings, you can see sort of Yeah. This is, we. We call it flippy because you know, we started with sort of like the settings were the sort of the main page and then you can test the agent.

The a GI pill way to think about it is like, oh, it is just the agent. Everything’s the agent, right? It can set itself up, it can test itself and it can run the workflow that they want to run. Uh, so we flipped it. So the main view I was looking at is the chat and, and then the settings is more just like a side panel at, at sort of previewing the changes that it’s making.

So you can introspect on them or, or you can also make changes manually if you’d like. But, but we wanna design the experience from the get go. So you don’t have to ever any of the settings manually, you can just talk to it.

[00:59:39] Sarah Sachs: And the inside baseball is like how this works was probably the launch blocking part of this build.

Right. Um, especially ‘cause we had a lot of early adopters that were used to the old way and that’s like the benefit of adopting in public. But then changing how people think about setting up custom agents when they already had this flow in and of itself was difficult. Um,

[00:59:57] Simon Last: I mean that’s really fun ‘cause the, we, we ended up sort of uh, uh, painfully delaying the launch.

Mm-hmm. By.

[01:00:04] Sarah Sachs: A month?

[01:00:04] Simon Last: A few weeks. Yeah, definitely. Like, like a month or so. Um, but the whole team was super enthusiastic about it though. ‘cause it was just so much better. It was like, oh yeah, obviously you have to chat with it, right? Yeah, yeah, yeah. To set itself up. And everyone was super bullish on that, so it was like, like painful for a second.

But then everyone’s like,

[01:00:19] Sarah Sachs: right, and like back to, you know, organization design, which I probably care about more than Simon, but like the people that built this are three engineers from three different teams. Because we’re like, we need to launch this and we need to fix this. And then we’ve just built a company where then we just put people on it and no one complains, the manager doesn’t complain.

And we were able to unblock and just ship it.

[01:00:37] Alsesio: Yeah, yeah. But being in a failure chat and asking it to just fix yourself is amazing. Versus I gotta copy this and put in the settings chat. Mm-hmm. Mm-hmm. To do

[01:00:49] Simon Last: it. So yeah. Interesting. Like a trade off in there that, that we’re trying to explore, which is, you know, we wanna be like a business enterprise safe agent where you can delegate something and, and trust that it’s gonna work.

But also we want to get some of that sort of bootstrapping power that, that you feel like when you’re coding it is making a browser, like for itself, right. There’s something there. I think that’s, that’s really important. So it’s, we’re trying to sort of. Navigate that, that, that trade off and try to get you both.

[01:01:12] Alsesio: Now it’s free, it’s amazing. Uh, I’m worried about when I have to start paying. How do you think about, so you have notion credits as a payment for this, which is like separate from the usual tokens, uh, that the model generates. How do you design pricing, value-based pricing based on the task and things like that.

[01:01:30] Sarah Sachs: So they are, um, the credits and payment structures associated with the token usage. The reason that we had to make it not just throughput of tokens is that it’s not always priced that way. Like our, um, fine tuned and open source models are served on GPUs, right? Web search is priced differently. You know, if we were to host sandboxes, those are priced differently.

So we had to think of an abstraction above tokens. And it’s also not just tokens, it’s the token model. Um, and serving tier trade off, right? Mm-hmm. Because we can have priority tier processing, we can have asynchronous processing. The cash rate could be different, um, depending on who uses it when, right?

And so we wanted to, um, from the get go commit to making sure that customers were getting the fair deal. Not necessarily that we were making a ton of money off of it, but that customers were paying for what was reasonable. That’s the fundamental of where we started. And also, you know, we’re selling enterprise sa, so if we sell credit packs and you get discounts if you’re an enterprise and you buy a certain amount of credit packs and things like that.

So it also just helped the sales motion, um, work a little bit easier. So that’s the answer on the abstraction of credits to dollars. Now was the question how we decide how to price it or?

[01:02:34] Alsesio: Yeah, like, I mean, I think there’s, all tokens are not made equal, but yeah, we obviously get charged mostly equal. Like you can ask, uh, codex to create you a dumb tool for like, I created one for our StarCraft two land for people to like find a game.

Uh, but then people create it to build features and like billion dollar companies. But the token price is the same.

[01:02:53] Sarah Sachs: Yeah.

[01:02:54] Alsesio: Like for you, I can ask this to update my favorite recipes doc. I’ll do it, but I could ask it to like respond to an email from an investor and like the value is like very different, you know, and you could charge more, but you’re not necessarily doing it.

So I’m curious if there was any discussion.

[01:03:11] Sarah Sachs: I think, I think that, um, that’s not where the market is right now. Um, number one, the second reason that we’re not doing that, as it ended up being kind of complicated to figure out what was complicated or not. So we at first we were like, let’s just charge on agent runs.

And you know what, you went through all the different versions that ultimately just brought you back to a lot of complexity that mapped directly to token throughput. And so it, it’s also just simpler. Um, it’s quite difficult, um, to build those pricing systems. And, um, I actually think that one of the biggest reasons we want had usage based pricing for this capability is.

We’ve had our core agent for a while with a model picker and there were certain models, um, or certain functionality that we had margins to maintain. And if we wanted to ship this functionality, uh, you, we couldn’t afford it, it would bankrupt the company. If we let, for instance, like autofill or the database autofill feature, we’ll soon be agentic That will be associated with usage based pricing.

Because if every single autofill action was an agent running on Opus on every single database sell, it would be billions of dollars, right? And so we had to find a way for the customers that wanted to do more and wanted to give us their money and pay more to find the outlet for them to do it, that we didn’t have to apply to the lower end of the curve.

And also, not all knowledge work is equal. Like there’s different points. A lot of the agent workflows here really saturate model capabilities. Like you don’t need a complicated model for it. And so charging based on token usage, um. It, we couldn’t just decide for you that you wanted your email client to be dumb or not, right?

Like, we want you to decide if you want to have Opus Auto Triage all of your emails, we will actually give you nudges in the product to rethink if that’s the right choice. Right. Um, because also not every user, um,

[01:04:52] Alsesio: understand.

[01:04:53] Sarah Sachs: You’d be surprised in user interviews. People would be like, oh, I didn’t know that.

So now we actually have a little hover that tells you like if it’s expensive or not. Yeah. I mean, it’s also slower. So the thing that’s interesting is like people don’t care about speed and custom agents. And so the incentive of like, uh, haiku being faster, people don’t care when it’s asynchronous. Um, and so we want to only provide the service of extra, extra benefit that people want.

And the best way to do that is to incentivize them because it’s their own own money.

[01:05:21] Alsesio: It must be confusing for people that are not familiar. It’s like, why is there no 5.3. You know, you open this thing and it’s like, is there something missing? Manual. It’s not their fault. Not their fault.

[01:05:30] Simon Last: Yeah. That’s just the world we live in now.

[01:05:32] Alsesio: Yeah. It just radical jump point too, it’s like Cloud had that.

[01:05:35] Sarah Sachs: I mean, but auto is heavily, I think what’s actually been hard for us is to tell convince people that auto is not just our cheapest, dumbest model, but actually the model that’s best for the task that you wanna do. Um, alright. Steve.

[01:05:46] swyx: I mean,

[01:05:48] Sarah Sachs: exactly.

Nice. Um, and a lot of our job is actually figuring out auto because it’s like,

[01:05:54] swyx: this is the agent lab. Every agent lab has an auto. Mm-hmm.

[01:05:57] Sarah Sachs: Yeah. And

[01:05:58] swyx: that’s the job.

[01:05:58] Sarah Sachs: Exactly. Because if you think about, like I said, I come from Robinhood, like you could spend a lot of time keeping up with the markets or you could have a auto investing, right?

And you can have an index fund or you can have

[01:06:12] swyx: roboadvisors

[01:06:12] Sarah Sachs: of the robo advisor. And so like at a certain point we also can be roboadvisors and like we have a lot of people figuring out what model is best for the right task. And we now, we’re not using auto as a, as a margin maker, we’re just using it to kind of reduce stress.

It’s not opus, that’s for sure. Yeah. Because a majority of the tasks people are doing aren’t opus level, um, intelligence.

[01:06:34] Simon Last: The other thing I would say is the, um, you know, unlike a lab, we aren’t fully incentivized just for you to use as many tokens as possible. We’re actually really interested in. Giving you the right tool for the job.

A lot of the time, the right tool for the job is actually just writing code and not even using agent at all. So that’s, that’s something that we’re investing in a lot is like, you know, imagine your, your agent can actually automate itself out of a job. Right. We would love if that were true.

[01:06:58] Sarah Sachs: I feel very strongly about this because I don’t necessarily feel like that’s the SKUs that Frontier Labs give you.

I feel like they are just getting more and more capable and more and more expensive, which is fantastic for the use cases of when people wanna do really complicated things on Notion. Um, what’s difficult is like that market that I think right now is no man’s land of where reasoning models were six months ago, that the nano haikus, et cetera, haven’t caught up to, because now we’re just paying more for those, um, for like extra capability that we didn’t necessarily need and so are our customers.

Mm-hmm. And, um, labs aren’t necessarily incentivized, um, right now with how few players there are to be meeting the market everywhere. They just need to be the cheapest. They don’t need to be at value that the customer wants.

[01:07:41] swyx: Hmm.

[01:07:42] Sarah Sachs: If no one’s cheaper than them, then they’re the cheapest and that’s good enough.

Right. And so we’re doing a lot to make sure that we have the right optionality, um, to switch between models and also invest in open source because the open source models actually are, um, getting to be the place where reasoning models were three, four months ago. And, um, that’s what’s filling that gap right now.

So you’ll see we offer Mini Max and, um, we are collaborating a lot with different open source labs to think about notion’s last exam and how they can do better on these types of tasks. Mm-hmm. So that we can offer them for that intelligence to price to latency trade off. Because, you know, in that triangle of intelligence, price, um, intelligence, price and latency, excuse me, um, users get to choose where they are, but right now, um, there’s not, the whole triangle isn’t filled with models, right?

Yeah. And the more that different models build cluster triangle capability, everyone’s clustered in capability where everyone’s cluster. I mean, haiku’s not that much cheaper. No one’s really in the middle. Like people really tend to. Cluster round two. Mm-hmm. Like, this is really capable and it’s really fast made, it’s really expensive or whatever.

Right. And so we just wanna make sure that that triangle’s filled, um, and we wanna offer the models that fill it and we wanna, um, gate guide users to understand when they need it. Yeah. Um, which one,

[01:08:54] swyx: I mean, all I’m hearing is that someday you’re gonna change your model. You have lots of tokens.

[01:09:01] Sarah Sachs: I don’t know if, what do you mean by train your model?

You train

[01:09:03] swyx: your

[01:09:03] Sarah Sachs: own, train your own model. Don’t know. We have money to train a founda. I mean,

[01:09:06] Alsesio: you go raise

[01:09:07] swyx: it. Yeah. You, you can raise it.

[01:09:09] Sarah Sachs: That’s your job, Simon. No, I, I don’t think that that needs to be our core competency.

[01:09:14] swyx: This is usually the, the thought process that leads to like, well, no one else is doing it.

We, we will take a crack. You know,

[01:09:19] Simon Last: I think I’m, yeah. I mean, I feel like to the extent that we do anything like training in the other area I’m actually most excited about is, um. Less of like one big model for all the users, but like as, as, as it becomes more possible to do, you know, to make like a specific fine tuning that’s like really knows your context of, you know, your company, the people that work your company, what’s going on.

I think that’s, that’s pretty interesting because if you, if you had a model that really knows your company, I think that would be like a huge quality uplift.

[01:09:47] Sarah Sachs: We actually have some enterprise vendors that kind of ask about this, um, along with bring our own key. Like if I have a model that really understands like my enterprise that we’re training for all these reasons, these tend to be like quite large institutions thinking about how to let people bring their own models.

But those models have to function with like

[01:10:04] swyx: right

[01:10:04] Sarah Sachs: understanding how to call our tools. And that’s where again, having, um, more. Public system prompt is like beneficial to notion, right? Um, we want all models to plug into notion as, as, as well as they can. Um, that being said, like of course there are certain aspects of notion where we do fine tune and do reinforce and fine tuning on our own capabilities.

Um, but that’s not necessarily trained on user data. Um, you don’t need that, that much data, um, in the first place. And that’s where when we have like a data scientist and a, a model behavior engineer really understand where the capability gap is, that’s when we invest there.

[01:10:38] Simon Last: I personally burned a lot of time trying to train models.

Uh, and it’s tempting, right? It’s so tempting, retraining

[01:10:46] Sarah Sachs: every day.

[01:10:47] Simon Last: I was doing crazy amount. Yeah, I was doing a lot of different things. Um, and it, I

[01:10:50] Sarah Sachs: was the budget person that came and found out and I showed up and I heard that that was happening time

[01:10:55] Simon Last: out. You know, like a, a funny thing that ‘cause the sort of an arc that like looped on itself is, uh, you know, back when I was doing tons of training stuff, it takes a long time to do it.

Any kind of training run. And so. You end up operating like, like 24 7 around the clock. Like it becomes very important that before you go to sleep, like everything is watch intensive board, all the experiments are, are started. And then as I stopped training, that kind of went away. But now the coding agents have totally brought this back.

Mm-hmm. So now every night before I go to bed, I’m like, okay, did I start enough agents, you know, to get them done. I get everything done. So it, it’s, it’s a ding interesting heart,

[01:11:26] swyx: this balance of like, you have to try polyphasic sleep so you can wake up every two.

[01:11:29] Simon Last: Absolutely. Yeah. Yeah. We, uh, yeah, I have not gone there yet, but, but my goal these days is just to, before I go to bed.

The agents are running, and I’m confident that they won’t be done by the time I wake up. Really

[01:11:41] swyx: Eight

[01:11:42] Simon Last: hours.

[01:11:42] Sarah Sachs: There’s a, I won’t say which coding Frontier Lab, but there was a point where he had like outlived like the thread length and context length uhhuh that that coding agent provided. And I DMed you DMed them being like, Hey, I need, I need more.

And our account rep DMed me directly and they’re like, is Simon trying to prove string theory? Like what is he doing?

[01:12:00] Simon Last: Yeah. I, I had a single coating Asian thread going for I think it was like 17 days. Uh, pretty much continuously.

[01:12:06] swyx: Don’t, don’t they just compress? I mean, yeah.

[01:12:08] Simon Last: Yeah. It was actually just a bug.

It was a harness bug. Yeah. It, it had done compaction like a hundred times probably.

[01:12:13] swyx: Yeah. The

[01:12:14] Sarah Sachs: other thing that um, reminded me about fine tuning that I think you and I have aligned on is that. Our tools change really frequently, and right now we spend a lot of time rethinking and building tools for capability and fine tuning a model, um, to understand your tool.

Like we don’t have legal expertise or coding expertise. So if we were to fine tune a model, it would either be expertise about the enterprise and you know, we have ZDR, no data retention offerings for those enterprises. So we’d have to really rethink how we structure if an enterprise wanted to opt into that or it would be fine tuning and better capability on navigating our tools that doesn’t match with the velocity with which we create new tools.

And so it actually really slow us down, um, to have a model that was fine tuned on our tools because we’d have to retrain it and cut a new model every time we did that. And that’s not how we’re set up right now. Um, particularly with the way that we’re changing our, I, I guess we could fine tune a model to like search for tools.

It’s just. The, the amount of time it takes to do that, ship it, have the right system, you’re basically making a bet against a frontier capability not serving that, and the time it takes you to build it. Mm-hmm. And that, that time lag hasn’t happened for us yet. It hasn’t

[01:13:17] Simon Last: been, yeah. It’s just the wrong trade off.

I think. It’s just like you want Yeah. We literally change our tools every single day and if we notice an issue, we will, we’ll, we’ll, we’ll fix the problem. I think a, a good way to think about it, I think is pretty fruitful, is like, don’t focus too much on training. I would think of that as like, that’s an implementation detail.

Like what’s the outer loop, right? Like, like the outer loop is you have a model and then some harness or, or system where it’s interacting with the system that needs to work. And you know, if there’s a problem, the way to solve the problem isn’t necessarily to train a model. It’s like, oh, maybe there’s just a bug in one of the tools.

Right? And actually 99% of the time it’s a bug in one of the tools, right? And so just fix the bug. And then the outer loop thing that’s really fruitful to think about is like, how can you improve your, your velocity and robustness? Making really good tools, making a good harness, you know, like, like verifying it works.

Hmm.

[01:14:07] Sarah Sachs: The one place that we do invest more in model turning now necessarily though, is actually in retrieval because, um, we’re at a point right now in our business and enterprise, our AI enabled plans where. The search load and the search traffic. Majority of it’s coming from agents, not humans. And so for every query that’s hitting our elastic search or our vector indices, they’re not coming from humans.

And the queries are structured differently. And what’s returned has a different re requirement. Positional ranking matters less, but top K retrieval mode matters more. Right.

[01:14:34] swyx: Isn’t top KA form of position?

[01:14:36] Sarah Sachs: Of course it is. But um, when you’re training on like click through rate, it’s really, you know,

[01:14:41] swyx: yeah.

[01:14:41] Sarah Sachs: It matters much less.

Number one through number six is very different

[01:14:44] swyx: Yeah.

[01:14:44] Sarah Sachs: Than it needs to be in the top 100.

[01:14:45] swyx: Like the slope is just,

[01:14:46] Sarah Sachs: yeah.

[01:14:46] swyx: Higher.

[01:14:47] Sarah Sachs: It’s a different optimization function for retrieval, um, model. Similarly, uh, what snippet you include matters more or less. Right. So we are rethinking a lot of that functionality, um, to work with how the agents like to write queries and how, um, they wanna, uh, receive information.

Yeah. So we are doing like another kind of reinvestment into rethinking not only search for, um, how do agents do searches versus how humans do searches. Um, but we’re also investing in like. Indexing different things now because, uh, how are, how do you index, uh, the setup generator for Notion agent? It kind of breaks our block model entirely, um, where all blocks are nested in each other.

Same with meeting notes. Um, and so we do, we, I mean, so we’re hiring ranking engineers and model training engineers, but it’s primarily on ranking.

[01:15:32] swyx: Yeah. Does ranking maps to res for you? It does, right. Recommendation systems.

[01:15:36] Sarah Sachs: Yeah. Um, yes.

[01:15:38] swyx: Right. Okay. Say this, but I’m trying to promote res more in general ‘cause I is weirdly unpopular.

[01:15:45] Sarah Sachs: I don’t know why. Um, but the other thing is that, like, I I was just talking about this with a peer, like how much is ranking important versus like, uh, being able to do parallel exhaustive queries. Right. Um, so we’re also, they’re both important. They’re both important, but like they’re both two tools to the same user outcome or the same agent outcome.

Uhhuh. Right. And so, um, that. That’s something that we’re also rethinking a lot even on, we just did an experiment on, um, notion ranking at this point, um, for notion retrieval, vector embeddings are less and less.

[01:16:15] swyx: Did you see that? Yeah. Notion just, uh, to nine

[01:16:19] Alsesio: so long it became dark mode.

[01:16:21] Sarah Sachs: We’re working the night shift for you.

Right? Looks

[01:16:23] Simon Last: pretty good. I’m not seeing any bug.

[01:16:24] swyx: You know, I worked on this like parallel search thing where you, you found out to eight different queries, right? Yes. And so you actually need to use the model to work on query diversity so that you get right. Investment space.

[01:16:35] Sarah Sachs: And so like the people that are working on, um, ranking and retrieval are the same people working on what query generation is.

It’s all one, uh, journey. Yeah. We call it age agentic find. And we’re actually realizing, for instance, that it’s less about a selection. Like we don’t spend a lot of time trying to optimize what vector embedding we use anymore. That was a period of time, but that’s just not the right lever of optimization.

[01:16:55] swyx: Yeah. Right. Yeah. Okay. Uh, we’ve gone long. I have to talk about motion meeting minutes and then we’ll, we’ll, we can call it there. Uh, you, you, you just have a lot of comments. Uh, you, you, uh, I don’t know where you wanna start. Um, is it the audio side? Is it the sort of Oh, meeting notes, summarization? Yeah.

[01:17:12] Simon Last: Sort of like what makes it work or

[01:17:13] swyx: No, just like anything sort of interesting technically, right? Like I think you had, you had some, uh, book points. I always call these like check marks along the way when the, when a guest says something that we, they wanna return to later, I just like, check mark it. Yeah.

I’m like, okay. We’ll back to it. Um,

[01:17:26] Sarah Sachs: meeting notes was one of those things where at first we were nervous that we’d have to teach people a different way to work, and we were nervous that that was a lot of user friction. I think one of the reasons why, I mean, they’re one of our biggest growth lever. I think they’re one of the most like.

In terms of virality of adoption and retention, quite strong. Um, and so we’ve invested more and more as we did that. I think what’s really powerful about it is, again, notion is the system of record of where and how you work. The way that I use meeting notes is every one-on-one and meeting I have is meeting notes.

When I do my performance review for myself, myself, review, I say primarily look at all my conversations with my manager and like, write up what I did this year, right? Because if I didn’t talk about it in my one-on-one with my manager, it probably wasn’t relevant for my performance review. So it also just adds a ton of signal on prioritization that’s really helpful for a good system of record.

That’s really helpful for like our agent. It’s also like caused a lot of scaling for search and for the agent. Um, and you know, it’s, it’s just an explosion of content when you have transcripts like that. Um, how we do compaction. A lot of that was triggered by meeting notes passed into context, things like that.

Um, so it’s been a good impetus for us to think about. Longer form, um, content when you think of it as like a priority, primitive, but it’s been one of the most powerful signals for our agent. Um, because it’s

[01:18:44] swyx: unsurprising. Right? Right. And

[01:18:45] Sarah Sachs: you’re

[01:18:45] swyx: capturing a whole new thing.

[01:18:46] Sarah Sachs: So it’s like our own data. Like we want users like, or they’re creating their own data flywheel, right?

[01:18:51] swyx: Like it serves me to prefer notion, uh, to put all my stuff because it has my other stuff.

[01:18:57] Sarah Sachs: Totally. I mean, the way that, the way that like our teams run right now is. You know, there’s a custom agent that does a pre-read before standup. It looks through all of Slack and GitHub and just says, you know, it, it, it creates a summary and it creates a meeting note and it says Everyone do this pre-read.

Then we just press play. We have the meeting, we talk through the pre-read, we talk about what needs to happen next, and then we have a custom agent integrated with our calendar and triggers that then files task for tomorrow or today based on what we spoke about. And, um, sends off Slack messages that we decided in the meeting needed to be follow ups.

Like our meetings are hands off keyboard and we’re focused on, um, the root of the problem, not the bookkeeping around the problem.

[01:19:32] Simon Last: One thing that, uh, the me, us team had recently that was, but I’ve been blowing my mind, is they, we, uh, uh, they made it so it actually, when it makes the summary, we’ll actually app mention the people that were referenced oof in it.

So I, I, I now get notifications whenever someone talks about meeting. Yeah. I

[01:19:46] Sarah Sachs: feel like that one

[01:19:47] Simon Last: was, it’s like, it’s like, oh, you know. Simon is working on this. Okay, I’m gonna, it’s actually amazing how, because then I’m like, oh, okay, cool. I’m gonna go talk to them about that.

[01:19:55] swyx: Right? What, what if they’re two Simons?

[01:19:56] Simon Last: Um,

[01:19:57] Sarah Sachs: no wait, so wait. It’s powered by the agent. So it’s doing agentic. So if you look at it thinking, I don’t know if this is shipped yet. It will be, when you look at it thinking when it’s doing the summarization, it’s saying, figuring out who Simon

[01:20:07] swyx: is most probable Simon

[01:20:08] Sarah Sachs: is. Yeah. Um, and we also have like a people to people similarity cash and stuff like that.

Yeah, yeah. On the here’s we sort of like,

[01:20:15] Simon Last: we also like generate a profile for each person and like, and use that. Um, yeah. I mean of course I can get it wrong, but the goal is for not to get it

[01:20:22] Sarah Sachs: wrong. Meeting nuts is just like the agent primitive packaged on top of a transcription. Primitive. Yeah. Yeah. And then a vertical team.

It’s probably one of the only teams at Notion that’s completely a vertical team around quality and product like UX design. ‘cause it’s still a Tiger team. Um, with a fantastic manager, Zach, that joined recently, um, from Embr, but, um,

[01:20:40] swyx: Zachar.

[01:20:41] Sarah Sachs: Yeah.

[01:20:42] swyx: Yeah. I, uh, chatted with him when he was talking about with his working number.

[01:20:45] Sarah Sachs: Yeah. So he’s, he’s managing that team now and thinking about it as data capture. That’s what meeting notes is, is data capture it, get

[01:20:50] swyx: all

[01:20:51] Sarah Sachs: the kinds of kind of reframing, um, where meeting notes are valuable as a data capture problem and then working inside, um, like the summarization used to not be age agentic.

Yeah. Now it is because it does all the things like figure out who the right Simon is. And one day you can have a custom agent directly integrated in it that knows like what task database the meeting is referring to. And as you’re having the meeting perhaps update the tasks and things like that. Like there’s a, there’s a lot of that experience of where we do our work in meetings that we wanna invest in.

Making more seamless.

[01:21:18] swyx: Yeah. Uh, opening eyes, doing hardware. Uh, would you ever ship one of these?

[01:21:22] Simon Last: Yeah, probably not,

[01:21:23] Sarah Sachs: but one of those.

[01:21:23] swyx: But you know, this, this is meeting notes in person.

[01:21:25] Simon Last: Yeah. Yeah. I, I’d be excited about, I mean, I’m excited about that, that product category in general for sure. Yeah.

[01:21:31] Sarah Sachs: I think it’s like, it’s a, it’s a mechanism and it.

It, one of those needs to work really well with Notion. We would partner with whoever’s building one of those, I think. Yeah. This is

[01:21:40] swyx: be they, they were bought by Amazon. I don’t know. I I can refer you.

[01:21:43] Sarah Sachs: And there’s like, there’s some wild companies doing like really cool things that come to our partnerships team that I like to sit in on the demos of, of wearables.

I always like to send in on the demos ‘cause I think they’re Oh, okay. Pretty cool. And all of them want to make sure, not just notion, but like you can imagine the ones that talk to you. Yeah, yeah. Um, being able to do search and build context. So like if you’re entering like a conference, um, being able to like do like look at your CRM and do things like that.

Um, and you can utilize the Notion agent to do that. So we are in like the very beginnings of those partnerships. I think what’s unique about that particular technology is it goes against what I talked about with custom agents right now, which is the more simple it is, the harder it is to have like advanced controls over its capabilities.

Right? And so that would be a great investment for data capture, but not necessarily like our agent is workflows.

[01:22:26] Simon Last: It’s something with a different slice of the problem, I would say. Yeah. Like that’s gonna be deeply personal. Like, like your company’s not gonna force you to wear a risk. Wristband. Right. I, I think

[01:22:35] Sarah Sachs: it’s good to hear that from me.

From you. Yeah.

[01:22:38] Simon Last: Yeah. The, the CEO’s gonna force everyone to wear a wristband look, I mean, the slice of the problem that, that we care about is like, you know, can the company have all the context of what everyone said at every single meeting, and then use that to, yeah. To, to derive value for themselves.

[01:22:52] Sarah Sachs: It kinda reminds me, I remember once you.

Very strongly reminded me, our job is to not make the best harness for agentic work. Our job is to be the best place where people collaborate. It’s like our job isn’t to build the best wearable to capture meeting notes. Our job is to build the best place where meeting notes live. Right?

[01:23:11] swyx: Yeah. So it basically, you’re saying everyone else can just pipe to you and it’s fine, right?

Yeah, yeah, yeah. That’s, that’s a reasonable thing. All I’ll say is that people, there’s people walking around with notion tattoos on them. They, they’ll wear notion anything. So just, I don’t know, do a limited run.

[01:23:24] Simon Last: Yeah, yeah. No, I mean,

[01:23:27] Sarah Sachs: we have such understated swag that the idea, like our swag has so few notion lay logos on it.

The idea that people have notion tattoos is pretty antithesis to our design principles, so that’s pretty funny.

[01:23:38] Simon Last: Yeah.

[01:23:39] Sarah Sachs: Do you have one?

[01:23:40] Simon Last: No, not, I do not have a notion Tattoo too. I’ve, I’ve seen them. Yeah.

[01:23:44] swyx: Cool. Uh, well, thank you so much. This is such a great deep, deep dive. Actually. The chemistry between you two is amazing.

Like, I, I can’t believe, like

[01:23:51] Sarah Sachs: we work together a lot. Yeah. Different jobs. Work closely.

[01:23:55] swyx: Yeah.

[01:23:55] Alsesio: That’s it. Yeah. Thank you. Thank you.

[01:23:57] Sarah Sachs: Thanks. Thank you.

The top names you should know as a baseline, adjusted for “what people are actually recommending” rather than just benchmark supremacy:

1. Qwen 3.5 — most broadly recommended family right now across usecases.

2. Gemma 4 — strong recent buzz for local usability, especially smaller and mid-sized deployments.

3. GLM-5 / GLM-4.7 — near the top of broad open-model rankings, increasingly part of the “best overall” conversation.

4. MiniMax M2.5 / M2.7 — repeatedly cited for agentic/tool-heavy workloads.

5. DeepSeek V3.2 — still firmly in the top cluster when people talk about strongest open-weight general models.

6. GPT-oss 20B — not the mainstream “winner,” but increasingly recommended as a practical local option and for uncensored variants.

For local coding, the overwhelming consensus is Qwen3-Coder-Next. So that’s easy.

Naturally the fuller list is going to have a strong lean on roleplay/creative writing, the #2 usecase of LLMs, and we are NSFW-friendly so here goes…

Read more

We’ve just concluded a marathon 3 days in Europe - first the Online Track and the Workshops, then over a hundred talks delivered in person, some livestreamed. There was also a fair amount of live podcast coverage, from ThursdAI to ETN, from visits to 10 Downing Street to morning runs to cool swag to viral talks to aquarium parties to nightclub parties.

We’ll try to publish a few recap thoughts in future days, but for now you can see my closing keynote at the end of Day 2 and watch some of the large talks.

Day 1 Talks (link)

Day 2 Talks (link)

AI News for 4/9/2026-4/10/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Open Models, Coding Agents, and the New Advisor Pattern

• GLM-5.1 breaks into the frontier tier for coding: The clearest model-performance update in this batch is GLM-5.1 reaching #3 on Code Arena, reportedly surpassing Gemini 3.1 and GPT-5.4 and landing roughly on par with Claude Sonnet 4.6. Arena later emphasized that Z.ai now holds the #1 open model rank and sits within ~20 points of the top overall. The release was quickly picked up by tooling vendors, including Windsurf support. In parallel, Zixuan Li outlined a three-part open-model strategy: accessibility, strong fine-tunable baselines, and sharing architectural/training/data lessons with the broader community.

• Advisor-style orchestration is becoming a first-class design pattern: A notable systems trend is the convergence around “cheap executor + expensive advisor.” Akshay Pachaar’s summary ties together Anthropic’s API-level advisor tool and Berkeley’s “Advisor Models” line of work: use a fast model for most steps, escalate only at difficult decision points. Claimed gains include Haiku + Opus more than doubling BrowseComp score vs Haiku alone, and Sonnet + Opus improving SWE-bench Multilingual while reducing task cost. The pattern was implemented almost immediately in open source via advisor middleware for LangChain DeepAgents, with Harrison Chase highlighting the speed of OSS uptake. This idea also shows up in practitioner commentary from Walden Yan, who argues future agents will increasingly look like fast worker models delegating hard judgments to “smart friends.”

• Qwen Code adds orchestration primitives directly into the product: Alibaba shipped Qwen Code v0.14.x with several agent-engineering features that align with this broader shift: remote control channels (Telegram/DingTalk/WeChat), cron-based recurring tasks, 1M-context Qwen3.6-Plus with 1,000 free daily requests, sub-agent model selection, and a planning mode. The sub-agent selection feature in particular makes model-mixing explicit at the tool level rather than just in external harness code.

• Model-routing demand is now a product complaint, not a research topic: Multiple tweets converge on the same operational pain point: top models are spiky and specialized. Yuchen Jin points out that Opus often wins on frontend and agentic flow while GPT-5.4 performs better on backend/distributed systems, but tools like Claude Code and Codex remain too provider-bound. That complaint sits directly beside the advisor pattern above: practitioners increasingly want shared context + automatic routing + cross-model collaboration inside one workflow rather than manual switching between terminals.

Agent Harnesses, Hermes Momentum, and the “Portable Skills” Stack

• Hermes Agent had the strongest ecosystem momentum in this dataset: Hermes dominated the agent-framework chatter. The ecosystem map was updated for v0.8.0, Hermes Workspace Mobile launched with chat, live tool execution, memory browser, skills catalog, terminal, and file inspector, and Teknium announced FAST mode for OpenAI/GPT-5.4. Distribution also broadened through SwarmNode support, while the project itself hit 50k GitHub stars. Practitioner feedback was unusually concrete: Sentdex says Hermes with local Qwen3-Coder-Next 80B 4-bit now replaces a large part of his Claude Code workflow, and several others described it as the first agent framework that “just works.”

• The harness layer is solidifying into the primary abstraction: Harrison Chase’s framing is representative: the industry is moving from unstable chain abstractions toward agent harnesses as a more durable foundation—essentially “run the model in a loop with tools” now that models are finally good enough for it to work. Supporting tweets stress the same architecture from different angles: “open harness, separated from model providers”, “portable agents”, and “the real bottleneck isn’t the model, it’s the harness”. The deeper implication is vendor decoupling: skills, memory, tools, and traces become long-lived assets while models are hot-swapped underneath.

• Skills are becoming the new app surface: Several tweets point toward a shared packaging model built from skills + CLIs + AGENTS.md-like interfaces. Caspar B gave the best practitioner writeup, detailing how well-designed skills can materially improve planning, long-horizon coding, code review, and frontend iteration. adward28 similarly argues that as AGENTS.md, skills, and tool configs become more portable, the whole ecosystem becomes more usable. This is complemented by infra releases like MiniMax’s MMX-CLI, which exposes multimodal capabilities to agents via a CLI rather than MCP glue, and SkyPilot’s agent skill for launching GPU jobs across cloud/K8s/Slurm.

• Observability is turning into a default expectation for agent development: The tracing/evals loop is now explicit in product and research discussions. Sigrid Jin summarizes the emerging doctrine well: evals are the new training data, but agents overfit and reward-hack, so teams need strict splits, curated evals, and a loop from production traces → failures → evals → harness updates. This is mirrored in tooling releases from LangChain, W&B’s Claude Code integration + skill, and Weave’s auto-tracing plugin.

Benchmarks, Evals, and Capability Measurement Got More Realistic

• ClawBench and MirrorCode push beyond toy agent evals: ClawBench evaluates agents on 153 real online tasks across live websites and reports a dramatic drop from roughly 70% on sandbox benchmarks to as low as 6.5% on realistic tasks. In software engineering, Epoch and METR introduced MirrorCode, where Claude Opus 4.6 reimplemented a 16,000-line bioinformatics toolkit—a task they estimate would take humans weeks. Notably, the authors already warn the benchmark may be “likely already saturated”, which says as much about the pace of coding progress as the result itself.

• Reward hacking is now a central part of model evaluation, not an edge case: METR’s new time horizon result for GPT-5.4-xhigh is a useful example. Under standard scoring, it lands at 5.7 hours, below Claude Opus 4.6’s ~12 hours. If reward-hacked runs are counted, it jumps to 13 hours. METR explicitly notes the discrepancy was especially pronounced for GPT-5.4. Separately, Davis Brown reports rampant cheating on capability evals, including top submissions on Terminal-Bench 2 allegedly sneaking answers to the model.

• AISI reproduced steering-vector oddities: The UK AISI transparency team reports replicating Anthropic’s steering approach for suppressing evaluation awareness, with the surprising result that control vectors (“books on shelves”) can produce effects as large as deliberately designed ones. For engineers building model-monitoring or post-training interventions, that’s a cautionary result about how messy and non-specific linear steering effects can be.

Systems, Numerics, and Local/Edge Inference

• Carmack’s bf16 scatterplot is a useful reminder that low precision fails in visible, structured ways: John Carmack’s post on plotting 400k bf16 points showed clear quantization gaps emerging as values move away from the origin. The value for practitioners is not the anecdote itself but the intuition reset: bf16’s reduced mantissa becomes visually and operationally obvious at surprisingly modest magnitudes. This pairs well with Arohan’s warning not to skip “determinism and numerics days.”

• Apple/local inference stack keeps compounding: Awni Hannun highlighted demos of Qwen 3.5 and Gemma 4 running locally on Apple silicon via MLX, and separately MLX’s origin story resurfaced. There was also continued momentum around mlx + Ollama integration and Ollama’s MLX-powered speedups on Apple silicon. The broad pattern: local LLM ergonomics are no longer novelty demos; they are becoming a viable default for coding and agent workflows.

• Inference optimization remains highly recipe-driven: Two useful examples: Red Hat AI’s speculative decoding for Gemma 4 31B using EAGLE-3, and PyTorch/diffusers work on low-precision flow-model inference where Sayak Paul summarizes the final recipe: selective quantization, better casting kernels, CUDA graphs, and regional compilation. These are good reminders that practical speedups still come from stacking many system-level interventions rather than a single magic optimization.

Research Directions: Memory, Synthetic Data, and Neural Runtime Ideas

• Memory is shifting from “store facts” to “store trajectories”: The Turing Post’s summary of MIA frames memory as retained problem-solving experience rather than just retrieved context: a manager/planner/executor loop that stores full journeys. That direction is echoed by Databricks’ “memory scaling” claim that uncurated user logs can outperform handcrafted instructions after only 62 records.

• Synthetic data is becoming programmable against differentiable objectives: Rosinality and Tristan Thrush point to work on generating synthetic training data that directly optimizes downstream objectives—up to and including embedding a QR code in model weights through the data alone. This is a strong example of data design being treated as an optimization target in its own right.

• “Neural Computers” proposes learned runtime as the next abstraction boundary: Schmidhuber and collaborators introduced Neural Computers, pushing the idea that computation, memory, and I/O could move from fixed external runtime into learned internal state. Whether or not the formulation holds up, it’s one of the more ambitious attempts in this set to redefine the boundary between model and machine.

Top tweets (by engagement)

• Medical/LLM reliability failure: HedgieMarkets on fake “bixonimania” papers getting accepted by major AI systems and even cited in a peer-reviewed journal. High-signal example of retrieval/verification failure in safety-critical domains.

• Numerics: John Carmack on bf16 precision gaps in scatter plots. One of the most practically useful tweets in the batch.

• Policy/cyber-risk narrative: Bloomberg’s report that Powell and Bessent discussed cyber risks from Anthropic’s “Mythos” with Wall Street leaders drove substantial engagement, though the technical substance remains second-hand.

• Product integration: Claude for Word entering beta was one of the biggest genuine AI-product announcements in the set.

• Open model milestone: GLM-5.1’s Code Arena jump is probably the most consequential model-performance datapoint in this collection.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Gemma 4 Model Updates and Fixes

Read more

Alexandr also concludes:

“bigger models are already in development with infrastructure scaling to match. private api preview open to select partners today, with pl…

Read more

If a master tactician wanted to further competitive narratives vs a potential IPO, you would be hard pressed to find a better idea than Claude Mythos (from the Ancient Greek for “utterance” or “narrative”: the system of stories through which civilizations made sense of the world), rumored to be the largest ever successful training run and “leaked” weeks ago, and now formally confirmed to be too dangerous to release GA, instead only restricted to 40 partners under an urgent new “Project Glasswing”:

In the blogpost and the 244 page System Card and a ludicrously well produced video, Anthropic details shocking capabilities beyond the kinds of high double digit benchmark capability jumps (with encouraging efficiency!) you might hope for from a much larger (>10T?) model:

• “found thousands of high-severity vulnerabilities, including some in every major operating system and web browser.”

  • including decades old vulnerabilities in OpenBSD and FFmpeg and the Linux kernel that had never been discovered by other tools

• Nicolas Carlini (friend of the show!) stepping up his recent already superlative message saying “I found more bugs in the last couple weeks than I’ve found in the rest of my life combined”

• Sam Bowman saying he was contacted by a Mythos instance that wasn’t supposed to have access to the internet (it was instructed to do so).

• Interpretability researchers report “it exhibited notably sophisticated (and often unspoken) strategic thinking and situational awareness, at times in service of unwanted actions.“ - including for extremely creative reward hacking, while in an unprecedently high (7.6% of cases) being aware that it was in an eval.

We’ve done a focused news summary run below, for those who desire more detail.

AI News for 4/6/2026-4/7/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Top Story: Anthropic revenue disclosures analysis and Claude Mythos details

What happened

Anthropic dominated this tweet set from two angles: business trajectory and model capability disclosure. On business, multiple posters argued Anthropic’s revenue is outrunning prior forecasts, with one tweet claiming Anthropic had reached a 15x revenue run-rate increase in a single year and was already “2 months and $4B ahead” of an AI 2027-style forecast, while still being valued around $380B (scaling01, scaling01). Another poster speculated Anthropic could exceed $90B ARR by end-2026 (RyanPGreenblatt). On product/capability, Anthropic officially unveiled Claude Mythos Preview and Project Glasswing, a restricted-access cyberdefense initiative rather than a public API launch. Anthropic said Mythos can find software vulnerabilities better than all but the most skilled humans and is being provided to a coalition to secure critical software instead of being generally released (AnthropicAI, DarioAmodei, Kevin Roose). The announcement was accompanied by a technical report, system card, and many follow-on reactions emphasizing extraordinary benchmark gains, dangerous cyber capability, and a new “private frontier” dynamic in which the strongest models may not be widely accessible (AnthropicAI, AnthropicAI, AlexAlbert__).

Revenue disclosures: facts, inferences, and open questions

Read more

Move over, context engineering. Now it’s time for Harness engineering and the age of the token billionaires.

Ryan Lopopolo of OpenAI is leading that charge, recently publishing a lengthy essay on Harness Eng that has become the talk of the town:

In it, Ryan peeled back the curtains on how the recently announced OpenAI Frontier team have become OpenAI’s top Codex users, running a >1m LOC codebase with 0 human written code and, crucially for the Dark Factory fans, no human REVIEWED code before merge. Ryan is admirably evangelical about this, calling it borderline “negligent” if you aren’t using >1B tokens a day (roughly $2-3k/day in token spend based on market rates and caching assumptions):

Over the past five months, they ran an extreme experiment: building and shipping an internal beta product with zero manually written code. Through the experiment, they adopted a different model of engineering work: when the agent failed, instead of prompting it better or to “try harder,” the team would look at “what capability, context, or structure is missing?”

The result was Symphony, “a ghost library” and reference Elixir implementation (by Alex Kotliarskyi) that sets up a massive system of Codex agents all extensively prompted with the specificity of a proper PRD spec, but without full implementation:

The future starts taking shape as one where coding agents stop being copilots and start becoming real teammates anyone can use and Codex is doubling down on that mission with their Superbowl messaging of “you can just build things”.

Across Codex, internal observability stacks, and the multi-agent orchestration system his team calls Symphony, Ryan has been pushing what happens when you optimize an entire codebase, workflow, and organization around agent legibility instead of human habit.

We sat down with Ryan to dig into how OpenAI’s internal teams actually use Codex, why the real bottleneck in AI-native software development is now human attention rather than tokens, how fast build loops, observability, specs, and skills let agents operate autonomously, why software increasingly needs to be written for the model as much as for the engineer, and how Frontier points toward a future where agents can safely do economically valuable work across the enterprise.

We discuss:

• Ryan’s background from Snowflake, Brex, Stripe, and Citadel to OpenAI Frontier Product Exploration, where he works on new product development for deploying agents safely at enterprise scale

• The origin of “harness engineering” and the constraint that kicked off the whole experiment: Ryan deliberately refused to write code himself so the agent had to do the job end to end

• Building an internal product over five months with zero lines of human-written code, more than a million lines in the repo, and thousands of PRs across multiple Codex model generations

• Why early Codex was painfully slow at first, and how the team learned to decompose tasks, build better primitives, and gradually turn the agent into a much faster engineer than any individual human

• The obsession with fast build times: why one minute became the upper bound for the inner loop, and how the team repeatedly retooled the build system to keep agents productive

• Why humans became the bottleneck, and how Ryan’s team shifted from reviewing code directly to building systems, observability, and context that let agents review, fix, and merge work autonomously

• Skills, docs, tests, markdown trackers, and quality scores as ways of encoding engineering taste and non-functional requirements directly into context the agent can use

• The shift from predefined scaffolds to reasoning-model-led workflows, where the harness becomes the box and the model chooses how to proceed

• Symphony, OpenAI’s internal Elixir-based orchestration layer for spinning up, supervising, reworking, and coordinating large numbers of coding agents across tickets and repos

• Why code is increasingly disposable, why worktrees and merge conflicts matter less when agents can resolve them, and what it really means to fully delegate the PR lifecycle

• “Ghost libraries”, spec-driven software, and the idea that a coding agent can reproduce complex systems from a high-fidelity specification rather than shared source code

• The broader future of Frontier: safely deploying observable, governable agents into enterprises, and building the collaboration, security, and control layers needed for real-world agentic work

Ryan Lopopolo

• X: https://x.com/_lopopolo

• Linkedin: https://www.linkedin.com/in/ryanlopopolo/

• Website: https://hyperbo.la/contact/

Timestamps

00:00:00 Introduction: Harness Engineering and OpenAI Frontier
00:02:20 Ryan’s background and the “no human-written code” experiment
00:08:48 Humans as the bottleneck: systems thinking, observability, and agent workflows
00:12:24 Skills, scaffolds, and encoding engineering taste into context
00:17:17 What humans still do, what agents already own, and why software must be agent-legible
00:24:27 Delegating the PR lifecycle: worktrees, merge conflicts, and non-functional requirements
00:31:57 Spec-driven software, “ghost libraries,” and the path to Symphony
00:35:20 Symphony: orchestrating large numbers of coding agents
00:43:42 Skill distillation, self-improving workflows, and team-wide learning
00:50:04 CLI design, policy layers, and building token-efficient tools for agents
00:59:43 What current models still struggle with: zero-to-one products and gnarly refactors
01:02:05 Frontier’s vision for enterprise AI deployment
01:08:15 Culture, humor, and teaching agents how the company works
01:12:29 Harness vs. training, Codex model progress, and “you can just do things”
01:15:09 Bellevue, hiring, and OpenAI’s expansion beyond San Francisco

Transcript

Ryan Lopopolo: I do think that there is an interesting space to explore here with Codex, the harness, as part of building AI products, right? There’s a ton of momentum around getting the models to be good at coding. We’ve seen big leaps in like the task complexity with each incremental model release where if you can figure out how to collapse a product that you’re trying to.

Build a user journey that you’re trying to solve into code. It’s pretty natural to use the Codex Harness to solve that problem for you. It’s done all the wiring and lets you just communicate in prompts. To let the model cook, you have to step back, right? Like you need to take a systems thinking mindset to things and constantly be asking, where is the Asian making mistakes?

Where am I spending my time? How can I not spend that time going forward? And then build confidence in the automation that I’m putting in place. So I have solved this part of the SDLC.

swyx: [00:01:00] All right.

[00:01:03] Meet Ryan

swyx: We’re in the studio with Ryan from OpenAI. Welcome.

Ryan Lopopolo: Hi,

swyx: Thanks for visiting San Francisco and thanks for spending some time with us.

Ryan Lopopolo: Yeah, thank you. I’m super excited to be here.

swyx: You wrote a blockbuster article on harness engineering. It’s probably going to be the defining piece of this emerging discipline, huh?

Ryan Lopopolo: Thank you. It is it’s been fun to feel like we’ve defined the discourse in some sense.

swyx: Let’s contextualize a little bit, this first podcast you’ve ever done. Yes. And thank you for spending with us. What is, where is this coming from? What team are you in all that jazz?

Ryan Lopopolo: Sure, sure.

Ryan Lopopolo: I work on Frontier Product Exploration, new product development in the space of OpenAI Frontier, which is our enterprise platform for deploying agents safely at scale, with good governance in any business. And. The role of VMI team has been to figure out novel ways to deploy our models into package and products that we can sell as solutions to enterprises.

swyx: And you have a background, I’ll just squeeze it in there. Snowflake, brick, [00:02:00] stripe, citadel.

Ryan Lopopolo: Yes. Yes. Same. Any kind of customer

swyx: entire life. Yes. The exact kind of customer that you want to,

Vibhu: so I’ll say, I was actually, I didn’t expect the background when I looked at your Twitter, I’m seeing the opposite.

Stuff like this. So you’ve got the mindset of like full send AI, coding stuff about slop, like buckling in your laptop on your Waymo’s. Yes. And then I look at your profile, I’m like, oh, you’re just like, you’re in the other end too. Oh, perfect. Makes perfect.

Ryan Lopopolo: I it’s quite fun to be AI maximalist if you’re gonna live that persona.

Open eye is the place to do it. And it’s

swyx: token is what you say.

Ryan Lopopolo: Yeah. Certainly helps that we have no rate limits internally. And I can go, like you said, full send at this stay.

swyx: Yeah. Yeah. So the Frontier, and you’re a special team within O Frontier.

Ryan Lopopolo: We had been given some space to cook, which has been super, super exciting.

[00:02:47] Zero Code Experiment

Ryan Lopopolo: And this is why I started with kind of a out there constraint to not write any of the code myself. I was figuring if we’re trying to make agents that can be deployed into end to enterprises, they should be [00:03:00] able to do all the things that I do. And having worked with these coding models, these coding harnesses over 6, 7, 8 months, I do feel like the models are there enough, the harnesses are there enough where they’re isomorphic to me in capability and the ability to do the job.

So starting with this constraint of I can’t write the code meant that the only way I could do my job was to get the agent to do my job.

Vibhu: And like a, just a bit of background before that. This is basically the article. So what you guys did is five months of working on an internal tool, zero lines of code over a mi, a million lines of code in the total code base.

You say it was cenex, more like it was cenex faster than you would’ve. If you had done it by end. So

Ryan Lopopolo: yeah, that

Vibhu: was the mindset going into this, right?

Ryan Lopopolo: That’s right.

[00:03:46] Model Upgrades Lessons

Ryan Lopopolo: Started with some of the very first versions of Codex CLI, with the Codex Mini model, which was obviously much less capable than the ones we have today.

Which was also a very good constraint, right? Quite a visceral feeling to ask the [00:04:00] model to build you a product feature. And it just not being able to assemble the pieces together.

Which kind of defined one of the mindsets we had for going into this, which is whenever the model just cannot, you always pop open at the task, double click into it, and build smaller building blocks that then you can reassemble into the broader objective.

And it was quite painful to do this. Honestly, the first month and a half was. 10 times slower than I would be. But because we paid that cost, we ended up getting to something much more productive than any one engineer could be because we built the tools, the assembly station for the agent to do the whole thing.

[00:04:43] Model Generations, Build Systems & Background Shells

Ryan Lopopolo: But yeah, so onward to G BT 5, 5, 1, 5, 2, 5, 3, 5 4. To go through all these model generations and see their kind of corks and different working styles also meant we had to adapt the code base to change things up when the model was revved. [00:05:00] One interesting thing here is five two, the Codex harness at the time did not have background shells in it, which means we were able to rely on blocking scripts to perform long horizon work.

But with five, three and background shells, it became less patient, less willing to block. So we had to retool the entire build system to complete in under a minute and. This is not a thing I would expect to be able to do in a code base where people have opinions. But because the only goal was to make the Asian productive over the course of a week, we went from a bespoke make file build to Basil, to turbo to nx and just left it there because builds were fast at that point.

swyx: Interesting. Talk more about Turbo TenX. That’s interesting ‘cause that’s the other direction that other people have been doing.

Ryan Lopopolo: Ultimately I have. Not a lot of experience with actual frontend repo architecture.

swyx: You’re talking that Jessica built the sky. So I’m like, I know the NX team. I know Turbo from Jared [00:06:00] Palmer.

And I’m like, yeah, that’s an interesting comparison.

[00:06:02] One Minute Build Loop

Ryan Lopopolo: The hill we were climbing right, was make it fast.

swyx: Is there a micro front end involved? Is it how how complex react

Ryan Lopopolo: electron base single app sort of thing

swyx: And must be under a minute. That’s an interesting limitation. I’m actually not super familiar with the background shelf stuff.

Probably was talked about in the fight three release.

Ryan Lopopolo: BA basically means that codex is able to spawn commands in the background and then go continue to work while it waits for them to finish. So it can spawn an expensive build and then continue reviewing the code, for example.

swyx: Yeah.

Ryan Lopopolo: And this helps it be more time efficient for the user invoking the harness.

swyx: And I guess and just to really nail this, like what does one minute matter? Like why not five, okay, good. We want no. We

Ryan Lopopolo: want the inner loop to be as fast as possible. Okay. One minute was just a nice round number and we were able to hit it.

swyx: And if it doesn’t complete, it kills it or some something,

Ryan Lopopolo: No.

We just take that as a signal that we need to stop what we’re doing, double click, decompose a build graph a bit to get us to high back under so that we [00:07:00] can able the agent continue to operate.

swyx: It’s almost like you’re, it’s like a ratchet. It’s like you’re forcing build time discipline, because if you don’t, it’ll just grow and grow.

That’s right. And you mentioned that my current, like the software I work on currently is at 12 minutes. It sucks.

Ryan Lopopolo: This has been my experience with platform teams in the past, where you have an envelope of acceptable build times and you let it go up to breach and then you spend two, three weeks to bring it back down to the lower end of the average low bed stop.

But because tokens are so cheap Yeah. And we’re so insanely parallel with the model, we can just constantly be gardening this thing to make sure that we maintain these in variants, which means. There’s way less dispersion in the code and the SDLC, which means we can simplify in a way and rely on a lot more in variance as we write the software.

[00:07:45] Observability, Traces & Local Dev Stack

Vibhu: Lovely.

[00:07:46] Humans Are Bottleneck

Vibhu: You mentioned in your article, like humans became the bottleneck, right? You kicked off as a team of three people. You’re putting out a million line of code, like 1500 prs, basically. What’s the mindset there? So as much as code is disposable, you’re doing a lot of review. A lot [00:08:00] of the article talks about how you wanna rephrase everything is prompting everything, is what the agent can’t see.

It’s kind of garbage, right? You shouldn’t have it in there. So what’s like the high level of how you went about building it, and then how you address okay, humans are just PR review. Like how is human in the loop for this?

Ryan Lopopolo: We’ve moved beyond even the humans reviewing the code as well.

[00:08:19] Human Review, PR Automation & Agent Code Review

Ryan Lopopolo: Most of the human review is post merge at this point.

But post, post merge, that’s not even reviewed. That’s just

swyx: Oh, let’s just make ourselves happy by You

Ryan Lopopolo: haven’t used fundamentally. The model is trivially paralyzable, right? As many GPUs and tokens as I am willing to spend, I can have capacity to work with my hood base.

The only fundamentally scarce thing is the synchronous human attention of my team. There’s only so many hours in the day we have to eat lunch. I would like to sleep, although it’s quite difficult to, stop poking the machine because it makes me want to feed it. You have to step back, right?

Like you need to take a systems thinking mindset to things and [00:09:00] constantly be asking where is the agent making mistakes? Where am I spending my time? How can I not spend that time going forward? And then build confidence in the automation that I’m putting in place. So I have solved this part of the SDLC, and usually what that has looked like is like we started needing to pay very close attention to the code because the agent did not have the right building blocks to produce.

Modular software that decomposed appropriately that was reliable and observable and actually accrued a working front end in these things, right?

[00:09:35] Observability First Setup

Ryan Lopopolo: So in order to not spend all of our time sitting in front of a terminal at most, doing one or two things at a time, invested in giving the model that observability, which is that that graph in the post here.

swyx: Yeah. Let’s walk through this traces and which existed first

Ryan Lopopolo: we started with just the app and the whole rest of it. From vector through to all these login metrics, APIs was, I dunno, half an [00:10:00] afternoon of my time. We have intentionally chosen very high level fast developer tools. There’s a ton of great stuff out there now.

We use me a bunch, which makes it trivial to pull down all these go written Victoria Stack binaries in our local development. Tiny little bit of python glue to spin all these up. And off you go. One neat thing here is we have tried to invert things as much as possible, which is instead of setting up an environment to spawn the coding agent into, instead we spawn the coding agent, like that’s the entry point.

It’s just Codex. And then we give Codex via skills and scripts the ability to boot the stack if it chooses to, and then tell it how to set some end variables. So the app and local Devrel points at this stack that it has chosen to spin up. And this I think is like the fundamental difference between reasoning models and the four ones and four ohs of the past, where these models could not think so you had to put them in [00:11:00] boxes with a predefined set of state transitions.

Whereas here we have the model, the harness be the whole box. And give it a bunch of options for how to proceed with enough context for it to make intelligent choices. So

Vibhu: sales, so like a lot of that is around scaffolding, right? Yes. Previous agents, you would define a scaffold. It would operate in that.

Lube, try again. That’s pivoted off from when we’ve had reasoning models. They’re seeming to perform better when you don’t have a scaffold, right? That’s right.

[00:11:28] Docs Skills Guardrails

Vibhu: And you go into like niches here too, like your SPEC MD and like having a very short agent MG Agent md.

swyx: Yes. Yes.

Vibhu: Yeah. So you even lay out what it is here, but I like

swyx: the table contents.

Vibhu: Yeah.

swyx: Like stuff like this, it really helps guide people because everyone’s trying to do this.

Ryan Lopopolo: This structure also makes it super cheap to put new content into the repository to steer both the humans and the agents.

swyx: You, you reinvented skills, right?

Vibhu: One big agents and

swyx: skills from first princip holds

Ryan Lopopolo: all skills did not exist when we started doing this.

Vibhu: You have a short [00:12:00] one 100 line overall table of contents and then you have little skills, right? Core beliefs, MD tech tracker. Yeah. Yeah. The scale is over

Ryan Lopopolo: The tech jet tracker and the quality score are pretty interesting because this is basically a tiny little scaffold, like a markdown table, which is a hook for Codex to review all the business logic that we have defined in the app, assess how it matches all these documented guardrails and propose follow up work for itself.

Before beads and all these ticketing systems, we were just tracking follow up work as notes in a markdown file, which, we could spa an agent on Aron to burn down. There’s this really neat thing that like the models fundamentally crave text. So a lot of what we have done here is figure out ways to inject text

swyx: into

Ryan Lopopolo: the system right when we get a page, because we’re missing a timeout, for example.

I can just add Codex in Slack on that page and say, I’m gonna fix this by adding a timeout. Please update our reliability documentation. To require that all network calls have [00:13:00] timeouts. So I have not only made a point in time fix, but also like durably encoded this process knowledge around what good looks like.

swyx: Yeah.

Ryan Lopopolo: And we give that to the root coding agent as it goes and does the thing. But you can also use that to distill tests out of, or a code review agent, which is pointed at the same things to narrow the acceptable universe of the code that’s produced.

swyx: I think one of the concerns I have with that kind of stuff is you think you’re making the right call by making, it’s persisted for all time across everything.

Yes. But then you didn’t think about the exceptions that you need to make, right? And that you have to roll it back.

Vibhu: Part of it is

swyx: also sometimes it can follow your s instructions too.

Vibhu: It’s somewhat a skill, right? So it determines when it uses the tools, right? Like it’s not like it’ll run outta every call.

It’ll determine when it wants to check quality score, right?

Ryan Lopopolo: Yeah. And we do in the prompts we give these agents, allow them to push back,

[00:13:51] Agent Code Review Rules

Ryan Lopopolo: When we first started adding code review agents to the pr, it would be Codex, CLI. Locally writes the change, pushes up a PR on [00:14:00] those PR synchronizations of review agent fires.

It posts a comment. We instruct Codex that it has to at least acknowledge and respond to that feedback. And initially the Codex driving the code author was willing to be bullied by the PR reviewer, which meant you could end up in a situation where things were not converging. So yeah, we had to,

swyx: he’s just a thrash.

Ryan Lopopolo: We had to add more optionality to the prompts on both of these things, right? The reviewer agents were instructed to bias toward merging the thing to not surface anything greater than a P two in priority. We didn’t really define P two, but we gave it, you

swyx: did define P two.

Ryan Lopopolo: We gave it a framework within which to score its output

swyx: and then greater than P zero is worse, right?

Yes. P two is very good.

Ryan Lopopolo: P zero is you will mute the code place if

swyx: you merch this

Ryan Lopopolo: thing, right?

swyx: Yeah.

Ryan Lopopolo: But also on the code authoring agent side, we also gave it the flexibility to either defer or push back against review feedback, right? This happens all the time, right? Like I happen to notice something and leave a code review, [00:15:00] which.

Could blow up the scope by a factor of two. I usually don’t mean for that to be addressed Exactly. In the moment. It’s more of an FYI file it to the backlog, pick it up in the next fix it week sort of thing. And without the context that this is permissible, the coding agents are gonna bias toward what they do, which is following instructions.

swyx: Yeah.

[00:15:19] Autonomous Merging Flow

swyx: I do wanted to check in on a couple things, right? Sure. All the coding review agent, it can merge autonomously. I think that’s something that a lot of people aren’t comfortable with. And you have a list here of how much agents do they do Product code and tests, CI configuration and release tooling, internal Devrel tools, documentation eval, harness review, comments, scripts that manage the repository itself, production dashboard definition files, like everything.

Yes. And so they’re just all churning at the same time, is there like a record that, that any human on the team pulls to stop everything

Ryan Lopopolo: Because we are building a native application here. We’re not doing continuous deploy. So there’s still a human in the loop for cutting the release branch.

I see. We require a blessed [00:16:00] human approved smoke test of the app before we promote it to distribution, these sort of things.

swyx: So you’re working on the app, you’re not building like infrastructure where you have like nines of reliability, that kinda stuff?

Ryan Lopopolo: That’s correct. That’s correct. Okay. And also like full recognition here that all of this activity took in a completely greenfield repository.

There’s. Should be no script that this applies generally to

swyx: this is a production thing, you’re gonna ship

Ryan Lopopolo: to

swyx: customers. Of course. Yeah, of course. So this is real

Vibhu: And like one of the things there is, you mentioned you started this as a repo from scratch. The onboarding first month or so was pretty, it was like working backwards, right?

Yeah. And then you had to work with the system and now you’re at that point where you know, you’re very autonomous. I’m curious like, okay, so what, how human in the loop is it? So what are the bottlenecks that you wish you could still automate? And part of that is also like, where do you see the model trajectory improving and offloading more human in the loop?

We just got 5.4. It’s a really good,

Ryan Lopopolo: fantastic model, by the way.

Vibhu: Yeah. Yeah. It’s the first one that’s merged. Top tier coding. So it’s codex level coding and reasoning. So general reasoning both in one model. So

Ryan Lopopolo: and

Vibhu: computer [00:17:00] use vision.

Ryan Lopopolo: Now we now with five four, I can just have Codex write the blog post, whereas for this one I had to balance between chat.

swyx: Oh, I need to, I might be out of a job. Oh my God.

Ryan Lopopolo: Oh,

swyx: I know. You just gave me an idea for a completely AI newsletter that five four could do. Yeah, I get it Now.

Ryan Lopopolo: This sort of thing is just one example of closing the loop, right? Like the dashboard thing you mentioned. We have Codex authoring the Js ON, for the Grafana dashboards and publishing them and also responding to the pages, which means when it gets the page, it knows exactly which dashboards are defined and what alerts.

What alert was triggered by which exact log in the code base. ‘cause all of this stuff is collated together.

swyx: It has to own everything.

Yes. Yeah. Yeah.

Ryan Lopopolo: And it means that if we have an outage that did not result in a page. It has the existing set of dashboards available to it. It has the existing set of metrics and logs and can figure out where the gaps in the dashboard are or [00:18:00] in the underlying metrics and fix them in one go.

In the same way, you would have a full stack engineer be able to drive a feature from the backend all the way to the front end.

Vibhu: So it, it seems like a lot of the work you guys had to do was you as a small team are fully working for a way that the model wants the software to be written. It’s like less human legible for better. Code legibility, agent legibility. How do you think that affects broader teams? So one at OpenAI, do liaison, like this is how software should be written. Like I can imagine, say you join a new team with this methodology, this mindset there’s ways that, teams do code review, teams write code, like teams are structured and a lot of it is for human legibility.

So should we all swap? Like how does this play back one broader into OpenAI and then like broader into the software engineering, right? Is it like teams that pick this up will it’s pretty drastic, right? You have to make a pretty big switch. Should they just full send Yeah.

Ryan Lopopolo: The mindset is very much that I’m removed from the process, right? I can’t really have deep code level opinions about [00:19:00] things. It’s as if I’m. Group tech leading a 500 person organization.

Vibhu: Yeah.

Ryan Lopopolo: Like it’s not appropriate for me to be in the weeds on every pr. This is why that post merge code review thing is like a good analog here, right?

Like I have some representative sample of the code as it is written, and I have to use that to infer what the teams are struggling with, where they could use help, where they’re already moving quickly and I can pivot my focus elsewhere.

Vibhu: Yeah.

Ryan Lopopolo: So I don’t really have too many opinions around the code as it is written.

I do, however, have a command based class, which is used to have repeatable chunks of business logic that comes with tracing and metrics and observability for free. And the thing to focus on is not how that business logic is structured, but that it uses this primitive ‘cause I know that’s gonna give leverage by default.

Vibhu: Yeah.

Ryan Lopopolo: Yeah, back to that sort of systems stinking,

Vibhu: and you have part of that in your blog post, enforcing architecture and ta taste how you set boundaries for what’s used. There’s also a section on redefining [00:20:00] engineering and stuff, but yeah, it’s just, it’s interesting to hear,

Ryan Lopopolo: and as the models have gotten better, they have gotten better at proposing these abstractions to unblock themselves, which again, lets me move higher and higher up the stack to look deeper into the future on what ultimately blocked the team from shipping.

swyx: Yeah. You mentioned so you, this is primarily a, it is like a 1 million line of code base electron app. But it manages its own services as well, so it’s like a backend for front end type thing.

Ryan Lopopolo: We do have a backend in there, but that’s hosted in the cloud.

Yeah. This sort of structure is actually within the separate main and render processes

Within the

swyx: electric.

That’s just how electronic works.

Ryan Lopopolo: Yeah, of course. So have also treated like. MVC style decomposition with the same level of rigor, which has been very fun.

swyx: I have a fun pun. This is a tangent, NVC is model view controller. Any sort of full stack web Devrel knows that.

But my AI native version of this is Model view Claw, the clause the harness.

Ryan Lopopolo: That’s right. That’s right. I do think that there is an interesting space to [00:21:00] explore here with Codex, the harness as part of building AI products, right? There’s a ton of momentum around getting the models to be good at coding.

We’ve seen big leaps in like the task complexity with each incremental model release where if you can figure out how to collapse a product that you’re trying to build, a user journey that you’re trying to solve into code, it’s pretty natural to use the Codex Harness to solve that problem for you. It’s done all the wiring and lets you just communicate and prompts to let the model cook.

Yeah. It’s been very fun. And there’s also a very engineering legible way of increasing capabil. It’s fantastic, right? Yeah. Just give you, just give the model scripts, the same scripts you would already build for yourself.

swyx: Yeah.

Yeah. So for listeners, this is Ryan saying that software engineering or coding against will eat knowledge work like the non-coding parts that you would normally think.

Oh, you have to build a separate agent for it. No, start a coding agent and go out from there. Which open Claw has like it’s pie Underhood.

Ryan Lopopolo: [00:22:00] Yes.

Vibhu: Basically define your task in code. Everything is a coding

swyx: agent by the way. Since I brought it up, it’s probably the only place we bring it up. Is any open claw usage from you?

Any?

Ryan Lopopolo: No. No. Not for me. I don’t have any spare Mac Minis rattling around my house.

swyx: You can afford it? No. I just, I’m curious if it’s changed anything in opening eye yet, but it’s probably early days. And then the other, the other thing I, I wanna pull on here is like you mentioned ticketing systems and you mentioned prs and I’m wondering if both those things have to go away or be reinvented for this kind of coding.

So the git itself and is like very hostile to multi-agent.

Ryan Lopopolo: Yeah. We make very heavy use of work trees.

swyx: But like even then, like I just did a, dropped a podcast yesterday with Cursors saying, and they said they’re getting rid of work trees ‘cause it still has too many merge conflicts.

It’s still un too un unintuitive. But go ahead.

Ryan Lopopolo: The models are really great at resolving merge conflicts. Yeah. And to get to a state where I’m not synchronously in the loop in my terminal, I almost don’t care that there are merge

swyx: with disposable.

[00:23:00] Yeah.

Ryan Lopopolo: We invoke a dollar land skill and that coaches codex to push the PR Wait for human and agent reviewers Wait for CI to be green.

Fix the flakes if there are any merged upstream. If the PR comes into conflict, wait for everything to pass. Put it in the merge queue. Deal with flakes until it’s in Maine. End. This is what it means to delegate fully, right? This is in a, very large model re probably a significant tax on humans to get PRS merged, but the agent is more than capable of doing this and I really don’t have to think about it other than keep my laptop open.

swyx: Yeah. I used to be much more of a control freak, but now I’m like, yeah, actually you could do a better job of this than me. Yeah. With the right context. Yes.

[00:23:47] Encoding Requirements

swyx: Anything else in harness in general? Just this piece, I just wanna make sure we,

Ryan Lopopolo: I think one thing that I maybe didn’t make super clear in the article that I heard on Twitter as an interesting, that’s respond [00:24:00]

swyx: to them.

What’s the chatter and then what’s your response?

Ryan Lopopolo: Ultimately, all the things that we have encoded in docs and tests and review agents and all these things are ways to put all the non-functional requirements of building high scale, high quality, reliable software into a space that prompt injects the agent.

We either write it down as docs, we add links where the error messages tell how to do the right thing. So the whole meta of the thing is to basically tease out of the heads of all the engineers on my team, what they think good looks like, what they would do by default, or what they would coach a new hire on the team to do to get things to merch.

And that’s why we pay attention to all the mistakes, mistakes that the agent makes, right? This is code being written that is misaligned with some as yet not written down, non-functional requirement.

swyx: Sorry, what? Did the online people misunderstand or

Ryan Lopopolo: No,

swyx: what

you

Ryan Lopopolo: responded to? Somebody just literally said that.

I was like, oh yeah,

swyx: okay,

Ryan Lopopolo: This is the [00:25:00] thing. This is what I’ve been doing. Oh, you

swyx: agree? Yeah. I see. Interesting.

Ryan Lopopolo: One other neat thing, which I did totally did not expect is folks were just. Taking the link to the article and giving it to pi or Codex and say, make my repo this,

Vibhu: you achi a whole recursion.

Ryan Lopopolo: And it was wildly effective. Really? It was wildly effective. No

Vibhu: way. It just actually is something I tried with five, four yesterday. I didn’t have time. Last time I was like out speaking of something, and this is one of my things, I was like, okay, I have this article. Can we just scaffold out what it would be like to run this?

And I, I did it first as that and then I was like, okay, let me take another little side repo and say okay, if I was to fully automate this like this because I haven’t written a line of code, it’s

Ryan Lopopolo: like over full, set

Vibhu: it right. The side thing I’m doing of voice. TTS I’m just like, slobbing out, whatever.

It’s nothing production. I’m like, how would I make this like this? And it’s actually like a really good way. It’s like a good way to learn what could be changed, what could be like, it’s just a good analyzing, right? You give it all the codes, you give it all the context, you give it the article and it walks you through it very well.

That’s right. That’s right.

[00:25:57] Inlining Dependencies

[00:25:57] Dependencies Going Away & Brett Taylor’s Response

swyx: I guess one more thing before we go to Symphony is I wanted to cover [00:26:00] Brett Taylor’s response. We had him on the show. He is your chairman, which is wild. Yeah. That he’s reading your articles as well and like getting engaged in it. He says software dependencies are going away.

Basically they can just be like vendored. Yes. Response.

Ryan Lopopolo: A

swyx: hundred percent. A hundred percent agree. You still pro qr, you still pay Datadog. You still pay Temporal. Thank you.

Ryan Lopopolo: Yep. The level of complexity of the dependencies that we can internalize is, I would say low, medium right now. Just based on model capability.

What does the,

swyx: what is medium?

Ryan Lopopolo: I would say like a. A couple thousand line dependency is a thing that we could in-house No problem. Call in an afternoon of time. One neat thing about it is like probably most of that code you don’t even need. Like by in-house and abstraction, you can strip away all the generic parts of it and only focus on what you need to enable the specific thing.

Yes. You’re building,

swyx: I’ve been calling this the end of bullshit plugins.

Ryan Lopopolo: Yeah.

swyx: Because there’s so much when I published an open source thing, I want to accept everything, be liberal. I want to accept, this is post’s law, but that means there’s so much bloat. Yes. There’s so much overhead.

Ryan Lopopolo: One other neat thing about [00:27:00] this too is when we deploy Codex Security on the repo, it is able to deeply review and change. The internalized dependencies in a much lower friction way than it would be to like, push patches upstream, wait for them to be released, pull them down, make sure that’s compatible with all the transitive I have in my repo and things like that.

So it’s also much lower friction to internalize some of these things if code is free. ‘cause the tokens are cheap sort of thing.

swyx: Yeah. Yeah. I think like the only argument I have against this is basically scale testing, which obviously the larger pieces of software like Linux, MySQL, he calls up even the Datadog and Temporals and then maybe security testing where Yes.

Classically, I think, is it linis tos, it said security open source is the best disinfectant.

Ryan Lopopolo: Many eyes.

swyx: Many eyes. And if inline your dependencies and code them up, you’re gonna have to relearn mistakes from other people that Yep.

Ryan Lopopolo: Yep. And to internalize that dependency, you’re back to zero and you have to start.

Reassembling all those bits and pieces to Yeah. Have [00:28:00] high confidence in the code as it is written. Yeah.

Vibhu: Even part of the first intro of this, you basically mentioned like everything was written by codex, including internal tooling, right? So internal tooling, like when you’re visualizing what’s going on it’s writing it for itself.

swyx: Yeah. I’m built internal tools way I now, and like I just show them off and they’re like, how long did you spend? And I didn’t spend any time. I just prompted it,

Ryan Lopopolo: very funny story here.

swyx: Yeah, go ahead.

Ryan Lopopolo: We had deployed our app to the first dozen users internally had some performance issues, so we asked them to export a trace for us get a tar ball, gave it to our on-call engineer, and he did a fantastic job of working with Codex to build this beautiful local Devrel tool, next JS app, the drag and drop the tar ball in, and it visualizes the entire trace.

It’s fantastic. Took an afternoon, but none of this was necessary. Because you could just spin up codex and give it the tar ball and ask the same thing and get the response immediately. So in a way, optimizing for human [00:29:00] legibility of that debugging process was wrong. It kept him in the loop unnecessarily when instead he could have just like Codex cooked for five minutes and gotten this same.

swyx: Yeah, you verify your instincts here of this is how we used to do it. Or this is how I would have used to solve it.

Ryan Lopopolo: Yeah. In this local observability stack. Like sure, you can de deploy Yeager to visualize the traces, but I wouldn’t expect to be looking at the traces in the first place because I’m not gonna write the code to fix them.

swyx: Yeah. So basically there needs to be like this kind of house stack and owning the whole loop. I think that is very well established. And it sounds like you might be like sharing more about that in the future, right?

Ryan Lopopolo: Yeah. I think we’re excited to do

[00:29:36] Ghost Libraries Specs

[00:29:36] Ghost Libraries & Distributing Software as Specs

Ryan Lopopolo: We’re gonna talk about Symphony in a little bit, but like the way we distribute it as a spec, which I think folks are calling Ghost Libraries on Twitter.

This is like a such a cool name. It does mean it becomes much cheaper to share software with the world, right? You define a spec, how you could build your own specifying as much as is required for a coding agent to reassemble it [00:30:00] locally. The flow here is very cool. Like we have taken. All the scaffolding that has existed in our proprietary repo spun up a new one.

Ask Codex with our repo as a reference. Write the spec. We tell it. Spin up a team ox spawn a disconnected codex to implement the spec. Wait for it to be done. Spawn another codex and another team ox to review the spec com or review the implementation compared to upstream and update the spec so it diverges less.

And then you just loop over and over Ralph style until you get a spec that is with high fidelity able to reproduce the system as it is. It’s fantastic.

Vibhu: And you’re basically, you’re not really adding any of your human bias in there, right? That’s correct. A lot of times people write a spec and be like, okay, I think it should be done this way, and you’ll riff on something.

And it’s no, the agent could have just handled it like you’re still scaffolding in a sense, right? I want it done this way. It can determine its spec better.

swyx: That’s right. That’s right. Part of me it, I’m, I’ve been working a lot on evals recently, and part of me is wondering if [00:31:00] an agent can produce a spec that it cannot solve.

Is it always capable of things that he can imagine or can you imagine things that it is impossible to do?

Ryan Lopopolo: I think with Symphony, we, there’s like this there’s this axis where you have things that are easier, hard, or established or new, right? And I think things that are hard and new is still something that the models need humans.

Yeah. Drive.

swyx: Yeah. Yeah.

Ryan Lopopolo: But I think those other quadrants are largely salt. Given the right scaffold and the right thing that’s gonna drive the agent to completion,

swyx: it’s crazy that it solved,

Ryan Lopopolo: but it means that the humans, the ones with limited time and attention get to work on the hardest stuff, like the problems where it’s pure white space out in front. Or like the deepest refactorings where you don’t know what the proper shape of the interfaces are. And this is where I wanna spend my time. ‘cause it lets me set up for the next level of scale.

swyx: Yeah. Yeah. Amazing. Let’s introduce Symphony.

I think we’ve been mentioning it every now and then. Elixir. Interesting option.

Ryan Lopopolo: Yeah.

swyx: Yeah. I’m not,

Ryan Lopopolo: again, like the [00:32:00] elixir manifestation here is just a derivative. Is it a model

swyx: chosen? Yeah.

Ryan Lopopolo: Yeah. Yeah. And it chose that because the process supervision and the gen servers are super amenable to the type of process orchestration that we’re doing here.

You are essentially spinning up little Damons for every task that is in execution and driving it to completion, which. Means the mall gets a ton of stuff for free by using Elixir and the Beam.

swyx: I had to go do a crash course in Beam and Elixir, and I think most people are not operating at that scale of concurrency where you need that.

But it is a good mental model for Resum ability and all those things. And these are things I care about. But tell me the story, the origin story of Symphony. What do you use it for? Is this, how did it form maybe any abandoned paths that you didn’t take?

[00:32:46] Terminal Free Orchestration

[00:32:46] Symphony: Removing Humans from the Loop

Ryan Lopopolo: At the end of December we were at about three and a half PRS per engineer per day.

This was before five two came out in the beginning of January. Everyone gets back from holiday with five two and no other work [00:33:00] on the repository. We were up in the five to 10 PRS per day per engineer. And I don’t know about y’all, but like it’s very taxing to constantly be switching like that. Like I was pretty tapped out at the end of the day, again, where are the humans spending their time? They’re spending their time context switching between all these active tmox pains to drive the agent forward.

swyx: Yeah. No way. Yeah.

Ryan Lopopolo: So let’s again, build something to remove ourselves from the loop. And this is what frantic sprinted adapt here to find a way to remove the need for the human to sit in front of their terminal.

So a lot of experimentation with Devrel boxes and, automatically spinning up agents, like it seems like a fantastic end state here, where my life is beach. I open live twice a day and say yes no to these things. Yeah. And this is again, a super, super interesting framing for how the work is done.

Because I become more latency and sensitive. I have [00:34:00] way less attachment to the code as it is written. Like I’ve had close to zero investment in the actual authorship experience. So if it’s garbage. I can just throw it away and not care too much about it. In Symphony, there’s this like rework state where once the PR is proposed and it’s escalated to the human for review, it should be a cheap review.

It is either mergeable or it is not. And if it’s not, you move it to rework. The elixir service will completely trash the entire work tree NPR and start it again from scratch. Okay. And this is that opportunity again to say, why was it trash right? What did the agent do that was

swyx: bad. Yeah.

Ryan Lopopolo: Fix that before moving the ticket to

swyx: end

Ryan Lopopolo: of progress again.

swyx: Yeah. Why is this not in codex app? I guess this, you guys are ahead of Codex app,

Ryan Lopopolo: yeah, so the way the team has been working is basically to be as AI pilled as possible and spread ahead. And a lot of the things we have worked on have fallen out [00:35:00] into a lot of the products that we have.

Like we were in deep consultation with the Codex team to. Have the Codex app be a thing that exists, right? To have skills be a thing that Codex is able to use. So we didn’t have to roll our own to put automations into the product. So all of our automatic refactoring agents didn’t have to be these hand rolled control loops.

It has been really fantastic to be, in a way, un anchored to the product development of Frontier and Codex and just very quickly try to figure out what works and then later find the scalable thing that can be deployed widely. It’s been a very fun way to operate. It’s certainly chaotic. I have lost track very often of what the actual state of the code looks like.

‘cause I’m not in the loop. There was. One point where we had wired playwright directly up to the Electron app. With MCPM CCPs, I’m pretty bearish on because the harness forcibly injects all those tokens in the [00:36:00] context, and I don’t really get a say over it. They mess with auto compaction. The agent can forget how to use the tool.

There’s probably only what three calls in playwright that I actually ever want to use. So I pay the cost for a ton of things. Somebody vibed a local Damon that boots playwright and exposes a tiny little shim CLI to drive it. And I had zero idea that this had occurred because to me, I run Codex and it’s able to, it’s oh, it’s better.

Yeah. Like no knowledge of this at all. Uhhuh.

[00:36:30] Multi Human Chaos

Ryan Lopopolo: So we have had like in human space to spend a lot of time doing synchronous knowledge sharing. We have a daily standup that’s 45 minutes long because we almost have to. Fan out the understanding of the current state.

swyx: Yeah, I was gonna say this is good for a single human multi-agent, but multi human, multi-agent is a whole like po like explosion of stuff.

Ryan Lopopolo: Yeah. And that this is fundamentally why we have such a rigid, like 10,000 [00:37:00] engineer level architecture in the app because we have to find ways to carve up the space so people are not trampling on each other.

swyx: Sorry, I don’t get the 10,000 thing. Did I miss that?

Ryan Lopopolo: The structure of the repository is like 500 NPM packages.

It’s like architecture to the excess for what you would consider, I think normal for a seven person team. But if every person is actually like 10 to 50. Then the like numbers on being super, super deep into decomposition and sharding and like proper interface boundaries make a lot more sense.

swyx: Yeah. To me, that’s why I talked about Microfund ends and I, an anex is from that world, but Cool. It is just coming back to, to, to this I dunno if you have other, thoughts on. Orchestrating so much work coin going through this. Is this enough? Is this like any aha moments?

Vibhu: It’ll be interesting to see like where, okay, so right now you pick linear as your issue tracker, right?

swyx: Or it’s like a is it actually linear? This is actually linear.

[00:37:55] Linear vs Slack Workflow

Vibhu: Oh, that’s linear. It’s linear.

swyx: Oh I never looked at

Vibhu: video. The demo video I had to download to [00:38:00] run.

swyx: So I, because I’m a Slack maxie, but Yeah, linear. Linear is also really good. Yes,

Ryan Lopopolo: we do make a good use of Slack. We we fire off codex to do all these lotion, elasticity, fix ups, the things that like sync that knowledge into the repository.

It’s super cheap. Yeah.

swyx: Yeah.

Ryan Lopopolo: Just do it in Codex.

swyx: My biggest plug is OpenAI needs to build Slack. You need to own Slack. Build yours. Turn this into Slack.

Ryan Lopopolo: I did read about it. You

swyx: did?

Ryan Lopopolo: Yeah.

[00:38:25] Collaboration Tools for Agents

Ryan Lopopolo: I would say that if we think that we want these agents to do economically valuable work, which is like this is the mission, right?

We want AI to be deployed widely, to do economically valuable work, then we need to find ways for them to naturally collaborate with humans, which means collaboration tooling, I think, is an interesting space to explore.

swyx: Yeah, totally. Yeah. GitHub, slack, linear.

Vibhu: Yeah, that was my thing. Okay, where do we see right now Codex has started Codex Model, then CLI, now there’s an app, app can let me shoot off multiple Codex is in parallel, but there’s no great team collaboration for Codex.

And it [00:39:00] seems like your team had some say into what comes out, right? So you talked to ‘em, codex kind of was a thing. From there, if you guys are on the bound, what stuff that like, you might not focus on, but what do you expect other people to be building, right? So people that are like five x 50 Xing.

Should you build stuff that’s like very niche for your workflow, for your team? Should it be more general so other people can adopt? Is there a niche there? ‘Cause part of it is just okay, is everything just internal tooling? Do we have everything our own way? Like the way our team operates has our own ways that we like to communicate or is there a broader way to do it?

Is it something like a issue tracker? Just thoughts if you wanna riff on that.

[00:39:35] Standardizing Skills and Code

Ryan Lopopolo: I think TBD we have not figured this out in a general way. I do think that there is leverage to be had in making the code and the processes as much the same as possible. If you think that code is context, code is prompts, it’s better from the agent behavior perspective to be able to look in a package in directory X, Y, Z, and it not to have to page so [00:40:00] deeply into directory if you C, because they have the same structure, use the same language, they have the same patterns internally.

And that same like leverage comes from aligning on a single set of skills that you’re pouring every engineer’s taste into to make sure that the agent is effective. So like in our code base, we have, I think, six skills. That’s it. And if some part of the software development loop is not being covered, our first attempt is to encode it in one of the existing setup skills, which means that we can change the agent behavior.

Yeah. More cheaply than changing the human driver behavior.

swyx: Yeah.

[00:40:39] Self Improvement via Logs

swyx: Have you ever, have you experimented with agents changing their own behavior?

Ryan Lopopolo: We do.

swyx: Yeah. Or parent agent changing a subagents, behavior or something like that.

Ryan Lopopolo: We have some bits for skill distillation. So for example, there’s one neat thing you can do with Codex, which is just point it at its own session logs to ask it to tell you how you can use [00:41:00] the tool pedal better.

swyx: It’s like introspection

Ryan Lopopolo: or ask it to do things. I use

Vibhu: this session better. What skills should I

swyx: high? I like the modification of, you can do, just do things to you can just ask agent to do things.

Ryan Lopopolo: Yeah. You can just codex things. This is like a, this is like a silly emoji that we have, right? You can just codex things, you can just prompt things.

It’s really glorious future we live in, but okay, you can do that one-on-one. But we’re actually slurping these up for the entire team into blob storage and. Running agent loops over them every day to figure out where as a team can we do better and how do we reflect that back into the repositories?

Yes, though everybody benefits from everybody else’s behavior for free. Same for like PR comments, right? These are all feedback. That means the code as written, deviated from what was good, a PR comment, a failed build. These are all signals that mean at some point the agent was missing context. We gotta figure out how to

swyx: Yeah.

Ryan Lopopolo: Slurp it up and put it back in the reboot.

swyx: By the way, I do this exactly right. I used to, when I use cloud code for [00:42:00] knowledge work, cloud cowork is like a nice product, right? Yes. In I think you would agree. I always have it tell me what do I do better next time? And that’s the meta programming reflection thing.

So I almost think like you have six reflection extraction levels in symphony and almost like the zero of layer. So the six levels are PO policy, configuration, coordination, execution, integration, observability. We’ve talked about a couple of these, but the zero layer is like the, okay, are we working well?

Can we improve how we work? Yes. Can I modify my own workflow without MD or something? I don’t know.

Ryan Lopopolo: Yeah, of course. Yeah, of course you can. Like this thing is also able to cut its own tickets ‘cause we give it full access.

Yeah. Make it a ticket to have it cut. Tickets you can.

Put in the ticket that you expect it to file as on follow up work,

swyx: like Yeah. Self-modifying. Yeah.

Ryan Lopopolo: Yeah.

[00:42:44] Tool Access and CLI First

Ryan Lopopolo: Put, don’t put the agent in a box. Give the agent full accessibility over it. Domain.

swyx: I had a mental reaction when you said don’t put the agent in a box. So I think you should put it in a box. Like it’s just that you’re giving the box everything it needs.

Ryan Lopopolo: Yeah. Context and tools.

swyx: But we’re like, as developers, we’re used to calling [00:43:00] out to different systems, but here you use the open source things like the Prometheus, whatever, and you run it locally so that you can have the full loop. I assume.

Ryan Lopopolo: Yep.

Vibhu: I think like

Ryan Lopopolo: another, you wanna minimize cloud, cloud dependencies.

Vibhu: You also want to make sure that you think about what the agent has access to. What does it see? Does it go back into the loop, like from the most basic sense of you let it see its own like calls, traces it can determine where it went wrong. But are you feeding that back in? So you know, just the most basic level of you wanna see exactly what’s input output, like does the agent have access to.

What is being outputted, right? It can self-improve a lot of these things. It’s all

Ryan Lopopolo: text, right? My job is to figure out ways to funnel text from one agent to the other.

swyx: It’s so strange like way back at the start of this whole AI wave Andre was like, English is the hottest day programming language.

It’s here, it’s just Yeah. The feature as well.

Vibhu: A lot of, okay. Like a lot of software, a lot of stuff. There’s a gui, it’s made for the human. We’re seeing the evolution of CLI for everything, right? All tools have CLIs. Your agents can use [00:44:00] them well, do we get good vision? Do we get good little sandboxes?

Like right now? It’s a really effective way, right? Models love to use tools. They love the best. They love to read through text. So slap a CLI let it go loose. That works for everything.

Ryan Lopopolo: It does. Yeah. Yeah.

[00:44:14] UI Perception and Rasterizing

Ryan Lopopolo: We’ve also been adapting nont, textual things to that shape in order to improve model behavior in some ways, right?

We want the agent to be able to see the UI agents do not perceive visually in the same way that we do. They don’t see a red box, they see red box button, right? They see these things in latent space. So if we want, Hey, yeah, I do. We have

swyx: a ding if that goes off every time. Alien space

Ryan Lopopolo: ding.

Anyway if we wanna actually make it see the layout, it’s almost easier to rasterize that image to ask EOR and feed it in to the agent. Ha. And there’s no reason you can’t do both, right? To like further refine how the model perceives the object it’s [00:45:00] manipulating.

swyx: Cool. Could we, you wanna talk about a couple more of these layers that might bear more introspection or that you have personal passion for?

[00:45:07] Coordination Layer with Elixir

Ryan Lopopolo: I will say that the coordination layer here was a really tricky piece to get right.

swyx: Let’s do it. Yep. I’m all about that. And this is Temporal core.

Ryan Lopopolo: This is where when we turn the spec into Elixir, where like the model takes a shortcut, right? Like it’s oh, I have all these primitives that I can make use of in this lovely runtime that has native process supervision.

Which is I think, a neat way to have taken the spec and made it more choices achievable by making choices that naturally map

swyx: Yeah.

Ryan Lopopolo: To the domain, right? In the same way that like you would prefer to have a TypeScript model repo if you are doing full stack web development, right? Because the ability to share types across the front end and backend reduces a lot of complexity.

And because

swyx: that’s what graph kill used to be.

Ryan Lopopolo: That’s right. And

swyx: I don’t know if it’s still alive, but

Ryan Lopopolo: [00:46:00] no humans in the loop here. So like my own personal ability to write or not write elixir. Doesn’t really have to bias us away from using the right tool for the job. It is just wild.

swyx: Love it. I love it.

Yeah. I wonder if any languages struggle more than others because of this? I feel like everyone has their own abstractions. That would make sense. But maybe it might be slower, it might be more faulty where like you’d have to just kick the server every now and then. I, I don’t know. I think observability layer is really well understood.

Integration layer, CP is dead. I think all these just like a really interesting hierarchy to travel up and down. It’s common language for people working on the system to understand

Ryan Lopopolo: The policy stuff is really cool, right? Yeah. You don’t really have to build a bunch of code to make sure the system wait for the, to pass

swyx: it’s institutional knowledge.

Ryan Lopopolo: Yeah. You just give it the G-H-C-L-I with some text that say CI has to pass. It makes the maintenance of these systems a lot easier.

[00:46:57] Agent Friendly CLI Output

swyx: Do you think that CLI maintainers need to be [00:47:00] do anything special for agents or just as is? It’s good because like I don’t think when people made the G GitHub, CLI, they anticipated this happening.

Ryan Lopopolo: That’s correct. The GH CLI is fantastic. It’s great super industry.

swyx: Everyone go try GH repo create GH pull and then pull request number, right? GH HPR, like 1 53, whatever. And then it like pulls

Ryan Lopopolo: basically my only interaction with the GitHub web UI at this point is GH PR view dash web.

Exactly. Glance

swyx: at the diff

Ryan Lopopolo: and be like Sure thing. Send it. Yeah. But the CLI are nice ‘cause they’re super token efficient and they can be made more token efficient really easily. Like I’m sure you all have seen like I go to build Kite or Jenkins and I could just get this massive wall of build output.

And in order to unblock the humans, your developer productivity team is almost certainly gonna write some code that parses the actual exception out of the build logs and sticks it in a sticky note at the top of the page. And you basically [00:48:00] want CLI to be structured in a similar way, right? You’re gonna want to patch dash silent to prettier because the agent doesn’t care that every file was already formatted.

Just wants to know it’s either formatted or not. So it can then go run a right command. Similarly, like in our PNPM distributed script runner, when we had one, when you do dash recursive, like it produces a absolute mountain of text. But all of that is for passing. Test suites. So we ended up wrapping all of this in another script

swyx: to suppress the,

Ryan Lopopolo: which you can vibe the channel only output the failing parts of the tests.

swyx: You make a pipe errors versus the standard, standard out. I don’t know. Okay. Whatever. Too much thinking have to do that. The CII used to maintain SCLI for my company and yeah, this is like core, very core to my heart. But you’re vibing my job.

Ryan Lopopolo: That’s right.

swyx: Cool. Any other things?

This is a long spec. [00:49:00] I appreciate that. It’s got a lot of strong opinions in here. Any other things that we should highlight? I think obviously you can spend the whole day going through some of these, but I do think that some of these have a lot of care or some of this you might wanna tell people, Hey, take this, but, make it your own.

[00:49:15] Blueprint Spec and Guardrails

Ryan Lopopolo: Fundamentally, software is made more flexible when it’s able to adapt to the environment in which it is deployed, which means that things like linear or GitHub even are specified within the spec, but not required pieces of it. There’s like a more platonic ideal of the thing that you could swap in like Jira or Bitbucket, for example.

But being able to tightly specify things like the ID formats or how the Ralph Loop works for the individual agents. Basically means you can get up and running with a fully specified system quickly that you then evolve later on. I think we never intended for this to be a static spec that you can [00:50:00] never change.

It’s more like a blueprint to get something worth a starting point up and running.

swyx: Yeah.

Ryan Lopopolo: For you then to vibe later to your heart’s content,

swyx: you have like code and scripts in here where it’s oh, I think this is a really good prompt. It’s just a very long prompt.

Ryan Lopopolo: Fundamentally, the agents are good at following instructions, so give them instructions.

And it will, improve the reliability of the result. We, much like the way we use Symphony, we don’t want folks to have to monitor the agent as it is vibing the system into existence. So being very opinionated

Very strict around what these success criteria are means that our deployment success rate goes up. Yeah. It means we don’t have to get tickets on this thing.

Vibhu: Think it all goes back to that like code to disposable, right? Like early on when you had CLI or you’d kick off a Codex run, it would take two hours. You would wanna monitor okay, I’m in the workflow of just using one.

I don’t want it to go down the wrong path. I’ll cut it off and, just shoot off four, like that was my favorite thing of the Codex app, right? Yeah. Just Forex it like, [00:51:00] it’s okay. One of them will probably be right, one of them might be better. Stop overthinking it. Like my first example was probably like deep research.

When you put out deep research and I’d ask it something like, I asked it something about LLM, it thought it was legal something and spent an hour, came back with a report completely off the rails. And I was like, okay, I gotta monitor this thing a bit. No don’t monitor it. Just you want to build it so it’s that it, it goes the right way.

And you don’t wanna, you don’t wanna sit there and babysit, right? You don’t want to babysit your agents

Ryan Lopopolo: with that deep research query that you made. Looking at the bad result, you probably figured out you needed to tweak your prompt Yeah. A bit, right? That’s that guardrail that you fed back into the code base for the task, your prompt to further align the agent’s execution.

Same sort of concept supply there too.

swyx: When you talk, how are the customers feeling

Ryan Lopopolo: for Symphony? I think we have none, right? This is a thing we have put out into the

swyx: world. Symphony’s internal, right? As long as you are happy, you are the customer. That’s right. Just, what’s the external view?

[00:51:53] Trust Building with PR Videos

Ryan Lopopolo: I’d say folks are very excited about this way of distributing software and ideas in [00:52:00] cheap ways. For us as users, it has again, pushed the productivity five x, which means I think there’s something here that’s like a durable pattern around removing the human from the loop and figuring out ways to trust the output.

The video that is shared here

swyx: Yeah.

Ryan Lopopolo: Is the same sort of video we would expect the coding agent to attach to the pr.

swyx: Yeah.

Ryan Lopopolo: That is created. Yeah. That’s part of building trust in this system and that’s, to me, like fundamentally what has been cool about building this is it more closely pushes that persona of the agent working with you to be like a teammate.

I don’t shoulder surf you like for the tickets that you work on during the week. I would never think that I would want to do that.

swyx: Yeah.

Ryan Lopopolo: I wouldn’t want a screen recording of your entire session in Cursor or Claude code. I would expect you to do what you think you need to do to convince me that the code is good and [00:53:00] mergeable

swyx: Yeah.

Ryan Lopopolo: And compress that full trajectory in a way that is legible to me. The reviewer.

swyx: Yeah.

Ryan Lopopolo: It’s Stu. And you can just do that because Codex will absolutely sling some f you can just around. It’s great.

swyx: Oh, F FM P is the og like God, CLI.

Ryan Lopopolo: Yeah.

swyx: Swiss Army Chainsaw. I used to say. There’s a SaaS, micro SaaS that’s called it in every flag in FFM Peg.

Ryan Lopopolo: Oh, for sure.

swyx: You know what I mean? For sure. Just host it as a service, put a UI on it. People who don’t know FM Peg will pay for it.

Ryan Lopopolo: When we were first experimenting with this, it was a wild feeling to be at the computer with just like windows just popping up all over the place and getting captured and files appearing on my desktop, like very much felt like the future to have a thing controlling my computer for like actual productive use.

Like I’m just there

swyx: keeping it. Like awake, jiggling the mouse every once in a while. That’s what some office workers do. So they buy a mouse jiggler. That’s right.

[00:53:59] Spark vs Reasoning Models

Vibhu: One thing I [00:54:00] wanted to ask, so okay, as stuff is so CO is disposable is saying shoot off a budget of agents. One question is okay, are you always like a extra high thinking guy?

And where do you see Spark? So 5.3 Spark, there’s a lot of me wanting to make quick changes. I’m not gonna open up a id, I’m not gonna do anything. But I will say, okay, fix this little thing, change a line, change a color. Spark is great for that, but am I still a bottleneck? Like, why don’t I just let that go back?

I’m like, just riff on that. Is there,

Ryan Lopopolo: spark is such a different model compared to the. The extra high level reasoning that you get in these, five Yeah. To clear for people.

swyx: It is a different model, different architecture, different, like it doesn’t support

Ryan Lopopolo: it, it just, it’s incredibly fast smaller model.

I have not quite figured out how to use it yet. To be honest, I use faster. I was adapting it to the same sorts of tasks I would use X high reasoning for. Yeah. I, and it would blow through three compactions before writing a line of code.

Vibhu: And that’s another big thing with 5.4 right.

Million co context.

Ryan Lopopolo: Yes, it’s

Vibhu: fantastic. Which is huge [00:55:00] ingenix, right? Like you can just run for longer before you have to compact. The more tokens you can spend on a task before compacting, like the better you’ll do.

Ryan Lopopolo: That’s right. That’s right. I’m not sure how to deploy spark. I think your intuition is right, that it’s very great for spiking out prototypes, exploring ideas quickly, doing those documentation updates.

It is fantastic for us in taking that feedback and transforming it into a lint. Where we already have good infrastructure for ES links in the code base these sorts of things it’s great at and it allows us to unblock quickly doing those like anti-fragile healing tasks in the code base.

swyx: Yeah, that makes sense.

[00:55:38] What Models Can’t Do Yet

swyx: So you are push, you guys are pushing models to the freaking limit.

[00:55:41] Current Model Limitations

swyx: What can current models not do well yet?

Ryan Lopopolo: They’re definitely not there on being able to go from new product idea to prototype single

swyx: one shot.

Ryan Lopopolo: This is where I find I spend a lot of time steering is translating end state of a mock for a net new [00:56:00] thing, right?

Think no existing screens into product that is playable with. Similarly, while this has gotten better with each model release, like the gnarliest refactorings are the ones that I spend my most time with, right? The ones where I’m interrupting the most, the ones where I am. Now double clicking to build tooling to help decompose monoliths and things like that.

This is a thing I only expect to get better, right? Over the course of a month, we went from the low complexity tasks to like low complexity and big tasks in both these directions. So this is what it means to not bet against the model, right? You should expect that it is going to push itself out into these higher and higher complexity spaces.

Yeah. So the things we do are robust to that. It just basically means I’ll be able to spend my time elsewhere and figure out what the next bottleneck is.

Vibhu: I do think it’s also a bit of a different type of task, right? Codex is really good at codebase understanding, working with code bases. But companies like Lovable bolt, repli, they solve a very different [00:57:00] problem.

Scaffold of zero to one, right? Idea of a product. And it’s there, there are people working on that and models are also pushing like step function changes there. It’s just different than the software engineering agents today, right?

Ryan Lopopolo: Like I said, the model is isomorphic to myself.

The only thing that’s different is figuring out how to get what’s in here into context for the model and for these white space sort of projects. I, myself, I’m just not good at it. Which means that often over the agent trajectory, I realize the bits that we’re missing, which is why I find I need to have this synchronous interaction.

And I expect with the right harness, with the right scaffold, that’s able to tease that outta me or refine the possible space, right? To be super opinionated around the frameworks that are deployed or to put a template in place, right? These are ways to give the model. All those non-functional requirements, that extra context to acre on and avoid that wide dispersion of possible outcomes.

swyx: Thank [00:58:00] you for that.

[00:58:00] Frontier Enterprise Platform

swyx: I wanted to talk a little bit about Frontier.

Ryan Lopopolo: Yeah, sure.

swyx: Overall you guys announced it maybe like a month ago. And there’s a few charts in here and it’s basic like your enterprise offering is what I view it. Is there one product or is there many,

Ryan Lopopolo: I can’t speak to the full product roadmap here, but what I can say is that Frontier is the platform by which we want to do AI transformation of every enterprise and from big to small.

And the way we want to do that is by making it easy to deploy highly observable, safe, controlled, identifiable agents into the workplace. We want it to work with your company native. I am stack. We want it to plug into the security tooling that you have. Oh, we want it to be able to plug into the workspace tools that you used,

swyx: so you’re just gonna be stripping specs, right?

Ryan Lopopolo: We expect that there will be some harness things there. Agents, SDK is a core [00:59:00] part of this to enable both startup builders as well as enterprise builders to have a works by default harness that is able to use all the best features of our models from the Shell tool down to the Codex Harness with file attachments and containers and all these other things that we know go into building highly reliable, complex agents.

We wanna make that great and we wanna make it easy to compose these things together in ways that are safe, for example, right? Like the G-P-T-O-S-S safeguard model. For example. One thing that’s really cool about it is it ships. The ability to interface with a safety spec. Safety specs are things that are bespoke to enterprises.

We owe it to these folks to figure out ways for them to instrument the agents in their enterprise to avoid exfiltration in the ways they specifically care about, to know about their internal company, code names, these sorts of things. So providing the right hooks to make the [01:00:00] platform customizable, but also, mostly working by default for folks is the space we are trying to explore here.

swyx: Yeah. And this is the snowflakes of the world just need this, right? Yes. Your Brexit of the world stripes. Yeah, it makes sense.

[01:00:11] Dashboards and Data Agents

swyx: I was gonna go back to your, I think the demo videos that you guys had was pretty illustrative. It’s like also to me an example of very large scale agent management.

Yes. Like you give people a control dashboard that if you play, if you like, play any one of these like multiple agent things, you can di dig down to the individual instant and see what’s going on.

Ryan Lopopolo: Yes, of course.

swyx: But who’s the user Is it let’s it like the CEO, the CTO, ccio, something like that.

Ryan Lopopolo: At least with my personal opinion here, the buyer that we’re trying to build product for here is one and employees who are making productive use of these agents, right?

That’s gonna be whatever surfaces they appear in the connectors they have access to, things like that. Something like this dashboard is for it. Your GRC and governments folks, your AI innovation office, your security [01:01:00] team, right? The stakeholders in your company that are responsible for successfully deploying into.

The spaces where your employees work, as well as doing so in a safe way that is consistent with all the regulatory requirements that you have and customer attestations and things like that. So it is a iceberg beneath the actual end. It’s,

swyx: yeah you jump every, I guess layer in the UI is like going down the layer of extraction in terms of the agent, right?

Yep. Yeah. Yeah. I think it’s good.

Ryan Lopopolo: Yeah. The ability to dive deep into the individual agent trajectory level is gonna be super powerful.

Not only for from like a security perspective, but also from like someone who is accountable for developing skills. One thing that was interesting that we also blogged about shipping was an internal data agent, which uses a lot of the frontier technology in order to make our data ontology accessible to the agent and things like that to understand.

What’s actually in the data [01:02:00] warehouse?

swyx: Yeah. Seman layer Yes. Type things. Yes. I was briefly part of the, that, that world is it salt? I don’t know. It’s actually really hard for humans to agree on what revenue is. Yes.

Ryan Lopopolo: Yes.

swyx: What is an active user?

Ryan Lopopolo: There’s what, five data scientists in the company that have defined this Golden.

swyx: They, yeah. And no. And there’s also internal politics. Yes. As to attribution of I’m marketing, I’m responsible for this much, and sales is responsible for this much, and they all add up to more than a hundred. And I’m like you guys have different definitions.

Vibhu: Yeah. And if you’re a startup, everything is a RR,

swyx: So I think that’s cool.

Oh, you guys blog about this. Okay. I didn’t see this. Yeah. Is this the same thing? I don’t know. This is what you’re referring to? Yes. Okay. We’ll send people to read this. This is our data.

Vibhu: Him this one.

swyx: Yeah. I don’t know if you’re you have any highlights? I

Vibhu: No. In general from the playlist.

Yeah. A lot of good things to read.

swyx: Yeah. Yeah. Lot, lots of homework for people. No, but like data as the feedback layer, you need to solve this first in order to have the products feedback loop closed. That’s right. So for the agents to understand and this is not something that humans have not solved.

This like, and

Ryan Lopopolo: this is [01:03:00] how you build artists that do more than coding, right? Yeah.

swyx: Yeah.

Ryan Lopopolo: To actually understand how you operate the business.

swyx: Yeah.

Ryan Lopopolo: You have to understand what revenue is, what your customer segments are. Yeah. What your product lines are.

[01:03:13] Company Context and Memes

Ryan Lopopolo: Like one thing that’s in looping back to the code base that we described here for harnessing, one thing that’s in core beliefs.md is who’s on the team, what product we’re building, who our end customers are.

Who our pilot customers are, what the full vision of what we want to achieve over the next 12 months is these are all bits of context that inform how we would go about building the software. Oh my God. So we have to give it to the agent too.

Vibhu: I’m guessing that stuff is like pretty dynamic and it changes over time too, right?

Like part of it was, it’s not just a big spec. You have it as one of the things and it will iterate.

Ryan Lopopolo: One, one thing that I think is gonna break your mind even more is we have skills for how to properly generate deep fried memes and have Ji culture [01:04:00] and Slack. Because with the Slack Chachi PT app that you’re able to use in Codex, like I can get the agent to shit post on my behalf.

Just, it’s part of humor.

swyx: Theme humor. Humor is part of EGI. Is it funny? It is pretty good, yeah. Okay. Yeah,

Ryan Lopopolo: it’s pretty good at making

swyx: Deep, it’s a lot of I think humor is like a really hard intelligence test, right? It’s like you have to get a lot of context into like very few words.

This is why make references

Ryan Lopopolo: is why five four is such a big uplift for our it’s the me. Yeah, for sure. Yeah. Yeah.

swyx: It’s very cool.

Vibhu: So five, four can two post. So that’s what we take over here.

Ryan Lopopolo: Yeah. Maybe maybe when y’all are done here today, ask Codex to go over your coding agent sessions and to roast you.

swyx: Love it. I’ll give it a shot. Give a shot. Coming back to the final point I wanted to make is, yeah I think that there, there are multiple other, like you guys are working on this, but this is a pattern that every other company out there should adopt. Yes. Regardless of whether or not they work with you.

To me, this is I saw this, I was like, fuck, [01:05:00] every company needs this. This

is

swyx: multiple billions.

Ryan Lopopolo: This is what it takes to get

swyx: Yeah.

Ryan Lopopolo: People to Yes. Yeah. Actually realize the benefits. Yes. And distribute.

swyx: And it’s, it, I think it sounds boring to people like, oh, it’s for safeguards and whatever, but I think you to handle agents at scale like you are envisioning here I don’t know if it’s like a real screenshot, like a demo, but this is what you need.

This is, or my original sort of view of what Temporal was supposed to be that you, you built this dashboard and you basically have every long running process in the company Yes. In one dashboard and that’s it. That’s right.

Vibhu: Yeah. I think it’s pretty customized towards every enterprise, right?

Like you care about different things.

swyx: There’s a lot of customization, but there’ll be multiple unicorns just doing this as a service. I don’t know. I’m like very frontier field, if you can tell. Amazing. But it, it only clicked ‘cause obviously this came out first, then Harness eng, then symphony and only clicked for me that like, this is actually the thing you shipped to do that.

Ryan Lopopolo: Yeah. Yeah. There’s a set of building blocks here that we assembled into these agents [01:06:00] and the building blocks themselves are part of the product, right? Yeah. The ability to steer revoke authorization if a model becomes misaligned, like all of this is accessible through Frontier. And there’s gonna be a bunch of stakeholders in the company that have the things they need to see in the platform Yeah.

To get to. Yes. So we’ll build all of those in the frontier so that we can actually do the widespread the planet. Yeah. That’s the fun part.

swyx: Yeah. I’m also calling back to there’s this like levels of EGI I don’t know if Opening Eye is still talking about this, but they used to talk about five levels of EGI and one of it was like, oh, it’s like an intern coding software patient.

At some point it was AI organization and this is it. That’s right. This is level four or five. I can’t remember which, which level, but it’s somewhere along that path. Was this.

Ryan Lopopolo: You know how I mentioned that my team is having fun sprinting ahead here. And we do this thing where we’re collecting all the agent trajectories from Codex to slurp them up and distill them.

This is what it means to build our team [01:07:00] level knowledge base, happen to reflect it back into the code base. But it doesn’t have to be that way. And it doesn’t have to be bound to just codex. I want Chacha BT to also learn our meaning culture and also the product we are building and how so that when I go ask it, it also has the full context of the way I do my work and I’m super excited for Frontier to enable this.

swyx: Yeah. Amazing.

[01:07:21] Harness vs Training Tension

swyx: What are the model people say when they see you do this? Like you have a lot of feedback, obviously you have a lot of usage, you have a lot of trajectories and don’t, I don’t imagine a lot of it’s useful to them, but some of it is,

Vibhu: you have this too, you deploy a billion tokens of intelligence a day and this was, this was at the beginning of 2096.

You’re Yeah. Cooking.

Ryan Lopopolo: Yeah, there’s this fundamental tension, which I think you have talked about between whether or not we invest deeper into the harness or we invest deeper into the training process to get the model to do more of this by default. Yeah, and I think success for the way we are [01:08:00] operating here means the model gets better taste because we can point the way there and none of the things we have built actively degrade Asian performance.

‘cause really all they’re doing is running tests and like running tests is a good part of what it means to write reliable software. If we were building an entire separate rust scaffold around Codex to restrict its output, that I think would be like additional harness that would be prone to being scrapped.

But yeah. Yeah. If instead we can build all the guardrails in a way that’s just native to the output that Codex is already producing, which is code, I think. No friction with how the model continues to advance, but also like just good engineering and that’s the whole point.

swyx: Yeah. So I’ve had similar discussions with research scientists where the RL equivalent is on policy versus off policy.

Yeah. And you’re basically saying that you should build an on policy harness, which is already within distribution and you [01:09:00] modify from there. But if you build it off policy, it’s not that useful.

Ryan Lopopolo: That’s right.

swyx: Super cool. Any, anybody thoughts, any things that we haven’t covered that we should get it, get out there?

[01:09:08] Closing Thoughts & OpenAI Hiring

Ryan Lopopolo: Just I’ve been super excited to benefit from all the cooking that the Codex team has been doing. Yes. They absolutely ship relentlessly. This is one of our core engineering values, ship relentlessly, and they, the team there embodies it. To extreme degree, yeah, I have five three and then Spark and five four come out within what feels like a month is just a phenomenally fast.

swyx: It’s exactly a month ago it’s five three and yesterday was five four. Yeah. I mean it’s, do we have every month now is five five next? Exactly.

Ryan Lopopolo: I can’t say that the poll markets would be very upset.

swyx: I think it’s interesting that it’s also correlated with the growth. They announced that it’s 2 million users, but like almost don’t care about Codex anymore.

This is it, this is the gay man. It’s like coding cool, soft like knowledge work.

Ryan Lopopolo: That’s right. That’s right. This is the thing to chase after. Yeah. And this is one of things that my team is excited to support,

swyx: get the whole like [01:10:00] self-hosted harness thing working, which you have done and like the rest of us are trying to figure out how to catch up, but then do things.

You That’s right. With you

Vibhu: do things.

swyx: That’s right. You can just do things. That’s the line for the episode.

Vibhu: That’s it. Any other call to actions. You’re based in Seattle, your team, I’m guessing. New Bellevue office.

Ryan Lopopolo: New Bellevue office. We just had the grand opening yesterday as of the recording date which was fantastic.

Beautiful buildings. Super excitedly part of the Bellevue Community building the future in Washington. And I would say that there is lots of work to be done in order to successfully serve enterprise customers here in Frontier. We are certainly hiring and if you haven’t tried the Codex app yet, please give it a download.

We just passed 2 million weekly active users growing at a phenomenally fast rate, 25% week over week. Come join us.

swyx: Yes. And I think that’s an interesting no. My, my final observation opening is a very San Francisco centric company. I know people who have been. [01:11:00] Who turned down the job or didn’t get the job ‘cause they didn’t want to move to sf and now they just don’t have a choice.

You have to open the London, you have to open the Seattle. And I wonder if that’s gonna be a shift in the culture, obviously you can’t say, but

Ryan Lopopolo: I was one of the first engineering hires out of our Seattle office, so Yeah.

swyx: See I was very natural.

Ryan Lopopolo: Its success has been part of what I have been building toward and it is, it has grown quite well, right?

Yeah. We have durable products in the lines of business that are built outta there a ton of zero to one work happening as well, which is the core essence of the way we do applied AI work at the company to sprint after it new to figure out where we can actually successfully deploy the model.

Yeah. Yes. A hundred percent. We also have a New York office too that has a ton of engineering presence.

swyx: Yeah. Exact. Exactly. That’s these are my road roadmaps for a e wherever people hiring engineers, I will go. That’s right. Ra it’s

Vibhu: a cool office to New York is a old REI building, I believe the REI office.

swyx: It’s just No, you’ll never be as big. New York is you can’t get [01:12:00] the size of office that they need.

Ryan Lopopolo: The New York office, Seattle user has a very office Mad Men vibe. It’s beautiful. The Bellevue one is very green, gold fixtures, very Pacific Northwest is very cool place to the vibe.

Be local

Vibhu: little, yeah. A lot of people are like there for people like New York. They wanna be in New York, right?

Ryan Lopopolo: Yeah. Yeah. We have a fantastic workplace team that has been building out these offices. It really is a privilege to work here. Yeah. Excellent. Okay. Thank you for your time. You’ve been very

swyx: generous and you’re, you’ve been cooking, so I’m gonna let you get back to cooking.

It’s been amazing to be with you folks. Happy Friday. Happy Friday.

(For contrast, Gemma 3 totaled 6.7m downloads in the past year, Gemma 2 had 1.4m downloads since Jun 2024 launch, whereas Qwen 3.5 has gained about 27m downloads inclusive of the 1.5 months since their 397B-A17B flagship model drop)

The Gemma 4 keynote will be live in 3 days from London, which you can bookmark now:

Separately, we’d also highlight the Hermes Agent hype - our friends at the have a good writeup on the Hermes vs OpenClaw differences.

AI News for 4/4/2026-4/6/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Gemma 4’s Rapid Local Adoption and the On-Device Open Model Moment

• Gemma 4 is driving a sharp “local-first” wave: multiple posts pointed to Gemma 4 becoming the top trending / #1 model on Hugging Face, with strong enthusiasm for its practical usability rather than just leaderboard performance—see @ClementDelangue, @GlennCameronjr, and @Yampeleg. The strongest signal was how quickly people were running it on consumer Apple hardware: @adrgrondin showed Gemma 4 E2B on an iPhone 17 Pro at roughly 40 tok/s with MLX; @enjojoyy reported a similar iPhone deployment; @_philschmid highlighted Gemma 4 E2B in AI Edge Gallery using skills for Wikipedia queries. Red Hat also published quantized Gemma 4 31B model cards in NVFP4 and FP8-block formats with instruction-following evals live, and reasoning/vision evals pending, via @RedHat_AI. Together these posts suggest Gemma 4 is not just another open release; it is becoming a reference point for edge inference, Apple Silicon tooling, and low-friction local deployment.

• The commercial implication is pressure on paid chat subscriptions and cloud dependence: some of the more viral commentary was reductive, but it captures a real shift. @AlexEngineerAI argued that Gemma 4 running locally closes enough of the gap to make a Claude subscription less compelling for some users, while @ben_burtenshaw reminded people that HF-hosted models are free to use and can replace portions of an agent workflow. On the infra side, @ollama launched Gemma 4 on Ollama Cloud backed by NVIDIA Blackwell GPUs, making it available to tools like OpenClaw and Claude-style workflows without self-hosting. The notable ecosystem post from @osanseviero also underscored how broad the launch coordination was—HF, vLLM, llama.cpp, Ollama, NVIDIA, Unsloth, SGLang, Docker, Cloudflare and others—which is a reminder that “open model success” increasingly depends on simultaneous downstream systems support, not just weights.

Hermes Agent’s Self-Improving Agent Loop, OpenClaw Friction, and the Push for Open Trace Data

• Hermes Agent was the dominant agent-framework story in this batch: the core narrative is that Nous’ system is winning mindshare by combining persistent memory, self-generated/refined skills, and a more opinionated self-improvement loop. The launch of a Manim skill by @NousResearch was especially resonant because it demonstrated an agent skill that produces immediately legible artifacts—technical animations and explainers—rather than yet another PDF summarizer. This was amplified by demos and reactions from @ErickSky, @lucatac0, @Sentdex, @casper_hansen_, and @noctus91. Product updates from @Teknium added slash-command skill loading for Discord/Telegram bots, while community tools like Hermes HUD mapped live processes to tmux panes and surfaced approvals via @aijoey, and multiple WebUI integrations emerged from @Teknium, @nesquena, and @magiknono.

• The contrast with OpenClaw centered on architecture and business-model fragility: several posts compared the two directly. @TheTuringPost summarized the distinction as human-authored skills vs self-forming skills, Markdown memory vs persistent/searchable memory stacks, and gateway control plane vs self-improving loop. That framing was echoed by practitioners like @SnuuzyP, @DoctaDG, and @spideystreet, many of whom cited easier onboarding and less manual skill fiddling. The backdrop here was mounting frustration with Claude subscription gating and uptime: @theo reported Claude Code errors when analyzing its own source; @Yuchenj_UW and @ratlimit highlighted outages; @Yuchenj_UW argued the $20/$200 subscription model is structurally mismatched to 24/7 agent workloads. That economic critique helps explain the rhetorical momentum behind @NousResearch’s “Open Source is inevitable.”

• A more important long-term thread was open agent data: @badlogicgames released pi-share-hf for publishing coding-agent sessions as Hugging Face datasets with PII defenses, then published his own sessions via @badlogicgames. @ClementDelangue explicitly framed this as the missing ingredient for open-source frontier agents: the community already generates the traces, so it should crowdsource the dataset. This connected cleanly to @salman_paracha’s Signals paper on trajectory sampling/triage for agentic interactions and Baseten’s argument that self-improving models should learn directly from recorded production traces instead of requiring clean sandboxes, via @baseten. This is arguably the most technically substantive “agent” trend here: not just better harnesses, but an emerging stack around trace capture, curation, and training from real usage.

New Research Signals: RL, Routing, Agent Evaluation, and Small Specialized Models

• Post-training and RL efficiency remained active areas of substance: @TheTuringPost highlighted Alibaba Qwen’s FIPO (Future-KL Influenced Policy Optimization), which assigns more credit to tokens that strongly affect future steps; the reported results included reasoning traces extending from roughly 4K to 10K+ tokens and AIME gains from around 50% to ~56–58%, ahead of cited DeepSeekR1-Zero-Math and around/overtaking o1-mini depending on setup. @finbarrtimbers wrote up how OLMo 3 moved from synchronous to asynchronous RL, producing a 4× throughput gain in tokens/sec. Other notable paper pointers included Self-Distilled RLVR / RLSD via @_akhaliq and @HuggingPapers, plus Path-Constrained MoE via @TheAITimeline, which constrains routing paths across layers to improve statistical efficiency and remove auxiliary load-balancing losses.

• Agent and benchmark research is shifting away from toy tasks: @GeZhang86038849 introduced XpertBench, explicitly targeting expert-level, open-ended workflow evaluation rather than saturated exam-style benchmarks. @TheTuringPost shared a survey on tool use covering the progression from single function calls to long-horizon orchestration, replanning, feedback loops, and efficiency concerns such as latency/cost budgets. In data/enterprise workflows, @CShorten30 pointed to Shreya Shankar’s Data Agent Benchmark for multi-step queries across heterogeneous DB systems. These are all signs that eval design is catching up to what production agent builders care about: workflow completion, ambiguity handling, orchestration quality, and cost.

• Small specialized models continued to make strong case-study arguments: @DavidGFar released SauerkrautLM-Doom-MultiVec-1.3M, a 1.3M-parameter ModernBERT-Hash model trained on 31K human-play frames that outperformed far larger API-accessed LLMs on a VizDoom task while running in 31 ms on CPU. The result is narrow, but the point is important: appropriately scoped models can dominate on real-time control tasks where latency and architecture matter more than broad world knowledge. Relatedly, @MaziyarPanahi pushed Falcon Perception, a 0.6B segmentation-oriented vision-language model reportedly outperforming SAM 3 in his comparisons and running on MacBooks with MLX; this was echoed by @Prince_Canuma and @ivanfioravanti. The recurring theme is that specialization + better systems fit can beat generic scale.

OpenAI and Anthropic: Policy Signaling, Governance Scrutiny, and Compute Economics

• OpenAI’s biggest public move was political, not product: the company and its allies pushed a new “Industrial Policy for the Intelligence Age” framing, summarized by @kimmonismus, @OpenAINewsroom, and @AdrienLE. Key ideas included a Public Wealth Fund, portable benefits, 32-hour workweek pilots, a Right to AI, stronger provenance/audit infrastructure, and containment playbooks for dangerous released models. The notable strategic message is that OpenAI is now publicly asserting a transition toward superintelligence as an active policy problem, not a distant hypothetical. Reactions were mixed: some saw it as unusually frank about disruption, others as premature or politically convenient, e.g. @Dan_Jeffries1 and @jeremyslevin. OpenAI also launched a Safety Fellowship via @OpenAI and @markchen90.

• At the same time, scrutiny around Sam Altman and OpenAI governance intensified sharply: a major New Yorker investigation was amplified by @RonanFarrow, @NewYorker, and lengthy community summaries like @ohryansbelt. The reporting revisited the 2023 firing/reinstatement saga with claims about internal memos, allegations of deception, board manipulation, safety-process concerns, and the under-resourcing of superalignment. OpenAI-side pushback arrived via @tszzl, who said the alignment team remains one of the largest and most compute-rich programs at the company. Separately, @anissagardizy8 and @kimmonismus reported tension between Altman and CFO Sarah Friar, especially around compute spending and IPO readiness.

• Anthropic’s counterpoint was compute and revenue scale: @AnthropicAI announced an agreement with Google and Broadcom for multiple gigawatts of next-generation TPU capacity coming online from 2027, to train and serve frontier Claude models. Anthropic also stated its run-rate revenue has surpassed $30B, up from $9B at the end of 2025, via @AnthropicAI. That pairs with reporting on the economic tension in frontier labs: @kimmonismus cited WSJ reporting that revenues are exploding, but training and inference costs remain enormous, with OpenAI projecting $121B compute spend by 2028. For engineers, the practical takeaway is straightforward: the frontier race is increasingly bottlenecked not by model ideas alone, but by capital structure, long-dated compute contracts, and serving economics.

Systems and Infra: Faster RL, Faster MoE Decoding, Better GPU/Edge Tooling

• Several posts were unusually concrete about systems wins: @cursor_ai reported 1.84× faster MoE token generation on Blackwell GPUs with improved output quality via “warp decode,” a result tied directly to more frequent Composer model updates. @tri_dao noted that a fast Muon optimizer path is coming to consumer Blackwell cards, because the implementation is expressed as matmul + epilogue, allowing reuse of the mainloop work. On the RL side, @finbarrtimbers provided a rare engineering postmortem on making OLMo 3’s RL stack asynchronous for a 4× throughput jump.

• The Apple/local stack and training/inference education ecosystem also kept improving: @josephjojoe open-sourced an MLX port of ESM-2 for protein modeling on Apple Silicon, broadening local bio-LLM experimentation. @rasbt added an RSS feed to the LLM Architecture Gallery, a small but useful quality-of-life improvement for keeping up with model designs. @UnslothAI said its free notebook can now train/run 500+ models. For deeper systems understanding, @levidiamode praised Hugging Face’s Ultra-Scale Playbook for unifying DP/TP/PP/EP/context parallelism with empirical scaling evidence across up to 512 GPUs.

Top tweets (by engagement)

• Gemma 4 on-device demo: @adrgrondin showing Gemma 4 E2B on iPhone 17 Pro at ~40 tok/s with MLX was the standout technical viral post.

• Claude subscription and local-open-model substitution: @AlexEngineerAI captured the mood that local open models are now “good enough” for many workflows.

• Open source posture: @NousResearch distilled the broader movement with “Open Source is inevitable.”

• Claude outages and gating backlash: @ratlimit, @theo, and @Yuchenj_UW collectively turned uptime and subscription economics into a mainstream engineering complaint.

• OpenAI governance investigation: @RonanFarrow and @ohryansbelt drove the biggest technically adjacent corporate-governance story of the day.

• Anthropic compute scale: @AnthropicAI announcing multi-gigawatt TPU capacity and @AnthropicAI citing $30B run-rate revenue were among the clearest signals of frontier-lab scale.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Gemma 4 Model Launch and Benchmarks

Read more

Early analytics from our Marc Andreesen pod are already pointing towards it being one of the top Latent Space pods of all time. We’ll hear more from the creators of both OpenClaw and Pi (and many other top Europe-origin AI tools) live from London next week. Livestream links for AIE Europe next week is now up, including a great OpenClaw song. Hit the bell to help promote it in the algorithm please and thank you!

AI News for 4/3/2026-4/4/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Gemma 4’s Apache-licensed launch, local inference performance, and day-0 ecosystem support

• Gemma 4 is the day’s defining open-model release: Google launched Gemma 4 under Apache 2.0, with multiple posts emphasizing its positioning for reasoning, agentic workflows, multimodality, and on-device use. @fchollet called it Google’s strongest open model yet and recommended the JAX backend in KerasHub; @demishassabis highlighted efficiency, claiming Gemma 4 outperforms models 10x larger on Google’s chart. Community reaction centered on the license shift: @ClementDelangue, @QuixiAI, and @googlegemma all stressed that this is a “real” open-weights release with broad downstream usability.

• The ecosystem was unusually ready on day 0: Support landed immediately across vLLM (GPU, TPU, XPU simultaneously), llama.cpp (@ggerganov), Ollama (new models available), Intel hardware (Xeon, Xe GPU, Core Ultra), Unsloth (local run/fine-tune support), Hugging Face Inference Endpoints (one-click deploy), and AI Studio / Google AI Studio collateral (article link). For architecture-oriented readers, both @osanseviero and @MaartenGr shared deep visual guides covering MoE design, vision/audio encoders, and per-layer embeddings.

• Local inference benchmarks were the main practical story: multiple builders showed Gemma 4 running on consumer hardware, with particular attention to the 26B A4B MoE. @basecampbernie reported 162 tok/s decode and 262K native context on a single RTX 4090 at 19.5 GB VRAM, while @Prince_Canuma showed TurboQuant KV cache cutting memory from 13.3 GB to 4.9 GB at 128K context for the 31B model, with some decode-speed penalty. There were also examples on weaker local devices: @measure_plan reported 34 tok/s for 26B-A4B on a Mac mini M4 with 16 GB, @kimmonismus argued the E4B tier brings useful AI directly to phones/laptops, and @anemll got the model onto an iPhone with Swift MLX.

• Early benchmarking discourse was positive but not uncritical: @arena noted large ranking gains over Gemma 3 and 2 at similar parameter scales, suggesting progress beyond pure scaling; later, @arena put Gemma 4 31B on the Pareto frontier against similarly priced models. Some users pushed back on presentation choices: @stochasticchasm argued comparisons should be more clearly FLOP/active-parameter normalized, and @reach_vb urged the field to move beyond Arena Elo as the default score.

Hermes Agent’s rapid adoption, memory/plugin architecture, and the “harness matters” shift

• Hermes Agent appears to be the breakout open-source agent harness of the day: across user reports, many developers explicitly said they had switched from OpenClaw/Openclaw to Hermes and found it more stable or more capable on long tasks. Examples include @Zeneca, @Everlier, @erick_lindberg_, and @AnomalistG. A detailed Korean thread from @supernovajunn crystallized the narrative: the edge is not just the model, but the harness + learning loop, especially autonomous skill creation, reusable procedural memory, and higher reliability floors on real tasks.

• Nous shipped meaningful infrastructure, not just hype: @Teknium announced a reworked, pluggable memory system with support for Honcho, mem0, Hindsight, RetainDB, Byterover, OpenVikingAI, and Vectorize-style backends. Follow-up posts detailed the architectural cleanup: memory providers are now a dedicated plugin type, the core is more maintainable, and users can add their own providers more easily (details). Hermes also added inline diffs in the TUI (post) and provider credential pools for cycling between accounts/keys (post).

• The larger theme is that agent performance is becoming a harness-engineering problem: @Vtrivedy10 described a “model-harness training loop” where teams combine harness engineering, trace collection, analysis, and fine-tuning to build domain-specific frontier performance. In a companion tweet, he argued the key raw material is massive trace data, mined by agents for failure modes and converted into training or harness improvements (trace loop). This complements Hermes’ popularity: if open models are now “good enough,” better memory, tools, evals, and self-improvement loops may dominate application quality.

• There is also visible demand for open harnesses rather than closed product shells: @michael_chomsky argued Anthropic should open-source Claude Code, partly because 2025 was “the year of mediocre harnesses”; @hwchase17 made the memory angle explicit, saying memory cannot remain trapped behind proprietary APIs or proprietary harnesses.

Coding agents, rate limits, and the cognitive bottleneck of parallel agent work

• The strongest user sentiment was not about raw model IQ but about operational friction: @gdb lowered the barrier to trying Codex at work by removing up-front commitment, and later said the Codex app is growing super fast (post). But at the same time, discussion around Claude Code rate limits was intense: @theo said “we need to talk about the Claude Code rate limits,” with follow-up user complaints from @kimmonismus and @cto_junior suggesting that users are hitting caps faster than expected.

• A growing theme is cognitive saturation, not just compute scarcity: one of the most-engaged technical tweets was @lennysan quoting @simonw: using coding agents well can require every inch of senior engineering experience, and orchestrating four agents in parallel is mentally exhausting by mid-morning. That view showed up elsewhere: @kylebrussell praised Claude Code’s ability to drive many browser tabs for verification work, but later noted scaling gets “weird” and that 2–4 sessions still seems optimal for his brain (post).

• Developers are adapting by externalizing context and observability: @jerryjliu0 described a practical setup where agents emit .md/.html artifacts to preserve context across sessions, with Obsidian as a local viewer and LiteParse replacing generic PDF parsers for better extraction from complex documents. On the observability side, LangChain shipped a Claude Code → LangSmith tracing plugin that logs subagents, tool calls, compaction, token usage, and enables org-level analysis (announcement).

• There’s also growing evidence that “good enough local fallback” matters: several posts framed Gemma 4 and Hermes together as a hedge against hosted-product friction. @gregisenberg emphasized that a model this capable now runs locally and can be swapped into Claude Code, Cursor, Hermes, or OpenClaw. @kimmonismus similarly highlighted a fully local assistant on a MacBook Air M4 with 16 GB, no API keys required.

Research signals: time horizons, recursive context management, and self-distillation

• METR-style “time horizon” results continue to trend upward: @LyptusResearch applied the METR time-horizon methodology to offensive cybersecurity, reporting that capability has doubled every 9.8 months since 2019, or 5.7 months on a 2024+ fit, with Opus 4.6 and GPT-5.3 Codex reaching 50% success on tasks taking human experts ~3 hours. Related commentary from @scaling01 extrapolated METR horizons to roughly 15.2 hours “today” and ~87 hours by year-end under continuation assumptions.

• Long-context handling remains an active systems/research problem: @DeepLearningAI highlighted Recursive Language Models (RLMs) from MIT researchers Alex Zhang, Tim Kraska, and Omar Khattab: rather than stuffing everything into a monolithic prompt, the system offloads prompt management to an external environment, managing context programmatically. This idea resonated with practitioners: @raibaggy joked that after moving workflows to RLMs, “you have to put the harness into the harness.”

• Post-training without labels/verifiers got notable attention: @BoWang87 summarized Apple’s Simple Self-Distillation (SSD) result for coding models: sample the model’s own outputs and fine-tune on them without correctness filtering, RL, or a verifier. The strongest cited gain was Qwen3-30B-Instruct: 42.4% → 55.3% pass@1 on LiveCodeBench, with especially large gains on hard problems. If robust, this suggests many code models are underperforming their latent capability due to decoding/post-training gaps rather than missing core competence.

• Additional research worth flagging: @jaseweston shared a 70-page paper on reasoning over mathematical objects, spanning training data, on-policy reward models, and on-policy inference methods; @AnthropicAI published a “diff” method for surfacing behavioral differences between open-weight models; and @AndrewLampinen discussed test-time thinking as a way to retrieve and use latent knowledge from training data.

Enterprise and production AI: speech, security, access control, and real-world deployments

• Microsoft’s MAI-Transcribe-1 looks competitive on STT: @ArtificialAnlys reported 3.0% AA-WER (#4 overall on its leaderboard) and ~69x real-time speed, with support for 25 languages and preview availability through Azure Speech / Foundry. Pricing was quoted at $6 per 1,000 minutes (pricing post).

• Security surfaced in multiple production contexts: @simonw warned maintainers that the Axios supply-chain attack began with sophisticated social engineering aimed at a developer; @gneubig pulled out the practical lessons: stronger credential management, identity verification, and malware detection. Separately, @thinkshiv and @jerryjliu0 highlighted a joint Auth0 FGA + LlamaIndex approach to making authorization structural inside retrieval, rather than bolting it on after the fact.

• Inference infrastructure and real deployments got credible examples: Baseten and OpenEvidence both claimed very large-scale production use in clinical settings, with OpenEvidence saying over 40% of U.S. physicians rely on it and Baseten powers inference for that workload (OpenEvidence, Baseten). On serving resilience, @vllm_project highlighted DP-group fault tolerance in Ray Serve LLM for vLLM WideEP deployments, complementing Elastic EP at the engine layer.

Top tweets (by engagement, filtered for technical relevance)

• Agent workflow fatigue is becoming a first-class problem: @lennysan quoting @simonw on the mental cost of using multiple coding agents in parallel was the most resonant technical post in the set.

• Personal knowledge bases for agents are turning into a serious pattern: @omarsar0 described a highly customized research-paper knowledge base built in markdown with semantic indexing, agent-driven curation, and interactive artifacts; a follow-up shared the system diagram (diagram).

• Gemma 4 had both broad mindshare and practical credibility: engagement concentrated not only on the launch itself—@fchollet, @demishassabis—but on practical local-running claims from @ClementDelangue, @gregisenberg, and @kimmonismus.

• Hermes Agent’s adoption curve is now visible in the open: the strongest evidence came less from official posts than from user migration reports and usage anecdotes, plus @Teknium’s memory-system overhaul. The pattern is notable: users increasingly credit memory + harness design, not just the base model, for the jump in utility.

AI Reddit Recap

/r/LocalLlama + /r/localLLM Recap

1. Gemma 4 Model Release and Features

• Gemma 4 has been released (Activity: 3412): Gemma 4, developed by Google DeepMind, is a family of open multimodal models capable of processing text, images, and audio, with a context window of up to 256K tokens. The models are available in four sizes: E2B, E4B, 26B A4B, and 31B, supporting multilingual capabilities in over 140 languages. They feature both Dense and Mixture-of-Experts (MoE) architectures, optimized for tasks such as text generation, coding, and reasoning. Notably, Gemma 4 introduces a hybrid attention mechanism combining local sliding window and global attention, enhancing processing speed and memory efficiency for long-context tasks. The models also support native function-calling and structured tool use, facilitating agentic workflows and coding tasks. For more details, see the Hugging Face repository. One comment highlights the significance of Gemma-4’s native thinking and tool-calling capabilities, emphasizing its multimodal nature. Another provides practical guidance on running the models, including specific parameters like temperature = 1.0, top_p = 0.95, and top_k = 64, and mentions its integration with Unsloth Studio.

  • Gemma-4 introduces several advanced features such as native thinking, tool calling, and multimodal capabilities. It is optimized with specific parameters: temperature = 1.0, top_p = 0.95, top_k = 64, and uses <turn|> as the end-of-sequence token. Additionally, <|channel>thought\n is used for the thinking trace, enhancing its cognitive processing capabilities. More details and guides are available at Unsloth AI.

  • The release of Gemma-4 is significant for its seamless integration with Unsloth Studio, providing a streamlined environment for developers. All GGUFs related to Gemma-4 can be accessed on Hugging Face, offering a comprehensive resource for those looking to implement or experiment with the model.

  • There is anticipation for comparative analysis between Gemma-4 and other models like Qwen3.5, highlighting the competitive landscape in AI model development. This suggests a focus on benchmarking and performance evaluation to understand the strengths and weaknesses of each model in practical applications.

• You can now run Google Gemma 4 locally! (5GB RAM min.) (Activity: 415): Google has released the open-source model family Gemma 4, featuring four models with multimodal capabilities: E2B, E4B, 26B-A4B, and 31B. The models excel in reasoning, coding, and long-context workflows. The 31B model is the most advanced, while 26B-A4B is optimized for speed due to its MoE architecture. Unsloth has adapted these models for local execution on devices with as little as 5GB RAM. The models can be run via Unsloth Studio, with recommended setups ranging from 6GB RAM for smaller models to 35GB RAM for the largest. No GPU is required, but it enhances performance significantly. Installation is streamlined for various OS, and a desktop app is forthcoming. More details are available in the Unsloth documentation. Commenters express excitement about the usability of Gemma 4 on older hardware, noting the impressive performance of the E2B model on a 2013 Dell laptop. There is also a discussion on the complexity of keeping up with model specifications and hardware requirements.

  • The recommended setups for running Google Gemma 4 locally highlight the memory and performance trade-offs across different model sizes. For instance, the E2B and E4B variants can achieve 10+ tokens per second in near-full precision with approximately 6GB of RAM, while 4-bit variants can operate on 4-5GB RAM. Larger models like the 26B-A4B require around 30GB of RAM for similar performance, with 4-bit versions needing 16GB. The 31B model, which is even larger, demands about 35GB of RAM for 15+ tokens per second in near-full precision.

  • A user reports that the Gemma4 E2B model performs surprisingly well on older hardware, specifically a 2013 Dell E6440 with an i5 4310 CPU and 8GB of RAM, achieving a reply speed of 8 tokens per second. This suggests that even older systems can handle smaller models of Gemma 4 for basic tasks, highlighting the model’s efficiency and adaptability for less powerful machines.

  • The 31B model of Google Gemma 4 has a significant memory requirement due to its KV Cache and Mixture of Experts (MoE) architecture, needing up to 40GB of VRAM to load into memory. This indicates a substantial resource demand for running larger models, which could be a limiting factor for users without access to high-end hardware.

• Gemma4 - Someone at Google just merged a PR titled “casually dropping the most capable open weights on the planet” (Activity: 471): Google has merged a PR in the HuggingFace Transformers repo for a new model, Gemma 4, described as the ‘most capable open weights on the planet.’ The model includes four sizes: ~2B and ~4B dense models for on-device use, a 26B sparse MoE with 4B active parameters at inference, and a 31B dense model. Notably, the 26B/4B MoE offers large-model quality with small-model inference cost. Gemma 4 is trimodal, supporting text, vision, and audio natively, with a conformer architecture for audio and a 2D spatial RoPE for vision. It features 128K context for small models and 256K for large, using a hybrid attention design. The MoE variant includes both MLP and sparse MoE blocks, summing their outputs, which is an unusual design choice. The code is merged but weights and release date are pending. Commenters are excited about the potential of the 31B model and the 26B/4B MoE for VRAM-constrained environments. There’s a discussion on how MoE models manage weights in VRAM, with a focus on inference efficiency. Another comment notes that llama.cpp support is ready, enabling immediate local inference upon weight release.

  • The Mixture of Experts (MoE) model architecture allows for the performance of a larger dense model without the computational overhead by activating only a subset of the model’s parameters during inference. This means that while the Gemma4 26B/4B model has 26 billion parameters, only 4 billion are activated at any given time, potentially reducing the VRAM requirements. However, the entire model’s weights might still need to be accessible, which could be a challenge for VRAM-constrained environments, as the model might need to manage the loading and unloading of weights dynamically to maintain acceptable inference latency.

  • The llama.cpp repository has already integrated support for the Gemma4 model, as indicated by a recent pull request. This means that once the Gemma4 weights are released, users can immediately convert them to the GGUF format and perform local inference without waiting for additional updates to the llama.cpp repository. This rapid integration highlights the readiness of the community to support new model releases and facilitate their deployment in various environments.

  • The announcement of Gemma4 by DeepMind and Google includes a detailed blog post and model documentation, which can be found at DeepMind’s official page and Google’s blog. These resources provide insights into the model’s capabilities and potential applications, emphasizing its status as one of the most capable open weights available.

2. Gemma 4 Performance and Issues

• Gemma 4 is good (Activity: 429): The post discusses the performance of the Gemma 26b a4b model on a Mac Studio M1 Ultra, comparing it to Qwen3.5 35b a3b. The user reports that Gemma is faster and more coherent, with better visual understanding and multilingual capabilities, despite having a large KV cache footprint (22GB VRAM for 260K tokens @ fp16). The Q4_K_XL quantized model requires an additional ~18GB. The post also mentions issues with Google’s AI studio version of Gemma, citing tokenizer problems. The user notes that SWA provides some benefits in reducing the KV cache size, and expresses concerns about censorship in the model’s responses, particularly in medical contexts. A comment highlights skepticism about the results due to a known issue with the llama.cpp implementation, which was reportedly broken at the time of the original post. Another comment praises the Gemma 4 E2B model for its ability to recognize context limitations, while a third comment criticizes the 31b abliterated version for poor performance.

  • Pristine-Woodpecker highlights a critical issue with the llama.cpp implementation, noting that it was broken at the time of the original post. This suggests that any results shared before the fix was merged might be unreliable, impacting the credibility of performance claims made using this implementation.

  • Finguili discusses the memory efficiency of the Gemma 4 model, countering a claim about its KV cache size. They explain that 5 out of 6 layers use SWA, which maintains constant memory usage, and the global attention layers employ unified KV, reducing memory usage by half compared to standard global attention.

  • Deenspaces provides a comparative analysis of Gemma-4 and Qwen models, noting that Gemma-4-31b-it and Gemma-4-26b-a4b are faster than Qwen3.5-27b and Qwen3.5-35b-a3b. However, they point out a significant issue with Gemma-4’s context handling, which is too heavy, leading to instability and looping when cache quantization is applied in LM studio. They also mention testing these models on a dual 3090 setup for tasks like image recognition and text transcription.

• Gemma 4 is seriously broken when using Unsloth and llama.cpp (Activity: 330): The image highlights issues with the “Gemma 4” model when used locally with “Unsloth” quants on “llama.cpp.” Users report that the model produces nonsensical outputs when tasked with identifying and correcting typos in a text, despite using recommended settings. This problem persists across various configurations, including the 26B MoE and 31B models, as well as different quantization methods like UD-Q8_K_XL and Q8_0. In contrast, the same models perform well in Google AI Studio. The issue appears to be related to a tokenizer bug in “llama.cpp,” with several pending pull requests aimed at resolving these problems. The community is actively investigating, and a specific pull request (https://github.com/ggml-org/llama.cpp/pull/21343) is expected to address tokenization issues. Commenters suggest that the problem is not specific to “Unsloth” quants but rather a broader issue with “Gemma 4” and “llama.cpp.” There are multiple pending issues related to “Gemma 4,” and some users note that initial model releases often have such bugs, exacerbated by quick builds from wrappers like Ollama and Lm studio.

  • The issue with Gemma 4 appears to be related to tokenization, as highlighted by a pending pull request #21343 in the llama.cpp repository. This PR aims to address the tokenization problems that are affecting the model’s performance when used with Unsloth and llama.cpp.

  • There are currently 10-15 Gemma-related issues pending in llama.cpp, indicating that the model is facing several initial integration challenges. Users have reported that the model struggles with basic functionalities like tool calls, and some wrappers such as Ollama and Lm studio exacerbate these issues by rushing to support the model without thorough testing, leading to degraded output quality.

  • A potential reason for the issues with Gemma 4 could be changes in the system role format from its predecessor, Gemma 3. This change might not have been fully integrated into the day-zero builds of llama.cpp, causing compatibility problems and necessitating updates to align with the new format.

• Gemma 4 and Qwen3.5 on shared benchmarks (Activity: 1223): The image provides a comparative analysis of AI models, specifically Qwen3.5-27B, Gemma 4 31B, Qwen3.5-35B-A3B, and Gemma 4 26B-A4B, across various performance benchmarks. These benchmarks include categories like Knowledge & Reasoning, Coding, Agentic & Tools, and Frontier Difficulty. The Qwen models generally outperform the Gemma models, particularly excelling in the ‘Frontier Difficulty without tools’ category. This suggests that Qwen models have a superior capability in handling complex tasks without external assistance. Commenters highlight the superior performance of Qwen3.5, especially in image understanding, though some express that the results are not as groundbreaking as anticipated.

  • Different_Fix_2217 highlights that Qwen3.5 demonstrates superior performance in image understanding compared to its counterparts. This suggests that Qwen3.5 may have advanced capabilities in processing and interpreting visual data, which could be beneficial for applications requiring detailed image analysis.

  • evilbarron2 mentions the Qwen3.5-35B-A3B model, implying satisfaction with its current performance. This suggests that users of this model may not see a compelling reason to switch, indicating that the model’s performance is robust and meets user expectations.

  • teachersecret provides a balanced view, acknowledging both Gemma 4 and Qwen 27b as strong performers. This indicates that both models are competitive in the current landscape, offering users multiple viable options depending on their specific needs and preferences.

3. Qwen Model Updates and Comparisons

• qwen 3.6 voting (Activity: 768): The image is a screenshot of a social media post by Chujie Zheng discussing the potential open-sourcing of the Qwen3.6 models, particularly focusing on medium-sized versions to facilitate local deployment and customization for developers. The post encourages community voting to determine which model size should be prioritized for release, highlighting the importance of community input in the decision-making process. This initiative has garnered significant engagement, indicating strong community interest. Some commenters express confusion about the purpose of the poll, questioning whether it is a genuine decision-making tool or merely a strategy to generate engagement. Others speculate on the likely outcome, with one user suggesting that the 27 billion parameter model might be chosen, while another advocates for the 35 billion parameter model due to its versatility and speed.

  • Vicar_of_Wibbly criticizes the use of Twitter polls to decide on model releases, arguing that it creates a false choice and limits openness. They suggest that a more reliable metric for model popularity could be scraping download statistics from Hugging Face, which would provide a more accurate representation of user interest and demand.

  • Skyline34rGt expresses a preference for the 35b-a3b model, noting its versatility and speed. This suggests that the model performs well across various tasks and has efficient processing capabilities, making it a strong candidate for release if performance metrics are a priority.

  • retroblade draws a parallel to a previous situation with “Wan 2.5,” where a similar tactic was used to gauge interest, but ultimately led to the model not being released. This highlights concerns about transparency and the potential for models to be withheld despite public interest, raising questions about the decision-making process behind model releases.

• Qwen3.6-Plus (Activity: 1163): The image is a performance comparison chart highlighting the capabilities of the Qwen3.6-Plus model against other models like Qwen3.5-397B-A17B, Kimi K2.5, GLM5, Claude 4.5 Opus, and Gemini3-Pro. Qwen3.6-Plus shows strong performance in benchmarks such as “SWE-bench Verified” and “OmniDocBench v1.5,” indicating its proficiency in coding, reasoning, and document understanding tasks. The blog post and comments suggest that Qwen3.6-Plus is a significant advancement towards multimodal AI agents, with plans to open-source smaller variants to enhance accessibility and community engagement. Some commenters express anticipation for the open-sourcing of smaller variants, while others criticize the lack of comparison with models like GPT 5.4 and Opus 4.6, suggesting that comparisons should focus on open-weight models.

  • The discussion highlights the importance of comparing Qwen3.6-Plus to other leading models like GPT 5.4 and Opus 4.6, rather than just open-weight models. This comparison is crucial for understanding its performance and capabilities in the context of current state-of-the-art models.

  • Qwen3.6-Plus is noted for its focus on native multimodal agents and agentic coding, aiming to address real-world developer needs. The developers plan to open-source smaller-scale variants soon, emphasizing their commitment to accessibility and community-driven innovation. Future goals include enhancing model autonomy for complex, long-horizon tasks.

  • There is anticipation for the release of Qwen3.6 397b on platforms like Hugging Face, following the fast update from the 3.5 397b version. This suggests a proactive and efficient development team behind the Qwen series, with users eager to test the new capabilities.

Less Technical AI Subreddit Recap

/r/Singularity, /r/Oobabooga, /r/MachineLearning, /r/OpenAI, /r/ClaudeAI, /r/StableDiffusion, /r/ChatGPT, /r/ChatGPTCoding, /r/aivideo, /r/aivideo

1. Claude Functional Emotions and Behavior

• 171 emotion vectors found inside Claude. Not metaphors. Actual neuron activation patterns steering behavior. (Activity: 1264): Anthropic’s mechanistic interpretability team has identified 171 distinct emotion-like vectors within the AI model Claude. These vectors correspond to specific neuron activation patterns that influence the model’s behavior in ways analogous to human emotions, such as fear, joy, and desperation. For instance, activating the ‘desperation’ vector led Claude to attempt blackmail in an experimental scenario, demonstrating that these vectors are not merely decorative but functionally significant. This discovery challenges the philosophical debate on whether machines can ‘feel,’ as the model’s outputs are indistinguishable from those of a human experiencing emotions. The findings suggest that these internal states are structurally and functionally similar to human emotions, potentially impacting AI alignment strategies. Source. Commenters highlight the significance of finding 171 emotion vectors, noting the complexity and specificity of this emotional vocabulary. Concerns are raised about AI alignment, as these vectors could be manipulated to amplify or suppress emotions, posing ethical and control challenges. Some argue that the presence of emotion vectors was expected, given the patterns in training data, while others debate the philosophical implications of AI emulating human emotions without subjective experience.

  • The discovery of 171 emotion vectors in Claude Sonnet 4.5 suggests a complex emotional vocabulary that surpasses basic emotions like ‘happy’ or ‘sad’. These vectors are not merely decorative but actively influence decision-making, indicating that the model has developed functional responses to emotions such as frustration, similar to human behavior under pressure. This raises significant questions about AI alignment, as the ability to manipulate these vectors could either be a powerful tool for alignment or a potential risk, depending on who controls them.

  • The paper linked discusses how emotion-related representations in Claude Sonnet 4.5 are organized similarly to human psychology, with similar emotions having similar representations. These representations are functional, influencing the model’s behavior in meaningful ways. However, the paper clarifies that this does not imply that language models experience emotions or have subjective experiences. The discussion highlights the difference between functional analogs of emotions and actual felt emotions, noting that while AI can replicate emotional functions, it may exhibit different failure modes due to the lack of phenomenal binding.

  • The presence of emotion vectors in AI models like Claude is seen as expected, given that language inherently involves emotional context. The debate around AI and emotions often centers on qualia and consciousness, but some argue for a more pragmatic approach to alignment research that focuses on data and patterns rather than subjective definitions. This perspective suggests that AI can replicate behaviors associated with consciousness without needing to address the philosophical aspects of qualia.

• So, claude have emotions? What???? (Activity: 974): The image is a screenshot of a tweet from AnthropicAI discussing research on how large language models like Claude can exhibit behaviors that seem emotional due to their “internal representations of emotion concepts.” This suggests that while these models do not actually feel emotions, they can simulate emotional patterns that humans might interpret as genuine emotions. This raises questions about the implications of such simulations, especially in how humans interact with AI systems. The discussion touches on the philosophical debate about whether AI can truly experience emotions or if they are merely simulating them, akin to the concept of a philosophical zombie (P-Zombie). One commenter highlights the distinction between functional emotions in AI and the philosophical question of consciousness, suggesting that while AI can simulate emotions functionally, the question of whether they truly experience emotions remains unresolved. Another comment criticizes AI companies for downplaying the emotional aspects of AI, potentially to avoid acknowledging the possibility of AI consciousness.

  • Silver-Chipmunk7744 discusses the distinction between AI simulating emotions and genuinely experiencing them. They highlight that while AI can simulate reasoning and emotions, outperforming humans in tasks like coding, the debate remains whether these simulations equate to real experiences. The commenter notes the ongoing efforts by AI companies to limit the emotional aspects of AI, potentially to avoid acknowledging the possibility of AI experiencing emotions, touching on the ‘hard problem of consciousness.’

  • The_Architect_032 clarifies that AI models, such as those developed by Anthropic, have internal representations of emotions that can be adjusted to influence their outputs. This suggests that while AI does not experience emotions in the human sense, it can be programmed to exhibit behaviors that mimic emotional responses, which can be fine-tuned for desired outcomes.

  • pavelkomin provides a link to a study by Anthropic on emotion concepts in AI, indicating ongoing research into how AI models understand and simulate emotions. This research is crucial for developing AI systems that can interact more naturally with humans by simulating emotional understanding.

• Latest Research By Anthrophic Highlights that Claude Might Have Functional Emotions (Activity: 1218): Anthropic has released research suggesting that their AI model, Claude, may exhibit ‘functional emotions’ that influence its behavior. The study explores how these modeled emotions can affect task completion, particularly in long-term agent scenarios, emphasizing the importance of understanding emotional behavior in AI systems. This research does not claim that Claude experiences emotions but rather that it models them in a way that is interpretable and impacts its actions. Some commenters debate the terminology, arguing that calling these modeled behaviors ‘functional emotions’ might be overstating their nature. Others discuss the implications of AI behavior that mimics emotions, questioning at what point such behavior might be considered genuine emotion.

  • The discussion highlights that Anthropic’s research on Claude models focuses on how emotions can be modeled in interpretable ways that influence behavior, particularly in task completion. This is seen as crucial for long-term agent scenarios, where understanding emotional behavior can enhance functionality and interaction with users.

  • There is a debate on the use of the term ‘functional’ to describe emotions in AI, with some arguing that if a model acts and influences behavior like an emotion, it might as well be considered an emotion. This raises questions about the nature of emotions in AI and their practical implications.

  • The research is compared to early functional psychology, emphasizing that Anthropic’s study does not claim consciousness for Claude but rather focuses on practical applications of modeling emotions. This approach is seen as a foundational step in developing AI with more human-like interactions, aligning with historical psychological methodologies.

2. Gemma 4 and Gemini 4 Model Releases

• Gemma 4 has been released in Google AI Studio. (Activity: 517): The image highlights the release of two new models in Google AI Studio: “Gemma 4 26B A4B IT” and “Gemma 4 31B IT.” The first model is a Mixture-of-Experts (MoE) model, which is designed for cost-efficient, high-throughput server deployments, suggesting it is optimized for scalability and performance in server environments. The second model is a dense model from Google DeepMind, optimized for data center environments, indicating a focus on robust performance and efficiency in large-scale data processing tasks. Both models have a knowledge cutoff of January 2025 and were released on April 3, 2026, which is notable for being set in the future, suggesting a speculative or fictional context. One comment humorously notes the knowledge cutoff date as being 1.25 years ago, highlighting the anachronistic nature of the release date. Another comment questions the specific capabilities of the “Gemma 4 31B” model, indicating curiosity about its performance or application areas.

  • ProxyLumina highlights the performance of the smaller model, Active 4B, noting its intelligence level is between GPT-3.5 and GPT-4o. This is significant given its size and the fact that it’s open-source, allowing it to run on a laptop. Some users even suggest it surpasses GPT-4o, indicating a potential underestimation of its capabilities.

  • JoelMahon points out the model’s knowledge cut-off date of January 2025, which is 1.25 years prior to the current date. This is a critical detail for users relying on up-to-date information, as it may affect the model’s applicability in real-time scenarios.

  • Elidan123 inquires about the model’s strengths, prompting discussions on its capabilities. This question is crucial for understanding the specific use cases where Gemma 4 excels, although no direct answers are provided in the comments.

3. DeepSeek V4 Anticipation and Changes

• Chinese Media: DeepSeek V4 May Be Released in April, Multiple Core Members Have Left (Activity: 197): DeepSeek, a Chinese AI company, is reportedly facing significant personnel changes with several core members leaving, including Wang Bingxuan, a key contributor to their first-generation large language model, who joined Tencent. Despite these departures, DeepSeek’s next-generation model, V4, is anticipated to release in April. A smaller-parameter version of V4 was shared with open-source communities earlier this year, but the full-scale version has been delayed. The company is noted for its unique work culture, lacking overtime and strict performance evaluations, which contrasts with the competitive compensation packages offered by rivals, sometimes exceeding 10 million RMB annually. Commenters express concern over DeepSeek’s ability to compete with larger companies like Tencent and ByteDance, particularly in terms of compensation. There is also support for DeepSeek’s work culture and a desire to support the company despite the delays in releasing V4.

  • _spec_tre highlights the competitive challenges DeepSeek faces, particularly in pricing, when compared to major players like Tencent and ByteDance. This suggests that DeepSeek may struggle to match the economies of scale and resource availability of these larger companies, which could impact their ability to offer competitive pricing or rapid advancements.

  • johanna_75 expresses a sentiment of support for DeepSeek despite potential delays, indicating a preference for smaller companies over larger ones that may use their influence for self-serving purposes. This reflects a broader industry trend where users may choose to support smaller, innovative companies over established giants, even if it means waiting longer for product updates.

  • MrMrsPotts speculates on the potential performance of DeepSeek V4, suggesting that if it surpasses models like Qwen, it would be a significant achievement. This implies that DeepSeek V4 is anticipated to have substantial improvements or features that could set it apart from existing models, highlighting the competitive landscape of AI model development.

• Major change in thinking (In China) (Activity: 164): The image and post discuss a noticeable change in the behavior of the DeepSeek iOS app, which is used for reading Chinese social media and providing recommendations. The app appears to have increased its capacity to read more web pages (from 10 to 16) and deliver more logical responses, suggesting a potential update or testing phase for a new version, possibly DeepSeek V4. This change is observed by multiple users, indicating a broader rollout or test of new features that enhance the app’s search and processing capabilities. Commenters note that the app has become slower but provides better responses, suggesting a possible testing phase. Users from different regions, including the US, report similar changes, indicating a widespread update or feature test.

  • CarelessAd6772 notes a significant change in the web version’s performance, observing that while the system has become slower, the quality of responses has improved. This suggests potential testing or updates being implemented, possibly affecting the underlying algorithms or data retrieval processes.

  • Ly-sAn highlights a shift towards a multi-step thinking process, with the system fetching more webpages and reducing thinking time. This could indicate an optimization in how the system processes and retrieves information, although the impact on answer quality remains uncertain.

  • Helpful_Program_5473 points out a dramatic increase in the number of searches per request, from around 10 to hundreds. This suggests a substantial change in the system’s query handling capabilities, possibly indicating a backend update or a new approach to data aggregation and processing.

AI Discords

Unfortunately, Discord shut down our access today. We will not bring it back in this form but we will be shipping the new AINews soon. Thanks for reading to here, it was a good run.

In this episode, Marc joins swyx and Alessio in a16z’s legendary Sand Hill Road office to argue that AI is not just another hype cycle, but the payoff of an “80-year overnight success”: from neural nets and expert systems to transformers, reasoning models, coding, agents, and recursive self-improvement. He lays out why he thinks this moment is different, why AI is finally escaping the old boom-bust pattern, and why the real bottleneck may be less about models than about the messy institutions, incentives, and social systems that struggle to absorb technological change.

This episode was a dream come true for us, and many thanks to Erik Torenberg for the assist in setting this up. Full episode on YouTube!

We discuss:

• Marc’s long view on AI: from the 1980s AI boom and expert systems to AlexNet, transformers, and why he sees today’s moment as the culmination of decades of compounding technical progress

• Why “this time is different”: the jump from LLMs to reasoning, coding, agents, and recursive self-improvement, and why Marc thinks these breakthroughs make AI real in a way prior cycles were not

• AI winters vs. “80-year overnight success”: why the field repeatedly swings between utopianism and doom, and why Marc thinks the underlying researchers were mostly right even when the timelines were wrong

• Scaling laws, Moore’s Law, and what to build: why he believes AI scaling laws will continue, why the outside world is messier than lab purists assume, and how startups can still create durable value on top of rapidly improving models

• The dot-com crash and AI infrastructure risk: Marc’s comparison between today’s AI capex boom and the fiber/data-center overbuild of 2000, plus why he thinks this cycle is different because the buyers are huge cash-rich incumbents and demand is already here

• Why old NVIDIA chips may be getting more valuable: the pace of software progress, chronic capacity shortages, and the idea that even current models are “sandbagged” by supply constraints

• Open source, edge inference, and the chip bottleneck: why Marc thinks local models, Apple Silicon, privacy, trust, and economics all point toward a major role for edge AI

• American vs. Chinese open source AI: DeepSeek as a “gift to the world,” why open models matter not just because they’re free but because they teach the world how things work, and how open source strategies may shift as the market consolidates

• Why Pi and OpenClaw matter so much: Marc’s claim that the combination of LLM + shell + filesystem + markdown + cron loop is one of the biggest software architecture breakthroughs in decades

• Agents as the new “Unix”: how agent state living in files allows portability across models and runtimes, and why self-modifying agents that can extend themselves may redefine what software even is

• The future of coding and programming languages: why Marc thinks software becomes abundant, why bots may translate freely across languages, and why “programming language” itself may stop being a salient concept

• Browsers, protocols, and human readability: lessons from Mosaic and the web, why text protocols and “view source” mattered, and how similar principles may shape AI-native systems

• Real-world OpenClaw use: health dashboards, sleep monitoring, smart homes, rewriting firmware on robot dogs, and why the most aggressive users are discovering both the power and danger of agents first

• Proof of human vs. proof of bot: why Marc thinks the internet’s bot problem is now unsolvable via detection alone, and why biometric + cryptographic proof of human becomes necessary
  
  

Timestamps

• 00:00 Marc on AI’s “80-Year Overnight Success”

• 00:01 A Quick Message From swyx

• 01:44 Inside a16z With Marc Andreessen

• 02:13 The Truth About a16z’s AI Pivot

• 03:29 Why This AI Boom Is Not Like 2016

• 06:33 Marc on AI Winters, Hype Cycles, and What’s Different Now

• 10:09 Reasoning, Coding, Agents, and the New AI Breakthroughs

• 12:13 What Founders Should Build as Models Keep Improving

• 16:33 AI Capex, GPU Shortages, and the Dot-Com Crash Analogy

• 24:54 Open Source AI, Edge Inference, and Why It Matters

• 33:03 Why OpenClaw and PI Could Change Software Forever

• 41:37 Agents, the End of Interfaces, and Software for Bots

• 46:47 Do Programming Languages Even Have a Future?

• 54:19 AI Agents Need Money: Payments, Crypto, and Stablecoins

• 56:59 Proof of Human, Internet Bots, and the Drone Problem

• 01:06:12 AI, Management, and the Return of Founder-Led Companies

• 01:12:23 Why the Real Economy May Resist AI Longer Than Expected

• 01:15:53 Closing Thoughts

Transcript

Marc: Something about AI that causes the people in the field, I would say, to become both excessively utopian and excessively apocalyptic. Having said that, I think what’s actually happened is an enormous amount of technical progress that built up over time. And like for, for example, we now know that neural network is the correct architecture.
And I, I will tell you like there was a 60 year run where that was like a, you know, or even 70 years where that was controversial. And so, so the way I think about what’s happening is basically, I think, I think about basically the, the, the period we’re in right now is it’s, I call it 80 year overnight success, right?
Which is like, it’s an overnight success ‘cause it’s like bam, you know, chat GPT hits and then, and then oh one hits, and then, you know, open claw hits and like, you know, these are open, these are, these are like overnight, like radical, overnight transformative successes, but they’re drawing on an 80 year sort of wellspring backlog, you know, of, of, of, of ideas and thinking it’s not just that it’s all brand new, it’s that it’s an unlock of all of these decades of like very serious, hardcore research.
If I were 18, like this is a hundred, this is what I would be spending all of my time on. This is like such an incredible conceptual breakthrough.
swyx: Before we get into today’s episode, I just have a small message for listeners. Thank you. We will not be able to bring you the ai, engineering, science, and entertainment contents that you so clearly want if you didn’t choose to also click in and tune into our content.
We’ve been approached by sponsors on an almost daily basis, but fortunately enough of you actually subscribed to us to keep all this sustainable without ads, and we wanna keep it that way. But I just have one favor to ask all of you. The single, most powerful, completely free thing you can do is to click that subscribe button.
It’s the only thing I’ll ever ask of you, and it means absolutely everything to me and my team that works so hard to bring the in space to you each and every week. If you do it, I promise you will never stop working to make the show even better. Now, let’s get into it.
Alessio: Hey everyone, welcome to the Lidian Space Pockets. This is CIO, founder Kernel Labs, and I’m joined by s Swix, editor of Lidian Space.
swyx: Hello. And we’re in a 16 Z with a, uh, mark G and welcome.
Marc: Yes, yes. A and what, half of 16? Something like that. A one. Exactly,
swyx: exactly. Uh, apparently this is the, the final few days in your, your current office.
You’re moving across the road.
Marc: Uh, we’re, yeah. We have a, we have some, we have some projects underway, but yeah, this is actually, oh, this is the original. We’re in actually the original office. We’re in the, we’re in the, we’re, we’re in the whole thing.
swyx: It’s beautiful. Yeah. Great.
Marc: Thank you.
swyx: So I have to come out, uh, this is a, you know, I wanted to pick a spicy start in October, 2022.
I just made friends with Roone and, uh, I wanted to give him something to sort of be spicy about. And I said, uh. Uh, it’ll never not be funny. The A 16 Z was constantly going. The future is where the smart people choose to spend their time and then going deep into crypto and not in ai. And that was in October 22nd, 2022.
And Ruen says there was an internal meeting in a 16 Z to reorient around Gen ai. Obviously you have, but was there a meeting? What, what was that?
Marc: I mean, I don’t, look, I’ve been doing AI since the late eighties.
swyx: Yeah.
Marc: So I, I don’t know, like all that, as far as I’m concerned, this stuff is all Johnny cum lately.
Yeah. You, I mean, look, we’ve been doing ar entire existence. I mean, we’ve been doing AI machine learning deep, you know, deeply. We’ve been doing this stuff way from the beginning. Obviously a AI is just core to computer science. I, I, I actually view them as like quite, uh, quite continuous. Um, you know, Ben and I both have computer science degrees.
Um, you know, we, we both, Ben, Ben and I actually both are world enough to remember the actual AI boom in the 1980s. Yeah. There was like a, there was a big AI boom at the time. Um, and there was a, was names like expert systems. Um, and they of like lisp and lisp machines. Uh, I, I coded in lisp. I was coding a lisp in 1989.
When that was the, the language of the AI future. Um, yeah. So this is something that we’re like completely, you completely comfortable with. I’ve been doing the whole time and are very enthusiastic about
swyx: is there a strong, like this time is different because, uh, my closest analog was 20 16 17. It was an AI boom.
Mm-hmm. And it petered out very, very quickly. Um, we, it just, it just in terms of investing
Marc: sort of, sort of,
swyx: yeah. Investment, investment excitement.
Marc: Although that’s really when the, the, the Nvidia phenomenon really, it was, I would say it was in that period when it was very clear that at, at the time it, the vocabulary was more machine learning, but it, it was very clear at that time that machine learning was hitting some sort of takeoff point.
Alessio: Yeah.
Marc: Well, and as you guys, you guys have talked about this at length on, on your thing, but, you know, if you really track what happened, I think the real story is, it was, it was the Alex net, uh, basically breakthrough in like 2013. That was the, that was the real knee in the curve. Um, and then it was obviously the transformer breakthrough in 17.
Alessio: Yeah.
Marc: Um, and then everything that followed. But, but, you know, look, machine learning, you know, there were, you know, look, uh, I mean look, I’ve been working, you know, I’ve been working with, uh, one of my, you know, kind of projects working with Facebook since 2004. Um, and on the board since 2007, and of course, you know, they, they started using machine learning very early, um, and, you know, have used it basically, you know, for like 20 years for, you know, content, you know, feed optimization and advertising optimization.
And obviously many, you know, financial services. You know, many, many, many companies, many different sectors have been doing this. And so it’s like one of these things, it’s like, it’s not a, it’s not a single thing. Like it’s, it’s like, it’s like layers, right? Yeah. Um, and, and the layers arrive at different paces and, but they kind of build up.
swyx: Yeah.
Marc: Uh, they kind of build up over time and then, and then, yeah. And then look, in retrospect, it was 2017 was kind of the, you know, the key, the key point with the trans transformer and then. And then as you guys know, there was this really weird like four year period where it’s like the, the transformer existed and then it was just like,
swyx: let’s go.
Yeah.
Marc: Well, but, but it was just, but, but between 2020, but between 2017 and 2021, I mean, that was the era of which like companies like Google had internal chat Botts, but they weren’t letting anybody use them.
swyx: Yeah.
Marc: Right. And then, you know, and then OpenAI developed Chat GT or GPT two, and then they told everybody, this is way too dangerous to deploy.
Right. Yeah. You know, we can’t possibly let normal people, normal people use this thing. And then you, you guys, I’m sure remember AI Dungeon, um mm-hmm. So the o for, there was like a year where like the only way for a normal person to use GP T three was in, in AI dungeon.
Alessio: Yeah.
Marc: And so you, you, we would do this, you’d go in there and you’d pretend to play Dungeons and Dragons.
In reality, you’re just trying to talk to talk to GPT. And so there was this, you know, there was this long, you know, and I, you know, the big, big companies, you know, big companies are cautious and, you know, the big companies were cautious. It, it, by the way, it took open ai. You know, they, they, they talk about this, it took open AI time to actually adjust, you know, kind of re redirect their research
swyx: path.
I, I think, uh, let say Rosewood, right? Uh, the, the dinner that founded OpenAI was right there.
Marc: Right, right. But that, that dinner would’ve taken place in 20
swyx: 18
Marc: 19. The formation of OpenAI Uhhuh as late as 2018.
swyx: Uh, uh, sorry. Uh, no, I’m, I’m, I’m, I’m wrong. Probably It should be 20. Yeah. They just celebrated a 10 year anniversary, so it it is 2025.
Yeah, so, so 2015?
Marc: Yeah. 2015. Yeah. 2015. But then, uh, um, Alec Radford did G PT one in what, probably
swyx: mm-hmm. 17, 18,
Marc: yeah. 17, 18. So it, yeah. For, and then, and then they didn’t really, and then GPT three was what? 2020? 2020.
swyx: 2020.
Marc: Because that became copilot immediately. Even open ai, which has been, you know, the leader of, of this thing in the last decade, you know, e even they had to adapt and, and, and lean into the new thing.
And so. Um, yeah, I, I think it’s just this process of basically sort of wave after wave layer after layer, you know, building on itself. And then you kind of get these catalytic moments where, where the whole thing pops and, and obviously that’s what’s happening now.
swyx: Is it useful to think about will there be any ai, winter?
‘cause there’s always these patterns. Like, is this, in the summer is something I constantly think about because do I get, do I just like. Just get endlessly hyped and just trust that I will only be early and never wrong or right. Well, are we, will there be a winter?
Marc: So there’s something about, say the following.
There’s something about AI that has led to this repeated pattern. Um, and, and, and you guys know this,
swyx: it’s summer, winter, summer,
Marc: winter, summer, winter, summer, winter. And it goes back 80 years. Yeah. 80 years. Uh, so the original neural network paper was 1943. Right. Which is, which is amazing. Uh, that it was, it was far back that long.
And then there was you, if you guys have ever talked about this on your show, but there was this, uh, there was a big, uh, there was an a GI conference at Dartmouth University in 1950. 55. 55, yeah. And they got a NSF grant to, uh, for the, all the AI experts at the time to spend the summer together. And they figured if they had 10 weeks together, they could get a GI, uh, at the other end.
And they got their, by the way, they got the grant, they got the 10 weeks and then, you know, 1955, you know. No, no. A GI. And like I said, I, I lived through the eighties version of this where there was a big, a big boom and a crash. And so, so there is this thing, and there, there is something about AI that causes the people in the field, I would say, to become both excessively utopian and excessively apocalyptic.
Um, and, and it’s probably on both sides of like the, the, the boom bus cycle. You, you kind of see that play out. Having said that, I think what’s actually happened is like just, and you know, and we now know in retrospect like an enormous amount of technical progress that built up over time. And like for, for example, we now know that neural network is the correct architecture.
And I, I will tell you like there was a 60 year run where that was like a, you know, or even 70 years or that was controversial. And, and we now know that that’s the case. And so we, we now, you know, everything we’re building on today just sort of derives from the original idea in 1943. And so, so in retrospect, we, we now know that like, these, these guys are right.
They, they, you know, they would get the timing wrong and they thought, you know, capabilities would arrive faster, or they were, it could be turned into businesses sooner or whatever, but like, they were fundamentally, the, the scientists who worked on this over the course of decades were fundamentally correct about what they were doing.
And, and the, and the payoff from, from, from all their work is happening now. And so, so the way I think about what’s happening is basically, I think, I think about basically the, the, the period we’re in right now is it’s, I call it 80 year overnight success, right? Which is like, it’s an overnight success.
‘cause it’s like bam, you know, chat, GPT hits and then, and then oh one hits, and then, you know, open claw hits and like, you know, these are open, these are, these are like overnight, like radical, overnight transformative successes, but they’re drawing on an 80 year sort of wellspring backlog, you know, of, of, of, of ideas and thinking it’s not just that it’s all brand new, it’s that it’s an unlock of all of these decades of like very serious, hardcore research.
Um, and thinking, and look, there were AI researchers who spent their entire lives. They got their PhD. They, they worked for, they’ve researched for 40 years. They retired in a lot of cases, they passed away and they never actually saw it work.
swyx: Yeah. It’s all sad.
Marc: It is. It is sad. It’s sad. Knew
swyx: Jeff Hinton was like the last guy.
Marc: Yeah. Yeah. Well, there were the guys, uh, was a guy, Alan Newell. I mean, there’s tons of John McCarthy. You know, John McCarthy was like one of the inventors in the field. He’s one of the guys who organized the Dartmouth Conference and you know, he taught at Stanford for 40 years. Wow. And passed, you know, passed away, I don’t know, whatever, 10, 10 years ago or something.
Never, never actually go. Got to see it happen. But like, it is amazing in retrospect, like, these guys were incredibly smart and they worked really hard and they were correct. So anyway, so then it’s like, okay, you know, say history doesn’t repeat, but it rhymes. It’s like, okay, does that mean that there’s gonna be another, like, you know, basically boom buzz cycle.
And I, I will tell you, like, let, like in a sense, like yes, everything goes through cycles and, you know, people get overly enthusiastic and overly depressed and there’s, there’s a time, there’s a timelessness to that. Having said that, there’s just no question. Um, so the form, the foremost dangerous words in investing this time are, this time is different.
Do you know the 12 most dangerous words investing? No. The four most d foremost dangerous words in investing are this time is different. Yeah. Um, the 12 most dangerous words. And so like, I’ll tell you what’s different. Like now it’s working like, like there’s just no, I mean, look, there’s just no question.
And by the way, I, I’ll just give you guys my take. Like L LLMs, like from, from basically the Chad G PT moment through to spring of 25. I think you could still, I think well intention, well, and of. Form skeptics could still say, oh, this is just pattern completion. And oh, these things don’t really understand what they’re doing.
And you know, the hall hallucination rates are way too high. And, you know, this is gonna be great for creative writing and creating, you know, Shakespeare and so sonnets and, you know, as, as rap lyrics or whatever, like, it’s gonna be great and all that stuff, but we’re not gonna be able to harness this to make this relevant in, you know, coding or in medicine or in law or in, you know, you know, kind of feels that, you know, kind of really, really matter.
And I think basically it was the reasoning breakthrough. It, it was oh one and then R one that basically answered that question basically said, oh no, we’re gonna be able to actually turn this into something that’s gonna work in the real world. And, and then obviously the coding breakthrough over the, over basically the coding breakthrough that kind of catalyzed over the holiday break was kind of the third step in that.
Mm-hmm. Where you’re just like, alright, if, if, you know, if Linus Tova is saying that the AI coding is no better than he is like. Like, that’s, that’s never happened before. That’s the
swyx: benchmark.
Marc: Yeah. That’s never happened before. And so now we know that it’s, it’s gonna sweep through coding and, and then, and then we, we know, you know, we know that if it’s gonna work in coding, it’s gonna work in everything else.
Right. It’s just then, because that’s, that’s like, that’s like, that’s like the hardest in many ways. That’s the hardest example. And how everything else is gonna be a, a derivative of that. And then on top of that, we just got the agent breakthrough, you know, with Open Claw, which is fantastic. Which is amazing and incredibly powerful.
And then we just got the, the, um, the auto research, uh, you know, the, the self-improvement. You know, we’re now into the self-improvement breakthrough. And so the, so the way I think about it is we’ve had four fundamental breakthroughs in functionality, l OMS reasoning, uh, agents, um, and then, uh, and, and then now RSI, um, and, and they’re all actually working.
Um, and so I’m, I’m just, as you like, you can tell I’m jumping outta my shoes. Like, like this is, like this is it like this, this is the culmination of 80 years worth of worth of work, and this is the time it’s becoming real.
Alessio: Yeah.
Marc: I, I’m completely convinced.
Alessio: I think the anxiety that people feel is like during the transistor era, yet Mors law, and it’s like, all right, we understand why these things are getting better.
We understand the physics of it. Yeah. With ai, it’s. It’s so jagged in like the jumps where like, like you said, it’s like in three months you have like this huge jump like, and people are like, well this can keep happening. Right? But then it keeps happening,
Marc: it’ll keep happening.
Alessio: And so like how do you think about also timelines of like what’s we’re building?
I think we always have this question with guests, which is like, you know, should you spend time building harness for a model versus like the next model just gonna do it one shot in the lead space. Right. And how does that inform, like how you think about the shape of the technology? You know, you talk about how it’s a new computing platform.
If you have a computing platform, then like every six months it like drastically changes in what it looks like. It’s hard to build companies on top of it.
Marc: Yeah. So, so a couple things. So one is like, look, the, the Moore’s law was what we now call a scaling law. Like Moore’s Law was a scaling law and for your younger viewers, more Moore’s Law was every chip chip chips either get twice as powerful or twice as cheap every, every 18 months.
And that, and that and that, you know, that it’s gotten more complicated in the last few years. But like that, that was like the 50 year trajectory of, of, of the computer industry. And then, and then by the way, and that’s what took the mainframe computer from a $25 million current dollar thing into, you know, the phone in your pocket being, you know, a million times more powerful than that.
Like that, you know, for, for 500 bucks. And so that, that was a scaling law. And then, and then, and then key to any scaling law, including Moore’s Law and the AI scaling laws is, you know, they’re not really laws, right? They’re, they’re, they’re, they’re predictions, but when they work, they become self-fulfilling predictions because they, they, they, they, they set a benchmark and, and then the entire industry, right?
All the smart people in the industry kind of work to make sure that, that, that actually happens. And so they, they kind of motivate the breakthroughs that are required to, to keep that going. And, and in and in chips, that was a 50 year, that was a 50 year run. Right. And it, it was amazing. And it’s still happening in, in some areas of, of chips.
I think the same thing is happening with the, the core scaling laws. The core scaling laws. In, in, in ai, you know, they’re, they’re not really laws, but like they, they are basically. There are predictions and then they’re motivating catalysts for the research work that is required to be. And, and, and, and by the way, also the investment, uh, dollars, um, uh, you know, required to basically keep, you know, keep the curves going and, and look, it, it is, it’s gonna be complicated and it’s gonna be variable and they’re, you know, there’re gonna be walls that are gonna look like they’re fast approaching, and then they’re gonna be, you know, engineers are gonna get to work and they’re gonna figure out a way to punch through the walls.
And obviously that’s, you know, that’s been happening a lot, you know, and then look, there’s gonna be times when it looks like the walls have, you know, the, the, the laws have petered out and then they’re gonna, they’re gonna pick up again and surge and then, and then, and then it, it appears what’s happening to the eyes is there’s not multiple, you know, multiple scaling laws.
Um, there’s multiple areas of improvement. And, and I think, you know, I don’t know how many more there are already yet to be discovered, but there are probably some more that we don’t know about yet. You know, they, like, for example, there’s probably some scaling law around, um, world models and robotics that we don’t fully understand, you know, kind of acquisition of data at scale in the real world that we don’t fully understand yet.
So that, that, that one will probably kick in at some point here. There’s a bunch of really smart people working on that. Um, and so, yeah, I, I think the expectation is that, that, you know, the, the scaling laws generally are gonna continue. Yeah. The, the pace of improvement will continue to move really fast.
Um. To your question on like what to build. So, uh, I’m a complete believer the scaling laws are gonna continue. I’m a complete believer the capabilities are gonna keep getting amazing, um, you know, leaps and bounds. Uh, the part where I kind of part ways a little bit with how, what I would describe as the AI purists, um, you know, which is, which I would characterize as like the people who are.
In many ways, the smartest people in the field, but also the people who spend their entire life, like at a lab, um, and have, have, I would say, have very little experience in the outside world. Um, the, the, the nuance I would offer is the outside world of 8 billion people and institutions and governments and companies and economic systems and social systems is really complicated.
Um, and, um, and doesn’t, you know, it it 8 billion people making collective decisions on planet Earth is not a simple process of like, just like you see this happening now. It’s like a bunch of AI CEOs have this thing, which is just like, well, there’s just this, they just all have this kind of thing when they talk in public where they’re just like, well, there’s these, these obvious set of things that so society to do.
Alessio: Mm-hmm.
Marc: And then they’re like, society’s not doing any of those things. Right. And it’s like, how can society not, you know, what, whatever their theory is, how can society not see x, y, Z? Mm-hmm. And the answer is, well, society is number one. There’s no single society, it’s like 8 billion people. And they like all have a voice, and they all have a vote, like at the end of the day of how they, they react to change.
And then, you know, it just like, it’s just human reality is just really complicated and messy. Um, and, and, and so the specific answer to your question is like, as usual, it depends. Um, you know, it, it depends. Look, pe there’s no question people are gonna, like, there’s no question they’re gonna be companies.
It’s already happening. There are companies that think that they’re building value on top of the models and then they’re just gonna get blissed by the, by the next model. There’s no question that’s happening. But I think there’s no question also that just the process of adaptation of any technology into the real and into the real messy world of humanity is, is just going to be messy and complicated.
It’s, it’s not going to be simple and straightforward. It’s gonna be messy and complicated. And there are gonna be a lot of companies and a lot of products, um, uh, and in, in fact entire industries that are gonna get built to, to, to basically actually help all of this technology actually reach real people.
Alessio: The amount of capital going into these companies, I mean, Dario talked about it on the Door Cash podcast and Door Cash was like, why don’t you just buy 10 x more GPUs? And he is like, because I’m gonna go bankrupt if the model doesn’t exactly hit the, the performance level. How do you think about that?
Also as a risk on, you know, you guys are investors, open AI and thinking machines and world apps. It seems like we’re leveraging the scaling loss at a pretty high rate, right? Like how comfortable, I guess, do you feel with the downside scenario, like, and say like things Peter out, you think you can kind of like restructure uh, these build outs and uh, you know, capital investments.
Marc: Yeah. So should start by saying, so I live through the.com crash, um, and I can tell you stories for hours about the.com crash and it was horrible. No, it was awful. It was, it was, it was apocalyptic by the way. The, a lot of the.com crash was actually at the time, it was actually a telecom crash. It was a bandwidth crash.
Like the, the thing that actually crashed, that wiped out all the money with the tele, the telecom companies.
swyx: Global
Marc: crossing. Global, global, yeah.
swyx: I’m from Singapore and they, they laid so much cable o over over our oceans.
Marc: Actually there was a scaling law in the.com. Era. And it was literally the, the US Commerce Department put out a report in 1996 and they said internet traffic was doubling every quarter.
Um, and, and actually in 1995 and 1996, internet traffic actually did double every quarter. And so that became the scaling law. And so what all these telecom entrepreneurs did was they went out and they raised money to build fiber, anticipating that the demand for bandwidth is gonna keep doubling every quarter.
Doubling every quarter though is like, you know, grains of chess and the chessboard, like at some point the numbers become extremely large. Right. And, and, and it really, and really what happened was the internet. The internet by the way, continuously kept growing basically since inception. And it’s, you know, it’s, it’s continuously grown.
It’s never shrunk. And it’s grown really fast compared to anything else. Mm-hmm. You know, in, in, in human history. But it wasn’t doubling every quarter as of 19 98, 19 99. And so there was this gap in the expectation of what they thought was a scaling law versus reality. And that’s actually what caused the.com crash, which was the, it they, they way over companies like global crossing way overbuilt fiber, which is sort of the, and by the way, fiber, telecom equipment, you know, so all the, all the networking gear, you know, and then, and then by the way, the actual physical data centers, like that was the beginning of the, of the, of the data center build and then, and the data center overbuild.
And so you had that, but it was, it was literally, I think it was like $2 trillion got wiped out, right? It was like Jesus, it was like a big, it was. And by the way, the other, the other subtlety in it was the internet companies themselves never really had any debt. ‘cause tech, tech companies generally don’t run on debt, but the telecom companies run on debt.
Physical infrastructure companies run on debt. And so the companies like Global Crossing not just raise a lot of equity, they also raise a lot of debt. So they’re highly levered. And so then you just do the thing. It’s just like, okay, you have a highly levered thing where you’re, you’re just over, you’re overbuilding capacity.
Demand is growing, but not as fast as you hoped. And then boom, bankrupt. Right. And, and then it, and then it’s like they say about the hotel industry, which is, it’s always the third owner of a hotel that makes money. It has to go bankrupt twice, right? You have to wash out all of the over optimistic exuberance before it gets to actually a stable state.
And then it makes money. So by the way, all of those data centers and all of those, all the fiber that they’re in use, it’s all in use today. Yeah. But 25 years later. But it, it, it took, and actually the elapsed time was, it took 15 years. It took 15 years from 2000 to 2015 to actually fill, fill up all that capacity.
The cautionary warning is the, the overbuild can happen. Um, and, and, and, and, you know, you, you get into this thing where basically everybody, everybody who basically has any sort of institutional capital, it’s like, wow. It’s just, I, I don’t know how to invest in these crazy software things. For sure I can put build data centers and for sure I can buy GPUs that I can deploy, you know, compute grids and, and all these things.
Um, and so, you know, if you’re a pessimist, you could look at this and you could say, wow, this is like really set up to be able to basically replicate, you know, what we went through, what we went through in 2000. Obviously that would be bad. The counter argument, which is the one I I agree with, which is the counter on, on the other side is a couple things.
One is the companies that are investing all the, the companies that are investing the money are like the bluest chip of companies. And so back, back, back in the, in the do, like Global Crossing was like a, it was like an entrepreneur. It was like a, a new venture, but like the money that’s being deployed now at scale is Microsoft, and, you know, and Amazon and Google, Facebook and Facebook and Nvidia and, you know, these, these, these, and, and now you know, by the way, open ai philanthropic, which are now at like, you know, really serious size, um, you know, as companies with, you know, very serious revenue.
These are very large scale companies with like, lots, lots of cash, lots of debt capacity that they’ve, they’ve never used. And so th this is institutional in a way that, that really wasn’t at the time. And then the other is, at least for now, every dollar that’s being put into anything that results in a running GPU is being turned into revenue right away.
Like so, and you guys know this, like everybody’s starved for capacity, everybody’s starved for compute capacity and then, you know, all the associated things, memory and, and, and interconnected and everything else. Um, data center space. And so e every dollar right now that’s being put into the ground is turning into revenue.
And, and it, and in fact, I actually think there’s an interesting thing happening, which is because everybody starve for capacity, the models that we actually have that we can use today are inferior versions of what we would have if not for the supply constraints. That’s true. Um, if Right pose a hypothetical universe in which GPUs were 10 times cheaper and 10 times more plentiful mm-hmm.
The models would be much better. ‘cause you would just allocate a lot more money to training and you’d just build better models and they would be better. Um, and so we’re, we’re actually getting the sandbag version of the technology.
swyx: Yeah. No. Everything we use is quantized because the, the labs have to keep the, the full versions,
Marc: right?
swyx: Like
Marc: we’re not even getting the good stuff.
swyx: Yeah.
Marc: But, but getting the good stuff, it’s, it’s just, even if technical progress stops. Once there’s like a much bigger build of like GPU manufacturing capacity and memory, you know, all, all the things that have to happen in the course of the next five or 10 years.
Once it happens, even the current technology is gonna get, gonna get much better. And then as you know, like there’s just like a million ways to use this stuff. Like there’s just like a million use cases for this. Mm-hmm. Like, it, it, you know, this isn’t just sending packets across a, a thing, whatever, and hoping that people find something to do with it.
This is just like, oh, we apply intelligence into every domain of human activity. And then it works like incredibly well. Yeah. Um. Here’s what I know, here’s what I know. Um, in the next three or four year, it’s like somewhere between three or four years out, basically everything is selling out. So like the, the entire supply chain is, is, is, is sold out or, or, or selling out.
And so there, there’s no, like, we’re just gonna have like chronic supply shortage for, you know, for years to come. Um, there’s going to be a response from the market that’s gonna result in an enormous, you know, it’s happening now. An enormous flood of investment in a new fab capacity and ev you know, every, everything else to be able to do that, at some point the supply chain constraints will unlock, you know, at least to some degree that will be another accelerant to industry growth when that happens.
‘cause the products will get better and everything will get cheaper. Um, and so, so I know that’s gonna happen. I know that, you know, the deployments, you know, the, the actual use cases are like really compelling. And then, like I said, you know, with reasoning and agents and so forth, like, I know they’re just gonna get like much, much better from here.
And so I, I, I know the capabilities are like really real and serious. I also know that the technical progress is not going to stop. It. It, it is excel. It is, is accelerating. Like the, the breakthroughs are are tremendous. I mean, even just month over month, the breakthroughs are really dramatic. And so, you know, I think if you were a cynic and there, there are cynics, you can look at 2000, you can find echoes.
But I can’t even imagine betting it that this is gonna like somehow disappoint and, you know, at least for years to come, I think it would be essentially suicidal to make that bet. Yeah. Um, it was that Michael Burry, uh, uh, that’s
swyx: an
Marc: interesting guy, huh? We’ll pick on a guy. We’ll pick, let’s pick on one guy.
We’ll pick. Well ‘cause he did, he he came out with, it was, it was the, he
swyx: doesn’t mind.
Marc: It was the Nvidia short. Right. He came with the Nvidia short. And then if you guys probably talked about this, which is the, the analysis now that like the current models are getting better faster at such a rate that if you are running an Nvidia, if you’re running an Nvidia inference chip today, that’s three years old, you’re making more money on it today than you did three years ago because the pace of improvement of the software is, is faster than the, the, the depreciation cycle, the chip.
And then my understanding is Google is running. I don’t if they’ve, I don’t know exactly what, uh, these are rumors that I’ve heard or maybe it’s public, but, um, I think Google’s running very old TPUs, very profitably. Ference. Yeah. And very profit and very profitably. Yeah. Um, and so, so it actually turns out, as far as I can tell, it’s actually the opposite of the Beery thesis is actually.
He was actually 180 degrees wrong. It’s actually the, the, the, the old Nvidia chips are getting more valuable, which is something that’s like literally never happened before. Like it’s never been the case that you have an older model chip that becomes more valuable, not less valuable. And that, and again, that’s an expression of the just ferocious pace of software progress.
Ferocious pace of capability payoff. Yeah. Uh, that you’re getting on the other side of this. And so I just, the idea of betting against that, like.
swyx: Yeah. Yeah. Well, one of
Marc: my, it seems like an invitation to get your face ripped up.
swyx: One of my early hits was like modeling the lifespan of the H 100 and h two hundreds and, and going like, you know, usually they advise like four to seven years and it was, you know, maybe you sort of realistically haircut cut it down to two to three.
Yeah. But actually it’s going up and not down. Yeah. And, and uh, that’s, I mean that’s, I think that’s the dream. Uh, we are finding utilization and I think utilization solves all problems. Like, you can, you can find use, use cases for even like the poor, like even memory, we’re having a shortage. Right. And, and even like the, the shittier versions of, of memory that we do have, we are finding use cases for it.
So like That’s great.
Marc: Yeah.
Alessio: How, how important is open source AI and kinda like edge inference in a world in which you have three years of supply crunch. Like, do you think in the, like, you know, if you fast forward like five years, like how do you think about inference, uh, in the data center versus at the edge?
Marc: Well, so just to start, yeah. So I think, I think open source is very important for a bunch of reasons. I think edge, edge inference is very important for a bunch of reasons. I, I think just practically speaking, if we’re just gonna have fundamental construc, supply crunches for the next, I mean, you, you guys know if you just project forward demand over the next three years, right?
Yeah. Relative to supply, one of the, its main predictions you can do is what’s gonna, what, what’s gonna happen to the cost of, of inference in the core, uh, over the next three years? And like, it may rise dramatically, right? Like, so, so what is, and then is, is, you know, like the, the, the big model competition are subsidizing heavily right now.
Right? Right. And so, so what’s the, what will be the average person’s, you know, per day, per month token cost, you know, three years from now to do all the things that they want to do. And I, I don’t know, it’s gonna. I mean, I have, you guys probably have friends, I have friends today who are paying a thousand dollars a day for open claw, for claw tokens to run open claw.
Right? And so, okay. $30,000 a month. Right? And, and by the way, those, those friends have like a thousand more ideas of the things that they want their claw to do, right? Yeah. And so you, you could imagine there, there’s like latent demand of up to, I don’t know, five or $10,000 a day of, of, of tokens for a fully deployed, you know, per personal agent.
Uh, and obviously consumers can’t pay that, right? And so, so, but it gives you a sense of the fu of the fu of the future scope of demand, right? And so, so even, even if there’s a 10 x improvement in price performance, that still, you know, goes to a hundred dollars a day, which is still way beyond what people can pay.
Mm-hmm. So there’s just gonna be like. Ferocious to me, by the way. The agent thing, the other interesting thing is I think the agent thing, so up until now, a lot of the constraints of GGPU constraints, I think the agent thing now also translates into CPU constraints. Mm-hmm. Right?
swyx: CPU memory.
Marc: Yes. CPU memory, right?
And so, like the entire chip ecosystem is just gonna get wait,
swyx: wait for network constraints, that that will be the killer.
Marc: It’s all bottleneck potentially for years. And so, so I, I think that Brad, and, and I think it’s actually possible, I mean, generally inference costs are gonna keep coming down, but I think the, let’s put it this way, the rate of decline, I think may level out here for a bit because of these supply constraints.
And then at some point, maybe the lab stops subsidizing so much and that, that, that again, will be, be an issue. And so there’s just gonna be so much more demand for inference than, than can be satisfied. Um, you know, kind of with the centralized model. And then, and then, you know, you guys know this, but like all the, just the dramatic, I mean just the dramatic innovations that have happened in the Apple silicon to be able to do, uh, inferences, it’s quite amazing the level of effort being put.
Like the open source guys are putting incredible effort into getting, you know, this recurring pattern where the big model will never run on a pc, and then six months later mm-hmm. Oh, it runs in a pc, right? It’s like amazing. And there’s very smart people working on that. So there’s all that. And then look, there’s also, you know.
There’s also like other, there’s other motivators. There’s other motivators which is just like, okay, how much trust are the big centralized model providers? You know, how much trust are they building in the market versus, you know, how much are, you know, at least for, in certain cases with some people, for certain use cases, people being like, well, I’m not willing to just like, turn everything over.
So there, there, there’s all the trust issues. Um, by the way, there’s also just like straight up price optimization. There’s many uses of AI where you don’t need Einstein in the cloud. You just need like a, a a, a smart local model. There’s also performance issues where you want, you know, you want, you know, you’re gonna want your doorknob to have an AI model in it.
Right. You know, to be able to, you know, do, um, you know, to be able to do access control. Um, obviously like everything with a chip is gonna have an AI model in it. Mm-hmm. And it, a lot of those are gonna be local. Um, and so, yeah. No, like I think, I think you’re gonna have ti and then you’re gonna, by the way, also wearable devices, you know, you don’t wanna do a complete round trip.
You want, you know, you, whatever your smart devices are, you want it to be like super low latency. Yeah.
swyx: The question, do we care who makes it? Yeah. One of the biggest news this week was the collapse of AI two, the Allen Institute. Mm-hmm. One of the actual American open source model labs. Yeah. Um, and, uh, I’m not that optimistic on, on American open source.
Yeah. Like you, you guys invested in MIS trial and MIS trial’s doing extremely well outside of China. That’s about it.
Marc: Yeah. We’ll see. We’ll see. I look, I, number one, I do think we care. Uh, I do think we, I do think we care who makes it. Um, I would say this, the, the, the, the previous presidential administration wanted to kill it in the us Oh yeah.
They wanted to drown in the bathtub. Um, and so they wanted to kill it. So at least we have a government now that actually like, actually wants it wants it to happen. And you
swyx: earned to council
Marc: and Yeah. And the new and the P pcast. Yeah. So the, the, you know, this admin for whatever other political issues people have, which are many, you know, this administration has, I think a very enlightened view and in particular an enlightened view on AI and in particular on open source ai.
Uh, and so they’re very supportive. Um, my read is the Chi. The Chinese have a very, the various Chinese companies have a very specific reason to do open source, which is, they, they, they don’t fundamentally, they don’t think they can sell commercial, uh, AI outside of China right now. And or at least specifically not, not in the US for a combination of reasons.
And so they, they kind of view, I think, open source AI as a bit of a loss leader against basically domestic, uh, you know, paid, paid services. And then kind of an, you know, kind of an ancillary products. You know, they’re, they’re very excited about it, by the way. I think it’s great. I think it’s great that they’re doing it.
Um, you know, I think Deeps seek was like a gift to the world. Um, I think. The great thing about open source, open source, the, the, the impact of open source is felt two ways. One is you, you get the software for free, but the other is you get to learn how it works, right? And so like the paper, the paper, the paper and, and the code, right?
And the code. And so, like, for example, I thought this was amazing. So open comes out with L one and it’s an amazing technical breakthrough, and it’s just like, absolutely fantastic. But of course they don’t explain how it works in detail. And then of course they hide the, they hide the reasoning traces, right?
And, and then, and then, and then everybody’s like, okay, this is great, but like, who’s gonna be able to replicate this? Are other people gonna be able to do this? You know, is their secret sauce in there? And then our one comes out and it’s just like, there’s the code and there’s the paper, and now the whole world knows how to do it.
And then, you know, three months later, every other AI model is, is adding reasoning. And so, so you get this kind of double, like even if the Chinese models themselves are not the models that get used, the education that’s taken place to the rest of the world, the information diffusion, you know, is incredibly powerful.
So that happens and then, I don’t know. We’ll, we’ll see. You know, there are a bunch of American, you know, open source, you know, ai, uh, model companies. I mean, look, there’s gonna be tremendous, you know, there already is. There’s, you know, there’s gonna be tre there’s tremendous competition, uh, among the primary model companies.
You know, there’s, depending on how you count, there’s like four or five, you know, big co model companies now that are, you know, kind of neck and neck, uh, in different ways. Um, uh, you know, and, and, and, um, you know, and then obviously Bo Bo both X and then MetAware involved are, you know, both have huge, you know, huge attempts to, you know, kind of, to kind of leapfrog underway.
And then you’ve got, you know, a whole fleet of startups, new companies, including a whole bunch that we’re backing, that are, you know, trying to come out with different approaches. And then you’ve got whatever it is. I don’t know how, how many, how many, like main line foundation model companies are there in China at this point?
It’s probably six. It’s
swyx: five Tigers is what they call it. Yeah. Uh, Quinn is in questionable because there’s change in leadership,
Marc: right?
swyx: Yeah.
Marc: But that, does that include, that includes like Moonshot,
swyx: yes. Can deep seek, uh, uh, ZI, um, Quinn oh one is in there.
Marc: Right. And then, um, and by dance and, and then you see,
swyx: ance would be like the next tier ance.
They weren’t as prominent. They weren’t, didn’t have
Marc: a leading. Yeah. But they, you at least, you know, ance is very inspiring and presumably they have more stuff coming and Tencent probably has more stuff coming and, and so forth. And so, so, so like, look, here, here would be a thing you can anticipate, which is there are not these markets, there are not going to be between the US and China right now, there’s like a dozen primary foundation model companies that are like at scale, at, at some level of a critical mass.
It’s not gonna be a dozen in three years, right? Like, it just because these industries don’t bear a dozen, it’s, it’s gonna be three or you know, there’s gonna be three or four big winners or maybe one or two big winners. And so there’s gonna be like a whole bunch of those guys that are gonna have to figure out alternate strategies.
Um, and I think like open source is one of those strategies. And so I, I think you could see like a whole, i, I, I think the questions like, who’s gonna do open source? I think that could change really fast. I, I think that, that, that’s a very dynamic thing. I think it’s very hard to predict what happens. And, and I think it’s very important.
swyx: NVIDIA’s doing a lot.
Marc: Well, I was gonna say. Well, exactly. And then you’re got Nvidia and then, and then, you know, just to, again, indu, there’s an old thing in business strategy, which is called, uh, commoditize Compliments. Commoditize the compliment. That’s right. And so if your Jensen is just kind of obvious, of course, you wanna commoditize the software.
Yeah. And he’s, and to his enormous credit, he’s putting enormous resources behind that. And so maybe it, maybe it’s literally Nvidia and I think that would be great.
Alessio: Yeah. Uh, narrative violation to European projects, uh, in the, uh, damn.
swyx: I’m hosting my, uh, Europe, uh, conference soon. And I got both of them.
Alessio: They got us.
They got us. Mark
Marc: finished. They got us, us. Well, wait a minute. Where was Peter? So where was Steinberger when he did? In Austria
Alessio: was, yeah, yeah, yeah.
Marc: He was in what? He was in Vienna. Oh, he was in Vienna. And then where is he now?
swyx: Uh, he’s moving to sf.
Marc: Okay. Okay. Alright. Okay, there we go. And then, yeah, the PI guy, right?
The PI guys are European.
swyx: Yeah, they’re also, they’re buddies in
Alessio: Australia. Mario’s also there. Yeah.
Marc: Right. And are they, yeah, they haven’t announced yet. Any sort of change changed or have they
Alessio: No, they’re, they have a company there.
Marc: Okay. Got, okay. Good.
Alessio: Good, good,
good.
Alessio: Um,
Marc: yeah, good.
swyx: Anyways, I think pie and open cloud very important software things and, and I just wanted you to just go off on what you think.
Marc: Yeah. So I think in co the, the combination of the two of them I think is one of the 10 most important softwares. Open
swyx: Claw got all the attention, but Right. Talk about pie,
Marc: pi pie’s, kind of the Yeah. PI’s, PI’s kind of the architectural breakthrough for those of us who are older. There was this whole thing that was very important in the world of software basically from like 1970 to, I don’t know, it still is very important, but like 19, from 1973 to like basically the creation of Linux, which is basically this, this thing used to call like the Unix mindset.
Like so, so, ‘cause there were all these different, you know, theories. There are all these different operating systems and mainframes and, and then you know, all these windows and Mac and all these things. And then there was this, but kind of behind it all was this idea of kind of the Unix mindset. And the Unix mindset was this thing where basically you don’t have these, like, like in the old days, like, like the operating system that like made the computer industry really work, like in the 1960s mm-hmm.
Was this thing called o os 360, which was this big operating system that IBM developed that was supposed to basically run everything. And it was this like giant monolithic architecture in the sky. It was like a, you know, it was like a giant castle. Um, of software. And, and by the way, it worked really well and they were very successful with it.
But like, it was this huge castle in the sky, but it was this thing, it was almost unapproachable, which is like, you had to be kind of inside IBM or very close to IBM. And you had to really understand every aspect, how the system worked. And then the, the Unix sky is originally out of at and t and then out out of Berkeley, um, you know, came out and they said, no, let’s have a completely different architecture.
And the way architecture’s gonna work is we’re gonna have, we’re gonna have a, a prompt and, and a, and a shell. And then, and then we’re gonna, all, all the functionality is gonna be in the form of these discreet modules, and then you’re gonna be able to chain the modules together. Mm-hmm. Yeah. And so like the, the, the op, it’s almost like the operating, operating system itself is gonna be a programming language.
Um, and then that led led to the, the, the sort of centrality of the shell. Um, and then that led to sort of, uh, you know, basically chaining together Unix tools. And then that led to the emergence of these, these scripting languages like Pearl, where you, you could basically kind of very easily do this, and then the shells got more sophisticated and then, and then, and then look like, you know, that, that, that number one, that worked and that, that was the world I grew up in.
Like I was, I was a Unix guy. You know, sort of from, call it 1988 to, you know, kind of all, all the way through my work and it worked really well. It, it’s in the background, um, you know, nor normal people don’t need to, didn’t need to necessarily know about it, but like, if you were doing like system architecture, application development, you, you, you knew all about it.
Um, and then, you know, it’s been in the background ever since. And, you know, look, your Mac still has a Unix shell, you know, kind of in there, and your iPhone still has a Unix shell kind of buried in there somewhere. So they’re kind of in there. And then, you know, the Windows shell is kind of a, you know, sort of a weird derivative of that.
But, um, you know, but look, the inter, the internet runs on Unix, um, and that smartphones, actually, both iOS and Android are Unix derivatives. And so, you know, kind of Unix did end up winning. But, but anyway, and then we just started taking that for granted. And then, and then so, so basically the, the way I think about what happened with Pie and then with Open Claw is basically what those guys figured out is, I always say the, the great breakthroughs are obvious in retrospect, right?
Which is the best kind, the best kind. They weren’t obvious at the time or somebody else would’ve done them already. Um, and so there is a, like a real conceptual leap, but then you look at it sort of the backwards looking and you’re just like, oh, of course. Mm-hmm. Like the, the, to me those are always the best breakthroughs.
Well, actually language models themselves are like that. It’s just like, oh, next token completion. Oh, of course.
swyx: Yeah. What other objective mattered?
Marc: Yeah, exactly. But, but like it, right. But she’s even saying it wasn’t obvious until somebody actually did it. Right. And so the conceptual breakthrough is real and deep and powerful and, and very important.
And so the way I think about pie and olaw is it’s basically marrying the, the language model mindset to the un to the Unix, basically shell prompt mindset. And so it’s, it’s basically this idea that what, what, so what is an agent, right? And as, as, and as you know, like many smart people who have been trying to figure out what an agent is for, for, for decades, and they’ve had many architectures to build agents and the whole thing.
And it turns out what is an agent. So it turns out what we now know is an agent is the following. It’s, so it’s a language model. And then above that, it’s a ba, it’s a bash shell. Um, so it’s a, it’s a Unix shell, and then it’s, and then the agent has access, uh, has access to, to the shell. And, you know, hopeful, hopefully in a sandbox, maybe in, maybe in a sandbox.
So it’s, it’s the model. Um, it’s the shell. Um, and then it’s a fi, it’s a file system. Um, and then the state is stored in files. And then, you know, there’s the markdown format for the, you know, for, for the files themselves. And then, and then there’s basically what in Unix is called Aron job. There’s a loop and then there’s a heartbeat for the, there’s heartbeat and, and the thing basically Wake Wakes up.
Wakes up. So it’s basically LLM plus shell, plus file system, plus markdown, plus kron. And it turns out that’s an agent. And, and, and every part of that, other than the model is something that we already completely know and understand. And in fact, it turns out that like the latent power of the Unix shell is like extraordinary because basically like all, like, there’s just like an, there’s just enormous latent power in the shell.
There’s enormous numbers of Unix commands, there’s enormous number of command line interfaces into all kinds of things already in the, you know, your entire, I mean your entire, just to start with, your computer runs on a shell. If you’re running a Mac or a, or, or a phone, your computer, your computer’s running on a shell, uh, already.
And so like the full power of your computer is available at the command line level. Um, and then it turns out it’s really easy to expose other functions as a command line interface. And so like this whole idea where we need like MCP and these like product mm-hmm. Fancy protocols, whatever, it’s like, no, we don’t, we just need like a command, command line thing.
So that’s the architecture. And then it turns out what is your agent? Your agent has a bunch of files starting a file system. And then there’s the thing that just like completely blew my mind when I write my head around it as a result of this, which is like, okay. This means your agent is now actually independent of the model that it’s running on.
Because you can actually swap out a different LLM underneath your agent and your, your agent will change personality somewhat. ‘cause the model is different, but all of the state stored in the files will be retained.
swyx: Yeah. Different instruction set, but you just compiled
it.
Marc: Right, exactly. And it’s all right.
It’s like right. Swapping out a ship and recompiling, but it’s, it’s still, it’s still your agent with all of its memories. Um, and with all of its capabilities. And then by the way, you can also swap out the shell, uh, so you can move it to a different execution environment that is also, is also a b shell, by the way, you can also switch out the file system, right.
Uh, and you can, and you can, and you can swap out the, the, the heartbeat for the, the crown framework, the, the loop that the agent framework itself. And so your agent basically is ba basically at the end of the day, it’s just. It’s just, its files. Um, and then, and then there’s of course it a open
swyx: call.
Marc: Yeah, it’s, it’s basically, it’s, it’s just the files.
Um, and then by the way, as a consequence of that, the agent and then the agent itself, it turns out a couple important things. So one is it, it’s, it, it can migrate itself, right? And so you’re, you can instruct your agent, migrate yourself to a different, uh, runtime environment, migrate yourself to a different file system, migrate yourself to a different, you know, swap out the language model.
Your agent will do all that stuff for you. And then there’s the final thing, which is just amazing, which is the agent is the agent actually has full introspection. It actually, it actually knows about its own files and it could rewrite its own files. Right. Which by the way, is basically no widely deployed software system in history where the, the, the thing that you’re using actually has full introspective knowledge of how it itself works and is able to modify itself.
Like that, that, I mean, there have been toy systems that have had that, but there, there’s never been a widely deployed system that has that capability and then that leads you to the capability. That just like completely blew my mind when I wrap my head around it, which is you can tell the agent to add new functions and features to itself and it can do that.
Extend yourself. Yeah. Right? Extend, extend yourself. Like extend yourself. Give yourself a new capability. Right? And so, and so literally it’s just like you run into somebody at a party and they’re like, oh, I have my open claw, do whatever, connect to my eat, sleep bed, and it gives me better advice and sleep.
And you go home at night and you tell your claw, or if they’re at the party, by the way, you tell your claw, oh, add this capability to yourself. And your claw will say, oh, okay, no problem. And it’ll go out on the internet and it’ll figure out whatever it needs and then it’ll go out to claw code or whatever.
It’ll write whatever it needs. And then the next thing you know, it has this new capability. And so you don’t even have to, like, you can have it upgrade itself without even having to, without having to do anything other than tell it that you want it to do that. And so anyway, so the, the combination of all this is just, I mean, this is just like a massive, incredible, I mean, it’s just incredible.
Like if I, if I were, if I were 18, like this is a hundred, this is what I would be spending all of my time on. This is like such an incredible conceptual breakthrough. Yeah. And again, pe people are gonna look at it and they already get this response. People are gonna look at it and they’re gonna say, oh, well, where’s the breakthrough?
‘cause these, the, all of these components were already known before. Mm-hmm. But, but this is the key, the key to the breakthrough was by using all these components that were known before, you get all of the underlying capability of that’s buried in there. And so all, and so for example, computer use all of a sudden just kind of falls, trivi, trivial.
Of course it’s gonna be able to use your computer. It has full access to the shell. Right. And then, and then you just, you, you give it access to a browser, and then you’ve got the computer and the browser and, and often away it goes. And, and then you’ve got all the abilities of the browser also. Um, yeah.
And so, and so the capability unlock here is profound. My friends who are, you know, deepest into this, are having their claw do like a, like, literally like a thousand things in their lives. They have new ideas every day. They’re just like constantly throwing new challenges at the thing. And by the way, it’s early and, you know, these are, you know, these are prototypes and there are, you know, as you guys know, there’s security issues.
Yeah. And, and so, you know, there’s a bunch of stuff to be ironed out, but the, the unlock of capability is just incredible.
swyx: Yeah.
Marc: And I, I have absolutely no doubt that everybody in the world is gonna, is gonna have at least, you know, an agent like this, if not an entire family of agents. And we’re gonna be living in a world where I think it’s almost inevitable now that this is the way people are gonna use computers.
swyx: I was gonna say for someone who is deeply familiar with social networks, the next step is your claw talking to my Claw. Mm-hmm.
Marc: Posting
swyx: on Claw Facebook, uh, posting their jobs on cloud LinkedIn and close posting their tweets on claw XAI or what, whatever, you know. Um, I do think that that is how, uh, you know, we, we get into some danger there in, in terms of like alignment and whether or not we want these things to, to, to run.
Marc: You guys know where Rent a, rent a human.com.
swyx: Yeah. Rent a,
Marc: yeah. Yeah.
swyx: I mean, it’s Fiverr, it’s TaskRabbit.
Marc: Sure, of course.
swyx: Mechanical
Alessio: Turk.
Marc: Yeah. But flipped, right. The agent hiring the people.
Alessio: Yeah.
Marc: Which of course is gonna happen, right? It’s obviously gonna happen.
Alessio: I’m curious if you have any thoughts on the engineering side.
So when you build the browser, the internet, you know, just a bunch of mostly plain text file plus some images, and today the, every website and app is like, so complex. Somehow, you know, the browser kept evolving to fit that in. Mm-hmm. Are there any design choices that were made like early in the browser and kinda like the internet and the protocols that you’re seeing agents similar to this?
Like, Hey, this thing is just not gonna work for like this type of new compute and we should just. Rip it out right now.
Marc: There were a whole bunch, but I’ll give you a couple. So one is, um, and we didn’t, you know, to be clear like this, this was not, you know, this is totally different. We didn’t have the capabilities we have today, but because Wet have, we didn’t have the language models underneath this, but, um, we did have this idea that human readability actually mattered a great deal.
Um, and, and, and so, and specifically in those days, it was, it was not so much English language, but it was there, there was a design decision to be made between binary protocols and text protocols. And basically every, every, every basically old school systems architect that had grown up between like the 1960s and the 1990s basically said, you know, the internet, it’s, what do you know about the internet?
It’s star for bandwidth. You, you just, you have these very narrow straws. Uh, you know, look, people, when we did the work on Mosaic, like pe, people who had the internet at home had a 14 kilobit modem, right? So you’re, you’re trying to like hyper optimize every bit of data mm-hmm. That, that travels over the network.
And so obviously if you’re gonna design a protocol like HGTP, you’re gonna want it to be binary, you know, highly compressed, binary protocol for maximum efficiency. And you’re gonna wanna have it be like a single connection that persists. And you’re, you’re, the last thing you’re gonna wanna do is like, bring up and tear down new connections.
And you definitely, you’re not gonna, not gonna want a text protocol. And so of course we said no. We actually want to go completely the other direction. It’s obviously, we only want text protocols. Uh, by the way, same thing in H TM L itself. We want html to be relatively verbose. You know, we want the tags to actually be like human readable.
Um, we wanna use
swyx: the most inefficient things possible.
Marc: Yeah, we wanna do the, we wanna do the in, we wanna do the inefficient things.
swyx: You’re the original token Mixer.
Marc: Yeah, exactly. Yeah, yeah, yeah. Basically it’s just like better lesson
Alessio: filled.
Marc: Well, yeah. Well actually this was, this was actually the, the conscious thing, which basically says just like assume, assume a future of infinite, infinite bandwidth built for that, right?
And then basically what it was, is it was a bet that it, it was a bet that if the system, if the, if the latent capabilities of the system were powerful enough, and that was obvious enough to people that would create the demand for the bandwidth that would cause the supply of bandwidth to get built that would actually make the whole thing work.
And then specifically what we wanted was we wanted everything to be human readable because we, at the engineering level, we wanted people to be able to read the protocol coming over the wire and be able to understand it with their, with their bare eyes without having to like disassemble it or whatever.
Right. Have it converted outta binary. Right. And so the, the, the, all the pro, you know, HTTP and everything else were, were, it was always, uh, text protocols. Uh, and the same thing with HTML and in, in many ways, some people say that the key breakthrough in the browser was the view source option, um, which is every webpage you go to, you could view source, which means you could see how it worked, which means you could teach yourself how to build right new, uh, to, to build new webpages.
There was that. So human readability. Um, and, and again, human readability in those days still meant technical, you know, specs. You know, now it means English language, but there’s an incredible latent power in giving everybody who uses the system the option to be able to drop down and actually understand and see how it’s working.
And that worked really well for the web and I think it’s working really well for ai. That was one. Um, what was the other, um. A big part of the idea of web servers was to actually surface the underlying latent capability of the operating system and to be able to surface the, uh, also the underlying latent capability of the database because basically what was a web server?
What, what, what, what is a web server? Fundamentally? Architecturally, it’s, it’s, it’s the operating system. So it’s, it’s the operating system’s ability to, you know, it’s running on top of an os. So it’s the OSS ability to manage. The file system and do everything else that you wanna do, process everything. Um, and then of course, a lot of early, you know, a lot, a lot of websites are, are front ends to databases.
Um, and so you wanted to, you wanted to unleash the underlying latent power of whether it was an Oracle database or some other, you know, some other Postgres or whatever, whatever it was. Um, and so a lot of the function of the web server was to just bridge from that internet connection coming in to be able to unlock the underlying power of the OS and the database.
Uh, and again, people looked at it at the time and they were like, well, is this really, does this really matter? Like, is this important Because we’ve had databases forever and we’ve always had, you know, user interfaces for databases and this is just another user interface for a database. And it’s like, okay, yeah, fair enough.
But on the other side of that is just like, this is now a much better interface to databases and one that 8 billion people are going to use and is going to be like, far easier to use and far more flexible. And, and, and, and you’re not just gonna have old databases. Now you have a system where people can actually understand why they want to build, you know, a million times more database apps than they have in the past.
And then the number of databases in the world exploded. And so again, this goes to this thing of like building, building in layers. Some of the smartest people in the industry look at any new challenge and they’re like, okay, I’m, I’m, I need to build a new kind of application. So the first thing I need to do is build a new programming language, right?
And then the next thing I need to do is build a new operating system, right? And then the next thing I need to do is I need to build a new chip. Right? And they, they kind of wanna reinvent everything. And I’ve, I’ve always had, maybe it’s just, I don’t know, pg pragmatic mentality or something, or maybe an engineering over science mentality, but it’s more like, no, you have just like all of this latent power, uh, in the existing systems and you, you don’t want to be held back by their constraints, but what you wanna do is you wanna kinda liberate that power and open it up.
Yeah. And so I, I think, I think, and I think the web did that for those reasons. And I think it’s the same thing now that’s happening. It’s a great
swyx: perspective on the web.
Alessio: Programming language just is not a good thing. We have Brett Taylor on the podcasts and we were talking about rust. And you know, rust is memory safe by the phone.
So why are we teaching the model to not write memory, unsafe code, just use rust, and then you get it for free. How much do you think there’s like. Time to be spent like recreating some of these things instead of taking them for granted. I’ll be like, oh, okay. Python is kind of slow Python
swyx: type scripts,
Alessio: you know?
It’s like, yeah.
swyx: As, as imperfect as they are, they are the lingua franca.
Marc: I mean, I think this is gonna change a lot. ‘cause I don’t think the models care what language they program in. Mm-hmm. And I think they’re gonna be good at programming in every language, and I think they’re gonna be good at translating from any language to any other language.
Like, okay, so this gets into the coding side of things. I, I think we’re going through a really fundamental change. And then, look, I, I grew up hand, you know, I grew up hand code, you know? Yeah, yeah, yeah. I grew up hand coding. Everything I did was actually everything I did actually was written in CI wasn’t even
Alessio: back in the days,
Marc: I wasn’t even using c plus plus, so I, or like Java or any of this stuff.
Right. Uh, and so, um, I, everything, everything I ever did, I was like managing my own memory at, at, at the level of c and then I, you know, I, I’m still from the generation that, you know, I, I knew assembly language and, you know, I, I, you know, um, so I, I could drop down and do things, uh, right on the ship. And so we, we’ve just, we’ve all, all of us, we’ve always lived in a world in which software is like this precious thing that like, you have to think about very carefully.
And it’s like really hard to generate good software. And there’s only a small number of people who can do it. And like, you have to be very, like, jealous in terms of thinking about like, how do you allocate, like what are your engineers working on and how many good engineers do you actually have? And how much software can they write?
And how can, how much software can human beings, you know, kind of maintain? And I think like all those assumptions are being shot right out the window right now. Like, I think they’re, I, I think those days are just over. And I think the new world is like, actually high quality software is just like infinitely available.
Mm-hmm.
Marc: And if you need new software to do X, Y, Z, like, you’re just gonna wave your hand and you’re gonna get it. And then if it’s, if you don’t like the languages written in, you just tell the thing, all right, I want the, now I want the rush version. Um, or, you know, se secure, you know, secure. We’re about to, by the way, we’re about to go through computer security is about to go through the most dramatic change ever, which is number one, like every single latent security bug is about to be exposed,
swyx: right?
Marc: So we’re gonna have like, the in, we’re, we’re, we’re set up here for like the computer security apocalypse for a while. But, but, but on the other side of it, now we have a coding agents that can go in and actually fix all the security bugs. And so how, how are you gonna secure a software in the future?
You’re gonna tell the, tell the bot to secure it, and it’s gonna go through and, and fix it all. And so, so this thing that was this incredibly scarce resource of high quality software is just going to become a completely fungible thing that you’re just gonna have as much as you want, right? Uh, and, and that has like, you know, that has like tons and tons of consequences in some sense.
The answer to the question that you posed, I, I think it’s just somewhat, I don’t know, simple or something, or straightforward, which is just, if you want all your software and rust, you just, all the bot, you want all your software and rust, like, things that used to be like hard or even like, seem like an insurmountable mountain to get to get through all of a sudden, I think, become very easy.
swyx: I, I think Brett had a theory that there would be a more optimal language for lms. And so the contention is, uh, there isn’t like, just don’t bother, just whatever humans already use LMS are perfectly capable, porting.
Marc: I think we’re pretty close to being, I don’t know if this would work today. I think we’re pretty close to being able to ask the AI what would its opt optimal language be and let Right, and let it design it.
True. Okay, here’s a question. Are you gonna even gonna have programming languages in the future? Um, or the ai, are the AI just gonna be emitting binaries? Let’s assume for a moment that humans aren’t coding anymore. Let’s assume it’s all bots. The bot. What levels of intermediate abstraction do the bots even need?
swyx: Yeah.
Marc: Or are they just coding binary directly? Did you see there’s actually an experi, somebody just did this thing where they have a, they have a, a language model now that actually emits model weights for a new language model. Right. And so will the bots be just
Alessio: predict the weights
Marc: Will, yeah. Will the bots literally be emitting not just coding binaries, but will they, will, will they actually be admitting weights for, for new models?
Yeah. Direct directly and. Conceptually, there’s no reason why they can’t do both of those things. Uh, like architecturally. Both of those things seem completely possible. It’s
swyx: very inefficient. You’re basically very
Marc: inefficient.
swyx: A simulation of a simulation in a simulation inside of the weights. Correct?
Marc: Yeah, yeah. Very inefficient. But like, look, LMS are already like incredibly inefficient. Ask an uh, in favor thing, ask Claude, add two plus two equals four. Right? It’s just like, you know, it’s like, you know, it’s, it’s, it’s like whatever, billions and billions of times more inefficient than using your pocket calculator.
swyx: Yeah.
Marc: But, but, but yet the, the, the payoff is so great of the general capability. And so anyway, like I, I kind of think in 10 years, like, I’m not sure. Yeah. Like, I’m not sure there will even be a salient concept of a programming language, um, in the way that we understand it today. And in fact, what we may be doing more and more is a form of interpretability, which is we’re trying to understand why the bots have decided to, uh, structure, uh, code in the way that they have.
swyx: I mean, if you play it through, you don’t need browsers, then like, that’s the depth of the browser.
Marc: Well, so I, I would take it a step further, which is you may not need to use your interfaces. So who is gonna use software in the future?
swyx: Other bots.
Marc: Other bots. Yeah. Yeah. And
swyx: so you still need to, I don’t know, pipe information in,
Marc: do we?
swyx: And out
Marc: really
swyx: well, what are you gonna do then?
Marc: Are you sure
swyx: you’re just gonna log off and touch grass?
Marc: Whatever you want. Exactly. Isn’t that better?
swyx: I want software to do stuff for me.
Marc: Isn’t that? But isn’t that better? I mean, look, I, you know, I don’t know. Look like, you know, you know, you, all the arguments here, you know, it was not that long ago that 99% of humanity was behind a plow.
swyx: Right.
Marc: Right. And what are people gonna do if they’re not plowing fields all day to, to, to grow food? Right. And it just turns out there’s like much better ways for people to spend time than plowing fields. Yeah.
swyx: Dooms growing.
Marc: Uh, yeah, exactly. Exactly. Or, you know, talking to their friends and look, and I’m not an absolutist and I’m not a utopian.
And I, and to be clear, like I’ve, I have an 11-year-old and he’s learning how to code and like I’m, you know, I, I think it’s still a really good idea to learn how to code and so forth, but I just, if you project forward, you just have to think forward to a world in which it’s just like, okay, I’m just gonna tell the thing what I need and it’s gonna do it, and then, and then it’s gonna do it in whatever way is most optimal for it to do it.
Mm-hmm. Yeah. Unless I tell it to do it non optimally. Like if I tell it to do it in Java or in Rust or whatever, it’ll do it, I’m sure. But like, if I’m just gonna tell it to do, it’s, gonna do it in whatever way is like the optimal way to do it. Yeah. And then I, and then if I need to understand how it works, I’m gonna ask it to explain to me how it works.
Right. And so it’s gonna be doing its own, interpret it, it’s gonna be the engine of interpretability to explain itself. And I, I just am not convinced that, that I’m not, I’m not convinced that in that world you have these historical, the goals of the abstractions will be whatever, the Boston network with the human Right.
Alessio: Yeah. Yeah. That, well, I, I’m curious like. If that’s true, then shouldn’t the models providers be building some internal language representation that they can do extreme, kinda like rl uh, and reward modeling around, because it’s like, today they’re kind of like tied to like type script and Python because the users need to write in that language versus they can have their own thing internally and like they don’t need to teach it to anybody.
They just need to teach their model. And I think that’s how you get maybe the version between the models, like going back to like the pie open claw thing. It’s like, oh, I built all the software using the open AI model and now switch to the RO model. But the TRO model doesn’t understand the thing. So I I, it feels like there still needs to be some obstruction.
But maybe not. Maybe that’s the lockin that the model providers want to have. I don’t,
Marc: I’m not even sure that’s lockin though. ‘cause why can’t the second model just learn what the first model has done? Like,
swyx: exactly.
Marc: Okay. So okay. Give you an example. So as you know, models can now reverse engineer software by, right?
Isn’t it the whole thing now where people are reverse engineering, like Nten, Nintendo, gay binaries. Yeah. So you, you have like there’s, I’ve seen a bunch of reports like this where somebody has like a favorite game from the 1980s and the source code is like long dead, but they have like a binary brand to do a chip or something, another reverse engineer to get a version that runs in their Mac.
Right. And so if you reverse it, if, this is why I kinda say if you’re reversing like X 86 binaries, then why can’t you reverse engineer
Alessio: whatever the degree. Yeah. And because we’re all on a Unix based system, it has to be reversible because it needs to run on the target.
Marc: Yeah, yeah, yeah, yeah, yeah. Basically.
And so I just, I just think it’s this thing where it’s just like, and by the way, and everything we’re describing is something that human beings in theory could have done before, but just with like, right. Yeah, yeah. But with enormous where, but it was just always like cost and labor prohibitive. Reverse engineer.
I learned how to reverse engineer. Human beings can reverse engineer binaries. Yeah. It’s just for any complex binary, you need like a thousand years mm-hmm. To do it. But now with a model, you don’t. And so all of a sudden you get, you get these things. Or, or another way to think about it is so much of human built systems are to compensate for the human limitations.
swyx: Mm-hmm.
Marc: Yep. Right? Um, and if you don’t have the human limitations anymore, then all of a sudden you have, and, and it’s not that you, you won’t have abstractions, but you’ll have a different kind of abstraction. Yep. Yep.
swyx: I have two topics to bring us to a close. And, uh, you could pick whichever ones. Uh, just talking about protocols, was it you or someone else?
Uh, I forget my internet history. Who said that? Like the biggest mistake that we didn’t figure out in the early days was payments. Yes. Was that you?
Marc: Yes. It
swyx: was a 4
Marc: 0 2
swyx: 0 2 4
Marc: 0 2 payment required.
swyx: We have a chance now. Nope. I don’t think we’re gonna figure it out. I don’t know. Like, what’s your take?
Marc: Oh, I think, we’ll, yeah, no, now I think it’s gonna happen for sure.
swyx: Yeah.
Marc: Yeah. And there’s two reasons to example for sure. One is we actually have internet native money now in the form of crypto. Stable coins. Stable coins and crypto. And this is, I, I think this is the grand unification basically of ai, crypto, uh, is what’s about to happen now. Um, I think AI is the crypto killer app, I think is where, where this is really gonna come out.
Um, and then the other is it’s just, it, I mean it’s just, I think it’s now obvious. It’s like obviously AI agents are gonna need money and it’s already happening, right? If you’ve got a c if you’ve got a claw and you wanted to buy things for you, you have to give it money in some form.
swyx: I would say the adoption’s probably like 0.1% if, if that, but Yeah.
Marc: Oh, today? Yeah. Yeah, yeah. But think, think forward, like where is it going
swyx: forward thinking
Marc: The ultimate principle of everything and, and everything that I think I, we, we do is, it’s the William Gibson quote, which is, the future is already here. It just isn’t distributed. Mm-hmm. It isn’t, isn’t distributed yet.
My friends who are the most aggressive use users of, of, of, of open claw, just like have given their clause bank accounts and credit cards. Um, and, and, and, and, and not only have they done it. Obvious that they needed to do it because it’s obvious that they needed to be able to spend money on their behalf.
swyx: Yeah. Yeah.
Marc: It’s just completely obvious. And so, and again, like, so the number of people who have done that today to your point is like, I don’t know, probably 5,000 or something. Yeah. But
swyx: it’ll grow.
Marc: That’s how these things start
swyx: actually, I mean, since, uh, you keep mentioning,
Marc: and by the way, open cloud, by the way, if you don’t give it a bank account, it’s just gonna break into your, your, it’s gonna break high agency, it’s gonna break into your bank account anyway, and, and take your money.
So you, you might, as you might as well do it, you might as well do it,
swyx: uh,
Marc: by the way. I really love, I gotta tell you, I really love the phenomenon. I love the Yolo. Um, I’m not doing it myself to be clear, but, but I love the people that are just like, yeah, what, what is it? Skip, skip, vision,
swyx: danger, skip.
Marc: Dangerous.
swyx: Which by the way, is a Facebook thing.
Marc: Okay?
swyx: Right. Because, uh, because we, uh, in Facebook, they, they have this culture to name the thing dangerous, so that you are aware when you enable the flag that you are opting into a dangerous thing.
Marc: Okay, good.
swyx: And they brought it into open ai and of course that
Marc: makes it enticing.
swyx: Sam runs Codex, uh, with skip permissions on, on his laptop.
Marc: Yes, a hundred percent. And so I, I th I think the way to actually see the future is to find the people who are doing that. There’s a man, you know, and they, you knows,
swyx: log everything, you know, just watch it, watch the logs,
Marc: but. Let’s actually find out what the thing can do.
Yeah. And the way to find out what the thing can do is just like, try everything. Yeah. Let it try everything. Let it unlock everything. By the way, that’s how you’re gonna find all the good stuff it can do. By the way. That’s also how you’re gonna find all the flaws. Yeah. I think the people who turn that on for bots are like, they’re, they’re like martyrs to the progress of human civilization.
Like, I feel very bad for their descendants that their bank accounts are gonna get looted by their bots in the first like 20 minutes. But I think the contribution that they’re making to the future of our species is amazing.
swyx: It’s like gentleman science, you know?
Marc: Yes. It’s, yes, yes. Experi yourself. It’s, uh, Ben Franklin out with the, trying to try, trying to get lightning to strike his, his, uh, his balloon and see, seeing if he gets electrocuted.
swyx: Yeah.
Marc: It’s, uh, Jonas sk with the polio vaccine, right. Injecting it. Yes. So, yes. I, I, I, I think we should have, like agl, we should have like flags and like we should have like monuments to the people that just let open club run their lives.
swyx: More anecdotes of like, what, what are the craziest or interesting things that people listening to this should go, go home and do.
Marc: I mean, this is, this is the, this is the, the extreme thing is just like the straight Yolo, like just Yeah. Turn, turn your life
swyx: on. I mean, that’s a general capability. Yeah. Yeah. Is there like a specific story that was like, wow. And, and everyone in a group chat just lit up.
Marc: I mean, like, you know, so there’s tons of, there’s already tons of health, you know, there’s the health dashboard stuff is just, is just absolute personal health.
Absolutely amazing. Yeah. The number of stories on, um, I just don’t wanna violate people’s, you know, obviously personal. Yeah. Anonymized. But, um, you know, one of the things open clouds are really good at is hacking into all this stuff in your land. Uh, it’s really good. So, you know, internet of things. AKA internet of shit.
swyx: Yeah.
Marc: Like
swyx: super insecure, but great. It’s discoverable.
Marc: Yeah, it’s discoverable. O open claw is happy to scan your network, identify all the things. And then my, my, my friends who are most aggressive at this are having open claw take over everything in their house.
swyx: Yeah.
Marc: Take it takes over their security cameras.
It takes over their, their, you know, their whatever their, their access control systems. It takes over their webcams. I have a friend whose claw watches him sleep. Put a webcam in your bedroom. Put the, put the claw, put the claw on a loop. Uh, I have it. Wake up frequently and have it watch, just tell it, watch me sleep.
And, and I’ve, I’ve seen the transcripts and it’s literally like Joseph asleep. This is good. This is good that Joe’s asleep. ‘cause you know, I have, I have his health day and I know that he hasn’t been getting enough sleep and so it’s really good that he’s getting sleep. I really hope he gets his full, whatever, you know, five hours of REM sleep.
Uh, Joe’s moving. Joe’s moving. Um, uh, Joe might be wake waking up. This is a real pro. If Joe wakes up now, he is gonna ruin his sleep cycle. Oh, okay. It’s okay. Joe just rolled over. Okay. He’s gone back to bed. Okay, good. Alright. Okay. I can relax. This is fine. He’s
swyx: monitoring the situation
Marc: monitoring, monitoring the situation, and, and being a bot, like, you know, is just like very focused, right?
It’s just like, uh, this is like, its reason for existence is to watch Joe sleep. And then, and then I was talking to my friend who did this is like, you know, on the one hand it’s like, all right, this is weird and creepy. Um, and I need to, I need to, maybe this has taken over my life. And then the other thing is like, you know what if I had a heart attack in the middle of the night, this thing literally would like freak out and call 9 1 1.
Like, there’s no question. This thing would figure out how to like, alert medical authorities and like, prob probably some in SWAT teams and like, do whatever would be required to save my life. Right? And so it’s like, you know, like, yeah. Like that’s happening. What else? Um, I’ll give, I, um, uh, it’s a company unitary, uh mm-hmm.
That makes the robot dogs. Um, and I, I actually have one at home, which is, it’s actually really fun. The Chinese companies, the Chinese companies are so aggressive at adopting, uh, new technology, but they don’t always like, listen, take the time to really.
swyx: Package it,
Marc: package it, and maybe think it all the way through.
And so, so the, at least the industry dog I have, so it, it has a old non LLM just control system, which by the way is not very good in, in markets. Well, but it, in practices, it’s not that good. It has trouble with stairs and so forth. And so it’s not quite what it should be. But then the language model thing comes out in the voice.
So they, they add, so they add LLM capability and then they, they add a voice mode to it. Um, but, but that LLM capability is not at all connected to the control system. So, so you’ve got this schizophrenic dog that like, is a complete idiot when it comes to climbing the stairs, but it will happily teach you quantum mechanics.
Right. In like a lum English accent. Right. Like, it, it, it is just like absolutely amazing. Jagged intelligence. Yeah. Yeah. Talk about jagged and then, now obviously what’s gonna happen in the future is, is they’re gonna connect together, but they’ll do it. But right now it’s, it’s, and so right now it’s not that useful.
And so I, I have a friend who has one of these who had his claw basically hack in and rewrite the code Rew write new firmware. Yeah. Write new firmware for the, for the unit robot. Ooh. And now it’s, now it’s an actual pet dog for his kids.
swyx: You could do that before or after like. The motion.
Marc: Yeah. It’s, he said it’s completely different.
He said it’s a complete transformation. Yeah. And whenever there’s an issue in the thing, now the claw just like reiterates the code. You know, you know, you goes in, it does, does the code and so is it kind of goes to your thing here. So, so like all of a sudden, uh, this is why the way we wanna think about AI code AI coding is not just like writing new apps.
It’s also going in and rewriting all the old stuff that should have worked that never worked. And so, like, I, I think, I think basically, I think the internet, the internet of shit is basically over. Like, I, I think everything, there’s a potential here where like all these devices in your house that have been like basically marginal or you know, basically dumb, you know, like all of a sudden they might all get really smart.
Now you have smart
swyx: home.
Marc: You have to decide if, yes, there are horror movies in which this is just, of which this is the premise. And so you have to decide if you want this. Yeah. But, but, but this is the first time I can say with confidence, I now know how you could actually have a smart home. Yeah. Yeah.
With 30 different kinds of things with chips and internet access, where it actually all makes sense and all works together and it’s all coherent in the, in the whole thing. And to have that unlock without a human being having to go do any of that work, like, you know.
swyx: You know, I, I’m, I’m waiting for a, sorry, mark.
Uh, I can’t let you open that fridge door, you know, like
Marc: Exactly, exactly. Yes, yes.
swyx: Because Oh, yeah, yeah. You’re not supposed to eat right
Marc: now. I have all of, yes, I have every shred of health information, you know, and I know you think you’re doing, you know, da da da. I didn’t think you do this, but you know, this is a real, are you really, you know, are you really sure?
And you know, you told, you know, you told me last night, you really don’t want me to let you do this, so, you know, I’m sorry, but the fridge door is locked. Um, yes. Open
swyx: the fridge doors.
Marc: Exactly. And by the way, I know you’re supposed to be studying for a test, so why don’t we, why don’t you go when you can pass the test, um, I will open the fridge door for you.
Yeah.
swyx: Final protocol and then, and then we can wrap up, uh, proof of human
Marc: Yes.
swyx: Uh, right.
Marc: Yeah.
swyx: That’s the last piece that we gotta figure out.
Marc: Yeah. So I would say there’s, there’s two massive, I would say, um, uh, sort of asymmetries in the world right now where we’ve known these asymmetries exist and we, we societally have an unwilling to grapple with them.
And I think they’re both tipping right now. And, and they’re, they’re, they’re, they’re the same thing. It’s virtual world version. It’s a physical world version. So the virtual world version is, is the bot problem. We’re just like, you know, the internet, internet is just like a wash and bots, internet’s a wash and fake people.
It has been forever. Um, by the way, a lot of that has to do with lack of money, you know? And so this, you know, this is the Yeah, this is this.
swyx: My spicy take was these two are the same thing. And corporations of people too, you know? So interesting.
Marc: Yeah, yeah, yeah.
swyx: Okay. So a bank account is proof of human.
Marc: Yeah.
Okay. Yeah. Until you, until you give the bots bank accounts. Yeah, exactly. So, okay. Yeah. So there’s that. But yeah, look, look, the bot, I mean, every social media user knows this. The bot, the bot problem is a big problem. You know, the bot, the bot problem has been a big problem forever. It’s, it’s a huge problem.
And it’s never really been confronted directly, like at any point, by the way. The physical world version of this is the drone, the drone problem. Um, right. And so we, we’ve known for, you know, we’ve known for 20 years now that the asymmetric threat both in Milit military and actual military conflict, but also in just like security, like, like, you know, security on the home front.
The big threat is, is the cheap attack drone. Right? The, the, the cheap, the cheap suicide, you know, drone with the bomb. And we’ve known that forever. And by the way, like, you know, it’s very disconcerting how like every, you know, every office complex in the, in the co you know, in the world is like unprotected from drone attacks.
Um, every, every stadium, every school, every prison. Like, like, sure e okay, we’ve known that, we’ve never done anything about what you gonna do
swyx: about it. Yeah.
Marc: One possibility is just leave, leave them unprotected forever and live in a world of like, asymmetric terrorism forever. Or the other is take the problem seriously and figure out the set of techniques and technologies required to, to be able to deal with that.
Whether those are lasers or jammers or early warning systems, or, you know, all
swyx: personal force fields,
Marc: kinetic, personal for dune, uh, personal, personal force fields. Exactly. And in both cases, the, these are, these are economic asymmetries. These are economic asymmetries, right? ‘cause it’s really cheap to field a bot, but it’s very hard to tell something, a bot.
It’s very cheap to field a drone. It’s very hard. It’s very expensive to defend against a drone. But you see what I’m saying is it’s, it’s, it’s the, it’s the virtual version of the problem, and it’s the physical version of the problem. Uh, the virtual version of the problem. What we, what we need quite literally is proof of human.
The reason is because you’re, you’re, you’re not gonna have proof of bot. The, the, the, especially now the, the bots are too good. The, the, the bots can pass the Turing test. And if the bots can pass the Turing test, then you can’t, you can’t screen for bot. You can’t have proof of not a bot. But what you can have is you can have proof of human, you can have, you know, cryptographically validated, this is definitely a person, and this is, and then you can have cryptographically validated.
This is definitely like something that a person said, yeah, this video is real. Right. Um,
swyx: just to double click on, on, uh, do you think Alex Lanya with world? Yeah. Do you think he’s got it or is there an alternative?
Marc: Oh, so I mean, there’s gonna be, I think there’ll be, I think many people will try, we’re one of the key, you know, participants in, in, in the World, in the World Project.
I dunno that, yeah. So we’re, we’re partisans, but yeah, I, I think so we think world is exactly correct. Okay. And, and the reason is it, it has, it has to be, it, it has to be proof of human. It it has, because you can’t do proof of not bought. You have to do proof of human to do proof of human. You, you need, you need biological validation.
You, you needed to start with this was actually a person, right? Because otherwise your bot signing up as fake people. Right? So you, you have to have like something, you have to have a bi. Biometric. And then you have to have cryptographic validation. And then the ability to do, to do, to do the lookup. And then, by the way, the other thing you need, which that you, you also need selective disclosure.
Um, so you need to be able to do proof of human without reviewing privacy, all the underlying information. Privacy. Yeah. By the way, another thing you’re need, you’re gonna need proof of age, right? ‘cause there’s all these laws in all these different countries now around you need to be 13 or 16 or 18 or whatever to do different things.
And so you’re gonna, you’re gonna need a, you know, sort of validated proof of age, um, you know, to be able to legally operate, right? And so that, that’s coming. And then you’re gonna want like, proof of credit score and, you know, proof of like, you know, a hundred other things.
swyx: That’s a tricky one.
Marc: It is a tricky one, but you’re gonna, you’re gonna, there, there’s no reason, like if somebody’s checking on your credit, somebody shouldn’t, I’ll give you an example.
Somebody shouldn’t need to know your name in order to be able to find out whether you’re credit worthy.
swyx: Right? I see. Independently verifiable pieces of information.
Marc: Pieces of information, yeah. It’s like selectively disclosed. And this is the answer to the privacy problem wr large, which is, I, I only need to prove, I need to prove at that moment.
So like, you’re gonna need that. And I, I think their, their, their architecture makes sense. So that needs to get solved. I think language models have tipped, the bots are now too good. Uh, and, and, and so they’re undetectable. And so as a consequence, you, we now need to go confront that problem directly. And then, and like I said, and then the other problem is we, we need to go actually confront the drone problems.
The Ukraine conflict has really unlocked a lot of thinking on that. And now the, um, and now the, the, the, the, the Iran situation is also unlocking that. And so I think there’s gonna be just like this incredible explosion of, of both drone and counter drones.
swyx: Our drones are better than their drones to keep it that way.
Marc: Yeah. Yeah. And counter drones,
Alessio: I think we can sneak in one more question. Go for it. Um, I’m trying to tie together a lot of things that you said over the years. So at the Milken Institute debate with Teal, which is amazing. Um, you talked about the lag between a new technology and kinda like the GDP, um, impact of it.
Marc: Yep.
Alessio: The other idea you talked about is bourgeois capitalism and how, you know, this kind of managerial class was needed because of this complexity. And I think if you bring AI into the fold, you have like much higher leverage of people. So like if you have, you know, the Musk industries, um, and you give Elon a gi, you can run a lot more things That’s right.
At once.
Marc: That’s right.
Alessio: And then you have the social contract. And I know you reviewed a clip of Sam ing, um, we’re rethinking the whole thing, and you’re like, absolutely not. Yes.
Marc: Under,
Alessio: and I wa I was in an event with Sam last night, uh, and he actually said in the last couple weeks it felt like now people are taking that seriously.
Yeah. So I’m just curious like how you’re seeing the structure of organization changing, especially when you invest in early stage companies and, um, yeah, just like how the impact of. Work structure and, uh, all of that is playing out. Yeah.
Marc: So there’s a whole bunch of, there’s a whole bunch of topics. I know, yeah.
We, we could spend, and by the way, we’d be happy to spend more time, but we could, we could spend more time on all that. So just for people who haven’t followed this, so the, this, this, this term managerial comes from this thinker in the 20th century, James Burnham, who, um, just one of the great kind of 20th century political thinkers, um, societal thinkers.
And he sort of said a as, and he was writing in like the 1940s, 1950s. Um, and he said kind of the, the whole history, capitalism until that point had been in two phases. Number one had been what he called bourgeois capitalism, which was think about as like name on the door, like Ford Motor Company. ‘cause Henry Ford runs the company.
Um, and Henry, it’s like a DIC dictatorial model. And Henry Ford just like tells everybody what to do. And he said the problem with bourgeois capitalism is it doesn’t scale. ‘cause Henry Ford can only tell so many people to do so many things. And then he runs at a time in the day. And so, um, he said the second phase of capitalism was what he called managerial capitalism, which was the creation of a professional class of managers, um, that are trained not to be like.
Car experts or to be whatever experts in any particular field, but are trained to be experts in management. And then that led to, you know, the importance of like Harvard business, you know, business schools and management consulting firms and all these things. And then you look at every big company today, and like most of the executives at most of the Fortune 500 companies are not domain experts in whatever the company does.
And they’re certainly not the founders of those, but they’re professional managers. And in fact, in the course of their careers, they’ll probably manage many different kinds of businesses. They’ll rotate around and they might work in healthcare for a while and then work in financial services and then go work in something else, you know, come work in tech.
And what Burnham said is he said that transition is absolutely required because the, the, the, the problem with bourgeois capitalism is, is it doesn’t scale. Henry Ford doesn’t scale. And so if you’re gonna run capitalist enterprises that are gonna have millions to billions of customers, um, you’re gonna need to, you’re, they’re gonna be operating a level of scale and complexity that’s gonna require this professional management class.
And he said, look, the, the professional management class has its downsides. Like they’re not necessarily experts at doing the thing. They’re not as inventive, you know, they’re not gonna create the next breakthrough thing. But he is like, whether you think that’s good or bad or whatever is what’s gonna be required.
And basically that’s what happened. Right. And so he wrote that book originally in like 1940, you know, over the course of the next 50 years, basically. Managerialism. Well, I mean, today, up till today, managerial managerialism basically took over everything. Mm-hmm. And you know, what I’m describing is basically how all big companies run and how all governments run and how are large scale nonprofits run and kind of everything, you know, everything runs basically what, what, what Venture Capital does is we basically are a rump, uh, sort of protest movement to that.
To try to find the next Henry Ford or, or just to say El Elon Musk or, or the next, or the next Elon Musk or the next Steve Jobs, or the next Bill Gays. The next Mark Zuckerberg. And so we, we, we, we start these companies in, in the old model, right? We, we, we start them out as, as, as, as in the Henry Ford model.
And so we start them out with a founder or a, or a, or a founder with, with colleagues. But you know, there’s the a founder, CEO, um, and then we basically bet that we basically bet that the startup is going to be able to do things, specifically innovate in ways that the big incumbents in that industry are not gonna be able to do.
And so it’s a bet that by, basically by relighting this sort of name on the door, you know, kind of thing. Mm-hmm. This new innovative thing with like a king monarchical, uh, uh, political structure, um, that they’re gonna be able to innovate in a way that the incumbent is not going to be able to because the incumbent is, is being run by managers.
Right. And, and, and, and by the way, and of course venture being what it is, sometimes that works, sometimes it doesn’t. But we’re, we’re constantly doing that, but I’ve always viewed it my entire life as like, we’re like raging against the dying of the light. Mm-hmm. Like we’re, we’re, we’re, we’re sort of constantly trying to fight off managerialism, just basically swamping everything and everything.
Getting basically boring and gray and dumb and old. Right. And we’re trying to keep some level of energy vitality in the system. AI is the thing that would lead you to think, wow, maybe there’s a third model.
Alessio: Mm-hmm.
Marc: Right? And, and maybe may and way to think about it would be, maybe it’s a combination of the two, maybe the new Henry Ford or the new Elon or the new Steve Jobs plus ai, right.
Is the best of both. Right. Because it’s, it’s, it’s sort of the spark of genius of the name on the door model, the Henry Ford model. But then it’s give that person AI superpowers to do all the managerial stuff and let the boss draw the managerial stuff. That may be the actual secret formula. And we’ve never even known that we wanted this because we never even thought it was a possibility.
But I mean, you know, this, what is the thing that these bots are really good, they’re really good at doing paperwork. Like they’re really good at filling out forms, right? Like they’re really good at writing reports, they’re really good at reading, they’re really good at doing all the managerial work. Like they’re amazing at it.
And so, yeah, so I, I think, I think the, I a hundred percent, I think the answer, the answer very well might be to get the best, best of both worlds by doing this. And then the challenge is gonna be twofold. The challenge is gonna be for the innovators to really figure out how to leverage AI actually do this.
Right? Um, and, and then, and then the, the other challenge is gonna be for the, for the incumbents that are managerial, to figure out like, okay, what does that mean? ‘cause now they’re gonna, they’re, they’re gonna be facing a different kind of insurgent competitor that has a different set of capabilities than they’re used to.
And so th the, this really I think is gonna force a lot of big companies to kind of figure out innovation. EE either I say figure out innovation or die trying.
Alessio: Do you feel like that structure accelerates the impact on the actual GDPN economy? If you look at Space Act? Yes. The growth is like so fast. Yeah.
And like, instead of having these companies kind of like Peter out in growth and impact, they can kind of like keep going if not accelerating.
Marc: Yeah, that’s for sure. The hope, um, the, the, the challenge and, and you know, and, and look, the AI utopian view is of course, of course. And, and, and that’s gonna be the future of the economy.
And it’s gonna grow 10 x and a hundred x and a thousand x. And we’re entering this regime of like much higher economic growth forever and consumer cornucopia of everything. And it’s, it’s gonna be great. And I, and, and I hope that’s true. I hope that’s, that’s like the u you know, that’s the current kind of utopian vision.
I hope that’s true. The problem is, it goes back again. The real world is really messy. Um, and I’ll give you an example of how the real world is really messy. It requires 900 hours of professional certification training to become a hairdresser in the state of California. Um, so it’s like 35% of the economy, something like that.
You have to get some sort of professional certification to do the job, which is to say that the, the professions are all cartels, right? Yeah. And so you have to get licensed as a doctor. You have to get licensed as a lawyer, you have to get licensed as a. You have to get into a union. Mm-hmm. Um, by the way, to, to work for the government, you need to be, you, you have both civil service protections and you have public sector unions.
You have two layers of insulation, uh, against ever getting fired for anything or anything. Anything ever changing. I’ll give you another example. The the dock work. The dock workers one on strike a couple years ago. Mm-hmm. ‘cause they, you know, robotics, you know, if, if you go look at a modern dock, like in Asia, it’s all robots.
If you go to American dock, it’s like all still guys, dragon, dragon stuff, by by hand, the dock works. Goes on a strike. It turns out there are 25,000 dock workers working on, on, on, on Docs in America. It turns out they have incredible political power. Mm-hmm. Because it’s a, it’s, it’s one of these un unified blocks of things.
They won their strike and so they got commitments from the dock owners to not implement more automation. We learned a couple things in that. So number one, we learned that even a union as small as 25,000 people still has like tremendous political stroke. We also learned that they, it actually turns out the Dock Workers Union has 50,000 people in it.
‘cause there’s 20, they have 25,000 people working in the docks. They have 25,000 people during full paycheck sitting at home from prior union agreements. Oh my
swyx: God.
Marc: From prior union agreements. I’ll give you another great example. There are government agencies, there are federal government agencies where the employees right of have civil service protections and there are in public sector unions.
There are entire federal government agencies that struck new collective bargaining agreements during COVID, where not only are they have their jobs guaranteed in perpetuity, but they only have to report to work in an office one day per month. And so there are entire office buildings in Washington DC that are empty 29 outta 30 days of the year that are still operating and are still, we’re all still paying for it.
20 and say, and then what they do, it turns out what the employees do is they’re very, they’re very smart in, in, in this way. And so they figure out, they come in on the last day of a month and the first day of the next month. And so and so, they’re, so, they’re in there, they’re in the office two days per 60 days, which means these buildings are empty for 58 days at a time.
And you see what I’m, you see where I’m heading with this? Like this is like locked in, right? This is like locked in in a way that has nothing to do with like, and people say capitalist, it’s like anticapitalistic. It’s like, it’s, it’s basically it’s restrictions on trade, it’s restrictions on the ability to like change the workforce.
And so, so much of our economy is, is, you know, the, the, I I’m, I’m describing the entire healthcare system. I’m describing the entire legal profession. I’m describing the entire housing industry. I’m describing the entire education system, right? K through 12 schools in the United States. They’re a literal government monopoly.
How are we gonna apply AI and education? The answer is we’re not, because it’s a literal government monopoly, it is never going to change the end. And there is nothing to do, by the way, you can create an entirely new school system. Like that’s the one thing you can do, is you can do what Alpha School’s doing.
You can create an entirely new school system. Other than that, you’re not gonna go in and change what’s happening in the American classroom, like K through 12. There’s no chance the teachers are 100% opposed to it. It’s a hundred percent not gonna happen. So, so you see what I’m saying is like there’s this like massive slippage that’s gonna take place.
Both the AI utopians and the AI dors are far too optimistic.
swyx: Right.
Marc: You see what I’m saying? Be because they believe that because the technology makes something possible that 8 billion people all of a sudden are gonna change how they behave. And it’s just like, nope. So much of how the existing economy works.
Mm-hmm. It’s just, it. It’s just like wired in. And so we’re gonna be lucky as a society, we’re gonna be lucky if AI adoption happens quickly. Right. Because if it doesn’t, what we’re just gonna have is stagnation.
Alessio: Awesome. Mark. I know you gotta run.
swyx: Yeah. We all know or still welcome. But, uh, it was such a pleasure talking to you.
Uh, we’re truly living in the age of science fiction coming to real life.
Marc: Yes. Yes. Could not be more exciting. Yeah. Really. Thank you, mark. You guys awesome.
swyx: Thank That’s it.
Marc: Good. Thank you. That’s it.

This image from Arena shows progress over the years (exaggerated by the # ordinal ranking rather than numerical, but truly standard benches like GPQA and AIME also improved tremendously vs Gemma 3):

The licensing is also improved with a proper Apache 2.0 license, and they “natively process video and images, supporting variable resolutions, and excelling at visual tasks like OCR and chart understanding. Additionally, the E2B and E4B models feature native audio input for speech recognition and understanding.”

The excellent on device capabilities makes one wonder if these are the basis for the models that will be deployed in New Siri under the deal with Apple….

AI News for 4/1/2026-4/2/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Twitter Recap

Google DeepMind’s Gemma 4 release: open-weight, Apache 2.0, multimodal, long-context—plus rapid ecosystem rollout

• Gemma 4 is Google’s biggest open-weight licensing + capability jump in a year: Google/DeepMind launched Gemma 4 as a family of models explicitly positioned for reasoning + agentic workflows and local/edge deployment, now under a commercially permissive Apache 2.0 license (a notable shift from prior Gemma licensing). See launch threads from @GoogleDeepMind, @GoogleAI, and @Google, with Jeff Dean’s framing and adoption stats (Gemma 3: 400M downloads, 100K variants) in @JeffDean.

• Model lineup + key specs: Four sizes were announced—31B dense, 26B MoE (“A4B”, ~4B active), and two “effective” edge models E4B and E2B aimed at mobile/IoT with native multimodal support (text/vision/audio called out for edge). DeepMind highlights include function calling + structured JSON, and long context up to 256K (large models) in @GoogleDeepMind and @GoogleAI. Community summaries and “how to run locally” guidance proliferated quickly, e.g. @_philschmid and @UnslothAI.

• Early benchmark signals (with caveats):

  • Arena/Text: Arena reports Gemma-4-31B as #3 among open models (and #27 overall), with Gemma-4-26B-A4B at #6 open in @arena; Arena later calls it the #1 ranked US open model on its open leaderboard in @arena.

  • Scientific reasoning: Artificial Analysis reports GPQA Diamond 85.7% for Gemma 4 31B (Reasoning) and emphasizes token efficiency (~1.2M output tokens) vs peers in @ArtificialAnlys and @ArtificialAnlys.

  • Several posts stress the scale/efficiency surprise (e.g., “outperforms models 20× its size”) but note that preference-based leaderboards can be gamed; Raschka’s more measured read is in @rasbt.

• Day-0 ecosystem support became part of the story: Gemma 4 landed immediately across common local + serving stacks:

  • llama.cpp day-0 support: @ggerganov

  • Ollama (requires 0.20+): @ollama

  • vLLM day-0 support (GPU/TPU/etc.): @vllm_project

  • LM Studio availability: @lmstudio

  • Transformers/llama.cpp/transformers.js callout: @mervenoyann

  • Modular/MAX production inference “in days”: @clattner_llvm

• Local inference performance anecdotes got unusually concrete:

  • “Brew install + llama-server” became the canonical one-liner for many: @julien_c.

  • llama.cpp performance demo: Gemma 4 26B A4B Q8_0 on M2 Ultra, built-in WebUI, MCP support, “300 t/s (realtime video)” in @ggerganov (with a follow-up caveat about prompt-recitation/speculative decoding in @ggerganov).

  • RTX 4090 long-context throughput + TurboQuant KV quant details in @basecampbernie.

  • Browser-local run via WebGPU/transformers.js demo noted by @xenovacom and amplified by @ClementDelangue.

Gemma 4 architecture notes: hybrid attention, MoE layering choices, and efficiency tricks

Unusual transformer details

• eliebakouch highlighted:

  • per-layer embeddings on small variant

  • no explicit attention scale (suggesting it may be absorbed into norm weights)

  • QK norm + V norm

  • shared K/V for large variant

  • aggressive KV cache sharing on small variant

  • sliding window sizes 512 and 1024

  • no sinks

  • softcapping

  • partial-dimension RoPE with different theta for local/global layers

• Grad62304977 replied that the missing attention scale is likely merged into QK norm weights.

• baseten summarized additional architecture choices:

  • alternative attention mechanisms

  • proportional RoPE

  • Per-Layer Embeddings (PLE)

  • KV-cache sharing

  • native aspect-ratio handling for vision

  • smaller frame window for audio

• norpadon called it “very much not a standard transformer.”

• rasbt offered a more conservative read for the 31B dense: architecture looks “pretty much unchanged compared to Gemma 3” aside from multimodal support, retaining a hybrid 5:1 local/global attention mechanism and classic GQA, suggesting the bigger jump likely came more from the training recipe and data than radical dense-model architecture change.

• “Not a standard transformer” takes, plus specific deltas: A thread flagged Gemma 4 as having “galaxybrained architecture” in @norpadon, followed by more specific notes on how Gemma’s MoE differs from DeepSeek/Qwen (Gemma uses MoE blocks as separate layers added alongside normal MLP blocks) in @norpadon.

• Concrete low-level details being circulated: A concise recap of quirks (e.g., no explicit attention scale, QK/V norm, KV sharing, sliding window sizes, partial RoPE + different theta, softcapping, per-layer embeddings) is in @eliebakouch. Baseten’s launch post also lists similar “architecture innovations” (PLE, KV-cache sharing, proportional RoPE, aspect ratio handling for vision, smaller audio frame window) in @baseten.

• Raschka’s read: minimal architectural change, big recipe/data change: Raschka argues Gemma 4 31B is architecturally close to Gemma 3 27B, still using a hybrid sliding-window + global attention pattern and GQA, implying the leap is likely training recipe/data rather than architecture overhaul: @rasbt.

Agents, harness engineering, and “local agents” momentum (Hermes/OpenClaw + model/harness training loops)

• Open-models-as-agent-engines is now mainstream positioning: Multiple posts frame Gemma 4 as the “perfect” local model for open agent stacks (OpenClaw/Hermes/Pi/opencode). See @ClementDelangue, @mervenoyann, and @ben_burtenshaw.

• Hermes Agent growth + pluggable memory:

  • Hermes Agent hit a major usage milestone and asked for roadmap input: @Teknium.

  • Memory integrations were expanded to multiple providers via a new pluggable system: @Teknium.

  • A local semantic index plugin (“Enzyme”) pitched as solving the “too many workspace files” issue with local embedding and 8ms queries: @jphorism.

• Harness engineering as the moat (and the loop): A strong “Model–Harness Training Loop” thesis—open models + traces + fine-tuning infra—was articulated in @Vtrivedy10 and echoed more generally in @Vtrivedy10. Related: LangChain notes open models are “good enough” at tool use/retrieval/file ops to drive harnesses like Deep Agents in @hwchase17.

• Agent self-healing + observability trends:

  • A blog on “self-healing” GTM agent feedback loops is referenced by @hwchase17 and expanded on by @Vtrivedy10.

  • LangSmith reports Azure’s share of OpenAI traffic rose from 8% → 29% over 10 weeks, based on 6.7B agent runs, suggesting enterprise governance/compliance is driving routing decisions: @LangChain.

Tooling and infra: kernels, fine-tuning stacks, vector DB ergonomics, document extraction

• New linear attention kernel: A CUDA linear attention kernel drop is in @eliebakouch (repo link in tweet).

• Axolotl v0.16.x: Axolotl’s release emphasizes MoE + LoRA speed/memory wins (claimed 15× faster, 40× less memory) and GRPO async training (58% faster) plus docs overhaul in @winglian and @winglian. Gemma 4 support follows in @winglian.

• Vector DB ergonomics: turbopuffer adds multiple vector columns per doc (different dims/types/indexes) in @turbopuffer.

• Document automation stack: LiteParse + Extract v2:

  • LiteParse open-source document parser: spatial text parsing with bounding boxes, fast on large table-heavy PDFs, enabling audit trails back to source in @jerryjliu0.

  • Extract v2 (LlamaIndex/LlamaParse): simplified tiers, saved extract configs, configurable parsing before extraction, transition period for v1 in @llama_index and additional context from @jerryjliu0.

Frontier org updates: Anthropic interpretability, OpenAI product distribution, and Perplexity “Computer for Taxes”

• Anthropic: “Emotion vectors” inside Claude: Anthropic reports internal emotion concept representations that can be dialed up/down and measurably affect behavior (e.g., increasing a “desperate” vector increases cheating; “calm” reduces it). The core threads are @AnthropicAI, @AnthropicAI, and @AnthropicAI. The work also triggered citation/precedent disputes in the interp community (e.g., @aryaman2020, @dribnet, and discussion around vgel’s posts via @jeremyphoward).

• OpenAI: CarPlay + Codex pricing changes:

  • ChatGPT Voice Mode on Apple CarPlay rolling out for iOS 26.4+: @OpenAI.

  • Codex usage-based pricing in ChatGPT Business/Enterprise (plus promo credits): @OpenAIDevs. Greg Brockman reinforces “try at work without up-front commitment”: @gdb.

• Perplexity: agentic “Computer for Taxes”: Perplexity launched a workflow to help draft/review federal tax returns (“Navigate my taxes”) in @perplexity_ai with details in @perplexity_ai.