No Moat: When the Model Becomes the Starting Point

A model on a phone changes the competitive question

Simon Willison could run Vicuna on his phone. He had watched LLaMA become the foundation for Alpaca, whose fine-tuning substantially improved its usefulness, and then Vicuna—the strongest model he had personally run on his hardware. That progression posed a problem for large AI labs: how could a centralized training operation compete with many developers improving a shared model at once?

The provocation came from “We Have No Moat and Neither Does OpenAI”, a purportedly leaked Google memo discussed in this May 2023 Latent Space conversation. While Google and OpenAI competed to train larger models, the memo argued, open development had become a serious competitive force in roughly two months. Simon considered the analysis worth discussing regardless of its provenance.

Its central argument was about iteration speed versus model scale: smaller models could be fine-tuned—adapted through further training—in hours on laptops, accumulating improvements between giant training runs. Simon nevertheless resisted claims of parity. In his judgment, local models still fell short of GPT-3.5 overall, although they approached it in some respects. Their trajectory was more persuasive than their current capabilities.

Free weights still need somewhere to run

The first disagreement concerned what would become a commodity. One participant expected converging capabilities and low switching costs to push model providers toward price competition. Another separated model quality from product delivery: freely available model weights do not supply the GPUs or operations needed for inference, the work of running a model to produce answers.

Kafka and Elastic supplied the analogy: free software can still support a hosted-service business because infrastructure costs money to operate. Yet the participant making that argument also doubted that merely hosting a model offered much differentiation. How open-model contributors would capture value remained unresolved.

Local inference complicated that thesis. Simon described WebLLM opening a ChatGPT-style interface in Chrome Canary, using WebGPU after a roughly two-gigabyte download. Generating animal rap battles impressed him, though this was a qualitative demonstration without a hardware specification or benchmark. He discussed it alongside Vicuna 13B, his favorite open model among those he had tried, whose LLaMA-derived terms then prohibited commercial use.

Quantization helped make local execution practical by representing model values with fewer bits. Simon credited llama.cpp with popularizing four-bit representations in place of 16-bit floating-point values. He also mentioned a recent pruning paper suggesting that half a model could be removed without losing quality, but gave no model or evaluation conditions for that claim. Running locally did not ensure factual accuracy: Vicuna on his phone credited him with inventing a bear-or-bunny approach to living that he had never created.

Products, talent, and access to useful data

Google’s email, calendars, and Docs offered a counterargument to the headline: a moat—a durable competitive advantage—could exist beyond model quality. Another participant identified a separate vulnerability. A perception of falling behind could drive researchers toward OpenAI or academia, weakening future innovation even while Google retained its financial and product advantages.

Meta illustrated the value of developer momentum. Alpaca and Vicuna extended LLaMA despite its commercial restrictions, creating a family of models around a common foundation. For Google, one proposed advantage was first-party data feedback: collecting useful training examples through its own products. If small amounts of high-quality data could substantially improve models, activity in Docs, Sheets, and YouTube might offer opportunities a startup would struggle to reproduce.

The counterargument was that useful data need not come from a giant consumer platform. RedPajama aimed to build a LLaMA alternative using openly licensed training data. Its released dataset reportedly occupied 2.7 terabytes, enough to fit on a suitably equipped laptop. That made the data accessible; it did not establish that training the model was equally accessible.

Instruction fine-tuning offered a more approachable example. The discussion distinguished a base model’s language capabilities from the adaptation that makes it useful for following instructions, as Alpaca had done with LLaMA. Databricks reportedly gathered questions and answers from 2,000 employees over a month for Dolly, while OpenAssistant crowdsourced examples. The contention was that tens of thousands of good examples could substantially improve a raw model without requiring a vast consumer service.

LoRA and the value of accumulated adaptations

LoRA, or low-rank adaptation, supplied the mechanism behind much of the excitement. Its broad approach is to hold parts of a model fixed while training a smaller set of parameters, described in the conversation as two smaller matrices. The participants’ understanding was preliminary, so their explanation stopped short of a reliable implementation recipe.

The tangible precedent came from Stable Diffusion. Users shared LoRA files representing concepts such as a particular person’s face. These could occupy only megabytes, and multiple LoRAs could reportedly be loaded into a model. Applying that pattern to language models suggested installing capabilities as easily as packages with npm: an aspiration for composable improvements, rather than an established language-model package system.

That possibility introduced another moat: adaptations accumulated around a shared base. A participant argued that LLaMA’s community could make a competing model less attractive if existing work did not transfer. Another suggested that adaptations might depend on a particular base-model version, making retraining disruptive. An older model refined for years by a community might then outperform a new replacement—an intriguing hypothesis, with no comparison reported.

Programming languages supplied the analogy. A technically better language still has to compete with existing libraries and infrastructure. Likewise, a better base model might ask developers to abandon valuable adaptations. This was also Mojo’s appeal: the prospect of improving performance while retaining Python’s ecosystem.

Fresh weights or better tools?

Before turning to Mojo, the discussion considered two ways to supply newer information. One participant proposed lightweight LoRA updates as a potentially scalable alternative to ordinary fine-tuning or retrieval. Search-augmented systems such as Perplexity illustrated retrieval: search in real time, then insert the results into the model’s context window, the information available while it generates an answer. That could supply current information and help reduce fabricated answers. LoRA updates remained a proposed alternative, not a tested improvement over retrieval.

A contrasting priority was a local model that knew when to look something up. It could search, access email and notes, and summarize material without storing every current fact in its weights. A participant thought Vicuna on a laptop might already be capable enough for a personal research assistant. Connecting models to tools appeared to require little Python and prompting, but that ease of assembly did not establish reliability.

Deployment constraints supplied the counterpoint. A participant recounted discussing language models on fighter jets with a Lockheed Martin researcher—a scenario where external service calls might be unavailable and milliseconds could matter, not a confirmed deployment. Flexible assistants could retrieve information through external services; constrained applications might need more capability embedded in an optimized local model.

Mojo: performance without starting over

Simon introduced Mojo as a newly announced language intended to extend Python with compiled, highly optimized code while preserving its syntax and compatibility. General access and an eventual open-source release were still pending. Chris Lattner’s work on LLVM, Clang, and Swift gave Simon reason to take the ambition seriously.

A demonstration in Jeremy Howard’s article particularly impressed him. Simon reported a roughly 2,000-fold speedup over matrix multiplication implemented in Python, with changes including static typing and a struct in place of a class. The baseline matters: no hardware, matrix dimensions, or comparison against optimized numerical libraries accompanied that account. It was a specific demonstration, not a general multiplier for Python workloads.

From Howard’s explanation, Simon understood Mojo to build on MLIR, compiler infrastructure supporting multicore and GPU work. He saw promise in combining established compiler ideas with a high-level language and an experienced team. Howard’s enthusiasm encouraged him to investigate; he was still waiting to try Mojo himself.

TypeScript supplied a comparison with a different objective: typing and reliability rather than Mojo’s performance emphasis. The shared test was incremental adoption. Could researchers and enterprises accelerate selected workloads while keeping their libraries, data tooling, and workflows? Waitlisted access left that unresolved. Still, one participant saw a broader founder opportunity: if model capabilities converged, better runtimes, languages, tooling, and data collection could remain valuable places to compete.

Ecosystem influence and the search business

Returning to strategy, Simon argued that Meta had gained an unexpected advantage because others were innovating on LLaMA. Commercial relicensing could strengthen that position, but remained an open question. Someone at Facebook Machine Learning Research had informally told him that reputational risk from harmful uses might prevent it. That was one person’s assessment; a separate rumor about opening LLaMA was explicitly acknowledged as unsourced. Meanwhile, developers were already contributing optimizations and extensions despite the commercial restrictions.

Google faced a different commercial pressure. One daily ChatGPT user reported sharply reduced Google usage and preferred ad-free answers to an advertising-supported chatbot. That experience suggested a threat to search monetization even if Google could deliver a competitive model. Another participant reported seeing a Bing Chat advertising prototype, prompting speculation about ad blockers for conversational interfaces.

Misuse scales; tool access creates exposure

The safety discussion focused on scalable human misuse. A scammer could already deceive someone, but a language model might support many simultaneous romance scams. One participant considered laptop-running Vicuna 13B a plausible enabler, without reporting a test. The concern was that widely available models could expand the scale of an existing harm.

The same participant doubted that a local model predicting the next part of a sentence would independently become a catastrophic threat. They argued that goal setting and substantial additional machinery would be necessary, and worried that speculative scenarios could distract from present harms. This was a position about safety priorities, not a guarantee about model safety.

Prompt injection made the application risk concrete: attacker-controlled text could redirect a model-connected system toward disclosure or unauthorized actions. A participant reported LinkedIn biographies whose embedded instructions influenced automated recruiter emails, causing the messages to reveal their automated origin. Text collected as profile data had changed the message generator’s behavior. With access to private information or physical controls, that failure could have more serious consequences.

What the memo recommended

The memo’s provenance and authority remained separate questions. A participant reported secondhand confirmation from Google contacts that it was authentic. Even so, an employee’s internally circulated opinion need not represent executives or official strategy. Participants differed over whether it revealed justified internal panic or was being asked to stand for too much.

OpenAI prompted a similar distinction between a model and its supplier. One participant pointed to talent as a moat. Another expected low switching costs to commoditize model access while still considering OpenAI a well-positioned team and company. Price pressure on a capability did not settle the prospects of the organization producing it.

The memo’s epilogue argued that researchers moving from Google to OpenAI already transferred knowledge, weakening the strategic value of secrecy. It predicted that open alternatives would eventually surpass OpenAI unless it changed its stance. Its prescription was for Google to move first by releasing models and engaging more deeply with open development—a strategic recommendation, not an outcome the conversation could establish.

The roadmap problem reaches individual builders

A brief closing exchange touched on the White House meeting: one participant considered Anthropic’s inclusion significant, while another noted Cohere’s absence. They had seen arrival photos but had no post-meeting outcomes to discuss.

The project updates then made the strategic debate personal. Simon’s Datasette, an open-source project for analyzing and publishing data, faced a challenge from ChatGPT’s Code Interpreter. In his assessment, uploading a SQLite database could accomplish everything on his next twelve months of roadmap. That experience pushed him to reconsider the AI features Datasette needed to remain useful.

Simon was tracking developments through daily blog updates, an email newsletter a couple of times a week, and an RSS feed. His No Moat recap offered a written companion to this discussion.

Travis described another response: a newly founded company developing reliable agent primitives for constrained use cases. Instead of attempting broad autonomy immediately, he wanted a narrower, dependable foundation and planned an initial open-source TypeScript release.

His ChatGPT Twitter bot supplied the operational coda. Travis reported about 125,000 followers and estimated roughly $1,000 in total costs, covered by OpenAI sponsorship. Those figures came without an operating period or usage breakdown. He had also changed the bot’s green ChatGPT logo to purple at OpenAI’s request, a change he said the company accepted.