Open AI Needs More Than Open Models

Access changes who gets to build

Growing up in India, Soumith Chintala had little internet access; even in college, he largely relied on a slow mobile connection. As knowledge became freely distributed, he could learn faster without paying for it. That experience grounds his commitment to open source: distributing software also distributes opportunity. He credits that access with helping his career and says he would pursue open source even without being paid.

Early in this Latent Space conversation, Chintala also separates opportunity from prestige. He originally wanted to become a 3D visual-effects artist, then stopped because he thought he was bad at it. Had that path worked out, he believes he could have been equally happy, even if others judged his success differently. Intrinsic interest becomes a recurring criterion for choosing what to build.

PyTorch: simplicity has a hardware cost

PyTorch’s design problem is to keep ordinary model programming easy while supporting increasingly complex research. Chintala describes an incrementally ambitious approach: start simply, then add capability while keeping frequent uses on an easy path. He invokes Huffman coding as an analogy for allocating interface friction: common uses get short paths; niche uses can require extra steps.

Tinygrad illustrates how scope grows beyond a small beginning. Chintala recalls it expanding beyond an early thousand-line aspiration and acknowledges that newer technology could also simplify a PyTorch rewrite. But much of the complexity comes from mapping computation onto hardware. Efficient execution must account for data moving through CPU caches, memory and storage, or through GPU memory and connections between accelerators.

His example is a sparse dot product taking three tensors: changing the input shapes can change the best execution strategy, even when the operation stays the same. PyTorch handles this variety through hundreds of configurations for individual operators, using templates and symbolic generation to produce CUDA or CPU code. A compact mathematical operation can therefore require a substantial implementation to run efficiently across inputs.

That cost can be borne in different places: - Encode configurations in the framework. More implementation complexity reduces the need to search for efficient mappings during compilation. - Search at runtime. A simpler framework can discover configurations, but users bear longer compilation times. - Narrow the workload. Supporting only Llama-style transformers can be much simpler than supporting the breadth of AI research. Chintala’s skepticism about George Hotz’s simplification vision is conditional: broad generality with near-zero compile time would require major breakthroughs.

Integrations follow users and product scope

Chintala explains Meta’s funding of PyTorch as an investment in a widely adopted standard that benefits its own work and gives it a timing advantage. Hardware support follows user demand. When users increasingly wanted Apple GPU support, the team encouraged Apple to contribute engineers and devoted substantial Meta-funded engineering time to the work.

Software dependencies face a similar test: ease of installation, smooth integration and enough user benefit to justify adoption. PyTorch’s dependency on OpenAI Triton illustrates that approach. Optional Mojo integration would face the same conditions, plus a more specific question: what does efficient arbitrary custom code add to a program already expressed through PyTorch’s roughly thousand-operator vocabulary?

One proposed use is for torch.compile to consume Mojo subgraphs—custom portions of an otherwise mostly PyTorch program. Interoperability has a clearer use case than replacing the frontend. Making Mojo the main way to express the program would replace PyTorch’s role rather than extend it. These are possible integration boundaries, not adoption commitments.

[Apple’s MLX](https://github.com/ml-explore/mlx), still early in the conversation’s March 2024 context, illustrates how product scope shapes complexity: - Stay Apple-focused. A smaller addressable market could still support a useful, focused framework. - Expand across server workloads. Distributed computing, NVIDIA and AMD support, a generalized backend and compilation overhead could recreate PyTorch’s complexity without enough differentiation.

Server competitiveness also depends on the interconnect—the connections that move data between accelerators. The respondent regards NVLink’s bandwidth and latency as a major NVIDIA advantage and makes Apple’s prospects conditional on solving its interconnect problem. Google’s accelerators also receive credit for their interconnect, with the qualification that users must access them through Google.

Evaluate inference on the workload you will buy

The early framework community provides a bridge to inference services. Among roughly twenty frameworks, the respondent recalls finding only four or five distinctive convolution kernels and building a benchmark suite whose results appeared in startup fundraising decks. Lepton and Fireworks come up as examples of the PyTorch/Caffe2 cohort’s subsequent influence.

Evaluating those founders as an investor and evaluating their services as a customer require different evidence: - For an investor, technical expertise is one capability. GPU programming and experience serving models at scale matter, but market understanding and financial discipline need separate assessment. - For a customer, the service must demonstrate usability, reliability and speed. An unknown supplier merits consideration if its product works well.

That distinction shapes the criticism of Anyscale’s benchmarks. The interviewer questions whether competing endpoints were compared on equivalent terms. The respondent focuses on coverage: latency numbers alone cannot support a purchasing decision. A useful evaluation needs: - Cost at realistic volume. Estimate the bill for an actual workload—illustratively, a million API calls per month—rather than relying on an isolated call price. - Performance across time. Measure latency and reliability at different hours and throughout the week. - Representative requests. Exercise real workloads instead of repeatedly submitting one generic, cacheable paragraph.

The respondent predicts a laundromat-like inference business: competition compresses margins around hardware costs, API prices and customers’ latency tolerance. Faster kernels may help, but the estimated advantage of a month or two is a judgment, not a measured duration. The reasoning is that many teams are optimizing similar Llama-style models on only a few hardware configurations. That narrow search space makes independent rediscovery likely even when techniques remain unpublished.

Synthetic data needs a source of structure

Beyond language-model serving, reported PyTorch applications range from drug discovery to Mars rover simulations. A visit to CERN supplies a memorable example: hearing about particle-physics research using both PyTorch and GANs made the visiting GAN researcher more nervous about the models than the framework.

The respondent singles out combining search and symbolic structure with learned models as particularly promising. AlphaGo and AlphaGeometry illustrate this direction, alongside attempts to combine language-model reward models with search. AlphaGeometry itself is not claimed as a PyTorch application; related work in biology and chemistry matters because those applications call for stronger guarantees on outputs.

Asked how geometry Olympiad problems connect to practical applications, the respondent turns to synthetic data as a way to transfer existing knowledge. Symbolic models encode knowledge through formal structures, such as those developed in mathematics, physics and language. Generating input-output pairs from those structures gives a neural network examples to learn through gradient descent. The network may then represent, with many more parameters, knowledge already encoded compactly in the symbolic model.

The key question is where the generated data’s useful structure comes from. The respondent is skeptical of treating synthetic data as a universal solution without a good underlying symbolic model. An interviewer introduces another case: Nous Research generating mock textbooks with GPT-4, inspired by Phi-2, to fine-tune open models such as Llama. The response accepts the synthetic-data label but emphasizes the specific objective: distillation, teaching another model to reproduce GPT-4’s behavior.

The interviewer calls opaque transfers through model outputs “dataset washing”: downstream training can obscure the original data’s provenance. This is a concern, not a demonstrated finding about a company. Using a model as a judge or preference ranker raises a related distinction: how should social and copyright boundaries change with the amount of information transferred? The conversation’s examples of one, twenty and two hundred bits are conceptual, not legal thresholds.

A final proposal connects document retrieval with fine-tuning: generate varied training examples from retrieved documents, then train on those examples. The intended mechanism is learning the same information across different phrasings, reducing sensitivity to wording and overfitting to raw documents. It remains a proposal without reported experimental results.

Whether symbolic reasoning difficulties are fundamental to transformers remains unresolved in the response. The relevant system includes tokenization, architecture and training dynamics, including when mathematical material appears during training. The respondent also cautions that some claimed failures disappear at sufficient scale, without specifying a benchmark or threshold.

Model generations compete for time

[OPT-175B’s contribution included its training logs](https://ai.meta.com/blog/democratizing-access-to-large-scale-language-models-with-opt-175b/), which exposed the practical complexity of training large models. The respondent believes retrospectively that OPT would have benefited from longer training. Llama 1 is credited to Guillaume Lample and his team, but the respondent disclaims close involvement and does not explain their scaling-law decisions.

Llama 2 is described from closer infrastructure involvement as an incremental evolution: more data, longer training and several unspecified architecture changes. Asked why training stopped despite apparently improving loss curves, the respondent offers a general framework rather than the specific explanation: starting the next generation may be worth more than extending the current run. New or cleaner data changes that opportunity cost. Time becomes the bottleneck in an iterative release cycle.

The March 2024 conversation describes projected year-end capacity of roughly 600,000 H100 equivalents across all of Meta’s data centers. This includes other accelerators and workloads; it is neither a physical H100 count nor the allocation to one Llama run. GPU allocation resembles a portfolio of competing investments, constrained further by which workloads and model sizes fit particular clusters.

More resources need not mean pursuing larger versions of the same idea. The respondent recalls leaving GAN research when its popularity made it hard to assess a personal marginal contribution. The career advice is secure a fundable baseline, then find the intersection of interest and strength. Fundability means credibility within a field and enough support for a reasonable livelihood, not an unspecified three-month compute budget. For work beyond one’s strengths, a collaborator can help identify a useful complementary contribution.

MTIA, the custom-silicon effort, brings the discussion back to hardware mapping. Specializing a chip for one or two workloads can improve power efficiency and simplify the software kernels needed to use it. The investment requires enough workload volume and expected stability: recurring computations must remain suitable for specialization long enough to justify building the chip.

Openness distributes opportunity and power

For model releases, the respondent uses open broadly, acknowledging uncertainty about what constitutes a model’s source and distinguishing commercial, noncommercial and conditional licenses. The benefit being defended is low-friction access: a student who can obtain and experiment with a model gains an opportunity even before understanding its licensing terms.

Two forms of openness contribute differently: - Corporate-funded releases absorb substantial costs. PyTorch, Llama and Segment Anything exemplify work whose capital and operating requirements would be difficult for volunteers to sustain. - Community-built projects pursue overlooked needs. Volunteers can coordinate around problems that established businesses consider commercially unattractive. The commercial rationale offered for FAIR, Meta’s AI research lab, is that AI capability constrained products and moderation. Advancing the field could improve those products without requiring exclusive ownership of the resulting ideas.

The respondent recalls rising pressure for open AI releases from roughly 2010 to 2017, followed by a reversal in the language-model era. The objections to closure have three parts: - Scientific isolation: a closed research community can reinforce its own assumptions. - Economic exclusion: paid access can shut out people who cannot afford continued use. - Concentrated control: a small cultural circle can make decisions that poorly serve people elsewhere. On this account, aligning AI with humans requires broad access and feedback from a broad range of people.

A personal hypothesis connects preferences for open or closed systems to formative experiences of institutional trust. People raised around trustworthy institutions may feel more comfortable with closed systems; those accustomed to corruption may favor openness. The respondent’s own position is that dispersing AI power reduces dependence on a few providers, making room for uses and priorities beyond Silicon Valley’s cultural norms.

Distributed usage needs shared feedback

Openness also creates a coordination disadvantage. A vertically integrated provider can connect product usage to model improvement. Open models may run through many unrelated interfaces while their fine-tuning datasets remain limited. The respondent’s diagnosis is that distributed usage was failing to become shared learning: many frontends lacked a way for users’ feedback to return to the ecosystem.

The proposed infrastructure has three responsibilities: - Collect feedback through a shared API. Accept text paired with a positive or negative signal. - Integrate where people already work. Add feedback controls to open frontends and proprietary applications serving open models. - Filter the data. Retain useful signals while removing spam and attempts to plant advertising in future models. Hugging Face is floated as a possible funder, without a commitment. Financing the service, coordinating integrations and sustaining data cleaning are all necessary to close the loop.

The respondent predicts a potentially compounding advantage against OpenAI, though that comparison rests on rumored usage. Google is considered harder to challenge because of its distribution through Android, Chrome, Gmail and Docs. Feedback coverage depends on product distribution: releasing models does not by itself connect their many uses to improvement.

Asked whether this leads toward artificial general intelligence, the respondent questions definitions based on a percentage of world economic output. Electricity illustrates the ambiguity: attributing output to an enabling technology does not provide a clear arrival test. The expected path is continuous—AI enters useful workflows, successive models improve those workflows, and users adopt better models as they emerge.

Open Assistant, LMSys Arena and OpenRouter enter the discussion as related efforts. The respondent praises the LMSys leaderboard’s model comparisons while emphasizing selection and task-coverage bias: visitors to a comparison site expose only some uses. Copilot-style coding assistance and Character AI-style interactions remain outside that coverage. One or two dedicated destinations cannot represent models embedded across hundreds or thousands of products; integrating even the top hundred frontends is proposed as a substantial first step.

Home robots need teachability and reliability

The closing discussion moves beyond text. The respondent forecasts home robotics entering commercialization five to seven years from the interview, with hardware as the main bottleneck. Wednesday afternoons with Lerrel Pinto’s team at NYU are spent working toward a robot that could handle chores such as dishes. Work in several tens of New York City homes had focused on simpler tasks: moving a cup, relocating cloth and opening a microwave. Dishwashing remained an aspiration.

Sample complexity, here the number of demonstrations needed to learn a task, is a user-experience constraint. Teaching a home robot fifty or three hundred times imposes a substantial burden. The team’s focus is two or three demonstrations, with a consequential qualification: fewer demonstrations come with less generalization. Certain initial conditions must hold for the learned task to work.

Hardware means motors, servos and sensors here, rather than the computer running the model. The respondent points to gaps in sensing, strength and dexterity while praising progress at Tesla, 1X and Figure. Another barrier is maintenance: a successful demonstration can conceal an unsustainable repair burden. Commercialization requires reliability sufficient for ordinary retail sale and home use, beyond what works in a controlled industrial setting.

The respondent also describes previous work in FAIR’s small robotics program before shifting Meta work toward infrastructure. In this account, robotics has a lower priority at Meta than Reality Labs’ broader strategy around virtual- and augmented-reality devices.

Smell opens a less crowded direction

Osmo’s work on smell digitization supplies the final example. The respondent explains an investment partly through knowing founder Alex Wiltschko from framework work: his Torch contributions and creation of Google’s Tangent automatic-differentiation framework, alongside his training in neurobiology. The technical attraction is smell’s relative immaturity compared with audio and imagery, which already have established representations such as frequency spectra and color spaces.

The possibilities span different horizons: - Near-term utility: perfume categorization and personalization, improved mosquito repellents and possible cancer detection through smell. These are suggested applications, with no diagnostic performance established in the conversation. - Long-term sensory media: watching a remote experience and receiving its accompanying smell, perhaps through future glasses. This is a speculative vision, not a demonstrated feature. A commercially useful early application could sustain research into the longer-term possibilities.

Asked for a final call for research or startups, the closing response declines to prescribe a direction. It returns to the opening principle: people should find work through their own intrinsic motivation.