Lin Qiao

Lin Qiao is the co-founder and CEO of Fireworks AI and previously headed PyTorch at Meta. Her work centers on making AI models practical for application developers: simplifying access while managing the infrastructure and workload-specific trade-offs among quality, latency and cost.[3][2]

Qiao co-founded Fireworks in 2022. In her November 2024 Latent Space interview, she recalled an initial plan for a PyTorch cloud, followed by customer conversations that led the team to focus on generative AI. They prioritized inference over training because, in her reasoning, inference demand grows with the population using applications, while training demand grows with the researchers building models. The team also chose OpenAI-compatible APIs early to make adoption easier for application developers and product engineers.[4][2]

That focus evolved from a shared inference endpoint toward customization for individual workloads. Qiao explained that customers’ inputs differ from a model’s training distribution, leaving opportunities to improve quality, latency and cost. She described Fire Optimizer as taking an inference workload and an objective function and producing a model setup and deployment configuration. The underlying engineering, performed by the Fireworks team, includes custom CUDA kernels, distributed execution, caching and quantization of components such as weights, the KV cache and communication between GPUs.[2][4]

The collaboration with Cursor gave this approach a concrete test. Qiao described Cursor as a technically capable team that deliberately sought an infrastructure partner so it could concentrate on its product. Its demands for interactive latency and high quality pushed Fireworks engineers to expand features and scale across regions. According to Qiao, workload-specific optimization was deployed for Cursor before becoming a separate product line. Speculative decoding was one technique within that work; the choice among a small-and-large-model pair, Eagle heads or Medusa heads depended on the workload.[2]

Qiao treats the resulting trade-offs as product decisions. During experimentation, she advises developers to establish whether a model delivers the intended experience; after product-market fit, quality, speed and cost require closer optimization. A lower-quality configuration can be appropriate if it improves speed and cost without harming that experience. As she put it, “The validation doesn't stop at offline eval.” She includes live inference and A/B testing in the evaluation of model versions and serving configurations.[2]

Qiao argues that a useful AI application often needs more than one model. Models have different specialties, can hallucinate and lack information outside their training data, including current information and private enterprise knowledge. Her compound-AI approach combines specialized models across modalities with public and private APIs, databases and other knowledge sources. Function calling supplies a routing mechanism: a model can select APIs that retrieve information or perform parts of a task.[2][4]

She favors declarative systems, where developers specify the result they want and the system handles execution. Her analogy is SQL: a user requests an answer without choosing execution nodes, indexes or a query plan. She recognizes value in imperative systems, where developers explicitly arrange steps and recovery behavior, and does not expect either approach to subsume the other. Her preference is grounded in ease of integration: “Focus on ease of use, and then let the system take on the hard challenges and complexities.”[2]

Qiao nevertheless leaves product behavior to the application’s designers. In her Training Data interview, she used an autocomplete feature inside a table cell as an example: should it extend the cell’s text, generate more rows or do nothing? Such an apparent model failure can be an unresolved design decision. Her proposed separation automates technical work such as collecting and labeling data and selecting tuning methods, while keeping decisions about how the product should respond with its developers and product managers.[4]

Her support for open models rests partly on a prediction that open and closed models of comparable sizes will converge in quality, making customization more consequential. She expects an ecosystem of specialists to excel at narrow tasks. These are forecasts, rather than established results. She also cautions against assuming model development has plateaued: at Meta, she recalled, apparently stable recommendation-model designs were later changed by significant innovations.[4][2]

Before Fireworks, Qiao led PyTorch at Meta. Her account of that earlier work explains the production-engineering experience behind her later emphasis on simple interfaces and automated infrastructure.[3][4]

Qiao recalled consolidating teams responsible for Meta’s research, server and mobile frameworks. An attempt to combine the PyTorch frontend with the Caffe2 backend proved harder than rebuilding: the frameworks had not been designed to work together. The team instead retained PyTorch’s frontend and rebuilt the backend, including TorchScript. Her formulation of the principle was succinct: “Simplicity. Simplicity scales.”[4]

What the team initially expected to be a six-month library migration became, in Qiao’s recollection, a five-year rebuilding of Meta’s training and inference stack. It required efficient data loading, distributed inference and scalable training, while supporting production concerns such as stability and low latency. She reported that the stack sustained more than five trillion inferences per day when the team left. That figure describes Meta’s infrastructure; her subsequent ambition for Fireworks was to shorten other organizations’ time to production from years to weeks or days.[4][2]

Qiao also co-authored the 2021 paper First-Generation Inference Accelerator Deployment at Facebook, alongside Michael Anderson, Benny Chen and a large collaborative team. The paper describes the co-design and production deployment of an inference-accelerator platform, including hardware, software tooling and optimizations, with support for accelerators from multiple vendors.[5]