Fireworks

Fireworks AI provides infrastructure for serving, customizing and training AI models. Led by co-founder and CEO Lin Qiao, formerly Head of PyTorch at Meta, it began with an inference focus and expanded into a platform connecting model training, evaluation and production serving. On August 31, 2026, it made its Training API and Fireworks Lab generally available, giving teams both control over their own training loops and access to embedded researchers and engineers.[3][2][10]

That training platform followed several years of inference engineering. Fireworks’ earlier history begins with a PyTorch-cloud proposal in 2022; its founders’ work on production AI at Meta and Google supplies context for the infrastructure they subsequently built.[2][3]

Fireworks’ founding team comprises Qiao, Benny Chen, Chenyu Zhao, Dmytro Dzhulgakov, Dmytro Ivchenko, James Reed and Pawel Garbacki. Their prior roles included Meta ads infrastructure, Google Vertex AI, PyTorch core maintenance, ranking and compiler work, and Meta Newsfeed machine learning. Before Fireworks, Qiao led work to make PyTorch support production requirements alongside research, including stability, reliability and low latency.[3][2]

In her November 2024 interview, Qiao recalled starting Fireworks in September 2022 with a plan to build a PyTorch-first cloud. Customer discussions led the team to choose generative AI as its initial vertical rather than build a broad framework platform. It then concentrated on inference: Qiao expected applications generating human-consumable content to create demand beyond the population of researchers training models. The public platform launched in August 2023 as a distributed inference engine with multiple models and an OpenAI-compatible API, designed to let application developers consume models without managing the serving stack.[2]

Qiao described the inference engine as distributing pieces of a model across GPUs according to their bottlenecks, rather than scaling the entire model uniformly. By November 2024, she said Fireworks served workloads in North America, EMEA and Asia, using regional affinity and global load balancing. It also matched heterogeneous hardware to workload characteristics such as context length and generation length.[2]

Fire Optimizer added workload-specific tuning across quality, latency and cost. Qiao explained that customers supplied inference workloads and an objective function, and the optimizer produced a model setup and deployment configuration. She traced its early deployment to work with Cursor, whose demands for interactive speed and quality prompted additional features and rapid expansion across regions. Speculative decoding was one technique within that system, with small-model pairs or specialized prediction heads selected and aligned to the workload. In July 2024, Fireworks reported that Cursor used its custom Llama 3-70B model at 1,000 tokens per second for code-generation tasks including instant apply, smart rewrites and cursor prediction.[2][4]

Customization also depended on multi-LoRA serving: many adapters shared a single base model, reducing memory requirements and spreading the dominant base-model serving cost across workloads. In the 2024 interview, Qiao reported that one base model could sustain hundreds to a thousand adapters, allowing uploaded LoRA adapters to receive their own endpoints at the same token price as the base model.[2]

Fireworks’ compound AI approach combined models with retrievers, external tools, data and knowledge sources. Its July 2024 announcement credited the approach to Matei Zaharia and collaborators at Berkeley Artificial Intelligence Research. Fireworks supplied FireFunction V2, an open-weight function-calling model that could coordinate models and APIs while retaining multi-turn chat capabilities. The company described Superhuman’s Ask AI as a concrete example: it used function calling to interact with search and calendar tools and answer questions from an inbox.[4]

FireAttention V2, published in June 2024, added Hopper FP16 and FP8 prefill support and multi-host deployment. Its long-context experiments separated model quality from serving performance, testing retrieval, variable tracking and question answering as well as speed. Performance comparisons used Qwen2 72B Instruct in FP16 and Mixtral 8x7B Instruct in FP8 against vLLM 0.5.0, with speculative decoding and prompt caching disabled. The reported gains varied by precision and workload; the headline’s 12× improvement was not a general speed multiplier. For long-generation tests, Fireworks doubled measured vLLM requests per second to approximate two-host capacity.[7]

FireAttention V3, published in October 2024, documented Fireworks’ use of AMD MI300 GPUs in production. PyTorch’s ROCm support and the hipify utility helped achieve functional parity, but performance required rewriting the attention kernel. AMD’s matrix-core layouts, shared-memory sizes and data arrangements demanded different design choices from NVIDIA hardware. Fireworks reported favorable results for memory-bandwidth-intensive workloads while retaining limitations for compute-intensive and mixture-of-experts workloads.[8]

FireAttention V4, published in May 2025, targeted NVFP4 on NVIDIA Blackwell. Fireworks revamped matrix-multiplication, grouped matrix-multiplication and attention kernels to use new TensorCore instructions because its architecture-specific Hopper operations were incompatible with Blackwell. Performance tests used DeepSeek V3 0324 on eight GPUs with NVLink and disabled MTP speculation. Comparisons with SGLang on H200 combined changes in hardware, precision and software; Fireworks also stated that it could not fully validate TRT-LLM’s B200 FP4 performance.[9]

V4 also explored quantization-aware training to recover quality lost at lower precision. In a text-to-SQL experiment, Fireworks trained the smaller DeepSeek V2 Lite Chat model on 10,000 rows and evaluated it on 1,000 rows. It reported that FP8 and FP4 quantization-aware training converged to the same final evaluation loss with more training data—a result bounded to that proxy experiment.[9]

By October 2025, Fireworks framed its platform around product-model co-design: application interactions, corrections and evaluations fed subsequent model improvements through fine-tuning and reinforcement learning. It announced plans to deepen tuning and inference research and extend its toolchain into model lifecycle management.[5]

The August 2026 Training API connected customer-controlled Python loops to managed trainers and rollout deployments. Customers controlled data, environments and loss or reward functions; Fireworks handled compute, weight synchronization, recovery from failed weight swaps and training–rollout alignment. Serverless training offered shared LoRA infrastructure with per-token billing. Dedicated training supported LoRA and full-parameter updates with per-GPU-hour billing, requiring Fireworks Lab consultation and provisioning. Managed Training supplied supervised fine-tuning, reinforcement fine-tuning and preference optimization workflows, while Fireworks Lab embedded researchers and engineers with customer teams.[10]

The training system addressed numerical consistency between the trainer and the inference engine generating rollouts. Fireworks aligned numerical formats and kernel implementations, used Router Replay to preserve mixture-of-experts selections between rollout and backward passes, and applied batch-invariant kernels and deterministic reductions for supported frontier MoE models. It checked alignment by running the same sequences through both paths and measuring train-inference KL divergence.[10]

For efficiency, Fireworks supported asynchronous reinforcement learning that overlapped rollout generation with training, with bounded weight staleness when the algorithm permitted it. Updated weights were hot-loaded into running rollout deployments. Full-parameter checkpoint transmission used XOR differences between successive weights, compressed with zstd, to reduce the payload. Fireworks reported experience operating reinforcement learning across more than 10,000 GPUs.[10]

The training announcement described Harvey post-training Kimi K3 for long-horizon legal work, Vercel combining reinforcement fine-tuning and speculative decoding for v0’s auto-fixer, and Factory training two LoRA adapters on an open Qwen base for secrets detection. Heidi Health CTO Yu Liu said Fireworks helped move its clinical scribe from closed models to fine-tuned open models, from proof of concept to production in four weeks.[10]

On July 11, 2024, Fireworks announced a $52 million Series B led by Sequoia Capital at a $552 million valuation, with NVIDIA, AMD and MongoDB Ventures participating. Total capital raised reached $77 million. At that time, the company reported more than 100 models, 140 billion tokens processed daily and 99.99% API uptime.[4]

On October 28, 2025, it announced a $250 million Series C at a $4 billion valuation, co-led by Lightspeed Venture Partners, Index Ventures and Evantic, with continued Sequoia support. The round included primary and secondary funding and brought reported total funding above $327 million. Fireworks reported serving more than 10,000 companies and hundreds of thousands of developers, processing more than 10 trillion tokens daily and exceeding $280 million in annualized revenue.[5]

In May 2026, Latent Space described financing talks at a reported $15 billion value as tentative. On July 15, Fireworks announced a $1.505 billion Series D at a $17.5 billion valuation, led by Atreides Management, Index Ventures and TCV. The company reported exceeding $1 billion in annualized revenue run rate and serving more than 40 trillion tokens daily, with more than 95% of those tokens coming from models specialized on customers’ proprietary data for particular jobs.[1][6]