Baseten

Baseten is an AI infrastructure company focused on running models in production under real traffic. Its platform provides hosted model APIs, dedicated deployments for custom models, and training infrastructure. It manages model containers, GPU capacity across clouds and regions, scaling, and observability, while maintaining inference engines for supported architectures. Its researchers and engineers also work alongside customers to post-train and optimize specialized models.[3][9]

The current inference and training platform followed an earlier, broader application-building toolkit. Baseten was founded at the end of 2019 by Tuhin Srivastava, Amir Haghighat, Phil Howes, and Pankaj Gupta. Its September 2026 leadership page identifies Srivastava as CEO, Haghighat as CTO, and Howes as Chief Scientist.[1][2]

In his April 2022 founder account, Srivastava recalled deployment, Docker, Kubernetes, backend integration, and user interfaces as obstacles between trained models and usable software. Baseten’s initial answer combined model serving, serverless backend APIs, and a drag-and-drop interface builder for model evaluation, data labeling, and internal tools. The company quietly announced the product in May 2021 after more than 18 months of development. Before launching its public beta in 2022, the team had, in Srivastava’s words, “painfully rethought and rewrote core abstractions.”[2]

• Model APIs expose hosted endpoints for a curated set of models. Baseten manages the model, engine, hardware, and scaling. Its September 2026 documentation lists support for OpenAI Chat Completions and, in beta, Anthropic Messages.[3]
• Dedicated inference lets teams deploy open-source models, private checkpoints, or custom models through configuration files, Python model classes, or Docker servers. Teams control hardware, autoscaling, and releases, with development, staging, and production environments. Deployments include logs, metrics, and request traces.[3]
• Training has two paths: Loops supplies dedicated trainers and samplers for supervised fine-tuning and reinforcement learning on supported base models; Training Jobs runs customer frameworks and training code in containers on dedicated GPU clusters. Both produce checkpoints that can be deployed to Baseten’s inference stack.[3]

Baseten maintains different engines for different workloads. Engine-Builder-LLM compiles dense text-generation models with TensorRT-LLM; BIS-LLM serves large mixture-of-experts models with distributed inference and KV-aware routing; BEI serves embedding, reranking, and classification models. Teams can also run servers such as vLLM or SGLang in custom containers.[3]

Truss, Baseten’s deployment CLI, handles containerization, dependencies, and GPU configuration. Its Qwen 2.5 3B Instruct example specifies model weights, FP8 quantization, an 8,192-token context limit, and TensorRT-LLM compilation in a configuration file. It assigns two GPUs to the build phase and an L4 GPU to serving, because compilation has different memory requirements from inference. A truss push initiates the build and deployment; the example then calls the model through an OpenAI-compatible endpoint.[4]

Baseten extended model deployment into multi-component workflows with Chains. In their July 2024 explanation, Marius Killinger and Rachel Rapp described customers coordinating models with cumbersome scripts, struggling to test locally, and paying for GPUs to perform CPU work. Chains composes typed services called Chainlets, each with its own container, hardware, and autoscaling. Chainlets declare their dependencies and call one another directly through ordinary control flow, with one designated entry point serving the client-facing API.[5]

At deployment, Chains generates request and response schemas, remote-call stubs, and FastAPI servers. The stubs handle serialization, authorization, and retries while retaining typed Python interfaces during development. Local testing can substitute fake implementations for components requiring unavailable accelerator hardware.[5]

The long-audio transcription example makes the resource separation concrete. An entry-point service divides a request into five-minute segments; CPU services download those ranges and split them into smaller, 30-second chunks for GPU transcription. Each service processes requests and scales separately, then returns results to its caller. Downloading and chunking therefore do not occupy the GPU assigned to transcription.[5]

Baseten built Multi-cloud Capacity Management (MCM) early in 2024, with more than six months of dedicated infrastructure engineering work. Its architecture account describes replacing isolated GPU pools and manual migrations with a global control plane above local Kubernetes clusters. Workload planes report capacity, utilization, and traffic demand; the scheduler chooses placement and delegates execution locally. Placement considers geography, preferred regions, and customer constraints, including enforced region and provider locks.[6]

In an August 2026 interview, Philip Kiely explained how routing interacts with long requests. Processing the input—prefill—creates a KV cache, the attention state used during subsequent token generation. Cache-aware routing seeks available workers that already hold reusable input state. On certain models, Baseten separates the GPUs processing input from those generating output, or decode, and transfers the cached state between them.[8]

Kiely and Ali Taha also described speculative decoding, in which a smaller draft model proposes tokens for the target model to verify. Its benefit depends on the traffic and acceptance rate: a coding-oriented draft may help coding requests more than unrelated workloads. Kiely described production support for new models as including quantization calibration, training speculators, infrastructure setup, and testing. For public Model APIs, he said Baseten trains general speculators using public datasets representative of popular workloads.[8]

Baseten’s Still research addresses the memory occupied by KV caches. The preprint submitted June 5, 2026, also published by Base Labs, is by Charles O’Neill, Alex Sandomirsky, Harry Partridge, Mudith Jayasekara, and Max Kirkby. Still uses a small per-layer Perceiver trained against a frozen base model to synthesize compact keys and values in a single forward pass.[17][18]

The authors reported Qwen and Gemma experiments spanning 8×–200× compression and 8k–128k context lengths, with 8–22-point gains over the strongest baseline on their long-context RULER evaluation grid. These results describe the evaluated settings, rather than a guarantee of lossless compression for arbitrary models or contexts.[17]

Taha described another internal engineering experiment in the August interview: Baseten used GLM-5.2 to help optimize kernels serving GLM-5.2. The model analyzed profiling traces, identified bottlenecks in SGLang, and wrote replacement kernels before another profiling pass. Taha said some kernels in Baseten’s inference engine came from this process, while noting that the model still had limitations in decision-making and could try to reward-hack its task.[8]

By its April 2022 Series A announcement, Baseten said it had raised a little over $20 million across seed and Series A. Sarah Guo at Greylock led Series A and had co-led the 2019 seed. The subsequent Series B announcement reported an additional $40 million, led by IVP and Spark. On February 19, 2025, Baseten announced the closing of a $75 million Series C, co-led by IVP and Spark, bringing its stated total funding to $135 million.[2][13][14]

Baseten’s Series D announcement, updated September 5, 2025, reported $150 million led by BOND, with Jay Simons joining the board. Its Series E announcement, updated February 5, 2026, reported $300 million at a $5 billion valuation, led by IVP and CapitalG, with NVIDIA among the participants.[15][16]

On June 22, 2026, Baseten announced a $1.5 billion Series F. Its financing release specifies investments across two tranches at $13 billion and $11 billion valuations, without allocating amounts between them. The company announcement names Altimeter, Conviction, and Spark as leads and Sands and Wellington as co-leads. Baseten said the financing would support compute, software, and talent, including embedded researchers and engineers adapting specialized models for customers.[12][9]

In the June release, Baseten reported more than one billion inference calls per day across 87 clusters and 18 clouds, approximately 20× year-over-year revenue growth, and more than $2 billion raised to date. Its Series F blog separately reported 40× inference-volume growth over the preceding year and named customers including Cursor, Notion, Lovable, Harvey, HubSpot, OpenEvidence, Abridge, Decagon, and Parallel. The growth figures are relative measures; they do not establish absolute revenue or a token count.[12][9]

On September 10, 2026, Baseten announced its acquisition of Blaxel. The joint announcement by Haghighat, Srivastava, and Paul Sinai described complementary infrastructure: Baseten supplied inference and training, while Blaxel supplied isolated microVM sandboxes, persistent Agent Drive storage, and connectivity for agents running code, calling tools, and carrying state across tasks. The teams described colocating these components as a way to reduce network latency repeated across agent loops.[11]

Blaxel continued operating with its existing product and support team. Baseten said it would add products based on Blaxel’s primitives over time, starting with Sandboxes. The acquisition was completed; the integrated platform bringing agent execution, inference, training, and shared storage together remained the announced direction.[11]