Modal

Modal is an AI infrastructure company that runs model inference, parallel batch jobs, training and agent-code environments in the cloud. Its founders are Erik Bernhardsson and Akshat Bubna, identified in May 2026 as CEO and CTO. Modal puts cloud execution close to the program: developers define dependencies, hardware requirements and scaling behavior alongside application code, while the platform manages containers and pooled compute capacity. This approach began with work on a runtime for data teams in 2021.[1][7][8][4]

Before Modal, Bernhardsson built the workflow engine Luigi and vector-search library Annoy at Spotify, then spent six years as CTO of Better. In his 2024 interview, he recalled wanting better tools for data teams that repeatedly built internal Kubernetes wrappers. Workflow scheduling appealed because it connected many parts of the data stack, but developing and monetizing a scheduler alone looked difficult. He turned toward the execution layer beneath it, focusing on the delay between changing code and seeing it run in the cloud.[2]

His April 2021 essay, “Software infrastructure 2.0: a wishlist”, supplies context for that focus. It advocated pooled serverless resources, temporary production-like environments for testing, fast infrastructure changes and client libraries instead of generated configuration. “I don’t want to pay for idle resources,” he wrote. These were proposals: faster, ephemeral infrastructure would move feedback from slow deployment cycles into the immediate loop of writing and running code.[3]

Bernhardsson recalled an early implementation combining the container runner runc with a custom network filesystem. Instead of downloading an entire image before execution, the runtime could fetch files as needed and cache commonly shared contents. The Python SDK addressed a separate problem: placing application and infrastructure definitions together, and making calls between functions in different container environments resemble ordinary function calls.[2]

Generative inference became the first strong use case for this general runtime. Bernhardsson recalled that Modal had little usage or revenue roughly a year and a half into development, around the emergence of Stable Diffusion. Inference's small prompts and responses, separated by substantial GPU computation, suited serverless execution. By February 2024, he described AI as Modal's largest use case, while customers also ran web scraping, video processing, sequence alignment and optimization jobs.[2]

Modal added initial users in 2022, opened a waitlist later that year and became generally available on October 10, 2023. The launch offered Python cloud functions, autoscaling GPU inference, batch jobs on thousands of containers, scheduled tasks, web endpoints and persistent storage defined in Python. In launch testimonials, Substack's Mike Cohen described parallel podcast transcription, Suno's Georg Kucsko described inference and data processing, and Sphinx Bio's Nicholas Larus-Stone described protein-folding jobs.[4]

The introductory example creates an application, defines a container image with Python dependencies and decorates a function with an H100 GPU requirement. The function loads a Qwen model through Transformers; modal run sends the workload to the cloud. Modal packages code into containers, adjusts container counts with traffic and selects capacity across providers. Python is the primary language for implementing applications and Functions; JavaScript/TypeScript and Go clients can invoke Functions, run Sandboxes and manage resources.[8]

Modal 1.0, announced on June 9, 2025 after launching the previous week, changed the balance between convenience and predictability. Earlier clients automatically included imported local packages, sometimes adding unnecessary dependencies or encouraging mistaken assumptions about available files. Version 1.0 made inclusion explicit, apart from a Function's own package, and consolidated filesystem setup under modal.Image. It also separated concurrency controls into their own decorator, reducing the configuration accumulated in the core function decorator.[14]

Training support also broadened. In February 2024, Bernhardsson distinguished bursty fine-tuning—such as Ramp running 100 models in parallel—from large foundation-model training, which Modal then supported poorly because of its networking requirements. By September 2025, the product suite included RDMA-connected training clusters, alongside inference, batch processing, Sandboxes and collaborative GPU notebooks. In July 2026, Bubna described multi-node training's focus as smaller post-training jobs and elastic research runs.[2][5][10]

A launched container is not necessarily a model-ready inference service. Healthy hardware must be available, image contents loaded, the host process initialized and device state prepared. Modal built its own filesystem, runtime, scheduler and image builder to address these steps. Its May 2026 architecture article, by Charles Frye, Jonathon Belotti, Bernhardsson and Bubna, explains four layers:[1][7]

• Healthy GPU buffers keep a small shared pool of idle, checked machines available, moving allocation and health checks ahead of incoming demand.[7]
• Lazy image delivery uses ImageFS, built with libfuse, to load metadata before execution and file contents as needed. A tiered, content-addressed cache reuses identical bytes across images even when their Docker layers differ.[7]
• CPU checkpoint and restore save initialized process state through gVisor's runsc, which emulates a subset of the Linux kernel in userspace. Restoration can skip imports and other host-side setup.[7]
• CUDA checkpoint and restore use NVIDIA driver support to move device state into host memory for checkpointing, then restore it to the GPU. This preserves expensive initialization artifacts such as CUDA graphs.[7]

Snapshotting skips computation but does not eliminate the transfer of model weights into GPU memory. Host compatibility can require multiple snapshots, applications may need adjustments, and multi-GPU communication complicates pausing and restoration. Modal's illustrative comparison of roughly 2,000 seconds to 50 seconds concerns inference-replica startup across these layers. In a separate experiment covering more than 10,000 cold starts of Qwen 3 0.6B in bf16, Modal reported mean vLLM boot latency falling from about 95.7 to 13.8 seconds with snapshots, and SGLang latency from 83.7 to 17.5 seconds.[7]

In 2024, Bernhardsson explained Modal's emphasis on custom models and workflows rather than competition primarily over commodity open-model API prices. By June 2026, Modal had extended its infrastructure work into serving-engine optimization and trained draft models. Its speculative-decoding research, by Charles Frye, David Wang and Shankha Biswas, described Qwen-series DFlash speculators trained with Z Lab and serving improvements developed with SGLang.[2][17]

Speculative decoding lets a small draft model propose tokens that the target model checks in parallel. The number of accepted tokens and the cost of drafting determine the useful speedup. Lossless speculative sampling preserves the target distribution, subject to implementation nondeterminism. DFlash, invented by Jian Chen and collaborators at Z Lab, drafts blocks in parallel using target-model features. Modal contributed trained speculators and engineering integration, including a custom operation and Triton kernel for efficient handling of target-model KV projections.[17][18]

A June 24 case study by Frye and Shankha Biswas describes applying this work to an inference subsystem in Decagon Voice. Modal combined regional serving, reduced host overhead, efficient kernels and task-specific draft-model training. Synthetic completions matching the application's output format supplied training data without exposing user data; additional datasets checked for regressions outside that distribution. Modal reported that custom speculators removed another 100 milliseconds, approximately 40% of server-side decode latency, from a baseline already using speculation.[18]

Auto Endpoints, launched June 23, 2026, packaged serving expertise into self-service LLM deployments through the CLI or dashboard. Endpoints expose serving code, model-specific recipes, autoscaling and engine metrics such as time to first token, inter-token latency and speculative acceptance length. The launch also brought regional Modal Servers out of beta. Modal described human-monitored internal agents generating deployment recipes; automated speculator retraining and distillation remained development directions.[15]

By September 17, 2026, endpoint documentation distinguished two serving modes. Shared Endpoints offer selected library models from Modal-managed capacity and bill per token. Dedicated Endpoints support library models and custom weights, expose adaptable generated source, and bill for compute resources with configurable autoscaling, including scale-to-zero. These modes extend Modal's earlier per-second serverless compute offering.[16][4]

Modal's agent runtime predates that managed-inference launch. In his July 2026 interview, Bubna recalled building Sandboxes in May 2023 after customers requested environments for generated code. The first example put swyx's smol developer into a loop that could iterate on its own code, although Bubna recalled models diverging after about ten iterations. Modal subsequently expanded Sandboxes into infrastructure for production agents and reinforcement-learning environments.[10][6]

Sandboxes provide a direct interface for defining containers at runtime, executing commands, reading outputs and terminating environments. Agents can check out repositories, install dependencies and run tests. Creation, scheduling, startup and optional readiness are distinct lifecycle stages. The documented default maximum lifetime is five minutes, configurable up to 24 hours; longer continuity uses filesystem snapshots restored into subsequent Sandboxes.[9]

Bubna described redirecting the SDK team toward agent experience, using Modal Bench to identify tasks agents struggled with and moving logs and metrics from the dashboard into the CLI. He also described sidecar support and private container networking, originally built for distributed training and subsequently used for other applications. His emphasis remained specialized, elastic compute, with hard sandbox boundaries and customer control over persistence, networking and the agent harness.[10]

Two earlier team additions brought systems and stateful-execution expertise. In November 2024, Tidbyt joined Modal. Its programmable LED-display business had shipped 100,000 devices, but Modal emphasized founders Rohan Singh and Mats Linander's previous work on Spotify container infrastructure and their business experience. The announcement said service for existing devices would continue while new device shipments would stop.[12]

Modal announced its acquisition of Jamsocket in July 2025, with founders Paul Butler and Taylor Baldwin joining. Jamsocket's products included the open-source orchestrator Plane, realtime document backend Y-Sweet and ForeverVM's stateful Python execution. Both teams used gVisor and had exchanged technical notes before the acquisition. Modal planned to migrate core functionality onto its platform while Jamsocket products continued operating.[11]

Modal's October 2023 launch announced a $16 million Series A led by Redpoint, identifying Amplify Partners as its seed-round lead. On September 29, 2025, it announced an $87 million Series B led by Lux Capital, a $1.1 billion post-money valuation and $111 million in total financing. At that point, Modal reported thousands of customers and tens of thousands of simultaneous Sandbox containers for customers such as Lovable.[4][5]

On May 21, 2026, Modal announced $355 million raised in its Series C, co-led by General Catalyst and Redpoint, with a final $4.65 billion post-money valuation. Menlo, Bain Capital Ventures and Accel joined as new investors. Reuters reported that the financing comprised two tranches: the first at a $2.5 billion valuation and the second at $4.65 billion.[6][13]

In the May announcement, Modal reported annualized revenue exceeding $300 million, fivefold growth since September, a team exceeding 120 across New York, San Francisco and Stockholm, and more than one billion cumulative Sandbox launches. Sandboxes contributed over one-third of revenue. The revenue figure was an annualized run rate; the launch count measured cumulative environments. Capacity for millions of Sandboxes in parallel was an expansion goal.[6]

Modal sourced compute through external providers while concentrating on its software layer. Reuters' May account described relationships with 13 cloud companies; Bubna's July-published interview described a pool spanning 17 providers and said Modal did not operate its own data centers. He explained the capital-light choice as a way to move quickly, while a compute-strategy team managed reservations, regional capacity and forecasts. Modal's May priorities included lower-latency inference, tighter integration of training and serving, and expanded agent runtimes.[13][10][6]