Cerebras

Cerebras is an AI computing company whose Wafer-Scale Engine (WSE) combines many small compute cores, local memory, and communication across a single silicon wafer. It builds complete Cerebras Systems (CS) computers around these processors and supplies both on-premises hardware and cloud services. Its products have evolved from accelerating neural-network training to serving interactive inference; in 2026, OpenAI committed to purchasing 750MW of inference capacity, and Cerebras became a public company.[1][6][11][12][3]

The hardware behind these inference services originated in Cerebras’s earlier work on shortening training runs, when long experiments constrained how many ideas researchers could test.[4][11]

Andrew Feldman, Gary Lauterbach, Michael James, Sean Lie, and Jean-Philippe Fricker founded Cerebras in 2015 to bring wafer-scale computing to market. Cerebras Systems Inc. was incorporated in Delaware in April 2016. Feldman, previously CEO of SeaMicro before its acquisition by AMD, has served as Cerebras’s CEO, president, and a director since April 2016; he also chairs its board.[1][16][2]

The founders focused on workloads requiring rapid data movement and memory close to computation. Feldman described the purpose in an IEEE Spectrum account of the first system: “The idea is to test more ideas,” by shortening training experiments. Cerebras introduced WSE-1 and CS-1 in 2019. WSE names the processor; CS names the computer built around it.[4][1]

Conventional manufacturing divides a wafer into individual chips. Cerebras worked with Taiwan Semiconductor Manufacturing Company (TSMC) to build interconnects across the scribe lines where chips would ordinarily be separated. Its small cores, nearby memory, and on-silicon communication mesh let regions exchange results without sending that traffic between separate chips. Manufacturing equipment and electronic design software also required adaptations for the unusually large processor.[4][6]

The whole wafer need not be flawless for the computer to work. Small cores limit how much silicon a manufacturing defect disables; spare cores replace defective units, and redundant communication paths permit routing around them. Cerebras’s defect-tolerance explanation describes WSE-3 as having 970,000 physical cores, with 900,000 active in the shipping product discussed there. Its approximately hundredfold comparison with an H100 concerns silicon area affected per defect, rather than a measured hundredfold improvement in manufacturing yield.[5]

CS-1 also required vertical power delivery: feeding power from the wafer’s edge would lose too much voltage before reaching its center. Cerebras supplied power from above through a million copper posts, with a water-cooled plate below the wafer. Lauterbach judged delivering the required current to be “much harder than cooling it or addressing the yield problems”. Different thermal expansion rates in the chip, circuit board, and cooling assembly created another problem; the team spent a year and a half developing an intermediate material to keep the connections intact.[4]

Cerebras introduced WSE-2 and CS-2 in 2021, but rising parameter counts challenged even a wafer-sized processor’s memory capacity. Feldman told IEEE Spectrum that “we probably underestimated how fast the world would get to very large parameter sizes.” Cerebras began adding engineering resources to weight streaming at the start of 2019. Co-founder Michael James explained the resulting architecture in September 2021.[1][8][6]

Weight streaming separates parameter storage from computation. A white paper by Stewart Hall, Rob Schreiber, and Sean Lie describes MemoryX, which stores model weights and optimizer state, streams weights to the wafers, and receives gradients to update them. The wafers ordinarily retain activations—the intermediate values produced as training examples pass through the network—while layers execute in sequence. Activation checkpointing can evict activations to external workers and refill them when needed.[7][18]

SwarmX broadcasts weights and combines returning gradients across systems. Each wafer processes a different shard of the training batch using the same model, providing data-parallel training without requiring the user to divide model layers between systems. The architecture also exploits unstructured sparsity: zero-valued weights are omitted from transmission and the corresponding multiplications are skipped.[7][8]

Cerebras engineers worked with Arvind Ramanathan’s lab at Argonne National Laboratory on genome-scale language models, or GenSLMs, for studying SARS-CoV-2 evolutionary dynamics. Cerebras reported training models from 123 million to 25 billion parameters with sequence lengths of 10,240, using a single CS-2 and clusters of up to 16 systems to accelerate training. The joint work with Argonne and NVIDIA received the 2022 ACM Gordon Bell Special Prize for HPC-Based COVID-19 Research.[7]

Cerebras-GPT, dated March 2023, comprised seven Apache 2.0 language models ranging from 111 million to 13 billion parameters, with weights available on Hugging Face and code in the Cerebras Modelzoo. The models were trained on the Pile using a 16-system CS-2 cluster named Andromeda. Following DeepMind’s Chinchilla methodology, the study examined roughly 20 training tokens per parameter as a compute-efficient pretraining regime.[9][18]ⁿ¹

A separate experiment used maximal update parameterization (µP) to transfer settings tuned on a 40-million-parameter proxy to models through 2.7 billion parameters. µP adjusts initialization, layer learning rates, and activation magnitudes as network width changes, making small-model tuning useful at larger scales. The researchers reported more predictable scaling and improved results relative to standard parameterization. These experiments were distinct from the initial seven-model release through 13 billion parameters, which used standard parameterization.[18]

The June 2023 SlimPajama release began with a change of plan. The team intended to use the upstream RedPajama dataset unchanged, then found missing files and extensive duplication. Cleaning and deduplication reduced the corpus from 1.21 trillion to 627 billion tokens. The release included distributed MinHashLSH preprocessing tools, which identified similar documents both within and between data sources.[10]

Cerebras contributors Daria Soboleva, Faisal Al-Khateeb, Joel Hestness, and Nolan Dey worked with Opentensor’s Robert Myers and Jacob Robert Steeves. They released the dataset before the planned training of a LLaMA-style model for Opentensor, alongside Apache 2.0 data and preprocessing tools. Cerebras reported that preprocessing took 2.5 days on a 64-core CPU, with peak memory consumption of 1.4TB.[10]

Cerebras also introduced the Condor Galaxy supercomputer network with G42 in 2023. The company described its first system, CG-1, as having 54 million cores and 4 exaFLOPs of FP16 performance.[1]

Cerebras introduced WSE-3 and CS-3 in 2024 and launched its public inference service on August 27 with Llama 3.1 8B and 70B. The API used the OpenAI Chat Completions format, and the service ran Meta’s original 16-bit model weights. Cerebras reported launch generation rates of 1,800 tokens per second for 8B and 450 for 70B.[1][11]

This serving design differs from training weight streaming: model weights reside in on-chip SRAM, with models exceeding one wafer’s capacity split across systems at layer boundaries. The launch configuration had 44GB of SRAM per WSE-3 and placed the 70B model on as few as four CS-3 systems. Keeping weights near computation reduces repeated data movement during autoregressive decoding, which generates successive tokens using the preceding context.[11]

Under a December 2025 agreement announced in January 2026, OpenAI committed to purchase 750MW of inference capacity, with deployment expected in tranches through 2028. It also received an option for another 1.25GW by the end of 2030. OpenAI supplied approximately $1 billion in working-capital financing in January and exercised a warrant for 10,033,508 Class N shares in July, making it a customer, financier, and shareholder. The 750MW commitment is phased capacity, rather than an assertion that all of it was already installed.[16][12]

OpenAI’s February 12 GPT-5.3-Codex-Spark research preview ran on WSE-3 as a latency-focused serving tier. OpenAI reported more than 1,000 tokens per second, with a text-only 128k context window at launch. It described Cerebras as complementary to the GPUs used across its training and inference pipelines, including the possibility of combining them within a workload.[13]

On August 13, OpenAI announced GPT-5.6 Sol Ultrafast, initially a limited API preview for selected customers. OpenAI reported up to 750 output tokens per second and up to 14 times Standard processing speed. These are customer-reported upper bounds, rather than equivalent speedups for an entire application; OpenAI’s financing and shareholding also give it a financial interest in Cerebras.[17][16]

Cerebras announced CS-4 on August 18, 2026. It combines three WSE-3 Turbo processors on the modular Nexus platform. Its rear-mounted wafer assemblies integrate power conversion, direct liquid cooling, I/O, and control electronics; connectivity includes Ethernet RDMA and direct links between wafers. Cerebras scheduled the first shipments for the third quarter of 2026.[15]

CS-4 supports disaggregated inference, separating prompt processing from token generation. A complementary platform performs prefill, processing the prompt and preparing model state, then transfers that state to Cerebras for decode. Cerebras named AMD Helios and AWS Trainium as complementary prefill platforms. Earlier in 2026, it announced an AWS partnership planning to pair Trainium 3 prefill with CS-3 decode.[15][14]

Cerebras claimed up to 30 times faster inference than GPU systems across the models shown in its August announcement, drawing on Artificial Analysis and internal benchmarks. Its claim of more than 1,000 tokens per second for models exceeding 10 trillion parameters was explicitly an extrapolation from internal benchmarking, rather than a reported run of such a model.[15]

Cerebras closed approximately $1 billion in Series H financing in February 2026 and a revolving credit facility of up to $850 million in April. Its Class A shares began trading on Nasdaq under CBRS on May 14. The IPO closed on May 15, selling 34.5 million shares at $185 each for approximately $6.38 billion in gross proceeds; the June-quarter filing reported approximately $6.2 billion in net proceeds after underwriting discounts, commissions, and estimated offering expenses.[14][3][16]

For the quarter ended March 31, 2026, Cerebras reported GAAP revenue of $193.4 million: $110.6 million from hardware and $82.8 million from cloud and other services. It reported a GAAP net loss of $14.0 million. Under the OpenAI agreement, Cerebras subsequently recognized $56.8 million in revenue in the June quarter and $74.4 million in the first half of 2026, net of customer-warrant amortization.[14][16]