DeepSeek

DeepSeek is an AI company founded in Hangzhou in 2023 by Liang Wenfeng, who previously co-founded the quantitative investment firm High-Flyer. Its work spans code and mathematical models, general-purpose language models, reasoning systems and GPU communication software. Across these projects, DeepSeek has addressed distinct engineering problems: how much computation each token activates, how much attention memory a conversation requires, how distributed GPUs exchange work, and how training rewards affect answer usefulness.[2][3][4][7][9][11][12]

DeepSeek released its first large language model in 2023. Its early DeepSeek-Coder family trained on 2 trillion tokens, comprising 87% code and 13% English and Chinese natural language. Dependency-based file ordering and concatenation preserved relationships across repository files, while repository-level deduplication reduced repeated examples. Infilling taught the models to supply missing code between existing passages, alongside ordinary next-token completion.[2][4]

Coder’s initial 1.8-trillion-token training used a 4K context window; another 200 billion tokens extended the window to 16K. In a 1.3-billion-parameter Python experiment, the developers found that an all-infilling configuration improved infilling while weakening ordinary completion, and chose a 50% prefix–suffix–middle infilling rate to balance the two. The Coder paper includes a Peking University affiliation, and the repository separately acknowledges Hugging Face’s support for its demo.[4][5]

The early DeepSeek LLM releases included 7B and 67B base and chat models; the 67B model trained on 2 trillion English and Chinese tokens. One experiment shaped a concrete data decision: adding 20 million Chinese multiple-choice questions improved multiple-choice benchmarks in the 7B setting without corresponding gains on other knowledge evaluations. The developers excluded that data from pretraining and fine-tuning to avoid benchmark overfitting.[6]

DeepSeekMath continued a 7B code model through a 500-billion-token mixed training run, drawing on a collected mathematical corpus of 35.5 million web pages totaling 120 billion tokens. Its paper, with DeepSeek, Tsinghua University and Peking University affiliations, introduced Group Relative Policy Optimization (GRPO). Instead of training a separate critic to estimate a reward baseline, GRPO scores several answers to the same prompt and compares their rewards within the group. Reward scoring remains necessary; replacing the learned critic reduces training resources.[7]

The developers reported 51.7% accuracy on MATH for DeepSeekMath-RL 7B using chain-of-thought reasoning without tools or voting. Their 60.9% result used self-consistency over 64 samples, a different evaluation procedure.[7]

DeepSeek-V2 combined two distinct efficiency mechanisms. Its mixture of experts routed tokens through selected feed-forward experts, using finer-grained specialists alongside shared experts. V2 had 236 billion total parameters, with 21 billion activated per token. Sparse activation reduces per-token computation; it does not reduce the full model’s weight storage to that of a 21B dense model.[8][9]

V2’s Multi-head Latent Attention (MLA) addressed the attention cache retained during generation. Rather than storing full keys and values for preceding tokens, MLA jointly compressed them into a smaller latent representation. Expert routing and attention compression therefore addressed separate expenses: selected computation and retained conversation memory.[9]

On its H800 training cluster, DeepSeek reported 172,800 GPU hours per trillion training tokens for V2, against 300,600 for its dense 67B model—a 42.5% reduction. It also reported generation throughput exceeding 50,000 tokens per second on an eight-H800 node, 5.76 times the comparison model’s maximum. These measurements used the reported training and serving configurations; the throughput result included deployment optimizations and the service’s prompt and generation-length distribution.[9]

In a July 22, 2024 interview with 36Kr, Liang described how a researcher’s proposed attention alternative took several months of team work to become operational. He characterized exploration as bottom-up: researchers brought their own ideas and recruited colleagues to discuss problems, while the organization allocated resources from above when an idea showed promise.[3]

Liang also described API pricing as cost-based, with a small profit above cost, rather than subsidies or excessive margins. He attributed lower costs to architectural exploration. At that time, he said DeepSeek had no short-term fundraising plan and identified restrictions on access to high-end chips, rather than money, as its constraint.[3]

The V3 technical report described 671 billion total parameters and 37 billion activated per token, retaining MLA and sparse experts while addressing several training bottlenecks together. FP8 mixed precision reduced precision in parts of computation; DualPipe overlapped computation with communication; load balancing distributed expert work; and multi-token prediction added an objective for predicting further ahead.[10]

The developers reported 2.788 million H800 GPU hours for V3’s official training, including context extension and post-training. At an assumed rental price of $2 per GPU hour, they calculated $5.576 million. That estimate excludes prior research and ablation experiments; it is neither total company investment nor the cost of developing R1. V3’s post-training already used internally generated R1 reasoning material, so the two research efforts overlapped.[10]

The R1 report distinguished two experiments. R1-Zero applied reinforcement learning to pretrained V3-Base without preliminary supervised fine-tuning; the developers reported reasoning capabilities alongside poor readability and language mixing. R1 added cold-start examples, reasoning reinforcement learning, a mixed supervised dataset and final alignment reinforcement learning. A language-consistency reward improved readability while slightly reducing performance in the reported ablations.[11]

The project also produced six smaller Qwen- and Llama-derived models, directly fine-tuned on 800,000 curated samples. The dataset included approximately 600,000 reasoning samples from an intermediate reinforcement-learning checkpoint and supervised material reused from V3. These models transferred larger-model examples into smaller upstream architectures; their training data was not solely output from final R1.[11]

DeepEP exposes expert-parallel communication as software: kernels dispatch token representations to experts and combine their results afterward. Its deployment requirements include SM90-capable GPUs, NVLink and RDMA networking, making hardware topology part of its practical use. The MIT license applies to this software artifact.[12][13]

DeepEP’s V2 switched from NVSHMEM to NCCL Gin, with acknowledged support from the NCCL team. The repository credits Tencent’s Network Platform Department for a contribution and AntGroup’s Network Platform Department for an optimization series. These are feature-specific contributions, alongside DeepSeek’s own systems work.[12]

V3.1, released August 21, 2025, combined thinking and non-thinking inference in one model. On September 29, V3.2-Exp introduced DeepSeek Sparse Attention (DSA) on V3.1-Terminus to explore training and inference efficiency for long contexts. The experimental release preceded final V3.2, released December 1.[14][15][16]

For final V3.2, DeepSeek described synthesized agent-training data covering more than 1,800 environments and 85,000 complex instructions—training scale, rather than customer adoption.[17]

The April 24, 2026 V4 preview made million-token context the default across official DeepSeek services. Its announced Pro model had 1.6 trillion total parameters and 49 billion active parameters; Flash had 284 billion total and 13 billion active. Token-wise compression combined with DSA extended the earlier work on managing attention memory and long-context computation.[18][15]

On July 17, Caixin reported that DeepSeek had raised approximately 50 billion yuan ($7.4 billion) in its first external funding round, at a post-money valuation above 350 billion yuan. The report cited a disclosure by Anhui Korrun, whose wholly owned subsidiary invested indirectly through a private-equity fund. This marked a change from Liang’s July 2024 statement that there was no short-term fundraising plan.[21][3]

Reuters’s July 16 account reported a fund’s 2.90-billion-yuan investment for an indirect 0.8265% stake, implying a valuation of 350.88 billion yuan, or $51.82 billion. Shanghai Securities News subsequently compared three listed-company disclosures and found differing implied valuations of approximately 325 billion–357.143 billion yuan. These stake-derived values do not establish a single precise company-announced valuation.[22][23]

The September 10 V4.1-Flash announcement added native visual understanding. DeepSeek specified a 552-billion-parameter mixture-of-experts model with a causal encoder–decoder architecture using 8 billion active parameters for input and 16 billion for output. It also announced support through WorkBuddy, including CodeBuddy, and OpenCode.[19]

DeepSeek initially announced that all V4 Pro requests would switch to V4.1-Flash from September 14 until V4.1-Pro launched. Its Models & Pricing documentation, as of September 17, instead stated that V4 Pro API service would continue with unchanged billing, attributing the decision to user demand. The documented offering retained Pro alongside V4.1-Flash.[19][20]

Earlier that year, Anthropic had accused DeepSeek of unauthorized capability extraction. Its February 2026 report alleged more than 150,000 exchanges with Claude, including rubric-based grading tasks that made Claude function as a reinforcement-learning reward model. This allegation concerns access to another provider’s service, distinct from the documented supervised training of DeepSeek’s Qwen- and Llama-derived models.[24][11]

Anthropic’s September 2026 report alleged both reasoning-trace extraction and selective forwarding of customer requests from coding harnesses such as Claude Code, the Claude Agent SDK and OpenCode. Anthropic assessed that sensitive information was likely forwarded without customers’ knowledge or consent, citing requests from an IT operator working with data from a Russian government agency associated with its Ministry of Defense. These remain Anthropic’s allegations and assessments.[25]