Qwen team

The Qwen team develops Alibaba’s Qwen foundation-model family, identified by its official organization profile as built by Alibaba Cloud. Its work spans language, vision and audio models, coding and mathematics specialists, and tools for agents and model interpretability. Open-weight releases coexist with hosted services: for example, Qwen3.5-397B-A17B provides downloadable weights, while its corresponding Qwen3.5-Plus service adds built-in tools and a default one-million-token context.[3][6][10]

The team’s later multimodal and agent work followed an earlier program of pretrained language models, chat post-training and specialized branches. That history begins with Alibaba’s first open-weight Qwen releases in August 2023.[5][6]

Alibaba published Qwen and Qwen Chat in August 2023, making their code, weights and documentation accessible through Hugging Face and ModelScope. The team’s early release table dates Qwen-7B to August 3, Qwen-14B to September 25, and Qwen-1.8B and Qwen-72B to November 30, 2023.[5][6]

The early product structure separated Qwen, the pretrained base language model, from Qwen-Chat, post-trained with supervised fine-tuning and reinforcement learning from human feedback. Code-Qwen and Math-Qwen specialized in coding and mathematics; Qwen-VL and Qwen-Audio extended the work to images and audio through modality alignment. The team reported training the first-generation language models on 2–3 trillion tokens. For tool use, it labeled ReAct-format examples connecting thoughts, actions and observations, and supported function calling, a code interpreter and Hugging Face agents. Its AgentFabric project let users configure a specialized agent by chatting with the model.[6]

Qwen2, announced June 7, 2024, offered five sizes from 0.5B to 72B, including a mixture-of-experts model with 57B total and 14B activated parameters. The team applied grouped-query attention across all five sizes to improve inference speed and memory use, and included training data in 27 languages beyond English and Chinese. It reported 128K-token context handling for Qwen2-7B-Instruct and Qwen2-72B-Instruct with YaRN augmentation, assessed through the Needle in a Haystack information-retrieval task.[7]

Qwen2 also marked a consequential licensing change: the 0.5B, 1.5B, 7B and 57B-A14B models adopted Apache 2.0, while Qwen2-72B and its instruction-tuned variants retained the Qianwen License. The team said greater openness could accelerate applications and commercial use.[7]

The September 19, 2024 Qwen2.5 release combined dense general-purpose language models with Qwen2.5-Coder and Qwen2.5-Math, alongside hosted Qwen-Plus and Qwen-Turbo APIs through Model Studio. Coder initially offered 1.5B and 7B models, with 32B still forthcoming in the announcement. Apache 2.0 licensing had stated exceptions for the 3B and 72B variants. The team described the general models’ pretraining dataset as encompassing up to 18 trillion tokens and reported training Coder on 5.5 trillion tokens of code-related data.[8]

Qwen’s release accounts credited external contributors across fine-tuning, quantization, serving and local inference, including Axolotl, Llama-Factory, AutoGPTQ, vLLM, SGLang, MLX, llama.cpp and Ollama. Qwen2.5 made a specific integration choice: its tool-calling template for vLLM, Ollama and Hugging Face Transformers was inspired by Nous’ Hermes, while retaining compatibility with Qwen2’s template and Qwen-Agent.[7][8]

On April 29, 2025, the team released Qwen3 with two open-weight mixture-of-experts models and six dense models under Apache 2.0. The MoE variants distinguished total from activated parameters: 235B/22B and 30B/3B. Qwen3 combined thinking and non-thinking modes, letting users choose step-by-step reasoning or direct responses. Its four-stage post-training pipeline used long chain-of-thought fine-tuning, reasoning reinforcement learning, fusion of thinking and ordinary instruction data, and general reinforcement learning.[9]

The team reported approximately 36 trillion pretraining tokens covering 119 languages and dialects. Earlier Qwen products supplied part of the data pipeline: Qwen2.5-VL extracted text from PDF-like documents, Qwen2.5 improved the extracted content, and Qwen2.5-Math and Qwen2.5-Coder generated synthetic textbooks, question-answer pairs and code snippets. The team described its direction as “transitioning from an era focused on training models to one centered on training agents.”[9]

The February 2026 Qwen3.5 series brought a unified vision-language foundation. Qwen3.5-397B-A17B combines a vision encoder with a hybrid language-model layout: three Gated DeltaNet/MoE blocks followed by one gated-attention/MoE block, repeated fifteen times. Its specification lists 397B total and 17B activated language-model parameters, with a native context of 262,144 tokens extensible to 1,010,000. The team also documented a deployment tradeoff: increasing the presence penalty can reduce repetition but may cause language mixing and slightly lower performance.[10][11]

Leadership departures disrupted the team in March. Latent.Space’s March 4 roundup reported Justin Lin’s stepping-down announcement and further contributor exits, alongside continuing Qwen3.5 fine-tuning and quantization activity. Reuters reported on March 5 that Alibaba formed a task force to boost AI development after the departure of Qwen division head Lin Junyang.[1][4]

The April 2026 Qwen3.6-27B continued the hybrid attention layout in a dense 27B language model with a vision encoder. It introduced an optional preserve_thinking setting, backed by additional training to use reasoning traces from earlier messages. The team described this as useful for iterative agent work, where retaining reasoning context could reduce repeated analysis. Its agent tools included Qwen-Agent for tool integration and Qwen Code, an open-source terminal agent optimized for Qwen models.[11]

Qwen-Scope extended the team’s work into sparse autoencoders, which decompose model activations into sparse, interpretable features. The paper submitted May 12, 2026, by Boyi Deng and seventeen coauthors, describes fourteen SAE groups across seven Qwen3 and Qwen3.5 variants, covering dense and MoE architectures. The authors reported four uses: steering language, concepts and preferences without changing model weights; analyzing benchmark coverage and redundancy; classifying multilingual toxicity and synthesizing safety data; and incorporating feature signals into supervised fine-tuning and reinforcement learning to mitigate code-switching and repetition.[12]

Latent.Space’s May 2 account described a concrete debugging example: using a heatmap of over-activated features to diagnose unexpected language switches.[2]

Qwen3.8-Flash-Next explored a new architecture that the team described as an experimental preview for Qwen4. It pairs Gated DeltaNet with Qwen Sparse Attention, which scores context in micro-blocks, widens the residual stream to four gated branches, and adds n-gram embedding tables prefetched from host memory. Its parameter accounting separates a 125B backbone with 6B activated parameters from 51B n-gram embedding parameters and 4B multi-token-prediction parameters. It retains historical thinking blocks by default. Qwen3.8-Flash-Next uses the Qwen Community 1.0 license.[16][17]

In their August 31 architecture paper, Zihan Qiu and thirty-five coauthors reported that the model led its 397B-A17B predecessor on eight of fourteen pretraining benchmarks and trailed by at most 2.6 points on the others, using one-third the activated parameters, one-third the training tokens and roughly one-ninth the training FLOPs. Their ablations also exposed a limit to embedding growth: enlarging the n-gram vocabulary kept lowering loss while downstream accuracy saturated.[17]

Qwen developed within Alibaba’s cloud and AI infrastructure investment. In February 2025, Alibaba announced plans to invest at least RMB 380 billion (US$53 billion) over three years in cloud computing and AI infrastructure. This was a parent-company infrastructure commitment, rather than a Qwen-specific financing round or budget. At WGS 2026, Alibaba co-founder and chairman Joe Tsai explained the business rationale: Alibaba monetized infrastructure used for training, developing and running inference alongside its open Qwen models.[14][15]

Alibaba’s June 27, 2025 account cited more than 200 open-sourced Qwen models and over 300 million collective downloads, attributing those figures to TIME, and reported more than 130,000 Qwen-based Hugging Face derivatives. By February 6, 2026, Alibaba reported over one billion model-family downloads and more than 200,000 derivatives. These figures count download events and derivative models, rather than unique users or paying customers.[14][15]

The consumer Qwen app launched in November 2025. In January 2026, Alibaba integrated shopping, instant-delivery and travel services into it as an agentic assistant. Alibaba’s February 6 account reported more than 100 million monthly active app users, a separate measure from adoption of the downloadable model family.[15]