Yineng Zhang

Yineng Zhang works on the performance and operation of large-language-model inference. He helped maintain SGLang during 2024–2025, coauthored the FlashInfer attention-engine paper, and contributed to Baseten’s published work on serving Qwen 3. His personal site identifies him as a senior director leading Together AI’s inference team and a governing-board member of the LightSeek Foundation, as of September 2026.[2][4][5]

Zhang says he joined Together AI in July 2025 and led its inference team as it grew from two people to more than ten, while helping shape the company’s inference-service strategy. His GitHub profile, under the handle zhyncs, also identifies him as the team’s leader.[2][3]

In March 2026, Zhang says he co-created TokenSpeed, an LLM inference engine, bringing together senior engineers and open-source maintainers from across AI infrastructure. He frames its backing by the nonprofit LightSeek Foundation as a basis for neutrality rather than control by a commercial company.[2]

Zhang emphasizes open source’s reach beyond an employer’s own staff. His personal site reproduces a New York Times interview quotation in which he says, “if you work on open source, you work with talent around the world.”[2]

Before joining Together AI, Zhang’s SGLang maintenance and Baseten model-performance work involved both supporting new model releases and explaining the tradeoffs of production inference.[2][1][5]

Zhang describes himself as a SGLang core maintainer from 2024 to 2025 who helped deliver day-zero support and performance optimizations for DeepSeek-V3. In the January 19, 2025 Latent Space episode, he appeared alongside Baseten cofounder Amir Haghighat as the company’s lead software engineer in model performance. The two jointly challenged the idea that choosing an inference framework was enough: the discussion also addressed horizontal scaling across clusters and regions, and developer workflows involving multiple models and steps.[2][1]

With Michael Feil and Philip Kiely, Zhang coauthored Baseten’s Qwen 3 day-zero benchmark article, published April 29, 2025 and updated May 19. The team reported optimizing and deploying Qwen 3 for customers within minutes of the weights becoming available. Their explanation of Qwen 3 235B paired tensor parallelism for attention with expert parallelism for sparse mixture-of-experts layers. They replicated the small routing layer on each GPU because recomputing it was cheaper than communicating its results.[5]

The coauthors recommended FP8 serving for Qwen 3 235B for what they described as nearly identical quality at lower cost; in their Hopper GPU setup, FP8 could run across four GPUs, while BF16 used eight. They also explained a workload tradeoff: smaller batches reduce latency but lower throughput, increasing effective cost per token. For their April 29 batch-size-32 benchmark of Qwen 3 235B on H200 GPUs, with requests containing 1,000 input and 1,000 output tokens, they reported about 250 milliseconds median time to first token, 45 tokens per second per user, and 1,400 tokens per second total throughput, including prefill.[5]

Zhang coauthored FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving with Zihao Ye, Lequn Chen, Ruihang Lai, Wuwei Lin, Stephanie Wang, Tianqi Chen, Baris Kasikci, Vinod Grover, Arvind Krishnamurthy, and Luis Ceze. First submitted January 2, 2025 and revised April 21, the paper was accepted by MLSys 2025.[4]

FlashInfer addresses varied KV-cache storage layouts with block-sparse and composable formats, and adapts attention kernels through customizable templates and just-in-time compilation. Its load-balanced scheduling accommodates changing requests while remaining compatible with CUDAGraph’s static-configuration requirements. The shared work was integrated into SGLang, vLLM, and MLC-Engine, connecting attention-kernel research to serving frameworks.[4]