Ce Zhang

Ce Zhang is a co-founder of Together AI and a machine learning systems researcher. As Together’s CTO, he has advocated improving AI inference by optimizing algorithms, model architectures and systems together. His work also addresses how training data are selected and how complex computing infrastructure can become easier to use.[1][5]

Zhang holds the Neubauer Associate Professorship of Computer Science, Data Science at the University of Chicago. He is also a co-founder of Together AI, where he was CTO in February 2024.[5][4][1]

Zhang co-founded Together with collaborators including Vipul Ved Prakash, Chris Ré, Percy Liang and Tri Dao. In their February 2024 interview, Prakash recalled that Ré introduced him to Zhang and that, within five minutes, he decided they should start the company. Prakash connected Zhang’s research on reducing communication to an early question: could GPUs distributed across a network provide cheaper compute for model training?[4][1]

Zhang described a recurring problem in his decade of work with Ré: “It's always about data movement across essentially all the stacks, right?” Distributed computing requires communication between machines; GPU algorithms must manage movement through different levels of memory. He used FlashAttention, developed by Together chief scientist Tri Dao, as an example of the latter.[1]

Zhang described Together’s infrastructure as disaggregated: a reliable layer over resources that differ in geography, reliability and hardware. The design goal was to hide those differences from users. His inference thesis likewise depended on coordination across layers: isolated improvements encounter diminishing returns, while algorithms, architectures and systems can sometimes compound their gains.[1]

Zhang explained RedPajama as a team effort to reproduce Llama 1’s published training-data recipe as closely as possible when the actual dataset was unavailable. Releasing the reproduction would let the community learn from its mistakes and build both models and further datasets. He credited earlier resources, including Eleuther’s Pile and Google’s C4, and pointed to Cerebras’s SlimPajama, which deduplicated RedPajama data, as an example of that reuse.[1]

In February 2024, Zhang described RedPajama V2 as containing 30 trillion tokens and approximately 40 precomputed quality signals. Its important design choice was to leave filtering flexible: the right selection and deduplication strategy could depend on the application. Users could combine signals and experiment with filters rather than receive only one fixed final dataset. Zhang also acknowledged AI2’s flexible data tooling and said RedPajama could be loaded through it.[1]

Zhang explained DSIR, crediting Percy Liang’s work, as a way to select training data using similarity to a target validation set or a desired corpus—for example, finding more documents like Wikipedia. He argued that a targeted application can sometimes support a better, smaller model that is easier to run. The choice involves tradeoffs among generality, accuracy, deployment cost, model size and robustness.[1]

Zhang distinguished the availability of data on Earth from access to data for open-model training. He expressed optimism about connecting open models to organizational data and sources beyond internet text, including transcripts, video and audio. He also acknowledged that some data might appropriately remain closed, leaving the community to find a balance between open models and usable training collections.[1]

Zhang’s interest in state-space architectures extended beyond long context. In the February 2024 interview, he explained that reducing the state carried by a model could also enable larger batches and faster execution on shorter contexts. He described StripedHyena’s mixture of state-space and transformer components as an exploration of whether different components could handle different aspects of a sequence better.[1]

He distinguished two research goals: StripedHyena sought to match a strong Mistral baseline using a hybrid architecture, while Mamba explored how far a pure architecture could be pushed, scaling up to three billion parameters. Zhang treated the optimal mixture as unresolved. Before committing to larger models, he advocated systematic study of scaling, data quality, architecture mixtures and hardware efficiency. He declined to assume that the best training data for these architectures would match those for transformers: “I think that's the result of the study, not the assumption.”[1]

Zhang supported publishing research ideas and papers, linking openness to attracting researchers and advancing the field. He distinguished that commitment from releasing every software artifact: some components were deeply embedded in Together’s infrastructure and mainly useful for building its disaggregated cloud. His position in February 2024 left their eventual release open.[1]

Discussing Together’s feedback on Anyscale’s inference benchmarks, Zhang focused on what benchmarks incentivize. He invoked database TPC benchmarks as a possible model for industry agreement and argued for transparent protocols aligned with user benefit. Beyond price and speed, he identified model quality, throughput, time to first token, reliability, capacity and latency percentiles as relevant dimensions.[1]

Before Together, Zhang pursued machine learning usability in academia. His earlier ETH work supplies concrete context for that concern: helping domain scientists assemble available techniques into usable systems.[8][7]

Zhang studied as an undergraduate at Peking University, advised by Bin Cui. He completed his PhD at the University of Wisconsin–Madison and then spent a year as a postdoctoral researcher at Stanford, both advised by Chris Ré. He joined ETH Zurich’s Department of Computer Science as a tenure-track assistant professor on September 1, 2016, and later became an associate professor.[6][5][8]

In his 2017 ETH interview, Zhang used a lung-cancer pathology collaboration to explain the gap between available machine learning methods and accessible tools. Although the methods were standard from a machine learning perspective, he recalled that pathologists and medical doctors spent considerable time putting them together. He wanted to connect systems researchers, machine learning researchers and scientists in other disciplines to make that process easier.[8]

Under Zhang’s leadership, ETH’s DS3 Lab worked on speed and scalability, automation, process management and human interaction through collaborative systems projects. Zhang said that working with domain scientists during his PhD had shaped his research, and he wanted to give his students similar opportunities to understand users’ problems. He left ETH in summer 2023. His later academic activities include serving as co-editor-in-chief of DMLR, a journal focused on data-centric machine learning.[7][8][5][6]