Charlie Snell

Charlie Snell is an AI researcher and coauthor of work on test-time scaling and sleep-time compute: improving language-model answers by choosing how to spend inference computation, including reasoning before a user's question arrives. His research also covers predicting emergent capabilities, evaluating model imitation, and offline reinforcement learning for language generation.[3][2][4][5][6]

In his undated personal biography, Snell describes himself as a Berkeley EECS computer-science PhD student advised by Dan Klein, a former Berkeley undergraduate, and a former Student Researcher at Google DeepMind. He says he now works at Cursor “on training frontier coding agents.” The April 2025 sleep-time compute paper identifies his affiliation as the University of California, Berkeley.[1][2]

Snell coauthored Sleep-time Compute: Beyond Inference Scaling at Test-time, dated April 17, 2025, with Kevin Lin, Yu Wang, Charles Packer, Sarah Wooders, Ion Stoica, and Joseph E. Gonzalez. The team addressed a practical feature of stateful assistants: documents, a codebase, or conversation history can be available before the next user request, and related requests may require the same intermediate reasoning.[2]

Their method prompts a model to draw useful inferences from that existing context and rewrite it before the question arrives. The resulting context is then supplied with the question, allowing the model to reuse work already performed. For a coding assistant, this could mean identifying architectural patterns or anticipating debugging strategies. The method changes the context supplied to the model; its implementation does not train new model weights.[2]

The team evaluated the method on Stateful GSM-Symbolic and Stateful AIME, constructed by separating mathematics problems into prior context and a later question. They reported approximately fivefold less test-time computation at matched accuracy, with additional preprocessing computation performed beforehand. Scaling sleep-time compute further yielded reported accuracy improvements of up to 13% and 18%, respectively. Gains depended on the model and computation budget: o1 showed limited gains, and standard test-time computation could perform better at high budgets.[2]

Sharing preprocessing across related questions offered a separate route to savings. On Multi-Query GSM-Symbolic, the team reported up to a 2.5-fold reduction in average cost per query with ten queries per context, compared with a single-query baseline. This was a modeled cost result that priced test-time tokens at ten times the cost of sleep-time tokens. Their analysis also found greater benefits when the future question was more predictable from its context; unpredictable or unrelated questions benefited less.[2]

The collaboration also tested repository exploration before a feature request on SWE-Features, a 33-example benchmark drawn from Aider and ComfyUI. At lower test-time budgets, sleep-time compute improved the reported results and achieved roughly a 1.5-fold decrease in test-time tokens. The evaluation measured agreement with the reference set of modified files, rather than successful feature implementation; standard test-time computation performed better at higher budgets.[2]

Before the sleep-time collaboration, Snell studied how to allocate computation after a prompt arrives. With Jaehoon Lee, Kelvin Xu, and Aviral Kumar, he coauthored Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters, submitted August 6, 2024. The team examined search guided by process-based verifier reward models and adaptive changes to the model's response distribution. Because the effectiveness of each approach varied with prompt difficulty, they proposed allocating computation adaptively per prompt.[3]

The authors reported more than fourfold greater efficiency than a best-of-N baseline. In a FLOPs-matched evaluation, test-time computation also allowed a smaller model to outperform a model fourteen times larger—but on problems where the smaller base model already achieved nontrivial success. The result tied the value of additional inference computation to what the model could already do.[3]

In Predicting Emergent Capabilities by Finetuning, Snell, Eric Wallace, Dan Klein, and Sergey Levine investigated whether smaller models could reveal capabilities that would emerge in larger ones. They finetuned models with varying amounts of task data and fitted parametric “emergence laws” to predict capability thresholds. Validation covered MMLU, GSM8K, CommonsenseQA, and CoLA; in some cases, the team could predict emergence in models trained with up to four times more compute. The paper's arXiv submission was November 25, 2024.[4]

Snell's earlier language-model work included Implicit Language Q-Learning, coauthored with Ilya Kostrikov, Yi Su, Mengjiao Yang, and Sergey Levine. Initially submitted in June 2022 and revised in May 2023, the paper proposed an offline reinforcement-learning method that learns from previously collected data. Conservative value learning and a dataset-support constraint guide generation toward user-specified utility. The team's evaluations included end-to-end dialogue and subjective reward judgments such as whether a comment was toxic.[6]

His project descriptions also identify two 2022 contributions. With Dan Klein and Ruiqi Zhong, he explored context distillation—internalizing gains that language models obtain from context tokens such as prompts or scratchpads. He lists JaxSeq as an October 2022 project built on HuggingFace Transformers to support language-model training in Jax with model and data parallelism across multiple devices and nodes.[1]

Snell was also one of eight authors of The False Promise of Imitating Proprietary LLMs, submitted in May 2023. The team initially found that models trained to imitate ChatGPT appeared much better at following instructions, and crowd workers rated their outputs as competitive with ChatGPT. Targeted automatic evaluations challenged that impression: on tasks not heavily represented in the imitation data, the models closed little to none of the capability gap. The authors described them as adept at “mimicking ChatGPT's style but not its factuality” and argued for developing stronger base language models. This was a collective conclusion about the methods evaluated in that study.[5]