Rishabh Agarwal

Rishabh Agarwal is a machine-learning researcher whose work includes on-policy language-model distillation, reinforcement learning for self-correction, and methods for evaluating deep reinforcement learning reliably. He is a founding member of Periodic Labs and an adjunct professor at McGill University. Previously, he worked on reinforcement learning and reasoning at Meta and was a staff research scientist at Google DeepMind.[2][3][5][6][7]

Agarwal and Nino Vieillard contributed equally to On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes, alongside Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist and Olivier Bachem. First submitted in June 2023 and accepted at ICLR 2024, the paper introduced Generalized Knowledge Distillation (GKD) to address a mismatch in autoregressive models: the sequences a student sees during training can differ from those it generates during inference.[5]

GKD trains the student on its own generated sequences, using feedback from the teacher on those sequences. This lets training address the outputs the student actually produces rather than relying solely on a fixed set of examples. The method also permits alternative losses between student and teacher when the student cannot fully express the teacher's distribution, and integrates distillation with reinforcement-learning fine-tuning. The collaborators evaluated it on summarization, translation, arithmetic reasoning and task-agnostic instruction tuning.[5]

Agarwal discussed this area in Latent Space's The Magic of LLM Distillation, which presented him with his Google DeepMind affiliation. His personal website links the appearance as a tutorial on post-training distillation.[1][2]

His subsequent collaborative work includes Training Language Models to Self-Correct via Reinforcement Learning, presented as an oral paper at ICLR 2025. The approach uses multi-turn online reinforcement learning to teach a language model to self-correct using entirely self-generated data.[7]

Before the 2023 distillation paper, Agarwal worked on how reinforcement-learning experiments should be compared. With Max Schwarzer, Pablo Samuel Castro, Aaron Courville and Marc G. Bellemare, he coauthored Deep Reinforcement Learning at the Edge of the Statistical Precipice, which received an Outstanding Paper Award at NeurIPS 2021. The team argued that aggregate mean or median scores from only a few training runs can obscure statistical uncertainty.[5][6]

Their Atari 100k case study found substantial discrepancies between conclusions based on point estimates alone and those from fuller statistical analysis. They advocated interval estimates, performance profiles that show variability, and robust aggregate metrics such as interquartile mean scores. The work supplied the open-source rliable library and also examined comparisons on the Arcade Learning Environment, Procgen and the DeepMind Control Suite.[6]

In their NeurIPS 2022 work on reincarnating reinforcement learning, Agarwal and the same collaborators proposed reusing prior computational work, such as learned policies, across agent designs. They argued for an alternative to repeatedly training from scratch: transfer an existing policy to another agent and continue learning. Their experiments included Atari games, locomotion and stratospheric-balloon navigation.[7][4]

Agarwal completed his PhD at Mila under Aaron Courville and Marc Bellemare. Earlier, he spent a year with Geoffrey Hinton's Google Brain team in Toronto and graduated in Computer Science and Engineering from IIT Bombay.[2][4]