Ashvin Nair

Ashvin Nair is a reinforcement-learning researcher whose contributions include learning robotic skills from images, demonstrations and previously collected experience. He later worked on OpenAI’s GPT-4o, o1 and o3; the December 2025 Latent Space episode described him as leading model development at Cursor. His coauthored research includes reinforcement learning with imagined goals (RIG), advantage-weighted actor critic (AWAC) and implicit Q-learning (IQL).[1][3][4][6]

Nair’s OpenAI career included a 2017 internship during the Dota era and later work on the o1/o3 reasoning team. By NeurIPS 2025, he had moved to Cursor and Composer. These career details and the views below were described in Latent Space’s December 2025 episode description, covering its interview with him at the conference.[1]

Nair argued that economically useful tasks must be brought into the training distribution through product–model co-design. For software engineering, that means encompassing writing code, checking logs, debugging and iterating. He also criticized the earlier RL research community’s overfitting to benchmarks and tendency to reward complex ideas over simpler approaches that generalize.[1]

Nair’s bet for the next paradigm shift was continual learning with persistent memory: models that learn from a bug, mistake or user pattern once and retain that experience in their weights. This was a vision for future systems, rather than a claim that Composer had already achieved it.[1]

Before his reasoning-model and coding work, Nair studied machine learning for robotics at UC Berkeley. He completed a PhD in the BAIR Lab under Sergey Levine, after undergraduate study at Berkeley working with Pieter Abbeel and Levine. This earlier research provides technical context for his later discussion of learning through interaction. On his personal research site, he describes his interest as “how to develop intelligent behavior through interaction, especially through offline reinforcement learning and self-supervised reinforcement learning.”[2]

In Overcoming Exploration in Reinforcement Learning with Demonstrations, first submitted in September 2017 and presented at ICRA 2018, Nair collaborated with Bob McGrew, Marcin Andrychowicz, Wojciech Zaremba and Pieter Abbeel. The team used a small set of demonstrations with Deep Deterministic Policy Gradients and Hindsight Experience Replay to address sparse-reward exploration: an agent can struggle to discover useful behavior when successful actions are the only ones rewarded. Their method learned multi-step tasks such as block stacking; the authors reported an order-of-magnitude speedup over RL on simulated robotics tasks.[5]

Nair and Vitchyr Pong shared equal-contribution credit on Visual Reinforcement Learning with Imagined Goals, with Murtaza Dalal, Shikhar Bahl, Steven Lin and Sergey Levine also credited on the 2018 paper. RIG uses a variational autoencoder to encode current and goal images into a low-dimensional latent space. Distance in that space supplies the reward, replacing pixel-wise error. Sampling from the VAE prior supplies goals for autonomous practice and for relabeling previously collected experience, allowing the same interactions to support learning about multiple goals.[3]

The RIG team demonstrated reaching and object pushing on a Sawyer robot using only 84×84 RGB images, without joint angles or object positions. To move from reaching to pushing, they added an object and a table and adjusted the camera. They reported about one hour of real-robot interaction for reaching and 4.5 hours for pushing. After autonomous practice, a user could specify a task with a goal image without additional training.[3]

Nair coauthored AWAC with Abhishek Gupta, Murtaza Dalal and Sergey Levine, first submitting it in June 2020. It addressed the difficulty of learning from an existing dataset and then continuing to improve through online interaction. AWAC combines dynamic programming with maximum-likelihood policy updates, using prior demonstrations or suboptimal transitions as a starting point for online fine-tuning. The team demonstrated it on simulated and real robotics tasks, including dexterous manipulation, drawer opening and valve rotation.[4]

With Ilya Kostrikov and Sergey Levine, Nair coauthored Implicit Q-Learning, submitted in October 2021 and listed at ICLR 2022. IQL tackles a central offline-RL problem: improving a policy without relying on unreliable value estimates for actions absent from the dataset. It fits an upper-expectile value function to estimate the value of better available actions, backs that value into a Q-function, and extracts a policy through advantage-weighted behavioral cloning. The authors reported state-of-the-art performance on the D4RL benchmark at publication and strong online fine-tuning after offline initialization.[6][2]