Kevin Wang

Kevin Wang researches reinforcement learning (RL). He is the first author of 1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities, which received a NeurIPS 2025 Best Paper award. Working with Ishaan Javali, Michał Bortkiewicz, Tomasz Trzciński and Benjamin Eysenbach, he studied how deeper networks could learn more capable goal-reaching policies through self-supervision. Delta Podcast’s May 28, 2026 episode description identified him as an OpenAI researcher working on RL and reasoning.[3][7][6][2]

Before his work at OpenAI, Wang studied computer science at Princeton. He belonged to Danqi Chen’s lab as an undergraduate during 2024–2025. His thousand-layer RL paper also lists his affiliation as Princeton, alongside fellow Princeton authors Javali and Eysenbach and collaborators Bortkiewicz and Trzciński at Warsaw University of Technology.[6][5][3]

Wang and his collaborators built on contrastive RL, an existing actor-critic algorithm, rather than simply enlarging a conventional reward-trained network. In their online, simulated tasks, agents explored from scratch without supplied rewards or demonstrations. The critic learned to distinguish state-action pairs and future goals from the same trajectory from pairs drawn from different trajectories, using an InfoNCE classification objective. Hindsight experience replay relabeled trajectories with goals the agent actually achieved, turning unsuccessful attempts at the commanded goal into useful training examples.[3][2]

The team combined residual connections, layer normalization and Swish activations, drawing these components from prior work. Each residual block contained four repeated Dense-layer, normalization and activation units; depth counted the Dense layers across the blocks. They generally increased the actor and both critic encoders together. At the largest scale, however, actor loss exploded at the start of training. Their adjustment was to keep the actor at 512 layers, while using 1,024 layers only in the two critic encoders.[3]

The team reported that scaled contrastive RL outperformed comparison baselines in eight of ten simulated locomotion, navigation and manipulation tasks. Performance was measured as time steps spent near the goal out of a 1,000-step episode. The behavioral comparisons made the gains concrete: a four-layer humanoid collapsed or threw itself toward the target, while a 16-layer agent walked upright. In Humanoid U-Maze, a 64-layer agent struggled and fell; a 256-layer agent learned to vault over the wall. These were distinct task-and-depth comparisons, rather than demonstrations that every capability required a thousand layers.[3]

To separate better exploration from greater learning capacity, the collaborators trained deep and shallow learners on the same replay buffer supplied by a separate data-collecting network. With a deep collector providing good coverage, the deep learner performed better. With a shallow collector providing poor coverage, additional learner capacity could not compensate. Their experiments therefore supported complementary roles for exploration and expressivity. They also found that deeper networks benefited more from larger batches and that increasing depth was more effective than increasing width in their tested comparisons.[3]

Wang and his coauthors argued that RL could develop large models while also improving its own data collection through exploration. Their results had important boundaries: greater depth increased compute costs, and preliminary OGBench experiments provided little evidence that deeper networks improved offline contrastive RL, where learning uses a fixed dataset. They proposed distributed training, pruning and distillation as future ways to extend scaling or reduce its costs.[3]

NeurIPS announced the shared Best Paper award on November 26, 2025. The selection committee praised the work for challenging the assumption that RL supplies too little information to train deep networks effectively, and highlighted its analysis of batch-size scaling in deeper contrastive RL networks.[7]