Ben Poole

Ben Poole is a machine-learning researcher and coauthor of DreamFusion, which generates 3D objects from text using a pretrained 2D image diffusion model. His subsequent collaborations include CAT3D, which generates views for 3D reconstruction, and CAT4D, which reconstructs scenes that change over time. On June 25, 2026, he announced his departure from Google after a 15-year span encompassing four internships and exactly eight years of full-time employment.[3][5][6][10]

Poole coauthored DreamFusion: Text-to-3D using 2D Diffusion with Ajay Jain, Jonathan T. Barron and Ben Mildenhall. Submitted on September 29, 2022, the paper addressed a practical obstacle: adapting text-to-image diffusion directly to 3D would require large labeled 3D datasets and suitable denoising architectures. The team instead used an existing image model as a prior, requiring neither 3D training data nor modifications to that image diffusion model.[4][3]

The team's Score Distillation Sampling (SDS) turns a diffusion model into a loss for optimizing a representation that maps differentiably to images. DreamFusion starts with a randomly initialized Neural Radiance Field, or NeRF, renders it from random angles, and uses the text-to-image model Imagen to guide optimization of the 3D scene. SDS supplies appearance guidance; additional regularizers and optimization strategies improve geometry. The resulting objects can be viewed from different angles, relit or composed into other scenes, and exported as meshes.[3][4]

On CAT3D, presented orally at NeurIPS 2024, Poole shares equal-contribution credit with Ruiqi Gao and Aleksander Holynski. Their collaborators were Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul P. Srinivasan and Jonathan T. Barron. CAT3D conditions a multi-view diffusion model on input images and camera poses to generate consistent views from new viewpoints, then feeds those images into a reconstruction pipeline using Zip-NeRF or 3D Gaussian Splatting. Its architecture resembles video diffusion, with camera-pose embeddings in place of time embeddings. The team reports total view-generation and reconstruction time of as little as one minute.[5]

Poole also coauthored CAT4D with Rundi Wu, Ruiqi Gao, Alex Trevithick, Changxi Zheng, Jonathan T. Barron and Aleksander Holynski. Released as a 2024 preprint and listed as a CVPR 2025 oral presentation, the work generates multi-view videos from a monocular video and reconstructs the dynamic scene as deforming 3D Gaussians. Its diffusion model separates camera motion from scene motion, allowing a fixed viewpoint with changing time, changing viewpoints at a fixed time, or both together.[6][2]

Poole presented “Inferring 3D Structure with 2D Priors” at ICML 2024, discussing work on NeRFs and DreamFusion. The recording appears at 03:06:20 in Latent Space's ICML 2024 compilation, published on December 10, 2024.[1]

In his June 25, 2026 departure announcement, Poole recalled moving from editing decision trees by hand as an intern to researching representation learning, latent-variable models and diffusion at Brain and DeepMind. He described his later work as spanning DreamFusion and scaling generative models with Veo, Genie and Omni. His outlook remained focused on world simulation: “We're just beginning to build systems that can understand and simulate the real world”.[10]

Before his generative-3D work, Poole studied biological and artificial neural networks. He completed his PhD at Stanford University under Surya Ganguli in the Neural Dynamics and Computation lab, with a thesis on computational tools for understanding both kinds of networks. His undergraduate study was at Carnegie Mellon University, where Tai Sing Lee advised him.[2]

With Eric Jang and Shixiang Gu, Poole coauthored Categorical Reparameterization with Gumbel-Softmax, first submitted in November 2016. The method addresses the difficulty of backpropagating through discrete categorical samples by replacing them with differentiable samples from a Gumbel-Softmax distribution. That relaxation can be smoothly annealed toward a categorical distribution, providing a gradient estimator for models with categorical latent variables.[7]

In the 2017 ICML paper Continual Learning Through Synaptic Intelligence, Poole and Friedemann Zenke share equal-contribution credit, with Surya Ganguli as coauthor. The method accumulates task-relevant information at synapses to help a network learn new tasks while retaining earlier information. The paper reports reduced forgetting in continual classification experiments.[8][2]

Poole's diffusion research includes Score-Based Generative Modeling through Stochastic Differential Equations, first submitted in November 2020 and presented orally at ICLR 2021. With Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar and Stefano Ermon, he coauthored a framework that gradually adds noise to data and generates samples by reversing that process using learned, time-dependent scores. The work unifies earlier score-based and diffusion approaches, introduces predictor-corrector sampling, and derives an equivalent neural ordinary differential equation that supports likelihood computation.[9]