Browse · People
Person · Latent Space wiki

Fan-yun Sun

Moonlake cofounder and AI researcher working on interactive worlds, embodied-agent training environments, and structured representations for reasoning.

Fan-yun Sun is a Moonlake cofounder and AI researcher whose work includes generating interactive environments for reinforcement-learning and embodied agents. At Moonlake, he advocates world models that let agents act, observe consequences, and practice repeatedly. His technical approach separates reasoning about a persistent world from generating its visual appearance, using physics engines and code as tools for the task.[1]

Building worlds that agents can learn from

In an April 2026 interview, Sun traced Moonlake's origins to his work with NVIDIA Research during his PhD years, generating interactive worlds for training agents. He recalled organizations paying outside vendors to manually curate environments and synthetic data. The opportunity he saw was to build reasoning models that could construct these worlds: agents needed interactive data to learn the consequences of their actions. He also credited his cofounder Sharon with introducing him to Chris Manning, correcting the interviewer's assumption that he was Manning's graduate student.[1]

Sun used a bowling demonstration to make the distinction concrete. A useful training world must let an agent pick up and throw a ball, cause pins to fall, connect the score to the fallen pins, and reset for another attempt. Looking like a bowling game is insufficient if the agent cannot practice and discover what produces a high score. For Sun, the representation should capture the interactions needed to learn the intended task.[1]

Sun argues that choosing an efficient abstraction is compatible with the bitter lesson—the emphasis on methods that benefit from more computation. His question is “what is the right abstraction level today?” He compares next-byte prediction with next-token prediction: a lower-level representation can encompass many modalities, but requires more computation. He similarly treats physics engines and code as “cognitive tools” that a model can employ when the task requires them, rather than committing every world to one representation.[1]

Sun described Moonlake's architecture as two trained models. The multimodal reasoning model handles causality, persistence, and deterministic logic; a separate rendering model learns to restyle the persistent representation into photorealistic or other visual styles while respecting its interactivity. He acknowledged that the reasoning approach does not immediately deliver the pixel fidelity of existing video models. He also described the boundary between symbolic and diffusion priors as fluid: the team revisits it as it acquires knowledge and learns what users need.[1]

Human control is another part of Sun's design position. He wants a layer through which users can specify the distribution of environments they need, whether for creative work or embodied-agent training. Evaluation, he argues, should follow the intended use: time spent in a generated game and robustness after deploying a policy trained in generated worlds measure different outcomes. He connects commercialization to a feedback loop in which creators and users reveal which model capabilities need improvement. His broader vision of generating environments for training and evaluating policies was a prospective platform goal.[1]

Before Moonlake, Sun's research included graph representation learning, multi-agent prediction, and language-guided 3D environments. These projects supply technical context for his later world-model work. His personal profile describes his earlier position as a computer-science PhD candidate at Stanford AI Lab, affiliated with the Autonomous Agents Lab and Stanford Vision and Learning Lab, and states that he received a master's degree in computer science as part of the PhD program.[2]

Earlier research: graphs, interactions, and 3D environments

Sun coauthored InfoGraph with Jordan Hoffmann, Vikas Verma, and Jian Tang. First submitted in July 2019 and presented as an ICLR 2020 spotlight, it learns whole-graph representations by maximizing mutual information with representations of substructures at different scales, including nodes, edges, and triangles. Its semi-supervised extension, InfoGraph*, links unsupervised graph representations with supervised representations, allowing a supervised encoder to learn from unlabeled data while preserving the semantic space of its task.[5]

For NeurIPS 2022, Sun coauthored Interaction Modeling with Multiplex Attention with Isaac Kauvar, Ruohan Zhang, Jiachen Li, Mykel Kochenderfer, Jiajun Wu, and Nick Haber. IMMA predicts multi-agent behavior using a multiplex latent graph to represent separate interaction types and attention to account for relationships of different strengths. The collaborators also introduced Progressive Layer Training. Their reported evaluations covered trajectory forecasting and relation inference in social navigation, cooperative task achievement, and team sports.[4]

Sun shared equal technical contribution with Yue Yang and Luca Weihs on Holodeck: Language-Guided Generation of 3D Embodied Environments, published at CVPR 2024 with a wider author team. He also coauthored FactorSim: Generative Simulation via Factorized Representation, published at NeurIPS 2024, with S. I. Harini, Angela Yi, Yihan Zhou, Alex Zook, Jonathan Tremblay, Logan Cross, Jiajun Wu, and Nick Haber.[2]

Sun and Weiyu Liu were equal contributors to LayoutVLM, a 2024 preprint presented at CVPR 2025, alongside Siyi Gu, Dylan Lim, Goutam Bhat, Federico Tombari, Manling Li, Nick Haber, and Jiajun Wu. The framework arranges 3D assets from open-ended language instructions. A vision-language model generates code specifying an initial layout and spatial relations among assets and walls; differentiable optimization then produces the final placements. This combines semantic planning with an explicit mechanism for resolving object placement in physically constrained scenes.[3]

Timeline

  1. research

    Coauthors InfoGraph for learning whole-graph representations
    InfoGraph[5]

  2. research

    Coauthors IMMA for modeling multi-agent interactions
    NeurIPS 2022[4]

  3. research

    Shares equal technical contribution on Holodeck's language-guided 3D environments
    CVPR 2024[2]

  4. research

    Coauthors FactorSim on factorized generative simulation
    NeurIPS 2024[2]

  5. research

    LayoutVLM combines language-guided planning with optimized 3D placement
    CVPR 2025[3]

From the Latent Space archive

Original posts and separately reviewed explanations.

Sources

  1. 1
  2. 2
    Sun, Fan-YunAccessed 2026-09-18
  3. 3
  4. 4
  5. 5

Model source and teaching reviews passed. Human editorial review pending. This profile describes the sources as of 2026-09-17.

Latent Space

Join Latent Space.

Sign in with Google, or get the newsletter by email.

Loading Google sign-in…

or subscribe by email

Free. Unsubscribe any time.