Chip Huyen

Chip Huyen is a writer and computer scientist whose work focuses on bringing AI into production. Her books, Designing Machine Learning Systems (O’Reilly, 2022) and AI Engineering (O’Reilly, 2025), explain how to design, evaluate, deploy, and maintain AI applications, with attention to the objectives of the whole system.[2][3][6]

AI Engineering provides a framework for adapting available foundation models—including language and multimodal models—to particular applications. It covers decisions about evaluation, prompting, context construction, retrieval-augmented generation, finetuning, security, latency, and cost. Huyen deliberately emphasizes fundamentals over particular tools or APIs: “Tools become outdated quickly, but fundamentals should last longer.” The book is not a tutorial built around code snippets; it helps readers choose among approaches and establish feedback loops for improving their applications.[6][3]

Huyen credits specific contributions to the book’s development: Luke Metz checked her assumptions, and Han-chung Lee pointed her toward resources she had missed; both reviewed early drafts before the next technical-review round. Kyle Krannen’s writeup about his colleagues’ finetuning process guided the finetuning chapter, while Mark Saroufim supplied resources on efficiency. She identifies their feedback as critical to Chapters 7 and 9.[6]

In her January 7, 2025 article “Agents”, adapted from AI Engineering, Huyen synthesizes existing literature around an agent’s environment, available actions, and planning capabilities. She distinguishes read-only actions, which supply information, from write actions, which change the environment: an SQL executor can retrieve a table or alter it, and an email API can read messages or send them. She describes the framework as experimental and subject to change as the field develops.[4]

Huyen recommends separating plan generation from execution so that invalid or excessively long plans can be rejected before they consume time and money. For risky operations such as database updates or code merges, she proposes explicit human approval or human execution. She leaves open whether language models can plan intrinsically, arguing that a model can still form part of a planner equipped with search and state tracking.[4]

Her evaluation advice separates planning failures, tool failures, and efficiency. A model may select an unavailable tool, supply invalid parameters, or use the wrong values; alternatively, it may select the correct tool and receive an incorrect output. She also recommends measuring steps, cost, and action duration rather than judging success alone. Tool inventories should be tested experimentally, including removing tools to see whether performance declines.[4]

Huyen was listed among the keynote speakers for the February 2025 AI Engineer Summit Online, with a talk titled “On Agents.”[1]

Huyen says she previously worked on ML tooling at NVIDIA, Snorkel AI, and Netflix, describing her NVIDIA role as a core developer of NeMo. A Stanford graduate, she also taught ML Systems there. Her lectures became the foundation for Designing Machine Learning Systems. She says she also founded and sold an AI infrastructure company.[2]

In her October 2020 course announcement, Huyen recalled beginning discussions with Stanford’s computer science department in early 2019 and announced plans to teach CS 329S: Machine Learning Systems Design in January 2021. The project-based course covered data pipelines, model selection, training, scaling, deployment, monitoring, and maintenance, alongside privacy, fairness, and security. She credited Christopher Ré, Jerry Cain, Mehran Sahami, Michele Catasta, and Mykel J. Kochenderfer with helping make the course possible.[5]

Designing Machine Learning Systems treats training data, feature selection, retraining frequency, and monitoring as interdependent decisions serving a system’s objectives. Its emphasis differs from AI Engineering: the earlier book concentrates on traditional ML applications, with more attention to data annotation, feature engineering, and model training; the later book concentrates on foundation-model applications, prompting, context construction, and parameter-efficient finetuning. Huyen presents the books as self-contained companions, noting that real systems can involve both kinds of models.[3][6]