Jack Hopkins

Jack Hopkins is an independent AI safety researcher based in London who built the Factorio Learning Environment (FLE), an open-ended benchmark for language-model agents. His work spans agents’ ability to plan and build complex systems, techniques for exposing hidden information in model weights, and whether models can reliably detect deception.[2][3][7][8]

Hopkins coauthored the Factorio Learning Environment paper with Mart Bakler and Akbir Khan, submitted on March 6, 2025. His project account separately credits building FLE with Neel Kant, Harshit Sharma and Kian Kyars, and acknowledges contributions from Yasaman Ansari, Bakler, Dipika Khullar and Khan. FLE was accepted at NeurIPS 2025.[4][3]

FLE uses Factorio’s factory-building simulation to test long-term planning, program synthesis and resource optimization. The original paper offers lab-play, eight structured tasks with fixed resources, and open-play, the unbounded task of building the largest factory on a procedurally generated map. Hopkins emphasizes that chaining production systems creates increasingly complex challenges without a fixed construction ceiling. Factorio’s deterministic engine also makes evaluation reproducible.[4][3]

The FLE repository describes a code-based interaction loop: an agent observes the previous program’s standard output and errors, writes a Python program, and receives execution feedback. Variables, classes and functions enter the execution namespace, allowing subsequent programs to build on earlier work.[5]

The March 2025 paper reports that tested models struggled with spatial reasoning in both settings. In lab-play, promising short-horizon skills did not translate into effective operation under resource constraints, which the authors connected to limitations in error analysis. In open-play, models discovered useful automation such as electric-powered drilling but failed to achieve electronic-circuit manufacturing.[4]

Hopkins also reports that agents entering failure states conditioned strongly on their own past failures and performed worse, tentatively suggesting that they might adopt the persona of a poor coding agent. His project account identifies instrumental convergence as FLE’s founding motivation, while explicitly saying the environment did not yet directly measure instrumentally convergent properties.[3]

Hopkins studied deceptive behavior as an Anthropic fellow and MATS scholar in 2025–26. His subsequent research includes collaborations with Dipika Khullar, Fabien Roger and Rowan Wang on techniques for auditing language models and the limitations of learned lie detectors.[2][7][8]

In Overthinking: Amplifying Reasoning Weights to Extract Learned Secrets, Hopkins, Khullar and Roger introduce a weight-based auditing technique. They amplify the parameter difference between an instruction model and its reasoning-distilled counterpart, with layer-wise attenuation to preserve output coherence. The ICML 2026 paper reports hidden information surfacing up to ten times more frequently than in the original reasoning model across four experimental settings using 2B–32B models. The effect depends on the secret: some require perturbations along the reasoning direction, while others emerge under sufficiently large weight perturbations generally.[7]

In the August 21, 2026 article Fine-Tuned Lie Detectors Failed to Generalize, Hopkins, Khullar, Wang and Roger describe eliciting lies across 12 controlled settings and training detectors on half the lie types while testing on the remainder. For the Gemma-3 detectors, AUROC—a measure of ranking performance—improved from 0.60 to 0.95 on lie types included in training, but plateaued at 0.70–0.75 on held-out categories. The specialized detectors barely outperformed prompted baselines on novel lie types.[8]

The team argues that detectors can learn the surface features of particular settings rather than a general signature of deception, making performance within the training distribution overstate what they can catch. Their conclusion has concrete limits: they used LoRA fine-tuning, approximately 25% of labels changed during a cleaning pass, and they did not test alignment faking, reward hacking or strategic deception during actual use.[8]

Hopkins’s neural-language research predates FLE. In July 2017, he and Douwe Kiela published Automatically Generating Rhythmic Verse with Neural Networks. One approach trained a language model on phonetic encodings to learn poetic form and content together; another combined neural generation with a weighted finite-state machine imposing form constraints. The paper reports that participants judged generated poems to be human-written 54% of the time in its evaluation.[9][6]

Before moving into safety research, Hopkins co-founded Paperplane, a conversational intelligence platform in YC W23, and Spherical Defence, an API security company. In his account of leaving startup work, he says it felt “a little solipsistic” and that he had not found an idea with enough impact to pursue even if it made no money. He studied computer science at Cambridge, earning an MPhil with Distinction, and at Royal Holloway, University of London, earning a first-class BSc.[2]