Josh Albrecht

Josh Albrecht is a co-founder of Imbue, formerly Generally Intelligent, and was its CTO during its June 2024 release of model-training tools and evaluation datasets. He co-authored CARBS, a cost-aware hyperparameter optimizer, and helped make infrastructure decisions for Imbue’s large-scale training work. His stated priority was practical coding and reasoning: agents that can generate, understand, test, and verify software useful in everyday work.[5][8][1]

In his June 25, 2024 interview with Jon Frankle, Albrecht explained Imbue’s decision to withhold its internally trained model weights while releasing resources that could help other teams train their own models. These included infrastructure scripts, CARBS, cleaned and newly written evaluation questions, CodeComprehension, and 450,000 human judgments about question quality. He described training from scratch as expensive and risky for smaller companies; the releases addressed the work required to make training run reliably and determine whether the resulting model worked.[1][11]

Imbue reported that about a dozen engineers and researchers completed the training project, including infrastructure, evaluations, and hyperparameter optimization. After thousands of smaller experiments, the team reported reaching 70B parameters on its first large-scale attempt with minimal instability and no loss spikes. Its model, pre-trained on 2 trillion tokens and fine-tuned on multiple-choice reasoning benchmarks, outperformed zero-shot GPT-4o in the specified comparison without chain-of-thought. That result did not establish general superiority or GPT-4o’s best attainable performance.[11]

Albrecht and his colleagues worked closely with Voltage Park on cluster architecture, component orders, and networking. He said direct control made it easier to debug with Dell, NVIDIA, or the responsible data-center partner, rather than passing problems through multiple intermediaries. Imbue’s written infrastructure account describes one cluster with 4,088 H100 GPUs across 511 computers and a three-tier InfiniBand network. Albrecht described an infrastructure team fluctuating between three and six people, while explicitly crediting substantial work by Dell, H5, and NVIDIA; the written account also credits Voltage Park.[1][10]

One failure Albrecht recalled was a GPU returning incorrect arithmetic without an explicit error. Another was training throughput gradually deteriorating over time. Profiling helped the team trace the slowdown to automatic garbage collection: in synchronous distributed training, a pause on one worker could hold up the others. The team’s written remedy was to disable automatic collection and schedule collection at specific intervals across all hosts.[1][10]

Albrecht favored simple, inspectable tools—Bash, Python, SSH, and Docker—and an orchestration abstraction tailored to experiments that run to completion. Machine failures and reboots were built into its assumptions. Imbue paired this approach with automated host checks, granular boot-log inspection, and relaunching failed training jobs on healthy machines. As he put it, the abstraction was “a sort of simpler, more tailored abstraction for the particular work that we're doing.”[1][10]

CARBS models both performance and GPU-hour cost, then searches locally near the performance-cost Pareto frontier: configurations offering the best observed tradeoffs between quality and expense. This lets it learn how settings such as learning rate, model dimensions, and token count should change with scale. Albrecht explained that the aim was to learn from inexpensive runs before committing to a costly 70B model, including when changes such as a custom tokenizer made established settings less dependable.[7][1]

The team also needed a metric sensitive enough to distinguish small models. It fine-tuned models to answer multiple-choice questions and measured the cross-entropy of the correct answer token. This captured changes that coarse accuracy could miss while targeting reasoning and code-understanding tasks. A held-out evaluation metric also allowed CARBS to compare training-data mixtures. Imbue reported using its scaling experiments and an intermediate 7B model trained on 200 billion tokens to predict the final 70B run’s performance.[9][1]

Albrecht argued that ambiguous questions and incorrect labels could obscure what benchmarks actually measured. Imbue sanitized 11 public multiple-choice datasets and commissioned handwritten recreations from human annotators to reduce contamination concerns. The team reported near-perfect performance on unambiguous questions in several datasets, suggesting that remaining score differences often concerned question quality. Its CodeComprehension benchmark instead asks models to predict outputs of procedurally generated Python snippets; success on those relatively low-complexity examples does not guarantee correct code generation.[1][12]

For realistic coding tasks, Albrecht emphasized the gap between passing tests and producing a complete solution: tests may miss production edge cases, and architectural judgments are harder to grade objectively. He preferred agents that reliably communicate uncertainty, even at some cost to raw correctness. He also treated abstract reasoning puzzles and difficult coding competitions as scientific tools whose scores might not translate into day-to-day programming usefulness.[1]

Albrecht’s approach to tool use centered on robust code writing, execution, debugging, and judging whether code was safe to run. He argued that code could give agents a broader action space than a fixed tool list, letting them choose suitable libraries and APIs for each problem. In June 2024, he described core coding and reasoning as Imbue’s focus, with external multimodal models available for particular visual needs. He also cautioned that long context did not remove the value of selecting relevant repository content or avoiding unnecessary inference cost.[1]

The optimizer used in the 2024 training project grew out of earlier collaborative research. Albrecht’s work at Imbue also followed a transition from an AI recruiting company to an independent research lab.[8][14][13]

Albrecht says he began programming by creating games as a child, worked professionally as a software engineer in high school, and published machine-learning research under Rebecca Hwa. He holds bachelor’s and master’s degrees in computer science from the University of Pittsburgh, co-founded Sourceress and CloudFab, led infrastructure at Addepar, and mentored Thiel Fellows, according to his Outset biography, which also records CloudFab’s acquisition. Before Generally Intelligent, he served as Addepar’s lead data architect and CTO of CloudFab and Sourceress, roles described in his February 23, 2021 Machine Learning Engineered episode.[5][6][14]

By February 2021, Albrecht was publicly identified as co-founder and CTO of Generally Intelligent, described as a pivot from Sourceress. In Nic Seo’s later profile, colleague Bryden Fogelman recalled asking Albrecht to work on machine learning and receiving an enthusiastic response. The profile associates that exchange with the transition to the research company that became Imbue. Fogelman subsequently explored hyperparameter optimization through CARBS and spearheaded the search for practical coding-agent use cases.[14][13]

Albrecht co-authored Tune As You Scale, submitted June 13, 2023, with Abraham J. Fetterman, Ellie Kitanidis, Zachary Polizzi, Bryden Fogelman, Maksis Knutins, Bartosz Wróblewski, James B. Simon, and Kanjun Qiu. The CARBS team reported reproducing Chinchilla’s C4 scaling result while tuning 19 parameters across 340 runs of 19M–1.5B-parameter models. It also reported substantial gains from tuning an existing PPO baseline on ProcGen, demonstrating the value of systematic tuning rather than introducing a new reinforcement-learning algorithm.[8][7]

Alongside his technical work, Albrecht invests through Outset Capital. On his undated personal website, he describes Imbue’s mission as “to reclaim human agency over technology,” through an ecosystem of open software and transparent, user-aligned AI agents.[5]