Jay Alammar

Jay Alammar creates visual explanations and interactive tools for understanding language models. His contributions include The Illustrated Transformer, the Ecco explainability library, and Hands-On Large Language Models, coauthored with Maarten Grootendorst. At Cohere, his work has also included helping companies and developers apply large language models to practical problems.[3][5][4][7]

O’Reilly published Alammar and Grootendorst’s Hands-On Large Language Models: Language Understanding and Generation in September 2024. Grootendorst describes their approach as “visual storytelling,” with almost 300 custom-made images and freely available code. The book also extends Alammar’s earlier Transformer explanation: his original post identifies Chapter 3 as an updated and expanded treatment covering developments such as multi-query attention and rotary positional embeddings.[7][8][3]

Their collaboration also includes Illustrated DeepSeek, which Grootendorst describes as a book about DeepSeek models, particularly DeepSeek-R1, created with Turing Education for a Chinese audience. His books page announces that the pair are writing An Illustrated Guide to AI Agents, covering memory, tools, MCP and context engineering; it describes that project as work in progress.[8]

Alammar joined Cohere a little less than a year before his March 7, 2022 account of the role. After working closely with pretrained Transformers for his blog and Ecco, he helped companies and developers put large models to use. He valued managed models because they freed his problem-solving work from restrictions involving model loading, deployment, memory and GPU management.[4]

He wrote and collaborated with colleagues on material that connected model mechanics to application decisions: distinguishing generative models from representation models, structuring prompts, building summarization systems, searching with sentence embeddings, fine-tuning representation models and controlling generation through top-k and top-p decoding. One semantic-search tutorial implemented a “similar questions” feature using sentence embeddings and Spotify’s Annoy vector-search library. He also singled out ranking, filtering and selecting among multiple generated outputs as an idea he wanted to explore further.[4]

Latent Space’s December 2023 NeurIPS recap featured Alammar as an Engineering Fellow at Cohere. A May 16, 2025 article, jointly credited to him and Edward Kim, listed his title as Director, Engineering Fellow.[1][9]

The explanatory work behind these applications and books began earlier: Alammar published The Illustrated Transformer in 2018 and presented Ecco’s model-exploration methods in 2021.[3][5]

Published on June 27, 2018, The Illustrated Transformer breaks down the architecture proposed in Attention Is All You Need for readers without deep prior knowledge. Alammar’s contribution is its visual, concept-by-concept explanation. He starts with an encoder–decoder overview, then follows the vectors through attention and feed-forward layers.[3]

His self-attention example asks what “it” refers to in “The animal didn't cross the street because it was too tired”. He explains how attention can associate the pronoun with the animal, then shows the calculation: query–key dot products produce scores, scaling and softmax turn those scores into weights, and a weighted sum of value vectors produces the output. Multiple attention heads allow different representation subspaces; decoder masking prevents attention to future output positions. He also distinguishes dependencies between positions in attention from the independently applied feed-forward network, where positions can be processed in parallel.[3]

Alammar credits Illia Polosukhin, Jakob Uszkoreit, Llion Jones, Lukasz Kaiser, Niki Parmar and Noam Shazeer with feedback on earlier versions. He continued revising the explanation: a July 2020 update clarified that the illustrated Tensor2Tensor positional encoding concatenated sine and cosine signals, while the paper interwove them. A 2025 update introduced an animated short course.[3]

Alammar developed Ecco to explore existing pretrained language models through interactive visualizations in Jupyter notebooks. Built on PyTorch and Transformers, it supports feature attribution, hidden-state inspection and analysis of feed-forward neuron activations. Its scope is exploring and understanding models rather than training or fine-tuning them. His Ecco paper appeared in the August 2021 ACL-IJCNLP System Demonstrations proceedings.[4][6][5]

In his January 19, 2021 article, Finding the Words to Say, Alammar uses Ecco to project intermediate hidden states into the output vocabulary. The resulting views show how candidate tokens’ scores and rankings change across layers, or trace the ranking of a selected token throughout a generated sequence. He credits related work by Voita and colleagues and Nostalgebraist’s logit-lens visualizations. These projections offer clues about processing rather than a complete causal account of a model’s decisions.[10]

One GPT2-Large example generates a list of European Union countries that incorrectly includes Chile. Alammar notes that Chile ranked 43rd among candidate tokens and interprets the mistake as better attributed to the sampling method than to the model itself. The example illustrates why inspecting candidate rankings can reveal more than reading the sampled output alone.[10]

The Ecco paper also reports that syntactic information could be retrieved from BERT’s feed-forward representations at levels comparable to its hidden-state representations. It found that hidden states accumulated syntactic information even when intermediate feed-forward representations showed diminished levels, extending the investigation beyond attention to the Transformer’s feed-forward sublayers.[5]