Shopify’s AI Phase Transition: What Happens After Code Gets Cheap

An agent should not rediscover yesterday’s work

An agent researches a problem, loses its context between runs, then repeats the work. Early in the conversation with Shopify CTO Mikhail Parakhin, that practical problem introduces two approaches to agent memory: Tobi favors QMD, while Parakhin prefers SQL and SQLite. Both aim to store information agents can retrieve on subsequent runs.

The larger change is how widely those agents are used. Parakhin says daily active AI-tool users approach 100% of Shopify employees at the time of the discussion. He places the inflection around December: incremental model improvements accumulated until the tools became useful enough for adoption to accelerate sharply.

Command-line tools and agents that do not require users to inspect code are growing fastest, including Claude Code, Codex, Pi and Shopify’s internal coding agent River. GitHub Copilot and Cursor are not necessarily losing users; their growth is slower. Employees choose their tools, with effectively unlimited token funding and a model-quality floor: Parakhin says Shopify discourages anything below Opus-4.6, with GPT-5.4 extra high another option in use.

Adoption is broad, but consumption is increasingly concentrated. Token usage was already growing exponentially before the inflection; now the heaviest users’ consumption is growing faster than lower percentiles. Parakhin is uncertain whether that widening gap is healthy. His hypothetical endpoint—one person consuming all the tokens—questions the trajectory rather than predicting an outcome.

Spend on critique; optimize time to deployment

Parakhin supports substantial AI spending, but token consumption is not the objective. Shopify learned that too many parallel agents without communication can burn tokens while adding little value over fewer agents.

His preferred critique loop gives the agents a sequence: 1. One capable agent produces the work. 2. Another, ideally using a different model, critiques it and proposes improvements. 3. The first revises in response. He reports substantially better code quality, at the cost of longer waits while the models take turns.

That tradeoff becomes consequential at pull-request review. Even if a model produces fewer bugs per unit of code, much greater output can still increase the total reaching production. Parakhin claims good models already compare favorably with average humans, without supplying a benchmark. His operational priority is review capacity that keeps pace with generation: he watches the ratio of generation spending to review spending and advocates expensive reviewers such as GPT-5.4 Pro and Gemini Deep Think, without prescribing a target ratio.

Shopify uses an internal review tool because available products do not fit its preferred approach: large models taking turns over an extended period. This can produce relatively few tokens while taking substantial time—a combination the respondent considers poorly aligned with users’ desire for fast responses and some vendors’ business models.

The host cites pull-request merge growth rising from 10% to 30% month over month, alongside increasing estimated complexity; neither the measurement window nor the complexity method is specified. More code also means more chances for tests to fail. Teams must identify an offending change, remove it and retest. The respondent’s argument is that an hour of review can shorten the whole deployment cycle by avoiding that downstream work, though no measured time comparison is given.

Shopify uses Graphite and stacked pull requests, yet its integration and deployment pipelines and repository interactions remain major bottlenecks. The host compares merge conflicts to a global mutex: a shared coordination point that becomes more restrictive as code arrives at machine speed. Stabilizing the workflow comes first; no compelling replacement has emerged. The respondent even tentatively reconsiders microservices, whose independent releases might reduce contention if automation can also manage their operational complexity.

Tangle makes experiments reusable across teams

A notebook workflow illustrates the next bottleneck. A practitioner exports data, adds cleanup and bot-filtering scripts, then runs experiments. Undoing a change fails to reproduce an earlier result. Reimplementing the workflow for production introduces mismatched feature names. Six months later, changed paths and forgotten folders make even a rerun difficult. An agent cannot reconstruct the work reliably if it is handed the wrong folder.

[Tangle](https://shopify.engineering/tangle) organizes data processing around collaborative experimentation. The respondent contrasts its focus with Airflow’s scheduled production execution: clone a pipeline, change a small component, run experimental variants and share results without first understanding the entire system. Once ready, the same experiment runs in production, avoiding a separate implementation and its opportunities for drift.

Tangle is described as open source, language-independent and visually composable, with full versioning for reproducible runs. Content-based caching avoids unnecessary computation: if a changed component produces the same output, downstream steps need not rerun. Concurrent experiments that require identical preprocessing share that execution without their authors coordinating it.

The largest claimed savings come from reuse across people and departments. One team finishes a task; another later benefits from its completed computation, even across versions when outputs remain identical. With everyone using Tangle, duplication that would otherwise be invisible becomes detectable. That is its network effect: one team’s work makes another team’s experiment cheaper.

Tangent turns objectives into repeated experiments

[Tangent](https://apps.shopify.com/tangent-1) adds an automated research loop to Tangle. An agent runs experiments, analyzes results, modifies the work and repeats toward a measurable objective. It can recombine existing components, such as XGBoost and PyTorch modules, or inspect code and create new ones. Tangle’s components are wrapped command-line calls with YAML files, giving the agent concrete implementations to modify.

The optimization examples span several workloads: - Search serving: index-serving code changes reportedly raised throughput from 800 to 4,200 queries per second with the same quality and number of machines. The benchmark workload, quality metric and precise role of the research loop are unspecified. - Latency: applications include faster Liquid themes and improved gisting, a prompt-compression technique associated here with lower latency and slightly higher quality. - Storage: agents found derivative datasets that did not need duplicate storage, including a large table mapping random IDs to other random IDs.

Usage reportedly spread from machine-learning engineers to product managers; one product manager was the organization’s highest user at the time. Domain expertise can guide experiments without manually implementing every change. Automated machine learning, or AutoML, already supplied parameter-search techniques inside Tangle. The proposed advance is broader flexibility: large language models can analyze unstructured information and improve individual stages of an otherwise familiar experimental loop.

The respondent’s qualification concerns the kind of improvement to expect. In their experience, autoresearch finds overlooked optimizations and applies standard practices people lack time to try. It is weaker at unfamiliar ideas requiring days of original thought. That assessment is explicitly personal and provisional.

A hobby project made the tradeoff concrete. After years of optimization, the speaker expected an automated run to find nothing. Several weeks and more than 400 experiments at production-like scale yielded one successful improvement. The speaker believes their own hit rate would have been higher, but estimates doing 400 experiments personally would have taken roughly three years. Low experimental yield still saved substantial human work.

SimGym grounds simulated shoppers in history

The next step is to combine reusable experiments and automated research with a customer simulator. [SimGym](https://apps.shopify.com/simgym) uses Shopify’s historical records connecting merchant changes to subsequent sales to optimize simulated shoppers. Those outcomes provide a behavioral target beyond what a prompt tells an agent to do. The respondent sees the combination of Tangle, autoresearch and SimGym as enabling work that would previously have required prohibitive human effort.

The historical records are noisy: merchant changes rarely happen in isolation or in controlled A/B tests, and versions often run at different times. Shopify aggregates observations, denoises them and applies a collaborative-filtering-like approach to extract signals for agent optimization. Almost a year went into this work. 0.7 correlation with add-to-cart events was an internal target, intended to align simulated outcomes with real experimental successes and failures; an achieved score and calculation method are not reported.

The agents must also encounter visual interface friction. Enlarging a product image changes the shopping experience while leaving most HTML text unchanged. The speaker says larger images often hurt sales, without giving an effect size. SimGym therefore combines visual inputs and browser actions, using a large browser farm and a capable multimodal model to assess what shoppers actually see.

The original product addressed small merchants’ lack of traffic: a week might not yield enough observations for a statistically significant test. The team then found an earlier obstacle: many merchants have no second design to test. SimGym evolved from comparing alternatives to examining an existing site or theme, predicting conversion values and suggesting modifications. That single-site workflow had recently launched at the time of recording.

Repeated simulations with many agents are expensive. As usage grew, priorities shifted toward model optimization, distillation and cheaper browser execution. Multi-instance GPUs initially suited the workload poorly, but making them work was considered necessary to control costs. The tuning depends on the objective: maximum throughput, minimum latency and minimum cost can require different configurations.

Model the journey, then compare interventions

An established store’s own history makes simulation more specific. Shopify reportedly creates agents that reproduce its previous customer distribution, then tests changes against them. The respondent says this substantially improves correlation with add-to-cart and conversion events compared with newly launched stores, without providing a numerical comparison. Relevant behavioral data remains central to the approach.

The host proposes thinking in trajectories: customer journeys whose next steps depend on earlier states. The response extends that idea to whole businesses. Shopify reportedly has a large system modeling merchant companies and their possible paths, representing each company as an entity taking actions over time. Its model is identified as HSTU, though the architecture is not explained in the conversation.

Counterfactual simulation compares modeled futures with and without an intervention. Insert a coupon, thank-you card or discount into a journey, then simulate forward. Moving the intervention earlier or later compares timing as well as action. Strong signals reportedly feed Shopify Pulse recommendations, such as fixing an apparent Canadian-sales misconfiguration or launching a campaign with particular parameters. Buyer-level applications include discounts and cashback.

Shopping categories differ in customer sensitivities and priorities. Shopify aggregates behavior at store and category levels and has returned to Chinese Restaurant Processes, described here as a way to grow clusters for studying those differences.

Catalog access moves product selection to runtime

A release discussed for the recording week brings Shopify catalog capabilities into [UCP](https://www.shopify.com/ucp), the commerce protocol discussed here: product search, lookup by a specific ID and bulk lookup. Applications can choose products at runtime instead of knowing beforehand which items to display, sell or check out. Personalized and non-personalized search are investment priorities, alongside identity linking—including Google identity—to reduce friction.

Liquid models: low latency online, high throughput offline

[Liquid AI](https://www.liquid.ai/blog/liquid-ai-announces-multi-year-partnership-with-shopify-to-bring-sub-20ms-foundation-models-to-core-commerce-experiences) enters the discussion through workloads needing small models, low latency or long context. The respondent characterizes Liquid models as more expressive relatives of state-space models, or SSMs, which carry prior information in a compact state. Their ability to process an ongoing context depends on that state retaining what matters; it does not imply unlimited faithful recall.

The architectural account is brief: Liquid uses a compact representation with subquadratic context-length scaling, described through a differential equation rolled out and computed as a convolution. The practical claim is that Liquid performed best among the models Shopify tried for these small-model workloads, with alternatives under continued evaluation.

Online query understanding prioritizes latency. A roughly 300-million-parameter model reportedly interprets a search query in about 30 milliseconds end to end, later described as below 30 milliseconds. It generates possible meanings beyond synonyms, incorporates previous queries for personalization and passes its interpretation to the search server. Achieving that result required substantial optimization with NVIDIA because overheads negligible for large models became consequential. Hardware, sequence lengths, concurrency and latency percentiles are unspecified.

Offline catalog processing prioritizes throughput: assign product categories, extract and normalize attributes, and identify equivalent products sold by different merchants. The respondent describes billions of products and an approximately quadratic matching problem handled through large batch jobs. For catalog and Sidekick Pulse tasks, Shopify distills very large models into task-specific Liquid models of roughly seven to eight billion parameters; some tasks also require long context.

These distilled models reportedly often outperform Qwen on Shopify’s batch workloads and are taking internal workload share, although no precise comparison metric or protocol is given. Qwen remains another internal standard. Liquid’s demonstrated role in this account is a specialized distillation target; the suggestion that Liquid–transformer hybrids could rival frontier models with comparable compute remains the speaker’s architectural opinion and extrapolation.

Sidekick Pulse also uses the enterprise-level counterfactual model described earlier. The closing hiring discussion highlights continuing work in machine learning, data science, data management and distributed databases, including collaboration with Yugabyte and exploration of how language models could change distributed database design.

Sydney’s personality—and the value of a platform

The final historical detour concerns Sydney. In the respondent’s recollection, it first shipped in India and went largely unnoticed for a long time. Its initial implementation used a Microsoft–NVIDIA collaboration model described as Megatron, rather than an OpenAI model. Some early users suspected a prank or hidden human operators; others complained of gaslighting.

Sydney’s personality was deliberately shaped, drawing on earlier experience developing Yandex’s Alice assistant. The speaker says experiments favored politeness with a slight edge that drew people in. Andrew McNamara, identified as Sydney’s first development lead, was leading Sidekick at Shopify at the time of the conversation, with Pulse also within his responsibilities.

When the host suggests a future Shopify Sydney, the response points to Sidekick as a somewhat different product. The closing exchange returns to platform value: undertaking technical work at a scale individual merchants could not reproduce themselves, then adding network effects. Shared experiment results and behavior-informed simulation give that principle concrete form.