TurboPuffer

TurboPuffer builds vector and full-text search for AI applications. Co-founded by Simon Hørup Eskildsen and Justine Li, the Canadian, remote-first company separates compute from durable object storage and caches active data on SSDs and in memory. Its central architectural decision makes infrequently searched data inexpensive to retain while accepting slower queries when that data is first loaded.[4][3][5]

TurboPuffer durably commits writes to object storage rather than eventually copying cold data out of a separate disk database. Its query tier consists of compute and a caching hierarchy. Each search namespace occupies a prefix in object storage; if a query node fails, another can load the namespace into its cache. Routing queries to consistent nodes helps preserve cache locality, but those nodes do not hold the only durable copy.[5][7]

Cold-query latency is the principal tradeoff. SSD and memory caches accelerate frequently searched data, while an uncached query can take hundreds of milliseconds. The query planner and storage engine work together to limit sequential object-storage round trips, sometimes downloading more data in each trip. This arrangement particularly suits workloads with many tenants whose datasets are active only intermittently. Eskildsen cautioned in March 2026 that TurboPuffer's write latency also made it unsuitable for high-transaction workloads.[5][1]

The architecture grew out of an earlier retrieval-cost problem at Readwise, alongside Eskildsen and Li's experience operating Shopify infrastructure.[5][3]

Eskildsen spent almost a decade on Shopify's infrastructure team, beginning around 2013. He and Li worked closely for eight years on its database and compute layers. After leaving, he undertook what he called “angel engineering”: roughly three-month stints helping friends' companies with infrastructure in exchange for equity. Readwise, Replicate and Causal were among those companies. He recalled self-hosted Elasticsearch around 2015 as particularly difficult to scale and operate, while acknowledging that this experience concerned an older version.[1][3]

In late 2022, Eskildsen helped Readwise prepare its infrastructure for the launch of Reader, its read-it-later app. The team wanted article recommendations and semantic search using embeddings. In his founder account, Readwise's relational database cost approximately $5,000 a month, while projected vector search over the same 100 million-plus documents would have cost more than $20,000 a month. The team shelved the feature. Development of TurboPuffer began in 2023, with object storage and modern NVMe drives supplying a different cost structure.[5]

Eskildsen reports that TurboPuffer launched in October 2023. Its first large customer was Cursor. Cursor's workload fit the architecture: caching only active codebase indexes avoids the cost of keeping every index in memory. Eskildsen recalled launching with a Rust binary on a single eight-core machine, then visiting Cursor after its founders contacted him. He brought Li on as co-founder during this period. Together they shipped the features Cursor needed and supported its migration. The company places the migration in November 2023 and reports a 95% cost reduction.[10][5][1]

Notion became the second large customer after Cursor. In his March 2026 interview, Eskildsen placed its cross-cloud challenge around mid-2024: Notion ran on AWS, while TurboPuffer had started on Google Cloud. Google Cloud Storage supported the compare-and-swap primitive TurboPuffer used to update metadata safely; S3 did not yet offer the needed primitive. Compare-and-swap lets a node write modified metadata only if the original has not changed, retrying otherwise. Rather than introduce a separate stateful metadata service, the team retained its Google Cloud deployment.[1]

Eskildsen recalled buying dark fiber, tuning TCP windows, prewarming connections and absorbing cross-cloud costs to meet Notion's latency requirements. He attributed the preference for fewer stateful systems to his and Li's shared on-call experience: outages were hardest when state in multiple places stopped agreeing. His succinct explanation was, “It doesn’t have state, I don’t want state and two systems.” He said the Notion workload still ran on Google Cloud at the time of the interview.[1]

TurboPuffer began with vector search and subsequently added full-text search. Its FTS v2 account, updated February 3, 2026 and authored by Adrien Grand, Morgan Gallant and Nikhil Benesch, described the upgrade as live for all customers. A redesigned inverted index reduced on-disk size by a reported tenfold and added metadata for skipping irrelevant postings. Lucene-style MAXSCORE pruning improved handling of long queries generated by agents. The company reported gains of up to 20×, with the greatest benefits on large datasets, low result counts and queries containing frequent terms; its examples used slightly more than five million English Wikipedia documents with top_k=100. Introduced features included Unicode-aware tokenization, conditional boosting, prefix queries and regex filtering.[8][1]

ANN v3 extended vector search to a single index containing 100 billion vectors. Chief architect Nathan VanBenschoten's technical account, updated May 5, 2026, described hierarchical clustering to narrow the search space and RaBitQ binary quantization to reduce bandwidth demands. The system first searches compressed vectors, then fetches and reranks full-precision candidates. Distributed shards supply the SSD capacity, and queries combine their results into a global top-k. For the described workload—1024-dimensional f16 vectors, approximately 200 TiB of dense data—the company reported 200 ms p99 latency with the index cached on SSD. Its theoretical 10,000 QPS calculation differed from observed saturation around 1,000 QPS in the discussed production load, where CPU work became the bottleneck. Eskildsen credited VanBenschoten's development work and assistance from much of the team.[7][1]

By March 2026, Eskildsen described a workload shift from one retrieval call before generation to a single agent issuing many concurrent searches. He announced a fivefold query-pricing reduction to accommodate the increased volume. Deployment options also varied by customer: he described Cursor using the shared SaaS service, Notion using a single-tenant cluster, and a bring-your-own-cloud option running inside a customer's VPC. Despite interest in aggregations, logging, time series and graph queries, he said search remained the reason to adopt TurboPuffer and warned against expanding the team's scope too quickly.[1]

Readwise later used TurboPuffer to ship Chat with Highlights in January 2025, combining vector and full-text retrieval across users' saved highlights. A July 2026 case-study entry described its migration to native query embeddings without re-embedding existing documents. Its indexed columns used qwen3-embedding-8b with dimensions truncated from 4096 to 1536, and TurboPuffer automatically matched query embeddings to the column dimensions. The vendor-hosted case study reported Readwise's median embedding latency falling from 1.58 seconds to 0.20 seconds—an approximately eightfold improvement in that step, rather than total chat latency.[6]

In a September 4, 2026 partnership post, Applied Compute's Edward Yu and Jacob Phillips described post-training Qwen3.6-35B-A3B with GRPO on search tasks drawn from approximately 9,000 open-source repositories. Narrow tasks required locating a repository and extracting implementation details; open-ended tasks required finding examples of a code pattern across repositories. The harness combined filesystem tools with TurboPuffer indexes built from AST-aware chunks, BM25 keyword retrieval and Octen-Embedding-8B vectors, followed by Qwen3 Reranker 4B. Rewards incorporated correctness, citation support and efficiency.[9]

The partners reported that the trained agent with TurboPuffer search finished 16% ahead of the bash-only agent on the narrow task and 25% ahead on the open-ended task. Increasing the corpus from 20 to 300 repositories raised ripgrep's tool latency elevenfold, compared with 1.2× for TurboPuffer search. Their specialist model led the tested frontier models on the narrow task's correctness-and-citation metric, but did not reach frontier-level quality on the open-ended task. Reported inference savings excluded the full costs of storage, indexing and maintaining changing corpora.[9]

Eskildsen said Cursor's initial adoption preceded outside funding and that investor Lachy Groom was involved by the time Notion adopted TurboPuffer. Early cloud bills ran on his credit card, creating immediate pressure to optimize the service. He recalled choosing Groom because he could speak openly with him, including a promise to return the investment if the company failed to find product-market fit by year-end. Taking the investment marked a deliberate commitment to building the company and accepting responsibility to employees and investors.[1]

A May 22, 2026 report relayed Eskildsen's announcement that TurboPuffer had crossed $100 million in run-rate in March 2026, 19 months after reaching $1 million, while profitable and with less than $1 million raised. His embedded announcement described a 34-person team and said Cursor searched more than one trillion code chunks on the service. These were founder-reported announcement-era figures; the financial milestone used run-rate rather than full-year revenue.[2][10]