Exa’s Neural PageRank: From Predicting Documents to Finding Every Match

The striped-shirt problem

Ask for shirts without stripes and get striped shirts back. Will Bryk uses this anecdote to explain the opportunity behind [Exa](https://exa.ai/): search can recognize a request’s words while missing its decisive constraint. In summer 2021, he saw GPT-3 handling paragraphs and linguistic subtleties that he felt Google missed. Exa entered the startup accelerator YC with the ambition of applying large language model (LLM) advances to search.

Bryk’s interest began with wanting better news consumption in high school, followed by building a small search engine in college. His technological optimism sometimes ran ahead of deployment: he postponed learning to drive because he expected autonomous cars sooner. By the interview, he reported riding in them nightly, roughly a decade later than he had expected.

He describes Exa’s early release as the first expression of a research project: build a better search algorithm, then develop applications around it. The organizing ambition was perfect web search. Products would follow from improvements in the underlying capability.

Train on the documents people point to

Exa’s original link-prediction objective uses existing web links as training examples: 1. Find a link and take the text surrounding it. 2. Hide the link. 3. Train the model to predict the document it references. Bryk’s hypothetical example is an article recommending an aerospace startup and linking to spacex.com. The recommendation supplies the input; the linked document supplies the target. At search time, a similar description can retrieve likely documents.

Document prediction is the more precise name: the task retrieves a document rather than generating a URL’s characters from memory. Bryk argues that repeating it billions of times can produce a search engine. He describes the architecture as transformer-inspired, with changes needed to make document prediction work, but leaves those details unspecified.

Exa had already moved beyond that initial model. Bryk compares its progression to language models that start with a self-supervised objective—learning from targets already present in the data—then add synthetic examples and supervised fine-tuning for greater control and robustness. This is an analogy for the progression, not a disclosure of Exa’s subsequent training recipe.

The model sits inside three connected systems: - Crawling: discover URLs and fetch their pages. - Indexing: process documents with embedding models or more complex models into a searchable representation. - Serving: use a vector database to retrieve from that index with high throughput and low latency. Exa reports typically assigning one or two people to each major subsystem. Learning which documents fit a description is only one part of making those documents available at search time.

A complete list changes the compute budget

Exa’s shirt slogan, 10^18 > 10^100, turns its naming story into a search philosophy: return exactly the matching items. Someone seeking hardware startups in San Francisco wants the qualifying companies, not an enormous advertised result count. If there were 325 matches, the goal would be all 325. That hypothetical count illustrates the ambition of the list-building product, which was still forthcoming when the conversation was recorded.

The interviewer separates two problems. Category interpretation asks whether companies describing themselves as robotics or wearables belong in a hardware search. Completeness asks whether every qualifying company has been found. The proposed control for interpretation is a feedback loop: show examples, let users judge them and eventually offer relevance thresholds. Those judgments define a classifier—a rule for deciding which items belong—for the particular request.

The proposed route to completeness is more computation. Finding every blog post about Swyx’s podcast from the past year could require much more searching than a simple lookup. The phrase o1 for search captures this idea: allocate work according to query difficulty. Running GPT-4 over the entire web to classify every page is the limiting thought experiment. It illustrates the expense of exhaustive evaluation, but leaves the practical question open: how does the system measure completeness and know when to stop?

This changes what users might accept from search. A minute, ten minutes or even a day could be worthwhile for a sufficiently useful list. The described workflow offers a preview before launching a larger job, with a compute limit at recording time. User-specified budgets are part of the direction; their bounds, exhaustion behavior and payment to continue remain product-design questions.

Reasoning needs access to knowledge

Venture sourcing makes the value of a list concrete. One participant describes gathering signals from GitHub, Twitter and Product Hunt to identify companies for evaluation and outreach. Exa aims to serve developers building those workflows while also offering a web search interface where people can log in, buy credits and get results before building an API integration.

The broader architectural argument separates reasoning ability from access to knowledge. Even a highly capable model could struggle to list every PhD in New York City who has thought about search if its tools cannot retrieve the relevant information. The proposed solution pairs a smaller language model with a separately optimized search system. The hoped-for efficiency comes from specializing each component instead of requiring the language model to memorize the web; no particular model size or efficiency gain is established.

Exa describes its approach as end-to-end neural search. Its competitive thesis is that incumbent engines’ combination of broad coverage, short response times and low serving costs constrains the algorithms they can deploy. Starting with neural models creates different research opportunities. The accompanying characterization of Google and Bing as tied to older keyword-oriented systems is a competitive assessment, not a measured comparison.

A question about [Perplexity](https://www.perplexity.ai/) and ChatGPT Search exposes a useful distinction: a web cache is not a search algorithm. A URL → document mapping provides content for a known address. Selecting relevant documents from a query is a separate capability. A service can own a crawler and cache while using another provider to choose results. The hypothetical Bing integration in the discussion illustrates this separation without establishing either competitor’s complete architecture.

Neural PageRank learns importance and description

The link-prediction objective returns under the name Neural PageRank. Predicting the documents people share gives frequently referenced material a greater chance of retrieval, incorporating a signal of importance. A Paul Graham fundraising essay illustrates the mechanism: the model can learn both that people repeatedly reference it and the many ways they describe it. This is the proposed advantage over PageRank, rather than a demonstrated performance result.

Popularity raises an obvious question: what about excellent essays with few links, perhaps shared by unusually knowledgeable readers? The proposed response is to curate high-quality training sources and adjust the distribution of examples. A separate mechanism for assessing each sharer’s authority is not specified. The same input-quality concern extends to search-backed language models: poor retrieved material can degrade the answers generated from it.

For application builders, retrieving a URL is only part of the job. Customers want full page content, cleaned and parsed into Markdown and sometimes divided into chunks. Exa describes serving content for batches ranging from ten to a thousand URLs. The feature began modestly: the first version retained each page’s first 1,024 tokens for debugging, making it easy to start returning content when users requested it.

Combining search and scraping simplifies that workflow, though Exa acknowledges its scraper is imperfect. Crawling restrictions also make coverage harder. Its proposed response combines searching the long tail of open sites with data partnerships, which it expects to become easier to arrange as the company grows.

Useful lists still need the right intent

Dating appears as a speculative application: public profiles and writing might help people find partners with shared interests or ideas. Its proponent acknowledges the Silicon Valley framing. The customer and internal examples are more concrete: - Research writing: a student writing assistant retrieves relevant papers, then summarizes and generates text with roughly 20 papers in the model’s prompt. - Company discovery: investors map sectors such as AI for law or construction. One customer reportedly used Exa for eight-hour stretches over several days. - Sales prospecting: Exa reports using its own search to find writing-assistant companies and attributes many customers to that process.

Recruiting extends the same pattern to people who have written about search, using blogs and some indexed tweets. This leads to a broader argument about publishing incentives: what search rewards influences what people create. Exa’s ambition is to reward substantive writing that matches a reader’s intent, encouraging authors to publish work that the right people can discover.

On the query side, users accustomed to a few keywords may never try a detailed information request. Yet more words do not automatically resolve intent. In a live search, a participant asks for people writing about learning in public and reports results about public schools, rather than the online practice associated with Swyx. A Google comparison uses different wording and remains inconclusive. The useful finding is ambiguity in the request, even when its words are understood.

The proposed interface uses an LLM to translate conversation and knowledge of the user into a detailed search request. Auto prompts are described as a lightweight precursor. The language model supplies context that a person may not bother typing; Exa’s task is to handle the resulting paragraph-length query.

A Lindy workflow for fantasy-football waiver recommendations introduces another distinction. Different websites recommend different players: should search collect those recommendations or decide which are best? Filtering determines whether a document matches the request; ranking orders the matches according to preferences. Exa’s stated first priority is reliable matching. Explicit ranking criteria—website visitor counts or company employee counts, for example—make best operational. Without them, the proposed fallback is to infer typical preferences.

Preview, refine, then launch

Exa’s developing search system combines an algorithm with an agent: it takes actions, decides when to investigate further and runs in a loop. That raises a disagreement about autonomy. One participant argues that submitting a paragraph and receiving a finished answer can disconnect users from research and reduce trust. Another argues that the right degree of involvement depends on capability: a sufficiently strong agent might eventually be slowed by human intervention. Both agree that capability has not yet been reached.

For a ten-minute search, the practical compromise is preview, refine, then launch. A preview can reveal a misunderstanding before the system spends the full budget. Once the intended result set is clear, the user can launch a batch job. This addresses a failure mode separate from insufficient capability: an agent may execute the wrong interpretation because it lacks context that another human would assume.

Visible assumptions and intermediate work would help users steer the search. A system prompt—the instructions setting the model’s general behavior—is compared to choosing a collaborator with a preferred working style: it cannot communicate every task requirement. One participant also criticizes ritualized instructions to avoid hallucination, arguing that such behavior should be trained into models rather than repeatedly requested.

Asked about searching company documents, Exa reports demand but emphasizes staying focused on web search. Personal and company information belong to the larger vision of search over everything, beyond the capabilities described here.

Who can judge a search result?

The later discussion of o1 turns to reward-driven self-training: generate coding or mathematics tasks, grade attempted solutions and train on successful examples. Even sparse successes could supply useful training data. One participant sees this as a path toward solving programming and mathematics—an optimistic extrapolation, not an established result or a verified account of o1’s training.

Improving a model through training is distinct from spending more computation while it answers a request. A stronger model need not always deliberate for minutes. The prediction is an ecosystem of different model sizes and providers, with latency-sensitive applications continuing to favor fast responses.

An interviewer proposes collecting human research trajectories: pay graduate students to investigate questions and record their search sequences. The response is skeptical of human-labeled search data. In an internal Exa exercise, engineers struggled to judge niche results, including biological experiments outside their expertise, and fell back on keyword overlap. That shortcut could miss the document’s meaning.

Exa reports using LLMs as relevance labelers, arguing that they can make judgments difficult for annotators without domain knowledge, though no quantitative comparison is given. The underlying distinction is between solving one deep research problem and asking relatively simple questions about billions of items: is this a startup, or did this person write about search? The latter is the search problem Exa emphasizes.

The hardware—and where to spend the computation

The closing discussion turns to Exa’s intense but playful culture and its practical approach to office decisions. Nap pods supply the memorable example: one participant describes closing their eyes for about twenty minutes to rest and think through problems. A pod ordered from China to save money arrived weighing roughly 500 pounds and required a passerby’s help to unload, in a maneuver the speaker acknowledges was dangerous. The motivation offered for the company’s intensity is the enjoyment of building something consequential with friends.

At recording time, Exa reported hiring, growing quickly and purchasing a $5 million H200 cluster. It used a mix of owned hardware and AWS, with an existing A100 cluster and cluster capacity devoted to training and research. The exact allocation of inference workloads and the training-versus-inference budget split remained unspecified.

The final economics question brings the architecture into focus. Advertising-funded search must keep request costs below the revenue those requests generate. The design described in response moves expensive processing ahead of the query: preprocess indexed items, usually with LLMs, because evaluating billions of documents with a model for every request is infeasible. At query time, reranking evaluates and reorders a much smaller candidate set.

Candidate count and transformer size provide two cost controls. Ten, thirty, a hundred or a thousand candidates are offered as possible reranking scales, not a production configuration. The closing argument is that falling LLM costs justify redesigning the search algorithm itself. The speaker’s approximate 200-fold decline has no specified model or workload baseline, but the design question is concrete: where can cheaper computation make richer preprocessing, broader evaluation or longer searches worthwhile?