NeurIPS 2023 Startups, Part 2 of 4: Search, Shipping, and Finding Talent

Cursor: growing without hiring too quickly

Cursor had seven people, six of them engineers, and more code-generation problems to tackle. Its response was to hire carefully and slowly. In this part of the NeurIPS 2023 startup conversations, the Cursor representative argues that companies often expand too quickly. The recruiting pitch rests on the belief that a small, strong team could make major progress in code generation.

The host closes the exchange by highlighting large language models (LLMs) as rankers or judges for synthetic data: models assess candidate outputs as well as generate them. He presents this as a more favored direction than directly distilling model outputs, without establishing a technical comparison between the approaches.

Perplexity: turn fresh web information into supported answers

Perplexity’s mobile rollout exposed an ordinary product problem: the apps were slow and crashed. Aravind Srinivas says user complaints prompted the company to recruit a stronger mobile team, producing faster, more reliable apps. The implementations were native; he considered that necessary for the responsiveness he wanted, though he offers no measured comparison with a shared React stack.

Srinivas confirms a milestone of one million people on Android and another million on iOS. The metric remains unspecified: the exchange does not distinguish downloads, registrations, or active users.

The developer offering, `pplx-online`, supplies web information to applications. Srinivas described it at the time as having no knowledge cutoff and claimed it was the only such language-model API. Its 7B and 70B versions were fine-tuned from Llama 2 for a specific task: combine multiple sources into an accurate, readable summary. Perplexity was not pretraining these models; support for the recently released Mixtral was planned.

Freshness also raises a question of source quality. Srinivas says he would welcome an API for Twitter information but does not regard its absence as a major disadvantage: shared links often originate elsewhere, and noteworthy opinions spread into journalism and other web content. Access alone does not settle what deserves inclusion; he distinguishes a curated human artifact from a tweet.

Sentence-level citations expose the evidence behind an answer, even when one article supports an entire paragraph. Srinivas connects this choice to his academic writing practice. The host identifies a remaining usability gap: following a citation can leave the reader searching a long page for the supporting passage. Direct navigation to that passage was still under consideration.

Serving your own model requires real traffic

Replacing a hosted model API with an internally served model requires operational testing. The Perplexity discussion identifies several ways the transition can fail: - Hardware failures: GPUs can crash and nodes can stop working. - Orchestration failures: Kubernetes may not behave as expected. - Latency–throughput tradeoffs: reducing the time an individual request takes can reduce the system’s capacity to handle requests, leaving it unable to absorb traffic spikes.

Perplexity Labs provides a public playground where users try open-source models and in-house models fine-tuned for factual accuracy and helpfulness. Actual requests expose infrastructure problems, and the lessons inform production deployment. The API serves a similar purpose while offering developers smaller, faster model alternatives. The respondent does not expect these APIs to displace GPT-3.5 APIs.

The interviewer challenges the testing argument: the main app receives far more traffic than Labs. The respondent agrees, but argues that new infrastructure cannot simply be shipped to the main app and that simulations cannot fully reproduce real use. Labs is a staging step with real users, not proof that the system can already handle production volume.

Labs also displays tokens per second and time to first token, making serving speed visible and helping attract infrastructure engineers. Yet infrastructure exists in service of the product: Perplexity buys GPUs primarily for its own application and does not position itself as competing with Together AI, Fireworks, or OctoML.

The respondent’s market argument explains that choice. As NVIDIA tooling standardizes serving stacks, providers offer increasingly similar services. In this view, competition shifts toward GPU access, economies of scale, and the substantial capital needed to subsidize prices and capture market share.

An answer engine shaped by startup incentives

Perplexity’s ambition was an answer engine that more people would use as a Google replacement. It reported operating its own crawlers and indexes. Its hiring pitch tied that ambition to short production cycles: clearly scoped projects, concrete deliverables, and announcements roughly every two weeks, with researchers training and shipping models into consumer search.

Srinivas attributes Google’s difficulty to unclear incentives and the prospect of cannibalizing its existing business, while acknowledging its technical talent. He agrees with the interviewer’s description of an innovator’s dilemma: a new product can threaten the business an incumbent depends on. Perplexity relied primarily on subscriptions and had no advertising—as of the day of the interview, Srinivas explicitly qualifies.

The host closes with Paul Graham’s Fierce Nerds concept, introduced through a My First Million clip about Dharmesh Shah: social quietness can coexist with intense competitiveness in someone’s own domain. He applies that characterization to Srinivas and Perplexity before turning to a different search strategy.

Metaphor: improve the results beneath the interface

Metaphor takes the infrastructure side of the search opportunity. Will Bryk describes an internet search engine trained from scratch for complex queries. He expects language models to become interfaces to information and wants developers to build those interfaces on Metaphor’s results. His claim that it handles complex queries better than Google is qualitative, without a benchmark or query-set evaluation.

One use case is finding nearby researchers with similar interests. Bryk reports that people had met in real life through these searches, crediting relevant results free of search-engine-optimization spam. He praises Perplexity’s combination of language models and search results while defining Metaphor’s priority as improving the underlying results.

Bryk initially sees Metaphor as complementary to Google and Bing, with a division based on the query: - Known keywords: conventional search should work well when an application knows which terms to use. - Complex natural-language requests: Metaphor targets requests whose intent is harder to express through keywords.

That ambition comes with an unusual input format. Metaphor is trained to predict links given text, so its preferred prompt resembles text that a link would naturally follow. Its auto-prompt feature uses ChatGPT to rewrite an incoming query into that form. Users can make ordinary requests while the search model retains its autocomplete-style interface.

The team reports that trying reinforcement learning from human feedback, or RLHF, reduced the model’s capabilities. It therefore preferred automatic query rewriting, though Bryk supplies no quantitative evaluation of that tradeoff. He also clarifies what link prediction means here: Metaphor searches an index rather than generating links. He prefers “search engine” or “research engine” to “search LLM.”

The longer-term goal is to answer requests with many simultaneous constraints—for example, researchers similar to a given person, in the Bay Area, with Rust experience, a specified college background, and suitability for a startup. This is an aspiration, not a demonstrated result. Bryk imagines customers combining such search with GPT-4 or a future GPT-5, with speed a central challenge.

The interview ends with Metaphor’s social gatherings, inspired by Jeff’s viral tweets: an anti-Huberman party with late hours and beer, and a “ghosted in SF” event where attendees coded together.

Two integration boundaries for web search

In the closing search commentary, the host organizes ChatGPT and the GPTs announced at DevDay around a language-model base with three added capabilities: - Retrieval over uploaded files: use supplied documents when answering. - Sandboxed code execution: generate code, run it, and correct it. - Web search: obtain information from the open web.

For developers adding the third capability, the two search APIs draw different boundaries: - PPLX integrates the language model and search. Developers receive the combined offering. - Metaphor supplies search. Developers bring their own language model. The host considers open-web search less contested than retrieval and code execution at that moment, naming Perplexity and Metaphor as leaders in his assessment.

Answer.ai: fund the work that makes AI useful

Jeremy Howard joins on the day Answer.ai is announced, alongside Tanishq Abraham and Jess Leao, who introduces herself as a Decibel partner. Howard frames the organization as a continuation of fast.ai’s mission to bring AI to more people. Its earlier funding arrangement had become unsustainable: he says he and his wife had paid for fast.ai themselves, without donations or revenue.

Howard says he initially approached Alessio for advice. Leao and her partner John joined the conversation and offered investment. Leao describes Howard’s existing work as reason to move quickly without a conventional pitch process.

Answer.ai’s agenda starts with practical conditions for adoption: AI needs to be cheaper, faster, easier to use, and integrated into everyday activities. Howard invokes Edison’s invention factory as inspiration for turning an enabling technology into useful applications. He wants deep technical experts who can identify and fill capability gaps, coupled with the curiosity and playfulness to imagine new products.

His intended collaborators span open-source communities, leading academics, and places where talented people rarely get opportunities. Recruiting would give people ways to demonstrate ability through work, including Kaggle-style challenges and open-source bounties, to uncover skills that conventional hiring overlooks.

Recognize persistence—and the conditions that sustain it

Howard presents Abraham, who says he is 20, as an example of how age can obscure leadership qualifications. Howard says Abraham finished high school at 10 and earned his first degree around 14. Another participant points out that these are still conventional credentials; Howard responds that ageism remains a barrier. Abraham adds that his entry into AI came through the fast.ai course, without a computer science degree.

For Howard, the stronger signal is roughly five years of sustained participation: Abraham answered forum questions, asked substantive questions, and stayed involved. During fast.ai’s diffusion course, which Howard calls its hardest to date, Abraham worked through the mathematics and collaborated with Waseem on a video explaining Stable Diffusion’s math. Working together changed Howard’s assumption that learning comes easily to child prodigies. Abraham still had to admit confusion, work hard, and persist.

Abraham adds a causal ingredient: a welcoming community helped him persist. Enjoying fast.ai made him want to keep participating. He emphasizes that building, nurturing, and growing such a community takes substantial work; his tenacity was supported by the environment in which he learned.

An investor offers distance traveled as a way to evaluate people: assess their progress relative to their starting circumstances. A résumé records current achievements but can conceal limited access to opportunities. The closing discussion connects this to company building. People and GPUs are inputs to valuable products and services; the speaker sees GPUs as relatively interchangeable and an exceptional team as the distinctive competitive advantage.

The final hiring question is what it took to reach the candidate’s current level. The speaker illustrates this with exclusion from school because of dyslexia, growing up in the mountains of Bangladesh without a PC until 16, and fighting institutionalized sexism. These are possible candidate backgrounds, not identified hires. The argument is that reaching a high level despite substantial barriers can signal greater potential than reaching the same level from an already privileged starting point.