Cursor’s early bet: redesign the editor around the model

A rectangle, an extrusion, and a failed assumption

Draw a rectangle, extrude it into a solid, then ask a model to continue the design. The next operation requires understanding the geometry created so far and knowing which part to reference. This was the problem behind the original Cursor: applying next-token prediction to computer-aided design, or CAD, before building a code editor.

Aman Sanger and his original cofounder, Michael, were enthusiastic Copilot users. Mechanical-engineer friends described tedious work in SOLIDWORKS, and the founders thought similar assistance could accelerate mechanical design. They trained models from scratch. CAD appeared technically feasible, commercially substantial, and less crowded than coding—a concern about competition that Sanger later became more comfortable setting aside.

Their training strategy rested partly on an overgeneralization. Sanger had read the original Codex paper as showing that, with enough Python data, training from scratch could match fine-tuning a model pretrained on language. He later understood that result as specific to its model and data regime. For CAD, he estimated that an exhaustive scrape would yield at most about 10 billion tokens. The team’s models overfit beyond a few billion parameters despite attempts at regularization: scaling the model did not overcome the limited data.

Spatial reasoning and integration imposed further constraints. In Sanger’s informal tests describing cubes on a table, GPT-3.5 failed as arrangements grew more complex, and GPT-4 failed at higher complexity; these were observations rather than a quantified evaluation. Plugins for SOLIDWORKS and Onshape were also difficult to build and distribute. Even a capable model needed a usable workflow. Sanger came to favor redesigning CAD around AI, encouraged by progress in text-to-image and text-to-3D generation.

GPT-4 changed the interface bet

A brief text-to-image experiment, Instill, reinforced another lesson: build something you use. Sanger says the founders were not substantial users of image-generation products. Coding was familiar territory, but older GPT-3.5 variants had disappointed them. Early GPT-4 access around December changed that. Pasting code into the playground revealed useful generation, refactoring, editing, and debugging capabilities.

An early playground experiment reportedly reached roughly 85% on HumanEval, a Python coding benchmark, against recalled figures of about 30% for the best open model at the time and 47% for Code Davinci 2. Unspecified prompting and sampling conditions make this an informal comparison. Access came through OpenAI’s Converge program, which allowed experimentation with unreleased models but prohibited production use. The team had not taken OpenAI money before building Cursor.

The resulting thesis was conditional: if models could produce 90–95% of code, autocomplete would be too narrow an interface. Copilot remained useful enough for daily use, but accepting successive completions did not seem like the final form of software development. Cursor was a bet on controlling the full integrated development environment, or IDE, so its interaction model could evolve with the models.

Speed and edits beyond the cursor

Copilot’s responsiveness raised a useful distinction between two kinds of caching: - Response reuse returns an earlier answer when the context changes only slightly. For code, that risks answering a different problem. - Attention key-value (KV) cache reuse retains previously computed attention state so the model need not process reusable prompt content again. It saves computation but requires storing and moving that state. Both were discussed as possible explanations for Copilot’s speed, not confirmed implementation details.

Owning the IDE was also a response to platform constraints. The team conceded that Cursor’s capabilities then were not dramatically different from those available in VS Code. Its concern was longer term: an extension vendor competes within an incumbent’s platform. A secondhand account that Copilot’s multiline ghost text required changes to VS Code itself helped motivate that concern.

One planned interaction made the argument concrete: suggest a change to a preceding line or delete existing code, rather than only complete what follows the cursor. Cursor had built the interface and was still training models; usefulness comparable to Copilot remained an aspiration. Command-K already let users request modifications or generation inside the editor. These small reductions in interaction friction were part of the case for controlling the whole environment.

A good answer is not yet an applicable edit

Persistent AI rules addressed recurring wrong assumptions. Cursor used Solid, a reactive interface framework, but GPT-4 tended to interpret a TSX component as React code. A standing instruction supplied that convention without repeated prompting. Rules could also request explanations in a preferred language. At the time, most users worked with GPT-3.5, while Pro users could access GPT-4.

Reliable edits were harder. The team found the models stronger at generating new code or rewriting a block than at emitting a diff—a compact description of changes to existing code. Line numbers frequently went wrong. One workaround used GPT-4 to propose changes and GPT-3.5 to repair them. The suggestion that shorter diffs left less computation for reasoning was a theory, rather than an established explanation for the failures.

Output format affected both capability and speed. Azure’s then-available GPT-3.5 completion interface could continue a supplied prefix, making it useful for completing code within a line. Separately, one participant favored comments before code as a reasoning aid, efficient implementations, and clarification when library versions mattered. Longer answers, however, increased waiting and reading. Compact diffs promised less output but produced less reliable edits with those models.

A usage complaint exposed the remaining gap: chat could abbreviate untouched code and show the changes, yet a participant still had to copy three suggested Python lines into a file when the conversation had not started there. Direct application was planned for the week of the conversation. Model economics complicated it: GPT-4 was expensive enough to use sparingly, while GPT-3.5 struggled with the task.

Find relevant code and keep changes inspectable

Retrieval had its own conspicuous failure: contextual chat repeatedly selected the license file for one participant. The response tentatively blamed the existing embedding-based retrieval. One alternative under exploration was inspired by Transformers as a Differentiable Search Index: train a model on a corpus to map a question directly to a relevant document. For code, it could emit the symbol identifying a function or class, rather than generate its source.

Other retrieval work included adapting embeddings to individual codebases and using rerankers to reorder retrieved candidates. Reported improvements were qualitative. Users could also choose context explicitly: @Nextjs in chat or Command-K brought in recent Next.js documentation.

Better context did not remove the review bottleneck. Experimental agents produced large amounts of code that were difficult to verify; debugging could take longer than writing it manually. A planned chat workflow borrowed from Code Interpreter: perform small, auditable actions and show outputs people can inspect. Generating a graph illustrated the idea—look at the result, then examine its code when needed. The team expected this workflow within two weeks of the conversation.

An interviewer offered a complementary approach, attributed secondhand to Itamar of Codium: develop the specification, tests, and source together, using test execution to validate code. The response treated rigorous tests as promising, while emphasizing the ability to see and steer changes step by step. Writing a massive up-front prompt imposed its own burden. With comparable capabilities, the team expected the easier, more enjoyable workflow to win.

Documentation versioning remained user-controlled. Cursor embedded current documentation by default, but users could provide a URL for a custom or version-specific set. A background crawler embedded that material, described as stored locally. Its scope was deliberately narrow: documentation’s regular, Markdown-like structure made it easier to crawl than general web content.

Tools, feedback, and execution boundaries

The planned interactive agent would execute locally, grouping commands into discrete steps for user approval. Fully unattended work would likely need a contained environment. Prompt injection made the boundary consequential: malicious instructions in a retrieved source comment or documentation could be mistaken for directions, causing an agent to execute dangerous terminal commands.

An interviewer proposed that an “agent cloud” should distinguish itself from an ordinary serverless sandbox through feedback designed for AI operators. The response emphasized giving models more tools. AlphaCode supplied an example of an inference process that generated tests, executed candidate solutions, and selected among them using machinery including clustering. The lesson was that execution and selection could improve on a standalone model response.

GPT-4 offered a richer loop than pass/fail filtering: interpret test results, then revise code or tests. Cursor’s VS Code fork also exposed language-server tools—the editor capabilities behind go-to-definition and workspace symbol discovery. The team reported substantial, unquantified improvements from giving models these tools. An agent could navigate definitions and inspect code much as a programmer does, rather than depend entirely on semantic search.

Priompt allocates a limited context budget

[Priompt](https://github.com/anysphere/priompt), an internal priority-prompting tool, addressed variable inputs competing for a fixed token budget. With a 4,096-token limit as an example, it let developers specify what to keep when chat history and code overflowed the available space.

The example policy was: 1. Include the chat history and current file while everything fits. 2. Remove older chat messages first. 3. At a chosen threshold, preserve the remaining conversation and begin truncating the file according to a selected strategy. The prompt’s composition changes with the available budget. The policy determines which information survives.

The team used JSX for structured prompt composition, drawing an analogy to responsive web design: different context lengths call for different arrangements of the same inputs. Its recommendation remained pragmatic—use strings for simple prompts and consider JSX when many possible inputs require explicit composition rules. The best long-term approach was still unsettled.

Longer windows did not eliminate the cost of context. For very large models, the respondent argued, feed-forward computation could dominate attention’s floating-point operations, making total compute look approximately linear up to around 8K tokens despite attention’s quadratic scaling. Attention and memory traffic became more consequential at longer lengths, and even linear growth felt expensive under then-current 32K pricing.

Recurrent approaches were appealing because they could retain a fixed state during inference. The state-space-model discussion combined parallelizable training with recurrent inference, tentatively estimating O(N log N) training work for sequence length N and roughly constant per-token inference cost. Hungry Hungry Hippos and Hyena were cited as related long-sequence work; the exact complexity claims were tentative.

Practical experience was mixed. Claude’s implementation and reports of forgetting remained speculative or anecdotal. The questioner had not tried Magic.dev’s announced five-million-token window. Another participant reported a working personal setup with roughly 55,400 prompt tokens and 1,800 completion tokens, without identifying enough task or model detail to generalize its reliability.

Input and output tokens have different economics

The comparison behind “Why GPT-3.5 is (mostly) cheaper than Llama 2” depended on prefill versus decoding. Prefill processes known prompt tokens, allowing parallel work across the sequence at each transformer layer. Autoregressive decoding generates one token before processing it to produce the next. A long input with a short answer therefore creates a different serving workload from a short input with a long answer.

Prefill was characterized as compute bound, while low-batch decoding was memory-bandwidth bound: generating tokens repeatedly requires reading model weights. Larger batches can shift the bottleneck back toward computation. The resulting recommendation was workload-dependent: GPT-3.5 could be cheaper for completion-heavy work, while Llama 2 looked more attractive for prompt-dominated tasks such as classification. Internal reranking experiments with Llama 2 were useful but had not reached production. The comparison did not include a complete pricing calculation or deployment configuration.

Quantization stores values at lower numerical precision. Reducing weights from 16-bit floating point, or FP16, to roughly four bits makes them four times smaller. The respondent estimated a three- to fourfold improvement in reading those weights—not an equivalent end-to-end speedup. Weight-only quantization helps when weight reads are the bottleneck; at larger batches, computation can dominate. For smaller Llama models using ordinary multi-head attention, KV-cache traffic could become the limiting factor instead.

The proposed serving direction was full eight-bit quantization of weights, activations, and the KV cache, addressing several potential bottlenecks. This was a judgment about the options available then. Idle GPU capacity also affected cost: the respondent described the claim that quantization savings roughly cancel imperfect utilization as hand-wavy. And serving cost was only one consideration—Llama 2’s ability to be fine-tuned was a substantial benefit.

Revising benchmarks and business assumptions

HumanEval returned with more skepticism than the early playground excitement suggested. Possible training contamination complicated both open and closed model scores, and strong Python results did not establish broad coding ability. BabelCode was proposed as a way to translate the problems into other programming languages. That would broaden coverage, though familiarity with the original problems could still help on translated versions. HumanEval’s ease of execution remained part of its appeal.

The business thesis had changed too. The respondent had expected startups to train specialized models and outsource large training runs to specialist companies—an outlook formed when GPT-3-era generalists seemed much weaker. The revised expectation was that most AI product companies would use foundation-model APIs, perhaps with fine-tuning. CAD illustrated the shift toward believing in generalist transfer, without establishing that its earlier spatial-design problems had been solved.

A proprietary-data argument received a similar revision. An earlier comparison of 92 TB of public GitHub history with 86 TB in Google’s monorepo had suggested nearly comparable corpus sizes. The respondent now believed the GitHub scrape substantially undercounted available code. The replacement estimate—5–10 trillion usable code tokens—was still an expectation, while Google’s internal code remained a meaningful resource in that assessment.

The company’s formation reflected the same surprise about model capability. Two teams independently tried GPT-4 and decided to build an IDE. After several months of separate work, they joined forces: Anysphere became the combined company’s name, while Cursor remained the core product. The respondent’s biggest surprise was how quickly one generalist model became strong at both text and code.

Persistent learning and the next capability jump

Efficient lifelong learning remained a central unsolved problem. The respondent contrasted efficient learning within a prompt with fine-tuning that required many examples, and saw persistent recurrent state as an appealing direction. Another participant suggested letting models update and query their own knowledge graphs. RETRO-style architectures—models that learn to retrieve from databases as part of their architecture—offered another direction of interest. These were proposals for durable memory, rather than demonstrated solutions.

The closing advice was to design for substantially stronger models. With GPT-4 only a few months old, the respondent expected another comparable capability jump to change software development through stronger reasoning, better understanding of engineering nuances, and longer context. GPT-4-level agents’ limits remained uncertain. The forecast was that future models could handle almost any reasonably well-scoped coding task—a qualification that tied the ambitious interface bet to work users could define and assess.