Normsky: building coding assistants around code intelligence

Understanding existing code was the first problem

After leaving his Google internship, Beyang Liu found programming substantially more tedious. The tool he missed most was Google Code Search. At Palantir, he and Quinn struggled with code navigation and encountered much larger versions of that problem inside clients’ sprawling corporate codebases. Those experiences motivated Sourcegraph: helping developers understand the code that already exists.

Google Code Search combined a trigram search index with a reference graph linking code references and definitions. Sourcegraph uses Zoekt, created by Han-Wen Nienhuys, which Beyang believes was heavily inspired by Google’s original search index. Beyang initially knew Steve Yegge through his engineering writing; a later talk connected Steve with the code-intelligence system he had used.

Steve’s tooling interests reached back to a video game he began around 1995, which exposed needs for better languages, databases and development tools. At Grab, he encountered another kind of information overload: a Slack workspace with 2,500 engineers whose conversations were impossible to catch up on. AI summarization, he reflects, would have helped. An introduction through Max at Temporal eventually brought Steve and Sourcegraph together, offering a return to developer tools after advertising work.

That history supplied the premise for Cody, Sourcegraph’s coding assistant. In the conversation published in December 2023, Beyang explains that context useful to humans navigating large repositories should also help AI generate code and answer questions. Cody offered inline completion, codebase-aware chat, and commands for generating tests or documentation. Its proposed differentiator was context quality and retrieval speed, beyond what a feature checklist could convey.

Repository knowledge beats a plausible convention

Retrieval-augmented generation, or RAG, supplies a language model with information looked up for the current task. The conversation compares retrieval to consulting someone who knows the codebase, while fine-tuning resembles on-the-job training. Training a model for a task still leaves it needing the particular repository’s implementation details.

Two examples recounted from Sourcegraph’s Copilot-versus-Cody comparison illustrate what happens when those details are missing: - Purpose gets confused with ingredients. Asked what an application did, Copilot reportedly listed implementation technologies, while Cody described its functions. - A familiar command overrides repository configuration. Asked how to start an application, Copilot reportedly suggested npm start, although package.json contained no start command. These comparison-post examples were not a controlled evaluation. They illustrate a concrete failure mode: substituting a common convention for repository knowledge.

Completion and chat require different model behavior. Inline completion must respond quickly, fill gaps within existing code and return code without conversational preambles. When Cody used Claude Instant for completions, the team had to prompt around its tendency to introduce an answer. At the time of the conversation, StarCoder powered completions. The team claimed a completion acceptance rate—the likelihood that a displayed suggestion is accepted—comparable to Copilot, but gave no numerical rates, evaluation population or comparison protocol.

Cody’s pluggable model layer allowed the team to adopt proprietary or open-source models as they improved. The expected specialization extended into retrieval: selecting code for a large language model (LLM) is itself a recommendation problem, potentially served by several models with different jobs.

Retrieve broadly, then rank for relevance

A basic RAG pipeline embeds documents and queries into representations it can compare, then retrieves nearest neighbors. Useful context selection also needs ranking and diversity among results. The discussion describes a two-stage search process: 1. Retrieve for recall: collect potentially relevant material so useful code is less likely to be missed. 2. Rerank for precision: move the most relevant material toward the top. The language model becomes the consumer of a search or recommendation system, rather than a reason to dispense with one.

Context placement also needs testing. Participants suggest that models use different positions in their input windows unevenly, without establishing a measured GPT-versus-Claude rule. Their practical recommendation is to test retrieved material at both the beginning and end for each model. One proposed explanation is that using deeply buried information requires training examples that reward finding it, whereas ordinary conversation often emphasizes recent material.

Parallel edits exposed the reliability limit

Non-Stop Cody, credited to Dominic and shipped in the first June launch, allowed multiple LLM requests to modify a source file in parallel. It handled diffs and displayed the regions each request would change. Some scaffolding carried into inline assistance, but the interface narrowed from general requests toward targeted actions such as fixing an error at a particular line.

The constraint was whether generated code worked reliably enough to leave unattended. Humans still needed to inspect responses, and the team drew a practical distinction: - Constrained tasks such as generating unit tests or fixing compiler and test errors were more promising for asynchronous assistance. - Broad implementation requests such as writing a class using existing libraries remained insufficiently reliable for unattended execution, even with good context. Reported user feedback reinforced the narrower interface: developers preferred a quick, correct fix and a return to coding over a long conversation embedded in the file.

Here, agentic execution means a multistep process with automated steps, no required human intervention at each step, and an LLM call at each or nearly every stage. One participant rejects agents as an immediate priority in favor of improving context sources and user control over context. Another preserves the long-term goal of automated issue implementation: the specific doubt concerns a transformer-based LLM alone serving as the workflow’s orchestrator.

Retrieval makes that architectural distinction concrete. A participant interprets Copilot’s @workspace behavior as sequential model calls deciding what to search and read, criticizing its speed and consistency; this is an observational assessment of its implementation. Sourcegraph’s described approach performs code searches and limited reference-graph traversal without LLM calls in those retrieval steps. The team claims faster, more predictable retrieval and better context, without a quantitative comparison.

Normsky gives formal tools and learned models different jobs

A chess analogy sharpens the proposed division of labor: a search algorithm controls exploration while a model suggests candidate actions or helps judge which paths look promising. A coding system could similarly use an LLM as a proposal function within a controlled search. A* appears as an analogy for that arrangement, not as a disclosed Cody implementation.

The participants use Norvig and Chomsky as shorthand for complementary approaches: - Learned behavior: data-driven models, including LLMs, generate plausible answers and proposals. - Explicit structure: parsers, compilers and formal systems construct precise code relationships that can supply context, constrain operations and check generated output. In this framing, Sourcegraph’s investment in code intelligence provides useful structure around the statistical model.

The internal name Normsky combines Norvig and Chomsky, with the stated expansion non-agentic rapid multi-source code intelligence. Its three design priorities are: - Use established tools for retrieval. Parsers and search indexes perform work that need not become an agent loop. - Avoid sequential model calls for simple requests. Speed matters in the developer’s immediate workflow. - Gather knowledge beyond source files. Search results lead outward through references, with documentation, product specifications, production logs and chat as further context sources.

Combining these sources creates a substantial evaluation burden. Search, graph traversal and AI retrieval may all contribute useful context to the same query, so their results must be ranked and tested together. Sourcegraph reported generating tens of thousands of tests, including tasks that fill gaps within code, to tune Cody’s behavior. No resulting scores were given.

Context packing starts with code structure

Ranking does not solve the capacity problem. Retrieved material can exceed the model’s context window—the input available for a request. Context assembly therefore resembles bin packing: decide whether to include an entire function, its opening portion or a particular expression. The discussed compression techniques include reducing each item to its smallest useful form, model-generated summaries and recursive summarization. Each choice needs testing across supported model configurations.

Parser-aware preprocessing gives the system meaningful units to work with. Separating function signatures, bodies and docstrings lets it use signals from each independently. Sourcegraph describes its advantage as an engine that quickly decomposes a customer’s codebase into fine-grained semantic units and serves them for retrieval. This is a preprocessing capability, rather than a stockpile of customer code. A participant also states Cody’s guarantee that private customer data would not train a generally available model.

Compact context raises a question about domain-specific languages (DSLs)—small languages for particular tasks. Could interfaces be designed primarily for efficient model input? A responding participant suggests that models’ tolerance for loosely structured input helps explain why a dedicated context language had not emerged. Context tagging was approaching that territory, with a formal language left as a possibility.

The separate question is how much syntax humans will need to learn. One prediction is that developers could understand the broad capabilities of regular expressions or XPath while letting models construct expressions. Another participant emphasizes that DSLs trade easier lower-level programming against an additional abstraction layer, and expects some to remain worthwhile. SQL offers the immediate example: models already help participants draft queries, without making review or expertise unnecessary.

Navigation protocols and code graphs serve different needs

OpenCodeGraph was proposed as a protocol for supplying context and hints from systems such as chat and logging to AI developer tools. It would give context providers a common interface across the software development lifecycle, with analogies to LSP and the original OpenAI plugins API. A preview was expected by December 14; that was a plan, not a confirmed release.

The protocol discussion turns on ease of integration versus explicit code relationships: - [Language Server Protocol (LSP)](https://microsoft.github.io/language-server-protocol/) simplifies editor navigation. An editor can ask about a source location and receive a destination range. Implementers need not first solve globally unique symbol naming across repositories, packages and registries. - [Kythe](https://kythe.io/) models relationships explicitly. Its storage format represents the graph the speakers want, but they argue that its richer schema requires more work before an implementation becomes useful. - SCIP aims to combine those strengths. Sourcegraph’s protocol seeks a simple schema and straightforward indexer development while retaining symbolic relationships useful to human navigation and Cody.

Sourcegraph’s reported LSP pain point was composing operations: finding references, moving through ranges, resolving the symbol at a range, then reaching its definition. Those intermediate steps added latency and complexity. LSIF is described as serializing LSP-style information into an index that helps language servers start quickly, while retaining limitations the team wanted to escape. An earlier claim that LSP already included cross-references was corrected during the exchange: the speaker had meant LSP together with LSIF.

Build-system integration was another obstacle. The described indexers compiled code in a special mode to produce indexing artifacts instead of binaries. A new SCIP-based graph, internally called BFG, was said to index in the background after plugin installation, without configuration or build-system integration. The mechanism removing that requirement was not explained, and its public name and availability remained tentative.

BFG’s concrete target was type errors in completions. A model might suggest an existing variable as a function argument even though its type is incompatible. The proposed remedy retrieves definitions and references before generation, placing the missing type context in the window. The team reported eliminating many such errors, without a measured reduction. Development targeted specific weaknesses, including call arguments and definition parameters, using thousands of generated tests and roughly half a dozen context sources.

What would change the argument about agents?

Cody supported supervised issue resolution: draft a plan, retrieve relevant context, identify files to modify and suggest changes after further prompting. Developers still needed to monitor the process and intervene. One participant maintained autonomous development as a long-term goal while arguing that a nontechnical person speaking a nontrivial feature into existence remained several key innovations away.

The counterpoint was a planning problem: capabilities that looked impractical in 2023 might arrive by 2030. A speculative human-compute analogy extrapolated a hundredfold increase per GPT generation toward enormous future capabilities; it was not a verified training measurement or capability forecast. The skeptic expected substantial hardware and inference progress but considered extrapolation less reliable higher in the software stack.

The skeptic offered a condition for reconsideration: a pure transformer LLM outperforming the best chess-playing algorithm would weaken the argument for other algorithms as the backbone. The disagreement concerns whether scaling transformers alone is sufficient, or whether they remain one component in a system using several kinds of algorithms.

Choose storage and models for the workload

A code graph need not require a graph database. Sourcegraph stored graph data partly in Postgres and partly in flat files. An early prototype tried an unnamed graph database, then switched to Postgres after a lengthy loading attempt apparently crashed. The speaker places that incident about a decade earlier and acknowledges that the database may have improved. The lasting design criterion is the query workload: single-hop lookups, including those enabled by denormalization, can suit ordinary storage.

The reported stack used Go, TypeScript and Rust, with SCIP-speaking indexers and a trigram search backend for regular expressions and string literals. Fuzzy, natural-language and keyword search were being added on top. The team’s infrastructure decisions divided along where it needed control: - Own the application experiments. Sourcegraph implemented recursive summarization itself. Although interested in LangChain and LlamaIndex, it wanted freedom to iterate throughout the stack while discovering useful workflows. - Delegate inference infrastructure. Running StarCoder on Fireworks let the team concentrate on context retrieval, the knowledge graph and model fine-tuning.

Model choice depended on context and prompts. StarCoder served completions while Claude and GPT-4 served chat and commands. The team explicitly declined to claim universal superiority for StarCoder over CodeLlama, which it was not using. StarCoder had worked best in tests with the team’s then-current retrieval and prompt construction. When context-fetching mechanisms changed, the team retested available models.

Rishabh, the newly hired head of AI, had fine-tuned StarCoder and started workstreams covering prompts, embeddings, evaluation, experimentation and benchmarking. A public Cody benchmark on Hugging Face remained an aspiration. Task-specific fine-tuning with synthetic data pursued a narrower goal: make at least one coding step reliable enough to automate. The team wanted clean datasets combining public and synthetic code, while questioning the business economics and recognizing that curated data’s competitive value could discourage sharing.

Provider dependence also affected operations. During OpenAI’s leadership crisis, one participant described moving to Microsoft as an emergency fallback. Anticipated customer questions about service reliability made provider stability a product concern alongside model quality.

Faster generation makes codebase understanding more important

Managing emergent complexity is the larger bottleneck. As a codebase grows, developers spend more time understanding how its components fit together so they can add functionality without breakage or unnecessary future complexity. Faster generation can compound that problem, particularly when the person requesting code does not understand its implementation. Quality assurance must assess architectural fit as well as local correctness.

The proposed tools for engineering leaders would help them understand architecture, investigate potential vulnerabilities and assess whether code is well tested. The team said it had rejected prospects exploring developer monitoring tied to firing decisions. Broader analysis also faced a cost constraint: a hypothetical six-billion-line security scan illustrated the expense of repeatedly processing an entire codebase, motivating analysis when code enters the system.

Cody Web brought repository questions into the browser. One participant used it to investigate a Twitter scraper inside a large repository after an unsuccessful attempt with Copilot—a single usage anecdote. Its Ask Cody button was difficult to discover, tucked into the upper-right corner. The exchange also distinguished two mechanisms: repositories available for questioning had been indexed, without necessarily being embedded.

Users could tag several repositories as context. In one demo, React and a tick-data API repository supplied context for a short request to build an app tracking Bank of America and Wells Fargo over the previous week. The speaker reported that the generated code worked when pasted into an app. This was RAG across selected repositories, rather than task-specific fine-tuning: a useful demonstration of the workflow, without a measured success rate.

Two broader opportunities followed: - Perform deferred maintenance. Dependency updates and tedious code-quality work could improve security and robustness. - Help explain engineering contributions. A future system could read Git history, delivered features, reviews and teammate support, then help leaders connect that work with engagement, revenue or adoption goals. The second proposal was assistance in explaining impact, not an established productivity measurement capability. Given limited capacity, the immediate priority remained making Cody compelling to developers before expanding to other decision-makers.

New interfaces still need reliable building blocks

The closing discussion looks beyond completions, chat and commands toward interaction patterns that might already be possible with existing models. The Sourcegraph team had not yet explored multimodal workflows, but participants suggested several directions: - Explain systems visually. Generate architecture or entity-relationship diagrams and accompanying explanations; a reference graph can produce diagrams without AI. - Turn drawings into code. Use a whiteboard sketch as input to generation. - Offer help without a chat prompt. A recounted demo periodically sent screenshots to GPT Vision for proactive suggestions. A participant’s hackathon variant commented on distractions such as browsing Twitter instead of coding.

The technical challenge returns to reliable individual steps before longer automated chains. Even a request to write one function needs dependable results before it can serve as a building block for multistep execution. Normsky expresses the team’s bet that future developer tools will need several forms of intelligence to reach that point.

The final recommendation is forceful: developers who abandoned coding assistants after an early disappointment should try again. Using an assistant effectively requires learning its behavior—how it responds under different query conditions and when it gives the user the runaround. The speaker predicts that developers who avoid that investment will be left behind. That call to practice sits alongside the unresolved engineering goal: code that works reliably on the first attempt.