A data copilot needs to know what your metrics mean

Statsbot could answer a request, but not clarify it

Which sales opportunities closed last week? Statsbot, the analytics bot Artem Keydunov began building in 2016, aimed to answer questions like this directly in Slack. But users had to know how to phrase the request. When their meaning was unclear, the bot could not ask follow-up questions and gather enough context to generate the right SQL.

Keydunov started Statsbot as a side project while working at a school software company. His team already worked in Slack; he wanted statistics from New Relic, Google Analytics, production databases, and Salesforce available there too. Early users arrived through Reddit, followed by more signups when Slack featured Statsbot on its application directory’s front page.

The first parser used regular expressions—an approach swyx also recalls using for a natural-language option-pricing tool in finance. Keydunov’s co-founder Pavel later explored models and other approaches, but Keydunov remembers few readily usable foundations in 2016. His optimism in this October 2023 conversation centers on what newer language technology could add: a dialogue that clarifies intent before producing a query.

Cube separated business meaning from SQL

Statsbot split the work into two transformations. First, it interpreted a request in terms of metrics—the quantities to calculate—and dimensions—the ways to group or examine them. Then [Cube](https://cube.dev/) translated that structured request into SQL. Keydunov identifies natural-language interpretation as the weak point.

Cube supplied a semantic layer: explicit mappings between business concepts and underlying data. Each Statsbot customer brought a different dataset, so active opportunities needed a customer-specific definition. Cube let customers define those metrics once, then let Statsbot query them. When users began applying the framework to other applications, the team extracted Cube and open-sourced it in 2019.

An analytical cube organizes measures across dimensions so they can be examined from different angles. Keydunov’s intuition is to imagine bringing together the separate result tables from SQL queries answering different questions. For an AI application, those explicit definitions also provide context. His argument is that humans and models need the same business meaning, making a semantic layer a useful shared home for it.

Describe the data, then narrow the query

A newly hired analyst cannot reliably interpret a warehouse from table access alone. Column names may be obscure, and even carefully structured data may lack descriptions. A model faces the same problem: it needs readable context about what the data means.

The proposed execution flow puts that context ahead of query generation: 1. Describe the metrics, dimensions, relationships, structures, and possible dimension values in text. 2. Index that description in a vector store to make it available as model context. 3. Have the model generate SQL against the semantic layer. 4. Let the semantic layer query the warehouse and return results to the application.

The intended advantage is a smaller query-generation problem. The model requests a metric, a grouping dimension, and perhaps a filter. The warehouse query behind that request might require five joins and logic to prevent incorrect aggregation. Keeping that complexity in the semantic layer is meant to reduce opportunities for generated SQL to distort the answer; the conversation supplies no measured error reduction.

For example, a revenue definition could encapsulate joining customers to orders and summing order amounts. The application requests revenue for last week, and the semantic layer expands it into the larger query. Execution can then repeat: the assistant asks a follow-up question, retrieves more information, and uses earlier results to guide subsequent queries, potentially assembling a report from several data points.

Shared metrics still require agreement

Revenue exposes the organizational difficulty behind this abstraction. Sales, accounting, and tax teams may each use a different definition; active users can provoke similar disagreements. A semantic layer can encode those divisions as readily as it captures shared understanding.

The proposed response is to treat metrics as a codebase. Version control tracks revisions, and pull requests give teams a place to debate calculations openly. Agreement remains the goal, but multiple definitions can be managed when necessary. The improvement is making differences visible and reviewable instead of leaving them in hidden spreadsheets.

Chatbots, automation, and copilots

Three application patterns emerge, with different expectations: - Conversational analytics: internal Slack bots answer warehouse questions; an unnamed service connects to Cube and offers a web or Slack interface. - Data-driven automation: an agent checks for changes and takes action, a general pattern described without a specific deployment outcome. - Analyst and engineer copilots: models generate repetitive SQL for technically skilled users to inspect and edit.

In 2023, the maturity of conversational analytics remained uncertain: an apparently popular bot could be a substantial business use case or an enthusiastic side project. SQL copilots had a clearer review workflow, provided the user knew SQL. Cube was also building copilot capabilities to help users create semantic layers.

The broader prediction is that natural language becomes a second interface to business intelligence (BI)—the dashboards, charts, and queries people use to understand business data. Some vendors could organize products around conversation; others could add it as a standard feature. A shared question box need not consolidate the market, any more than shared line and bar charts make existing products interchangeable.

These interfaces create ongoing infrastructure work. Upstream changes must reach the context the model uses, including descriptions stored in a vector database, while query correctness remains a separate concern. Cube’s existing customer-facing analytics infrastructure could also support generated summaries: a few lines explaining a crowded dashboard, with room to expand into detail.

How much of the data stack changes?

One expectation is incremental: warehouses, transformation tools, catalogs, data-quality systems, and integration tools gain copilots without a fundamental architectural shift. Separately, consolidation could follow the preceding two or three years’ proliferation of specialized tools. A Fivetran–dbt combination is floated as a hypothetical new-generation Alteryx, not an announced transaction.

A host proposes a more expansive future: models read documentation, connect systems, and translate descriptions of desired data into transformation rules. A semantic layer would help explain the data’s purpose and destination. The response accepts generating boilerplate for transformations and integrations, but regards full automation as premature: a human still needs to operate and oversee the copilot.

The useful assistant asks a better next question

The conversation returns to Slack with an example of the missing dialogue. A user asks for metrics; the bot asks which ones. The user chooses active users; the bot asks how activity should be defined and whether the analysis should examine groups of users or changes in behavior over time.

Informed follow-up questions bridge the gap between a vague business request and a precisely specified query. Analysts do this using knowledge of their company’s data, and the bots described here were beginning to do the same. The reported improvement over Statsbot came from trying several applications, rather than a benchmark.

Production implementations still required substantial code around the model. GPT-3.5 and GPT-4 were common among the applications observed, without a demonstrated advantage for either. Arithmetic and post-processing often ran in Python outside the model. A lightweight conversational demo could therefore conceal much of the work needed for a demanding application.

The infrastructure advice is to adopt a warehouse or query engine early. MySQL is described as potentially adequate at small scale but a poor warehouse choice as performance demands grow. Open-source analytical infrastructure is suggested as a lower-cost starting option, without a specific workload or sizing threshold.

A semantic layer supplies the model’s business context, with an acknowledged preference for Cube. Cube’s LangChain integration offers a connection to the application stack. The surrounding tools for developing, testing, and documenting AI applications were still unsettled, leaving room for tools better suited to particular jobs.

Better interfaces do not settle the business model

Embedded analytics gives a platform’s users dashboards over their own slice of its data, as distinct from the company’s internal view across customers. Keydunov describes this as a difficult market to monetize and says Cube is more excited about internal BI, including chatbots for internal data consumption and workflows.

His explanation is structural. Established BI vendors already serve embedded analytics and can extend natural-language features to those offerings. Larger organizations may build custom implementations, leaving many vendors competing for smaller and mid-market customers. Because access to models through OpenAI is broadly available, Keydunov doubts that adding natural language alone creates enough competitive advantage to change those economics.

Foundation models reopened an old product idea

Keydunov’s biggest surprise was the arrival of general foundation models capable of serving many applications. While building Statsbot, he had expected customized models for narrow tasks to remain the norm for years. The move to broadly useful models arrived faster than he anticipated.

He still places AI within software engineering. Existing software and data practices remain useful, even as AI-specific development methods, testing approaches, and tools take shape. He compares that emerging ecosystem to DevOps: engineering practices and tools growing around a distinct set of operational needs.

His closing advice is to build analytics chatbots and agents. Having attempted the product years earlier, he believes foundation models have made applications feasible that were much harder in the Statsbot era. The opportunity is to carry a business question through clarification and shared definitions until it becomes a useful analysis.