AI Startups in 2024: Cheaper Models, Harder Product Questions

Model choice becomes a product decision

Choosing a model API looked different at the end of 2024. OpenAI’s earlier lead of more than 100 points on LM Arena had given way to a ranking led by Google. LM Arena compares user preferences between two model responses to the same prompt; the presenter described a roughly 100-point Elo rating gap as corresponding to preference about two-thirds of the time. Builders now had several plausible suppliers, although the ranking applied to that evaluation and moment.

Spending suggested customers were exploring that choice. In a Ramp chart covering OpenAI and Anthropic, OpenAI’s share fell from nearly 90% in November 2023 to roughly 60% at the later endpoint. The presenter interpreted this as evidence of experimentation and relatively easy API switching. These were shares of spending on two providers, not the entire AI market.

That widening choice framed the NeurIPS 2024 keynote by Sarah Guo, Conviction’s founder, and Pranav Reddy, a Conviction partner, at Latent Space’s first in-person miniconference in Vancouver. Guo introduced the firm’s founding bet: AI could change products, markets and founder profiles enough to undermine established investors’ mental models. The same possibility ran through the discussion of software incumbents.

Smaller models and lower prices widen the options

Open-source models were becoming competitive, but unevenly. On the Scale leaderboards presented, a Llama model ranked among the top three evaluated models for math, instruction following and adversarial robustness. Agentic tool use—having a model use tools to perform tasks—remained a counterexample, with the gap attributed to differences in training data or expertise. Strong results on one task did not make models interchangeable across workloads.

Smaller models also appeared to be narrowing the capability gap. In one benchmark comparison, an unnamed 70-billion-parameter model sat about two points behind the leaders. The presenter contrasted Mistral 7B, recalled as scoring around 60 a year earlier, with Llama 8B, described as more than ten points better. Exact versions were unspecified and the older score came from memory, making this a directional comparison rather than a reproducible measurement.

API prices were falling alongside those gains. The presenter estimated an 80–85% reduction in flagship OpenAI API costs over the preceding year to year and a half, with Artificial Analysis pricing comparisons showing broader declines. A rough analogy suggested that a few thousand dollars could generate a token volume comparable to the data in a product such as Notion or Coda. Cheap generation does not reproduce proprietary data: no dataset size or pricing assumptions accompanied the analogy, and the generated data would have a different distribution.

From responding to acting—and spending more on an answer

The capability survey then moved beyond text. Low-latency voice changes the interaction, allowing an experience different from earlier transcription-first systems. Execution was another emerging category: computer use, code execution inside OpenAI Canvas and Devin, reported as becoming available to try for $500 per month, suggested products could increasingly act as well as respond.

Guo recalled that surpassing roughly 13% on SWE-bench had seemed out of reach a year earlier, whereas that level now felt accessible. No benchmark variant or evaluation setup accompanied the recollection; its point was how quickly expectations had changed.

Video illustrated two different uses: - Sora generated a scene: The presenter used a video depicting the shrimp-fried-rice joke to demonstrate emerging generation capabilities. - HeyGen transformed an existing speech: The portfolio company’s demonstration translated Javier Milei from Spanish into English with dubbing and lip synchronization. The presenter praised the preserved tone and emotion—a subjective assessment, not a measured fidelity result.

The final technical theme was test-time compute: spending more computation while producing an answer. The presenter saw limits emerging in the benefits of increasing scale, while viewing OpenAI’s work as evidence for another scaling approach. The key condition was verifiability. Math, physics and, secondarily, software engineering offered ways to define an objective value function—a measure for judging results. The open question was how to construct useful measures where a good answer is less clearly defined.

Applications reach work and users previously overlooked

The business discussion began by separating foundation-model financing from the rest of the startup market. The presented PitchBook analysis suggested a funding recovery, but a small number of large labs accounted for roughly $30–40 billion raised in 2024. The figures were probably incomplete. Removing those exceptional rounds made the broader environment look more rational to the speaker than the headline totals suggested.

Several application patterns stood out: - Work left undone: Hiring, providing context and managing people can cost more than a task justifies. Scribing and customer support, including Sierra and Decagon, illustrated automation opportunities; Harvey illustrated an opening in professional services. - Search and companionship: Text alone had enabled compelling interactions. Perplexity and Glean represented the productivity side, with richer information formats offering possible next steps. - Creative and technical access: Tools for audio, video, images, writing, code and functioning applications could serve people the venture industry had previously overlooked.

Midjourney demonstrated demand beyond established professional workflows. Its generated images were difficult to edit and incomplete for some professional uses, yet many people still wanted them. Judging the market only by professional requirements missed that appetite for creation. Quality and controllability still had substantial room to improve.

Compute and data supported these applications, but the data opportunity was becoming more specific. Training increasingly called for expert examples and domain reasoning traces: records of how a decision was reached. The startup question was therefore not simply how much data was available, but which information would make the product better.

Selling completed work changes the market

The keynote rejected the broad claim behind the dismissive term “GPT wrapper”: that value belonged only to companies training base models. Specialized models, price competition and open-source alternatives gave application builders options, while turning capability into a useful product remained difficult. Even highly capable general AI leaves substantial deployment work across specific use cases.

Test-time scaling suggested a different financing pattern. If extra inference—the computation used to run a model—produced something a customer valued, that customer could pay for it as part of receiving the result. A startup might then avoid funding a large training effort upfront. Small models offered another option where the domain was well understood.

Legal, healthcare, defense, pharma and education had often looked difficult under venture expectations of rapid, capital-efficient growth. The speaker argued that useful new capabilities costing orders of magnitude less could change buying behavior. Selling completed work opens services budgets, extending beyond software budgets for tools that help people perform a workflow.

Lower prices could also expand demand: customers might buy work they previously could not justify. Software development supplied the example. The speaker predicted that making high-quality software a hundred times cheaper would produce more software rather than fewer people doing development. This was an elastic-demand forecast—demand increasing as cost falls—not an observed employment result.

Distribution, workflow code and data are different advantages

Incumbent advantages needed separate examination: - Distribution still has to be won: A startup needs a better or more inventive product, or a different business model, to reach customers. - Seat pricing can conflict with automation: Doing work for a customer may reduce the number of paid seats that customer needs. - Workflow code can lose its purpose: Years of customer relationship management software implementation matter less if people no longer need to update its database manually every Friday. - Stored outputs may omit the useful training example: Automating knowledge work may require inputs, reasoning traces and final decisions. An incumbent database may hold only the final output.

The practical data test was to identify what would improve a particular product, then determine whether the incumbent possessed it. In the speaker’s portfolio experience, the desired record of knowledge work often had never been saved. A large database alone did not establish an advantage.

The speakers called the broader rethink Software 3.0, spanning business models, product design and engineering: - Price the work and account for its cost: Outcome pricing remained unsettled, while model and compute spending made gross margins a central concern. - Evaluate the actual task: Academic benchmarks could not replace product-specific evaluations. Users also needed understandable behavior when outputs varied and graceful handling of failure. - Treat compute as a company capability: Acquiring, managing and optimizing infrastructure mattered again.

The keynote closed with a startup thesis: rapid change could favor small organizations, while even capable incumbent leaders had enormous teams to coordinate. Foundation-model labs’ ambitions left substantial work elsewhere in the economy. The speakers viewed 2024 as more favorable to startups than 2023 and urged builders to pursue that opening.

Fast growth does not establish durability

The audience discussion tested that optimism by separating three questions: - User value: Does usage reflect an enduring need in work, play or communication? - Defensibility: Can competitors take that demand away? - Operating maturity: Can leadership, customer service and delivery quality keep up with growth? Revenue could outrun organizational capacity. One unnamed example had twenty employees serving hundreds of thousands of users.

The wrapper criticism retained a narrower truth: simple prompts and pages designed to attract search traffic were unlikely to form a durable technology company. A product could attract real demand and still collapse under competition. That distinction qualified the application thesis without invalidating it.

Nor did every AI company require a huge initial investment. One investor described a portfolio ranging from profitable or break-even businesses to companies with substantial upfront costs, including ordinary sales and marketing hiring. Test user value before committing heavily to GPUs was a useful heuristic, with explicit exceptions for businesses that needed graphics processing units earlier. Model APIs allowed incremental, low-cost experimentation.

Some repeat founders intended to use AI to keep teams small. Investors faced their own capacity constraint: intensive founder support limited how many companies they could serve. Embed extended support to more companies through smaller ownership stakes, a network and guidance; applications were said to open in January following the talk.

Enterprise multimodality needs a capture layer

An audience member working at Writer challenged the enterprise case for multimodality—using media such as images, audio and video alongside text. Customers asked about audio and video, the questioner said, but often could not identify a concrete use case beyond vision. A responding panelist argued that average enterprises mostly held text and structured data.

The proposed opportunity required capturing work before models could use it: communications, decisions and created material needed to become available across workflows. Gong illustrated an earlier version, capturing video to extract keywords and sales insights. Highlight, described as a prosumer company in the latest Embed batch, combined screen and audio capture. Such systems could let people work without first organizing everything into schemas and hierarchies.

Cheaper calls can still produce expensive results

A later question asked what happened if intelligence became almost free. The response returned to elastic demand: cost per call differs from cost per useful output. Search and validation systems may make hundreds of calls to deliver one result. Lower unit prices can encourage more computation per output, leaving substantial total spending.

Large models also remained demanding to run. The speaker reported inference-capacity constraints even at foundation-model providers and pointed to model compression and domain-specific approaches as ways to reduce the burden.

Moving cost from training to inference can move who bears it. Providers had absorbed large training costs to make models broadly useful; test-time scaling could instead leave customers paying for thousands of generation attempts to obtain a better result. A panelist emphasized that the referenced scaling chart used a logarithmic axis: the additional computation could be substantial.

Build around a specific prediction of model progress

One panelist refined the advice to welcome new model releases: predict which core capabilities will improve, then build around that prediction. Copywriting startups illustrated the risk of relying on temporary limitations. Some advantages—eliciting writing intent and incorporating business information—became easier to reproduce as models improved, accepted more context and could more easily incorporate website information.

Other constraints lay outside model intelligence. A pharmacy-services company needed scale and the ability to negotiate pricing contracts. Other businesses depended on access to particular people or delivery mechanisms. Better computation alone would not supply those assets.

Competitive threats required the same precision. One investor regretted passing on an unnamed company with strong people and metrics because incumbents seemed likely to crush it. Ability to compete differs from commitment to compete. Reading a business website or adding suggested prompts looked relatively easy; developing taste in developer products and distributing an excellent product required more sustained work. The speaker worried more about underestimating near-term model progress and overestimating incumbent execution than the reverse.

Hardware may change when usage changes

The hardware discussion offered two speculative openings for new consumer devices: - Different usage demands different specifications: Continuous image or video capture would affect sensors and battery life. Continuous local model use could make compute a much larger part of the device. - General AI could absorb single-function apps: A water-drinking reminder illustrated functionality that might no longer require a dedicated app, potentially weakening an existing platform’s ecosystem advantage.

Building such hardware would be difficult. Robotics nevertheless prompted renewed interest: one investor described moving from longstanding skepticism about its economics to making an investment during the year, because algorithms and generalization from broader machine learning appeared applicable to physical intelligence.

Agent infrastructure needs evidence from working agents

Agent infrastructure was harder to specify because successful examples were few and recent. The panel informally debated whether working “true agents” numbered near zero or in the double digits, eventually settling around a dozen—not a defined census. Meanwhile, model providers were still shaping interfaces, from specific clicks to rumored higher-level browser actions. The requirements were changing alongside the applications.

Several proposed capabilities were concrete even while their packaging remained unclear: - Get current information: Crawl, access live data and extract available information from a URL. - Act with authorized access: Take actions on a webpage using a user’s login, with identity and access management identified as a missing utility. - Operate and assess: Retry operations and return feedback about how well the agent works.

One panelist expected strong frameworks to emerge from successful agent companies solving their own operational problems, using React’s internal adoption at Facebook as the analogy. A crawler could remain useful while its interface and commercial packaging changed. A useful component does not automatically make a durable standalone company. The investment approach was to look for recurring needs among agents already operating at some scale.

Consumer products also wait for founders to adapt

The final question asked why there were not more consumer AI companies. The response proposed that innovation diffuses through entrepreneurs as well as customers. Researchers recognized capabilities early; very young founders could experiment with lower opportunity costs. Experienced consumer product leaders needed to develop intuition for AI and make the larger commitment of leaving established jobs.

The speaker saw two plausible routes to a breakthrough: a new founder discovering a form of social interaction built around AI, or an experienced consumer builder combining product intuition with newly acquired AI intuition. More of the latter group appeared to be starting companies and experimenting. The closing prediction was that consumer companies would emerge over the next few years as that learning spread.