ChatGPT’s App Store Moment: From Conversation to Computation

A video edit becomes a compute platform

Upload a video, describe an edit, and receive a download link. The example that opens this Latent Space emergency episode is ChatGPT generating and executing code that uses FFmpeg to process a file. An instruction becomes computation, and computation produces a usable artifact. The user does not need to learn the video tool’s syntax first.

In this March 2023 launch-day discussion, participants react to the ChatGPT Plugins announcement, while access is still waitlisted. Participants call it an App Store moment, but the video example suggests a broader possibility: a conversational programming environment that generates code, runs it and returns files. One participant sees a new form of interactive notebook. Their impression of sufficient memory, storage and compute is qualitative; no resource limits are established.

The launch spans web browsing, code execution and document retrieval. The companion article links OpenAI’s ChatGPT retrieval plugin to retrieving documents from external datastores. Participants immediately see integrated retrieval as a threat to standalone document-search products. Their predictions of startup extinction express competitive anxiety, not observed business outcomes.

A small manifest, a large distribution prize

Plugin development supplies the second surprise. A developer provides a small JSON manifest, an English description of the service, and an OpenAPI specification—a description of the API the model can use. One participant estimates roughly ten lines of manifest JSON and finds the approach easier to understand than a browser-extension manifest. The appeal is that readable descriptions help the model operate an integration.

Easy integration does not settle who captures the customer. Participants contrast acquiring users for a standalone application with building inside ChatGPT, invoking an audience of 100 million as the distribution prize. That is potential reach, not demonstrated plugin availability: access remains restricted. Launch partnerships with Expedia, Kayak, OpenTable, Wolfram and Zapier also raise concern that large companies have occupied valuable categories before small developers arrive.

The early brainstorming identifies two openings: - Improve the experience: an existing video-summary application still leaves room for a better workflow or interface. - Build discovery and reputation: submissions, reviews and ratings could support a third-party plugin directory while the platform lacks a popularity ranking.

Let the model write the backend—or run it?

The pace of announcements makes choosing what to learn difficult. A participant recommends following a few people who aggregate developments well, enabling selected notifications and accepting that some news will be missed. Capability excitement also crowds out scrutiny: the same participant admits skipping the announcement’s discussion of safety testing to focus on what the system can do.

The first engineering boundary is where the model belongs in execution. A participant relays a secondhand account of a Scale AI hackathon winner that used GPT as a backend: the simple demo worked, but reportedly broke down when pushed. They consider GPT-3 inadequate for that approach and leave later models’ suitability open. Their broader objection is cost and latency from invoking a large language model on every API request, though no benchmark accompanies the judgment.

The proposed decision rule separates two workloads: - Scaffolding and occasional interactive tasks: let the model help write software or perform an unfamiliar operation, such as an ad hoc video edit. - Repeated production requests: let the model help build the backend, rather than assume it should execute the backend’s logic on every call.

Personalized education introduces a different concern. A participant argues that effective tutoring requires a model of the learner: representing what someone understands and rapidly testing hypotheses about them. They call this theory of mind and urge caution because of its potential power. Language tutoring seems more constrained, but their experience of tutors failing to understand the individual student leaves them unconvinced that personalization is ready.

Demonstrations of generating an app from a sketch and reading a screenshot sharpen the employment question. One participant predicts that reading an entire screen will lead to coding it. Another argues that product judgment remains valuable as syntax becomes easier to generate. Even a sketch requires decisions about where buttons belong and how people should interact with them. The proposed shift is toward specifying what software should do.

Platform dependence and unequal API calls

Users can often describe a data transformation or cross-service workflow more easily than they can connect its APIs. Plugins promise to reduce that implementation work, but introduce dependence on the platform that supplies the interface. Participants contrast OpenAI’s appealing integration path with the greater openness of LangChain, a tool for connecting model-driven operations. Whether plugins use an ordinary base model or specialized training remains unanswered at this stage.

If users reach Expedia or Zapier through ChatGPT without visiting those services’ websites, how do providers get paid? Monetization terms are unknown to the participants. They compare two possibilities: - Subscription royalties: divide a ChatGPT subscription among providers, as Spotify distributes royalties. The objection is that a meme-generation call and a business-summary call can have different costs and values. - Separate plugin charges: pay for services or usage through OpenAI. This accommodates differentiated prices but could produce surprising monthly bills.

Data-use terms also remain unresolved. One participant asserts a promise against training on usage data; another questions whether plugin request patterns fall under it and acknowledges not having read the terms. A more concrete distinction emerges in the interface: browsing and third-party plugins are separate launch modes. Users cannot simply enable browsing alongside an Expedia plugin.

Personal assistants must remember what matters

Assistant ideas become more compelling when grounded in what existing products fail to remember. A proposed travel assistant would combine calendar destinations and dates with personal preferences. A basketball fan wants price alerts for preferred seats, frustrated that Ticketmaster repeatedly shows the cheapest options despite their purchase history. Remembering the user’s constraint makes the recommendation useful.

LindyAI is introduced as an assistant for workflows expressed in natural language. A participant describes recruiting ten software engineers by finding candidates on LinkedIn, drafting outreach, obtaining email addresses through a backend service and sending messages in the background. They are still waiting for access, so this is a description of the intended experience rather than a workflow they have tested.

The scheduling examples show why natural language is attractive: no Friday meetings, Monday one-on-ones, and meetings grouped with five-minute gaps. Preferences can include exceptions, such as wanting an aisle seat unless a flight lasts less than an hour. The business question is frequency versus value per use: infrequent tasks need to deliver substantial value, while frequent tasks can justify less value per occasion.

Nader’s GPT-3 travel app provides a concrete example. During monthly work trips, he spent about an hour using TripAdvisor and Google to plan a day or two of sightseeing. Supplying GPT-3 with a city, duration and other parameters produced itineraries he liked. He built a polished interface around that interaction and linked recommendations to Google searches.

Nader reports roughly 100,000 daily visitors, including 60,000–80,000 daily unique visitors, after sharing the app. Yet traffic brought costs without revenue, and he lacked time to maintain it or implement affiliate monetization. He sold it within about a day of announcing the sale; no price is disclosed. Attracting users and operating a business were separate problems.

From installed plugins to services chosen on demand

Returning to plugin setup, participants describe configuration containing a model-facing description, authentication information, a logo URL and a legal-information URL. Installation involves hosting the JSON associated with an API specification, entering its location in ChatGPT and passing validation. Bearer authentication is mentioned; OAuth2 support is assumed, not confirmed. This simplicity renews concern about defensibility: if OpenAI absorbs common features, niche or proprietary applications may offer more durable advantages.

Interaction design extends beyond typing. Participants want speech input and spoken replies, but disagree about dictating long prompts. One proposes supporting both speech and text, with a system instruction to keep responses concise. Another reports LangChain’s new “ask human” tool, allowing a workflow to ask the user a question when uncertain instead of simply continuing.

The brainstorming explores several possible uses: - Content tied to physical places: an imagined augmented bulletin board at a coffee house or bar. - Games extending beyond the game window: multiplayer play inside ChatGPT, with characters that might text or call for help when a virtual village is burning. - Everyday assistance: meal suggestions from refrigerator photos, proposed conversational calorie estimates and diet planning, or a phone-call interface backed by Twilio.

A participant then challenges the app-store analogy itself. Instead of making users choose plugins to install, a natural-language routing layer could select services on demand. For a travel request, the proposed system would query Expedia and Booking.com, present both sets of results and potentially charge providers for click-throughs. This is a possible future design, not launch behavior. It shifts the competitive question from winning an installation to being selected within a shared interface.

An assistant must work while you are away

Offline models such as Alpaca prompt a parallel vision: privately run models paired with personal collections of tools. Participants imagine a wearable Raspberry Pi assistant and a local model that explains compiler errors. They expect inexpensive local inference to broaden access, but provide neither tested hardware requirements nor a reliable timeline.

Alex identifies an architectural gap closer to the existing chat experience: scheduled work breaks the synchronous request-and-response loop. His example is a trading bot that checks wheat prices at midnight each day and sends the result back to a large language model. Another proposed assistant monitors a thread and alerts the user to relevant developments. Both require work to begin after the original conversation has paused.

Alex also describes Instacart lookups as slow and blocking, without latency measurements. He proposes asynchronous fetching followed by a completion notification, allowing the user to do something else while waiting. The assistant accepts work now and returns when it is ready—a different interaction from waiting for every response in sequence.

Logan clarifies the launch boundaries

Logan points to Zapier’s roughly 5,000 existing integrations as a way for ChatGPT users to reach services before each publishes a plugin. He believes Zapier may also support the proposed scheduling workflows, but explicitly says he has not checked. He gives a practical explanation for the initial partner lineup: those engineering teams were willing to iterate with OpenAI while it built the platform and worked through implementation challenges.

The rollout is described as intentionally small to investigate new avenues for harm, including poorly understood risks. Waitlist project descriptions help select useful applications and screen out potentially harmful work. Developers can prepare a backend before receiving access, but only a small group can install and test developer-unverified plugins through ChatGPT.

Plugin specifications compete with conversation for context. The implementation places OpenAPI information, descriptions, sample requests and sample responses inside the model’s context window—the limited space available for those instructions and the conversation. More plugins therefore leave less room for conversation. Wider availability of GPT-4’s 32K context window is presented as a prospective way to fit more plugins, without a demonstrated plugin count.

Several other boundaries become explicit: - Plugin-enabled API calls are unavailable: developers cannot yet programmatically call ChatGPT with plugins enabled. Whether or when that will ship is unresolved. - Local simulations cannot fully reproduce the system: Logan says some functionality uses a different model unavailable to developers. Injecting descriptions into a base model might approximate its behavior. - Browsing, plugins and Python remain separate experiences: the plugins experience cannot communicate with browsing.

Returning to the opening video demonstration, Logan says his understanding is that it uses the ordinary interpreter, without special setup. He also suggests generating code for scheduling and dispatch, but does not demonstrate scheduled execution or establish its operating conditions. The video example supports the compute-platform interpretation without establishing every proposed workload.

Product quality after connectors get cheaper

Peter proposes a marketplace business built around trust: paid tiers of validation, compatibility testing and compliance certification. In his view, validation needs an ongoing component, because data flowing through plugins can vary in quality or interfere with safety systems. Citing JetBrains, Postman and cloud marketplaces as precedents, he predicts that meaningful approval would be worth paying for.

Flo shifts attention to products built around foundation-model APIs: applications that use the models as a component. Where the host praises Lindy’s interface, Flo argues that the opportunity is moving beyond parameter counts toward product quality. The plugin announcement makes capabilities that Lindy had built itself more widely available, lowering the barrier to entry. Product expertise remains a source of differentiation.

The exchange also revisits a prediction that OpenAI would never prioritize Gmail or Google Calendar. The response rejects that certainty and argues for compatibility driven by the user’s interests. An independent assistant could work across competing ecosystems without needing to protect one incumbent’s products.

Lindy’s integration approach is described as supplying API documentation and letting the assistant work out how to use it. The reported integration time is about fifteen minutes, though no complexity range or validation procedure accompanies that estimate. A proposed future workflow would let power users supply documentation and an API token themselves. Participants argue that cheaper integration development could weaken a traditional advantage of open-source connector platforms: communities’ ability to expand coverage faster than a single vendor.

The quality debate exposes two different concerns. One response treats integration quality as correctly calling endpoints and supporting their parameters. Another asks about data quality and interference with safety systems. An email-drafter developer makes the distinction concrete: they deliberately excluded automatic sending because constructing an email correctly does not establish authority to send it.

Confirmation tokens and the limits of model behavior

Lindy’s described guardrail separates read actions from write actions, particularly flagging POST endpoints as writes. For a write, user confirmation issues a token required to execute the action. Later details specify that some tokens expire and authorize only one action. The design places an authorization check around execution instead of relying solely on instructions to the model.

The speaker describing this design distinguishes assistant failures and misinformation from long-term existential risk. They view the former as engineering challenges that guardrails can address, while expressing much greater concern about the latter. Their product outlook remains optimistic: language alone can support substantial applications, even as audio and images promise another qualitative advance. The analogy is a CPU—a component that keeps improving while product teams build useful systems around it.

The Gmail execution path reveals the scope of the guardrail. The model writes code against supplied functions, such as an email-sending function with recipient, subject and body parameters, rather than normally constructing REST requests through a library. But the initial assertion that unauthorized writes are impossible is subsequently qualified: generated code could theoretically bypass those functions and call endpoints directly. The speaker has not observed that behavior and says the model was not trained to behave that way. Not observing a bypass does not prove it is blocked.

The discussion separates two permission problems: - Authorize the action: require confirmation and an action-specific token before a write executes. - Restrict the execution environment: control which network destinations generated code can reach. A proposed proxy could enforce such restrictions; access to only part of a domain remains an open design question.

Deno’s permission model raises a usability concern. One speaker criticizes permissions applied to an entire executable, arguing that people encountering errors may keep broadening access until protection is weakened. Another reports using a separate subprocess with restricted access, but finds the workaround cumbersome. The precise isolation design remains unresolved.

A conversation with a computer that does the work

The closing response welcomes ChatGPT’s capabilities without yet seeing them as direct competition, and predicts a market large enough for many winners. The discussion returns to its starting point: FFmpeg, generated Python, stored files and downloadable results. A participant wants to build for this emerging compute platform, even without knowing what to build.

The final prediction is that language models become an operating-system interface: people converse with computers that perform work on their behalf. The video-editing example gives that ambition a concrete form—interpret a request, execute work and return the result. The permissions debate supplies its unresolved counterpart: how to ensure the system acts only within the authority it has been given.