OpenAI DevDay, Part 2 of 4: From Configuring Bots to Running Workflows

An assistant that notices what you forgot

During investor pitches, Avi used an AI companion that sent phone notifications reminding him of points he had forgotten to mention. The example opens this part of the DevDay conversation with a concrete ambition: assistance during an ongoing task, while a reminder can still help.

A participant imagines extending that experience into a hardware assistant loaded with several GPTs—customized bots with their own behaviors. The hardware idea is speculation. The same discussion expresses confidence that OpenAI will leave room for independent builders, even as the pace makes planning difficult: ChatGPT’s first anniversary on November 30 is still ahead, and one participant struggles to forecast even four weeks.

Jim Fan: cheaper APIs and shareable behavior

Jim Fan makes API economics tangible with back-of-the-envelope workload estimates: - Text: roughly $15 to process all seven Harry Potter books as input, and $45 to generate an equivalent volume of text. - Images: approximately $180 to process about 20 hours from the eight movies, sampled at one frame per second. These are cost calculations, not completed runs or demonstrations of book-length generation. Exact model variants are unclear, and the image resolution is transcribed as “260p,” limiting reproducibility.

Fan sees economies of scale as a competitive advantage that other providers will struggle to match. He predicts exponentially cheaper APIs as compute costs fall and scale increases—a forecast about pricing, rather than an established law.

Integration is his second theme. He praises the Assistants API as a common interface for stateful interactions—interactions that retain state—and capabilities such as Code Interpreter. Switching among capabilities makes the interface more flexible. His description also includes browsing; compatibility with every store plugin is something he speculates about, not a confirmed feature.

GPTs turn that integration into a user-generated-content platform where people can share complete bot behaviors. Fan imagines programming through dialogue: describe the desired behavior, refine it, and answer the builder’s clarification questions. The creation interface gives this idea a concrete form, with conversation in one pane producing instructions in another. Users need not know their full specification before they start.

The exchange compares this to onboarding an employee: explain expected behavior, then resolve unclear details through follow-up questions. But a better configuration interface does not remove the underlying constraint. Fan argues that agent robustness is bottlenecked by the model. He interprets GPT-4 Turbo’s improved function calling and tool switching as signs of tuning toward agent use, while leaving open what builders will achieve. Shawn’s suggestion that a unified interface could use internal model routing remains speculation.

Fan identifies customization as the event’s central theme, with three ways to specify behavior: - Through APIs: combine assistant capabilities, including managed retrieval. - Through dialogue: refine instructions conversationally. - Through data: fine-tune GPT-4 on a supplied dataset. He calls this programming by data and anticipates that it could produce new behaviors; he presents no experimental result.

When native tooling removes a startup’s moat

Asked what excited him, Shawn starts with the breadth of releases. He had doubted that the DALL-E 3 API and text-to-speech would be ready. He recalls the Whisper 3 announcement receiving no applause, illustrating how much news competed for attention.

His next reaction is unease: OpenAI is incorporating retrieval and agent-building features previously supplied by startups. Managed retrieval reduces the work of building a document-backed chatbot by putting the capability directly into the playground, API, and ChatGPT. Shawn compares the competitive dynamic to AWS launch events; the accompanying “red wedding” analogy is explicitly a joke.

A conference organizer describes the work being compressed. Their summit bot required a LangChain stack and a website widget. Now a simple bot could be configured without code in the playground and offered there for people to try. The strategic condition is specific: if easy chatbot setup is the entire moat, native tooling can remove it. Another example is a stateful API that a speaker considered building, then abandoned because they expected the platform to provide it.

That leaves the question of where developers should invest. One participant criticizes the presentation for offering ecosystem builders little assurance about where they could work without competing with OpenAI. They accept disruption as part of frontier development, but still see substantial tension between the platform and startups depending on it.

Raza Habib: easy configuration still needs control

Raza Habib views GPTs and the GPT store as a second attempt at plugins, with a thoughtfully designed builder. Its mechanism is simple: conversation helps a nonexpert formulate the system prompt—the instructions governing the bot’s behavior—and select its abilities. The builder elicits a specification from someone who might struggle to write one unaided.

Among longer context windows, support for multiple media types, and cheaper, faster models, Habib singles out JSON mode: structured output without requiring a function call. An interlocutor describes inventing a fictitious function solely to obtain JSON through the function-calling interface. Habib argues that structured output has broader uses and relays reports of invalid JSON in complex function-calling cases. He welcomes greater reliability without claiming a measured improvement.

Managed retrieval draws a more skeptical response. Habib argues that uploading a collection of documents is rarely sufficient for enterprise applications. The discussion separates three requirements: - Document treatment: how documents are handled affects retrieval’s usefulness. - Permissions: users may be allowed to access different portions of the data. - Context transparency: developers need to understand how a stateful API maintains history and selects what the model receives.

Delegating state management also delegates context decisions. The threaded interface is appealing, but the participant asking how those decisions work reports leaving the API booth without answers. Convenience alone does not resolve the need to understand what enters the model’s context.

Adding image and audio inputs and outputs to Humanloop is described as a relatively straightforward extension of the existing product, still an expectation rather than a completed integration. Fine-tuning receives a practical sequencing rule: get the application working, then fine-tune it. The analogy is compilation—an optimization applied once useful behavior exists.

Voice quality and the cost of idle compute

The text-to-speech demo earns praise for its prosody, with hands-on testing still needed. A participant also relays a secondhand claim of internal voice cloning from roughly 30 seconds of speech. Release plans remain unknown, and the participants identify safety concerns as a possible barrier.

Repeated price cuts are described as building confidence in OpenAI and weakening the economic case for self-hosting even a smaller fine-tuned model. The comparison depends on utilization. Avoiding cold starts—the delay of starting a deployment—can require keeping compute running continuously. Sparse traffic then leaves paid capacity idle.

Two operating conditions lead to different choices: - Steady, sufficient traffic: usage may justify continuously running compute. - Intermittent traffic with a need to avoid cold starts: idle capacity can make a third-party API competitive. No numerical break-even volume is established; the economics remain use-case dependent.

Surya Dantuluri: migrating functionality leaves distribution unresolved

Surya Dantuluri built plugins for chatting with PDFs, videos, websites, and Git, initially to help with homework. Chat with Instructions saved additional custom instructions for GPT-4 conversations. He reports paying users in 85 countries, without disclosing revenue. To handle monetization, he built Custom Plugins, avoiding another provider’s stated 10% fee.

His post claiming that Sam Altman had killed his startup began as a joke to help him meet people at DevDay. Hours later, he opened the new ChatGPT interface on his phone and found the plugin option missing. He believed that interface was limited to event attendees and had not checked his usage statistics. The immediate problem was migration uncertainty; customer losses had not been measured.

Dantuluri says he was told to create a new GPT and import the plugin’s schema. His concern was preserving access to existing users: he did not know of a way to link them to the replacement. Recreating the functionality could still leave people needing to discover the product again.

He nevertheless considers GPTs a better abstraction and describes his plugins as a side project. He welcomes several changes: - Shareable links improve discovery: in his account, plugins required searching through the store. - Revenue sharing reduces payment plumbing: monetizing plugins had required cumbersome implementation. - Instructions, context, and retrieval package behavior together: these are capabilities he thinks plugins should have included.

Dantuluri compares GPTs with Poe bots, which he says had already let developers combine custom instructions and a selected model for purposes such as poetry or essays. But more capability has not yet given him a clear product plan. He is reluctant to reveal ideas after seeing earlier ones copied, and acknowledges having few strong ideas for products using the new releases.

The exchange returns to retrieval’s limits. A loose reference to an internal vector database is clarified as retrieval, without establishing a separately exposed database or its storage architecture. A participant argues that simple configurations can serve initial use cases, but developers eventually want control over chunking.

A closing comparison with Claude adds a practical qualification. One participant had relied on its 100K context window and convenient PDF uploads, but reports refusals that week when documents contained copyright markings. Another suggests that asserting authorship could override them. Neither behavior is demonstrated in the exchange.

Zapier: constrain tool choice and translate API data

Reid Robinson introduces Zapier AI Actions, launched for GPTs alongside an integration update supporting the Assistants API. AI Actions rebrands the Natural Language Actions API introduced with ChatGPT Plugins. Robinson sees tighter control over prompts and available apps as a key improvement.

The original plugin sometimes searched Gmail when asked for next week’s agenda, even with Google Calendar available, reasoning that invitations arrive by email. A plausible tool choice can still miss the intended workflow. GPTs let builders specify which apps to use and how, reducing that ambiguity.

Across Zapier’s roughly 6,000 apps at the time, Robinson describes three responsibilities: - The model supplies intent: for example, check a user’s calendar for Monday. - Zapier constructs the request: it translates the natural-language instruction into the service’s API call. - Zapier reshapes the response: it processes lengthy Gmail or Calendar payloads into the information the model needs, instead of returning the entire payload.

AI Actions are individual operations, not one-for-one ports of existing Zaps. They can be chained—the demo combines a Calendar search with a Slack action—but the human in ChatGPT triggers the work. The Assistants API opens another route: assistant workflows that run automatically outside an immediate chat session.

Scheduled analysis can become an ongoing conversation

Robinson proposes a weekly analytics workflow to make asynchronous execution concrete: 1. Receive a scheduled CSV export from Looker, Mixpanel, or Amplitude—perhaps every Friday at 8 a.m. 2. Send it to an instructed assistant to identify the top ten blog articles and analyze keywords, search-engine optimization tags, and conversions, depending on the supplied data. 3. Deliver the response to Slack, potentially with charts and graphs. This is a proposed workflow, not a reported production result.

He says recipients could reply to the report in Slack and continue talking with the assistant, though implementation details are not provided. Automation starts the analysis; conversation continues it.

Part 2 closes on the distinction between one-off actions in ChatGPT and asynchronous work. API support for vision, Code Interpreter, and retrieval gives assistants capabilities to use in those workflows. A person can initiate work in chat, or an automated process can deliver a result that people pick up afterward.