DevDay, Part 2 of 4: Configurable Assistants and Platform Risk

An assistant that notices what you forgot

During investor pitches, Avi reportedly received phone notifications reminding him of points he had forgotten to mention. Tab, a personal AI companion capable of real-time conversation, supplied the reminders: assistance delivered during an activity, without waiting for a separate question.

The GPTs announcement prompts speculation about extending that experience to hardware carrying several specialized GPTs. This is a proposed direction, not an announced OpenAI product. The opening discussion also argues that opportunities remain for independent builders because OpenAI will not build everything. These reactions in the DevDay recording come before ChatGPT’s first anniversary on November 30, after roughly eleven months of change.

Cheaper inference and programming through dialogue

Jim Fan begins with the economics of processing much larger amounts of material. His back-of-the-envelope estimates give two illustrations: - Books: about $15 in input charges for all seven Harry Potter books, or $45 for an equivalent volume of output. - Film frames: roughly $180 for the eight films—about twenty hours—sampled at one frame per second, at a resolution transcribed as “260p.” These are cost estimates, not comprehension experiments. Inconsistent model names and the ambiguous resolution in the transcript prevent precise reproduction of the calculations. Fan expects prices to keep falling as compute becomes cheaper and economies of scale grow, making OpenAI’s scale difficult for competitors to match.

Fan’s next theme is integration. He welcomes the Assistants API for bringing together code execution, stateful interaction that retains information across exchanges, and switching between capabilities. His suggestion that it could integrate all store plugins remains speculative. He sees GPTs as shareable behaviors: configured bots that people can distribute, extending what they previously shared as prompts.

The GPT builder makes this concrete through programming by dialogue. Users describe desired behavior, answer clarification questions, and refine the result. The interviewer points to the two-pane interface, where a conversation produces instructions in another pane, as useful when users do not yet know exactly what they want. The exchange compares this with onboarding an employee: explain expectations, then resolve unclear details through follow-up questions.

These configured assistants are treated as an early step toward agents, with an important limit: agent robustness depends on the underlying model. GPT-4 Turbo appears promising because of better function calling and tool switching, but what builders will produce remains open. The discussion also proposes an internal model router to explain the unified interface; that is a hypothesis about implementation.

Customization connects three ways of shaping an application: - Through APIs: combine assistant capabilities, including retrieval handled by the platform. - Through dialogue: refine instructions and behavior conversationally. - Through data: fine-tune GPT-4 with a custom dataset, potentially producing behavior beyond prompting alone. That last possibility is a direction for experimentation, without a demonstrated result here.

When built-in integration removes differentiation

The interviewer’s reaction begins with the density of releases: the DALL-E 3 API, voice, and text-to-speech exceeded his expectations. He recalls Whisper 3 receiving no applause when announced, illustrating how crowded the day felt. That excitement leads to a harder question for startups selling capabilities OpenAI now includes.

A conference website chatbot provides the comparison. The interviewer had assembled a LangChain stack and a website widget; he now sees a comparable bot as configurable without code in the playground. Built-in retrieval reduces the external tooling needed for a simple chatbot that answers using supplied documents. His competitive argument is conditional: if assembling that integration is the entire moat, built-in support removes the differentiation.

He had also considered building a stateful API, then stopped because he expected the platform to supply one. His criticism of DevDay is that it offered ecosystem developers little reassurance about where to build as OpenAI expanded. The joking “red wedding” comparison expresses that anxiety rather than documenting startup failures. He accepts disruption as a risk of frontier work while arguing that the tension deserves attention.

Raza Habib: structured output and controllable retrieval

Raza Habib sees GPTs and the GPT store as a promising second attempt at plugins. He particularly likes how the builder helps people without prompting expertise. Its mechanism is simple: the conversation fills out the system prompt—the instructions governing behavior—and selects capabilities. Making that configuration accessible is the product achievement.

Among longer contexts, multimodality, and cheaper, faster models, Habib singles out JSON mode: structured output without a function invocation. The interviewer had been inventing functions with no implementation simply to obtain JSON through function calling. Habib distinguishes that workaround from a dedicated output mode: JSON is useful beyond tool calls, and users had reported invalid JSON in complex function-calling cases. He welcomes the prospect of greater reliability without claiming a measured improvement.

His reservations concern managed retrieval, where developers upload documents and the platform retrieves material for the model. This is convenient for low-code GPTs, but he questions whether it exposes enough control for enterprise applications: - Chunking: how documents are divided into pieces matters to retrieval. - Permissions: different users may be entitled to different portions of the data. Simply uploading documents does not resolve those requirements.

Stateful assistants introduce a related question: what information actually enters the model’s context? Habib welcomes the threaded Assistants API but wants transparency about how it maintains state and selects information for the model. He says his questions at the API booth went unanswered, and considers that transparency important for adoption.

For Humanloop, Habib describes multimodal support as a straightforward extension of existing workflows to image and audio inputs and outputs. Fine-tuning belongs later in development: get the application working, then optimize it through fine-tuning. His analogy is compilation—a step applied after useful behavior has been established.

Voice quality and the cost of avoiding cold starts

The text-to-speech demo impresses Habib through its prosody and variation, though he wants more hands-on testing. He also relays a secondhand report of internal voice cloning from roughly thirty seconds of speech. The participants are unsure whether that information is public or whether the capability will be released, citing safety concerns. It remains distinct from the demonstrated text-to-speech offering.

Pricing returns the conversation to infrastructure. Habib argues that repeated price cuts build developer confidence and make hosting even a smaller fine-tuned model less obviously economical. Utilization and cold starts determine the comparison: compute billed by the hour or minute may need to remain running to avoid startup latency, including periods with few requests.

The workload changes the decision: - Sustained usage can justify continuously available compute. - Sparse usage with latency requirements can make an API cost-competitive because a separately hosted model must remain running between requests. The interviewer counters that an hourly bill might still be manageable, using $5 an hour as a hypothetical. Habib agrees that the tradeoff depends on the use case; neither supplies a utilization or concurrency threshold.

A plugin developer faces a distribution reset

Surya Dantuluri brings platform risk down to a developer’s business. He describes plugins for chatting with PDFs, videos, websites, and Git, plus a tool for storing additional custom instructions. These projects reportedly attracted paying users in 85 countries. He built custom monetization infrastructure to avoid a third-party fee of 10%.

When he opened the new ChatGPT interface on his phone at DevDay, the plugins option was absent, along with separate Code Interpreter and DALL-E 3 options. He believed the rollout was initially limited to attendees. He says he was told that restoring a plugin’s functionality required creating a GPT and importing the existing schema.

His concern was user continuity beyond schema compatibility. He knew of no way to connect the old plugin to its replacement, potentially forcing users to discover him again. This was a migration concern, not measured user loss: he had not checked his statistics, and the interface had not rolled out widely.

Despite the disruption to his side project, he considers GPTs a better abstraction. He compares them with Poe bots configured through custom instructions and a selected model. Three benefits stand out: - Direct distribution: shareable GPT links avoid the plugin-store search process he describes. - Packaged behavior: custom instructions and context shape the specialized experience. - Built-in monetization: revenue sharing could replace awkward custom billing work, although no rates or payout results are given.

Enthusiasm for the capabilities has not yet produced many strong product ideas, and Dantuluri says earlier plugin ideas he shared were copied quickly. The conversation also distinguishes built-in retrieval from an exposed, configurable vector database: the interviewer expects demanding uses to require control over chunking.

Dantuluri had used Claude heavily for its 100K-token context window and PDF uploads, which worked well for him. He reports more recent copyright-related refusals when documents contained a copyright symbol or name. This is a user observation rather than a systematic evaluation.

Zapier: constrain tool choice and translate both directions

Reid Robinson introduces Zapier AI Actions, a natural-language interface to application searches and actions. It rebrands the Natural Language Actions API introduced with Zapier’s ChatGPT plugin. Zapier also updated its ChatGPT integration to support the Assistants API.

For Robinson, GPTs improve control over which tools an assistant uses. The original plugin needed access to every action a user might want. Given Gmail and Google Calendar, it sometimes searched Gmail for next week’s agenda because calendar invitations arrive by email. GPTs let builders specify both the relevant applications and how to use them, reducing that ambiguity.

Across roughly 6,000 Zapier applications at the time, AI Actions performs two kinds of translation: - Requests into API calls: the model asks for a calendar check on a specified date; Zapier converts the instruction into the application’s API call. - Responses into relevant information: Zapier reshapes long, confusing payloads before returning them to the language model. The integration layer therefore does more than expose endpoints. It prepares the information the assistant receives.

From a human trigger to a scheduled assistant

Robinson distinguishes individual AI Actions from existing Zaps transferred one-for-one. Actions can be chained—a Google Calendar search followed by a Slack action—but in the ChatGPT interaction, the human supplies the trigger. Through the Assistants API, Zapier can instead run workflows automatically and asynchronously.

His proposed weekly analytics workflow makes the sequence concrete: 1. Export blog traffic every Friday as a CSV from an analytics tool. 2. Send it to a configured assistant to identify the week’s top ten articles and examine the supplied keywords, search-engine optimization tags, and conversion data. 3. Post the analysis to Slack, potentially with charts and graphs. 4. Continue in replies, letting team members ask the assistant follow-up questions. The depth of analysis depends on the supplied data. This is a proposed use case, not a measured result.

Robinson says creating and running an assistant through this kind of automation was already supported. He then begins another use case: provide company policy documents to an assistant supporting a team that approves budget requests, with Slack as a possible interaction point. Part 2 ends as that policy-retrieval example begins, before the Slack connection or approval behavior is specified.