From o1’s Thinking Budget to Working Applications — Part 3 of 3

Reasoning can use the budget before an answer arrives

Counting the Rs in “strawberry” gives the closing Q&A of the OpenAI API and o1 recording a concrete example: participants describe o1 answering correctly and quickly, while complex prompts can take much longer. But waiting does not guarantee an answer. Reasoning can exhaust its token allowance before producing a final response.

The discussion establishes no universal maximum thinking time or reasoning-token cap; a respondent describes behavior beyond the documented API as still flexible. One participant reports a 121-second query, although the prompt and timing conditions are unclear. A question about scaling compute through further orders of magnitude receives no additional results beyond the published blog post.

The remaining questions separate two issues: - Another avenue of innovation: a respondent rejects the inference that adding reasoning means foundational models have reached a limit. - Performance at matched compute: an audience member asks for a comparison against GPT-4o with a matched reasoning budget. Respondents point to research blogs and a technical report, but do not identify that specific evaluation.

The AMA: long reasoning inside a single model

The recording next presents a narrated recap of the research team’s AMA. Romain Huet explains the naming reset as marking a new level of reasoning capability, with “o” standing for OpenAI. Shengjia Zhao and Max Schwarzer distinguish the releases: - o1-preview is an early version of o1, with no size difference from the full model described in the AMA. - o1-mini is smaller and faster, not a preview of a future model. Possible near-term updates were not guaranteed. Precise model sizes were not disclosed.

In the AMA recap, Noam Brown describes o1 as trained to generate a long chain of thought—the intermediate reasoning before its final answer. Hyeong Won Chung emphasizes that this is a single model. Its answer is typically, though not always, much shorter than its reasoning.

The displayed thinking summary is not guaranteed to faithfully represent the reasoning. Brown says there were no plans at the time to expose the raw chain of thought through ChatGPT or the API. Chung explains that the visible thinking stage summarizes the process, making it appear slower than it is. The interface therefore does not provide a complete trace of the model’s intermediate work.

Jason Wei argues that more aggressive prompting of GPT-4o would probably not yield an IOI gold result—a judgment offered without a supporting evaluation here. His task-specific qualification is equally useful: o1-preview could be little better than GPT-4o, or slightly worse, on personal writing. He distinguishes that comparison from strong evidence that increasing scale worsens performance.

Model capability outpaced the available API

The AMA describes several gaps between model capability and what developers could use at the time: - Tools and structured responses: Chung says o1-preview did not use tools. Michelle Pokrass says work was underway on function calling, structured outputs, developer/system messages, and standard parameters. Ahmed Elkishky describes iterative deployment of code interpreter and browsing, without an exact preview duration. - Thinking-time control: Brown expresses a desire to let users control how long the model thinks, without announcing an available control or delivery date. - Multimodal access: Wenda Zhou describes built-in multimodal capabilities and claims state-of-the-art performance on MMMU, an evaluation not explained in the excerpt. Safety work remained before release; no score or evaluation conditions accompany the claim.

Access and economics were also evolving. Nikunj Handa ties batch support to higher rate limits. Shengjia Zhao predicts continued price reductions based on historical trends, not a pricing commitment. Łukasz Kondraciuk offers an early development example: a couple of pull requests to OpenAI’s codebase had been authored solely by o1, although their scope and review outcomes are not described.

The recording then reads Tibor Blaho’s recap of the same AMA. It attributes improvements in o1’s chain of thought to reinforcement learning and says instructions can influence how the model approaches a problem. It also describes researchers using o1 to build a GitHub bot that pings the appropriate code owners for review.

That recap adds practical distinctions for integration and model selection: - Reasoning could not be interrupted with new context. Pausing inference to insert information was being explored. Streaming was planned; API visibility into reasoning progress was under consideration. - Smaller did not mean weaker on every task. o1-mini could outperform o1-preview on some science, technical, and coding tasks despite limited world knowledge. Its reported data cutoff was October 2023, with fresher data planned. - Input accounting remained familiar. o1 used the same input tokenizer as GPT-4o. - Access limits were provisional. The recap reports 50 o1-mini prompts per week for ChatGPT Plus users, all counting equally. Broader API access and higher limits were planned; prompt caching and fine-tuning had no announced timelines.

The prompting advice is to supply useful edge cases or reasoning styles and select context carefully. Retrieval-augmented generation supplies retrieved material as context for an answer. The recap says relevant material helps, while irrelevant chunks may worsen reasoning. No quantitative comparison accompanies that advice.

Lighthouse: translating research into immigration documents

The first demo turns from the o1 model to the O-1 visa. Min introduces Lighthouse, an AI-assisted immigration-document platform, and describes a typical application as a 300–400-page research report connecting an applicant’s achievements to immigration criteria. The central task is translating technical work for a nontechnical officer: a research abstract written for peers may need substantial rewriting for that audience.

Min presents the platform as a ready-to-use paralegal that reduces drafting and attorney back-and-forth. The live example selects a person identified as Daniel and five of his papers, then follows this workflow: 1. Fetch and summarize the papers. Min says this usually takes one or two minutes. 2. Apply legal-domain guidance developed with expert feedback. The demonstrated synthesis emphasizes machine-learning products. 3. Edit, revise, and obtain client review so the applicant can check that the account is sound.

Technical knowledge and legal presentation meet in the draft. Min argues that applicants understand their work but may not know how to present it for an application, while lawyers may struggle to summarize the technical contribution. Lighthouse combines domain guidance with AI drafting and also supports tasks such as assessing compensation comparables.

Min estimates that Lighthouse reduces roughly 120 hours of work with a traditional provider to about 20, recalling a roughly 500-page document prepared for physical printing and shipping. The calendar-time comparison is preparation in about 30 days or fewer versus a traditional process that can take six months. These are presenter estimates. A further estimate of a government response within two to three weeks is given without specifying processing conditions.

o1-mini was a prospective upgrade. Lighthouse was an OpenAI customer, but Min requests access to the model, hoping its reported technical strengths could improve drafting precision. The demo does not establish existing o1-mini use or a measured accuracy gain. Min closes by saying Lighthouse supports O-1 and all US employment-based visas.

From o1’s plan to an executable script

The closing demo tackles the missing API features directly. A presenter describes a prototype built over the preceding three hours to compensate for o1’s lack of structured outputs and tool calling. It uses E2B, a cloud execution sandbox, and the presenter discloses being an investor. The task is to visualize the growth of the Roman Empire.

o1 plans and writes code; 4o mini packages it for execution. The application prompts o1 as though it has access to a code interpreter. Its response contains a plan, explanations, and code, which 4o mini converts into one script. The application then follows an execution-and-repair sequence: 1. Extract structured output containing the code, required pip packages, filename, and execution command. 2. Open a sandbox, install the dependencies, and run the generated command. 3. On failure, feed the error back to the model, use 4o mini to extract revised code, overwrite the script, and retry.

The presenter calls this a self-healing loop, though no aggregate repair success rate is reported. The demo exposes two distinct failures: - The date representation breaks execution. A negative year triggers a Pandas date-handling error. The presenter says o1’s initial code missed the issue and 4o mini could change the representation to handle dates before the common era; the exact conversion is not specified. - The output path blocks artifact retrieval. The live sandbox integration cannot retrieve its generated PNG files. The presenter subsequently runs copied code locally and reports showing population and territorial-area charts, including an overlaid view.

The closing proposal is dataset generation from information learned through text: drawing fragmented material into a structured form. The Roman Empire example demonstrates that ambition, but its generated historical values are not validated. Producing runnable charts and establishing the dataset’s accuracy remain separate tasks.