OpenAI Runs Autonomous Coding Agent for 1,000 Hours at $16 an Hour

What Will You Build With Coding Agents That Don’t Quit? | DevDay 2026 — OpenAI

OpenAI

Engineering teams can eliminate continuous integration maintenance by delegating debugging to autonomous multi-agent hierarchies, which now resolve complex code failures around the clock for $16 an hour.

AI-assisted summary of What Will You Build With Coding Agents That Don’t Quit? | DevDay 2026 — original source. Check the original for full context.

Source published:

Key ideas

  • OpenAI showcased autonomous Codex deployments running continuously for weeks, including a 1,000-hour CI maintenance agent that authored over 100 pull requests per week and directly merged 213 fixes. (5:46)
  • Prompting methodologies have evolved from hyper-prescriptive execution plans to concise, outcome-driven specifications: a 200-word prompt without code governed the six-week CI agent run. (4:12)
  • The operational token cost of running continuous 24/7 maintenance agents was approximately $32 per hour initially and dropped to $16 per hour with GPT-6 Sol. (24:01)
  • Long-horizon agent stability relies on native context compaction, Git worktrees for parallel isolation, and remote cloud environments with periodic heartbeats for asynchronous code review loops. (15:11)

Detailed notes

## Background Published on YouTube by OpenAI on November 11, 2026, this DevDay presentation details insights gained from an autonomous Codex run that operated for more than 1,000 hours maintaining continuous integration builds and test suites. The talk covers criteria for selecting recurring engineering tasks, structuring feedback loops, optimizing token spend, and establishing clear boundaries for human intervention.

## TL;DR OpenAI engineers Friel and Vivi showcase how frontier models have transitioned from brittle, micromanaged execution plans to durable, outcome-driven agents capable of running autonomously for weeks. Drawing on a flagship experiment where Codex ran for over 1,000 hours to resolve flaky CI failures and landed over 200 fixes, they break down the architectural patterns that enable multi-week stamina—such as sub-agent orchestration, Git worktrees, context compaction, and persistent documentation via `AGENTS.md`. The presentation highlights how operational economics have plummeted to roughly $16 per hour with GPT-6 Sol and provides a five-step playbook for engineers looking to offload repetitive operational toil.

## Atmosphere & style The session balances visionary frontier demos with pragmatic, trench-level engineering realities, driven by easygoing camaraderie and self-deprecating humor between the co-presenters. Moving from creative spectacles like autonomous 21-kilometer Nürburgring 3D renders to the unglamorous grind of debugging floating-point CI flakes, the narrative feels like an honest, insider post-mortem rather than a polished sales pitch. Watching it evokes both relief at the prospect of delegating tedious maintenance backlogs and excitement about transitioning from hands-on coders to directors of autonomous agent teams.

## Bulletin - [05:46](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=346s) OpenAI showcased autonomous Codex deployments running continuously for weeks, including a 1,000-hour CI maintenance agent that authored over 100 pull requests per week and directly merged 213 fixes. - [04:12](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=252s) Prompting methodologies have evolved from hyper-prescriptive execution plans to concise, outcome-driven specifications: a 200-word prompt without code governed the six-week CI agent run. - [24:01](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1441s) The operational token cost of running continuous 24/7 maintenance agents was approximately $32 per hour initially and dropped to $16 per hour with GPT-6 Sol. - [15:11](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=911s) Long-horizon agent stability relies on native context compaction, Git worktrees for parallel isolation, and remote cloud environments with periodic heartbeats for asynchronous code review loops.

## Key takeaways - [00:19](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=19s) Autonomous agents can synthesize complex 3D assets and animated fly-throughs from minimal inputs, including full crash recovery across multi-day rendering runs. - [04:12](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=252s) Prompt engineering has transitioned away from micromanaged step-by-step scripts toward outcome-focused prompts that define boundaries and trade-offs. - [05:46](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=346s) A continuous integration maintenance agent ran continuously for six weeks (>1,000 hours) using a prompt of just 200 words. - [07:25](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=445s) Delegating debugging tasks to parallel sub-agents transformed the lead agent into an orchestrator, scaling throughput across 5,800+ sessions. - [07:25](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=445s) The autonomous CI agent resolved complex non-deterministic issues, such as a floating-point Heisenbug in GPU test environments. - [10:28](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=628s) Agents can independently isolate physical hardware issues down to specific cluster nodes and coordinate technical inquiries with human system owners. - [10:28](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=628s) To prevent aggressive or reckless test deletions, the system instantiated reviewer subagents to audit proposed code removals before human escalation. - [12:12](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=732s) Aligning one long-lived thread to a single objective alongside an AGENTS.md file provides persistent guidelines, off-limits boundaries, and operational scratchpads. - [15:11](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=911s) Default context compaction eliminates context rot, enabling uninterrupted agent threads to run stably for months at a time. - [17:55](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1075s) Git worktrees give parallel child threads isolated checkouts, preventing merge conflicts and race conditions when subagents build concurrent features. - [20:46](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1246s) Ultra mode equips Codex to tackle deep, complex bottlenecks within a single thread by coordinating specialized diagnostic and patching subagents. - [22:27](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1347s) Automated heartbeats wake agents on scheduled intervals to inspect pull request feedback, handle reviewer remarks, and resume blocked work. - [24:01](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1441s) Running continuous engineering agents around the clock cost $32/hr initially and halved to $16/hr with GPT-6 Sol, offering high leverage relative to developer time. - [25:17](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1517s) Deploying production agents successfully requires identifying operational time sinks, establishing closed automated verification loops, and enforcing explicit escalation boundaries.

## Detailed walkthrough ### [00:19](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=19s) Pushing Models to Build 3D Formula 1 Worlds

Friel, working on engineering acceleration at OpenAI to help engineers and researchers keep shipping, introduces the session alongside Vivi from the developer experience team. Vivi explains that much of his day involves experimenting with frontier models to discover how far they can be pushed. Lately, that experimentation has centered on generating rich 3D worlds using Codex and Astra. As a passionate Formula 1 enthusiast, Vivi wanted to see whether Astra could dissect an F1 vehicle and render its internal anatomy, answering basic curiosity about what the cars are composed of.

```mermaid flowchart TD accTitle: F1 Modeling and Circuit Generation Architecture accDescr: Pipeline showing how Codex and Astra transformed reference assets into 3D models and an animated fly-through of the Nürburgring. subgraph Ferrari["Ferrari 3D Exploration"] A["7 Reference Photos"] --> B["Codex + Astra on Remote Machine"] B --> C["Rendered 3D Model"] B --> D["Clay Version"] B --> E["Wireframe Version"] B --> F["Interactive Explorer Website"] end subgraph Circuit["Nürburgring Nordschleife Fly-Through"] G["Track Name Prompt"] --> H["Codex Autonomous Controller"] H --> I["Research Sub-Agents: Geometry & Track Data"] H --> J["Blender Frame Rendering Engine"] J --> K["Crash Recovery: Resumed from Saved Frames"] K --> L["21 km, 70-Turn Cinematic Video"] end ```

To explore the car, Vivi executed Codex paired with Astra on a remote machine under a single objective: construct a Ferrari alongside an interactive website allowing users to explore all its underlying components. The model received only seven reference photographs. Over several days of autonomous iteration, Astra not only assembled the complete rendered 3D vehicle and exploration site, but also generated clay and wireframe visualizations without ever being prompted to do so.

- Goal given to the agent: “Build a Ferrari and a website where I could explore all that it's made of.” - Inputs required: Only seven reference photographs provided to Codex. - Autonomous output: Full 3D interactive model, website, unprompted clay render, and unprompted wireframe view.

[01:55](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=115s) Generating a 21-Kilometer Nürburgring Fly-Through

Vivi then expanded the scope from a single vehicle to an entire racing circuit, tasking Codex with creating a full fly-through animation of the Nürburgring. The agent was provided nothing beyond the name of the circuit, leaving it to scour references online, map out the track geometry, and chart its own camera path. Rather than settling for a short layout, Codex autonomously opted to recreate the complete Nordschleife circuit—spanning 21 kilometers and 70 distinct turns, an environment roughly three times the size of a standard Grand Prix circuit.

The resulting video took slightly over two days to generate. The bulk of that duration was pure compute latency, with Codex delegating the heavy lifting to Blender while standing by for frame renders. Codex dispatched sub-agents to independently research track layouts, verify geometric fidelity, and schedule rendering batches. Because rendering a 21-kilometer course is compute-intensive, the system encountered several hard interruptions caused by RAM exhaustion and CPU overheating. Codex demonstrated resilience by automatically detecting saved frames upon recovery, resuming progress where it left off, and completing the run across two days without human babysitting.

- Circuit scope: Full Nordschleife course covering 21 kilometers and all 70 turns (3× larger than an average F1 track). - Production duration: Slightly over two days, with most time spent waiting on Blender frame rendering. - Sub-agent utilization: Dedicated sub-agents handled track research, geometric verification, and render scheduling. - Fault tolerance: Automatically recovered from system crashes caused by RAM limits and CPU overheating by parsing cached render frames. - Vivi on agent stamina: “What was best about this was Codex automatically picked up from where these saved frames were and kept going for these two days.”

[04:12](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=252s) From Micromanaged Execution Plans to Outcome-Driven Agents

Reflecting on how quickly agentic capabilities have expanded, Friel looks back at the prior year's DevDay stage presentation where he introduced "exec plans"—a rigid scaffolding system designed to help Codex tackle hard multi-hour problems. At that time, keeping Codex running productively for seven hours was considered a massive milestone, requiring prompts so lengthy and hyper-prescriptive that they had to be broken across slides. Those early prompts dictated exact code samples, libraries, and precise implementation workflows, while demanding that the model produce an upfront plan for human review and iterative debate before taking action.

Today, prompting paradigms have shifted dramatically. Instead of prescribing low-level code mechanics or negotiating multi-step roadmaps, engineers now specify the target outcome, boundary constraints, and trade-offs, allowing models like Astra and Sol to chart their own implementation paths. This newfound reliability prompted Friel to examine recurring, maintenance-heavy engineering workflows that drain engineering time through relentless manual supervision.

- Evolution of prompts: Shifted from massive, multi-slide, code-heavy instructional prompts down to goal- and trade-off-oriented descriptions. - Autonomy comparison: A 7-hour run required constant hand-holding a year prior, whereas modern workflows run unprompted for days across complex tasks. - Friel on trusting modern models: “Today, with Astra and Sol, I'm much more comfortable just telling it specifically that. I tell it the goal and the outcome I want, what I care about, what are the trade-offs that I'm concerned with. And then, I tell it it can figure out the steps.”

[05:46](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=346s) The 1,000-Hour CI Maintenance Agent

Friel's team is responsible for engineering acceleration at OpenAI, which includes keeping continuous integration (CI) green across the organization so researchers and engineers can merge code and ship models. In large codebases, CI suites frequently fail due to flaky tests—tests that randomly fail on one execution and pass on the next—triggering deep investigations that can consume hours or days of an engineer's time while incoming failures continuously queue up. To break this cycle, Friel initiated an experiment to determine if an autonomous agent could shoulder CI maintenance indefinitely without ongoing human intervention.

```mermaid flowchart TD accTitle: Autonomous CI Triaging and Delegation Flow accDescr: Hierarchical agent architecture showing the Sol controller managing CI runs and directing parallel worker sub-agents. subgraph Manager["Primary Sol Agent (Controller)"] A["200-Word Rule Prompt"] --> B["Trigger CI Test Suites"] B --> C["Monitor & Collect Failures"] C --> D["Orchestrate & Dispatch Sub-Agents"] D --> E["Slack Coworker Communication & PR Landings"] end subgraph Workers["Sub-Agent Fleet (5,800+ Sessions)"] D --> F["Sub-Agent: Pull Code & Reproduce"] F --> G["Iterate on Fixes"] G --> H["Push to CI & Verify Pass"] end H -->|"Merge Candidate"| E ```

The resulting agent ran continuously for six weeks, clocking over 1,000 hours of runtime before being deliberately shut down—not due to failure, but because coworkers had begun deploying parallel agents of their own to handle similar workloads. In stark contrast to the massive prompts of the prior year, this autonomous CI run was launched with a prompt of just 200 words. It contained zero code and zero prescriptive fix patterns; it simply pointed the agent to the repository, defined rules for triggering test suites, instructed it on using CI compute resources conscientiously, and outlined how to communicate with human engineers on Slack.

- Total runtime: Over 1,000 operating hours across six consecutive weeks. - Prompt footprint: Approximately 200 words, establishing behavioral guardrails rather than implementation code. - Shutdown rationale: Turned off intentionally after colleagues began adopting and running their own maintenance agents. - Friel on the recurring CI burden: “Now, a test can fail legitimately because it's broken code, but often it will fail one time and pass the next. That's a flaky test. Someone then has to look at that and determine what the cause is... and the trickiest of these can take hours or days to debug.”

[07:25](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=445s) Delegating to Sub-Agents and Managing CI at Scale

The 1,000-hour deployment initially launched on model version 5.6 Sol running in a single thread. In this early setup, Sol sequentially diagnosed failures and authored patches within the same execution loop. However, four days into the experiment, Friel noticed the agent would get trapped debugging a single complex failure, stalling progress on newly arriving flakes. Friel interrupted the agent with a crucial course correction: delegate the repetitive debugging legwork to sub-agents.

This structural shift transformed the primary Sol thread into an orchestrator that gathered test signals, prioritized incoming failures, and assigned targeted issues to sub-agent sessions while coordinating PR reviews with human engineers over Slack. Sub-agents handled the time-consuming cycles of pulling repositories, reproducing flakiness, drafting code changes, and verifying test suites against CI. Joking about his evolving title, Friel observed that after describing himself as a "manager of agents" the previous year, he had effectively become a "director of agents," supervising a lead agent that commanded its own team.

```mermaid flowchart LR accTitle: Sol Autonomous Maintenance Metrics accDescr: High-level metrics tracking commits, PRs, tool calls, and merged fixes across the 1,000-hour agent run. A["1,000+ Operating Hours"] --> B["~500,000 Tool Calls"] B --> C["5,800+ Sub-Agent Sessions"] B --> D["67,000+ Wait Loops"] A --> E["1,000+ Commits"] E --> F["100+ PRs / Week"] F --> G["213 Self-Authored Merges"] A --> H["600+ Slack Coordination Messages"] ```

Over its six-week lifespan, the agent generated significant engineering throughput, authoring over 100 pull requests per week, pushing upwards of 1,000 commits, and broadcasting over 600 Slack messages to coordinate landing patches. In total, the system directly merged over 213 of its own code fixes. Whenever a proposed patch was skipped, it was typically because the agent noticed an engineer had already authored an equivalent fix, gracefully deprioritizing its own PR in favor of human work.

During a post-run retrospective, the agent demonstrated self-awareness regarding its workflow dynamics; it noted that its peak five-day stretch of productivity occurred when Friel completely stopped sending it messages. Across its half-million tool calls and 5,800 sub-agent sessions, more than 67,000 invocations were patience loops—waiting for long CI runs to complete, mirroring the exact downtime human engineers experience when stepping away for coffee while pipelines execute. Rather than picking easy fruit, the agent knocked down some of the most elusive technical debt in the repository, including fixing a non-deterministic Heisenbug triggered by floating-point math anomalies in GPU-based test environments.

- Operational velocity: Produced >100 pull requests per week and >1,000 total commits across the run. - Collaboration metrics: Sent >600 Slack messages to teammates and merged 213 autonomous fixes into the mainline. - Tool usage: Logged ~500,000 tool invocations, >5,800 sub-agent sessions, and >67,000 asynchronous wait checks. - Benchmark fix: Solved an intermittent Heisenbug involving non-deterministic floating-point discrepancies inside GPU tests that human engineers routinely bypassed by retrying. - Friel on team hierarchy: “Last year at DevDay, I joked that I'd kind of become a manager of agents, and this year, I've been promoted to a director of agents. I give an agent a job, and then it manages a team of agents to get it done for me.” - Friel on the agent's candid retrospective feedback: “I did a retrospective with the agent after the run, and while it didn't say anything directly about my managing style, it did say that it was most productive for a five-day stretch when I didn't message it once.”

[10:28](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=628s) Debugging Hardware and Interrogating Deletions

Reflecting on a particularly thorny investigation that was completely outside his wheelhouse, the speaker describes spending significant time reading through an agent thread to reconstruct what had occurred. Over several weeks of correlated logs, the agent managed to isolate an intermittent failure to a single specific node. The subagent spent several days teaching itself how to execute tests repeatedly on that exact machine until it reproduced the bug. Then came what the speaker considers the most remarkable breakthrough: rather than getting stuck or silently failing, the agent initiated contact with the researchers who owned the machine. It framed a structured technical collaboration, detailing how it reproduced the failure and asking whether the issue stemmed from an undeclared, out-of-scope machine-level software optimization or an actual physical hardware defect like a faulty GPU.

```mermaid flowchart TD accTitle: Agent Bug Investigation and Verification Loop accDescr: An autonomous agent correlates logs to locate a bad node, reproduces intermittent bugs, checks with system owners, and verifies solutions with reviewer subagents before pinging a human.

subgraph Investigation["Log & Node Investigation"] A["Correlate Weeks of Logs"] --> B["Isolate Faulty Node"] B --> C["Subagent Learns Node Test Loop"] C --> D["Intermittent Reproduction"] end

subgraph Escalation["System Owner Outreach"] D --> E["Engage System Researchers"] E --> F{"Software Optimization or Bad GPU?"} end

subgraph Resolution["Patch & Safety Check"] F --> G["Draft Fix or Test Removal"] G --> H["Spawn Reviewer Subagent"] H --> I["Interrogate Proposed Code"] I --> J["Human Pull Request Review"] end ```

The speaker notes that while the fastest way to get a continuous integration build back to green is often to simply delete the failing test, teams naturally harbor skepticism toward that shortcut. Whenever the agent questioned whether deleting a test was legitimate, it spawned a secondary reviewer agent tasked specifically with interrogating and verifying the first agent’s logic before ever escalating to a human engineer. Delegating this operational burden gave the speaker hundreds of hours back over the summer to focus directly on scaling infrastructure. As the speaker highlights: “And Codex giving me this time back allowed me to focus on scaling our infrastructure.”

- The agent correlated weeks of cluster logs to track intermittent failures down to a single physical machine. - Debugging and test-loop generation on that machine took several days of autonomous subagent execution. - Reviewer subagents audited proposed test deletions and fixes prior to human notification to prevent trivial test gutting. - Running Codex around the clock unlocked equivalent engineering effort spanning thousands of machine-hours.

[12:12](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=732s) Single Outcome Threads and AGENTS.md Context

Vibi takes over to break down the architectural features that make these multi-week agent runs possible, noting humorously that his co-presenter’s agent was dubbed "Frillbot." While the agent produced hundreds of commits and pull requests across the summer, Vibi boils the entire paradigm down to six essential Codex capabilities, starting with dedicating one thread to each specific outcome. Within a single dedicated thread, developers can map plans, monitor progress, supply iterative feedback, and redirect the agent on the fly. For engineers who prefer hands-off execution, Vibi points to slash commands: “Some of us are slash goal enjoyers over here. So, for those of you, you can even use slash goal and tell Codex to relentlessly go after that task and not stop until it is completed, no matter what happens.” In a continuous integration scenario, a thread’s objective can be framed to watch the build system, reproduce test failures, iterate on candidate fixes, and open PRs for review.

To ground that mission in institutional knowledge, developers configure an `AGENTS.md` file within the repository. This document serves as persistent operational guidance, defining where test suites live, outlining explicit engineering "dos," and staking out strict boundaries that are entirely off-limits. Vibi strongly emphasizes adding a requirement in `AGENTS.md` for Codex to maintain an active work log detailing both completed actions and pending explorations. If the agent ever runs into an environmental hiccup or execution boundary, it can simply read its own historical notes and resume seamlessly without starting from scratch.

- **Outcome-focused threads**: Keep one long-lived thread aligned with a single durable objective rather than scattering goals across disjointed chats. - **Relentless execution**: The `/goal` command locks the model onto an end-state and instructs it to cycle through obstacles without stopping. - **Rules of engagement**: `AGENTS.md` codifies repository boundaries, test execution recipes, and strict off-limits files. - **Persistent scratchpad**: Requiring Codex to log completed steps and unexplored hypotheses ensures clean continuity if interrupted.

[15:11](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=911s) Infinite Context with Compaction and Remote Environments

Addressing the widespread concern that maintaining a single conversation over weeks or months inevitably triggers context rot or output degradation, Vibi explains that context compaction completely changes the equation. Compaction condenses prior history while retaining critical operational context, enabling agents to operate across massive horizons. Vibi quips about its efficacy across engineering workflows: “A lot of the times I would joke about saying that compaction is a solved problem, which means that long horizon task is a solved problem.” He confirms he and his team run individual Codex threads continuously for three months at a time without manual resets, and compaction works natively out of the box with zero configuration changes.

Carrying out long-horizon runs on a personal workstation, however, creates an obvious physical bottleneck—engineers cannot realistically keep fleets of laptops awake and connected for six consecutive weeks. Remote environments solve this by letting Codex work over SSH connections across desktop, mobile, and web interfaces. For example, in a multi-day racetrack rendering project, Vibi pointed Codex to a remote development server provisioned with Blender, project assets, and internet access, leaving it to run unattended for days. Expanding on this, Vibi introduces the newly upgraded Codex Cloud, which provisions managed, pre-configured virtual machines without host-management overhead. Codex Cloud environments snapshot state automatically, periodically refresh git branches and dependencies, support custom plugins, and persist seamlessly across devices—a capability Vibi demonstrated by merging launch pull requests from backstage on his mobile device during the event.

- Compaction ships as default Codex behavior, preserving reasoning quality in active threads spanning over three months. - Remote SSH connectivity lets agents access dedicated servers with custom tooling (such as Blender runtimes and web browsing tools). - Codex Cloud delivers snapshotted, fully managed execution environments that stay up to date with remote git repositories automatically. - Unified cross-device state allows engineers to inspect, steer, and merge agent-authored pull requests from mobile, desktop, or browser.

[17:55](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1075s) Plugins, Child Threads, and Worktrees

Giving an agent compute is only half the battle; it also requires ambient organizational context so it does not waste cycles investigating solved problems. Vibi cautions that when a build breaks, teammates may already be coordinating on Slack or tracking the failure in a ticketing system. Without external tool awareness, Codex will inevitably run in circles: “Codex can continue sort of working on that problem, but it would just be mindlessly running around like a headless chicken if it didn't know what is happening.” Connecting plugins for services like GitHub and Linear equips the agent to search existing issues, read discussions, and inspect related pull requests before writing redundant patches.

```mermaid flowchart TD accTitle: Parallel Child Thread and Worktree Architecture accDescr: A parent agent thread delegates subtasks to parallel child threads, each operating in an isolated Git worktree to avoid file conflicts.

A["Parent Coordinator Thread"] -->|"Oversee Overall Goal"| B["Branch Task Execution"]

subgraph ParallelExecution["Parallel Child Threads"] B --> C["Child Thread: Blender Assets"] B --> D["Child Thread: Web UI Viewer"] end

subgraph GitWorktrees["Isolated Git Worktrees"] C --> E["Worktree A (assets branch)"] D --> F["Worktree B (frontend branch)"] end

E --> G["Combined Pull Request / Merge"] F --> G ```

For large, multi-faceted initiatives—such as simultaneously modeling a 3D Ferrari in Blender and constructing the web application to display it—Vibi advises against cramming everything sequentially into one thread. Instead, Codex can decompose broad objectives into child threads managed by an overarching parent thread. To keep parallel subagents from corrupting shared local files, each child thread operates inside an isolated Git worktree—a dedicated disk checkout linked to the same repository. This gives every subagent free rein to test, build, and commit in parallel without stepping on adjacent work. Finally, to steer these expansive multi-month runs without derailing ongoing execution, Vibi introduces `/side`, a command designed to fork the parent thread's full context into an isolated exploratory workspace.

- **Ecosystem plugins**: Integrating GitHub and Linear feeds issue histories, ticket statuses, and team discussions directly into agent reasoning. - **Parent/child delegation**: Parent threads maintain high-level supervision while offloading discrete components to dedicated child threads. - **Git worktree isolation**: Allocating isolated worktrees per subagent prevents concurrent file writes and git index collisions during parallel feature development. - **Steering with `/side`**: Forking a thread’s context into a side branch allows developers to probe specific decisions without disturbing the main execution loop.

[20:46](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1246s) Worktrees, Threads, and Ultra Mode for Hard Problems

VB highlights how essential Git worktrees and branching threads are to daily engineering flow, noting that using them avoids cluttering the primary conversation or corrupting local repository state. Being able to work across multiple projects and simultaneous feature branches without running into merge conflicts saves massive headaches, with VB emphasizing: “And we all hate merge conflicts.”

```mermaid flowchart TD accTitle: Thread Strategy vs Ultra Mode accDescr: Diagram distinguishing parallel execution across threads from deep reasoning in Ultra mode. subgraph ParallelTasks["Embarrassingly Parallel Problems"] A["Parent Thread"] --> B["Child Thread 1"] A --> C["Child Thread 2"] A --> D["Child Thread N"] end subgraph SingleHardTask["Deep / Difficult Bottlenecks"] E["Ultra Mode"] --> F["Investigate Root Cause"] E --> G["Try Potential Fixes"] E --> H["Think Outside the Box"] end ```

When an agent thread gets stuck after exhausting multiple conventional attempts without diagnosing what went wrong, Codex provides an escape hatch called "Ultra mode." Ultra mode gives the underlying model extensive time to deliberate and reason through difficult bottlenecks. Within that single thread, Codex can automatically spin up specialized sub-agents—tasking one with investigating the bug's root cause, directing another to prototype a speculative patch, and letting a third search for unconventional out-of-the-box angles. VB contrasts thread strategies by explaining: “when I have a problem that's embarrassingly parallel, I would have Codex use threads to split that task and work on that. And when I'm dealing with one really hard problem, much like the GPU problem that you mentioned and the floating point precision problem, I would switch to ultra.”

- Worktrees allow engineers to spin up sub-tasks dozens of times a day across disparate repos and features without merge conflicts. - Ultra mode manages sub-agents internally within a single thread context rather than forcing the user to coordinate multiple independent threads manually.

[22:27](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1347s) Heartbeats and Scheduled Tasks

Long-running workflows frequently stall on external dependencies outside an agent's control, such as awaiting human code review on an open pull request. Rather than forcing an engineer to babysit the pull request or repeatedly message colleagues on Slack while they are away on a coffee break, Codex can register automated recurring checks called heartbeats. A heartbeat operates as an automated follow-up loop within the existing thread: every half hour or at an interval of choice, Codex wakes up with the complete accumulated context of prior attempts, inspects the pull request for fresh reviewer comments, addresses actionable feedback directly, and continues its work.

Scheduled tasks also power net-new tasks that run on recurring chron-like triggers. Users can instruct Codex to compile overnight status reports across their active repositories or generate weekly catch-up summaries for legacy projects they have not monitored recently. These autonomous jobs are configured via natural language prompts, parameterized by execution frequency, and managed directly from the "Scheduled" tab in the Codex application interface.

[24:01](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1441s) Token Economics and the Real-World Cost of 24/7 Agents

Addressing the natural skepticism of an audience suspecting that running continuous agents for six weeks is an unaffordable luxury reserved only for OpenAI insiders with infinite token budgets, Frey breaks down the real production costs. Running Codex continuously around the clock across OpenAI’s entire engineering surface area—debugging build pipelines, stabilizing continuous integration, and repeatedly executing millions of test cases per loop—initially cost roughly $32 per hour. With subsequent model releases, that cost dropped sharply: “But with GPT-6 Sol last week, the price dropped to $16 an hour.”

| Execution Tier / Model | Approximate Run Cost (USD / Hour) | Operational Role & Efficiency Impact | |:--- |:--- |:--- | | **Initial Deployment** | ~$32 / hr | Continuous agent running 24/7 for 6 weeks fixing company-wide CI | | **GPT-6 Sol** | ~$16 / hr | Halves operating expenditure; delivers 2x token budget longevity | | **GPT-6.1 Sol** | <$16 / hr equivalent | Heightened token efficiency; highly competitive operational profile |

Frey emphasizes that the token cost represents only one piece of continuous automation, sitting alongside compute infrastructure for CI runners and human code reviews. Nevertheless, paying $16 an hour to automatically stabilize builds and eliminate engineering friction across an entire enterprise was universally considered a bargain by engineering management.

[25:17](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=1517s) Playbook for Hiring Codex Today

Closing out the session, Frey shares a practical framework for how developers can begin delegating significant engineering workflows to Codex immediately:

```mermaid flowchart LR accTitle: Playbook for Deploying Codex Agents accDescr: Flowchart outlining the five steps for onboarding Codex into developer workflows. A["1. Identify Time Sink<br/>(Backlog, PRs, CI, Logs)"] --> B["2. Establish Verification Loop<br/>(E2E Tests on Laptop/Cloud)"] B --> C["3. Define Trade-Offs<br/>(Let Model Delegate)"] C --> D["4. Structure Workspace<br/>(Sidebar Sections & Inbox)"] D --> E["5. Set Ground Rules<br/>(Escalations & Spend Limits)"] ```

1. **Target repetitive operational drag**: Select tasks that consume valuable focus—triaging ticket backlogs, speculatively implementing roadmap features via PRs, inspecting error logs, or fixing failing CI runs. 2. **Provide closed feedback loops**: Equip the agent with clear mechanisms to verify its own work and discern when a task is completed, such as end-to-end test suites running on a local laptop, a dedicated dev box, or Codex Cloud. 3. **Declare high-level objectives**: With GPT-6 generation models, engineers no longer need to micromanage individual steps; they can articulate desired outcomes and acceptable trade-offs, allowing the model to delegate subtasks autonomously. 4. **Organize using sections and inboxes**: Treat the Codex sidebar like an email inbox, using custom sections to group hundreds of active issues and letting background threads flag only items that require human eyes. 5. **Establish hard operational boundaries**: Set explicit escalation triggers for when an agent must alert an engineer or page the on-call responder, and configure strict concurrency caps or budget limits (such as constraining maximum simultaneous CI runs).

Frey leaves the DevDay audience with a closing challenge: “What could Codex take off of your plate? And what could you do with the time that it gives you back?”

## Notable quotes > [07:25](https://www.youtube.com/watch?v=U1IVZy9FhVQ&t=445s) Last year at DevDay, I joked that I'd kind of become a manager of agents, and this year, I've been promoted to a director of agents. I give an agent a job, and then it manages a team of agents to get it done for me. — Friel

## Mentioned OpenAI, DevDay 2026, Codex, Codex Cloud, Astra, Sol, GPT-6 Sol, GPT-6.1 Sol, Friel, Vivi, VB, Ferrari, Nürburgring Nordschleife, Blender, Slack, GitHub, Linear, Git

## Video description Learn from a Codex run that spent over 1,000 hours keeping builds and tests passing. Explore how to choose recurring tasks, improve feedback, manage token budgets, and decide when engineers should step in.

[Source](https://www.youtube.com/watch?v=U1IVZy9FhVQ)

Reading summary

At DevDay 2026, OpenAI engineers Friel and Vivi demonstrated how frontier models have transitioned from heavily micromanaged execution plans to autonomous agents capable of operating continuously over weeks. By orchestrating sub-agents, handling system faults, and operating within dedicated cloud environments, these systems can sustain long-horizon engineering tasks like 3D world generation and continuous integration maintenance. Autonomous Generation of Complex 3D Assets (0:19) To test the limits of agent autonomy, Vivi tasked Codex paired with Astra to build a 3D Ferrari model and an interactive anatomy website using just seven reference photos. Working on a remote machine over several days, the agent assembled the vehicle and site while independently generating unprompted clay and wireframe renders. Expanding the scope, Vivi instructed Codex to create a fly-through animation of the Nürburgring circuit with no input beyond the track's name. Codex autonomously chose to recreate the entire 21-kilometer, 70-turn Nordschleife course rather than a standard layout. Operating for over two days, the agent delegated rendering passes to Blender and coordinated research sub-agents to verify track geometry. When the pipeline suffered crashes caused by RAM exhaustion and CPU overheating, Codex parsed cached render frames and resumed execution automatically without human intervention. Managing Continuous Integration at Scale (5:46) Friel reported deploying an autonomous agent powered by Sol to remediate flaky continuous integration (CI) test failures across OpenAI's codebase. Initiated with a 200-word prompt detailing operational constraints rather than implementation code, the primary controller agent ran continuously for six weeks and accumulated over 1,000 runtime hours before being deliberately decommissioned. To prevent bottlenecks, the controller delegated discrete debugging workflows to a fleet of more than 5,800 sub-agent sessions. Throughout the 1,000-hour deployment, the system generated over 100 pull requests per week, made more than 1,000 commits, issued approximately 500,000 tool calls, and merged 213 autonomous fixes into the mainline. Over 67,000 tool calls were asynchronous wait loops monitoring long CI runs. The system resolved complex technical debt, including a non-deterministic GPU Heisenbug caused by floating-point math anomalies, and isolated an intermittent hardware defect to a single node by correlating weeks of cluster logs. Architectural Patterns for Multi-Week Agent Lifespans (12:12) Vivi outlined several core mechanisms that prevent agent degradation over multi-month deployments. Long-lived workflows rely on dedicated single-outcome threads governed by an AGENTS.md file, which codifies repository rules, testing recipes, and scratchpad logging requirements. Native context compaction condenses conversational history to preserve reasoning across horizons spanning up to three months, while remote execution environments and Codex Cloud maintain stateful, pre-configured runtimes across devices. To manage complex parallel initiatives without stepping on shared local states, parent coordinator threads spawn child threads that operate inside isolated Git worktrees. Developers can use the `/side` command to fork context into exploratory branches, or activate 'Ultra mode' to instruct sub-agents to deeply deliberate over singular, difficult bottlenecks. For intermittent dependencies such as waiting for pull request reviews, automated heartbeats periodically awaken the agent to address feedback, while scheduled tasks execute recurring background jobs. Production Economics and Operational Onboarding (24:01) Friel addressed the financial feasibility of continuous agent operation, noting that 24/7 CI maintenance initially cost approximately $32 per operating hour. Following the release of GPT-6 Sol, operating costs decreased to roughly $16 per hour, which engineering management viewed as cost-effective relative to the engineering hours saved. To onboard agents into production workflows, Friel advised identifying repetitive operational bottlenecks, establishing closed feedback loops via end-to-end verification tests, and declaring outcomes and trade-offs rather than micro-managing steps. Teams must also set hard operational boundaries, including explicit spend caps, concurrency limits, and escalation triggers that require human or on-call engineer notification. The presenters conclude that modern agentic models are capable of durable, multi-week autonomy when provided closed verification loops, boundary guardrails, and hierarchical sub-agent delegation, prompting engineering organizations to reconsider which operational burdens can be handed off permanently.

Source quotes

[7:25] Friel: Last year at DevDay, I joked that I'd kind of become a manager of agents, and this year, I've been promoted to a director of agents. I give an agent a job, and then it manages a team of agents to get it done for me.

Original transcript

[0:19] Hi, I'm Friel, and I work [0:21] on engineering acceleration at [0:22] OpenAI, helping our engineers [0:24] and researchers keep shipping. [0:26] And I'm VIVI. [0:27] I work on the developer experience [0:28] team at OpenAI. [0:30] I spent a lot of my time [0:31] experimenting with the latest models [0:33] and seeing how far I can push them, [0:36] just in general, right? [0:38] Lately, that's meant building [0:39] 3D worlds with Codex and Astra. [0:43] So I am a huge Formula 1 fan, [0:45] and I want to see if Astra could [0:48] help me explore an F1 car in 3D. [0:52] And really, what I [0:54] wanted to do was to just learn [0:55] what these cars are made of. [0:56] What are the components within this? [0:58] And just see what they're composed of. [1:01] So I ran Codex with Astra [1:04] on a remote machine [1:06] and asked it to set itself a goal. [1:08] The goal was very simple. [1:10] Build a Ferrari and a website where I [1:13] could explore all that it's made of. [1:22] So I gave Codex a few reference [1:25] photos of the Ferrari itself and worked [1:28] with it over a few days until [1:30] it got the 3D model right. [1:33] And what's sort of incredible [1:35] is that all of this was [1:36] made by just seven pictures. [1:39] What blew my mind in all [1:41] of this was Astra unprompted [1:44] essentially built not only [1:46] the 3D version and the rendition [1:48] of this, but also a clay version [1:50] and the wireframe version that you [1:52] see on the screen as well. [1:55] And this sort of brought [1:57] me to the next project, [1:59] which I'm just going to show you, [2:01] which was to build a fly-through [2:03] of one of my favorite racetracks, [2:05] which is the Nürburgring. [2:07] Again, I tasked Codex [2:08] with literally just the name [2:10] of the track to go out, [2:13] build a fly-through, and find [2:16] its own way around it. [2:18] Here's what it made. [2:20] What you see is 100% something [2:23] that does not exist in the sense [2:25] that this video does not exist. [2:27] This is all created just by Codex [2:29] by looking at references online [2:31] and so on and so forth. [2:33] So Codex set out [2:35] with this goal to build [2:37] this Grand Prix circuit. [2:38] And when it started, it decided [2:41] not to make the short version, [2:43] but the longer, noted, life-way version. [2:45] That's 21 kilometers. [2:48] This entire video is 21 [2:49] kilometers down [2:52] to all 70 turns [2:54] of the circuit itself. [2:56] And just for context, [2:58] this track is three times [3:01] bigger than your average [3:03] Formula One circuit track. [3:05] So this took a little over two days. [3:08] And most of the time went into [3:10] rendering the frame itself. [3:11] And Codex was just waiting [3:14] for Blender to do its thing. [3:15] Codex used sub-agents to help [3:18] out research the track, [3:19] figure out whether the geometry [3:21] is correct or not, and really [3:23] just planned the render itself. [3:26] What's interesting here is [3:27] that because this is a really [3:29] compute-intensive render, [3:31] Codex stopped a few times. [3:33] This was because of RAM issues, [3:35] CPU overheating, and, you know, [3:37] all the things that could go wrong. [3:38] What was best about this was [3:40] Codex automatically picked up from [3:43] where these saved frames were [3:45] and kept going for these two days. [3:48] Now, I pointed Codex and Astra [3:51] for these F1 questions out [3:53] of my own personal interest. [3:55] But there's so much more you can [3:57] accomplish in your day-to-day. [3:59] And all of this really to say [4:01] that we're clearly at a point where [4:02] agents, when given the right outcome, [4:05] can keep working for a really long time. [4:08] Pray, can you walk us [4:10] through this journey so far? [4:12] Absolutely, VB. [4:13] If you were here last [4:15] year, on this very stage, [4:17] I talked about exec plans. [4:18] A skill for Codex to make [4:20] it run for long periods [4:21] of time on hard problems. [4:24] Back then, [4:25] making Codex run for seven hours [4:26] felt like a big deal. [4:28] I guess today you can just unprompted [4:30] make it run for several days [4:32] to build an F1 circuit. [4:34] What you're seeing here is the prompt [4:35] that I had last year. [4:37] You'll notice that it's actually [4:38] too big to fit on. [4:39] We had to split it up. [4:40] It's really long. [4:42] There's a lot of prescriptive [4:43] guidance in here. [4:44] I'm giving it a code sample. [4:46] I'm telling it what libraries to use [4:49] and how to go about implementing things. [4:51] And then, still, I'm telling it [4:54] to produce a plan that I can [4:55] interrogate and go back and forth [4:57] with before I have it start working. [5:00] I was a lot less comfortable with giving [5:02] the model then a goal and trade-offs [5:04] and having it just start working. [5:08] Today, with Astro and Sol, [5:10] I'm much more comfortable just [5:11] telling it specifically that. [5:12] I tell it the goal and the outcome [5:14] I want, what I care about, what are [5:16] the trade-offs that I'm concerned with. [5:19] And then, I tell it it [5:20] can figure out the steps. [5:22] I can tell it how I know [5:23] when the work will be good [5:25] and let it just start working. [5:27] I don't feel like I need to tell [5:28] it half of the plan at the start. [5:30] I don't feel like I need to go [5:31] back and forth with it. [5:32] And that gave me that got me [5:34] thinking about what kind of jobs [5:35] I can give it today, [5:36] and especially the kind that requires [5:38] constant manual supervision. [5:40] It takes time out of my day. [5:41] I wondered what could our models [5:42] do this year, and how long [5:44] could they work today? [5:46] The answer is over [5:50] 1,000 hours. [5:52] One of my team's jobs is keeping [5:54] continuous integration, or CI, [5:56] healthy across all of OpenAI. [5:58] CI runs automated tests on changes [6:01] that engineers and researchers [6:02] can tell whether their code can [6:03] merge and they can ship. [6:06] Now, a test can fail legitimately [6:08] because it's broken code, [6:10] but often it will fail one [6:11] time and pass the next. [6:13] That's a flaky test. [6:15] Someone then has to look at [6:17] that and determine what the cause is, [6:19] what kind of failure they're looking at, [6:20] and the trickiest of these can [6:22] take hours or days to debug. [6:24] Meanwhile, another one comes in. [6:26] Someone on my team [6:27] has to go look at it, [6:28] or someone else has to debug it. [6:30] And if you've ever worked [6:31] on a team like this, you know [6:33] that this just keeps recurring. [6:34] I wanted to see if an agent could [6:35] pick this up for me without me [6:37] having to ask it every time. [6:39] This agent ran for six weeks [6:41] before I shut it down, and not [6:43] for any reason like it wasn't working. [6:44] It was actually because my colleagues [6:46] had started running agents of their own [6:47] to do the same work. [6:50] The starting prompt for this agent [6:52] was only 200 words. [6:53] What changed from last year? [6:55] Well, I'm still giving it some ground [6:57] rules of how I want it to work. [6:58] Like, here's our repo. [7:00] I want you to... But I'm telling it, like, you [7:02] figure out how to run tests. [7:03] You figure out how to run [7:04] our entire suite. [7:06] And I'm going to tell it then some [7:07] rules about how I want it to message [7:08] my colleagues and how to use [7:10] CI resources effectively. [7:12] There's no code here. [7:13] I'm not prescribing how it needs [7:15] to fix anything. [7:16] And so, this allows it to keep working [7:19] and do this iterative loop of solving [7:22] CI problems all on its own. [7:25] For this run, I started with 5.6 Sol. [7:28] At first, Sol did [7:29] everything in one thread. [7:31] It investigated the failures, [7:32] and it wrote the fixes itself. [7:34] But I did notice that sometimes [7:35] it would get bogged down in one [7:37] particularly difficult problem, [7:38] and then it would work on [7:39] that for a long period of time while [7:41] other flakes and issues were coming in. [7:44] So, about four days in, I [7:46] interrupted it, and I said, actually, [7:47] let your subagents do that busy work. [7:50] This was a paradigm shift [7:51] for how productive it could be. [7:53] The main thread was then taking care [7:55] of triggering the tests and collecting [7:56] the information about what was failing, [7:59] and then organizing subagents [8:00] and threads and messaging people. [8:03] The subagents did that slow [8:04] work of pulling down code, [8:07] working on a particular failure, [8:09] iterating, pushing back up to CI, making [8:11] sure that it passed or sometimes failed, [8:14] and repeating when it needed to. [8:16] Last year at DevDay, I joked [8:17] that I'd kind of become a manager [8:19] of agents, and this year, I've been [8:21] promoted to a director of agents. [8:23] I give an agent a job, [8:25] and then it manages a team [8:26] of agents to get it done for me. [8:30] The outcomes here are pretty incredible. [8:32] Over the six weeks that it [8:34] ran, it opened more than 100 [8:35] pull requests per week. [8:37] Across the run, it pushed [8:39] more than 1,000 commits, [8:41] and it sent more than 600 messages [8:43] to coworkers on Slack to coordinate, [8:45] not just telling them what to do, [8:47] but actually coordinate on [8:48] landing these fixes. [8:50] And in total, it merged more [8:52] than 213 of its own fixes. [8:54] For those that it didn't merge, [8:56] it often found that actually someone [8:57] else had already produced a similar [8:58] change, and it prioritized [9:00] their work ahead of its own. [9:02] The agent recognized [9:03] my role too, and even [9:04] did a good job of managing up. [9:06] I did a retrospective [9:07] with the agent after the run, [9:09] and while it didn't say anything [9:10] directly about my managing style, [9:12] it did say that it was most [9:14] productive for a five-day stretch [9:16] when I didn't message it once. [9:19] Oof. [9:22] Over those 1,000 hours, [9:24] the agent made just under [9:25] half a million tool calls. [9:26] That's roughly how many steps [9:27] that it took, and it used more [9:29] than 5,800 sub-agent sessions. [9:32] Over 67,000 of those calls were [9:34] just waiting for things to happen, [9:36] and if you've ever worked on CI [9:37] systems, you've ever had to debug issues [9:39] like this, you know how this works. [9:40] You pull down the code. [9:41] You push up a change. [9:42] Maybe you go get a coffee [9:43] or a snack, because it's going to take [9:45] a little bit, and then you come back. [9:47] The agent here was tireless. [9:48] It was able to keep working on [9:49] this, and then pause and wait [9:51] for something to happen, and then work [9:53] on the next step when it came in. [9:55] And a couple hundred fixes to CI [9:57] might not sound like a lot, [9:58] but keep in mind, [10:00] these were the hardest bugs [10:01] that were remaining open after many, [10:03] many other humans and their agents [10:05] were looking at these problems, maybe [10:06] encountering them on their changes, [10:08] and deciding, actually, rather [10:10] than fix this, it's easier for me [10:12] to skip or retry that test. [10:15] As an example, one [10:17] of the hardest bugs that we had [10:18] to fix with that this agent fixed [10:20] was a Heisen bug, a non-deterministic [10:23] error in the floating-point calculations [10:25] of one of our GPU tests. [10:28] Now, this is really well [10:29] out of my wheelhouse. [10:30] I really had to spend [10:31] a lot of time reading this thread [10:33] and following up on figuring [10:34] out exactly what happened. [10:36] It was really impressive. [10:38] The agent reproduced this and correlated [10:40] logs over weeks in order to find [10:42] actually there was a particular [10:43] node that this happened on, [10:46] and it then taught itself how to run [10:48] the test repeatedly on that node. [10:50] This actually took several days [10:51] for the subagent to debug, [10:53] so it taught itself how to run [10:55] this test on this node repeatedly, [10:57] and it was able to reproduce [10:58] this, but not 100%, intermittently. [11:01] And then this was [11:02] the most impressive part. [11:04] It actually reached out to researchers [11:05] that owned this system, and it [11:07] started a collaboration with them, [11:08] describing what the problem was [11:10] and how it reproduced the bug, [11:12] and asking them questions. [11:14] You know, is this a software [11:15] optimization that maybe wasn't in scope, [11:17] wasn't declared in the repo, [11:19] but maybe part of that machine, [11:21] or was this a hardware issue? [11:22] Maybe there was actually a faulty GPU. [11:25] It was really impressive that it [11:26] was able to do this, and it [11:27] was able to solve this problem. [11:30] Of course, sometimes this idea [11:31] of getting back to green is just [11:32] to delete the failing test, right? [11:36] And sometimes that's right. [11:38] But we often have reasonable [11:39] skepticism when we [11:41] want to just delete a test, right? [11:42] That's the easy way of getting [11:44] things to pass. [11:45] When the agent noticed that it was [11:47] not sure about whether that was [11:48] the right solution, it often created [11:50] another agent to interrogate and review [11:52] the first agent's code, make sure [11:54] that it was the right solution before [11:55] asking a human to review it. [11:58] And Codex giving me this time [11:59] back allowed me to focus on [12:01] scaling our infrastructure. [12:02] This gave me back hundreds of hours [12:03] of my time over the summer. [12:05] And obviously, Codex can [12:07] work for 1,000 hours a day. [12:08] So, Vibi, can you talk us through [12:09] the features in Codex that enabled this? [12:12] Sure. [12:14] So, first of all, one of the things [12:16] that Frill didn't mention is that his [12:17] agent was called Frillbot, which I [12:19] find very cool and cute as well. [12:22] And, you know, so Frill [12:24] walked us through a bunch [12:25] of things that his agent did. [12:27] And also through, like, [12:29] you know, quite literally hundreds [12:30] and thousands of commits, [12:33] pull requests, and so on. [12:34] But if we were to sort [12:35] of, like, summarize [12:37] all the features that you [12:38] require within Codex to get you [12:41] to that state, it's these six. [12:44] And what we're going to do [12:45] in the next couple of slides [12:46] is essentially just, like, [12:47] talk through those. [12:50] And hopefully, by the end of this, like, [12:51] you would have an understanding of how [12:53] you can set up agents that can [12:54] run for that long as well. [12:57] So, number one, use one [12:59] thread for each outcome [13:01] that you want to accomplish. [13:03] So, this is where, like, within [13:06] this thread, you can iterate, [13:08] build up a plan with Codex, [13:09] and really just, you know, like, [13:11] check in on, like, how Codex is [13:13] doing at that particular job. [13:16] You can even ask Codex to work [13:18] towards it in a certain specific [13:20] direction, or even just, like, [13:22] change directions as it goes as well. [13:24] Some of us are slash [13:26] goal enjoyers over here. [13:27] So, for those of you, you can [13:29] even use slash goal and tell Codex [13:31] to relentlessly go after that task [13:34] and not stop until it is completed, [13:37] no matter what happens. [13:39] Now, let's take Frail's example. [13:42] If I was to set up a thread [13:44] for that, I would say something like, [13:46] hey, keep an eye on the CI, [13:48] reproduce these failing tests, propose [13:51] fixes, and, you know what, open a PR [13:54] for both Frail and me to review. [13:56] This gives Codex an outcome, [13:58] which is to fix and, like, you [14:00] know, keep an eye on the CI. [14:02] It also gives it a way to check [14:04] in on its work, which is the tests. [14:06] And, you know, we've overall [14:08] figured out a way for Codex [14:10] to run those tests, as well as [14:12] make a change, run it again, [14:13] figure out what it needs to do. [14:16] Next up, you can further [14:18] contextualize these long-running [14:20] tasks using agents .md. [14:23] Agents .md would look something [14:25] like this for the CI project. [14:27] You've got some sort of an understanding [14:29] of where these tests are, [14:31] what are some do's for your agents, [14:33] and, hey, what are some things [14:35] which are completely off-limits? [14:37] And this is something which I [14:39] recommend to everyone to, you know, [14:42] continuously sort of maintain as you [14:44] sort of work through your project. [14:47] Now, one of the things, or, like, [14:50] one tip that I always add [14:51] to my Agents .md is to ask [14:53] Codex to keep notes of what [14:55] it has done so far, and also [14:58] for what is still left to explore. [15:00] So in case of scenarios wherein [15:02] Codex cannot continue the work as [15:04] it happens, Codex can still sort [15:07] of, like, read from that work [15:08] log and go back to it. [15:11] Now, at this point, you must be [15:14] wondering, you're using one thread [15:16] for such a long task, wouldn't [15:18] that result in context rod or, you [15:20] know, degraded model outputs? [15:23] And this is where compaction comes in. [15:25] Compaction allows our models to run [15:28] for really long horizon tasks [15:30] without losing track of context. [15:32] A lot of the times I would [15:34] joke about saying that compaction is [15:36] a solved problem, which means that long [15:37] horizon task is a solved problem. [15:39] And, you know, I mean [15:41] this in the most strictest sense, [15:43] that I've got threads which are [15:45] running for three months, and I'm sure, [15:46] like, Frail's got a bunch which [15:48] are running for that long as well. [15:50] So you can quite literally [15:51] have endless conversations [15:53] without worrying about [15:54] the output itself. [15:56] And just to make it clear, [15:58] you do not need to change [15:59] anything in your default Codex [16:01] behavior to have this compaction. [16:03] This is built in. [16:06] Now, running long horizon [16:07] tasks on your laptop can [16:10] be a little inconvenient. [16:11] Imagine Frail having to keep [16:13] a fleet of laptops plugged [16:15] in for six weeks straight. [16:17] This is where remote [16:18] environments come in. [16:19] They allow your threads [16:21] to continue working without [16:23] having to have a laptop [16:26] plugged in 24-7. [16:28] With Codex, you can connect remote [16:30] machines over SSH, across desktop, [16:33] mobile, as well as web. [16:35] And that allows these threads to run [16:38] on this virtual machine with the files, [16:40] the context, the Git repos, [16:42] and the tools that it has access to. [16:43] For the racetrack example, [16:45] I used the Dev server. [16:47] That Dev server had Blender. [16:49] It also had some relevant project files. [16:51] It also had access to web search, [16:53] so it could, like, pull [16:54] in resources as it needed. [16:56] And Codex was good to go [16:57] for quite literally a couple days. [17:00] And starting today, you can [17:02] also use our new and improved [17:05] Codex Cloud experience as well. [17:07] You get the full Codex agent [17:09] with a powerful virtual machine without [17:11] the need to manage the machine yourself. [17:15] The environment is snapshotted, [17:17] which means in case you [17:19] come back to a task after [17:20] a couple days or even weeks, [17:21] you can come back to it. [17:23] It also refreshes the checked-out [17:25] code, dependencies periodically [17:27] to make sure that your tasks [17:28] work the way that they do. [17:30] In fact, fun fact, today for all [17:32] the stuff that we launched, I was [17:34] actually merging those PR from [17:36] the backstages via Codex Cloud. [17:38] So it does work. [17:42] It also works across the three [17:43] surfaces that I mentioned before, [17:45] which is desktop, mobile, and web. [17:47] And the best part is you can bring [17:49] in all of your Codex plugins as [17:51] you use directly in it as well. [17:55] Speaking of plugins, [17:57] now that Codex has somewhere [17:58] to run, it's got either a VM [18:00] or Codex Cloud and so on, [18:03] you do want to make sure [18:04] that Codex has the full picture [18:06] of what it's trying to do. [18:08] Going back to Frail's example, [18:10] let's say there is a test [18:11] that is failing. [18:13] Now, there might already be [18:15] a GitHub issue for it, or there [18:17] might be that the teammates are [18:18] already discussing that on Slack. [18:21] Now, [18:22] Codex can continue sort of working [18:24] on that problem, but it would [18:26] just be mindlessly running around [18:28] like a headless chicken if it [18:30] didn't know what is happening. [18:33] And this is where plugins come in. [18:34] You can connect plugins like GitHub, [18:37] Linear, and really just give Codex [18:39] access to tools that you work with so [18:42] that it can search through issues, [18:44] discussions, pull requests, and so on. [18:46] I am a huge believer that plugins [18:48] is the way to accelerate the most [18:51] that you get out of Codex. [18:55] Now, when you give Codex a big job, [18:57] there might be a few things it [18:59] could work on at the same time. [19:01] Like, running a six-week long task [19:03] in just one single thread can be [19:05] a little messy, especially over days. [19:08] Take the Ferrari project, for example. [19:10] There is the Blender model, but there's [19:12] also the website which needs to be [19:15] built where you can sort of explore [19:16] the model and so on and so forth. [19:18] These two pieces do not need [19:20] to be worked on sequentially. [19:22] So what you could do is you [19:23] could just ask Codex to split [19:25] this task into child threads. [19:27] The main thread can keep an eye [19:30] over the main job. [19:33] And, you know, you can even [19:34] open these individual threads [19:36] and sort of give them direction [19:38] or just ask them to stop [19:39] or just really treat them as [19:42] individual projects in itself as well. [19:44] This is how it would typically look [19:46] like on the Codex app UI as well. [19:50] And if you're using a Git repo, [19:53] each thread or each of these child [19:55] threads can have its own worktree. [19:58] A worktree is a separate [20:00] checkout of the code itself [20:01] so that Codex sub-agents [20:04] and the child threads can make [20:07] changes without sort of messing [20:09] around with one copy of your code. [20:11] This is particularly useful as [20:14] parallel threads are working on [20:16] different features together. [20:17] It allows your Codex agent to continue [20:20] working faster and in a much [20:23] more secure way as well. [20:25] Now, you might be wondering, how do [20:27] you steer long-running tasks, [20:31] especially like those which are running [20:32] over a couple of days and even [20:34] months and so on and so forth? [20:36] This is where slash side comes in. [20:39] Slash side is a special form [20:40] of child thread which forks [20:42] the entire context that your main [20:44] thread has and allows you to ask [20:46] questions without messing around [20:49] with the main thread itself. [20:51] I personally use both of these [20:53] features over a dozen times a day. [20:56] Especially with worktrees, [20:57] I can work on multiple projects [20:59] and multiple features within a project [21:01] without worrying about merge conflicts. [21:03] And we all hate merge conflicts. [21:08] Now, while those threads are working, [21:10] one of them might get stuck. [21:13] Maybe Codex has tried a couple [21:15] of approaches and still hasn't really [21:17] figured out what exactly is going wrong. [21:20] This is where ultra mode comes in. [21:23] And I have to say I personally [21:24] love this visualization a lot. [21:28] Codex can use ultra to give [21:31] the model a lot of time [21:33] to reason through the problem itself. [21:35] It can also spin up sub-agents [21:37] within that thread that could, [21:40] you know, one of those sub-agents [21:41] could go investigate the problem. [21:43] One of them can try a potential fix [21:45] or just, you know, really another one [21:47] could just like think outside the box. [21:49] And ultra gives Codex [21:51] the capability to sort of manage [21:53] that all in a package itself. [21:55] Now, to draw parallels between [21:58] having Codex spin up child threads [22:00] and having Codex work on ultra, [22:03] the main difference really is, [22:04] or rather the way that I see [22:06] it is, when I have a problem [22:08] that's embarrassingly parallel, [22:11] I would have Codex use threads [22:13] to split that task and work on that. [22:15] And when I'm dealing with one [22:17] really hard problem, much like [22:19] the GPU problem that you mentioned [22:21] and the floating point precision [22:22] problem, I would switch to ultra. [22:27] Now, sometimes a job can get [22:29] held up waiting for something [22:31] that is outside its own control. [22:33] This could be, let's say I've [22:34] opened a pull request or my Codex [22:36] has opened a pull request and it's [22:38] waiting for Frail to review. [22:39] Now, I have two options. [22:42] One is that I can continue messaging [22:44] Frail on Slack to come back from [22:46] his coffee break and review my PR. [22:49] Or I can just have Codex, [22:50] you know, set up a scheduled task [22:53] and look at that for me. [22:55] An example of this would be, hey, [22:57] Codex, every half hour, check [23:00] for, you know, some sort of a review [23:01] and see if there's a feedback. [23:04] If there is something that you can [23:05] address, address it. [23:07] If not, then, you know, [23:09] just carry on as usual. [23:11] And this is what [23:12] is called a heartbeat. [23:14] A heartbeat is a scheduled follow-up [23:16] within the same thread where Codex can [23:19] come back with the context of all [23:20] the work it's done so far [23:22] and all that it's done so far, [23:25] what's changed, and really whatever [23:27] needs to be done sort of next. [23:30] You can also use scheduled tasks [23:32] to set up a new task each time. [23:34] A good example of this is give [23:37] me a summary of all the projects [23:39] that I've done or, like, [23:41] everything that's happened, you [23:42] know, in my project overnight. [23:45] Or really just, you know, [23:46] give a weekly summary of different [23:48] projects which I used to work on [23:49] and don't work on now. [23:51] You can just try a prompt, set [23:53] it as how often you want it [23:55] to run, and you will find it [23:57] in the scheduled tab on the Codex app. [23:59] Now, [24:01] Frey, you let all of this, all [24:03] of these features run for six weeks. [24:05] And I can imagine literally everyone [24:07] in this room thinking right now [24:09] that, hey, you work at OpenAI, [24:11] you have infinite tokens. [24:14] And the rest of us [24:15] surely can't afford that. [24:16] So let's demystify that. [24:19] Walk me through how much [24:21] did it actually cost. [24:22] It's a great question, VB. [24:23] And, of course, I and my manager [24:25] were asking the same question. [24:27] And it turned out we did the we [24:29] worked back from the pricing and it [24:31] worked out to about $32 an hour [24:32] to have Codex running for six weeks [24:34] fixing CI across all of OpenAI. [24:38] But with GPT-6 Sol last week, [24:41] the price dropped to $16 an hour. [24:47] GPT-6 stretches your token [24:49] budget twice as far. [24:51] And with 6.1 Sol, it's [24:54] even more token efficient [24:55] and competitive with Astra. [24:57] And this was for an agent [24:58] that was fixing bugs across systems [25:00] all across OpenAI, getting CI Green [25:03] repeatedly running millions of tests [25:05] each time it ran its loop. [25:06] It was well worth it to reduce [25:08] friction for all of us. [25:09] Of course, there's a lot more than [25:11] just an agent that goes into fixing CI. [25:13] There's code review. [25:14] There's the CI resources themselves. [25:16] There's other factors. [25:17] But I do want to talk to you [25:18] about how you can get started [25:19] hiring Codex today. [25:22] First, find something that's taking [25:24] up time and keeping you from doing [25:26] the work that matters most to you. [25:28] This could be going [25:29] through your backlog, [25:31] making PRs maybe speculatively [25:33] to implement features [25:34] that are on the roadmap. [25:35] It could be watching errors [25:37] and exceptions and logs [25:38] and making fixes for those. [25:39] Or it could be fixing CI [25:41] like I did. [25:43] You need to give the model [25:44] a way that it can check [25:45] its work and run in a loop. [25:48] Do you have end-to-end tests that it [25:49] can run either in Codex cloud or on [25:52] a dev box or on your laptop? [25:55] And does it have some way to know [25:56] when its work is done? [25:58] With the GPT-6 models, I find that I [25:59] can just tell them the trade-offs [26:01] that I care about and the outcome [26:02] that I'm looking for. [26:04] And I let it delegate. [26:06] I use Ultra for the hardest problems, [26:08] but I also use Codex as an organizer. [26:10] Like with your Formula One example, [26:12] I use sections in the sidebar [26:14] of the app, and I have it organize [26:15] huge sections for me with hundreds [26:17] of issues for me to look at, [26:19] and I manage this as an inbox. [26:20] I let it create these threads [26:22] and let me know what needs my attention. [26:25] And I set my ground rules. [26:27] I tell it when it should escalate [26:28] to me and needs my attention, [26:31] or maybe even to ping an on-call. [26:34] And then finally, I tell it [26:35] how much I'm willing to spend, [26:37] or in my case, how many CI [26:38] jobs can it run at one time? [26:40] So think back to the beginning [26:42] and the thing you had [26:44] in mind at the start. [26:45] What could Codex take off of your plate? [26:48] And what could you do with the time [26:50] that it gives you back? [26:52] Thanks, everyone, and have [26:53] a great DevDay.