OpenAI Upgrades Agents SDK With Codex Harness and Cloud Sandboxes

Build Hour: Agents SDK — OpenAI

OpenAI

The updated Agents SDK decouples control logic from compute environments to allow autonomous AI agents to execute multi-day tasks reliably without losing state.

AI-assisted summary of Build Hour: Agents SDK — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by OpenAI on May 28, 2026, this Build Hour session focuses on building long-running agents with a model-native harness using the updated Agents SDK. Presented by Steve Coffey and Nish Singaraju, the video details how to provide agents with tools, memory, and sandboxed execution environments to work reliably across complex files and systems.

## TL;DR Steve Coffey introduces major updates to the OpenAI Agents SDK, featuring a Codex-inspired control harness that decouples execution logic from ephemeral compute sandboxes. The presentation details key primitives—such as git-backed skills, bash streams, state rehydration via snapshot tarballs, and dynamic tool approvals—before demonstrating a live conference-planning task tracker built with Docker and Modal/Cloudflare R2 integrations. The session concludes with a detailed Q&A with Nish Singaraju addressing persistent state management, sandbox lifecycle strategies, and multi-agent coordination.

## Atmosphere & style The session carries an technical, collaborative, and developer-focused energy typical of live engineering demonstrations. Steve Coffey leads with a conversational yet precise delivery, combining light humor about 'vibe coding' with grounded technical insights into container infrastructure. The dynamic transitions smoothly into an engaging Q&A alongside Nish Singaraju, offering an authentic behind-the-scenes look at OpenAI's internal tools and developer workflows.

## Key takeaways - [01:48](https://www.youtube.com/watch?v=tK32trvj_b4&t=108s) Internal OpenAI applications like Codex demonstrate that modern agents can run continuously for days or up to a week using goal-driven execution. - [01:48](https://www.youtube.com/watch?v=tK32trvj_b4&t=108s) Building production agents presents major friction points, including balancing provider flexibility, managing state rehydration in sandboxes, and integrating custom skills. - [01:48](https://www.youtube.com/watch?v=tK32trvj_b4&t=108s) The updated Agents SDK introduces a Codex-style harness featuring first-class support for isolated compute sandboxes like Docker, Modal, and E2B. - [05:54](https://www.youtube.com/watch?v=tK32trvj_b4&t=354s) Decoupling the harness (control loop) from compute (sandbox execution) keeps environments ephemeral, isolates secrets, and allows seamless state rehydration. - [05:54](https://www.youtube.com/watch?v=tK32trvj_b4&t=354s) The SDK provides built-in capabilities such as auto-context compaction, asynchronous shell interaction loops, and git-backed skills. - [11:21](https://www.youtube.com/watch?v=tK32trvj_b4&t=681s) Recent OpenAI API releases include a Hosted Shell Tool, fine-grained container network controls, a Skills API, and complete TypeScript SDK support. - [15:36](https://www.youtube.com/watch?v=tK32trvj_b4&t=936s) The new `SandboxAgent` class provides containerized isolation and automatically streams user files into the sandbox working directory. - [20:45](https://www.youtube.com/watch?v=tK32trvj_b4&t=1245s) State persistence across execution turns is achieved by pairing compressed tarball file system snapshots with serializable conversation rollouts. - [23:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1388s) The SDK enables cloud-native execution with providers like Modal and Cloudflare R2, enabling rapid container spin-up and remote snapshot storage. - [27:22](https://www.youtube.com/watch?v=tK32trvj_b4&t=1642s) Developers can extend agent functionality using the `@function_tool` decorator, which includes options for custom execution guardrails and timeout rules. - [30:37](https://www.youtube.com/watch?v=tK32trvj_b4&t=1837s) Dynamic approval functions allow condition-based human-in-the-loop oversight before high-impact tool calls execute. - [33:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1988s) Remote object storage buckets (S3/R2) can be mounted directly into sandboxes to query live network datasets without heavy local file downloads. - [37:42](https://www.youtube.com/watch?v=tK32trvj_b4&t=2262s) While the Responses API fits lightweight single-turn tasks, complex multi-step workflows benefit from the SDK's model-native harness. - [43:12](https://www.youtube.com/watch?v=tK32trvj_b4&t=2592s) Sandboxes can be managed automatically by the SDK on a per-turn basis or pre-provisioned as warm, user-owned persistent environments. - [46:27](https://www.youtube.com/watch?v=tK32trvj_b4&t=2787s) Specialized agents can collaborate asynchronously through shared file systems, databases, or direct handoffs, establishing a foundation for parallel agent workflows.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=tK32trvj_b4&t=0s) Introduction

Christine from the startup marketing team opens the session by welcoming viewers back to OpenAI Build Hours. She introduces Steve Coffey, a Member of Technical Staff on the API team at OpenAI, reminding the audience of his previous presentation on the Responses API. Christine explains that Build Hours aim to empower developers with best practices, tools, and AI expertise to build using OpenAI's models and APIs. She notes that past sessions, as well as the on-demand recording of this stream, are accessible via the homepage linked on screen.

Christine outlines the agenda for the session, which includes: * Major updates to the Agents SDK, including a new Codex-style harness. * Recent API feature releases. * A lengthy live demonstration focused on building an agentic task tracker. * A live Q&A segment supported by OpenAI team members and a special guest answering questions in the chat and on stream.

---

[01:48](https://www.youtube.com/watch?v=tK32trvj_b4&t=108s) Agents SDK Updates

Steve Coffey takes over to discuss recent evolutions in the Agents SDK over the past few months. He highlights how AI models are continuously improving at executing work over long, autonomous trajectories. Drawing parallels to internal OpenAI applications, Steve points to Codex, their agentic coding tool. While typical user tasks run anywhere from minutes to an hour, internal teams have leveraged features like goals to run Codex continuously for days or up to a week. He also highlights a specialized security agent built on Codex that scans repositories for vulnerabilities in legacy code and dependencies, as well as an internal data agent connected to data lakes that converts plain prompts into SQL queries in minutes.

Despite these advancements, Steve notes that building production-grade agents remains challenging due to several friction points: * Balancing performance with flexibility and customization across different model providers while avoiding off-distribution model behavior. * Handling computer-using and coding agents running inside isolated sandboxes, especially managing state rehydration when containers expire or crash. * Bringing custom data, tools, and skills into a adaptable framework without bloating the core application logic.

Steve explains how the Agents SDK addresses these problems. Originating as an open-source, flexible framework released in March, the SDK now includes first-class support for sandbox environments. Developers can run sandboxes wherever their infrastructure lives, with built-in integrations for platforms like E2B, Modal, Cloudflare, Vercel, Daytona, Docker, or local machines. Atop this compute layer sits a Codex-style harness providing out-of-the-box features like skill usage, computer control, and agent memory.

- **Codex Capabilities:** Capable of running multi-step tasks continuously for minutes, hours, or even up to a week. - **Internal OpenAI Agents:** Uses a Codex-based security agent for code repo auditing and a data agent to query data lakes without writing manual SQL. - **Supported Sandbox Environments:** E2B, Modal, Cloudflare, Vercel, Blaxel, Daytona, Docker, and local laptop setups.

---

[05:54](https://www.youtube.com/watch?v=tK32trvj_b4&t=354s) Harness vs. Compute and Key Features

Steve elaborates on the philosophy of the Agents SDK, explaining that it abstracts away the complex orchestration loop. In older paradigms, developers manually managed loops—sending prompts to an LLM API, executing returned tool calls, updating context, and repeating until completion. The Agents SDK provides a model-native harness that handles this agent loop automatically, including web search, file search, Model Context Protocol (MCP) integrations, Code Interpreter, and skills.

Crucially, the SDK splits the **harness** (the control loop) from the **compute** (the execution environment/sandbox). When harness and compute are coupled within the same container, sandboxes become fragile and load-bearing. If a container dies, all session state is lost, and placing secrets inside the sandbox risks prompt injection or exfiltration. By decoupling them, the sandbox can remain completely ephemeral. The harness—running in robust infrastructure such as AWS or a Temporal workflow—handles snapshotting, state rehydration, and secret isolation smoothly.

Steve summarizes core features brought into the Agents SDK: * **Codex-Inspired Loop:** Features auto-context compaction and an async shell interaction loop where the agent can run bash commands, execute background tasks, check status, and manage multi-step terminal operations. * **Sandbox & File Working Memory:** Provides isolated environments where agents can create, edit, and read files across complex projects (e.g., large codebases, PDFs, slide decks). * **Open Source & Model-Agnostic:** Compatible with any model adhering to the Responses API format, supporting multi-tenant architectures, simplified session management, custom MCPs, and tools. * **Multi-Modal & Memory:** Native support for text, voice, skills, containers, and agent memory that allows tasks to improve iteratively.

---

[11:21](https://www.youtube.com/watch?v=tK32trvj_b4&t=681s) Recent API Releases

Steve shifts gears to outline new releases directly available in the OpenAI API: * **Hosted Shell Tool (Responses API):** A lightweight alternative to full SDK setups. Developers pass files into an API call, and OpenAI automatically spins up an ephemeral container where the model executes code and manipulates files before returning the result. Containers can be managed in `auto` mode or manually created and attached via the `/v1/containers` endpoint. * **Container Network Controls:** Fine-grained security parameters allowing developers to set up domain allowlists or completely disable inbound and outbound network access for sandboxed containers. * **Skills API:** Solves the challenge of maintaining a central source of truth for agent skills. A skill is a structured directory/bundle containing a `SKILL.md` file along with supporting scripts and resources (for instance, a tax preparation skill containing IRS rules and 1040 form generation scripts). Developers can upload `.zip` archives of skills, create version histories, set defaults, and seamlessly reference them within the Hosted Shell or Agents SDK. * **TypeScript SDK Support:** Following the April release of the Python Agents SDK, full support for sandbox agents and the updated harness is now available in TypeScript.

---

[14:13](https://www.youtube.com/watch?v=tK32trvj_b4&t=853s) Demo: Building an Agentic Task Tracker

Steve transitions to the screen-share portion of the session to demonstrate building a task tracker powered by the Agents SDK. He presents a custom-built, Linear-style task board interface that he jokes he "vibe coded," explaining that the goal of the demo is to make the board fully agentic and offload manual work to the model.

[14:40](https://www.youtube.com/watch?v=tK32trvj_b4&t=880s) Demo Overview: The Conference Planning Task Tracker

Steve Coffey sets up the practical demo for the new OpenAI Agents SDK: an automated task tracker for conference planning. In a typical event organizing scenario, tasks involve creating program assets, refining speaker presentations, and uploading content to a web portal. Instead of doing the manual labor himself, Steve's goal is to automate the entire process by delegating tasks to autonomous agents.

``` +-------------------------------------------------------------------+ | TASK TRACKER WORKFLOW | | | | [ User Creates Task ] ---> [ Upload Files ] ---> [ Assign Agent ] | | | | | v | | [ Resume Snapshot ] <--- [ Return Output ] <--- [ Docker Sandbox ] | +-------------------------------------------------------------------+ ```

He demonstrates creating a task titled `"create program assets"` and attaching mock document files directly inside the web UI. In a pre-agentic setup, this task would be manually assigned to a human teammate or self-assigned. Instead, Steve prepares to hand off execution directly to an agent configured with the SDK.

- **Use case:** Conference planning workflow (asset generation, talk editing, web publishing). - **Core target:** Automating task execution via SDK-driven agents without manual human intervention.

[15:36](https://www.youtube.com/watch?v=tK32trvj_b4&t=936s) Running the First Agent in a Docker Sandbox

Steve introduces the `SandboxAgent` class—a brand-new primitive in the updated Agents SDK that inherits directly from the existing `Agent` class. Because it is a subclass, developers retain familiar parameters and instructions while gaining native sandbox isolation capabilities.

When Steve assigns the task to the **Program Editor Agent** with the prompt *"Please take a look at these files and edit them for clarity,"* the agent loop triggers locally inside a Docker container. The Agents SDK seamlessly handles uploading the user's attached task files straight into the sandbox directory.

``` +--------------------------------------+ | Host Environment | | | | +------------------------------+ | | | Docker Sandbox Agent | | | | | | | | - Isolated Workspace | | | | - Exec Command | | | | - Async Bash Stream | | | | - Python 3.12 Runtime | | | +------------------------------+ | +--------------------------------------+ ```

Inside the running Docker container, the agent inspects the file tree, executes shell commands to examine content, and begins processing. After running a sequence of terminal commands, the agent produces an execution log confirming what actions were taken and highlighting any missing data.

- **`SandboxAgent` class:** Subclass of `Agent` that provides out-of-the-box containerized isolation. - **File hydration:** Automatic streaming of user attachments directly into the agent’s working directory. - **Default local backend:** Docker running the official `python:3.12` base image.

[18:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1088s) Structuring Agent Capabilities and Git-Backed Skills

To refine how the agent behaves, Steve introduces the concept of **Skills** and **Capabilities**. Rather than hardcoding huge prompts, domain knowledge can be stored in structured skill repositories. Steve points out that GitHub is an ideal place to host skills because teams can leverage version control, pull requests, and code reviews to manage prompt and tool evolutions.

He explains the SDK's new `Capability` primitive—an object designed to package tools, system instructions, and computer startup manifests into reusable modules. The SDK includes a set of built-in default capabilities:

| Capability | Purpose | Key Tools / Functions | |:--- |:--- |:--- | | **File System** | Inspect media and modify code in place | Image viewing, `apply_patch` tool | | **Shell** | Asynchronous terminal access | Combined `exec_command` and `write_stdin` flows | | **Compaction** | Context window optimization | Truncates and summarizes long histories for indefinite runs | | **Skills** | Extensible domain knowledge | Git-backed skill loading (defaults to GitHub main branch) |

By adding the `Skills` capability configured to pull from a main branch on GitHub, Steve prompts the agent: *"Tell me the most interesting thing from your skill file."* The agent spins up, greps through its git-pulled skill repository inside the sandbox, and returns the requested insight.

> *"Compaction allows the model to compact its context and keep going even when it exceeds the context window... this allows the model to keep working for a super long period of time—hours, weeks, technically there's no limit to how long it can work."* — Steve Coffey

[20:45](https://www.youtube.com/watch?v=tK32trvj_b4&t=1245s) How Pausing and Resuming Works Across Container Sessions

Steve breaks down how the Agents SDK handles state persistence between execution turns without losing work. When a task completes or pauses, the SDK automatically stops the active sandbox container, takes a complete tarball snapshot of the file system, and saves it to a designated storage location.

``` Active Sandbox (Docker / Cloud) │ ▼ (Task ends / Pauses) Tarball Snapshot Created │ ▼ Stored in Local Disk or R2/S3 │ ▼ (New Prompt / Resume) Rehydrated onto New Container ──► Model resumes seamlessly ```

When a user posts a new prompt or resumes a paused session, the SDK grabs the saved tarball, provisions a fresh container if the previous one was terminated, and rehydrates the exact file system state onto the new instance. The underlying LLM remains completely oblivious to the fact that it is operating on a new container, enabling continuous execution across cold starts.

- **Snapshot format:** Compressed `.tar` archives storing the workspace state. - **Local default:** Saves tarballs directly to the developer's local file system. - **Integrations:** Direct plug-and-play support for cloud storage providers like Cloudflare R2, AWS S3, Azure Blob Storage, and Modal Volumes.

[23:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1388s) Offloading Sandboxes to Cloud Providers (Modal & R2)

While Docker is ideal for local development and testing, Steve notes that deploying local Docker containers to production is cumbersome. The Agents SDK provides first-class support for cloud-native sandbox platforms, including **Modal**, **Cloudflare Workers/Sandbox**, **E2B**, **Vercel**, **Daytona**, and **Blaxel**.

``` +-------------------------------------------------------------------+ | CLOUD EXECUTION ENVIRONMENT | | | | +-----------------------+ +-----------------------+ | | | Modal Sandbox | | Cloudflare R2 | | | | (Remote Execution) | <=======> | (Remote Snapshots) | | | +-----------------------+ +-----------------------+ | +-------------------------------------------------------------------+ ```

Steve updates the configuration to use `ModalSandboxClient` paired with Cloudflare R2 for snapshot storage. When he triggers a new task, Modal provisions a remote container in under two seconds. The SDK automatically uploads the workspace manifest and synchronization files to the remote server, executes the instructions, and writes snapshot state back to R2.

- **Supported sandbox providers:** Modal, E2B, Cloudflare, Vercel, Daytona, Blaxel, or custom implementations via `BaseSandboxClient`. - **Manifest system:** Declares desired directory structures and remote resource bindings (S3/R2/GitHub) before container startup. - **Spin-up time:** Cold starts on platforms like Modal execute in ~1-2 seconds.

[27:22](https://www.youtube.com/watch?v=tK32trvj_b4&t=1642s) Exposing Custom Function Tools to the Agent

To make the agent genuinely useful within an application, developers can supply custom python or TypeScript functions using the `@function_tool` decorator. Steve walks through three custom tools added to the conference program editor:

1. `update_task_status`: Changes a task's state in the tracking database. 2. `update_task_assignee`: Reassigns the task to another team member or agent. 3. `search_assignees`: Queries the system directory for valid users.

When the LLM decides to trigger a tool, the Agents SDK intercepts the function call request, matches it against decorated local or remote functions, executes the code, and routes the returned output back to the model automatically.

Steve demonstrates this by issuing the command: *"Please assign this task to Steve and mark it as ready for review."* The agent invokes `search_assignees`, locates Steve, executes `update_task_assignee`, and updates the status tool in real time, reflecting the changes immediately in the web frontend.

- **Decorator syntax:** Uses `@function_tool` to convert standard native functions into model-readable schemas. - **Advanced features:** Tool overriding names, custom instructions, dynamic enable/disable flags based on feature gates, timeout configurations, and argument guardrail hooks.

<FollowUp label="Want to cover the next section of the transcript on Human-in-the-loop approvals and multi-agent coordination?" query="Continue with the next transcript section covering tool call approvals, human-in-the-loop controls, and multi-agent workflows."/>

[30:37](https://www.youtube.com/watch?v=tK32trvj_b4&t=1837s) Dynamic Tool Approval and Multi-Agent Handoffs

Steve Coffey demonstrates how to configure human-in-the-loop approvals for agent tools. Instead of passing a static boolean, developers can supply a dynamic function that evaluates parameter values before granting execution rights. In the demo example, any request to set a task status to "done" triggers a mandatory approval pop-over, preventing the agent from finalizing work without manual oversight. When a user requests to mark a task complete, the platform rehydrates the state inside a freshly spun-up sandbox, evaluates the tool call, and surfaces a pending approval request in the UI activity log.

``` +-------------------------------------------------------------------+ | Human-in-the-Loop Approval | | | | [ Agent Task ] ---> ( Modifies Status) | | | | | v | | { Dynamic Function Check } | | | | | Status == "done"? | | / \ | | (Yes)/ \(No) | | v v | | [ Pause & Request ] [ Auto-Execute ] | | [ Human Approval ] | +-------------------------------------------------------------------+ ```

While waiting for the approval workflow to execute, Steve highlights the SDK's built-in handoff mechanisms that allow multiple specialized agents to collaborate on a single task. In his system, a **Program Editor Agent** and an **Asset Producer Agent** work in tandem. By creating an issue requesting content refinement and asset generation, the user assigns the initial task to the Program Editor Agent. Once its work finishes, it hands off execution to the Asset Producer Agent, which generates the required media files and places them back into the sandbox. Meanwhile, approving the pending status change resumes the primary task loop, successfully marking the issue as done.

- **Dynamic Approvals:** `approval: (params) => params.status === 'done'` allows condition-based human oversight without blocking safer read-only tool calls. - **Agent Handoff Pattern:** Specialized agents pass context and execution state back and forth through unified task assignment and sandbox management.

---

[33:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1988s) Mounting Remote Data Volumes with S3 and R2

Steve transitions to explaining how the Agents SDK natively handles large datasets and high-freshness data by mounting external storage buckets directly into sandbox manifests. Copying hundreds of large files or PDFs into a sandbox on every spin-up creates heavy bandwidth overhead and risks using stale data. By mounting a centralized source of truth—such as Cloudflare R2 or Amazon S3—agents operate over live network data seamlessly without requiring local copies.

In the code manifest, Steve replaces a standard empty input directory with an `S3` mount configuration using an `R2AttachmentStore`. Because Cloudflare R2 provides S3-compatible APIs, a single object type handles both providers. Developers supply the bucket name, access key, secret key, and a provider strategy (e.g., Modal cloud bucket strategy). For generic container setups or Docker deployments, the SDK includes out-of-the-box support for `rclone` and `FUSE` file system mounts.

- **Storage Mount Types:** S3-compatible object storage (R2/S3) mounted directly to local path structures inside the manifest. - **Provider Integrations:** Built-in modal cloud bucket strategies alongside generic `FUSE` and `rclone` implementations. - **Abstraction layer:** The remote bucket mount is completely opaque to the model; agents issue standard file system commands (`ls`, `cat`, `grep`) over the network. - **Trade-off:** Minimal network latency on heavy file listings in exchange for near-instant sandbox boot times and strict data freshness.

---

[37:42](https://www.youtube.com/watch?v=tK32trvj_b4&t=2262s) Q&A: SDK Harness vs. Responses API

Steve is joined by Nish Singaraju (Product Manager for Agents at OpenAI) to answer community questions. The discussion opens with guidance on when developers should build using a harness-based Agents SDK versus using the low-level Responses API directly. Steve explains that while the Responses API excels at lightweight, single-turn tasks—such as translation or converting unstructured text to JSON—the Agents SDK provides the robust infrastructure needed for complex multi-step workflows.

Steve notes that model behavior is increasingly tightly coupled with harness design (such as the Codex harness). Utilizing the official SDK ensures developers receive optimal model performance, built-in tool loops, and native file-editing capabilities. However, for esoteric programming languages lacking an official SDK framework (e.g., Elixir or Haskell), building directly on the Responses API remains a strong choice.

> "The Codex harness with OpenAI models is sort of becoming more and more coupled as time moves on... it's very possible that you will be able to get the best performance from the harnesses that the models are trained with."

---

[39:54](https://www.youtube.com/watch?v=tK32trvj_b4&t=2394s) Q&A: Persistent State Management and Rehydration

Addressing state persistence, Steve and Nish outline how the Agents SDK handles long-running pause-and-resume behaviors natively out of the box. State is bifurcated into two synchronized components: 1. **File System State:** Captured via full snapshotting that zips up container file system changes and persists them to local disk or remote storage (like Cloudflare R2). 2. **Conversation Rollout:** Captured as a serializable JSON object containing all messages, tool calls, and run parameters.

Nish emphasizes that combining file system snapshots with conversation rollouts sets the Agents SDK apart from other frameworks. Because the entire run state can be saved as a simple database record, multi-node applications can pause an agent execution and losslessly resume it hours or days later on a completely different compute node.

- **Snapshot Mechanics:** Automatic archiving of modified workspace files upon turn completion. - **Rollout Payload:** Standardized JSON object representing complete agent history for easy database storage and cross-node hydration.

---

[41:24](https://www.youtube.com/watch?v=tK32trvj_b4&t=2484s) Q&A: Agents SDK vs. Codex Binary Capabilities

Nish compares the execution harness inside the Agents SDK (via the Sandbox Agent) with the native Codex binary execution environment. While the Codex binary is a multi-threaded system built for local execution, the Agents SDK ports core Codex capabilities—such as the Bash tool, file editing paradigms, skills, and memory systems—into a highly distribution-friendly framework.

While multi-agent orchestration in the Codex binary relies on native threading, the Agents SDK achieves equivalent modularity using structured **agent handoffs**. Nish notes that features exclusive to the Codex binary are actively being ported to the Agents SDK to align both developer environments.

---

[43:12](https://www.youtube.com/watch?v=tK32trvj_b4&t=2592s) Q&A: Sandbox Lifecycle and File Population Strategies

The panel addresses container lifecycle patterns and strategies for populating sandbox file systems. Steve clarifies that the Agents SDK supports two distinct operational modes for sandboxes:

* **SDK-Owned Sandboxes (Default):** The SDK manages container provisioning and destruction. Developers provide container credentials (e.g., Docker or E2B client), and the SDK automatically boots the environment for the execution turn and tears it down immediately afterward, preventing accidental infrastructure costs. * **User-Owned Sandboxes:** Developers manually provision a persistent sandbox out-of-band and pass its reference to the agent. The SDK keeps the container warm across multiple user interactions, eliminating cold-start latency at the cost of continuous compute expenses.

``` +---------------------------------------------------------------------+ | Sandbox Ownership Models | | | | SDK-Owned (Default): | | [ Start Turn ] -> [ Spin Up ] -> [ Execute ] -> [ Spin Down ] | | | | User-Owned (Warm): | | [ Pre-provision Container ] -> [ Turn 1 ] -> [ Turn 2 ]... | +---------------------------------------------------------------------+ ```

Nish summarizes the file population tradeoffs defined in the SDK manifest. Developers can choose between **copying** files directly into the sandbox (yielding fast local read performance during runtime but slower container startup) or **mounting** files over network storage (yielding instant sandbox initialization but incurring small network latency penalties during file reads).

---

[45:42](https://www.youtube.com/watch?v=tK32trvj_b4&t=2742s) Q&A: Multi-Agent Coordination at Scale

Closing the session, Steve and Nish address whether a single supervisor agent can coordinate hundreds of specialized sub-agents simultaneously. Steve confirms that this architecture is entirely possible today by providing the supervisor agent with management tools to query, check progress on, and communicate with active sub-agents via messaging layers. He hints that OpenAI is actively building deeper multi-agent framework abstractions, which will be released in the coming weeks and months to make high-density agent orchestration even easier.

<ElicitationsGroup message="Explore related Agents SDK topics:"> <Elicitation label="Learn about Agents SDK tool loop mechanics" query="Explain how the Agents SDK manages tool loops, function calling, and execution state."/> <Elicitation label="Compare sandbox providers for OpenAI Agents SDK" query="What are the differences between Docker, E2B, and Modal sandbox providers in the OpenAI Agents SDK?"/> </ElicitationsGroup>

[46:27](https://www.youtube.com/watch?v=tK32trvj_b4&t=2787s) Coordinating Parallel Agents and Envisioning the Future

Addressing the mechanics of multi-agent architectures, the speakers explain how multiple agents can be coordinated using a shared underlying state. Agents can seamlessly communicate through a unified file system, a centralized database, or a variety of other inter-agent communication mechanisms. There are virtually no technical barriers stopping developers from orchestrating these complex configurations today. Looking ahead, Steve Coffey emphasizes that this style of "massively parallel work will become sort of the default in the near future," signaling a major shift in how complex software tasks and autonomous agent loops will be structured.

- Agents can communicate and coordinate work asynchronously using shared mechanisms like a single file system or a central database. - Massively parallel agent architectures are expected to become the industry standard for handling complex multi-step workflows.

[46:45](https://www.youtube.com/watch?v=tK32trvj_b4&t=2805s) Resource Wrap-Up and Next Steps

Nish Singaraju wraps up the session by highlighting the key resources required to get started with the Agents SDK. Addressing the high volume of chat questions regarding Steve's live demo, Nish reassures attendees that all the code from this session and previous Build Hours is hosted in the official build-hours GitHub repository (`[github.com/openai/build-hours](https://github.com/openai/build-hours)`). The repository will be updated immediately after going off camera. Attendees are encouraged to visit the Build Hours homepage (`[webinar.openai.com/buildhours](https://webinar.openai.com/buildhours)`) to register for future sessions.

Following the live broadcast, attendees will receive a follow-up email containing a feedback survey to suggest future topics, as well as a direct link to the full video recording for those who missed the beginning. With a final word of appreciation, the team concludes the stream until the next session.

- **Agents SDK Docs**: `[developers.openai.com/api/docs/guides/agents](https://developers.openai.com/api/docs/guides/agents)` - **TypeScript SDK**: `[github.com/openai/openai-agents-js](https://github.com/openai/openai-agents-js)` - **Python SDK**: `[github.com/openai/openai-agents-python](https://github.com/openai/openai-agents-python)` - **Agents SDK Quickstart**: `openai.github.io/openai-agents-python/quickstart/` - **Code Repository**: `[github.com/openai/build-hours](https://github.com/openai/build-hours)` - **Upcoming Build Hours**: `[webinar.openai.com/buildhours](https://webinar.openai.com/buildhours)`

## Notable quotes > [18:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1088s) Compaction allows the model to compact its context and keep going even when it exceeds the context window... this allows the model to keep working for a super long period of time—hours, weeks, technically there's no limit to how long it can work. — Steve Coffey

> [37:42](https://www.youtube.com/watch?v=tK32trvj_b4&t=2262s) The Codex harness with OpenAI models is sort of becoming more and more coupled as time moves on... it's very possible that you will be able to get the best performance from the harnesses that the models are trained with. — Steve Coffey

> [46:47](https://www.youtube.com/watch?v=tK32trvj_b4&t=2807s) massively parallel work will become sort of the default in the near future — Steve Coffey

## Mentioned Build Hour: Agents SDK, OpenAI, Steve Coffey, Nish Singaraju, Christine, Codex, Agents SDK, Responses API, Hosted Shell Tool, Skills API, Model Context Protocol, Docker, Modal, Cloudflare, Cloudflare Workers, Cloudflare R2, E2B, Vercel, Daytona, Blaxel, AWS S3, Azure Blob Storage, Temporal, Linear, GitHub, Python, TypeScript, Elixir, Haskell, IRS, Form 1040

[Source](https://www.youtube.com/watch?v=tK32trvj_b4)

中文简报

OpenAI 升级 Agents SDK,引入 Codex Harness 与云端沙盒 更新后的 Agents SDK 将控制逻辑与计算环境解耦,使自主 AI 智能体能够可靠地执行多日任务且不丢失状态。 Steve Coffey 介绍了 OpenAI Agents SDK 的重大更新,其核心是引入受 Codex 启发的控制框架(harness),将执行逻辑与临时计算沙盒解耦。演讲详细阐述了多项关键原语,包括基于 Git 的技能、Bash 流式传输、通过快照 tarball 恢复状态以及动态工具审批,并现场演示了基于 Docker 与 Modal/Cloudflare R2 集成构建的会议筹备任务跟踪器。活动最后,Nish Singaraju 参与了深入的问答环节,探讨了持久化状态管理、沙盒生命周期策略以及多智能体协同等议题。 OpenAI 发布更新版 Agents SDK,引入源自 Codex 的原生控制循环(Harness),原生支持 Docker、Modal 及 E2B 等隔离计算沙箱。 (1:48) SDK 采用控制循环(Harness)与计算执行沙箱解耦架构,结合自动上下文压缩、异步 Shell 交互循环与 Git 支持的 Skills 功能,保障密钥隔离与容器故障后的无缝恢复。 (5:54) OpenAI API 同步推出 Hosted Shell 工具、容器网络细粒度访问控制、Skills API,并全面支持 TypeScript SDK。 (11:21) 新增的 SandboxAgent 原语可自动将文件注入容器运行目录,并通过压缩 Tarball 快照与可序列化对话 Rollout 的结合实现跨会话持久化。 (15:36)

Reading summary

OpenAI's updated Agents SDK introduces a model-native, Codex-style harness that decouples agent orchestration from ephemeral sandbox execution. Presenters Steve Coffey and Nish Singaraju explain that this architecture enables long-running, multi-step workflows with robust state persistence, modular skill loading, and dynamic human oversight across local and cloud environments. Decoupling Orchestration from Ephemeral Compute (5:54) In traditional setups, developers manually manage execution loops between model API calls, tool invocations, and context updates. The Agents SDK abstracts this process through a model-native harness that automatically orchestrates shell interactions, file operations, skills, and Model Context Protocol (MCP) integrations. Crucially, the architecture isolates this control loop from the execution environment. Coffey points out that coupling the orchestration harness inside the execution container makes sandboxes load-bearing and exposes credentials to prompt injection or container crashes. By running the control harness in external infrastructure like AWS or Temporal workflows, the underlying compute sandbox remains entirely disposable. Developers can run sandboxes locally in Docker or deploy them across third-party cloud environments such as Modal, E2B, Cloudflare, Daytona, and Vercel. The SDK manages container interactions through the `SandboxAgent` primitive, exposing async shell commands and file-editing tools without tying session survival to container uptime. State Persistence via Snapshots and Rollouts (20:45) To allow tasks to pause, resume, or survive container termination, the SDK divides session state into two synchronized mechanisms: conversation rollouts and workspace snapshots. Conversation history, run parameters, and tool call traces are captured as serializable JSON objects, enabling workflows to be logged and transferred across multi-node infrastructures. In parallel, changes made to the sandbox file system are packed into compressed tarball archives and saved to local disks or cloud object stores such as Cloudflare R2 and Amazon S3. When a session resumes, the SDK provisions a container and unpacks the tarball snapshot, rehydrating the workspace seamlessly without model awareness of the underlying reboot. Singaraju notes this persistence layer differentiates the SDK from frameworks that lose context during container restarts. Sandboxes operate in two modes: SDK-owned, which spins containers up and down per execution turn to minimize compute expenses, or user-owned, which maintains warm containers across interactions to avoid 1–2 second cold-start initialization delays. Modular Skills, Compaction, and Human Oversight (18:08) Rather than hardcoding large instructions into system prompts, the SDK structures functionality through reusable `Capability` objects. The skills capability allows agents to ingest external domain repositories directly from version-controlled sources like GitHub branches, enabling teams to govern prompts and accompanying scripts through standard code review processes. To maintain multi-day autonomous runs without breaching context boundaries, the SDK includes an automated context compaction capability. Coffey notes this mechanism compacts conversational history, technically removing upper duration limits on continuous agent execution. For operational governance, custom tools exposed via the `@function_tool` decorator support dynamic, programmatic human-in-the-loop controls. Instead of relying on static approval flags, developers can define evaluation functions that inspect tool parameters before granting execution rights. In Coffey’s demonstration, routine updates executed automatically, whereas invoking a status change to "done" triggered a mandatory review pop-over, preventing unauthorized task completion until an operator reviewed the container logs. Cloud Sandboxing and Multi-Agent Orchestration (23:08) For enterprise scaling, the SDK natively mounts remote storage systems rather than downloading extensive asset directories into containers on startup. Developers configure bucket credentials to mount S3 or Cloudflare R2 buckets directly to file paths via FUSE or rclone integrations, or through native platform drivers like Modal cloud bucket strategies. This approach trades slight network latency on file reads for near-instant container startup times and live data freshness across distributed agents. Multi-agent coordination is supported through structured agent handoffs, where specialized sub-agents share sandbox environments and conversation states to perform complementary tasks. Coffey and Singaraju emphasize that architectures where supervisor agents orchestrate hundreds of parallel sub-agents are achievable today via shared file systems or databases. Coffey forecasts that this massively parallel execution model will become the industry standard for autonomous engineering and operational tasks. While the low-level Responses API remains recommended for single-turn calls or programming environments lacking official SDK bindings, OpenAI positions the Agents SDK as its primary framework for complex, multi-turn applications. However, native multi-threaded coordination remains exclusive to the local Codex binary, with unified multi-agent abstractions for high-density supervision still under active development.

精炼阅读摘要

OpenAI 在 Build Hour 活动中展示了 Agents SDK 的最新架构升级,重点将控制编排环(Harness)与代码执行环境(Sandbox)彻底解耦,并引入了类 Codex 的运行模式。主讲人 Steve Coffey 指出,该设计旨在通过沙箱快照与对话回放机制解决容器崩溃导致的状态丢失与安全凭证暴露问题,使智能体能够长期稳定执行复杂的多步自主任务。 控制环与计算环境的解耦架构 (5:54) 在传统开发范式中,开发者通常需要手动编写循环来调用语言模型、执行工具调用并更新上下文,同时若将编排逻辑与执行环境绑在同一容器内,一旦容器崩溃就会丢失所有会话状态,且在沙箱内置入密钥会引发提示词注入或凭证外泄风险。Agents SDK 提供的类 Codex 模型原生 Harness 将控制逻辑与计算沙箱(Compute)分离,让沙箱本身保持完全无状态和短暂性(Ephemeral)。控制环可在 AWS 或 Temporal 工作流等稳健基础设施中运行,负责调度、密钥隔离以及环境状态管理。 该 Harness 自动处理网页搜索、文件检索、Model Context Protocol(MCP)集成、Code Interpreter 与技能系统。为了让智能体突破上下文窗口限制长期运行,SDK 内置了自动上下文压缩(Auto-context compaction)与异步 Shell 交互环,支持执行后台进程与多步骤终端操作。Steve Coffey 表示,该机制在理论上取消了模型持续工作的时长上限,能够支撑数小时乃至长达数周的任务轨迹。 沙箱隔离与快照恢复机制 (20:45) 为了在容器崩溃或跨节点调度时保留工作进展,Agents SDK 引入了双重状态持久化方案:文件系统快照与对话回放(Conversation Rollout)。当任务暂停或单轮执行结束时,SDK 会自动将工作区沙箱的文件变动打包为 `.tar` 压缩包并存入本地磁盘或 Cloudflare R2、AWS S3 等对象存储;同时将完整的交互历史序列化为标准 JSON 数据并写入数据库。当恢复执行时,SDK 启动全新容器并解压快照,模型无需感知底层容器的重置即可无缝衔接。 在底层计算层面,SDK 提供了 `SandboxAgent` 原语,支持本地 Docker(默认采用 `python:3.12` 基础镜像)以及 Modal、Cloudflare、E2B、Vercel、Daytona 和 Blaxel 等云端沙箱。在容器生命周期管理上,开发者既可采用 SDK 托管模式(单轮调用按需拉起并立即销毁,节约资源成本),也可采用用户自有模式(保持热容器以消除冷启动延迟)。针对大型数据集,SDK 支持通过 FUSE 或 rclone 将 S3/R2 存储桶直接挂载为文件系统,避免每次拉起容器全量复制文件的带宽开销。 API 新特性与代码化技能体系 (11:21) 针对不需要完整 SDK 的轻量场景,OpenAI 推出了 Responses API 下的托管式 Shell 工具(Hosted Shell Tool)。开发者可直接在 API 调用中附带文件,OpenAI 会在云端动态创建临时容器供模型执行代码和编辑文件,并支持通过 `/v1/containers` 手动管理容器。API 同时上线了容器网络访问控制,允许开发者配置域名白名单或完全切断容器出入站流量以防数据泄露。此外,Agents SDK 除 Python 外现已正式支持 TypeScript。 针对提示词和领域知识难以集中管理的问题,OpenAI 推出了基于文件结构的 Skills API,支持将提示词规范、说明文档(如 `SKILL.md`)和可执行脚本打包为 `.zip` 进行版本化管理。在 SDK 中,`Capability` 原语可将工具集、系统指令与启动清单模块化组合。Steve Coffey 建议团队将技能库托管在 GitHub 主分支上,借助 Git 的版本控制与拉取请求(PR)流程规范智能体知识演进,使智能体可在沙箱中动态克隆或检索最新的领域技能。 动态人工介入与多智能体协作 (30:37) 在任务自动化执行中,Agents SDK 通过 `@function_tool` 装饰器将原生代码转换为结构化模型工具,并提供了细粒度的条件化人工审批机制。开发者可以编写动态逻辑函数而非依赖静态布尔值;例如,当且仅当智能体请求将任务状态变更为“已完成(done)”时,系统才会触发审批弹窗暂停流转,其余低风险读写操作则自动放行,从而兼顾执行效率与系统安全性。 在多智能体协同方面,SDK 目前通过明确的智能体移交(Handoff)机制实现职责分工,例如会议策划任务中,负责文案审阅的 Program Editor Agent 完工后可将沙箱环境与任务指针移交给 Asset Producer Agent 生成配图。产品经理 Nish Singaraju 指出,Codex 二进制环境依赖本地多线程,而 Agents SDK 采用移交与消息总线机制以适配分布式架构;多智能体共享统一文件系统或中心数据库的设计使得高并发协同成为可能,Steve Coffey 预测大规模并行智能体作业将在不久后成为行业主流范式。 OpenAI 团队总结指出,轻量单轮任务仍推荐直接使用 Responses API,而多步骤、依赖文件与终端口的长程任务则应优先采用 Agents SDK,因其 Harness 正在与模型能力深度绑定。演讲者提示,用于大规模监督智能体调度百级子智能体的更深层多智能体框架目前仍在开发中,计划在未来数周与数月内陆续发布。

Detailed notes — zh

## Background 该视频由 OpenAI 于 2026 年 6 月 3 日发布在 YouTube 官方频道。页面简介指出,本期 Build Hour 重点展示了最新演进的 Agents SDK,指导开发者如何利用模型原生 Harness(脚手架)、隔离执行沙箱、长期记忆与工具链,构建能够跨文件和系统稳定执行长程任务的生产级智能体。 ## TL;DR OpenAI 工程师 Steve Coffey 与产品经理 Nish Singaraju 深入解析了全新 Agents SDK 的核心架构演进,重点阐述了调度 Harness 与计算沙箱彻底解耦的设计理念,以及对 Docker、Modal 等容器环境和 Skills API 的原生支持。分享通过现场实操,完整演示了如何从零构建一个具备文件审查、人机协作审批、多 Agent 接力移交以及通过 Cloudflare R2 进行状态快照与恢复的 Linear 风格任务追踪器。最后在 Q&A 环节中,团队深入对比了 Harness 架构与 Responses API 的选型差异,并就沙箱生命周期控制、文件挂载策略及未来大规模多智能体并发架构进行了专业解答。 ## Atmosphere & style 整场直播洋溢着极客氛围与开放活跃的技术交流感,Steve“氛围编程”(vibe coding)的幽默调侃与现场真实的小插曲让演示显得格外亲切务实。讲解风格由浅入深,从底层架构原理自然过渡到云端沙箱的极速冷启动实操,视觉呈现与代码联动流畅连贯。随着 Nish 加入问答,讨论节奏紧凑而充满干货,展现出技术团队对前沿 Agent 基础设施演进的笃定与热情。 ## Bulletin - [04:40](https://www.youtube.com/watch?v=tK32trvj_b4&t=280s) OpenAI 宣布升级开源的 Agents SDK,采用源自 Codex 的解耦架构,将调度腳手架(Harness)与即用即弃的执行沙箱(Compute)彻底分离,支持长时间自主运行与无缝断点续跑。 - [11:24](https://www.youtube.com/watch?v=tK32trvj_b4&t=684s) Responses API 推出托管 Shell 工具、细粒度容器网络访问控制(包括全隔离模式)及统一分发业务脚本的 Skills API,同时补齐了 Sandbox Agent 的 TypeScript 原生版本。 - [25:36](https://www.youtube.com/watch?v=tK32trvj_b4&t=1536s) Agents SDK 原生接入 Docker、Modal、E2B、Cloudflare R2 等多类沙箱与云存储,通过 Manifest 契约实现文件系统预构建或秒级对象存储网络挂载。 - [45:40](https://www.youtube.com/watch?v=tK32trvj_b4&t=2740s) OpenAI 正在全力研发多智能体框架(Multi-Agent Frameworks),预计将在数周或数月内发布,未来系统默认将趋向于数百个 Agent 共享存储的大规模并行工作模式。 ## Key takeaways - [01:38](https://www.youtube.com/watch?v=tK32trvj_b4&t=98s) 随着大模型长程自主任务执行能力的跃升,生产级环境对容器崩溃、敏感凭据隔离及长时间状态持久化提出了严苛要求。 - [04:40](https://www.youtube.com/watch?v=tK32trvj_b4&t=280s) Agents SDK 采用 Harness 与 Compute 解耦设计,由稳健的基础设施负责状态调度与密钥管理,沙箱则作为完全临时、即用即弃的隔离计算环境。 - [08:50](https://www.youtube.com/watch?v=tK32trvj_b4&t=530s) SDK 深度继承了 Codex 的异步 Shell 交互循环,智能体在下发耗时命令后无需阻塞等待,可轮询状态并持续执行多任务。 - [11:24](https://www.youtube.com/watch?v=tK32trvj_b4&t=684s) Responses API 增加了轻量级托管 Shell 工具与域名白名单控制,并提供基于 ZIP 包版本化分发的 Skills API。 - [14:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=848s) SandboxAgent 类扩展了标准 Agent,能够将业务任务及相关附件自动化同步挂载至 Docker 隔离容器中并执行排查与编辑。 - [18:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1088s) SDK 将外部工具、系统指令与沙箱资产统一打包为 Capability,原生支持通过 Git 仓库管理、版本审查和动态注入 Skills。 - [18:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1088s) 内置上下文自动压缩(Compaction)机制,使智能体能够在超出 Token 上下文窗口限制时自主压缩历史,持续运行数天甚至数周。 - [20:43](https://www.youtube.com/watch?v=tK32trvj_b4&t=1243s) 状态休眠与恢复机制通过将工作区文件系统打包为 tarball 并记录历史上下文 rollout,实现了模型无感知的跨容器冷启动与断点续跑。 - [23:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1388s) 支持从本地 Docker 平滑迁移至云端 Modal 沙箱与 Cloudflare R2 对象存储,实现秒级容器启动及云端快照归档。 - [25:36](https://www.youtube.com/watch?v=tK32trvj_b4&t=1536s) Manifest 清单机制通过显式声明初始目录结构并向上下文自动渲染工作区文件树,大幅减少了模型初始盲目扫描文件的 Token 消耗。 - [27:32](https://www.youtube.com/watch?v=tK32trvj_b4&t=1652s) Function Tools 机制通过装饰器自动化接管参数解析与调用路由,并支持参数守卫、超时熔断及 Feature Flag 动态开关。 - [30:14](https://www.youtube.com/watch?v=tK32trvj_b4&t=1814s) 引入人机协同(Human-in-the-loop)审批机制,支持开发者针对特定关键动作(如标记任务已完成)设置强制拦截和动态核准。 - [31:22](https://www.youtube.com/watch?v=tK32trvj_b4&t=1882s) 多个专业智能体(如程序编辑与视觉生成)可通过任务移交(Handoffs)与状态分派,形成无缝协作的接力执行链路。 - [33:04](https://www.youtube.com/watch?v=tK32trvj_b4&t=1984s) Manifest 支持直接挂载 S3 / R2 兼容外部存储桶,避免海量数据启动时的全量拷贝开销,保证沙箱实时读写中心化的最新数据源。 - [38:15](https://www.youtube.com/watch?v=tK32trvj_b4&t=2295s) 单次轻量任务与小众语言适合直接调用 Responses API,而高复杂度、深度沙箱操作的长程任务则需依赖与 OpenAI 模型高度对齐的 Codex Harness。 - [43:12](https://www.youtube.com/watch?v=tK32trvj_b4&t=2592s) 沙箱生命周期支持 SDK 托管的单轮次自动销毁以防止账单意外,也支持用户自管模式跨轮次保活预热以降低冷启动延迟。 - [44:31](https://www.youtube.com/watch?v=tK32trvj_b4&t=2671s) 针对文件系统初始化,全量拷贝具备运行时本地极速读取优势,而网络挂载则在冷启动速度与大体量数据吞吐上表现更优。 - [46:27](https://www.youtube.com/watch?v=tK32trvj_b4&t=2787s) 未来智能体架构将朝着大规模并行工作发展,多个 Agent 通过共享文件系统或分布式数据库实现高效并发与协作。 ## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=tK32trvj_b4&t=0s) 开场介绍与议程概述 OpenAI 的初创团队营销负责人 Christine 主持了本次 Build Hour,并欢迎观众回到直播间。她隆重介绍了老朋友——来自 OpenAI API 团队的工程师 Steve Coffey,两人之前曾合作开展过一期关于 Responses API 的深度分享。Christine 向新老观众解释了 Build Hour 的初衷:通过分享最佳实践、开发工具以及前沿的 AI 专业经验,帮助开发者更好地基于 OpenAI 的模型和 API 进行构建。屏幕下方提供了官方主页链接,便于观众随时回看往期录播(包括上一期关于 Responses API 的内容),本次分享在结束后也会提供全套录播。 随后,Christine 简明扼要地梳理了本次活动的整体流程: 1. **Agents SDK 重磅更新**:重点介绍最新引入的类似 Codex 风格的 Harness(运行调度脚手架),Steve 会详尽拆解各项新特性; 2. **API 最新发布**:快速回顾近期 API 层面推出的关键功能与调整; 3. **现场实操演示(Live Demo)**:这是本次活动的核心大头,Steve 将现场基于 Agents SDK 搭建一个具备自主能力的 Task Tracker(任务跟踪器); 4. **观众 Q&A 问答**:屏幕右侧常设 Q&A 提问窗口,后台有多位技术团队成员实时打字解答,尾声阶段还会有一位神秘特邀嘉宾连线,共同挑选高频核心问题进行现场解答。 --- ### [01:38](https://www.youtube.com/watch?v=tK32trvj_b4&t=98s) 长程任务能力的飞跃与生产环境 Agent 的挑战 Steve 接过麦克风,切入关于 Agents SDK 过去几个月迭代的核心背景。他指出,当前大语言模型正在经历一次本质上的能力跃迁——模型在漫长的时间跨度内独立自主完成复杂任务的能力呈现出强劲的上升势头。许多业内人士都见过反映模型自主工作时长上限的走势图,那条曲线正在持续地“向右上方狂飙”。以 OpenAI 内部的明星产品 Codex 为例,许多开发者日常使用它进行全流程软件构建时,可能习惯了它持续运行几分钟甚至一个小时;但如果启用 Goal(目标)功能,Codex 在内部测试中已经可以连续运行数天乃至整整一周来攻克极具挑战性的工程任务。 ```mermaid flowchart LR accTitle: 传统模式与分离架构对比 accDescr: 对比 Harness 与计算环境耦合和分离两种架构在安全与状态管理上的差异 subgraph Coupled["耦合架构(传统 / 本地)"] direction TB A1["Agent 循环调度<br/>(Harness)"] --- B1["执行沙箱 / 文件系统<br/>(Compute)"] B1 -.-> C1["单点失效 / 状态丢失<br/>密钥暴露风险"] end subgraph Decoupled["分离架构(Agents SDK)"] direction TB A2["持久化 Harness<br/>(Temporal / AWS / 业务后端)"] -->|"跨网络安全指令"| B2["无状态计算环境<br/>(沙箱: E2B / Modal / Docker)"] A2 -->|"外部快照与重水化"| D2["安全持久化存储 / 密钥隔离"] end ``` 这种长程执行能力在 OpenAI 内部孵化出了诸多关键生产级 Agent: - **安全扫描 Agent**:基于 Codex 构建,利用模型极强的主动搜寻与代码审计能力,自动扫描内部代码仓库和第三方依赖库中的遗留漏洞,守卫系统底层安全。 - **内部数据分析 Agent**:同样依托 Codex 连接到内部数据湖(Data Lakes),大幅颠覆了研发与业务的交互方式。以往工程师查证“Agents SDK 目前有多少人在用”或“两天前 Responses API 产生了多少调用量”这类指标时,可能需要绞尽脑汁花上一个小时手写复杂的 SQL 关联查询;如今只需在聊天框输入一句话提示词,几分钟之内就能拿到精准的数据报表。 然而,Steve 坦言,将原型 Agent 转化为**生产可用(Production-Grade)系统**仍然极具挑战,开发者主要面临三大痛点: - **性能最大化与模型灵活性的两难**:现代前沿模型高度适配于特定的执行 Harness,开发者既想让模型处于最高效的“分布内(In-Distribution)”表现,又希望能保留灵活切换跨模型供应商底层平台的能力,往往顾此失彼。 - **现代 Computer-Using Agent 的基础设施脆弱性**:本地开发时 Codex 直接运行在笔电上,沙箱与调度逻辑合为一体;但部署至云端生产环境时,隔离运行的容器沙箱很容易发生崩溃、超时过期。如果容器本身承载了关键业务状态,容器消亡便意味着任务彻底失败,状态恢复(Rehydration)极其棘手。 - **高度定制化与扩展需求**:企业既需要方便地挂载专有数据、注入自定义技能(Skills),又不想被死板的原语绑死。 --- ### [04:40](https://www.youtube.com/watch?v=tK32trvj_b4&t=280s) Agents SDK 的解耦架构与生态支持 针对上述生产落地难题,Agents SDK 确立了全新架构。Steve 强调,自去年 3 月发布以来,Agents SDK 始终保持**完全开源**且**高度可定制**,而此次升级带来了划时代的变化: 第一是**将 Harness(调度脚手架)与 Compute(计算沙箱)彻底分离**。过去在笔电或单个容器里运行 Agent 时,负责发起 LLM API 调用的调度循环(Agent Loop)与底层执行命令的文件系统紧密耦合在一起。一旦放到大规模生产环境,这种“全包容器”就会变成不可承受的“承重墙”(Load-bearing):容器挂掉,内存和上下文全毁,难以从外部恢复;更致命的是,为了让容器里的 Agent 执行外部调用,开发者被迫把各类 API 密钥或敏感凭据直接塞进沙箱,极易遭受 Prompt 注入攻击或凭据外泄。 Agents SDK 通过将两者切开,将沙箱彻底转变为**完全临时、即用即弃(Ephemeral)的消耗品**: - **Harness 负责大脑与状态管理**:运行在企业原有的健壮基础设施中(例如 Temporal 定时任务、AWS 计算节点或内部微服务),统一掌控上下文快照、状态重水化(Rehydration)与密钥管理,沙箱内不留任何敏感秘密。 - **Compute 仅作为隔离执行环境**:沙箱即便随时崩溃、失效或被轮替,Harness 也能从容拉起一个崭新的空沙箱,重放快照继续执行,彻底消除安全外泄隐患。 第二是**对主流沙箱基础设施的一等公民支持(First-Class Support)**。开发者无需重复造轮子,SDK 原生开箱支持众多主流平台与本地环境: - 云原生代码沙箱平台:**E2B**、**Modal**、**Cloudflare Workers** - 现代云平台与运行时:**Vercel**、**Blaxel**、**Daytona** - 标准化容器与本地测试:**Docker** 以及本机的直接运行支持。 第三是**模型原生的 Codex 风格 Harness 体验**。SDK 将沉淀自 Codex 的整套最佳实践与内建工具链原汁原味地抽象出来,包含模型最熟悉的标准工具(在分布内高度对齐,同时兼容其他厂商模型)、自动化技能调用(Skill Use)、计算机使用(Computer Use)、长程记忆机制(Memory),以及开箱即用的 Web 搜索、文件检索、MCP(Model Context Protocol)扩展和 Code Interpreter 代码解释器。开发者不再需要把 80% 的精力浪费在手写 `请求模型 -> 解析 Tool Call -> 触发执行 -> 拼装上下文重新请求` 的冗长编排外壳上,而是可以把注意力 100% 聚焦在业务产品功能的打磨上。 ### [08:50](https://www.youtube.com/watch?v=tK32trvj_b4&t=530s) 深度融合 Codex 的执行框架:沙箱、异步 Shell 与长期记忆 在此次 Agents SDK 的重大迭代中,团队带来了多项令人振奋的新特性,其中包括对技能(Skills)的一等公民(first-class)支持、在 API 和 SDK 层面推出的容器化环境(Containers),以及帮助任务随着时间推移不断进化、越跑越聪明的 Agent 记忆机制(Agent Memory)。整个架构深受 Codex 体系的启发,核心目的在于为模型配备一整套在虚拟化沙箱中长期运行的工具链。模型不仅能够直接读写并执行 Bash 和 Python 等脚本,还能安全地在隔离的虚拟环境中处理长程任务。这一整套工具链自始至终保持开源与高度可定制性。 具体到所谓“Codex 风格的执行框架(Codex-inspired harness)”,其精髓在于借鉴了 Codex 处理高复杂度任务的几项底层能力:例如上下文的自动压缩(auto-compaction),以及通过 Shell 终端实现计算机操作的核心交互循环。在此循环中,模型以异步(async)方式执行终端命令——如果一条命令执行耗时较长,模型无需死等,而是可以记录当前后台任务、转头去处理其他子任务,稍后再回来检查运行结果。这种异步跟踪命令状态的能力被完整植入到 Agents SDK 中,使代码编写型 Agent 能在各种业务场景和底层基础设施中游刃有余。 ```mermaid flowchart TD accTitle: Codex-inspired Agent Harness Architecture accDescr: 展现 Agents SDK 基于 Codex 理念打造的沙箱隔离、异步执行与状态恢复循环。 subgraph Core["Agent 核心循环与记忆"] Agent["Sandbox Agent\n(支持长期记忆与自动压缩)"] AsyncLoop["异步 Shell 执行循环\n(追踪后台命令 / 状态轮询)"] end subgraph Sandbox["隔离执行沙箱 (Docker / 容器 API)"] Workspace["工作区文件系统\n(代码仓库 / PDF / Office 资产)"] Runtime["受控执行环境\n(Bash / Python 运行时)"] end Agent -->|"下发异步命令"| AsyncLoop AsyncLoop -->|"执行脚本与文件修改"| Runtime Runtime <-->|"读写与生成文件"| Workspace AsyncLoop -.->|"返回执行结果与日志"| Agent ``` 沙箱环境在现代工作流中是绝对不可或缺的基石。在现实工作中,人类每天都要面对极其庞杂的文件系统:可能是在庞大的代码库中穿梭修改多份源码,也可能是在处理堆积如山的 PDF、撰写 Word 文档或制作 PPT 幻灯片。Agent 同样需要这样的执行实体,它们必须拥有读取、创建和存储这些文件的专属空间。沙箱就是专为 Agent 划出的隔离空间,让模型在受控环境下自由调用文件、产出真正有业务价值的交付成果。此外,Agents SDK 坚持完全开源和模型不可知(Model Agnostic),只要遵循 Responses API 格式的模型均可无缝接入;同时原生支持多租户高并发架构,并内置会话快照(snapshotting)与状态恢复(rehydration)机制,让任务即使中途暂停也能精准地“从上次停下的地方直接继续”。 - **技能一等公民支持**:内置对技能文件的标准化解析与调度机制。 - **异步 Shell 交互模型**:支持下发耗时任务并转入其他工作,定期轮询后台进程状态。 - **状态快照与断点续跑**:内置任务级别的状态持久化机制,轻松实现多租户并发隔离与会话恢复。 - **全方位开放兼容**:除了 OpenAI 模型外,允许接入任何兼容 Responses API 的外部模型,支持自定义工具和 MCP(Model Context Protocol)。 --- ### [11:24](https://www.youtube.com/watch?v=tK32trvj_b4&t=684s) Responses API 新特性:托管 Shell、容器网络权限与 Skills API 团队还同步介绍了过去几个月在底层 API 层面密集交付的多项重要特性。首先是 Responses API 中新增的“托管 Shell 工具”(Hosted Shell Tool)。该功能面向轻量级即时调用场景:开发者只需发起一个普通 API 请求并传入待处理文件,OpenAI 就会在后台自动拉起一个临时容器、载入目标文件,模型即可在容器中编写并运行代码完成任务,最终直接将结果返回给开发者。你可以将其理解为 Agents SDK 的极简托管版(lightweight version)。开发者既可以使用自动模式——传入文件由系统全自动创建并在完成后销毁容器;也可以通过独立的 Containers 端点提前创建容器、上传依赖资产,再将其挂载至 Responses API 请求中。 网络安全控制方面,API 现已支持对拉起的容器进行细粒度的网络访问策略配置。为了满足企业级安全合规要求,开发者可以为容器设置域名白名单(domain allow lists),甚至直接开启全封闭模式,完全切断容器的一切出网(egress)和入网(ingress)流量,杜绝任何数据泄露风险。 ```mermaid flowchart LR accTitle: Skills API and Container Integration Flow accDescr: 展示技能包通过 Skills API 管理并挂载到托管 Shell 容器中运行的流程。 subgraph SkillsRegistry["Skills API (统一真实数据源)"] SkillPkg["技能包 (ZIP)\n• skill.md (顶层说明)\n• 专用脚本 / 依赖资源"] Versions["多版本管理与迭代\n(设定默认版本)"] SkillPkg --> Versions end subgraph Execution["Responses API 托管容器"] Container["Hosted Shell 容器\n(支持域名白名单 / 网络物理隔离)"] Run["模型调用技能脚本\n执行特定业务任务"] end Versions -->|"挂载指定版本"| Container Container --> Run ``` 另一个重量级发布是 Skills API。以往使用技能包时,团队通常面临缺乏“单一真实数据源”(single source of truth)的痛点,不得不依赖公共 GitHub 仓库或云存储桶。全新的 Skills API 提供了一套标准的技能分发与托管方案。一个技能本质上是一个打包文件(通常为 ZIP),顶层包含一个说明性质的 `skill.md` 文件,同时内嵌该技能专属的可执行脚本和辅助资源。例如,你可以构建一个“2025 年报税技能”,内部封装 IRS 官方报税规则、自动化解析财务票据的脚本以及 1040 表格填写逻辑。开发者可将该技能上传至 Skills API,持续迭代多版本、指定默认版本,并与托管 Shell 工具无缝联动。最后,针对 Python 版本发布后社区的强烈呼声,官方正式推出了 Sandbox Agent 的 TypeScript 原生版本,彻底补齐了前端与全栈开发者的生态版图。 - **Hosted Shell 容器生命周期**:支持一键自动创建与销毁的瞬态模式,以及通过独立 Containers 端点进行精准预分配的高级模式。 - **容器网络安全沙箱**:提供针对网络流量的精细化访问控制,支持域名白名单筛选及完全物理断网模式。 - **Skills API 规范**:支持直接上传 ZIP 技能包,集中托管 `skill.md` 顶层文档及业务配套脚本,原生支持版本回溯与迭代。 - **双语言生态就绪**:在继 4 月发布 Python 版本后,TypeScript 版本的 Sandbox Agent 现已全量上线。 --- ### [14:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=848s) 现场实战演示:从零构建基于 Docker 沙箱的任务追踪器 为了直观展示这些能力的威力,Steve Coffey 进入了实战演示环节。他开门见山地调侃道,自己今天通过“氛围编程”(vibe coded)手搓了一个类似 Linear 的敏捷看板工具,而他今天的终极目标就是“把所有工作全扔给 Agent,自己什么活都不用干了”。整个 Demo 的设定是一个大型技术峰会的筹备场景,主策划人需要处理繁杂的议程事务,包括校订演讲内容、生成大会节目单资产以及更新官网宣传等。 现场演示中出现了一个真实而有趣的技术插曲:Steve 在展示新建任务弹窗时,原本准备给观众留作惊喜的 Agent 分配选项提前泄露了,他笑着坦白:“刚才大家不小心提前偷瞄了一眼即将登场的功能”,随后果断刷新页面重置演示状态。重置后,他在看板中创建了一个名为“创建节目单物料资产”(create program assets)的任务,并将一组伪造的演示素材文件拖拽上传到任务中。在传统的非 Agent 时代,这类任务只能分派给自己或其他同事手动熬夜处理。 ```mermaid flowchart TD accTitle: Demo Task Assignment and Sandbox Execution accDescr: 任务看板将文件与指令同步至底层 Docker 容器并执行 Sandbox Agent 流程。 A["Linear 风格任务看板\n创建任务: create program assets\n上传会议素材文件"] --> B["指派给 Program Editor Agent\n(底层基于 SandboxAgent 类)"] B --> C["Agents SDK 自动接管\n启动本地 Docker 沙箱容器"] C --> D["文件同步挂载\n将素材注入沙箱工作目录"] D --> E["Agent 执行 Shell 检查命令\n识别文件内容并输出编辑建议"] ``` 随后,Steve 切到后台代码,取消了一段预先写好的 Agent 声明代码注释。他解释道,如果之前用过 Agents SDK,这里的代码会显得极其熟悉:`SandboxAgent` 本质上就是之前标准 `Agent` 类的子类,继承了原有的全部核心参数与配置模式,同时新增了与沙箱宿主交互的能力。在配置好基本指令提示词后,看板页面重新加载,指派人列表中正式出现了“节目单编辑 Agent”(Program Editor Agent)。 Steve 在任务描述框中输入指令:“请仔细检查这些文件,并对它们进行针对性的清晰度编辑(edit them for clarity)”,随即点击发送。在后台,由于该 Agent 配置了本地 Docker 作为沙箱后端,Docker 容器立刻被唤醒并处于运行状态。Agents SDK 自动接管了文件传输流程,将看板中上传的所有会议素材同步推送到 Docker 容器内的指定隔离工作目录中。模型被安全限制在该工作目录下,开始自主执行 Shell 命令遍历并分析文件。短短几秒钟后,Agent 顺利完成了首轮文件排查,在控制台打印出完整的操作日志,并敏锐地指出素材中缺失的部分数据,展现了沙箱化 Agent 独立执行复杂本地任务的强悍执行力。 ### [18:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1088s) 基于 GitHub 仓库的 Skills 能力与 SDK Capabilities 机制 单纯让智能体运行通用代码还远远不够,开发者通常需要赋予智能体高度定制化、针对具体业务场景的特定上下文,而技能(Skills)正是达成该目标的核心方式。Steve 展示了一个为大会日程编辑(Conference Program Editor)预先编写的技能示例。尽管该技能的内容带有一定的随性编写风格("entirely vibe coded"),但其核心逻辑在于展示如何组织与分发这些技能。 ```mermaid flowchart TD accTitle: Capabilities 架构体系 accDescr: 展示 Agents SDK 中 Capabilities 如何聚合文件系统工具、Shell 工具、上下文压缩以及外部技能能力 subgraph Capabilities["Agents SDK Capabilities"] FS["File System: view images & apply patch"] SH["Shell: exec_command & write_stdin"] CP["Compaction: 自动压缩上下文,支持长任务运行"] SK["Skills Capability: 动态拉取与挂载"] end GH["GitHub / Git 仓库 (版本控制 & PR 审查)"] -->|拉取 main 分支| SK ``` 在架构设计上,GitHub 成了托管技能仓库的绝佳选择。相比直接调用纯粹的技能 API,将技能存放在 Git 仓库中天然继承了完整的工程化能力:团队可以通过版本控制跟踪提示词与指令的演进,借助 Pull Request 进行协同代码审查(Code Review),实现高质量的资产治理。OpenAI Agents SDK 对 Git 提供了原生一等支持(first-class support)。 在更新后的 Agents SDK 中,**Capability(能力)** 被抽象为一个高度复合的对象,能够将多种系统概念无缝打包在一起: - **合并多重要素**:单个 Capability 能够打包外部工具(Tools)、追加系统指令(Instructions),并在沙盒启动时向机器文件清单(Manifest)注入预设资产。 - **默认内置能力集合**: - **文件系统(File System)**:赋予模型读取文件、检查图片以及调用 `apply_patch` 进行行内精确代码编辑的能力。 - **Shell 工具**:由底层异步执行工具对 `exec_command` 与 `write_stdin` 复合而成,实现类似异步 Bash 命令流的操作。 - **上下文压缩(Compaction)**:使模型在处理长周期任务并在超出上下文窗口(Context Window)限制时,能够自主压缩历史上下文继续运行。从技术上讲,该机制去除了运行时间的绝对上限,让智能体持续工作数小时、数天甚至数周成为可能。 - **Skills Capability 的声明与引入**:在代码中添加 Skills Capability 时,只需将其指向指定的 Git 仓库 URL 并指定目标分支(如 `main`)。SDK 默认使用 GitHub 作为托管服务,但同时也支持灵活切换至其他自托管 Git 服务端。 - 演示中 Steve 给智能体发送了测试任务指令:“告诉我你的技能文件里最有趣的内容是什么?(Tell me the most interesting thing from your skill file.)”以验证技能加载机制。 --- ### [20:43](https://www.youtube.com/watch?v=tK32trvj_b4&t=1243s) 沙盒环境的状态休眠与快照复原机制 长运行智能体不可避免地需要处理跨会话的暂停(Pause)与恢复(Resume)。当启动一个 Sandbox Agent 时,Agents SDK 会根据代码中声明的配置自动初始化运行沙盒。在本地开发阶段,默认使用 Docker 沙盒客户端(`DockerSandboxClient`),配合标准运行镜像(如 `python:3.12`)。 ```mermaid flowchart LR accTitle: 沙盒暂停与状态恢复流程 accDescr: 展示任务停止时沙盒打包文件系统,以及任务恢复时在新容器中无感知解包回填的闭环流程 subgraph Suspend["任务暂停 / 结束阶段"] A["Agent 执行任务"] --> B["停止 Sandbox 容器"] B --> C["抓取文件系统快照生成 tarball"] C --> D[("存储后端: 本地磁盘 / Cloudflare R2")] end subgraph Resume["任务恢复阶段"] D --> E["拉取对应任务 tarball"] E --> F["启动新容器并解压回填文件系统"] F --> G["Agent 无缝继续执行 (模型完全无感知)"] end ``` 当任务完成某阶段并暂停或停止运行时,SDK 内部的生命周期管理逻辑会自动介入: 1. **停止运行实例**:系统立即停止当前的沙盒容器。 2. **截取文件系统快照**:自动抓取该容器当前的文件系统状态,并打包归档(默认采用 `tarball` 压缩包)。 3. **分发至存储目标**:快照文件会被写入用户指定的存储位置。在本地测试时,该路径即为开发者笔记本电脑的本地磁盘。 4. **无感知恢复(Rehydration)**:当再次唤醒该任务时,SDK 从存储中检索对应的 tarball 包;若前一个容器已不存在,则冷启动一个全新的空容器,并将文件系统完整解包回填至新沙盒中。 对于大语言模型而言,它所处的工作区文件树与之前的状态毫无二致,模型“完全意识不到自己正在一个全新的容器实例上运行”。虽然针对不同生产级云提供商(例如 Modal 的 Volumes 卷存储或快照 API)支持其专属的高级持久化驱动,但 Agents SDK 默认的朴素 tarball 快照方案兼顾了极高的通用性,可以在任意环境中开箱即用。 --- ### [23:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1388s) 生产环境迁移:Modal 沙盒与 Cloudflare R2 云端快照存储 本地 Docker 适合原型验证,但面对生产部署则面临扩展与运维挑战。Steve 强调,业内涌现了一批优秀的沙盒基础设施服务,Agents SDK 对 Modal、Cloudflare、E2B、Vercel、Daytona、Blaxel 等平台均提供了第一方或插件式扩展支持,同时也支持开发者实现自己的沙盒后端。 为了将智能体提升(hoist)至云端,Steve 演示了将本地 Docker 切换至 Modal 沙盒环境的过程。在代码实现上,底层统一继承自 `BaseSandboxClient`,开发者只需提供符合 SDK 规范的客户端对象及参数: ```python # 示例:通过统一接口切换至云端沙盒客户端 sandbox_provider = ModalSandboxClient( image="python:3.12", snapshot_storage=R2SnapshotStorage(...) ) ``` 在云端沙盒方案中,持久化存储同步迁移到了云端对象存储 Cloudflare R2。不仅沙盒算力转移到了 Modal 云端,快照归档也不再占用本地磁盘,彻底实现了云端闭环。 - **极速冷启动**:在向会议日程编辑智能体下发“为提升清晰度编辑这些文件(edit these files for clarity)”的任务后,Modal 沙盒在仅耗时约 1 秒左右的时间内即完成了容器启动。 - **状态跨云流转**:任务执行结束后,Cloudflare R2 控制台即时出现了最新的任务快照归档文件。后续如果需要从断点唤醒长任务,生产环境实例可直接从 R2 并发拉取状态,极大降低了分布式调度的开销。 --- ### [25:36](https://www.youtube.com/watch?v=tK32trvj_b4&t=1536s) 沙盒初始化核心机制:Manifest 清单与工作区透视 智能体在多步骤任务中往往需要预置初始环境与基准资产,SDK 通过 **Manifest(清单对象)** 优雅地解耦了环境构建逻辑。Manifest 的核心概念非常直接:它是一种用于显式声明“当智能体沙盒启动那一刻,文件系统目录结构到底该长什么样”的契约。 Manifest 支持跨任务、跨智能体复用一套标准的文件骨架。在较为简单的场景中,开发者只需在 Manifest 中定义一个目录层级树(Tree),为沙盒划定初始工作目录,随后再按需上传业务文件。而在复杂的工业级场景中,Manifest 的扩展能力涵盖: - 直接从运行 Harness 的宿主机器复制代码文件; - 挂载来自 Cloudflare R2、Amazon S3 或 Azure Blob Storage 的远端存储桶资产; - 原生绑定远程 Git 仓库。 更关键的是,SDK 在沙盒启动时会基于 Manifest 自动渲染一份结构化的**工作区文件树视图(File Tree View)**直接注入模型的初始上下文中。这意味着模型在介入任务的第一时间即可对全局文件拓扑一目了然,无需在执行初始阶段耗费大量模型轮次调用 `ls` 或 `grep` 进行盲目探测,极大地节省了 Token 开销并压缩了首步任务的决策延迟。结合可配置的节点描述信息,智能体得以在启动后迅速切入具体的代码与文档编辑流程。 ### [27:32](https://www.youtube.com/watch?v=tK32trvj_b4&t=1652s) 自定义 Function Tools 与自动化工具循环 在构建智能体时,将系统特有的上下文和业务功能注入到模型中是不可或缺的。Steve 回到程序编辑器智能体(Program Editor Agent)的代码中,取消了一组预定义工具的注释。演示中提前准备了三个核心工具:更新任务状态(update status)、更新任务经办人(update assignee)以及搜索所有候选经办人(search assignees)。使用过 Agents SDK 的开发者对这套机制会非常熟悉——所谓的 Function Tools 本质上就是普通的 Python 函数(在 TypeScript SDK 中则是普通的 TypeScript 函数),开发者只需为其添加 `@function_tool` 装饰器即可。 ```mermaid flowchart LR accTitle: Agents SDK 自动工具调用流程 accDescr: 展示从模型发出工具调用指令到 Agents SDK 自动路由并回传结果的闭环过程。 subgraph ModelLayer["模型交互层"] A["LLM 模型"] end subgraph SDKLayer["Agents SDK 运行机制"] B["API 架构描述<br/>(参数定义)"] C["工具执行分发器<br/>(自动路由)"] end subgraph ToolLayer["业务代码层"] D["本地函数<br/>(@function_tool)"] end B -->|"声明可用工具"| A A -->|"发起工具调用指令"| C C -->|"执行底层函数"| D D -->|"返回结果"| C C -->|"回传执行结果"| A ``` 这个装饰器的核心作用在于充当桥梁:它能够自动将本地语言的函数定义解析并转换为 API 规范的函数工具描述格式,精确声明每个输入参数,从而让大模型明确知道该工具的用途以及调用时需传入的参数。更重要的是,当 Agents SDK 从 API 接收到调用具体工具的指令时,它会在后台自动将请求路由到对应的函数上执行,并将执行结果无缝打包回传给大模型。开发者完全不需要亲自动手编写复杂的工具调用循环(function calling loop)。Steve 在任务界面中输入指令:“请将此任务分配给 Steve,并将其标记为待审查(ready for review)”,触发模型进行实际的工具调用。 在任务启动期间,Steve 深入介绍了 Function Tools 提供的进阶控制能力: - **名称与描述覆盖**:允许对外暴露与本地函数名不同的名称,并通过精细的描述性文本对模型的调用意图进行定向引导。 - **参数守卫(Guardrails)与执行钩子(Hooks)**:能够在模型尝试传入特定参数时进行拦截和校验,控制底层的执行流程。 - **超时控制(Timeouts)**:针对耗时或不稳定的外部调用设置明确的超时中断。 - **动态开关(Dynamic Toggling)**:允许直接绑定业务系统的 Feature Flag 动态启用或禁用特定工具,无需在代码中反复手动增删工具列表。 返回演示界面后,系统已成功执行相应工具,将任务正式分配给了 Steve。这种模式为开发者接入专属数据库或通过 MCP(Model Context Protocol)接入 Slack 等外部工作流提供了标准路径。 --- ### [30:14](https://www.youtube.com/watch?v=tK32trvj_b4&t=1814s) 人机协同:工具调用的审批机制(Tool Call Approvals) 在某些关键业务节点上,完全放任智能体自主调用工具可能会引发合规或业务风险,此时引入“人机协作(Human-in-the-loop)”验证环节显得尤为关键。Steve 举例指出,老版本的 Codex 用户对此深有体会——模型在尝试执行 Shell 命令或访问互联网时,经常会停下来向用户确认许可。Agents SDK 将这一审批模式原生封装为了一个极度灵活的接口。开发者既可以简单粗暴地将审批配置硬编码为布尔值(如直接传 `True`),也可以传入一个动态校验函数,由该函数根据模型即将传入的入参动态决定是否需要人工干预。 在本次演示中,代码设定的规则是:当智能体尝试将任务状态变更为“已完成(done)”时,必须强制经过人工审批。Steve 明确表示:“我绝不会在未经快速审查前,就允许你把某件事直接标记为完成。”随后他在界面中测试下达指令:“请将此任务标记为已完成。”智能体后台立即拉起沙箱容器并重新挂载(rehydrate)任务状态,并在识别出调用工具的动作触发了设定的拦截规则后,准时暂停等待授权。 在任务详情的活动动态面板中,界面随即弹出了醒目的审批请求卡片(request for approval)。当用户在 UI 上点击“批准(Approve)”按钮后,被挂起的执行流被重新激活,工具调用顺利执行,任务最终才被正式推进至完成状态。这一机制为安全策略与自动化执行之间提供了可靠的缓冲带。 --- ### [31:22](https://www.youtube.com/watch?v=tK32trvj_b4&t=1882s) 多智能体协作与任务移交(Handoffs) 借助前面配置的经办人搜索与更新工具,多个各司其职的智能体能够形成自动化的协作接力流程。系统内此时并存两个专业智能体:一个是前面一直演示的程序编辑器智能体(Program Editor Agent),另一个是专门负责生成图像及相关视觉素材的资产生成智能体(Asset Producer Agent)。 Steve 在系统中创建了一个新任务,并上传了一批文件附件,在任务描述中写道:“请优化此内容并生成部分资产……请先优化内容,然后将其分配给资产生成智能体。”随后,他将该任务的首位经办人指派给程序编辑器智能体。 整个接力执行链路清晰明了: 1. **程序编辑器智能体领单**:首先拉起独立的沙箱容器并挂载初始文件,执行文本优化和代码处理等核心工作。 2. **状态更新与经办人移交**:完成前置任务后,程序编辑器智能体调用预置工具查询目标角色,并将任务经办人变更为资产生成智能体。 3. **资产生成智能体接管**:资产生成智能体被自动触发唤醒,在对应的沙箱环境中承接中间产物,继续完成资产的批量生成,并最终将新生成的文件写入沙箱目录中。 --- ### [33:04](https://www.youtube.com/watch?v=tK32trvj_b4&t=1984s) 外部存储桶挂载:R2 与 S3 数据流优化 随着任务复杂度提升,在沙箱环境中频繁读写大容量或高时效性数据成为了新的瓶颈。Steve 重点讲解了在沙箱清单(Manifests)中直接挂载外部对象存储存储桶(External Buckets)的高级特性,并梳理了该特性的两大核心应用场景: - **海量文件处理**:当模型需要解析数百个 PDF 或巨量数据集时,每次启动沙箱都全量拷贝数据会带来极大的网络开销与存储浪费。 - **数据强时效性(Freshness Constraints)**:若底层数据在不断高频变动,预先静态拷贝进沙箱的文件可能在智能体刚开始工作时就已经过时;直接基于中心化的真实数据源(Source of Truth)动态挂载,能保证智能体始终工作在最新数据之上。 针对这一需求,SDK 实现了原生挂载方案。Steve 在代码中解开了 `R2AttachmentStore` 相关配置。在此之前,前端上传的文件会先缓存到本地笔记本电脑再逐一复制进容器;而接入 R2 后,文件直接上传至 Cloudflare R2 对象存储,将其确立为应用层唯一的真实数据源。在环境清单配置中,容器内部的结构保持不变(仍然包含统一的 `task` 目录及四个标准子目录),但其中的输入目录(input directory)不再映射为空目录,而是通过 `S3Mount` 类型直接挂载。由于 Cloudflare R2 完全兼容 AWS S3 的 API 规范,SDK 内部直接复用了统一的 S3 挂载对象,并填入 R2 存储桶名、Access Key 和 Secret Key。 ```mermaid flowchart TD accTitle: 外部存储桶挂载架构对比 accDescr: 对比传统沙箱全量拷贝与基于对象存储直接挂载两种模式的数据流差异。 subgraph RemoteStorage["远程存储层 (Source of Truth)"] R2["Cloudflare R2 / S3 兼容存储桶"] end subgraph SandboxContainer["沙箱容器内部文件系统"] subgraph MountLogic["挂载驱动 (Modal Cloud Bucket / Rclone / FUSE)"] VDir["/task/input/ (虚拟本地目录)"] end Agent["智能体执行进程<br/>(视网络挂载为普通本地磁盘)"] end R2 <==>|"按需网络读取/写入"| MountLogic Agent -->|"标准 POSIX 文件操作 (ls / cat / open)"| VDir ``` 在挂载实现策略上,不同的沙箱底层有不同的支持方式: - **Modal 原生云存储桶策略**:演示所采用的 Modal 沙箱环境内置了对云存储桶的原生挂载加速支持,SDK 可直接声明使用其专用策略。 - **通用容器挂载**:针对 Docker 或其他无特定基础设施优化的沙箱环境,SDK 开箱即用支持两种底层驱动模式——**Rclone** 和 **FUSE**。开发者可以根据具体系统的兼容性和性能表现自由选择。 Steve 现场上传了一批新资产创建任务,并指派给程序编辑器智能体。通过后台可以确认,文件已经实时出现在 R2 存储桶的任务路径下。当智能体启动并对容器内的挂载路径发起 `ls` 或读取操作时,整个网络交互过程对大模型是完全透明的,它感知到的就像是一个读写速度极快的本地文件系统。Steve 也坦承这种方案存在权衡(trade-off):由于必须跨网络交互,当智能体在大规模存储桶中列出海量目录(`ls`)时可能会产生轻微的网络延迟,但相比于频繁全量拷贝受限于沙箱磁盘容量以及数据滞后的巨大缺陷,这种网络挂载无疑是一种优雅得多的解决方案。至此,整个构建任务追踪器的核心功能演示圆满收尾。 ### [37:25](https://www.youtube.com/watch?v=tK32trvj_b4&t=2245s) Q&A 环节开启与 Nish Singaraju 登场 在演示完基于沙箱的长程任务执行与代码编辑能力后,演讲进入了观众问答环节。主持人热情欢迎负责 Agents 产品线的产品经理 Nish Singaraju 登台。Nish 此前一直在直播频道中实时解答开发者的疑问,他对开发者社区提出的高水准问题感到非常振奋,并正式加入讨论,与技术团队成员 Steve Coffey 一同解答关于 Agents SDK 的核心架构、运行机制与未来规划。 --- ### [38:15](https://www.youtube.com/watch?v=tK32trvj_b4&t=2295s) Harness 架构与原生 Responses API 的适用场景对比 现场提出的第一个问题直指架构选型核心:“什么时候应该使用基于测试套件/脚手架(Harness)的智能体,什么时候又该直接使用 Responses API 构建?”Steve 对此进行了深度拆解。他指出,Responses API 是一套在默认状态下就具备智能体特性的 API,非常擅长应对广泛的任务。特别是在某些较为小众或特别的编程语言环境中(例如 Elixir 或 Haskell),如果开发者找不到心仪的智能体框架,直接基于原生的 Responses API 封装并重构这些概念往往是更直接的选择。此外,针对文档翻译、非结构化数据转 JSON 等单次调用(one-shot)任务,或者仅需通过托管工具进行简单循环的轻量化流程,Responses API 开箱即用,单个 API 调用即可轻松完成编排。 然而,对于复杂的长程任务,Harness 则是不可或缺的核心底座。Steve 强调,随着模型迭代,“Codex Harness 与 OpenAI 模型的结合正变得越来越紧密”。在未来,开发者很可能会发现,只有在模型训练时所针对的原生 Harness 中运行,才能激发出模型的最佳性能。如果开发者的生产环境能够直接接入这些 Harness 组件,Agents SDK 能让高复杂度智能体的搭建变得异常轻松。 ```mermaid flowchart TD accTitle: Responses API 与 Agents SDK Harness 选型决策 accDescr: 对比轻量单次任务与复杂长程开发任务在技术架构选型上的差异 subgraph Direct["Responses API 原生调用"] D1["单次任务 / 结构化提取 / 快速翻译"] D2["小众语言环境 (如 Elixir, Haskell)"] D3["单次 API 托管工具轻量调用"] end subgraph Harness["Agents SDK / Harness 架构"] H1["长程、多步骤复杂工作流"] H2["深度代码编辑与沙箱执行"] H3["与 Codex 原生环境紧密耦合的最佳性能"] end Direct ---|轻量灵活| Output1["快速构建无状态轻量应用"] Harness ---|长程执行| Output2["构建稳定可靠的深层 Agent 循环"] ``` --- ### [39:54](https://www.youtube.com/watch?v=tK32trvj_b4&t=2394s) 原生持久化状态管理与暂停恢复机制 针对智能体是否具备原生开箱即用的持久化状态管理(以支持运行暂停与断点续跑),Steve 给出了明确的肯定答复。Agents SDK 内置了此前演示中展示的快照(snapshot)机制。开发者唯一需要配置的,只是快照文件的存储路径——可以按默认设置存放在本地磁盘,也可以无缝对接 Cloudflare R2 等云端对象存储。该机制不仅能将沙箱的文件系统完整打包压缩并异地转储,以便后续由其他沙箱节点拉取还原,还会同步接管智能体的完整运行轨迹。 Steve 解释道,智能体的持久化状态由两部分构成: 1. **沙箱文件系统的全量快照**; 2. **当前对话上下文中的所有历史消息(完整 rollout)**。 Agents SDK 将这两部分数据统一打包为一个轻量的 JSON 对象。该对象记录了快照的具体存储指针以及全量运行历史,开发者只需将其写入数据库,即可在多节点分布式系统中无损加载并恢复任务。Nish 随后补充强调,这种既保存上下文 rollout 又对文件系统进行实时快照的设计,是 Agents SDK 区别于其他框架的独特优势,使得智能体在面对持续数小时乃至整天的长周期工作流时,能够极其稳健地实现“长程断点恢复”。 --- ### [40:31](https://www.youtube.com/watch?v=tK32trvj_b4&t=2431s) Agents SDK 沙箱与 Codex 原生 Harness 的能力差异 在被问及 Agents SDK 中的沙箱智能体(Sandbox Agent)与 Codex 官方原生二进制文件之间的能力差异时,Nish 详细罗列了二者的异同。Agents SDK 继承了 Codex 大量底层的核心能力,包括 Bash 命令工具、文件修改机制、技能(Skills)体系以及记忆(Memories)存储模式。开发者目前完全可以使用 SDK 达成绝大多数 Codex 二进制工具所支持的操作。 不过二者在并发架构上仍有区分: - **Codex 二进制系统**:天生采用多线程架构,原生支持并发运行多个不同的子智能体。 - **Agents SDK**:现阶段主要通过智能体交接(Handoffs)的机制来实现类似的分工协作模式。 Nish 表示,团队目前正持续将 Codex 二进制文件中的高级特性移植到 Agents SDK 当中,目前 Agents SDK 在能力分布上已经与当前的 Codex Harness 保持了高度的一致性与紧密协同。 --- ### [42:29](https://www.youtube.com/watch?v=tK32trvj_b4&t=2549s) 长程工作流自动化持久化与状态恢复实现 关于长程工作流的恢复是否能够完全自动化,Steve 结合演示代码进一步消除开发者的顾虑。在实际代码中,整体编排逻辑出奇地简短精炼:开发者只需要触发 SDK,将保存有 rollout 轨迹的 JSON 数据写出到本地或数据库中,后续便能随时原样重载继续运行。 开发者无需在“暂停、停止、启动与恢复”等底层编排状态机上耗费精力,这些均属于 SDK 封装完好的开箱即用能力。工程师真正应该聚焦的核心工作,是“思考应该为智能体引入哪些工具、注入哪些量身定制的 Skills 技能”,其余底层控制完全由 SDK 自动化托管。 --- ### [43:12](https://www.youtube.com/watch?v=tK32trvj_b4&t=2592s) 沙箱的生命周期管理:自动销毁 vs 常驻复用 对于“沙箱是在任务结束后自动销毁更优,还是保持常驻运行更好”的技术疑问,Steve 介绍了 SDK 支持的两种典型模式: - **SDK 托管模式(SDK-owned Sandbox)**:开发者向 SDK 提供沙箱创建配方(Recipe)及运行客户端(例如 Docker 客户端或 E2B 客户端)。SDK 会在任务触发时自动拉起沙箱,并在本轮(turn)执行结束后自动将其销毁。该设计是默认行为,旨在保护开发者,“防止在写完 Demo 后突然发现云端常驻着 100 多个正在计费的闲置沙箱”,造成高昂的账单意外。 - **用户自管模式(User-owned Sandbox)**:针对性能敏感型业务,开发者可以在主流程之外提前预热(eagerly load)拉起沙箱,仅将沙箱引用实例传递给智能体。在此模式下,SDK 不会在回合结束时关停实例,沙箱可以跨多个轮次长期保活(keep warm),免去频繁创建和销毁的冷启动开销。 --- ### [44:31](https://www.youtube.com/watch?v=tK32trvj_b4&t=2671s) 文件系统填充与持久化策略:Copy 与 Mount 的技术权衡 Nish 就如何在沙箱中填充和持久化文件系统进行了深入剖析。Agents SDK 引入了**清单(Manifest)**的核心概念,为文件系统的准备与转储提供了高度的定制空间。目前主流支持两种接入方式,二者在不同性能维度上各有优劣: | 接入方式 | 启动延迟 | 运行时读取速度 | 适用场景与优劣分析 | |:--- |:--- |:--- |:--- | | **全量复制 (Copy)** | 较慢(存在冷启动传输开销) | 极快(本地磁盘直接 IO) | 启动时需将云存储数据整体拉取并解压写入沙箱;但在运行期间模型能够极速读取本地文件。 | | **网络挂载 (Mount)** | 极快(秒级瞬间挂载启动) | 稍慢(受制于网络传输延迟) | 运行时所有文件读取都需要经由网络向存储端发起请求,适合大体量或启动频率极高的任务。 | Nish 总结道,Manifest 赋予了开发者按需选择网络挂载或直接拷贝的自由度,并能在沙箱实例销毁关停时,精准地将变更沉淀回持久化存储层。 --- ### [45:40](https://www.youtube.com/watch?v=tK32trvj_b4&t=2740s) 监控并协调数百个特化智能体的架构可行性 针对“主管智能体(Supervisor Agent)能否同时监控并协调数百个特化子智能体”的压轴提问,Steve 给出了乐观且务实的解答:“答案是肯定的,不过要在当下搭建起来确实需要多花一点工程心思。” 他透露,OpenAI 内部正在全力研发多智能体框架(Multi-Agent Frameworks),相关功能预计将在接下来的数周或数月内陆续推送,届时编排大规模子智能体网络将变得极其直观简易。在现阶段的落地实践中,开发者完全可以大胆放手尝试大规模智能体编排,常见的高效协调模式包括: - **基于消息的协同机制**:智能体之间通过标准消息总线建立解耦通信。 - **基于工具探查的父子协调模式**:为主管智能体(Parent Agent)装配专属工具,使其具备查询当前有哪些活跃子智能体在运行的能力,并支持主动打点轮询(check in)各个子任务的实时进展。 ### [46:27](https://www.youtube.com/watch?v=tK32trvj_b4&t=2787s) 多 Agent 协同与大规模并行机制展望 在探讨多个 Agent 之间如何协作时,技术团队指出完全可以让所有 Agent 通过同一个文件系统或数据库进行通信,开发者可以根据具体架构选用多种不同的通信机制来协调它们的动作。在技术实现上,没有任何障碍会阻止开发者采用这种架构;甚至从长远发展趋势来看,这种“大规模并行工作”(massively parallel work)在不久的将来将会成为构建智能体系统的“默认模式”(default)。通过共享底层存储与状态通道,多个专注于不同子任务的 Agent 能够实现高效并发与信息同步,从而成倍提升复杂工程的执行效率。 ```mermaid flowchart TD accTitle: Multi-Agent Parallel Communication Architecture accDescr: 多个并行 Agent 通过共享文件系统或集中式数据库进行状态协调与通信。 subgraph Agents["并行执行层"] A1["Agent 1 (代码编写)"] A2["Agent 2 (测试验证)"] A3["Agent 3 (文档检索)"] end subgraph Storage["共享协作层"] FS["共享文件系统 (Filesystem)"] DB["集中式数据库 (Database)"] end A1 <--> FS A2 <--> FS A3 <--> FS A1 <--> DB A2 <--> DB A3 <--> DB ``` ### [46:45](https://www.youtube.com/watch?v=tK32trvj_b4&t=2805s) 资源汇总、代码仓库发布与活动结语 随着问答环节告一段落,活动进入尾声。主持人对多 Agent 并行的愿景表示赞同("Love that"),并切换幻灯片集中展示了上手 Agents SDK 所需的全部关键资源。针对直播聊天区中大量观众反复询问的“本次演示示例代码在哪里可以找到”这一高频问题,官方明确指出,所有的演示例程都将收录在屏幕最底端显示的官方 Build Hour 代码仓库(`build-hours`)中。该仓库不仅沉淀了往期所有 Build Hour 的完整工程代码,本场演示构建的任务追踪器(Task Tracker)代码也会在下播后立即推送到 GitHub 上供开发者查阅和复现。 此外,主持人引导大家关注官方主页以获取后续 Build Hour 的最新直播排期,并通过下一张幻灯片再次强化展示了相关链接。在本次活动彻底结束之后,团队还会通过电子邮件向注册参会者发送一份问卷调查,收集社区对于未来主题的期望和建议;邮件中还将附带本场直播的完整录像回放链接,方便中途加入或错过开场的开发者随时补看追赶进度。在致谢全体参会者的热情参与后,本期 Build Hour 正式闭幕。 - **演示代码获取**:本场及往期 Build Hour 的所有演示代码均汇聚在官方 GitHub 代码仓库,直播结束后会以最快速度更新上线。 - **后续活动与回放**:官方主页持续更新未来各期 Build Hours 的日程;会后将统一发送包含视频回放链接和主题调研问卷的邮件,听取社区开发者的反馈。 ## Notable quotes > [01:38](https://www.youtube.com/watch?v=tK32trvj_b4&t=98s) 那条曲线正在持续地“向右上方狂飙”。以 OpenAI 内部的明星产品 Codex 为例,许多开发者日常使用它进行全流程软件构建时,可能习惯了它持续运行几分钟甚至一个小时;但如果启用 Goal(目标)功能,Codex 在内部测试中已经可以连续运行数天乃至整整一周来攻克极具挑战性的工程任务。 — Steve Coffey > [01:38](https://www.youtube.com/watch?v=tK32trvj_b4&t=98s) 以往工程师查证“Agents SDK 目前有多少人在用”或“两天前 Responses API 产生了多少调用量”这类指标时,可能需要绞尽脑汁花上一个小时手写复杂的 SQL 关联查询;如今只需在聊天框输入一句话提示词,几分钟之内就能拿到精准的数据报表。 — Steve Coffey > [04:40](https://www.youtube.com/watch?v=tK32trvj_b4&t=280s) 开发者不再需要把 80% 的精力浪费在手写 请求模型 -> 解析 Tool Call -> 触发执行 -> 拼装上下文重新请求 的冗长编排外壳上,而是可以把注意力 100% 聚焦在业务产品功能的打磨上。 — Steve Coffey > [14:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=848s) 把所有工作全扔给 Agent,自己什么活都不用干了 — Steve Coffey > [18:08](https://www.youtube.com/watch?v=tK32trvj_b4&t=1088s) 从技术上讲,该机制去除了运行时间的绝对上限,让智能体持续工作数小时、数天甚至数周成为可能。 — Steve Coffey > [20:43](https://www.youtube.com/watch?v=tK32trvj_b4&t=1243s) 完全意识不到自己正在一个全新的容器实例上运行 — Steve Coffey > [30:14](https://www.youtube.com/watch?v=tK32trvj_b4&t=1814s) 我绝不会在未经快速审查前,就允许你把某件事直接标记为完成。 — Steve Coffey > [38:15](https://www.youtube.com/watch?v=tK32trvj_b4&t=2295s) Codex Harness 与 OpenAI 模型的结合正变得越来越紧密。在未来,开发者很可能会发现,只有在模型训练时所针对的原生 Harness 中运行,才能激发出模型的最佳性能。 — Steve Coffey > [43:12](https://www.youtube.com/watch?v=tK32trvj_b4&t=2592s) 防止在写完 Demo 后突然发现云端常驻着 100 多个正在计费的闲置沙箱 — Steve Coffey > [45:40](https://www.youtube.com/watch?v=tK32trvj_b4&t=2740s) 答案是肯定的,不过要在当下搭建起来确实需要多花一点工程心思。 — Steve Coffey > [46:27](https://www.youtube.com/watch?v=tK32trvj_b4&t=2787s) 这种“大规模并行工作”(massively parallel work)在不久的将来将会成为构建智能体系统的“默认模式”(default)。 — Steve Coffey / Nish Singaraju ## Mentioned OpenAI, Christine, Steve Coffey, Nish Singaraju, Agents SDK, Responses API, Codex, E2B, Modal, Cloudflare Workers, Vercel, Blaxel, Daytona, Docker, Model Context Protocol (MCP), Skills API, Hosted Shell Tool, Linear, GitHub, Cloudflare R2, Amazon S3, Azure Blob Storage, Temporal, AWS, Rclone, FUSE, Slack ## Video description Build with the next evolution of the Agents SDK. In this Build Hour, you’ll learn how to use the updated Agents SDK to build long-running agents with a model-native harness. Give agents the tools, memory, and execution environment they need to work across files and systems — while staying flexible enough to fit your own stack. In this session, Steve Coffey (Member of Technical Staff) and Nish Singaraju (Product, Agents) will cover how to: • Build agents that can inspect files, run commands, edit code, and operate across multi-step tasks • Use the updated Agents SDK harness for more reliable agent loops across documents, tools, and systems • Work with core agent primitives like MCP, skills, AGENTS.md, shell, and apply patch • Run agents safely in controlled sandbox environments with the files, dependencies, and tools they need 👉 Agents SDK Docs: https://developers.openai.com/api/docs/guides/agents 👉 Typescript SDK: http://github.com/openai/openai-agents-js 👉 Python SDK: http://github.com/openai/openai-agents-python 👉 Agents SDK Quickstart: https://openai.github.io/openai-agents-python/quickstart/ 👉 Follow along with the code repo: http://github.com/openai/build-hours 👉 Sign up for upcoming live Build Hours: https://webinar.openai.com/buildhours 00:00 Introduction 01:48 Agents SDK updates 11:21 Recent releases in the API 14:13 Demo: Building a task tracker with the Agents SDK 37:50 Q&A [Source](https://www.youtube.com/watch?v=tK32trvj_b4)

Source quotes

[1:38] Steve Coffey: 那条曲线正在持续地“向右上方狂飙”。以 OpenAI 内部的明星产品 Codex 为例,许多开发者日常使用它进行全流程软件构建时,可能习惯了它持续运行几分钟甚至一个小时;但如果启用 Goal(目标)功能,Codex 在内部测试中已经可以连续运行数天乃至整整一周来攻克极具挑战性的工程任务。 [1:38] Steve Coffey: 以往工程师查证“Agents SDK 目前有多少人在用”或“两天前 Responses API 产生了多少调用量”这类指标时,可能需要绞尽脑汁花上一个小时手写复杂的 SQL 关联查询;如今只需在聊天框输入一句话提示词,几分钟之内就能拿到精准的数据报表。 [4:40] Steve Coffey: 开发者不再需要把 80% 的精力浪费在手写 请求模型 -> 解析 Tool Call -> 触发执行 -> 拼装上下文重新请求 的冗长编排外壳上,而是可以把注意力 100% 聚焦在业务产品功能的打磨上。 [14:08] Steve Coffey: 把所有工作全扔给 Agent,自己什么活都不用干了 [18:08] Steve Coffey: 从技术上讲,该机制去除了运行时间的绝对上限,让智能体持续工作数小时、数天甚至数周成为可能。 [20:43] Steve Coffey: 完全意识不到自己正在一个全新的容器实例上运行 [30:14] Steve Coffey: 我绝不会在未经快速审查前,就允许你把某件事直接标记为完成。 [38:15] Steve Coffey: Codex Harness 与 OpenAI 模型的结合正变得越来越紧密。在未来,开发者很可能会发现,只有在模型训练时所针对的原生 Harness 中运行,才能激发出模型的最佳性能。 [43:12] Steve Coffey: 防止在写完 Demo 后突然发现云端常驻着 100 多个正在计费的闲置沙箱 [45:40] Steve Coffey: 答案是肯定的,不过要在当下搭建起来确实需要多花一点工程心思。 [46:27] Steve Coffey / Nish Singaraju: 这种“大规模并行工作”(massively parallel work)在不久的将来将会成为构建智能体系统的“默认模式”(default)。

Source quotes

[18:08] Steve Coffey: Compaction allows the model to compact its context and keep going even when it exceeds the context window... this allows the model to keep working for a super long period of time—hours, weeks, technically there's no limit to how long it can work. [37:42] Steve Coffey: The Codex harness with OpenAI models is sort of becoming more and more coupled as time moves on... it's very possible that you will be able to get the best performance from the harnesses that the models are trained with. [46:47] Steve Coffey: massively parallel work will become sort of the default in the near future

Original transcript

[0:00] Hey everyone, welcome to OpenAI Build [0:02] Hours. I'm Christine. I'm on the startup [0:05] marketing team and I'm here with Steve. [0:07] >> Hi everybody. [0:08] >> So you might might remember Steve. [0:10] Welcome back to build hours. [0:12] >> Thank you. It's great to be back [0:13] >> from our last session on responses API. [0:16] Um but today we're going to be talking [0:18] about the agents SDK. [0:20] So if this is your first build hour, uh [0:24] the purpose of build hours is to empower [0:25] you with the best practices, tools, AI [0:28] expertise to help you build with our [0:29] APIs and models. Down below on your [0:32] screen is our homepage. So you can find [0:34] all of our past sessions including [0:36] Steve's one on responses API on demand [0:39] and this will also be on demand after [0:41] our live session. So to give you a [0:44] little snapshot on what we're going to [0:46] be talking about today. So, first we'll [0:49] be going through a lot of the new [0:50] updates on agents SDK. There's now a [0:54] codec style harness um that's really [0:56] improved it and Steve will walk you [0:58] through all of these new releases. Um we [1:01] also have some new releases in the API [1:03] that we'll walk through and then we'll [1:04] move into a live demo. So, this will be [1:06] the majority of the time that we'll be [1:08] spending today. Um and we'll be building [1:09] an agentic uh task tracker and then as [1:12] always we have a lot of time set aside [1:14] for Q&A. So on the right side of your [1:17] screen, you'll see a Q&A tab. So feel [1:19] free to submit your questions. Um we [1:22] have our team who is dialing in [1:24] virtually to answer them during the [1:25] session as well as saving some to answer [1:28] live towards the end. And we'll have a [1:30] very special guest be joining us [1:31] virtually uh to help answer some of [1:33] these. So with that, I will turn it back [1:36] over to Steve to walk through some of [1:37] these slides. [1:38] >> Awesome. Thanks, Christine. Uh so great [1:40] to be back on Build Hour. My name is [1:42] Steve. I'm an engineer on the API team [1:44] here at OpenAI. First thing I want to [1:46] talk about is some of the updates we've [1:48] made to the agents SDK in the last [1:49] couple months. Um kind of uh kind of the [1:53] backstory here is that models are [1:54] getting way better at doing work over [1:57] long periods of time. Uh a lot of folks [1:59] have seen the sort of meter chart that [2:00] shows the ability of models to kind of [2:02] work uh on their own for longer and [2:04] longer periods of time that's just kind [2:05] of going continually up and to the [2:07] right. And there are a lot of great [2:08] examples of this uh even within our own [2:10] products. So many of you know and love [2:12] Codeex, our agent coding tool that you [2:14] can use to kind of build software and do [2:17] tasks from end to end. You've probably [2:19] seen it run for maybe minutes or up to [2:21] an hour. If you've used the goal [2:22] feature, maybe you've seen it run for [2:23] much longer than this internally. Folks [2:25] have gotten Codex to run for days, up to [2:27] a week on tasks. So these models are [2:30] really really able to kind of do things [2:32] over really long trajectories, which is [2:33] really cool. Um we also uh have this [2:36] sort of security agent that's built on [2:38] top of Codeex. um you know the models [2:40] are getting really really good at [2:41] finding security vulnerabilities [2:43] especially in legacy software and so we [2:45] have an agent that's powered by codecs [2:47] that actually scans repos and our our [2:50] own code and code that we depend on for [2:52] vulnerabilities that might affect the [2:54] security of our systems and then we also [2:56] have an internal data agent that's kind [2:59] of powered again by codeex that's [3:01] connected to our data links able to [3:02] answer any questions so I can say [3:04] something like how many people are using [3:06] the agents SDK how many requests were [3:08] there in the responses API two days ago [3:10] and where previously I might have spent [3:12] an hour writing that SQL query, now it's [3:15] just a simple prompt and a few minutes [3:16] later I have an answer which is which is [3:18] really awesome. Uh the reality is though [3:21] that building production grade agents in [3:22] your own systems is is still pretty [3:24] hard. Uh it's pretty hard to kind of [3:27] strike this balance between maximizing [3:29] performance and getting and having sort [3:32] of a really uh flexible maybe cross [3:34] model provider platform. um because it [3:38] more and more the models are kind of [3:39] being tailored to the individual [3:41] harnesses and so it's kind of hard to [3:42] stay in distribution while also [3:43] maximizing that flexibility and that [3:46] customization. Um there's also a lot of [3:49] challenges that come with sort of modern [3:51] computer using agents. So we think of [3:53] codeex as a coding agent uh that can do [3:55] things within a sandbox. Most most of [3:57] the time when you're running codeex that [3:59] sandbox is just your local laptop. So [4:01] those two things are together. We'll [4:02] talk about how we switched that up with [4:03] the agents SDK. But if you're deploying [4:05] agents in production, oftentimes you [4:07] want to be able to run these agents in [4:10] sort of an isolated environment and then [4:12] you have to deal with problems like the [4:14] container dies, it expires, it goes [4:16] away, what do you do? Does that thing [4:18] loadbearing? Did you have state on that [4:20] then rehydrate? Uh this stuff is is [4:22] really really hard. Uh and then kind of [4:24] the third thing is like at the end of [4:26] the day, you want to be able to have [4:27] this sort of like really customizable [4:29] framework you can use to bring your own [4:30] data in and uh maybe add your own skills [4:33] and and all of this stuff like this. And [4:35] so this combination of things is kind of [4:36] makes deploying agents in production a [4:38] little bit challenging. Uh the cool [4:40] thing about the agent SDK, we've added [4:42] we've made some updates to it, but sort [4:44] of the baseline is this open- source [4:45] customizable framework that maybe you've [4:47] used since we released the agent SDK [4:49] last March. It's been around for a [4:51] while, continues to be open source and [4:53] and super flexible. Uh we've added in [4:56] this concept of sandbox using agents and [4:59] you get to decide where these sandboxes [5:01] are run. So if you are you know maybe a [5:03] modal user or an EDB user or a [5:05] cloudflare user and you're kind of [5:06] already using those sandbox products or [5:08] maybe you're new to sandboxes and new to [5:10] sandbox using agents and you want to but [5:12] you're already using one of these [5:13] platforms we have provided first class [5:16] support for using the sandbox products [5:19] in a bunch of these uh different [5:21] platforms like ETB modal cloudflare in [5:23] addition versell uh blackel daya uh and [5:28] just things like docker and uh if you [5:30] just want to use your local laptop for [5:32] testing. Those are kind of all supported [5:33] first class. And then on top of this is [5:35] this Codex style harness. So bringing in [5:37] all of the great stuff that makes CEX [5:39] Codex, all the great tools that are in [5:40] distribution with our models, but work [5:42] great with other models as well. And [5:44] then some of the cool things like skill [5:46] use, uh, computer use, memory, and we'll [5:49] be demoing some of this stuff here in a [5:50] little bit. So uh, yeah. So basically [5:54] what the agents SDK is is it provides [5:56] this sort of model native harness that [5:58] the that the agent can use in order to [6:00] kind of do productive work over long [6:01] periods of time. So if we think about [6:03] maybe what building an agent might have [6:04] looked like you know a year a year and a [6:06] half ago where you're just kind of using [6:07] an API directly an LLM API directly you [6:10] would have done something like you know [6:12] you set up a loop you make a request to [6:14] an LLM API you get some stuff back maybe [6:17] you get a tool you execute that tool you [6:19] update the context and then you kind of [6:21] repeat until the model has no is no [6:23] longer calling any tools you might be [6:25] layering in stuff in addition to that [6:26] like web search file search MCP kind of [6:29] like building spending most of your time [6:30] building this orchestration layer around [6:32] this loop where actually you should be [6:34] spending most of the time building [6:35] product building stuff into your product [6:37] that that makes it it much better for [6:39] your end users and so agents SDK is kind [6:41] of designed to help you do all that [6:43] stuff and not really think about the [6:45] orchestration. So the agent loop is [6:47] built in again very CEX-like in its uh [6:49] design. Uh also web search, file search, [6:52] MCPS, code interpreter, skills, these [6:54] are things are all you get out of the [6:56] box or really easy to turn on. And then [6:58] we've kind of added in some of this like [6:59] sandbox using functionality that I'll [7:02] talk about uh pretty shortly. The thing [7:04] that I'm most excited about that we've [7:06] kind of done here is we've split the [7:08] harness from the compute. And uh what [7:12] does that mean basically? So if you can [7:14] kind of imagine a world where the model [7:17] is working sort of directly on a uh [7:20] computer that it itself is running on. [7:22] So if you can kind of imagine like [7:23] codecs running on your laptop is the [7:25] sort of example where the harness and [7:26] the compute are tied together. Uh [7:28] imagine putting that into production [7:30] right so you have sort of a container [7:32] where the agent is both running. So like [7:34] the the loop that is calling your LLM [7:37] API in addition to working on the same [7:38] file system those things are together. [7:40] This kind of creates a few problems when [7:42] you run this in production at scale. A [7:44] your sandboxes become all of the sudden [7:47] loadbearing and if one dies or goes away [7:49] then all of a sudden all that state is [7:51] gone and you don't have some sort of [7:52] external place where maybe you can [7:54] refresh from. In addition, you have to [7:57] sort of do interesting uh sort of [8:00] gymnastics to manage your secrets. You [8:02] ideally don't want to have any secrets [8:03] on your sandbox otherwise you might be [8:05] vulnerable to prompt injection attacks [8:08] or exfiltration. And so if you kind of [8:10] split those things up, then you can [8:11] treat the sandbox as this totally [8:13] ephemeral thing that you don't really [8:15] have to worry if it lives or dies. It [8:17] can expire, go away. And then the uh [8:19] harness, which is running maybe in a [8:21] temporal job or in AWS or you know kind [8:23] of wherever your infrastructure is [8:24] already running can sort of handle the [8:26] rehydration and snapshotting and all [8:29] this stuff. And the agents SDK is going [8:30] to make that really easy. And I'll kind [8:32] of show some examples of this. Um, so [8:36] agent SDK kind of like already fully [8:37] packed with a ton of features that you [8:40] might have already been using things [8:41] like web search, file search, uh, you [8:44] know, agent memory, text in addition to [8:46] some modalities that you've probably [8:48] been using, uh, text and voice. Um, [8:50] there are some new things that are we're [8:52] pretty excited about that we've added. [8:53] So this is kind of first class support [8:54] for skills. Uh, containers is kind of a [8:57] new thing sort of in the API, also in [8:59] the SDK. And then we've also added agent [9:01] memory so your tasks can improve over [9:03] time um and get better as they as they [9:06] go. So some of the kind of core features [9:09] here again uh code a really you know [9:12] codeex inspired harness a lot of those [9:14] same tools sandbox use so your models [9:16] can work with files in sort of a [9:18] virtualized environment you know do work [9:20] over long periods of time write bash [9:22] write python write whatever to kind of [9:24] accomplish tasks and then as always open [9:27] source super customizable [9:29] socy harness what kind of concretely [9:32] does that mean um the kind of things [9:34] we've brought from codeex into the [9:36] agency SDK are these things like uh [9:39] autoco compaction uh especially you know [9:42] this computer use through through uh the [9:44] shell. So the model is able to write [9:46] shell commands. Um we've you know the [9:49] codec style loop is this sort of async [9:51] shell interaction loop where it can [9:52] write a command maybe wait a little bit [9:54] if it doesn't finish it can go do [9:56] something else and then kind of come [9:57] back to those things and it kind of [9:58] keeps track of what commands it has [10:00] running at any given time. And so we've [10:02] kind of brought that into the agent SDK [10:03] to allow you know coding agents to work [10:06] kind of in any domain in any [10:08] infrastructure. And then sandboxes are [10:10] super crucial because you know modern [10:12] work means working over a lot of files. [10:15] If you think about kind of the work that [10:16] you and I do every day maybe you're [10:17] working in a big codebase with a bunch [10:19] of files editing files and and making [10:21] your changes. Maybe you're working with [10:22] a bunch of PDFs. Maybe you are you know [10:25] creating word documents or powerpoints. [10:27] a agents are no different and they need [10:28] access to those files and a kind of a a [10:30] a place to create those files and store [10:32] them. And so that's kind of where [10:33] sandboxes come into the picture. It's an [10:35] isolated environment where the model has [10:37] access to some files and can do stuff [10:39] and and produce uh meaningful output. [10:42] And uh kind of again agent SDK open [10:45] source and customizable. It's been open [10:47] source since day one. Super, you know, [10:50] uh model agnostic. If you want to use [10:52] other model providers, you're more than [10:53] welcome to. Anything that kind of uses [10:55] the responses API format will plug in [10:56] really well. Uh it you can make this [10:58] multi-tenant super easily. So if you [11:00] want to have a system that's running the [11:01] agents SDK where you're kind of [11:03] processing many requests from users at [11:05] once again super easy. Made session [11:07] management really simple. So agents SDK [11:09] is going to manage the snapshotting uh [11:13] rehydration of tasks. You can kind of [11:14] start from where you left off. All that [11:15] stuff is really simple. Kind of show a [11:17] demo of that. And then really [11:19] configurable. You can bring your own [11:20] tools, bring your own MCPs and and kind [11:22] of all that stuff. So I want to switch [11:24] gears a little bit and talk about some [11:26] other cool things that we've shipped in [11:27] the API in the last couple months. Uh [11:30] first one is this hosted shell tool in [11:32] the responses API. So idea here is I [11:35] want to make a just a quick API call. I [11:38] want to provide some files there that I [11:40] want the model to do something with. Um [11:42] and behind the scenes we're going to [11:43] spin up a container, load that up with [11:45] the files that you've asked us to. The [11:46] model can write code, do some stuff in [11:48] that container and then return the [11:50] results back to you. So think about this [11:52] as like a lightweight version of the [11:53] agents SDK. It's kind of this ephemeral [11:55] container. You can put a few files in [11:57] there. The model can work in that [11:58] container and then it can finish and you [12:00] can you know create these out of band. [12:02] We have a containers endpoint where you [12:04] can go and create a container, upload [12:05] files to it, spin it up, attach it to a [12:08] responses API request or you can kind of [12:10] use it in auto mode where you just add [12:12] some files and then we'll spin up the [12:14] container for you and then spin it back [12:15] down when you're done. Really easy way [12:17] to kind of get started with this, you [12:18] know, sandbox using paradigm. [12:20] We've also added the ability for you to [12:24] kind of control network access in the [12:26] containers you spin up. So if you want [12:27] to do things like domain allow lists, [12:29] lock it down entirely so you have no [12:31] egress or ingress into that container. [12:33] Uh these are kind of like all things [12:35] we've added so you have a little bit [12:36] more security control over your [12:37] containers. And we've also added the [12:40] skills API. So if anybody has used [12:42] skills in the past, there's kind of like [12:44] a pretty common problem that comes into [12:46] play which is that you kind of need a [12:47] central source of truth for these [12:49] skills. whether that's maybe a GitHub or [12:51] a bucket or now we have a skills API [12:54] where you can kind of upload your skills [12:55] too. Uh and a skill is kind of this [12:58] bundle for for those who haven't used [12:59] it. It's kind of a bundle of files that [13:01] has sort of one skill MD file that has [13:03] all the top level stuff in it, but they [13:04] can also contain scripts or other [13:06] resources that the model can use to kind [13:08] of do a pretty specific task. So, the [13:11] example I always use is like maybe you [13:12] have a tax prep skill that kind of [13:14] defines like, okay, here's all the [13:15] things that I need to know as an agent [13:17] in order to do somebody's taxes for like [13:19] tax year 2025. Has all the like IRS [13:22] rules and maybe some scripts to like [13:23] process documents and like fill in a [13:25] 1040, something like that. Um, in the [13:27] with the skills API now, you could [13:28] upload that, you can iterate on it over [13:30] time and create versions. You can set up [13:31] default version and then reference those [13:33] versions with the hosted shell and all [13:35] that stuff works together super [13:36] seamlessly. [13:39] Uh yeah, so kind of a couple curl [13:42] snippets here of how you might use this [13:43] thing. Pretty easy. If you already have [13:45] zip files of your skills, you can just [13:46] kind of upload them directly to the API. [13:50] And then the last thing is we've also [13:52] shipped uh the sandbox agent stuff that [13:55] we will be talking about today in [13:57] Typescript. This is a big ask from a lot [13:58] of folks after we ship the Python [13:59] version in April. Uh sand the TypeScript [14:02] version is now available. So if you are [14:03] a TypeScript user, uh knock yourself [14:06] out. So, uh, want to switch gears a [14:08] little bit and kind of go over to the [14:09] demo. Uh, we're going to be kind of like [14:12] building a task tracker with the agents [14:13] SDK and kind of making it aic and then [14:15] offloading all the work so I don't have [14:17] to do anything anymore. [14:19] So, we're going to [14:24] Oops. [14:31] Flip over here. And uh I kind of put [14:33] this kind of like vibe coded this uh [14:37] kind of like sort of like a linear [14:39] board. Doesn't really do anything right [14:40] now, but I can kind of create tasks and [14:43] uh assign them to folks. So the sort of [14:45] idea here is that we are going to be [14:48] doing some conference planning. So you [14:50] can kind of imagine this is a tool that [14:51] I might use as a conference planner to [14:53] put together some talks. So I might have [14:56] tasks for you know creating program [14:59] assets or refining somebody's talk or [15:03] you know uploading stuff to the website [15:04] things like that. Um my goal here is not [15:06] to do any work myself. So we're going to [15:08] kind of automate a lot of this with the [15:09] agents SDK. So I'm going to create a [15:11] fake task here and I'm going to say uh [15:13] you know create program assets and we'll [15:18] say we can leave this blank and we have [15:20] a couple of things down here. where we [15:22] have we can have a assignment. Uh I need [15:25] to refresh this. [15:30] Sorry guys, I need to reset my demo. [15:34] Cool. [15:36] Okay, there we go. So I can uh I can [15:39] create a blank task here that's we can [15:41] say create program [15:44] assets [15:46] and uh I can assign this. Right now it's [15:48] Right now it's just me, but just got a [15:50] sneak peek at what might be coming in a [15:51] few minutes. Um, and I can upload files [15:54] to this. So, if I just go ahead and grab [15:57] my fake files here, [16:01] can drag these in. I can create an [16:04] issue. Uh, and in a pre-agentic world, I [16:07] would assign this to somebody. Maybe I'd [16:09] assign it to myself and I would I would [16:10] do this work, but that's not really that [16:12] much fun. So, we're going to go ahead [16:13] and create an agent to kind of help us [16:14] automate this. So, if I hop back here, [16:21] uh, we are just going to comment in this [16:25] agent definition. So, I want to take a [16:27] sec to kind of walk through what we're [16:28] looking at here. So, if you've been an [16:30] agent SDK user in the past, this will [16:32] look pretty familiar to you. Uh, the [16:34] sandbox agent is a new type of agent. [16:36] Uh, it actually is just a subclass of [16:38] the agent class that you might have been [16:40] using before. So, a lot of the same [16:42] things will still apply. A lot of the [16:44] same parameters are still going to be [16:45] there. We are having we have a couple [16:48] new things that we'll kind of talk about [16:49] in a sec, but suffice to say that we are [16:52] adding some instructions that are kind [16:53] of defined up here. Uh, and then we'll [16:55] be adding some other stuff later. So, if [16:57] I go back to my demo, refresh, [17:01] should be able to assign this to the [17:03] program editor agent. And what I'll say [17:05] is like, please take a look at these [17:08] files and uh, edit them for clarity. [17:14] I'll hit send [17:16] and then we should be able to see that [17:18] kickoff. So the first thing that's going [17:19] to happen is uh because I've configured [17:22] this first agent to use docker as its [17:25] sandbox can actually see that docker [17:27] container running and uh the agents SDK [17:30] is going to handle uploading these files [17:32] to that container. So if I pop this [17:34] might be a little hard to see but uh we [17:37] can see that a lot of these files have [17:39] now made it into the container. So the [17:41] agent is going to be sandboxed to just [17:43] this folder and we'll have access to all [17:44] of these files that it can then use to [17:47] look at. We can as we kind of look over [17:49] here, we can see that it's executing [17:50] commands. It's kind of like looking [17:52] through the files and figuring out [17:54] what's what. And then it's going to kind [17:55] of produce some output. And so it just [17:57] finished and it gave us it just kind of [18:00] gave us a little log of of what it did [18:02] and kind of told us that it's missing [18:04] some data here. That's kind of okay for [18:05] now. So, [18:08] uh, this is great, but we kind of want [18:09] to be able to customize this agent and, [18:12] uh, kind of give it some like really [18:13] specific context that is pretty [18:16] particular to what it needs to do. And a [18:18] really good way to do that is through [18:19] skills. So, prior to this demo, I kind [18:22] of put together this skill that's for a [18:25] conference, the conference program [18:26] editor. It's got some stuff in here. [18:28] Again, this is kind of entirely vibe [18:29] coded, so the actual content doesn't [18:31] matter. But what I want to point out is [18:32] that GitHub is actually a really great [18:34] place to store skills and kind of [18:35] addition to the skills API. The reason [18:37] GitHub is so great is because you can [18:40] have a lot of this a lot of these uh [18:42] kind of concerns of producing and [18:44] editing skills are kind of taken care of [18:46] for you. So you know if you want to have [18:49] version control over your skills, if you [18:50] want to have pull requests so that you [18:52] can review changes to skills, this is [18:54] all really great and a pretty common way [18:56] that people are starting to create [18:57] repositories of their skills is kind of [18:59] in Git. And we have first class support [19:01] for that in the agent SDK. So if I come [19:03] back here and I comment this stuff back [19:06] in [19:08] and hop down, we can see that we have [19:10] this uh skills capability. And I want to [19:13] pause for a sec to kind of talk about [19:14] what a capability is in the new SDK. [19:17] Capability is kind of uh this object [19:20] that is able to combine a bunch of [19:22] different concepts all into one. So if [19:24] you want to bring in uh tools, if you [19:27] want to bring in additional [19:27] instructions, maybe add some stuff to [19:29] the manifest so that a capability can [19:31] put things into the computer when it [19:33] starts up. These are kind of all things [19:35] capability can do. The sort of default [19:37] set is file system. So this is the [19:39] ability for the model to view images and [19:42] run the apply patch tool. So kind of [19:43] like edit files in line. And then we [19:45] also have the shell tool which is [19:47] actually a combination of of two tools. [19:48] This is as I mentioned earlier exact [19:50] command and write standard in. This is [19:52] this async bash command flow. And then [19:55] we have compaction which is allows the [19:57] model to compact its context and keep [19:59] going even when it exceeds the context [20:01] window. And this allows the model to [20:02] keep working for a super long period of [20:05] time, hours, weeks, technically uh [20:07] there's no limit to to how long it can [20:09] work. So these this is kind of the [20:11] default set. We're also adding the [20:12] skills capability. And then what we're [20:14] saying here is that we want to load our [20:15] skills from this git repo. And so I'm [20:17] pointing it at this git repo that I just [20:20] created. Uh we're saying we want to pull [20:22] from main. If you're not using GitHub, [20:24] you can actually change the host here, [20:25] but uh it's we're kind of like using [20:27] GitHub by default. So I'll go ahead and [20:29] save this. We'll flip back to our demo. [20:32] I'm going to ask the question, tell me [20:35] the most [20:37] interesting [20:39] thing from your skill file. [20:43] Will it send? This is going to kind of [20:44] spin back up, but I want to take a sec [20:45] now to actually talk about how we do [20:47] resume uh like pausing and resuming [20:50] behavior between sessions at the agents [20:52] SDK. When you start a sandbox agent, the [20:54] agents SDK will handle spinning up that [20:56] sandbox for you based on how you define [20:58] it. So, if I hop back to the code here, [21:02] we can go and say for by default, we're [21:04] using Docker. And so, I'm kind of just [21:06] providing these two classes here to kind [21:08] of define where I want to run the agent. [21:09] In this case, I'm saying I want to use [21:10] the Docker sandbox client. the and I can [21:13] provide some options. Different backends [21:15] might have different options. Docker, I [21:17] can provide an image. In this case, it's [21:19] just uh the Python 312 image. And uh you [21:23] can kind of easily swap this out for any [21:25] of the backends that that I mentioned [21:26] before. And we'll kind of show an [21:27] example of that in a little bit. Um but [21:29] what's cool is that when that when this [21:31] task spins down, when it stops, the [21:33] agent will automatically stop that [21:35] container and then we'll take a snapshot [21:36] of the file system and put it somewhere [21:38] you define. Right now that place is just [21:41] my laptop. So if I go over here [21:45] uh we can see some of the snapshots. So [21:48] this is kind of the a couple tasks that [21:50] I recently created. And if I untar one [21:52] of these and look, it's basically just [21:55] the exact file system that the model was [21:57] using. So when I resume that task, it is [22:01] grabbing that tarball from the place [22:02] that I defined, spinning up a new [22:04] container if the last one doesn't exist [22:06] anymore and then rehydrating that file [22:08] system onto that new container so that [22:11] it to the model looks exactly the same [22:13] as it did when it left off and the model [22:15] is actually unaware of the fact that [22:16] it's running on a new container is [22:18] totally oblivious to that. And this [22:20] makes resuming really easily easy and [22:21] seamless. And if you don't want to use [22:23] your local file system or tarballs, we [22:25] have sort of provider specific ways that [22:28] you can do this. So if you're a modal [22:29] customer and you use volumes or their [22:31] snapshot concept, this stuff plugs right [22:33] in super easily. But kind of by default, [22:36] we're using this sort of like naive [22:37] tarball approach to kind of help make [22:39] this easy across any provider. So uh we [22:42] can kind of go back and if I refresh [22:46] should be done. And it was able to kind [22:48] of like look at the skill file. If I [22:50] look at the output here, we can see that [22:53] it probably was kind of graping over [22:55] this skill file. Uh, it did quite a bit [22:59] quite a bit of stuff. So, um, but it was [23:01] able to kind of give me some output [23:02] here, kind of, uh, telling me what was [23:05] the most interesting thing here. So, now [23:08] we have an agent. It can use a skill. We [23:10] want to kind of hoist this thing up into [23:12] the cloud. We don't want to be running [23:13] this in Docker forever. Docker's great [23:14] for testing, especially on your laptop, [23:16] because just kind of works out of the [23:17] box. It uh is a really great place to [23:21] kind of do local development, but it's a [23:23] bit hard to deploy in production. [23:24] There's a bunch of companies that have [23:26] built really great first class agent [23:28] sandbox products that you can use that [23:30] we've added first class support for. In [23:32] this demo, I'm going to be using modal, [23:34] but you could also be using Cloudflare, [23:35] ETB, Verscell, Daytona, Blackel, uh or [23:39] you know any of the other you can kind [23:41] of bring your own implementation if if [23:42] you choose to. So I've already set up a [23:45] uh modal kind of app here for us. So I'm [23:49] going to go back to my [23:52] uh code here and we'll go to step three [23:57] and we will enable modal and here we're [24:00] kind of returning this sandbox provider. [24:02] Um but the kind of the key thing here is [24:04] just this client and options which are [24:05] the things that the the agents SDK [24:07] expects and here we're just kind of [24:09] providing this modal sandbox client. If [24:11] I go look at the definition of this [24:13] thing, uh this just inherits from base [24:15] sandbox client just like the docker [24:16] version and we are saying here that uh [24:20] we are going to use the same image. So [24:22] that same Python 312 image and then uh [24:26] in addition we're going to be using R2 [24:28] for snapshots. So that both our [24:29] snapshots are now off my laptop and into [24:31] the cloud. Additionally, the modal [24:34] sandbox is going to be moving into the [24:35] cloud as well. So we'll kind of show an [24:37] example of what this looks like. So save [24:38] all this. Go back to my [24:42] here and we'll say uh edit these files [24:46] for clarity. [24:49] Go back, [24:59] load this, [25:02] and we'll assign it to our program [25:05] editor and we'll say get started, [25:08] please. [25:10] And so now we're starting it. We can see [25:12] that we're starting a modal sandbox [25:13] instead. And if I hop over to modal, I [25:16] can see that now we have a sandbox uh [25:18] that's starting. Zoom in a little bit if [25:20] that's hard to see. And this will take [25:23] uh usually they start up in just a few [25:25] seconds or around a second. So now it's [25:27] running. What's happening behind the [25:28] scenes is the agent SDK is handling sort [25:31] of the applying all of the files that we [25:34] have defined onto that sandbox. And so [25:36] want to take a sec now to actually go [25:37] and look at this manifest object that we [25:40] talked about. And a manifest is is [25:42] pretty simple in concept. You're [25:43] basically saying this is what I want the [25:44] file system to look like when the agent [25:46] spins up. Here's how I kind of attach [25:48] files to this task in a way that I can [25:50] share across tasks or share across [25:51] agents. So manifest is just kind of a [25:54] simple class. Um we're saying the [25:56] entries are this tree here basically [25:58] kind of defining a directory structure [26:00] and this this is a pretty simple [26:01] example. So we're just saying we want to [26:03] kind of establish this base directory [26:04] structure and then we'll upload files to [26:06] the sandbox directly. But you could also [26:08] here plug in things like files if you [26:10] want to copy things from, you know, [26:11] wherever the harness is running. Or if [26:14] you want to plug in an R2 bucket, S3 [26:16] bucket, an Azure blob store account, all [26:18] these things are fair game, a GitHub [26:20] repo, kind of a whole flexible set of [26:22] primitives here that allow you to put [26:24] the files on the container that the [26:26] model needs in order to kind of get [26:27] started. And what happens is that we [26:29] render a version of the file tree to the [26:31] model when it starts up so that it can [26:33] kind of see an example of its workspace. [26:36] Uh so that it doesn't really have to do [26:37] too much grepping or lsing to understand [26:39] what's on the file system. And so you [26:40] can kind of add descriptions to these [26:41] things too, but we won't get too far [26:43] into that. Um so now if I go back to the [26:49] demo, we can see that this finished. Uh [26:52] and what I want to show here is that now [26:55] if I go to my Cloudflare account and [26:57] refresh, we can see that we now have [27:00] snapshots that are stored in the cloud. [27:01] So if I want to resume this task and I [27:06] dep have this app deployed in [27:07] production, it's really easy for the the [27:11] agent to just kind of like pull its [27:12] context for the agent to pull its [27:14] context and the snapshot from R2 instead [27:16] of you know anywhere else. Uh it just [27:18] provides like a fast pretty clear [27:20] interface for this and so just want to [27:22] give a quick demo of that. So now uh we [27:25] kind of want to we have this agent it's [27:27] deployed in the cloud. It's pulling its [27:29] snapshots from another cloud provider [27:31] and we want to go ahead and add our own [27:32] tools into it because you know bringing [27:34] our own context and our own [27:35] functionality into agents is is pretty [27:37] critical. So if I go back to my program [27:39] editor agent, I'm going to comment this [27:41] back in [27:44] and we have some predefined tools that [27:46] we built sort of before this demo. [27:49] There's there's three of them. We have [27:50] one for updating the status of a task. [27:52] We have one for updating the assenee of [27:54] a task. And then we have one for [27:56] searching all of the assignees. So, uh, [27:58] if you've used the agents SDK before, [28:00] function tools work exactly the same way [28:02] as they always have. They're essentially [28:03] just Python functions or in TypeScript, [28:05] TypeScript functions, and you decorate [28:07] them using this function tool decorator. [28:10] And what this does is it allows us to [28:13] translate this function call into sort [28:15] of the API representation of the [28:17] function tool with all the parameters [28:19] uh, clearly defined so the model knows [28:21] what to call. And then when the agents [28:23] SDK receives from the API an instruction [28:25] to call a specific tool, it routes it [28:27] automatically to this function, calls [28:29] this function and then sends the [28:30] response right back. So you don't have [28:32] to sort of like be implementing this [28:34] function calling loop yourself. Uh and [28:36] so we will uh just show a quick demo of [28:40] this. I'll go back here and I'll say um [28:45] please assign this [28:50] task to Steve and mark it as ready for [28:55] review. [28:57] Let's send. While this is spinning up, [28:59] I'll kind of cover some of the other [29:00] cool things you can do with the function [29:02] tool. Um kind of a bunch of things you [29:04] can do here. So if you want to kind of [29:06] override the name, like what the model [29:07] sees is different from what the name of [29:09] the function is, you can do that. You [29:10] can provide a description to kind of [29:11] help steer the model to how to use this [29:13] tool. You can provide guard rails around [29:17] uh whether you know if the model's call [29:19] trying to call the tool with specific [29:20] arguments. You can add hooks around that [29:22] to kind of you know control the [29:24] execution flow. You can add timeouts and [29:26] things like this which is which is [29:27] pretty cool. Um and you can kind of like [29:30] set whether it's enabled kind of [29:32] dynamically maybe based on a feature [29:33] flag or something else. Uh so you don't [29:35] have to like comment and comment out [29:36] these these lists of tools which is [29:37] which is pretty cool. So go back to our [29:40] demo here [29:43] and we should see it uh assign the task [29:48] to me which is cool. So we can kind of [29:49] see like using these tools and this is a [29:51] great way to sort of bring your own [29:53] context or your own functionality from [29:55] your app into your agent whether maybe [29:57] you have want to give it access to your [29:59] database or want to have it be able to [30:02] you know uh use Slack through an MCP or [30:04] something like this. These are kind of [30:05] like great ways to bring in your own [30:07] functionality into the SDK and into your [30:09] agent. So, uh, now we are going to talk [30:14] a little bit about like tool call [30:15] approvals. So, ideally, um, there are [30:19] certain tools that maybe kind of require [30:21] a human of the loop step in order for [30:22] you to have confidence that it's kind of [30:24] doing what you ask and not going off the [30:26] rails. If you have been a Codex user in [30:28] the past, you're pretty familiar with [30:29] this. It tends to ask you a lot, do you [30:30] want me to run this bash command or am I [30:32] allowed to access the internet? This [30:34] sort of thing. You can do this in the [30:35] agent SDK pretty easily by just [30:37] providing a function. Uh you can I can [30:40] either say you know true directly or I [30:43] can provide a function that returns true [30:45] or false dynamically based on the [30:47] parameters. And so in this example we've [30:49] said if you're trying to set the status [30:51] to done then it needs my approval first. [30:54] I'm not going to let you mark something [30:55] as done without taking a quick look at [30:57] it first. And so if I refresh this and [31:00] go back and we'll go back to this other [31:01] task we have and I'll say please mark [31:04] this task as done. [31:07] We'll send that and this is going to [31:08] spin up a sandbox rehydrate the state. [31:11] Uh it will kind of like examine its [31:13] current state and then it should call [31:14] the tool and then we should kind of see [31:16] a tool call approval um pop over and [31:19] over here. Uh while this is working [31:22] another thing we can kind of show is the [31:23] ability for the agents to kind of hand [31:25] off to each other. So let's say I want [31:28] to, you know, I kind of have this [31:30] built-in mechanism here where we have [31:32] the ability to search and update the [31:33] assigne. And this is great for agents to [31:35] kind of work together. So I have two [31:37] agents now in my system. I have one for [31:39] producing assets. Um, and then I have [31:41] one for just like the program editor [31:43] agent that we looked at earlier. And so [31:45] I can go ahead and create an issue new [31:46] issue here and say like uh please [31:50] um [31:52] refine and uh [31:56] this content [31:59] and produce some assets. We'll hit uh [32:03] drop some files in our same set and then [32:06] we'll assign this to the program editor [32:09] agent and we'll say like please uh [32:12] refine this content [32:15] and then assign to the what's the other [32:20] one called asset producer agent [32:26] and we'll create that. Then we'll kick [32:28] this task off. the program editor agent [32:30] will be the first one to pick it up. It [32:31] will kind of do its thing and then it [32:33] will uh reassign this to the asset [32:36] producer agent and then that agent will [32:38] kind of kick off and then you know be [32:40] able to create some assets and then put [32:42] them back in the sandbox. And so kind of [32:44] coming back to my uh previous task here [32:48] it looks like this marked as done. If I [32:50] go to activity here I can see that we [32:52] now have this request uh for approval. [32:56] And if I go ahead and approve this, then [32:57] this will kind of kick the flow back off [33:00] and then it will eventually kind of get [33:01] marked as as done here. [33:04] So, uh let's see. I think the last thing [33:08] I want to show here is sort of the [33:09] ability to [33:11] to mount uh external buckets to your [33:14] manifests in a way that uh allows you to [33:18] kind of bring in external data. So, what [33:20] are the reasons you might want to do [33:21] this? Let's say you have either a ton of [33:23] files, let's say you have like hundreds [33:24] of PDFs that you need the model to be [33:26] able to work through and it's kind of [33:28] annoying to just, you know, have to copy [33:29] all of those to a sandbox every time you [33:32] want to spin it up and have it do some [33:34] work. Or let's say you have data with a [33:36] a strong freshness constraint, so the [33:38] data is changing all the time and you [33:40] want the agents to be able to work over [33:41] data that is like super live. In either [33:43] one of these cases, mounting data from a [33:45] centralized source of truth instead of [33:47] copying it in where it might already be [33:48] out of date by the time the agent starts [33:50] working is pretty critical. And we've [33:52] built that natively into the agents SDK. [33:55] So if I go to [33:58] uh [34:03] if I go here, what I'm going to do is [34:05] kind of comment this in. And we've built [34:06] this uh R2 attachment store. And so this [34:08] is kind of doing a couple things. This [34:10] is a taking our uploaded files and then [34:13] instead of just putting them on my [34:15] laptop and then copying them to the [34:17] container, we're actually now uploading [34:19] them straight to R2. So that's going to [34:21] be our source of truth just for our [34:22] application layer. And then back in the [34:26] manifest, oops, [34:32] uh we're saying if we are using this R2 [34:33] attachment store, then we're actually [34:34] going to use a different manifest here. [34:36] And we have a similar layout. So we have [34:39] task and then these four folders. But [34:40] instead of the input being an empty [34:43] directory that we then copy files into, [34:45] we're instead saying I want to use this [34:46] S3 mount type. And we're technically [34:49] using an R2 bucket, but we're using the [34:50] S3 mount. The reason is is because R2 [34:52] and S3 are API compatible. So we have [34:54] one object for them both. But as we look [34:56] through the parameters here, we can see [34:58] we're using this R2 bucket name passing [35:00] passing we're passing the access key and [35:03] secret key. And then we're using this uh [35:05] modal cloud bucket strategy. A lot of [35:08] these sandbox providers, modal included, [35:09] have native ways to mount external [35:12] volumes to their sandboxes. And so we're [35:14] using the sort of built-in one for [35:15] modal. But if you're using a Docker [35:17] container or sort of any generic [35:19] container, we provide a couple out of [35:20] the box. You can either use our clone or [35:23] fuse. And these both work equally well. [35:24] And it will depend a little bit on your [35:26] systems which one will work better for [35:27] you, but they're both provided out of [35:29] the box. So pretty easy to just plug [35:31] this stuff in. And so now uh I will go [35:35] and create another issue here. We'll say [35:39] edit these assets again please. And [35:43] we'll grab our files. [35:46] Drop these in. [35:48] Hit create. [35:54] And [35:56] we should see so this has been I didn't [35:59] assign this but we'll sign in a sec. Uh [36:01] what I want to show here though is now [36:02] we have if we go back to our bucket we [36:05] now have tasks uh and we have a a [36:08] reference to our task here that has all [36:10] the files in it. So now if I go back and [36:14] then assign it to our program editor, [36:16] program editor and say get started, [36:19] hit send, it's going to be able to we'll [36:22] watch this thing spin up and then it [36:24] will be able to read and write to this [36:26] volume just like it would if it were on [36:28] the file system local to the container. [36:31] So you can kind of bring in these places [36:33] of context of you know file storage from [36:36] kind of anywhere across the internet and [36:37] the model will be able to access it as [36:39] if it were on the local file system. [36:40] It's completely opaque to the model that [36:42] it's actually over operating over the [36:44] network. So there are some trade-offs [36:46] here. You might incur some additional [36:47] latency as the model is kind of like [36:49] lsing maybe a large bucket that takes [36:51] you know a long time to list all the [36:52] files through. Uh, but there's sort of [36:54] this, you know, really nice trade-off [36:56] that you get where you get to, you know, [36:58] maintain your freshness constraint or [36:59] have it work over a ton of files without [37:01] having to sort of like copy all them all [37:03] onto the container where you might be [37:05] sort of like limited in space. Um, so [37:08] cool. We can wait a sec for it to see [37:10] the spin up, but that kind of brings us [37:12] to the end of the demo here. [37:15] Uh, you know, we kind of covered a bunch [37:18] of cool things. Uh we showed how we can [37:22] build agents that both inspect files, [37:24] spin up sandboxes, and kind of any [37:25] environment that you choose, maybe one [37:27] that you're already using. Uh agents [37:29] that can edit code and kind of like work [37:31] on these long horizon tasks for maybe [37:34] minutes or hours or longer uh over, you [37:37] know, whatever period of time that you [37:39] want and whatever context you want. So [37:41] yeah, with that, I think we'll do some [37:42] questions. [37:43] >> Yeah. [37:44] >> Cool. [37:45] >> Okay. Um next slide. I'm really excited [37:50] to welcome Nish on stage with us. So [37:52] Nish is product for agents. Nish, do you [37:56] want to give a quick intro? [38:00] >> Hey guys. Uh yes. So product manager on [38:02] agents. Uh really excited about the [38:04] stuff we're doing with agents SDK. I've [38:07] been answering some of the questions in [38:08] the channel. Really great questions and [38:10] uh yeah, excited to get into it. [38:12] >> Awesome. Okay. Um next slide here. [38:15] Perfect. Um, so the first question, when [38:17] are harnessed based agents best versus [38:20] building with just responses API? [38:22] >> Cool. Yeah. So I think that if you the [38:25] responses API is great for a huge [38:27] variety of tasks, you know, I think the [38:29] places where it really shines is and [38:31] it's an agentic model by default and or [38:34] agentic API by default and we kind of [38:35] talked about that in the responses API [38:37] build hour from October. um it really [38:39] shines I think if you are using if maybe [38:42] you're using a language where you can't [38:43] find a really good uh sort of agentic [38:46] framework that you like. So maybe you're [38:47] building an elixir, maybe you're [38:49] building in HASLL or something a little [38:51] bit more say esoteric, you can always [38:53] just build on the API. You can kind of [38:55] recreate a lot of these concepts [38:56] yourself. It might be a little bit more [38:57] work. Um the you know sort of like [38:58] harnesses are becoming uh sort of really [39:01] key to how the models operate. So in [39:03] particular, the codeex harness with [39:05] openAI models is sort of like becoming [39:07] more and more coupled as time moves on. [39:08] And so you know in the future, it's very [39:10] very possible that you will be able to [39:12] get the best performance from the [39:13] harnesses that the models are trained [39:14] with. And uh so that's where kind of the [39:17] harnesses are are really going to shine. [39:20] Um and also if you you are using uh an [39:23] environment where you can kind of drop [39:24] these things in, it just is making if [39:26] you're building agents, it's going to [39:27] make a lot of this stuff super easy. Um, [39:28] however, if you're doing, you know, kind [39:30] of oneshot tasks like maybe a [39:32] translation use case where you just need [39:33] to like just oneshot translate a [39:35] document or, you know, taking [39:37] unstructured data and structuring it [39:38] into JSON or some other format, [39:40] sponsor's API just out of the box is [39:42] going to be really good at that stuff. [39:43] And you can use all the hosted tools to [39:44] kind of have these like really light [39:46] agentic loops that are uh pretty easy to [39:48] to orchestrate in just one API call. [39:52] >> Cool. [39:54] >> Okay, next question. Does the agents SDK [39:57] feature any built-in out ofthe-box [39:59] persistent state management to handle [40:00] this pause and resume behavior natively? [40:03] >> Yeah, totally. So, I showcase a little [40:05] bit that snapshot mechanism. That's [40:06] totally out of the box. That works if [40:08] you just start using it. Um, the only [40:10] thing you have to define is kind of [40:11] where you want to store the snapshots, [40:12] whether it's in R2 as I mentioned or [40:14] just kind of local to the to disk which [40:16] is the default. Um, and so this is kind [40:18] of that, you know, really out of the box [40:21] thing where it will take one sandbox's [40:23] file system and then zip it up and then [40:26] put it somewhere else. Uh, so that [40:28] another sandbox can resume from the [40:29] point where it left off. Um, but also [40:31] the run state is sort of managed. So you [40:33] can think about two pieces of state [40:35] here. It's kind of like the file system [40:36] and then all of the messages in that [40:38] conversation so far that the agent will [40:40] need to continue from agents SDK manages [40:42] handling kind of like both of those [40:44] pieces in in one shot. And so you can [40:46] take that thing that has both a pointer [40:48] to where the snapshots stored and then [40:50] also the full roll out and then you can [40:52] store in your database. It's just a JSON [40:53] object and then load it back up later [40:55] and then you can kind of resume [40:56] losslessly maybe if you have sort of a [40:58] multi-node system or something like this [41:00] that just has the database to share [41:02] state. [41:04] >> Nish anything to add there or we can [41:06] move on to next question. Yeah. No, I [41:08] think that covers it. Like essentially [41:10] we store the roll out and we also [41:11] snapshot the file system which is uh a [41:14] little bit unique compared to other [41:15] SDKs. Um so you should be able to uh [41:19] resume very like long running uh dayong [41:22] agents pretty easily. [41:24] >> Cool. Okay. Next question. How would you [41:27] compare the code executing harness in [41:29] the agents SDK via the sandbox agent [41:32] versus the actual codeex harness? Any [41:34] capability? Uh Nish, you want to take [41:36] this one or? [41:37] >> Yeah. Yeah. Um so there are some sort of [41:40] small uh differences between the sort of [41:43] like actual codeex binary and the agents [41:45] SDK. Agents SDK does come with a lot of [41:48] those core capabilities. Uh some of [41:50] those are like the bash tool. Uh the way [41:52] codeex edits files, skills, memories. Um [41:57] there are some things in the codeex [41:59] binary that uh we are still going to [42:01] like build into agents SDK eventually. [42:04] uh the codex binary is like this sort of [42:05] like multi-threaded thing that you can [42:07] run uh multiple different sub aents in [42:10] agents SDK we have this concept of [42:12] handoffs so you can do something like [42:13] very similar in agents SDK you should be [42:15] able to do like a lot of the things that [42:17] you can do in the codex binary and [42:18] agents SDK um but yeah ultimately like [42:21] agents SDK is like very much uh in [42:24] distribution uh through the Codex [42:27] harness today. [42:29] >> Awesome. Thanks Nish. Um, next question. [42:32] Can agents persist and restore [42:33] longrunning workflows automatically? [42:35] >> Yeah, definitely. This is kind of what I [42:37] showed in in the demo. It's kind of like [42:39] very easy with very little. We didn't [42:40] look at a lot of my orchestration code, [42:42] but there's not really that much to it. [42:44] Just kind of kicks off the SDK, stores [42:46] the the roll out, that JSON file in a [42:49] file just on the laptop, and then is [42:52] able to kind of resume uh pretty easily. [42:54] So yeah, I think mostly most of the sort [42:57] of engineering that you'll do is [42:58] figuring out what tools to bring in, [43:00] what what skills should I add to the [43:02] agent and uh the sort of orchestration [43:04] of starting and stopping and resuming [43:06] tasks is is sort of uh just out of the [43:08] box. [43:12] >> Okay. Can the sandbox get auto destroyed [43:15] after the task is completed or is it a [43:17] better pattern to keep the sandbox [43:19] running? [43:19] >> Yeah, this is a good this is a great [43:20] question. Uh so I I actually only demoed [43:23] one mode of the ages SDK. But there's [43:24] two ways that you can use sandboxes. You [43:27] can do the sort of what we call um you [43:29] know SDKowned sandbox where when you [43:31] spin the task up if you just provide it [43:34] the recipe for creating the sandbox. So [43:36] you provide it one of the clients like a [43:37] docker client or an ETB client it will [43:40] handle spinning up that sandbox for you [43:42] and then when the when the turn is done [43:44] it will spin that sandbox down. But you [43:47] can also eagerly load a sandbox. So you [43:49] can kind of create a sandbox out of band [43:51] and then pass the reference of the [43:52] sandbox to the agent and then that's [43:54] what we call like user own sandbox and [43:56] so you would handle managing the [43:58] lifetime of that thing. It the SDK won't [44:00] automatically spin it down at the end of [44:01] the turn. So you can kind of keep it [44:03] warm and then use it across like [44:04] multiple turns instead of spinning it up [44:06] and spinning it down. Um, but the out [44:08] of- thebox behavior is that we'll handle [44:10] spinning it up and and and down so that [44:12] you don't end up in a situation where [44:14] you've built your demo and then all of a [44:15] sudden you have 100 sandboxes running in [44:18] the cloud that you're paying for and [44:19] it's kind of providing a bad outcome. So [44:21] there's a bunch of different ways that [44:22] you can use this. Um, but yeah, by [44:24] default we the HSDK will will handle the [44:27] lifetime. [44:31] >> Hey, how should I populate and persist [44:34] my file system? Yeah. [44:36] >> Yeah. Happy to take this one. Um, so [44:38] essentially we have this concept called [44:40] the manifest uh in agents SDK. Uh, the [44:43] thing that I really like about the [44:44] manifest concept is you can kind of [44:46] choose how you want to populate and [44:48] persist uh the file system into the [44:49] sandbox. Uh, so you can essentially copy [44:52] over all of your files from cloud [44:54] storage into the sandbox or you can [44:56] mount uh, you know, all of those files [44:59] uh, onto the sandbox. And there's sort [45:01] of like different benefits of of either [45:04] if you copy over everything like during [45:06] run the runtime the the model will be [45:08] able to read the files a lot more [45:09] quickly but you know startup will be [45:11] slower because you are copying like sort [45:13] of this big blob uh into the sandbox and [45:15] there's sort of the second world where [45:17] you can optionally mount something on [45:19] the sandbox and uh startup will be like [45:22] uh very very fast but like over the [45:25] runtime it has to like read these files [45:27] over the network. Um, but the nice thing [45:29] about agents SDK is we kind of like let [45:30] you do multiple different ways to kind [45:33] kind of like mount or copy over things [45:34] in the sandbox and then persist that uh [45:37] after the sandbox spins down. [45:40] >> Awesome. Thanks, Nish. [45:42] >> Okay, last question here. Can a [45:44] supervisor agent monitor and coordinate [45:46] hundreds of specialized agents [45:48] simultaneously? [45:49] >> I think the answer is yes. It would [45:51] definitely it would take a little bit of [45:52] work to kind of spin this up. And we [45:53] have some uh cool stuff around sort of [45:55] like multi- aent frameworks that's kind [45:57] of coming in the near like weeks and [45:59] months. Um so it will get a lot easier [46:01] to do this in the future, but uh there's [46:04] nothing kind of stopping you from doing [46:05] this now. You can kind of, you know, go [46:07] crazy with agents and spin up a ton and [46:10] then have one that can kind of check on [46:11] the status between them. I think like [46:13] ways we've seen good coordination [46:15] patterns is either by through messages. [46:17] You can give the parent agent a tool [46:19] that allows it to understand what sub [46:21] aents are running and then check in on [46:23] them. So kind of like check in on their [46:25] progress or you can sort of like have [46:27] them all communicate via the same file [46:29] system or like a database or something [46:31] like this with kind of a variety of [46:32] communication mechanisms you can use to [46:34] coordinate them. But yeah, nothing [46:35] stopping you from from doing this. And [46:37] um you know, I think that this sort of [46:39] massively parallel work will become sort [46:41] of the default in the near future. [46:45] >> Love that. Okay, so let's wrap up with [46:47] some resources. So on the screen here [46:50] are everything you need to know to kind [46:51] of get started with agents SDK. Um we [46:54] had a ton of questions also in the chat [46:56] about your demo and where they can where [46:58] they can find this. So on the very [46:59] bottom is our build hour repo. Um we [47:02] have all the code from the last build [47:04] hours and we'll be we'll be posting that [47:06] um very shortly right after right after [47:08] we're off camera. Um then for any [47:12] upcoming build hours, feel free to check [47:14] back on our homepage. Um so if you hit [47:17] the next slide, we show you just the [47:19] link again. Um and after this session, [47:22] we'll email out a survey. Let us know [47:23] what topics you want to hear about next. [47:25] And we'll also include the link for this [47:28] recording so you can catch up if you [47:30] missed the beginning. Um you can catch [47:31] get up to speed um there. And that's all [47:36] for now. Thanks so much for joining us [47:37] and we'll see you next time.