AI Engineer
Long-running autonomous research requires automated outcome rubrics that force agents to evaluate and retry their own work until deliverables meet formal specifications.
AI-assisted summary of Build a Research Agent with Claude Managed Agents (Part 2 of 4) — Anthropic — original source. Check the original for full context.
Source published:
Key ideas
- Anthropic's Claude Managed Agents introduces "Outcomes," a self-supervision mechanism combining task definitions, evaluation rubrics, and automated grader subagents that trigger retries until criteria are satisfied. (17:41)
- To secure API credentials, the platform uses "Vaults" that expose only placeholder tokens to model sandboxes, swapping in live secrets strictly at the egress proxy when requests match designated domain whitelists. (23:53)
- Cross-session state and organizational intelligence are preserved through shared Memory Stores, supplemented by an offline "dreaming" engine that consolidates transcripts and notes into structured knowledge. (29:14)
- Private enterprise data can be integrated using three deployment architectures: public MCP endpoints backed by Vaults, reverse-connection MCP Tunnels that bypass inbound firewall rules, or self-hosted sandboxes inside private VPCs. (33:15)
Detailed notes
## Background Published on YouTube by AI Engineer on October 12, 2026, this video documents part two of Anthropic's four-part Claude Managed Agents workshop recorded at the AI Engineer World's Fair in San Francisco. Presented by Anthropic technical staff members Priyanka Phatak and Gabe Cemaj, the session guides developers through evolving a single-prompt console agent into an end-to-end 'research desk' application equipped with custom skills, subagents, persistent memory, and self-supervising outcome rubrics.
## TL;DR Following a quick demonstration of creating a hosted agent from natural language in Anthropic's web console, Gabe Cemaj leads a hands-on architecture build of a multi-session financial research desk. He demonstrates how to stream real-time events to a chat interface, provision an SEC EDGAR filing skill, isolate child subagents, and wire up a shared memory store that persists context across runs. The build culminates in a self-supervised Nvidia research run where a dedicated grader subagent inspects deliverables against a predefined rubric and triggers retries until quality standards are met. Throughout the session, the speakers field audience questions on Vault credential egress swapping, differences between the Agent SDK and managed infrastructure, loop guardrails, session steering, and private enterprise data connectivity via MCP tunnels.
## Atmosphere & style The session has the candid, pragmatic energy of a live technical workshop where engineers speak peer-to-peer. When venue Wi-Fi constraints leave most attendees unable to clone dependencies, Gabe and Priyanka smoothly adapt the format, turning potential friction into an open, high-bandwidth architectural discussion. The dynamic between the speakers is collaborative and grounded, balancing transparent admissions of active roadmap gaps with deep, reassuring explanations of sandbox caching, credential virtualization, and multi-agent coordination.
## Bulletin - [17:41](https://www.youtube.com/watch?v=FGftGOzfohc&t=1061s) Anthropic's Claude Managed Agents introduces "Outcomes," a self-supervision mechanism combining task definitions, evaluation rubrics, and automated grader subagents that trigger retries until criteria are satisfied. - [23:53](https://www.youtube.com/watch?v=FGftGOzfohc&t=1433s) To secure API credentials, the platform uses "Vaults" that expose only placeholder tokens to model sandboxes, swapping in live secrets strictly at the egress proxy when requests match designated domain whitelists. - [29:14](https://www.youtube.com/watch?v=FGftGOzfohc&t=1754s) Cross-session state and organizational intelligence are preserved through shared Memory Stores, supplemented by an offline "dreaming" engine that consolidates transcripts and notes into structured knowledge. - [33:15](https://www.youtube.com/watch?v=FGftGOzfohc&t=1995s) Private enterprise data can be integrated using three deployment architectures: public MCP endpoints backed by Vaults, reverse-connection MCP Tunnels that bypass inbound firewall rules, or self-hosted sandboxes inside private VPCs.
## Key takeaways - [00:00](https://www.youtube.com/watch?v=FGftGOzfohc&t=0s) Session storage was re-architected to decouple event ingestion, persistence, and session mechanics by virtualizing interfaces across service boundaries and the event layer. - [01:04](https://www.youtube.com/watch?v=FGftGOzfohc&t=64s) Claude Managed Agents abstracts runtime infrastructure, enabling cloud-hosted agent deployment, sandbox environment provisioning, and event streaming via prompt-driven console workflows or the /v1/agents API. - [10:16](https://www.youtube.com/watch?v=FGftGOzfohc&t=616s) Execution compute supports multiple topologies, including Anthropic-managed cloud hosting, AWS Bedrock using enterprise billing credits, or self-hosted sandboxes on providers like Modal, Daytona, and Vercel. - [11:21](https://www.youtube.com/watch?v=FGftGOzfohc&t=681s) Client-agent interactions rely on a continuous, bidirectional event stream using Server-Sent Events (SSE) rather than standard synchronous request-response REST patterns. - [13:41](https://www.youtube.com/watch?v=FGftGOzfohc&t=821s) When spawning subagents, child sessions inherit system prompts and tool schemas but do not inherit the parent session's conversation transcript, preventing context pollution and token inflation. - [17:41](https://www.youtube.com/watch?v=FGftGOzfohc&t=1061s) Outcomes provide autonomous quality control by having a dedicated grader subagent evaluate finished work against an explicit rubric before marking tasks complete. - [22:09](https://www.youtube.com/watch?v=FGftGOzfohc&t=1329s) Skills function as live, versioned artifacts; updating a skill definition immediately propagates improved instructions and tool behaviors to newly created sessions without redeploying the core harness. - [23:53](https://www.youtube.com/watch?v=FGftGOzfohc&t=1433s) Vaults mitigate secret exfiltration by restricting live credential injection to the network egress proxy level, ensuring placeholder tokens are only substituted when calling whitelisted domains. - [26:08](https://www.youtube.com/watch?v=FGftGOzfohc&t=1568s) The Claude Agent SDK operates as a standalone headless binary intended for developer-hosted compute, while Managed Agents provides a multi-tenant, fully hosted cloud execution service with sandboxes and event streaming. - [27:15](https://www.youtube.com/watch?v=FGftGOzfohc&t=1635s) While infinite-loop prevention in custom harnesses currently requires developer-side termination logic, native platform features for hard financial caps and per-session token budgets are in development. - [28:04](https://www.youtube.com/watch?v=FGftGOzfohc&t=1684s) Active sessions support live operator steering by accepting user message events queued into the live stream, whereas session forking and history rewinding remain roadmap items. - [29:14](https://www.youtube.com/watch?v=FGftGOzfohc&t=1754s) Memory stores support concurrent layering, permitting an agent session to draw simultaneously from private, user-specific memories and global, organization-wide knowledge bases without vendor lock-in. - [31:54](https://www.youtube.com/watch?v=FGftGOzfohc&t=1914s) Outcome rubrics currently grade execution at the macro-session boundary rather than independently evaluating intermediate child subagent operations. - [33:15](https://www.youtube.com/watch?v=FGftGOzfohc&t=1995s) MCP Tunnels allow enterprises to connect internal data repositories and RAG pipelines to Claude Managed Agents over secure reverse tunnels without opening inbound public firewall ports. - [34:27](https://www.youtube.com/watch?v=FGftGOzfohc&t=2067s) Sandbox container environments resolve and cache package dependencies during environment creation, avoiding package installation overhead across consecutive session reinstantiations.
## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=FGftGOzfohc&t=0s) Evolving Session Storage: Decoupling and Virtualization
The session opens with an audience question about the architectural evolution of Claude Managed Agents: specifically, what changed in the session storage design between the first and second iterations. Priyanka Phatak directs the question to Gabe Cemaj to explain how the underlying data model shifted as the platform matured.
Gabe explains that the core motivation for redesigning the storage system was modularity and decoupling. In earlier iterations, the storage layer was too tightly bound to session mechanics, making it difficult to introduce new event types or support alternative backends without breaking runtime assumptions. Gabe Cemaj summarizes their architectural philosophy: “So we basically apply the same principle of building and virtualizing interfaces all the way down, not only across service boundaries but also at the event layer.” By virtualizing these boundaries, the team ensured that storage mechanisms could handle arbitrarily complex event streams going forward.
- **Component Isolation**: The storage redesign partitioned event ingestion, persistence, and session mechanics into independent abstractions rather than a monolithic session loop. - **Extensible Event Layer**: New event types can be registered and stored without requiring modifications to the core session orchestrator.
[01:04](https://www.youtube.com/watch?v=FGftGOzfohc&t=64s) The No-Code Console Quickstart: Creating a Hosted Agent in Seconds
Acknowledging that audience members are battling spotty venue Wi-Fi and may need to follow along on mobile hotspots or review the slides later, Priyanka transitions to a quick live demonstration in the web console. She sets out to prove how minimal the barrier to entry is by building a fully functional cloud-hosted agent from a single natural language instruction: *"build me an agent that tells me jokes every time given a topic."*
Upon submitting the prompt, Claude immediately synthesizes a full agent specification in the background. It generates an appropriate agent name, writes a descriptive overview, authors a specialized system prompt, selects a model, and provisions access to the standard agent toolset. Priyanka shows that developers can override any of these defaults, such as swapping the underlying foundation model, before clicking "create agent"—which hits the platform's `/v1/agents` API endpoint. Next, she configures an environment template for the sandbox. While she leaves access unrestricted for this demonstration, she emphasizes that production setups should enforce strict network access controls, whitelisting only authorized egress domains or specific Model Context Protocol (MCP) servers. After defining the environment and dependencies, she spins up a session and tests it with *"Tell me jokes about cats."* Claude displays its internal thinking block and immediately streams back a cat joke.
Priyanka Phatak underscores the operational simplicity of the managed model: “All you have to do is start a session with the agent, send events, get back events. Everything else is managed by us.” Rather than forcing developers to stand up sandbox fleets, write execution harnesses, manage session scaling, or maintain distributed event logs, the platform abstracts the entire runtime infrastructure so developers can focus strictly on product logic. With the console demo complete, she hands the stage to Gabe for the hands-on engineering track.
[03:47](https://www.youtube.com/watch?v=FGftGOzfohc&t=227s) Introducing the Hands-On Build: The Research Desk Architecture
Gabe Cemaj takes the microphone, inviting attendees squeezed into the back of the room to move forward into empty seats. He displays a QR code linking to the workshop GitHub repository and instructs the audience to clone the project, change into the `research-desk` directory, and install dependencies using either `npm install` or `bun i`.
Gabe then maps out the technical journey of the workshop. The goal is to evolve a basic single-agent chat endpoint into a sophisticated, multi-session financial research desk. The build begins with a custom chat interface running directly over the session event stream. From there, the agent is upgraded into a multi-agent system capable of delegating specialized work to subagents. They will equip the agent with an SEC EDGAR skill so it can query the U.S. Securities and Exchange Commission's filing systems, locate individual company tickers, pull down regulatory filings, and extract specific forms. Gabe notes that this setup powers long-running asynchronous agentic workflows, culminating in a map-reduce orchestration pattern where a primary agent delegates sub-tasks across multiple parallel sessions. Finally, he outlines the use of the Deployments API to schedule recurring background runs that keep financial analysis artifacts continuously refreshed.
```mermaid flowchart TD accTitle: Research Desk Workshop Progression accDescr: Progression from a bare chat interface to a multi-agent scheduled research system. subgraph UI["Interface Layer"] A["Chat UI on Event Stream"] end subgraph Core["Agent Capabilities"] B["SEC EDGAR Skill"] C["Subagent Delegation"] end subgraph Scaling["Orchestration & Automation"] D["Map-Reduce Multi-Session Harness"] E["Deployments API for Scheduled Runs"] end UI --> Core Core --> Scaling ```
- **Interactive Event Stream**: Interfacing directly with real-time session events rather than using blocking request-response patterns. - **SEC EDGAR Integration**: Automated retrieval and structural analysis of regulatory filings (such as 10-K and 10-Q forms) for public equities. - **Map-Reduce Agent Harness**: Orchestrating multiple agent sessions concurrently to process large data corpora before reducing findings into a unified report. - **Recurring Deployments**: Running autonomous analysis jobs on a cron-like cadence to produce self-updating research desks.
[06:10](https://www.youtube.com/watch?v=FGftGOzfohc&t=370s) Project Setup: Claude Code, Environment Variables, and the Six-Act Workshop Skill
Gabe transitions to his development environment, selecting Claude Code as his terminal agent while remarking with a laugh: “I don’t think anybody’s writing code by hand, so I’m not going to do that either.” He explains that the repository includes an automated interactive tutor packaged as a Claude Code skill. By invoking `workshop start`, attendees can execute a guided, six-act tutorial that manages project state, verifies code, and advances using `workshop next`. Gabe humorously apologizes that there will be no intermission between the six acts before enabling Claude Code's auto mode and instructing it to hold his hand through the setup steps.
Running `bun run dev` spins up the local Next.js research desk server, presenting a setup checklist in the browser. The automated skill initiates Act 1 by duplicating `.env.example` into `.env.local`. Gabe warns the audience that this configuration requires two mandatory credentials: 1. **Anthropic API Key**: Grants access to Claude Managed Agents endpoints. 2. **SEC EDGAR User-Agent Identifier**: Formatted as a user's name and contact email address (e.g., `Sample Company AdminContact@<sample company domain>.com`), which the SEC requires on all automated API calls to monitor traffic and enforce rate limits.
``` # Research Desk Environment Configuration (.env.local) ANTHROPIC_API_KEY=your_anthropic_api_key_here EDGAR_IDENTITY="Your Name your.email@example.com" ```
A quick show of hands in the room reveals that venue Wi-Fi failures have left nearly the entire audience unable to clone or run dependencies, with only a single attendee successfully keeping pace. Gabe and Priyanka pivot smoothly, urging participants to treat the session as an open architectural discussion, ask deep technical questions, and rerun the guided skill at home using their own network connections.
[10:16](https://www.youtube.com/watch?v=FGftGOzfohc&t=616s) Q&A: Infrastructure, Cloud Hosting, and Self-Hosted Sandboxes
As Gabe returns to the setup dashboard to click the button that provisions the agent and sandbox environment, an audience member asks where the managed infrastructure physically executes and whether it runs on Amazon Web Services (AWS).
Gabe details the deployment topology supported by Claude Managed Agents. Anthropic provides a fully managed first-party (1P) hosting environment that runs directly on Anthropic-managed infrastructure, abstracting all server operations. For enterprise customers who must maintain compute within existing cloud billing agreements or utilize enterprise discount programs, the service is also accessible through AWS Bedrock using existing AWS credits.
Furthermore, Gabe highlights that execution environments do not have to be hosted by Anthropic at all. Gabe Cemaj notes: “What we also offer and maybe we can talk about this after this demo is self-hosted sandboxes. So if you want to bring your own sandbox through different providers like Vercel or Modal or Daytona or a bunch of others, you can do that.” By supporting self-hosted sandbox integrations across providers like Modal, Daytona, Vercel, Google Cloud Platform (GCP), and AWS, engineering teams can keep sensitive execution environments and proprietary runtime packages entirely within their own virtual private clouds while continuing to rely on Claude Managed Agents for agent reasoning, session lifecycle management, and event routing.
[11:21](https://www.youtube.com/watch?v=FGftGOzfohc&t=681s) Setting Up the Event Stream and Chat Interface
Gabe Cemaj begins by configuring the initial environment for the research agent. For this specific demonstration, the agent is granted network access to external resources, specifically the U.S. Securities and Exchange Commission (SEC) website, along with predefined Python packages installed in its environment so it can query SEC endpoints. At this starting stage, the agent is completely bare-bones: it possesses standard basic tools and a system prompt, but lacks domain-specific skills, custom orchestration, or sub-agent coordination. Gabe prompts Claude to advance the build and wire up the application's conversational interface.
He explains that communication with a Claude Managed Agents session centers around an event stream. Rather than relying on simple request-response REST patterns, the application listens to an incoming stream for all lifecycle and output events emitted by the agent, while a dedicated ingress endpoint allows the client to push user events back up into the session. Gabe points out: “So really everything that happens in the communication for these more chat-like applications happens on that stream.” Claude writes the code to capture enter keypresses in the chat input, package the text into a `user_message` event targeting the active session, and pipe the resulting Server-Sent Events (SSE) back from the backend API to the browser for real-time frontend rendering.
Testing the freshly wired integration under the application’s "Desk" tab, Gabe sends a quick greeting and confirms the agent replies over the stream. While functional, he emphasizes that this is merely table stakes. If an engineer only wants a standard chatbot, they could stop here, but the true capability of managed agents lies in sustained, autonomous execution. As Gabe emphasizes, “The whole point of Claude Managed Agents is that you can do long-running asynchronous tasks—not just for minutes, but hours, days, weeks, and even months.”
```mermaid flowchart LR accTitle: Client-Agent Event Stream Architecture accDescr: Flow diagram showing bidirectional event communication between browser UI, backend server, and managed agent session.
subgraph Client ["Frontend"] UI["Chat UI"] end
subgraph AppServer ["Application Backend"] Endpoint["Ingress API"] SSEProxy["SSE Relay"] end
subgraph Platform ["Managed Agents Session"] Stream["Event Stream"] end
UI -->|"user_message event"| Endpoint Endpoint -->|"Push event"| Stream Stream -->|"SSE stream"| SSEProxy SSEProxy -->|"Render response"| UI ```
---
[13:41](https://www.youtube.com/watch?v=FGftGOzfohc&t=821s) Upgrading the Agent: Skills, Sub-agents, and Shared Memory
Having established basic connectivity, Gabe moves to augment the agent so it can climb what he calls the "ladder of intelligence." By entering `workshop next`, Claude updates the infrastructure-as-code provisioning definitions to equip the agent with specialized domain skills, custom tooling, and hierarchical sub-agent delegation. The agent receives an SEC EDGAR skill detailing how to use the pre-installed Python libraries and how to identify itself using the SEC EDGAR user-agent header requirements. Claude also defines a custom tool named `dispatch_analysis`, which will later let a primary "Head of Research" agent coordinate and spin up multiple parallel analysis sessions.
When Claude updates the primary agent definition, it produces a new version configured with dedicated sub-agents and, crucially, a recursive self-reference so the primary agent can clone its own prompt and tool persona if unexpected workloads arise. Gabe addresses a frequent point of developer confusion regarding sub-agent spawning: “It doesn't copy the transcript of that session into the sub agent. It just copies over the tool description and the schemas and the prompt into that sub agent.” This isolation prevents prompt context pollution and token bloat while giving the child agent a clean slate.
Gabe runs into a brief "cooking show" hiccup when trying to click the "Staff the desk" button—a pre-existing skill from an earlier run causes a provisioning conflict. He quickly deletes the stale skill and re-provisions the environment. In addition to sub-agent definitions and skills, the provisioning creates a long-running shared memory store. This persistent layer allows isolated sessions to write and read accumulated learnings about various financial tickers and market instruments, passing institutional context forward over months or years of automated runs.
- **SEC EDGAR Skill**: Encapsulates EDGAR querying rules, rate limits, identity headers, and Python library usage. - **`dispatch_analysis` Custom Tool**: Enables programmatic multi-session dispatch from the top-level agent. - **Sub-Agent Isolation**: Child sessions inherit system instructions and tool schemas without copying the parent session's conversational history. - **Cross-Session Memory Store**: Provides persistent, shared knowledge retention across independent agent executions.
---
[17:41](https://www.youtube.com/watch?v=FGftGOzfohc&t=1061s) Outcomes and Self-Supervision: The Nvidia Research Run
With sub-agents and memory established, Gabe transitions away from conversational interaction to launch a long-running, asynchronous task governed by "outcomes." In the Claude Managed Agents architecture, an outcome acts as a self-supervision primitive combining a task objective, an explicit evaluation rubric, and an automated grading loop. Gabe summarizes the mechanism: “In an outcome, you tell Claude, 'This is your task. This is your rubric to evaluate your task.' It's going to go off and do that work and then when it thinks it's completed, it's going to ask a sub agent to look at the rubric and the work that Claude did and evaluate: did it do it correctly and did it do it to completion?” If the work falls short of the criteria, the agent automatically retries until it passes or hits a defined maximum iteration ceiling.
Claude implements the underlying `user_defined_outcome` event, after which Gabe navigates to the application's scorecard view and triggers an analysis of Nvidia. Because a full SEC filing audit takes around ten minutes, Gabe switches over to review an identical session he ran the previous day. The prompt instructed Claude to analyze Nvidia's most recent quarterly SEC filings using valid EDGAR credentials, inspect shared memory first to avoid redundant research, extract key metrics into a standardized JSON scorecard saved to a designated local path, and log analytical notes back into the memory store.
Tracing the run through the console's observability interface, Gabe points out individual visual threads representing sub-agent lifecycles:
1. The root agent verified the shared memory store and discovered no prior notes for Nvidia. 2. It dispatched two specialized sub-agents in parallel: a Financial Expert Analysis extractor to pull filing numbers and a Risk Evaluator to dissect qualitative risk factors. 3. Upon sub-agent completion, the root agent synthesized the gathered data, wrote the formatted JSON scorecard deliverable to disk, and updated shared memory. 4. The outcome self-supervision loop automatically engaged: a dedicated grader sub-agent evaluated the final artifacts and execution log against the rubric. 5. The grader verified that all metrics were well-defined and that memory updates occurred, formally marking the outcome as satisfied.
---
[21:55](https://www.youtube.com/watch?v=FGftGOzfohc&t=1315s) Scaling Up: From Single Tickers to Multi-Agent Harnesses
After demonstrating that the completed scorecard extracts successfully from the session sandbox to the local environment (or into an external database in a production deployment), Gabe poses the next architectural hurdle. Analyzing a single company like Nvidia via a manual UI trigger works well for demonstration purposes, but institutional research workflows demand processing ten, fifty, or a hundred tickers concurrently. He pivots to the next phase of the workshop: leveraging the conversational interface to instruct the Head of Research agent to build and manage a custom multi-agent execution harness capable of orchestrating ticker evaluations at scale.
[22:09](https://www.youtube.com/watch?v=FGftGOzfohc&t=1329s) Updating Skills as Live Artifacts and Multi-Session Orchestration
Gabe Cemaj outlines how the custom orchestration harness operates inside the server. When communicating with the primary agent, that agent can autonomously decide to spawn parallel worker sessions—for instance, spinning up ten distinct sessions to execute isolated work in parallel before aggregating the results back into a final set of scores. Before demonstrating the orchestration layer, however, Gabe catches himself skipping an essential prerequisite: updating the underlying skill.
Skills in Claude Managed Agents are not static files frozen at deployment time; they operate as dynamic, versioned resources that can be continuously refined. In this case, Gabe demonstrates modifying the SEC skill to inspect a new SEC filing form. As soon as a revised skill is published, any subsequent agent session referencing that skill automatically inherits the updated instructions and schema without requiring a rebuild of the entire agent harness. Gabe emphasizes that developers can continuously improve skills by inspecting historical session performance, noting that Anthropic automates part of this process through a capability called "dreaming" demonstrated later in the session.
As Gabe transitions toward the orchestration setup, Priyanka Phatak steps in to handle audience logistics and kick off a live Q&A segment. Addressing attendees who were waiting on API promotional credits, she clarifies that although the signup form warns of a one- to two-day turnaround, credits typically provision in about five minutes.
- “I wanted to show you that these skills are live artifacts that you can update, so you can improve the skill.” — Gabe Cemaj - Once updated and published, any newly instantiated agent session instantly references the revised skill artifact. - Skills can be evolved over time based on edge cases encountered in earlier runs or consolidated automatically via offline "dreaming" passes.
---
[23:53](https://www.youtube.com/watch?v=FGftGOzfohc&t=1433s) Q&A: Vaults and Egress Credential Swapping
An attendee asks whether accessing SEC EDGAR filings requires credentials and how credential injection or brokering is securely handled by the system. Gabe clarifies that while the public SEC API technically only requires a name and contact email in the user-agent header—which they simply inject into the system prompt for the quick demo—production deployments should never expose raw credentials to the model. Instead, managed agents rely on a dedicated security abstraction called **Vaults**.
In a Vault, developers configure environment variables and secrets using placeholder tokens rather than raw API keys or passwords. While Claude is executing tools and writing code inside its execution sandbox, it only ever observes and manipulates the placeholder token. The sensitive swapping happens strictly at the network egress boundary, and only when the request destination matches an explicit domain whitelist assigned to that specific credential. If an agent attempts to exfiltrate an SEC credential to an unauthorized domain such as `anthropic.com`, the egress proxy refuses to substitute the placeholder, neutralizing credential theft or leakage.
Priyanka and Gabe walk through the console management interface for Vaults. Beyond static environment variables, Vaults natively handle OAuth token flows for Model Context Protocol (MCP) servers and third-party SaaS integrations like Notion or Google Workspace CLIs installed directly inside the sandbox environment. Each Vault yields an alphanumeric ID that can be passed directly into session creation calls to instantly grant safe, proxied access.
```mermaid flowchart LR accTitle: Vault Egress Swapping Mechanism accDescr: Diagram showing how Claude works with placeholder credentials inside the sandbox, while egress proxies swap placeholders for real secrets only when calling whitelisted external domains.
subgraph Sandbox["Managed Agent Sandbox"] Claude["Claude Agent"] Env["Environment Variable<br/>(Placeholder Token)"] Claude -->|"reads"| Env end
subgraph SecurityBoundary["Anthropic Network Egress Proxy"] Proxy["Vault Egress Interceptor"] Whitelist{"Is destination<br/>domain allowed?"} Proxy --> Whitelist end
subgraph External["External APIs"] SEC["Whitelisted API<br/>(e.g., sec.gov)"] Untrusted["Unauthorized Host<br/>(e.g., untrusted.com)"] end
Env -->|"HTTP Request with Placeholder"| Proxy Whitelist -->|"Yes: Swap for Real Secret"| SEC Whitelist -->|"No: Send Placeholder / Block"| Untrusted ```
- “Instead of putting the real secret in, what we put is a placeholder, and then when Claude is doing work and it wants to use that environment variable, it just injects the placeholder value; we at egress swap it out for the real value only on the host domains that you specified are allowed for that secret.” — Gabe Cemaj - **Domain Whitelisting**: If a secret is scoped exclusively to SEC endpoints, any outbound call targeting another domain receives only the inert placeholder string. - **Console & API Provisioning**: Vaults can be created programmatically or via the web console, returning an ID that can be attached to any session. - **Protocol Support**: Vaults support static API keys, environment credentials, and automated OAuth lifecycles for external MCP tool servers.
---
[26:08](https://www.youtube.com/watch?v=FGftGOzfohc&t=1568s) Q&A: Claude Agent SDK vs. Managed Agents
An audience member asks for architectural clarification regarding the boundary between the Claude Agent SDK and Claude Managed Agents, wondering whether the managed service is built as a direct wrapper on top of the SDK. Gabe explains that the two serve distinct operational paradigms despite sharing common architectural patterns.
The Agent SDK was originally packaged as a headless, developer-facing harness—effectively an unbundled, standalone binary of Claude Code that developers can download, containerize, and execute inside their own local or self-hosted compute environments. Managed Agents, by contrast, represent a fully hosted cloud infrastructure running on Anthropic's managed servers. While the managed agent harness draws heavily on design lessons, prompt harnesses, and execution primitives proven in Claude Code and the SDK, the team built a generalized, cloud-native orchestration harness capable of running arbitrary, non-coding research workflows, keeping the two codebases closely coordinated rather than in a strict parent-child dependency.
- “Agent SDK is basically like Claude Code in the form of a binary that you can take and run wherever you want; this is a fully hosted solution running in our cloud and we have our own harness that's heavily inspired by Claude Code's harness and we frequently stay in touch for like any enhancements across the board.” — Gabe Cemaj - **Agent SDK**: A headless, standalone distribution of the Claude Code harness intended to run anywhere developers choose to host it. - **Managed Agents**: A multi-tenant, fully hosted cloud execution service featuring managed sandboxes, persistent session streaming, and out-of-the-box infrastructure.
---
[27:15](https://www.youtube.com/watch?v=FGftGOzfohc&t=1635s) Q&A: Loop Guardrails and Token Budgets
A participant raises concerns about runaway multi-agent orchestration, asking what guardrails exist to prevent subagents from triggering infinite communication loops and rapidly burning through massive amounts of context tokens.
Gabe acknowledges that in custom agent harnesses, infinite-loop prevention and multi-agent coordination logic are currently left largely in developer hands. Because developers control the server-side orchestration harness, they are free to implement custom terminating conditions, call out to alternative LLM providers, or enforce internal cycle detection. However, Gabe shares that Anthropic is actively developing native, fine-grained budgeting primitives. The roadmap includes controls where developers can set hard monetary caps or strict session-level token thresholds, causing the runtime to automatically halt execution once an agent crosses its assigned resource ceiling.
- “We are exploring the idea of what it means to do like budget tighter budget controls that you can say use no more than this amount of money per session, use no more than this amount of tokens per session, but that's not something we offer today.” — Gabe Cemaj - Developers building custom orchestration harnesses are currently responsible for circuit breakers and recursion limits. - Hard currency ceilings and per-session token limits are in active research and planned for future platform updates.
---
[28:04](https://www.youtube.com/watch?v=FGftGOzfohc&t=1684s) Q&A: Live Steering vs. Session Forking
Referring to the web console transcript viewer, an attendee asks whether developers can step in when an agent starts drifting down an erroneous path—either to correct history, rewind state, or branch execution with updated instructions.
Gabe demonstrates that live steering is supported out of the box through session events. By selecting "Send Event" in the console or dispatching an API call, a human operator can inject a new user message event directly into the active session stream. The runtime automatically queues the message, allowing developers or end users to deliver course corrections or stack multiple instructions while the agent runs. However, rolling back state or editing prior history is not currently supported. Priyanka adds that true "session forking"—the ability to snapshot an agent's sandbox and conversation history at a specific turn, branch off in a new direction, and explore alternative paths without mutating the original session—is a heavily requested feature from enterprise customers and is slated on Anthropic's active product roadmap.
- “Session forking is essentially what you're asking for; we've had a few customers who have asked for it, it's on our roadmap, so hopefully we'll have it at some point.” — Priyanka Phatak - **Active Steering**: Operators can post real-time steering events into an running agent session, which queues and processes them sequentially without terminating execution. - **Session Forking**: Rewind-and-branch capabilities are not yet supported in the public API, but remain an active item on the platform roadmap.
---
[29:14](https://www.youtube.com/watch?v=FGftGOzfohc&t=1754s) Q&A: Shared Memory Stores, Dreaming, and Cloud Flexibility
A participant asks whether managed agents support a shared, collective memory layer across hundreds of different users working on financial research within the same organizational deployment. Gabe confirms that shared memory is already implemented and active in the workshop demo through the platform's **Memory Store** construct.
During the Nvidia research workflow, multiple autonomous agents concurrently read and write facts, intermediate findings, and edge-case learnings into a shared memory store. To prevent memory stores from accumulating fragmented, contradictory, or redundant context, Anthropic implements an offline consolidation process called "dreaming." During dreaming passes, background routines synthesize raw session logs and agent notes into a unified, clean knowledge base that improves subsequent agent runs across the entire organization.
When the attendee asks about infrastructure lock-in—specifically whether teams bound to AWS Bedrock or proprietary self-hosted sandboxes can decouple their data from Anthropic's cloud—Gabe and Priyanka emphasize that the architecture is fully open and decoupled. The memory store acts as a standard API resource: teams can extract memories, pipe them into external databases or vector stores, or wire in third-party memory systems via custom MCP tools. Furthermore, Priyanka explains that agents support layered memory hierarchy: a single session can attach both an individual user-level memory store and a broader, organization-wide memory store simultaneously, enabling personalized context alongside shared institutional intelligence.
- “None of this is locked in; you can take your data and go anywhere else if you wanted to, everything is an API call.” — Priyanka Phatak - “All of this is API based, so you can extract the memories and put them in different systems if you want to.” — Gabe Cemaj - **Multi-Store Attachment**: Sessions can concurrently attach multiple memory stores to isolate private user context while drawing from global organizational knowledge. - **Dreaming**: An automated background synthesis mechanism that processes disparate session records and edge-case outcomes to consolidate knowledge stores over time. - **Vendor Agnosticism**: Memory primitives, custom tools, and sandbox workflows can interface with external clouds and storage systems using Model Context Protocol (MCP) bridges.
---
[31:29](https://www.youtube.com/watch?v=FGftGOzfohc&t=1889s) Q&A: Grading Subagentic Workflows
An attendee observes that in the workshop architecture, the rubric grader evaluates outputs at the primary agent level, and asks how developers should grade and manage memory across nested subagentic tasks.
Gabe begins breaking down the two operational dimensions of this architecture, explaining that subagents are intentionally designed as context-augmentation primitives. Rather than forcing the root agent to ingest overwhelming volumes of raw SEC filings or tool responses, subagents take on focused delegation scopes to insulate the primary context window and preserve model reasoning capacity.
- “Subagents in the way that we've built them are meant to augment your context and leverage the...” — Gabe Cemaj
[31:54](https://www.youtube.com/watch?v=FGftGOzfohc&t=1914s) Session-Level Outcomes and the "Dreaming" Memory Engine
Addressing how outcome verification interacts with hierarchical subagents, Gabe Cemaj clarifies that evaluation rubrics operate strictly at the top-level session scope rather than on individual subordinate agents. The developer specifies the target outcome for the overarching session, and the primary agent coordinates the necessary subagents to accomplish the work, delegating tasks and reviewing intermediate progress until it determines the objective is met. Gabe notes that there is currently no native mechanism to assign discrete, isolated sub-outcome definitions to individual subagents.
An audience member asks a follow-up about the platform's "dreaming" feature, wondering how the system learns when subagents drift off course or perform unintended actions. Gabe explains that dreaming is currently scoped to memory stores. The system periodically inspects the transcripts and accumulated memories generated across prior sessions, consolidating and refining them so subsequent agent runs know precisely which contextual knowledge to pull. While today this background consolidation is restricted to memory updates, Anthropic plans to extend continuous learning workflows to refine prompts, agent definitions, and skills over time.
As Gabe Cemaj explains regarding the execution model: “The outcome definition happens at like an overall arching session level so you define the outcome and you say this is what I want you to accomplish.” Later, outlining the scope of autonomous memory consolidation, he adds: “Basically the way dreaming works is it will analyze the transcripts and the memories created across multiple sessions so that it can update its memories so that future agents know what to pull.”
- Outcomes evaluate the macro-session rather than grading individual subagent steps independently. - The dreaming engine runs asynchronously across session transcripts to synthesize durable memories for future sessions. - Long-term roadmap goals for dreaming include autonomous self-tuning of system prompts, agent tools, and skill definitions.
[33:15](https://www.youtube.com/watch?v=FGftGOzfohc&t=1995s) Ingesting Private Enterprise Data: MCP Tunnels and Self-Hosted Sandboxes
A workshop attendee raises a common enterprise security question: if a company has internal proprietary data housed in an internal retrieval-augmented generation (RAG) pipeline, custom database, or command-line interface, what is the safest pathway to expose those resources to managed agents without leaking them to the public internet? Gabe outlines three distinct integration architectures depending on the organization's security posture and firewall restrictions.
If an organization can expose an internal service via an API or Model Context Protocol (MCP) server, they can register the MCP server with the platform and protect credentials using an authentication vault to mediate outbound requests. For teams strictly prohibited from exposing internal services to the public internet, Anthropic provides "MCP tunnels," an egress-free networking bridge that punches a secure tunnel from the customer's private corporate network directly into Anthropic's managed agent environment. Finally, for organizations with rigid compliance boundaries that cannot use managed compute at all, Claude Managed Agents supports self-hosted sandboxes, allowing teams to bring their own compute instances running inside their private virtual private cloud (VPC).
```mermaid flowchart TD accTitle: Enterprise Data Ingestion Options for Claude Managed Agents accDescr: Three architectural patterns for connecting internal data stores to Claude Managed Agents including Direct MCP, MCP Tunnels, and Self-Hosted Sandboxes.
subgraph AnthropicCloud["Anthropic Managed Environment"] Agent["Claude Managed Agent"] Vault["Authentication Vault"] end
subgraph EnterpriseNet["Enterprise VPC / Private Network"] DataStore["Internal Data & RAG Stores"] TunnelClient["MCP Tunnel Client"] PrivateSandbox["Self-Hosted Sandbox"] PublicMCP["Public/Exposed MCP Server"] end
Agent -->|"1. Secure API Call with Tokens"| Vault Vault --> PublicMCP PublicMCP --> DataStore
Agent <-->|"2. Secure Bidirectional Stream"| TunnelClient TunnelClient --> DataStore
Agent -->|"3. Orchestrates Tasks"| PrivateSandbox PrivateSandbox -->|"Local Network Peering"| DataStore ```
Highlighting the secure networking middle ground, Gabe Cemaj states: “If you don't want to expose your MCP to the internet, we have a feature called MCP tunnels which lets you create a tunnel from your network into ours so that you don't expose it to the internet and we would do it that way.”
- **Public MCP with Vault:** Standard remote MCP server where runtime secrets and API tokens are swapped securely at outbound boundaries via managed vaults. - **MCP Tunnels:** Reverse-tunnel architecture connecting on-premise or private cloud MCP tooling directly to Anthropic without opening inbound public firewall ports. - **Self-Hosted Sandboxes:** Fully managed by the enterprise, running directly within corporate network perimeters to preserve direct internal peering with enterprise services.
[34:27](https://www.youtube.com/watch?v=FGftGOzfohc&t=2067s) Environment Dependencies, Caching, and Sandbox Runtimes
The discussion transitions to container environments and package dependency management inside sandbox runtimes. An attendee asks whether environment dependencies are fetched dynamically at runtime during agent execution or pre-baked during sandbox provisioning. Gabe explains that environment dependencies are resolved upfront during environment creation time. The resulting execution environment is cached, enabling rapid re-instantiation across consecutive agent runs without incurring the latency penalty of pulling and reinstalling packages repeatedly.
When asked whether the sandbox can consume proprietary or internal dependencies that do not exist on public registries, Gabe clarifies that the automated provisioning hooks currently expect packages accessible via public registries such as npm, Cargo, or pip. However, developers needing internal libraries can upload private wheels, tarballs, or raw source files directly into the session workspace, or clone proprietary GitHub repositories directly inside the sandbox during initialization. As another attendee spots a "Deployments" tab in the console UI and asks for details, Gabe pauses the audience questions to steer the room back to the live coding demonstration.
Explaining how container setup latency is minimized, Gabe Cemaj notes: “It's at creation time, and then we'll cache it and reuse that environment across reinstantiations of the sandbox.”
- Sandbox dependencies are resolved at environment creation time rather than on every individual agent action loop. - Caching environments across sandbox reinstantiations avoids redundant package installations during multi-turn executions. - Standard public registries supported include npm, PyPI (pip), and Rust crates (Cargo). - Internal or private code can be loaded by uploading binary artifacts or files straight into the active session file tree or checking out source repositories directly.
## Notable quotes > [11:21](https://www.youtube.com/watch?v=FGftGOzfohc&t=681s) The whole point of Claude Managed Agents is that you can do long-running asynchronous tasks—not just for minutes, but hours, days, weeks, and even months. — Gabe Cemaj
> [01:04](https://www.youtube.com/watch?v=FGftGOzfohc&t=64s) All you have to do is start a session with the agent, send events, get back events. Everything else is managed by us. — Priyanka Phatak
## Mentioned Claude Managed Agents, Priyanka Phatak, Gabe Cemaj, Anthropic, Claude Code, Claude Agent SDK, Model Context Protocol (MCP), SEC EDGAR, Nvidia, Amazon Web Services (AWS), AWS Bedrock, Modal, Daytona, Vercel, Google Cloud Platform (GCP), Next.js, Bun, Python, Notion, Google Workspace, PyPI, Cargo, npm
## Video description From a one-line console agent to a research desk with skills, subagents, memory and self-grading outcomes.
NEXT UP: AI ENGINEER CODE SUMMIT Join us in SF for the second annual AIE CODE Summit, Nov 10–12, our last flagship summit of 2026. Applications live now: https://aie.codes
Priyanka Phatak finishes a no-code agent in the console, then Gabe Cemaj leads the hands-on build: a 'research desk' app with a chat UI on the session's event stream. He adds an SEC EDGAR skill, subagents, a custom tool and a shared memory store, then runs an outcome-graded analysis of Nvidia, where a grader checks the work against a rubric and Claude retries until it passes. Q&A covers vaults, the Agent SDK vs managed agents, guardrails, memory sharing and MCP tunnels.
This is part 2 of 4 of the Claude Managed Agents workshop from Anthropic.
In this video: • Building a chat UI on the session event stream • Outcomes: a task, a rubric and a grader that makes Claude retry • Vaults that swap placeholder secrets at egress • Shared memory stores and self-hosted sandboxes
SPEAKERS Priyanka Phatak, Member of Technical Staff, Anthropic (leads the Claude Managed Agents engineering team)
Gabe Cemaj, Member of Technical Staff, Anthropic GitHub: https://github.com/gcemaj
LINKS Claude Managed Agents docs: https://platform.claude.com/docs/en/managed-agents/overview Scaling Managed Agents: Decoupling the brain from the hands: https://www.anthropic.com/engineering/managed-agents
CHAPTERS 0:00 Q&A: how session storage changed 1:04 Console quick start demo 3:47 The workshop repo 4:41 What we're building 6:10 Workshop skill and project setup 10:16 Q&A: where managed agents run 11:52 Chat interface over the event stream 13:41 Skills, subagents and custom tools 17:41 Outcomes and self-supervision 19:05 Nvidia research session walkthrough 22:40 Updating skills as live artifacts 23:53 Q&A: vaults and credential injection 26:08 Q&A: Agent SDK vs managed agents 27:15 Q&A: guardrails for agent loops 28:04 Q&A: steering and session forking 29:14 Q&A: shared memory and lock-in 31:29 Q&A: grading subagents 33:16 Q&A: private data and MCP tunnels
Recorded at the AI Engineer World's Fair 2026 in San Francisco. Subscribe for more talks from the engineers building with AI.
AI Engineer: https://ai.engineer YouTube: https://www.youtube.com/@aiDotEngineer X: https://x.com/aiDotEngineer LinkedIn: https://www.linkedin.com/company/aidotengineer/
#Claude #AIAgents #AIEngineer
[Source](https://www.youtube.com/watch?v=FGftGOzfohc)