Anthropic Automates Corporate Workflows With Managed Claude Agents

Anthropic通过托管Claude Agent实现企业工作流程自动化 转向云托管自主Agent让系统能够在夜间自我恢复并优化成果，推动人类生产力从个人执行转向系统化委托。 本场对话梳理了AI Agent的发展历程，即从脆弱的提示词循环演变为能够自我恢复、通过模型上下文协议（MCP）执行工具并优化产出的云托管自主系统。结合一家虚构杂货店的数据分析师现场演示，Jess阐释了Anthropic的预构建基础设施如何让开发者在无需人工监督的情况下委托复杂任务并运行迭代评估循环。在Anthropic内部，产品经理将这些Agent用作积极主动的协作伙伴，用于监控企业Slack频道、清理庞大的候补名单、审查代码库，并作为中立裁判打破架构僵局。 AI Agent 正在从基础的提示词循环演变为能够整夜自主运行的长程系统，其核心架构由底层模型智能层与负责调度工具、记忆及人机协作介入的 Harness 支架组成。 (0:00) Anthropic 推出 Claude Managed Agents 托管架构以支持自愈和自我调试循环；现场演示中数据分析师 Agent 在无人工干预的情况下自主处理数百万行数据并生成完整的 HTML 可视化报告。 (3:45) 通过标准化的模型上下文协议（MCP）安全连接企业系统，Anthropic 内部产品经理与工程师广泛利用 Claude Code 监控代码库 PR、处理客户安全 RFP，并在设计分歧时将其作为中立裁判。 (19:39) 企业与个人实践逐渐转向低成本的“一次性临时 Agent”（如半小时内构建 Agent 清洗并筛选 4,000 家机构的等候名单），未来垂直 AI 的核心壁垒在于深度嵌入现有工作流与聊天界面。 (25:08)

Anthropic product manager Jess Yan argues that software development is transitioning from manual human execution to autonomous delegation, enabled by cloud-managed agent infrastructure that self-corrects and operates independently. Under this model, developer productivity is increasingly bounded by how many tasks can be orchestrated simultaneously rather than individual human capacity. The Harness and Self-Recovering Agent Infrastructure (3:45) Technically, an agent comprises a foundation model, a system prompt, behavioral instructions, and a scaffolding harness. While early agent designs relied on brittle, synchronous prompting loops where a single dropped message or unexpected error caused the system to fail, modern systems rely on a tightly integrated harness to manage tool execution, context recall, and human-in-the-loop intervention thresholds. Anthropic couples its models with proprietary harnesses to provide standardized primitives through Claude Managed Agents. This architecture enables self-recovery: an agent can autonomously inspect its session history, debug execution errors, and adjust its plan to reach an established target without manual intervention. By incorporating the Model Context Protocol (MCP), these cloud-hosted agents securely interface with external services and internal databases behind an authentication layer, running multi-step workloads—such as processing multi-million-row dataset analyses and outputting browser-rendered reports—independently in the cloud rather than relying on local machine uptimes. Internal Adoption and Disposable Utility Agents (19:39) Internally at Anthropic, staff deploy agents to query their codebase, observe pull request states, track deployments, and summarize customer feedback across enterprise Slack channels. Rather than requiring engineers to manually context-switch, product managers can interrogate systems directly via Claude Code or deploy agents to automate repetitive administrative duties like filling out customer security RFPs. Claude is even brought into technical decision meetings as a neutral judge to break deadlocks when human design debates reach an impasse. Yan emphasizes using disposable, single-purpose agents rather than over-engineering massive multi-team orchestrators with complex compliance burdens. Because managed agents can be assembled in roughly 30 minutes, they serve as disposable utilities for short-term bottlenecks. For example, when a feature waitlist swelled to 4,000 organizations with duplicates and low-signal submissions, Yan spun up a temporary agent with database access to clean the records, estimate conversion probabilities, and surface high-value candidates over several weeks until the feature launched as self-serve. Evaluation Evolution from Vibes to Automated Benchmarks (10:06) Agent development traditionally begins with informal 'vibe testing' to evaluate early conversational behavior, but Yan notes that systems inevitably outgrow this stage because qualitative signals cannot be aggregated at scale. Moving to formal evaluation frameworks requires simulating complex, multi-string user interactions, executing competitive A/B tests between model versions, and employing LLM-as-judge scoring models to assign structured letter grades. Modern agent harnesses also incorporate trigger evaluations to ensure specific functional skills activate autonomously when appropriate, replacing manual slash commands. Furthermore, because agents can now run iterative self-correcting loops against predefined targets—termed 'outcome optimization'—systems can evaluate their own intermediate work in isolated context windows. This allows agents to iteratively tune machine learning models to meet specific benchmark accuracy targets or refine visual content without human oversight. Vertical Specialization and Workflow-Native Defensibility (34:34) As underlying foundation models absorb broad general knowledge, industry SaaS applications are pivoting toward hyper-specialized vertical tasks, such as accounting specifically tailored to solopreneurs. Yan and host Peter Yang observe that defensibility for these tools will not stem from isolated destination websites requiring manual form inputs. Instead, sustainable products must meet users inside existing work environments, primarily through cloud-hosted coding platforms and integrated chat interfaces. Rather than forcing users into fragmented web portals, the emerging operational model pairs user-specific context with ambient, always-on agents that trigger on cron jobs or incoming events. By delegating complex backlogs and bug remediation to cloud agents overnight, organizations increasingly operate like networks of decentralized, single-person startups supported by autonomous tooling. While cloud-managed agents demonstrate high utility for delegating operational overhead, building resilient applications still requires moving beyond early vibe testing to rigorous multi-string evaluations, alongside resisting the organizational urge to deploy over-engineered multi-team workflows before validating simple, single-user use cases.

Anthropic 的 Jess Yan 指出，AI Agent 正在由脆弱的即时提示循环演进为以模型与支架系统深度结合的自主托管架构；未来的生产力边界不再受限于人类个体的工作时长，而取决于多任务委托的能力，其技术演进与产品壁垒则在于深度嵌入现有工作流的超垂直化交付。 架构跃迁：支架系统与自主自愈循环 (3:45) 在技术架构上，现代 Agent 不再局限于被动问答的简单提示词循环，而是演进为由底层模型、系统提示词、行为指令和支架（Harness）共同构成的长周期运行实体。模型承担核心智能层角色，而支架则提供工程脚手架，用于调度工具执行、调用内存记忆，并自主判定何时请求人类介入权限。传统的裸提示循环由于高度依赖前置步骤的线性执行，在复杂业务中容错能力极弱；Claude 托管 Agent（Claude Managed Agents）则提供开箱即用的基础设施，赋予系统分析会话历史、自查错误与实时纠偏的自愈能力，使其在面对非预期输出时仍能自主校准并推进目标。 在演示中，预配置的数据分析 Agent 获得了本地文件系统和 Python 环境的调用权限。面对名为“Just in Time”虚构杂货店的数百万行数据文件，Agent 在无人工干预的情况下自主编写并执行代码，在数分钟内生成了涵盖订单模式关联、顾客行为热力图以及复购概率预测模型的三份浏览器交互式 HTML 文件。这一流程证实了通过前置数据结构与分步规划，随机性模型能够在脱离微观指令的状态下稳定输出高复杂度的业务分析结果。 评估演进：从感觉测试到结果驱动优化 (10:06) 将托管 Agent 接入敏感业务系统与内部数据库需依托模型上下文协议（Model Context Protocol, MCP）。该协议为外部服务提供标准化通信接口，并在前置环节设立身份验证层，以此规避未授权的数据暴露风险。在系统的开发与验证方面，团队强调“感觉测试”（Vibe Testing）是产品早期的必要起点，但由于该方式依赖非结构化反馈且无法在规模化场景下聚合信号，项目成熟后必须引入正式评估机制。现代评测体系综合运用二元合格判定、基于大语言模型裁判（LLM-as-judge）的等级评分，以及专门确保高阶能力在适当时机自动激发的触发评估（Trigger Evals），甚至通过独立上下文窗口运行自检回路以消除评分偏差。 与此同时，工程范式正从编写大量胶水代码以微观管理 JSON 结构化数据，转向“结果驱动优化”（Outcome Optimization）。得益于托管基础设施的自我纠偏机制，开发者可以设定明确的最终目标（例如模型需达到的特定准确率基准），由 Agent 在独立的循环中持续迭代、测试与修正，直至交付符合品质预期的成品，大幅削减了人工组装中间产物的工程损耗。 内部实践：“一次性代理”与组织效能 (19:39) 在 Anthropic 内部，技术和产品团队通过 Claude Code 及自主代理直接穿透至代码库，实时追踪拉取请求（PR）的合并与部署进展，避免了反复向工程师询问进度的内部消耗。内部部署的 Agent 还广泛承担了填写客户安全合规问卷（RFP）、诊断客户现场架构问题以及利用定时任务（Cron jobs）持续汇总企业客户 Slack 频道动态等重度运营工作。此外，团队在面临复杂棘手概念时频繁借助 Claude 开展前期深度梳理，甚至在团队陷入设计分歧时将其作为中立裁判引入 API 评审，以剥离人为偏见并打破决策僵局。 在日常运营中，托管 Agent 常被作为能在半小时内搭建上线的“一次性工具”使用。例如，面对一份长达 4,000 家机构且混杂大量冗余数据的产品候补名单，团队耗时 30 分钟构建起临时代理，通过数据库访问权清洗无效信息并评估转化概率，迅速筛选出具备高反馈价值的测试对象。对于企业级落地，演讲者明确建议摒弃一开始就构建跨 20 个团队的大型协同编排器的做法，提倡通过单人模板降低启动阻力，在大型组织内部培育自主运转的“单人初创模式”。 行业终局：超垂直化代理与嵌入式护城河 (34:34) 展望未来数月的演进方向，随着通用大模型内化了极其广泛的基础常识，垂直 SaaS 领域的价值正向极端细分的垂直场景迁移（例如专门服务于独立创业者的财务记账代理）。行业共性已不再体现为特定行业的固有开发流程，而是转向共享的上下文管理与任务编排逻辑，个体能够低成本构建满足极窄需求的专属软件并实现对外交付。 在产品竞争与护城河层面，讨论认为要求用户跳出工作环境并访问独立网站填写表单的传统模式将逐步式微。能够在竞争中存活的 AI 产品，必须具备“在用户开展工作的地方相遇”的能力，即以常驻协同者的形态无缝嵌入云端开发环境或即时通讯工具之中。正如 Vercel Chat SDK 所代表的发展趋势，对话界面正演变为人机协同的核心入口，AI 产品的长期防御性将完全取决于其能否紧密整合用户的个人上下文并深度融入既有工作流。 该访谈表明，AI Agent 正在推动软件应用从单步交互向全自主委托模式转变，但企业在落地时仍普遍面临过早过度工程化与编排复杂度失控的挑战。同时，通用基座模型能力的不断下沉，意味着无法深度契合真实工作流界面与私有上下文的独立产品将迅速失去生存空间。

[0:00] Jess Yan: The limits of what can be accomplished are no longer bound by personal human capacity, but by how much work can be delegated to these systems at once. [0:00] Jess Yan: It is the harness that elevates AI from a random sampling of tokens into a highly capable, actionable product. [21:07] Jess Yan: I talk to Claude more. I'll be honest. [21:07] Jess Yan: I'll be spending a lot of therapy time with Claude. [25:08] Jess Yan: It's okay if I throw that away agent away. Like it doesn't have to be the most beautifully productized thing in the world. [28:58] Jess Yan: Instill the kernel of creativity and sort of like autonomy. [33:59] Vibe testing is honestly the most important first step when developing an AI product. [34:34] Peter Yang: Hey, just take care of everything for me, [34:34] Accounting agent for solopreneurs. [37:18] Meet their users where they are [37:18] Just go figure it out [37:18] Everything is chat now, [39:33] Jess Yan: That I assure you humans still read.

[0:00] We've really evolved from agents being [0:02] prompting loop to agents being [0:03] autonomous, self-discovering and [0:06] longunning actors. We set them tasks [0:08] overnight and then we wake up and [0:09] backlog is resolved and bugs are are [0:12] squashed. All of that is 10,000 times [0:14] easier because of all the agents that we [0:16] [music] have internally. My personal [0:18] favorite is like a predictive model that [0:20] based off various attributes of the [0:22] customer and the product can predict [0:23] whether this customer is going to [0:25] return. And it's able to produce this [0:27] really rich level of insight in just [0:29] minutes. Limits of what we can achieve [0:30] will really be based off of how much we [0:32] can delegate at once more so than like [0:34] what our personal capacities are. [0:39] >> Hey everyone. Uh my guest today is Jess, [0:42] product lead at Anthropic for Cloud [0:43] Managed Agents. Uh really excited to get [0:46] Jess to demo uh how to build an agent [0:48] from scratch, maybe talk about how [0:50] Anthropic uses agents internally and [0:52] maybe even just talk about what an agent [0:54] even is. So welcome Jess. [0:56] >> Thanks for having me Peter. [0:57] >> Yeah, it's great to have you. Um so [0:59] everyone's talking about agents and the [1:01] agent tech stuff and everything. So let [1:03] me ask you, so how would you define what [1:05] an agent is and what are the main [1:07] components of an agent? [1:09] >> Man, what a what a loaded question. Um [1:11] yeah, so once upon a time agents were [1:13] really just prompting loops where you [1:16] were just trying to get questions and [1:17] responses uh in a loop and I think [1:20] that's really evolved towards [1:21] permissioning and access to thirdparty [1:23] systems, internal tooling and sensitive [1:26] data. Um and that that level of access [1:29] now requires permissioning um [1:32] observability [1:34] uh steering not not in the same way that [1:37] it was just questioned answered before. [1:39] >> Got it. And so they're really the [1:41] underlying components are still you know [1:42] the model uh the system prompt and [1:45] behavioral instructions and uh the [1:48] actual harness driving the loop but the [1:51] sophistication of what we are asking [1:53] agents to achieve is higher. So that has [1:56] made the sophistication of the harness [1:57] higher as well. [1:58] >> Okay. Yeah. Because there's no you know [2:00] agents can use tools they can have [2:01] memory right there's all kinds of stuff. [2:03] >> Yeah. [2:04] >> And why don't we also just define what a [2:06] harness is like what what is a harness? [2:08] Yeah, the harness is the core [2:10] scaffolding around the model that gives [2:12] it the ability to run those tools and to [2:14] call its memory um and to know when to [2:17] ask for human in the loop input versus [2:19] to just continue executing on its task. [2:22] So the harness is really what elevates [2:24] us from the sort of random sampling of [2:27] just tokens in and tokens out to actual [2:29] actionable uh products [2:31] >> and and let me throw you a curveball [2:33] actually. Do you think the model should [2:34] be developed the hardest like like are [2:36] they kind of joining at hip? I think I [2:39] am quite biased but I also think that it [2:42] is impossible to get the maximum [2:43] possible performance without tying [2:46] together the harness and the model. Now [2:48] the the uh components of the harness and [2:52] maybe the thickness of the harness will [2:54] change over time as models get more and [2:56] more capable. Um however [2:59] you know we when we test our models and [3:02] when we are uh assessing their [3:04] performance we always have to test it in [3:07] con conjunction with a harness and are [3:09] we going to test it with all the [3:10] different harnesses of the world we're [3:12] going to select the harnesses that we [3:13] have built and so there is an aspect of [3:17] you know the necessity of building [3:19] models is that you have to be testing [3:22] them with harnesses and that sort of [3:23] keeps them paired together. [3:25] >> Okay. like you know you test them with [3:26] like cloud co-work and cloud code and [3:28] maybe some third party cares right [3:31] >> yeah yeah yeah and of course we run [3:32] against like open source evals and [3:34] whatnot but at a certain point you know [3:37] every single model distribution now is [3:39] through a harness and so we also need to [3:42] be testing them through harnesses as [3:44] well [3:45] >> okay now let's talk about your product [3:46] what is manage agents and you know how [3:49] is it different from just like you know [3:50] me talking to the messages API [3:52] >> yeah so manage agents is really the [3:54] evolution of where we see uh task [3:57] orchestration going. And earlier [3:59] [clears throat] I talked about how we've [4:01] really evolved from agents being [4:02] prompting loops to agents being [4:04] autonomous uh self-discovering [4:08] and [4:09] uh you know uh longunning actors with [4:14] access to lots of thirdparty systems and [4:17] uh need for both permissions and [4:20] guardrails. And so cloud manage agents [4:22] was developed with that in mind. [4:25] um rather than just being a prompting [4:27] loop that we've sort of like added um [4:29] different capabilities onto, this is a [4:31] pre-built harness and uh companion [4:34] infrastructure to allow an agent to run [4:37] complex uh tasks at scale. And so for [4:41] us, the core motivation behind cloud [4:44] manage agents is that the return on [4:46] effort for building an agent should be [4:48] extremely extremely high. So we wanted [4:50] to build easy to stack primitives and [4:52] easy to use flexible developer APIs with [4:55] out of the box infrastructure and all of [4:57] that should be really really low effort [4:59] but then you should be able to delegate [5:01] hugely complex work that might have [5:03] taken you days out days months weeks to [5:06] to actually execute. [5:08] >> Okay. So it takes care of like a lot of [5:09] the infra stuff that you have to build [5:10] to even get agent to run. Is that kind [5:13] of [5:13] >> Yes. Exactly. [5:14] >> Got it. Okay. All right. All right. [5:16] Well, then without further ado, do you [5:17] want to show us uh how easy it is or [5:19] hard it is to build a manage agent? [5:22] >> Yeah, absolutely. Hopefully, we find it [5:23] easy. All right. Here I am in our cloud [5:26] console and I have a pre-built agent um [5:29] that I've already configured. Um so here [5:33] you see the core components of this [5:35] agent. So one is the model selection. Um [5:38] this continues to be what drives sort of [5:40] like the intelligence layer underneath [5:42] the agent. Um here are the system prompt [5:45] which is the raw text that the model [5:49] gets to define its behavior, its [5:52] guardrails and a highle awareness of the [5:55] kinds of test tasks that you will [5:57] orchestrate to it. I've given it access [6:00] to a builtin tool set that we ship with [6:03] every cloud manage agent. Um and I've [6:06] given it the ability to basically [6:08] interact with its file system and [6:10] produce uh produce results. I've [6:13] actually set its permissions to always [6:15] allow each of these tools. Um, but we [6:17] also have the flexibility to configure [6:20] these as um ask requesting permissions. [6:24] So, keeping a human in the loop for any [6:26] of these actions. [6:28] >> Um, in this particular case, I didn't [6:29] grant it any skills, but here's where I [6:31] would grant it skills as well. [6:33] >> So, this is a this is a agent to analyze [6:36] data and uh kind of like a data analyst. [6:38] >> Exactly. Yes. [6:40] >> Got it. Okay. [6:41] Um so uh this particular agent runs [6:45] analysis on a fictitious uh grocery [6:48] store called just in time [6:50] >> and I actually give it um uh an initial [6:54] prompt that gives it the data schemas um [6:58] a guidance on how to actually execute [7:00] the task and um I'll I give it also a [7:04] file a really large like multi-million [7:07] line file for it to analyze as well. Um [7:10] throughout the course of the session run [7:12] it you can see it running all of its uh [7:14] tools and calling the model selectively [7:18] to run the analysis and at the very end [7:22] um it outputs HTML files that can be [7:25] rendered in the browser so that you can [7:27] see the results of the analysis. Um, so [7:31] all of these particular t all of these [7:33] particular events are the actions that [7:35] the model is taking. And you can see [7:36] that I'm actually not steering it except [7:38] for the initial event. And at the very [7:41] end, [7:42] >> these outputs are are produced for me. [7:44] >> This episode is brought to you by [7:46] Riverside. I've used Riverside for years [7:48] to record my podcast because it records [7:51] in 4K resolution each person locally. So [7:55] the audio and video still comes through [7:56] clean even if a guest Wi-Fi gets shaky. [7:59] But the reason I love it now more than [8:01] ever is what happens after we stop [8:03] recording. If I go in here that I can [8:05] use these AI tools to remove pauses, [8:07] remove filler words and just clean up [8:09] the recording. And I I can also edit the [8:11] transcript directly and it will [8:13] automatically generate clips with [8:15] captions ready to publish to YouTube, [8:17] Spotify, and all types of social media [8:20] platforms all from one place. As a [8:22] oneperson creator business, that matters [8:24] a lot. Riverside is the upgrade your [8:27] content workflow needs. Try it at [8:29] creators.side.com/peteryang [8:33] and use code peter Yang at checkout to [8:35] get one month completely free. That's [8:38] creators. [8:40] Peter Yang. Now, back to our episode. [8:43] And what is the initial prompt for this [8:45] uh particular conversation? Is it just [8:46] like go analyze this huge file? [8:50] Yeah, you you could think of it as um as [8:53] basic as that, but I had a very specific [8:55] schema that I wanted it to be aware of. [8:57] So, for example, I wanted it to know [9:00] exactly what the structure of the data [9:02] set was beforehand. And that way I can [9:04] frontload the initial exploration that I [9:06] would have or ordinarily had to do. Um [9:09] and then I wanted to break down the [9:11] steps into discrete t uh discrete [9:13] segments so that I could get very [9:16] predictable outputs at the end of the [9:18] task because these agents are you know [9:20] randomized actors. So you do need to be [9:23] somewhat prescriptive sometimes if you [9:25] want to have uh very predictable [9:27] outputs. [9:28] >> Okay. And this will help with like [9:29] debugging and stuff too, right? Because [9:31] like they can figure out which step. And [9:33] to your point about debugging actually [9:35] um directly in cloud console um we have [9:38] an agent that runs and analyzes actually [9:41] the full uh session history associated [9:43] with the agent um and so uh after I've [9:47] run this I actually can um use this [9:50] debug agent to to look for areas where I [9:53] could have um improved this agent even [9:55] more. [9:56] >> I What is the output again? You said [9:58] there's like a HTML file. [10:00] >> Yeah. Yeah. So it produces uh three [10:02] different HTML files and I can pull this [10:04] up right now actually. [10:06] >> So uh okay so basically the components [10:08] are there's a there's a model there's a [10:10] prompt right there's tool access and [10:13] then there's skills that are optional. [10:15] >> Yes. [10:16] >> Uh for tool access like uh like if I [10:19] want to hook up to my internal database [10:20] or something like there there's like a [10:21] whole set of processes for for that [10:23] right or Yeah. [10:24] >> Yes. Yeah. So uh one of the ways in [10:28] which you can hook up an agent to a uh [10:31] thirdparty database or a third party [10:33] system is through MCP. MCP exposes a [10:37] standardized way to communicate to [10:39] external services and it includes an [10:41] authentication layer in front. So that [10:43] allows you to safely access these [10:46] internal services that you might not [10:48] want just anyone to access. Why don't [10:50] you just like briefly talk about if I [10:51] didn't use cloud manage agents like if I [10:53] had to set this up from scratch what [10:55] kind of scaffolding is to go into this [10:56] thing [10:57] >> to to build from scratch? Yeah. [10:59] >> Yeah. So I think that if you're working [11:01] with a raw prompting loop um then all of [11:05] your work is highly highly synchronous [11:07] and you are constantly dependent on like [11:09] the prior request that you strung [11:11] together to complete successfully in [11:14] order to get to the next step. And I [11:16] think that that worked in a world where [11:19] we were just asking the um you know the [11:23] chat bots to write haikus for us and we [11:26] were doing very very simple tasks um and [11:29] then over time that has become [11:31] increasingly unscalable. So if I [11:34] delegated a really large task to an [11:37] agent um and for example my like [11:40] something went wrong in you know that [11:42] first initial message that I sent and [11:44] either I dropped the message or it was [11:47] like slightly off from my expectations [11:49] then my ability to pivot my integration [11:52] and to handle that gracefully is is [11:56] significantly lower. So, it's important [11:59] for us to evolve towards these more [12:02] self-running [12:03] um you know uh self-reovering agent [12:06] loops that can recover from errors, [12:09] recover from going slightly off course, [12:12] uh resteer themselves back and then just [12:14] keep you in the loop as they're doing so [12:16] so that you're aware of their process. [12:18] >> When you say self-reovery, you mean like [12:20] they run into error, they can like debug [12:21] their own error, they can do some [12:22] searches and figure it out, right? [12:24] >> Yes. Yeah. [12:25] >> Got it. Um, and then even more baseline [12:28] than that, you know, if they um, uh, if [12:32] they produce an output that is [12:34] unexpected, but then they're aware of [12:36] what a good output is supposed to look [12:37] like, they can, uh, you know, revise [12:40] their thinking and really adjust their [12:42] course of action. That is very difficult [12:45] to wire together in just a raw prompting [12:48] loop. [12:48] >> All right. Uh, let's go back. Do do you [12:50] think the output generator generate now [12:52] or is it still working? [12:52] >> Yeah. All right. So this agent produces [12:55] three primary artifacts. So first an [12:57] analysis into the products. Um so you [13:01] know just an overall uh highle [13:04] in inspection of like common common [13:06] order patterns in shopping carts. It [13:10] also produces a analysis into the [13:12] shoppers and sort of heat maps on when [13:14] they're shopping um in a and these [13:18] really interesting like radar charts. [13:20] And then lastly, like my personal [13:22] favorite is like a predictive model that [13:24] based off of various attributes of the [13:27] customer and the products um can predict [13:30] whether this customer is going to [13:32] return. Um so all of this was just with [13:35] like a simple prompting and access to [13:38] Python packages in the agents [13:41] environment and it's able to produce [13:42] this really rich level of insight in [13:44] just minutes. And so the prompt has like [13:47] uh some stuff about like the format of [13:49] the three reports that we want and the [13:51] data that we want, right? [13:52] >> Yes. So um what I what I put into the [13:56] system prompt is general performance [13:59] optimizations because I do want this to [14:01] be a general agent that I can reuse [14:02] across a lot of different uh data sets. [14:05] And then what I put into the initial [14:07] prompt I sent to it is that schema that [14:12] highly specific schema discovery and the [14:14] desri the actual task description on how [14:18] to run um its analysis in sequence. [14:21] >> Well, okay. Well, let me ask you this. [14:22] So, so now you have a bunch of traces [14:24] and a bunch of outputs like you know [14:27] like how do we build evals for this [14:28] Asia? How do we know it's like not going [14:30] off the rails? [14:31] Yeah, eval is definitely uh the toughest [14:34] part about building agents today because [14:37] I think that what has traditionally been [14:42] a [14:43] uh you know traditional eval development [14:46] is evolving as the tasks get more [14:48] complex. I think that you know the [14:51] traditional eval setup of you know [14:54] here's a set of initial prompts and [14:55] here's how we want the agent to uh [14:58] produce results in response. um that [15:00] still works. I think that we're also [15:02] seeing folks uh do more sophisticated [15:05] things like um [15:08] uh replays of more complex like [15:11] multistring uh kinds of interactions [15:14] with agents or um you know AB testing [15:18] different versions by sending the like [15:21] same string of sort of user interactions [15:23] and seeing how that uh ultimately [15:25] changes uh the responses. I think [15:28] another trend that we're we're also [15:30] enabling in cloud manage agents is like [15:33] a built-in eval loop where if the agent [15:36] itself knows to grade its outputs then [15:39] you can actually pull the eval directly [15:42] into the agent's work rather than having [15:45] it be outside of the course of the [15:47] session. [15:48] Okay, this is kind of what I do in cloud [15:50] code. Like it spits out an output and I [15:52] try to get another agent to run the eval [15:54] and then if the eval sucks then I try to [15:56] get the other agent to work again [15:59] >> basically. Yeah, when you when you can [16:01] have agents evaluating their own work uh [16:03] potentially in separate context windows [16:05] to avoid bias, then you're always going [16:07] to get a better output. And um just real [16:10] quick on evals like um do you guys do [16:13] like past fail evals or or like do you [16:15] do uh like scoring evals or all kinds of [16:18] evals? [16:19] >> We do a mix. There's definitely binary [16:21] pass fails. There's definitely scoring [16:23] which is more like LLM as judge and like [16:25] applying sort of more of a [16:28] you know letter grading type approach. [16:29] And then there's also triggering evals [16:32] um for things like you know make sure [16:34] that this type of action is actually [16:35] triggered in general. Um, for example, [16:38] with skills, something that we worked on [16:40] very early on was making sure that [16:42] skills triggered properly at the right [16:44] time because the whole point behind [16:46] skills is, you know, progressive [16:47] disclosure. [16:48] >> Oh, yeah. It wasn't that good at I think [16:50] that's gotten better over time. I used [16:51] to have to manually trigger the skills [16:52] with like slash commands. [16:53] >> Yeah. [16:54] >> But it's got a little bit better over [16:55] time. Got it. Okay. I feel like as these [16:57] agents become more autonomous, right, [16:59] and they can do longer running tasks. I [17:01] feel like it's more about like what is [17:02] your goal and what's the outcome that [17:04] you wanted to have? Like what do you [17:06] think about like that kind of changes [17:07] the prompting and everything, right? [17:08] Like [laughter] [17:09] you know. [17:10] >> Yeah. Yeah, it definitely changes [17:11] things. Um I think that you know once [17:15] upon a time there were structured [17:16] outputs, right? We told an agent [17:20] specifically your output must adhere [17:23] rigidly to this structured uh you know [17:27] code formatting. [17:28] >> Yeah. And then we will string together a [17:30] lot of glue to make sure that these big [17:33] blobs of of JSON um structured [17:35] structured data turn into something like [17:38] beautifully rendered in the browser. I [17:40] think that as models have become more [17:42] capable and as harnesses have evolved [17:45] the sort of outcome has become the [17:48] structured output. The the uh outcome is [17:51] sort of a meta a meta structured output [17:54] where [17:55] we don't need to tell the agent anymore. [17:58] Okay, like you know this is the exact [18:00] structure of your response and uh I will [18:04] glue all of these together and create [18:06] something rich and interactive. We're [18:09] just skipping straight ahead and saying [18:11] like let's build this rich and [18:12] interactive thing and this is sort of my [18:15] tastemakers assessment of what good [18:18] would look like. And I and because we [18:21] now have the infrastructure for these [18:23] agents to run autonomously, the agents [18:25] can actually self-correct along the way [18:27] rather than relying us to string [18:29] together all these intermediate outputs. [18:31] >> Okay, so it's not like, you know, here's [18:32] like five seconds you got to adhere to, [18:34] here's the character count, here's like [18:35] what you got to do. It's more like I [18:37] think it's actually kind of harder now [18:38] because like what's an example of a [18:40] tastemaker output? like, hey, you got to [18:42] make it beautiful and interactive or you [18:44] like what? [18:45] >> Yeah, I mean, yeah, we we see people [18:47] using it for like slide generation and [18:49] and content creation. Um, I I think [18:51] that's like one area where outcome [18:55] optimization is is really useful just [18:57] like visual artifacts and editorial [18:59] content. Um the other place where we see [19:02] it being useful is on uh you know for [19:05] example in the predictive model that I [19:07] just showed. Let's say that I needed it [19:09] to achieve like a specific score of 90% [19:13] in a sort of like accuracy benchmark. [19:16] Then um I've I've run tests with this [19:20] agent before to try to always optimize [19:22] for building a model that will hit that [19:25] score and it's iterated until it got [19:27] there. [19:27] >> Okay. Got it. Okay. That that that makes [19:29] sense. Yeah, it makes sense to try to [19:31] get the agents to do their own loops [19:32] first before the human eyes have to come [19:34] into play, you know, just like make as [19:36] much progress as possible. Yeah. [19:38] >> Yeah. [19:39] >> Yeah. [19:39] >> Well, let me let me switch gears a [19:40] little bit. Let's talk about how you [19:42] guys use agents internally and [19:44] anthropic. Um and yeah, maybe you can't [19:46] screen share Slack, but I'm curious how [19:48] anthropic PMS, you know, technical [19:50] staff, what what kind of help and [19:51] leverage has agents helped you guys [19:52] internally during your work? Yeah, for [19:56] me it's really been about depth and I [19:58] think that access to our codebase has [20:01] been the biggest unlock for me. I think [20:04] that one it helps me just like manage [20:08] state more easily. You know rather than [20:10] um you know poking a bunch of engineers [20:12] on what they're doing, I can just track [20:15] the PRs directly and see which ones are [20:17] merged, which ones are deployed. Um, I [20:20] think there's also an aspect of like I [20:23] deeply understand and interact with my [20:25] product so much more than I've ever been [20:27] able to in the past because it's so much [20:29] faster for me to either prototype an [20:32] agent on cloud manage agents uh because [20:34] I can use cloud code um or just [20:37] interrogate the codebase on like exactly [20:39] how everything is working. So, anything [20:43] from, you know, going into a customer [20:46] RFP and filling out all of their, you [20:49] know, security check marks to, you know, [20:52] diagnosing problems in the field, [20:54] unblocking users who are having trouble [20:56] or helping them scale by like helping [20:58] them understand like our specific [21:00] architecture. Um, all of that is 10,000 [21:03] times easier because of all the agents [21:06] that we have internally. [21:07] >> Okay, so let's kind of break that down [21:08] into a few things. So um do you come [21:10] into work on Monday and be like hey you [21:12] know hey cloud what what did my [21:14] engineers ship last week like [laughter] [21:16] do you ask that kind of question or or [21:17] like how do you [21:19] >> um for for for me I have some scheduled [21:23] runs going that like summarize activity. [21:26] Um, but I still do a lot of the deep [21:30] dives on a more sort of ad hoc basis [21:32] based off of the questions I'm getting [21:34] or like the the pitches I need to uh uh [21:37] prepare for or the customer [21:39] conversations I'm about to have. [21:41] >> Okay. Got it. Okay. Got it. So, like [21:43] you're going to enterprise to like talk [21:44] about manage agents and you need to like [21:46] personalize the pitch for for that [21:48] company or something, right? [21:49] >> Yeah. Yeah. Um, and and and do you guys [21:53] have like a feedback? Like I I know some [21:54] people are active on Twitter even so [21:55] it's pretty toxic, but like how do you [21:58] get feedback for your prof product? Like [21:59] do you have like a enterprise Slack [22:01] group or something? [22:02] >> I do have agents monitoring our Slack [22:04] channels. Uh I sit in a bunch of Slack [22:06] channels with all of our with a bunch of [22:08] customers. Um and I love talking to [22:10] customers directly, but for those I'm [22:12] not able to talk to directly, it is [22:14] useful to have agents sort of [22:16] summarizing the activity that's [22:17] happening there. Um I I I think that [22:20] like we have started to evolve towards [22:23] thinking about agents as you know always [22:26] on like you should be able to tag them [22:28] anywhere but they also should be [22:31] proactively surfacing things toward uh [22:34] for you in the way that a co-orker truly [22:36] would. And so I think that like there's [22:40] two aspects to which our agent usage is [22:44] really powerful. One is the level of [22:46] access of data we give it. And then two [22:48] is the interaction styles that we expect [22:51] of our agents which is it should be [22:53] humanlike. It should be proactive and [22:56] not just reactive. [22:58] uh is proactive through like triggered [23:00] events and like chron jobs or like how [23:02] do you make it proactive just like [23:03] >> yeah proactive is on triggered events [23:05] and chron jobs and like continuously [23:08] refreshing the data that has available [23:10] so that if it is long running and it's [23:12] and it's like constantly slurping up [23:14] information. It's it needs to be as [23:16] upto-date as you are and that shouldn't [23:19] be just you know on an ad hoc basis that [23:21] should be proactive. [23:23] >> Got it. Yeah. Yeah. It's got to have the [23:24] most up-to-ate context right. Yeah. [23:26] >> Yeah. [23:26] >> So why don't you ask this question? How [23:28] do you compare your conversations with [23:29] claudin agents versus your conversation [23:31] with co-workers? Like what happens more [23:32] throughout the day? [laughter] [23:35] >> Interesting. That's a great question. [23:37] >> Yeah. [23:38] >> Um [23:39] >> I I feel like I talk to cloud more. I'll [23:41] be honest. [laughter] [23:42] >> It is true. I I I do I think [23:45] particularly when I'm in a new space, I [23:50] find myself spending a ton of therapy [23:52] time with Claude just trying to wrap my [23:54] hands, wrap my arms around uh a thornier [23:57] concept. And I think that it uplevels my [24:00] conversations with my my teammates [24:03] because I'm able to come to a [24:04] conversation with like a true opinion [24:06] and a lot of baseline research done very [24:10] quickly. And so I'm not asking the like, [24:14] you know, please spin me up questions. [24:16] We're able to engage at a deeper level. [24:18] >> I mean, I I guess that's probably [24:20] expected with an right here because like [24:21] you encouraged to use cloud. Yeah. Do do [24:24] you guys pull up cloud in like a [24:26] decision m meeting or like a team [24:28] meeting and like try to you know [24:30] >> Yeah. [laughter] [24:31] >> Yes. [24:31] >> You do that? [24:32] >> Yeah. Uh Claude is a really good neutral [24:36] judge for certain things. Uh we have an [24:39] API review claude that basically if [24:43] we're really stuck at an impass with how [24:46] we want to shape certain components of [24:48] our API then we do tag in quad to tell [24:53] us when our biases are getting the [24:56] better of us. [24:56] >> Okay. [24:57] >> But we all of our primitives are [25:02] definitely there to be able to allow for [25:04] this kind of interaction. It's sort of [25:06] like agent to agent communication. All [25:08] right. So, just to wrap this section, [25:10] how would you say you typically spend [25:12] your day or I guess no real typical day, [25:14] right? But like do you talk to a lot of [25:16] enterprise customers? Do you do like a [25:17] lot of road map planning? You're like [25:19] shipping stuff yourself like how do you [25:20] typically spend your day? [25:22] >> Yeah, I think that my day now is spent a [25:28] lot more in the customer discovery and [25:31] like sort of uh integration journey [25:34] process than it has been before. And I [25:36] think that's awesome. I think it's [25:37] because frankly like we're all moving so [25:40] fast that the like I said the kind of [25:43] conversations that we're all having has [25:45] been like seriously upleveled. I think [25:47] previously, you know, if I spent this [25:49] much time in customer conversations, it [25:52] might be like, oh, like please debug [25:54] this like tiny little thing that is [25:57] like, you know, some a problem that I've [25:59] debugged a hundred times. But now, [26:01] because they have agents and because we [26:03] have agents and because we're pushing [26:04] the boundaries, the conversations are [26:06] now like, okay, how would we build this [26:09] super futuristic thing together and like [26:11] what are our principles around it and [26:13] how can we push that forward in the next [26:15] like two weeks? So, um, those [laughter] [26:17] kind of those kind of conversations are [26:19] really exciting for me and like I really [26:23] really do love spending time with our [26:24] customers. Um, I also spend a ton of [26:26] time prototyping and so um, one thing I [26:30] love about managed agents is because [26:32] it's so easy to spin up an agent. Um, [26:34] and because my work changes dayto day [26:37] and it never really looks the same um, [26:40] you know, week over week. uh I need to [26:43] be able to spin up an agent that is [26:46] perfectly suited for that specific task [26:49] of the moment. And it's okay if I throw [26:52] that away agent away. Like it doesn't [26:53] have to be the most beautifully [26:55] productized thing in the world. [26:57] >> Okay. [26:58] >> Um and so I spend a lot of time bashing [27:01] my own product by basically automating [27:03] my own work. and that it it takes me [27:08] maybe half an hour to to spin up an [27:10] agent, but I try to have like a [27:11] different one going every couple of [27:13] weeks. [27:13] >> Can you give a example? [27:15] >> Yeah, I'll give an example. So, we had a [27:18] weight list um for some of our advanced [27:20] features and it was like a 4,000 [27:24] organization long wait list and it was [27:27] filled with invalid entries and [27:30] duplicates and you know all the kind of [27:32] stuff that you get on like a traditional [27:33] published web form. [27:35] And so I know that this agent or I I [27:38] know that this wait list is only really [27:43] uh going to be relevant for the next few [27:45] weeks until we get this feature out into [27:48] the public and make it self-s serve. Um [27:50] so I just needed to spin up an agent [27:52] that would automate my next few weeks of [27:54] work with this weight list like um [27:56] parsing out all the invalid entries like [27:58] assessing which ones are the highest [28:00] likelihood to convert and actually give [28:02] really high feedback. and I basically [28:04] embedded it with like access to our [28:06] internal systems and our databases and [28:08] whatnot to make that assessment and [28:10] figure out who to pull in off the weight [28:12] list on a on a daily basis. Um, that's [28:15] just like a few weeks of work and [28:17] there's no point in like building [28:20] something super shiny for it. And so [28:23] having the a really really good easy to [28:26] use pre-built infrastructure [28:29] >> that just automates building an agent is [28:32] like a huge unlock for me and I can take [28:35] those kinds of tasks and just repeat [28:36] them. [28:37] >> Okay. So basically the this agent uh [28:39] looks at a big weight list and then like [28:41] uh cleans it up and then like sends uh [28:44] invites to people on the wait list. Is [28:46] that kind of [28:46] >> Yeah. Yeah. like people who are highest [28:49] likelihood to be high value testers for [28:52] us. [28:53] >> Got it. People who who are like the most [28:55] active Twitter complainers basically. [28:56] No, I'm just kidding. [laughter] [28:58] No. Um Okay. I don't have to be a [29:01] company to use this product, right? Like [29:02] like I I I think you were going to show [29:04] me how to do this in cloud code or [29:07] >> like can I just use this as an [29:08] individual? [29:09] >> Yeah. [29:09] >> As long as you have an API key with us, [29:11] then you should be able to use it. In [29:13] fact, we have a ton of usage from [29:14] individuals. Um, it actually kind of [29:17] surprised me, but like I think that [29:19] we're seeing a lot of individuals just [29:21] like automate their lives with cloud [29:22] manage agents. [29:24] >> But like when should I use this product [29:26] versus just like try to build a skill or [29:27] like a week weekly crown job or [29:30] something like you you said there's a [29:31] lot of individuals using it, right? [29:33] >> Yeah. Keep in mind that these are really [29:35] longunning cloud hosted sessions. So [29:37] anything that's running directly in your [29:39] cloud code is is bound by the [29:42] constraints of your laptop and and when [29:44] it's on. And [29:46] so using cloud manage agents basically [29:48] pushes that all to the cloud. It [29:50] increases the capacity of the work that [29:51] you're able to do and also the longevity [29:53] of it. [29:54] >> Okay. So stuff like uh maybe like some [29:56] sort of online competitive research or [29:58] like um like what are what are people [30:01] using it for uh [30:03] >> in in their personal lives? [30:05] >> Yeah. I um I have a friend who's like a [30:09] new parent who basically has her entire [30:11] like child's um sort of like the hourly [30:14] schedule of like feedings and like um [30:17] tummy time and all of the things that [30:19] new parents have to worry about [30:20] basically fully managed by Claude. Um [30:24] and then she also has like these like uh [30:26] fridge monitors and like grocery uh uh [30:31] grocery management agents running as [30:33] well. Uh [30:34] >> yeah. So yeah, I I I don't actually have [30:38] insight into what people are doing at [30:40] scale because like you know uh we redact [30:44] a lot of that information internally, [30:46] but um from anecdotal usage like [30:49] honestly just using these as like your [30:51] personal uh personal assistant uh is you [30:55] know they're ultim ultimate customizable [30:58] agents, right? You can do them for [30:59] whatever you need. Like I I I'll tell [31:02] you my bias. Like I feel like uh [31:03] sometimes companies tend to set up [31:05] agents. Like they make it too [31:06] complicated right off the bat, you know? [31:07] They just like they want to have like or [31:09] orchestrators. They want to have like [31:10] all all kinds of stuff. And then my bias [31:12] is just like why don't just build one [31:14] agent and see if it actually works and [31:15] people actually use it. But like like [31:17] you know, you talked to a lot of [31:18] enterprises. What what are some best [31:20] practices to roll this out? Yeah. I [31:24] think that [31:26] a lot of enterprises make the immediate [31:28] jump to like how could I automate this [31:31] like crazy 20 team workflow that would [31:36] have required like a lot of like [31:38] crosscutting coordination and these like [31:40] multimonth processes like I mean super [31:43] ambitious very exciting but I do think [31:45] that there is something really valuable [31:47] about just like okay how do we unlock [31:50] the individual like how do How do we [31:53] make any individual on any team feel [31:57] exponentially more powerful if like one [32:01] Peter and one Jess is suddenly like wait [32:04] I don't need to make dependency requests [32:06] because I have these agents that are [32:09] able to extend the kind of work that I'm [32:12] able to do do the design work that [32:16] previously I would have had to request a [32:17] human for etc. um then like you've [32:23] supercharged like one one individual, [32:26] right? You might you might not have [32:28] completely like eviscerated an entire [32:32] like multi-quarter process like [32:34] compliance process that everyone hates [32:37] but you've like instilled the kernel of [32:40] like creativity and sort of like [32:42] autonomy within like the individuals of [32:45] your organization. So starting there and [32:47] starting by just like getting your [32:50] employ individual employees to raise a [32:52] ceiling of what they can do creatively [32:54] and like what they can ship in isolation [32:57] is the first starting point and then [32:59] from there you can start working on [33:00] these like multi- multi- teamam like [33:03] mega processes and start raising the [33:06] ceiling of complexity there. But like [33:08] there is a huge amount of value that's [33:10] unlocked simply by making everybody feel [33:13] like they have their own power to [33:15] develop products as sort of like a one [33:18] person startup. [33:19] >> Like you have a bunch of one person [33:20] startups inside a large company [33:22] basically. [33:22] >> Yeah. Yeah. [33:23] >> Uh that's interesting. Okay. Yeah. So [33:25] basically um and and and like do you [33:27] just give do you just let anyone make [33:29] their own individual agent or do you [33:30] kind of like I think it's probably best [33:31] practice to actually give some like [33:33] spotlight examples, right, of people who [33:35] actually know what they're doing, you [33:36] know? [33:37] >> Yeah. I I think giving people templates [33:39] and then letting them iterate freely off [33:40] those templates is always a good place. [33:43] Um you know you avoid the writer's block [33:46] of like what do I do? Um and but then [33:48] you give them the creativity to iterate. [33:50] >> Cool. This is very good. And and I think [33:52] getting the agent into the hands of [33:54] actual users qu quickly maybe not all [33:56] users but at least like some beta users [33:59] you know that's kind of where the rubber [34:00] meets the road right. So like like don't [34:02] go too crazy on evals before you even [34:03] get in the hands of any users like do do [34:05] yeah just that. Yeah. The vibe testing [34:08] is honestly the most important first [34:11] step. And at a certain point, you [34:14] uh outgrow the vibe testing because you [34:16] can't really like do uh you know [34:19] aggregate vibe signals at at scale and [34:21] that's when you do eval. [34:23] >> Okay. Okay. The it's it's it's it's hard [34:26] to present the vibe testing. I I just a [34:28] way to quantify the vibes. I guess you [34:30] can have a bunch of quotes and stuff but [34:32] it's hard. [34:32] >> Yeah. [34:33] >> Yeah. [34:34] >> Um Okay, cool. So I guess last question. [34:35] So, I mean, this stuff is moving so [34:37] fast. Where do you think this stuff is [34:39] going to go? Like, let's just say like [34:40] three or six months from now. Like, you [34:41] think we're gonna like, you know, like [34:43] before I go to bed and be like, "Hey, [34:45] just just take care of everything for me [34:46] and [laughter] then when I wake up, it's [34:48] is done or Yeah, [34:50] >> honestly, uh, to some degree, these [34:53] longunning agents are kind of doing [34:54] that. Like, we set them tasks overnight [34:56] and then we wake up and, you know, [34:58] backlog is resolved and bugs are are [35:00] squashed." Um but I think I so I I think [35:05] that we will see the workday like [35:09] becoming sort of uh sort of the the [35:13] limits of what we can achieve will [35:14] really be based off of how much we can [35:16] delegate at once um more so than like [35:19] what our like personal capacities are [35:21] because we are going to be able to [35:24] increasingly um lean on these agents as [35:27] partners. Um, so that's one thing. I [35:30] think like on an industry level, what [35:33] I'm really fascinated by is I'm starting [35:35] to see that vertical SAS is sort of just [35:38] like becoming increasingly specialized. [35:41] And so the idea that you would have like [35:44] an accounting agent versus a uh healthc [35:48] care agent like that is starting to [35:50] become like more and more and more [35:52] narrow as [35:54] people realize that like uh you know the [35:58] models are getting smarter and so like [36:01] broad domain expertise is is sort of [36:03] there and so like the real value ad is [36:06] like like these incredibly specific and [36:09] and niche use cases. And so, um, it's [36:13] really interesting because, uh, I I [36:16] think that like getting the right agent [36:19] to do the job now is getting so, uh, [36:24] specialized and tailored that having um [36:27] the really the shared thing now is like [36:29] the context patterns and the task [36:32] orchestration patterns more so than like [36:35] okay, this is a canonical way to build [36:37] like a finance agent or like a [36:39] healthcare agent. [36:40] So for example, like maybe instead of a [36:42] general accounting agent, it's like a [36:43] accounting agent for like solarreneurs [36:44] or something like like just like more. [36:46] >> Yeah. Yeah. Everything is just becoming [36:48] incredibly specialized. um particularly [36:50] as as uh people are able to build [36:54] products um you know for themselves and [36:57] then and scale them externally like so [37:00] so what we're seeing is as as you think [37:04] of as you realize that you can now build [37:07] software for like these hyperspecific [37:09] use cases people are scaling those [37:10] things and so that means that the kind [37:12] of products that we're seeing [37:14] distributed are increasingly [37:16] verticalized [37:16] >> yeah I I think this is actually really [37:18] interesting I actually have more [37:19] questions about this just real quick. Um [37:22] because I feel like I can build a [37:24] product like a AI product for accounting [37:26] for solarreneurs or something, but I [37:28] feel like someone can just build a scale [37:29] that's just as good. Like [laughter] [37:31] it's kind of hard to think about like a [37:33] long-standing SAS product, you know, you [37:34] know, like around this. [37:36] >> Yeah. I don't really have a good answer [37:38] but yeah [37:40] >> I think that where the products that [37:42] will sort of survive this transformation [37:45] are the ones that meet their users where [37:47] there are where their workflows are [37:49] right so it's about being [37:52] one hyper specific to like the tasks at [37:55] hand and adaptable um I guess beyond [37:58] those specific tasks but then two like [38:00] being where exactly where you need it to [38:03] be and so like um that always on kind [38:06] agent pattern is definitely important [38:08] because uh you know you you you want [38:11] that agent to pop up at the right time [38:13] but it also needs to be in like the [38:14] discoverable place where you would [38:16] expect those workflows to be handled [38:18] >> increasingly that is just like in clock [38:19] mode or some of these apps right like I [38:21] don't want to navigate to a website and [38:22] like fill a bunch of forms like I feel [38:24] like I have I want to bring my personal [38:26] agent like with all my contacts and I [38:28] don't want to get it to go talk to [38:30] someone's accounting agent just like go [38:31] go figure it out [laughter] [38:33] you know what I mean [38:34] >> yeah I wherever you're wherever your [38:36] your work lives and so for like you know [38:41] for a lot of engineering teams that will [38:43] be increasingly in cloud code and and [38:45] everyone sort of is becoming engineering [38:46] team um but also like just like you know [38:51] even if we think about uh Verscell's [38:53] chat SDK and how um you know they had [38:57] the sort of vision to realize that like [39:00] everything is chat now because like [39:02] agents interact best in chat and like [39:05] these sort of like um interactions with [39:07] our colleagues are getting like more and [39:09] more compressed because like the the the [39:11] speed that we're iterating at is is so [39:13] much faster, right? And so [39:16] like I do think that a lot of these I [39:19] mean it it sounds kind of basic but like [39:21] these sort to to meet users where you [39:23] are you have to it has to live in cloud [39:24] code and it has to live in chat. [39:26] >> Yeah. It's got to live in chat and the [39:27] chat has to be connected to all your [39:28] personal contacts. It's not just like a [39:30] random chat window on web website you [39:32] know. [39:33] >> Yeah. Well, fascinating world, [39:35] fascinating world of living. [laughter] [39:37] Yeah. Um but uh but this is super [39:39] helpful, Jess. Um where can people um [39:42] find you or learn more about cloud [39:44] manager agents? [39:45] >> Yeah. Um so definitely use the cloud [39:48] code skill to learn more about cloud [39:50] manage agents. We also have traditional [39:52] arteal documentation that I I assure you [39:55] humans still read um at [39:57] platform.cloud.com. [39:59] Uh, and uh, I'm Jess double_yan on [40:04] Twitter if anyone wants to reach me. [40:06] >> All right, Jess. Well, this has been a [40:07] fascinating conversation. I hope the [40:08] agents don't take over everything, but I [40:10] look forward to having them save time on [40:12] work we don't want to do. Yeah. Thanks [40:14] so much.