OpenAI Pushes Computer-Use Agents Past Human Speed to Bypass APIs

OpenAI推进计算机操作智能体超越人类速度以绕过API 随着智能体执行速度超越人类交互速度，图形用户界面将取代定制API，成为贯穿传统企业软件与云端设备的通用集成层。 OpenAI负责人Ari Weinstein和James Sun详细阐述了他们最新发布的模型（如Astra）如何推动计算机操作超越普通人类速度，迈向跨图形界面的超人类、高可靠操作。他们审视了核心的企业应用场景——从连接陈旧的ERP、对账财务总账到自动化采购，同时详述了确定性沙盒、审批触发器以及Guardian/Auto Approve模型等治理工具。在探讨消费者端动态时，他们讨论了化解跨碎片化会话的界面摩擦、通过Dots等系统涌现的主动式辅助，以及为何能操作计算机的智能体将与精心制作的以人为本的软件共存，而非使所有图形用户界面走向淘汰。最终，他们认为计算机操作已迎来关键的扩展拐点，为创始人构建端到端垂直解决方案提供了巨大的机遇。 OpenAI报告称，能操作计算机的智能体速度已超越普通人类，并接近超人类水平，使屏幕和操作系统交互成为替代定制API的通用方案。 (0:00) GPT-6 Astra的发布带来了每项任务更少的思考标记（thinking tokens）、对提示词注入更高的抵御能力，以及Guardian等自动化防护栏系统，以安全管控企业自主操作。 (1:21) 企业应用重点在于连接缺少现代API且已有15至20年历史的传统软件，实现采购、财务对账和CRM数据管理等重复工作流的自动化。 (3:24) OpenAI计划向移动用户扩展无头（headless）、云托管的计算机环境，旨在将复杂的桌面级智能体执行能力带给超过十亿的目标受众。 (41:11)

OpenAI engineers Ari Weinstein and James Sun assert that computer-using AI agents have crossed human execution speeds and are nearing superhuman performance, operating standard graphical interfaces directly rather than requiring custom API connectors. Technical Strides in Astra and Universal UI Interfaces (1:21) OpenAI's latest model, Astra, significantly reduces the number of thinking tokens required to finish computer-driven tasks, enabling lower execution costs and faster turnaround. Weinstein emphasizes that practical user trust depends on tiny statistical improvements, pointing out that a jump from 97% to 99% reliability determines whether an operator will allow an agent to run unattended. Internal testing demonstrated this stability by having Astra navigate complex game environments like Fallout over 30 hours of continuous multi-step planning. In product demonstrations, Astra exhibited simultaneous multimodal execution, such as holding voice conversations while generating freehand sketches in Microsoft Paint. Rather than relying on dedicated Model Context Protocol (MCP) servers or bespoke APIs, computer use treats standard graphical user interfaces as a universal fallback. Weinstein explains that this design lets models interact out of the box with any software built over the last several decades. When tasks lack explicit detail, Astra adapts by asking clarifying questions or inferring intent from ambient context. While current systems predominantly handle asynchronous background tasks, Weinstein projects that crossing expert human speeds will soon trigger a sea change where delegating digital work becomes the default user behavior. Enterprise Guardrails and Proactive Infrastructure Triaging (5:43) Because enterprises manage mission-critical data, OpenAI pairs rapid execution with deterministic administrative boundaries. Sun details the "Guardian" (or Auto Approve) model, an inspection layer that assesses each planned agent action against safety policies before execution. IT administrators can enforce deterministic sandboxes that strictly allow or block specific desktop applications and web domains. Configurable human-in-the-loop triggers mandate explicit manual authorization whenever an agent attempts high-risk actions, such as uploading files, downloading content, or navigating to unfamiliar web domains. To build user comfort, client interfaces feature an animated cursor that renders agent interactions step by step. Internally, OpenAI deploys "Dots," an agent workflow demonstrating how models can move beyond scheduled automation by exercising context-aware judgment. Traditional cron bots often flood communication channels with unhelpful noise, but Dots uses open-ended guidelines to filter for urgent messages across reporting lines and critical product feedback. In developer triage, Dots operates across local desktops and isolated cloud virtual machines to flag bugs reported in Slack, log directly into Sentry to pull error stacks, pipe the diagnostic output into Codex for root-cause analysis, and deliver a concise brief directly to engineering teams. Legacy Software Automation Versus Bespoke Native Tools (20:04) The commercial demand for computer use splits between bridging archaic infrastructure and crafting consumer experiences. Sun explains that corporate IT environments remain saturated with 15- to 40-year-old legacy ERPs and CRMs lacking modern interfaces or APIs, making them notoriously inefficient for human staff. Computer-using agents capture immediate value by automating tedious multi-step enterprise workflows—such as cross-referencing isolated ledgers during month-end financial reconciliations, copying prospective sales leads into Salesforce, or manually transferring 1,000-row procurement spreadsheets into legacy purchasing databases. At the same time, Sun and Weinstein reject the idea that graphical software will dissolve entirely into invisible background APIs. Humans still seek tactile engagement, direct manipulation, and audit logs that clearly document operational decisions, such as invoice discounting terms. Modern AI makes software creation so inexpensive that developers can build specialized tools for small groups or single individuals. This dynamic creates a hybrid software model: developers build intuitive, modern front ends for direct human manipulation that quietly instruct computer-using agents to drive clunky enterprise back ends. Adoption Friction, Web Economics, and Startup Opportunities (11:24) Widespread adoption faces technical and economic hurdles. On the open web, platforms relying on ad revenue face business friction because non-human agents do not generate billable page impressions, prompting OpenAI to develop protocols like WebMCP to bridge automated interaction with existing publisher models. Operational friction also stems from fragmented user sessions across disconnected Chrome extensions, client browser windows, and isolated cloud virtual machines. Sun notes that consolidating these states into unified, cross-device authentication is an immediate engineering priority to ensure computer use works smoothly for non-technical users. Despite the rapid pace of foundational model releases, Sun argues that startups possess strong moats. Large frontier labs lack the operational bandwidth to address every vertical industry. Founders can build durable businesses by coupling computer-use capabilities with deep regulatory knowledge, specialized customer support, and tailored operational workflows. An example includes automated restaurant management platforms that combine visual storage room inspection with automated web grocery purchasing and ERP staff scheduling. OpenAI plans to extend computer-using agents beyond desktop environments by deploying headless instances in the cloud, allowing mobile phone users to dispatch complex desktop computing tasks. However, long-term consumer adoption will require shifting user habits from manual interaction to proactive delegation, alongside resolving publisher monetization conflicts across the open web.

OpenAI的Ari Weinstein与James Sun在a16z播客中指出，计算机使用（Computer Use）技术已跨过超越普通人操作速度的临界点，并正迈向超人类水平。该能力将操作系统和屏幕视为通用接口，既能直接盘活缺乏现代API的企业遗留软件，也正推动人机协作模式向主动式和云端化演进。 技术跃迁：从思维标记削减到超人类执行速度 (1:21) Ari Weinstein表示，相较于早期的Soul模型，OpenAI最新的Astra模型在执行相同工作流时所需的思考标记（thinking tokens）显著减少，从而直接提升了端到端操作的速度与流畅度。在可靠性方面，系统从97%提升至99%的细微增益，在实际体验中决定了用户是否敢于放手让代理独立执行任务。此外，内部基准测试显示Astra在复杂游戏中表现出色，能在约30小时内通关《辐射》（Fallout）等开放世界游戏，展示了其长周期的多步骤规划与界面交互能力。 Weinstein强调，计算机使用代理已超越普通人类的操作速度，目前正逼近超越熟练专家用户的“超人类”临界点。一旦代理在速度上系统性压倒手动操作，用户的心理预期将发生逆转，即默认将绝大多数日常数字任务委托给AI处理。为了兼顾不同的业务场景与成本敏感度，OpenAI正通过多层级的模型梯队支持差异化的质量与成本诉求，使非同步后台批处理与高要求的即时交互均能获得适配。 双轨落地：企业遗留系统改造与消费级习惯重塑 (3:24) 企业环境与消费场景呈现出截然不同的落地逻辑。James Sun分析认为，大型企业普遍充斥着使用达15至20年且缺乏API或MCP连接器的遗留ERP、CRM与内部系统，普通员工上手极其困难。计算机使用代理将视觉图形界面作为一种通用回退API，企业无需推倒重构即可直接自动化处理跨表格采购、跨系统账目对账和销售线索录入等枯燥流程。这类“枯燥流程”往往蕴藏着极高的商业回报。 而在消费端，Weinstein指出核心挑战在于打破固有的人类操作习惯并建立信任。当前代理能协助完成退订垃圾邮件、处理政府表格或自动化比价等繁琐事务，甚至支持如个性化健身计划编排等过往因耗时过长而放弃的个人项目。与企业追求严格审计不同，消费级交互更强调消除日常数字摩擦并带来愉悦感，但在网页生态中，代理跳过广告展示的行为也正与依赖展示广告收入的数字出版商产生商业模式层面的摩擦。 信任架构：确定性沙箱管控与可视化交互感知 (10:01) 随着代理运行速度加快，人工实时监控每一个按键和点击已不再现实，企业必须依靠确定性的安全治理机制。Sun介绍，OpenAI在Astra中引入了Guardian系统（应用内亦称为Auto Approve），通过专门的模型对代理的每一步操作进行审查与风险判定。企业管理员可以配置严格的确定性沙箱规则，明确限制代理可访问的应用名单和网络域名，并在代理触发文件上传、下载或跳转未知域名等高危操作时，强制要求人工介入审批。 在用户感知与心理层面，界面交互设计对缓解失控焦虑起到了关键作用。客户端通过显式的动画虚拟光标，实时向用户呈现代理的每一步移动与操作路径，使原本黑盒化的自动化过程变得透明可视。Sun和Weinstein指出，这种兼具确定性权限拦截、第三方基准评测以及可观测光标反馈的设计，能够让企业和个人逐步建立起对代理无监督运行的长期信任。 从被动响应转向主动执行与全平台云端化 (5:43) 在工作流范式上，代理正从传统的“提示-响应”模式演进为具备定性判断力的主动助理。OpenAI内部应用的“Dots”展示了这种演变：它不仅能主动发现邮件中未同步的外部日程邀请并在Gmail中确认，还能在无需死板脚本监控的情况下，依据上下文理解自动抓取Slack中的严重故障反馈，调动云端虚拟机登录Sentry拉取错误日志，再将诊断数据交由Codex排查根本原因并向工程师汇报。 Sun同时指出，当前计算机使用面临的工程瓶颈在于环境的割裂：本地浏览器扩展、桌面客户端内置浏览器与云端沙箱之间的凭证、Cookie和会话状态尚未打通，导致非技术用户体验受阻。团队下一步的重点是消除这些实现细节的裂缝，使跨设备会话保持统一连贯。同时，由于全球绝大部分普通用户以手机为主，通过在云端托管无头（headless）计算机环境，移动端用户未来亦可远程调用完整的桌面级运算与操作能力，从而将潜在受众拓展至十亿级规模。 创业壁垒与人机协作软件的新形态 (25:57) 面对前沿模型实验室的快速迭代，Sun基于前创业者的经历指出，即便基础模型厂商规模持续扩大，其内部工程资源依然有限，无法通吃所有垂直细分场景。在非科技传统行业（如餐饮、零售、供应链等），将视觉识别、采购网站操作与老旧排班ERP串联成完整解决方案依然存在巨大的市场空白，深入行业合规、掌握定制销售渠道并快速交付的垂直创作者仍具备坚固的护城河。 关于未来软件形态，Weinstein与Sun一致认为，未来的软件生态不会演变为纯粹由代理互相调用的“无面孔”冷冰世界。直接操纵（Direct Manipulation）符合人类的天性本能，AI工具的普及大幅降低了小众软件的开发门槛，催生了大量针对单人或小团队的高粘性定制界面。未来的典型架构将是由精致友好的现代前端负责与人类进行触觉交互并呈现审计轨迹，底层则调度计算机使用代理去操纵笨重繁琐的遗留后端，实现人机协同共存。 计算机使用技术正迎来消费级与企业级双重爆发的拐点，但其全面普及仍受制于跨平台会话割裂、用户主动委托习惯的欠缺，以及开放网页广告商业模式与自动化抓取之间的利益冲突。如何在提升代理自主推理速度的同时，维持确定性的安全边界并保护人类直接操纵的软件乐趣，仍是全行业演进中的核心课题。

[0:00] Ari Weinstein: We've gotten computer use now to a place where it's faster than the average human. And I think we're getting really close where it is superhuman. [40:02] Ari Weinstein: I'm really excited about a world where the agent can sort of adapt to you instead of you adapting to it.

[0:00] We've gotten computer use now to a place [0:01] where it's faster than the average [0:03] human. And I think we're getting really [0:05] close where it is superhuman. [0:07] >> What sorts of tasks do you think are [0:09] ready to hand over to the agent versus [0:10] the ones like okay it still requires [0:12] some checking and correcting [0:14] >> computer using agents now since Astra is [0:16] now capable of doing anything that a [0:17] person would do. We think a lot about [0:19] what new consumer experiences can we [0:20] enable with a faster better model. And [0:22] then on the enterprise side that's kind [0:24] of naturally why the safety becomes more [0:25] interesting. How do we give the ability [0:27] to control the amount of human the loop [0:30] there is and then the auditability? It's [0:32] unlike any other thing that we've built [0:34] where the developer has to do some work [0:36] to adopt it. Someone has to build a [0:37] connector MCP server. You know, computer [0:39] use works with everything. Now we have [0:40] like capable models. They're getting [0:42] cheap enough to run at scale. What [0:44] should companies think of automating? [0:46] >> I think of it as in two big categories. [0:48] The first category is like [0:51] >> Welcome to the A16Z podcast. I'm Sema [0:54] Amble. I'm joined by my colleague [0:56] Fbritzio Saraphiniti. In August, we [0:58] wrote about uh computer using agents and [1:02] how they're starting to do the real work [1:03] inside companies. Uh today I'm joined by [1:06] Ari Weinstein and James Sun from OpenAI [1:09] um who are going to help unpack what is [1:11] going on with computer using agents from [1:13] their perspective and uh share some [1:15] thoughts on the future. Ari James, [1:16] thanks for joining us. Of [1:17] >> course, happy to be here. Yeah, thanks [1:18] for having us. Ari, kick it off and tell [1:21] us a little bit about um what in Astra [1:24] has made computer using agents move [1:26] forward and what what's really unlocked. [1:28] >> Yeah. Um so Astra's an amazing model. [1:32] It's good at so many things. Our [1:35] computer use models were already so good [1:37] before Astra, you know, like Soul itself [1:39] is amazing at computer use. So um Astra [1:43] really helps with a few things around [1:45] the edges in my sort of anecdotal [1:47] experience. One is it is a little bit [1:50] faster on average because it uses fewer [1:52] thinking tokens to do the same task. [1:54] It's better at certain kinds of creative [1:56] work like for example we had this demo [1:58] where um in our launch video where you [2:01] were talking to it and it's drawing [2:02] things on Microsoft Paint. Um it's way [2:04] better at that sort of thing which I [2:05] think probably transfers to other kinds [2:06] of creative tasks also. And then the [2:09] last thing I find is it's just more [2:11] reliable around the edges. Like you know [2:13] it's kind of I feel like I trust it a [2:15] little more. like it's sort of the kind [2:16] of thing where there's a little [2:18] difference that is hard to perceive but [2:20] really important between something being [2:21] 99% reliable and 97% reliable. And I [2:24] feel like it's sort of just a little a [2:26] little higher in a way that feels uh [2:28] pretty pretty concrete. [2:29] >> Yeah. And I think adding on to some of [2:31] those things, Astra is also our most [2:32] aligned model. Yeah. Right. And so, you [2:34] know, talking about trust a little bit. [2:36] Yeah. [2:36] >> Um you know, if you look at the system [2:38] card for it, it is much better at being [2:40] resistant to prompt injection. um it's [2:42] much better at not taking any kind of um [2:45] unintended actions. And so when you [2:47] think about it for like enterprise work, [2:49] especially for companies using it for [2:51] sensitive things, whether it's like [2:52] procurement or you know talking to [2:54] vendors or talking to customers, it's [2:57] going to be just a you're going to feel [2:59] a lot safer kind of using it and that's [3:00] really really exciting. And then so some [3:02] some fun things on the creative side. I [3:04] think internally we had um you know [3:06] Astra play video games against uh you [3:08] know 5.6 and it wins every time. And so [3:12] there are some cool tweets that people [3:13] have made where, you know, Astro is able [3:15] to beat Fallout in like 30 hours or [3:17] something like that. And so it's also [3:19] just really amazing at playing video [3:20] games. So I don't know that probably [3:21] translates to something, but yeah, it's [3:24] very exciting. [3:24] >> Maybe just to double click on that [3:26] point, um consumer versus enterprise use [3:29] cases of metering agents, very uh very [3:32] different implications around kind of [3:34] like all the scaffolding around I would [3:36] say the um computer using agents. How do [3:39] you guys think about where things are [3:41] more exciting between consumer and [3:43] enterprise if if you were to pick? [3:45] >> I think you know our goal at OpenAI is [3:47] really about building products that's [3:48] useful for everyone. And so um you know [3:51] actually it's interesting because Ari is [3:52] definitely more of a consumer guy here [3:53] and I spend a little bit more time on [3:55] the enterprise side. Um I think the [3:57] consumer is always the place where we [3:58] try to push the frontier on what's [4:00] possible. uh you know uh with Astra like [4:02] Ari mentioned there is it's a much [4:04] faster compared to the previous models [4:06] and so we think a lot about like what [4:08] new consumer experiences can we enable [4:10] with a faster better model and then on [4:12] the enterprise side that's kind of [4:13] naturally why the safety becomes more [4:14] interesting because I think the [4:16] enterprise whenever we talk to large [4:18] companies I think the use cases for [4:20] comput is actually fairly obvious right [4:22] inside enterprises there's lots of [4:24] legacy software that doesn't have any [4:25] MCP tools they are software that's been [4:29] used for like 15 20 years and so there's [4:32] and and and a lot of times it's actually [4:33] kind of hard to use for a normal person. [4:35] right? Lots of like UI that is like from [4:37] the last, you know, decade. And so you [4:39] don't even know how to operate some of [4:41] the key key key things. And so we've [4:43] seen companies build skills that kind of [4:44] teach the model to be able to use them [4:46] better. And so I think maybe the way the [4:48] way to think about it is enterprise is [4:50] where, you know, the capabilities have [4:53] been there maybe for a little bit longer [4:54] and the companies are catching up trying [4:56] to understand how to deal with it and [4:58] consumers where we're trying to figure [4:59] out the next types of use cases that's [5:01] not maybe possible today with software. [5:04] Maybe you can share more. [5:05] >> Yeah, I mean the consumer use cases are [5:08] so fun for me because it's kind of like [5:11] how can you put your agent to work doing [5:13] the things in your daily life that are [5:15] so annoying and save you tons of time [5:16] and just have more time to focus on the [5:18] things that [5:19] >> um that you're excited about. Um I think [5:20] it's been really fun to see dot uh where [5:24] you know each dot gets its own computer [5:26] and you can ask it to do things for you [5:27] in the cloud. Um it gets a full web [5:30] browser. It can buy things for you. can [5:33] um you know book haircut appointments [5:34] and uh you know anything else you wanted [5:37] to do. So um so that that's been really [5:39] fun and I think there's a lot more on [5:41] the horizon. That's going to be really [5:42] fun. [5:43] >> Yeah. Know that was going to be my [5:44] question on on on dots. I think James [5:47] you probably use a dot to walk through [5:50] product feedback. It would be cool if [5:52] you could share share more about it or [5:54] I'm just so curious to know on on your [5:56] end guys like how are you using dots and [5:58] like how does computer use play into [6:01] kind of what you can do with them. [6:03] >> Yeah, I mean I think dots is really cool [6:05] for a few reasons, right? One is like [6:07] really proactive and so um I've been [6:10] very surprised a lot of times when in [6:12] both my personal life and work when dots [6:14] will just flag something that um I just [6:18] completely slip through the cracks, [6:19] right? Just the other day. So internally [6:21] like on our our email system whenever we [6:23] get an external email for an invite it [6:26] doesn't actually show up on our [6:27] calendars and so I end up missing a [6:28] bunch of like external meetings and so [6:30] dots now is like catching these things [6:31] for me. It's like oh there's this invite [6:33] that's in your inbox. It's not on your [6:35] calendar. Do you want me to like accept [6:36] it in your Gmail so it shows up? So [6:38] that's been a lifesaver. It's a small [6:39] thing but it matters a lot. Um, but I [6:41] also have it proactively kind of looking [6:43] at I think it's really good at [6:44] proactively looking for tasks where it's [6:47] the shape of it is not super uh, let's [6:51] say concrete every single time. So, an [6:53] example that I also haven't set up to do [6:54] is um we use Slack internally and [6:57] there's there's a ton of messages that's [6:59] going on at all times and uh I used to [7:01] have this Slack job uh using automations [7:04] and codecs that's like every hour or [7:06] every 30 minutes go and try to pull [7:08] things and I've just ended up tuning out [7:10] a lot of the notifications because it's [7:12] just like sometimes useful sometimes [7:14] not. With DOTS, I can give kind of a [7:15] high level goal. It's like hey if [7:17] anybody in my reporting chain says [7:18] anything flag it to me, right? if [7:20] anybody who's giving some critical [7:22] feedback, use your judgment uh about [7:24] certain products, go for it. And then [7:26] because it has a cloud computer, um you [7:28] know, I've had this pretty cool workflow [7:30] where when someone reports, they usually [7:32] have like a sentry log. I could have it [7:33] go use the cloud computer, um you know, [7:36] actually drive my local local browser [7:39] while the DOS's going to control both [7:41] your local computer as well as the cloud [7:42] computer. Log into Sentry, pull the [7:45] logs, analyze it, give me feedback. And [7:47] so it's saving that entire loop of [7:49] figuring out which feedback is [7:50] important, looking at it, then going to [7:53] Sentry, downloading the logs, feeding [7:54] into codecs, asking for thoughts, and [7:56] all that can now just kind of be [7:58] automated. And so, um, yeah, DOS is [8:00] great for for at least these proactive [8:02] and kind of longunning tasks where [8:03] they're not the same shape every single [8:05] time. [8:05] >> We talked a little bit about [8:07] reliability, speed, token usage, and [8:09] efficiency. How do you guys think about [8:10] balancing all these things uh as you uh [8:13] as you keep making improvements on what [8:15] what you're releasing? I think we're [8:17] really excited about computer use at [8:19] OpenAI and we've invested in it a lot [8:21] and we've gotten to a place where I [8:23] think we have the fastest, cheapest and [8:26] most reliable computer use product. Um, [8:28] and I think there's actually so much [8:31] further we can take that. Um, we spent a [8:33] lot of time looking at speed. You know, [8:35] speed is really interesting because on [8:37] one hand the power of computer use is it [8:39] can do tasks in the background. So in [8:41] some cases it doesn't matter how long it [8:42] takes because you can be going going and [8:44] doing other things while it's [8:44] accomplishing the task for you. But also [8:46] sometimes you want to watch it to make [8:48] sure it does it correctly. You need to [8:50] wait for it because it needs to complete [8:51] this before something else can happen. [8:53] And so the speed is so important because [8:57] um you know you don't want to get this [8:58] feeling like oh I could have just done [8:59] that myself and it would have been so [9:01] much easier. So um you know we've gotten [9:03] computer use now to a place where it's [9:04] faster than the average human at using a [9:07] computer. And I think we're getting [9:09] really close to a place where it is [9:11] superhuman, meaning it is faster than an [9:14] expert computer user like probably one [9:16] of us at this table. Um, and I think [9:18] that's going to actually be a sea change [9:20] when that happens. I think it's going to [9:21] change the way we all interact with [9:23] computers because all of a sudden [9:24] anything that you you know that that you [9:27] could do manually, it could do faster. [9:28] So unless you really want to do it [9:30] yourself for some reason, which I think [9:31] actually there may be many reasons [9:32] people would want to do that, but in a [9:34] default case, you probably just want the [9:35] agent to do it for you. And I think [9:36] that's going to let us build all kinds [9:37] of cool user experiences. So speed is [9:39] something we think about a lot. And then [9:41] obviously we have a strategy of offering [9:43] multiple different models at multiple [9:44] different you know qualities and price [9:46] points. And so for different use cases [9:48] you may want to use a different model. [9:50] Um at the same time our um our less [9:53] expensive models are getting much better [9:54] at computer use over time. And so I [9:56] think there will actually be more [9:58] variation over time in which models [9:59] people want to use for which tasks. [10:01] adding on saying like I think another [10:03] thought is like how do we give customers [10:05] you know control over the [10:08] safety reliability autonomy versus speed [10:11] right like if you have like you know [10:14] when when the model is slow you can kind [10:15] of watch it do stuff but then you're [10:17] limited to that to the task that you can [10:19] do because you can't just sit here [10:20] watching the thing doing it like it's [10:22] actually less effective right because [10:23] it's slower than than you um but once [10:26] you can get to a point where you can no [10:28] longer observe and audit everything it's [10:30] doing um how do we give especially [10:32] enterprises and businesses the ability [10:34] to control um the amount of human the [10:38] loop there is and then the auditability [10:39] right like I think a few months ago it's [10:41] part of access a few months ago it's [10:43] part of Astra you know we we kind of [10:45] launched a set of enterprise controls [10:46] that let um businesses choose one like [10:49] how much human in the loop is there like [10:51] do they want the agent to automatically [10:53] approve everything we have this really [10:54] awesome system called guardian right or [10:56] auto I think auto approve in the app [10:58] that uh is a really smart model looking [11:01] at every single action that the computer [11:02] use agent is taking to decide if it's [11:05] safe it's if it's aligned so there's no [11:06] intent um a consumer might be like I [11:08] just want it to do everything and I [11:10] don't want it be asked for approval [11:12] whereas an enterprise is able to kind of [11:14] decide which type of actions actually go [11:16] through and so I think the the the [11:18] control part becomes more and more [11:20] important as our computing agents become [11:23] also superhuman and doing things [11:24] >> totally I think as you try to you know [11:26] roll out more enterprise use cases I [11:28] think employers They're excited about [11:30] the efficiency, but they're also, you [11:31] know, wary of like what is what are [11:34] what's sort of being unleashed. [11:35] >> Maybe even on the other side, [11:37] >> have you guys seen the response from [11:39] application software around okay, now [11:42] there's computer using agents accessing [11:44] my legacy workflow software and how do [11:47] you do you guys think at all about that? [11:49] Are you more focused on just the the [11:51] agent that's being used by the you know [11:53] consumer enterprise user? M yeah I think [11:56] there's like I think this problem [11:57] becomes really interesting on the open [11:58] web actually. I think when it's like [12:01] within software that is like approved [12:03] and used by the company it's actually [12:05] more straightforward because it's like [12:07] oh yeah 100% we want to have uh people [12:12] automate and do work in here that they [12:14] don't have to do tediously themselves. [12:16] But when it comes to the internet, I [12:17] think you kind of have a few different [12:18] camps of folks, right? One is we run a [12:22] our business model is not in line with [12:25] having agents use our product, right? [12:27] You think about advertisers, publishers [12:30] whose business model is entirely on [12:31] advertisement. If the agents using the [12:33] website for them, then where's the ad [12:35] revenue coming from? And I think most [12:37] places a agent's ad impression does not [12:40] actually count towards real uh revenue. [12:42] Um and so that's like an interesting [12:44] question that we're always working [12:44] through and working with our [12:45] partnerships team on. Um there's another [12:48] question kind of around um for the [12:50] companies who do really want to embrace [12:52] having computer use agents using their [12:53] sites. A lot of them are adopting kind [12:55] of standards that let humans or people [12:58] and agents to be able to use it [13:00] together. So like WebMCP is a standard [13:02] that we're really excited about. We we [13:04] added support for in the app browser. [13:06] And so you can kind of imagine these [13:08] scenarios where the agent is elevating [13:11] the experience of using the site, but [13:13] it's not entirely replacing it. So like [13:15] I think browsing the internet is still [13:17] very fun in my opinion, right? I still [13:18] like doing shopping. I still like [13:20] looking at all the hotels and flights [13:23] and places. And so, you know, you can [13:25] you can imagine places where um the the [13:28] agent does all the tedious work, but it [13:31] kind of carves out an user experience [13:34] where you're doing the browsing be like, [13:35] "Okay, now I want to finish the the [13:36] transaction and then put in my credit [13:38] card and do all this stuff, then you can [13:40] hand it back to the agent." And so, Web [13:41] MCP is kind of an interesting way where [13:42] your interface with the website is still [13:44] the site itself versus an API, but then [13:47] the person is kind of using it together. [13:49] And so, those those are some thoughts. [13:51] Yeah. Shifting gears, we want to double [13:52] click a little bit on what it means to [13:54] do the job or for get the end to end job [13:57] done. Um, we'd love to hear a little bit [14:00] from you guys around, [14:01] >> you know, what works well. How do you [14:03] guys define the right the right narrow [14:06] repeatable task uh ver you know and what [14:08] sorts of tasks do you think um are ready [14:10] to hand over to the agent versus the [14:12] ones like okay it still requires some [14:14] checking and correcting as the on the [14:16] user side. M I mean generally I found [14:19] the model to be pretty capable to do [14:21] anything that you can do on a computer [14:23] now like um and and you know one [14:25] exercise that I try to force myself to [14:27] do um is that whenever I have a task the [14:30] first thing I think about it is just can [14:31] I just have the agent do it now there [14:32] are obviously some sensitivities around [14:34] you know if I'm sending a message to [14:35] someone I want to feel like it's coming [14:37] from me um but in general I think the [14:39] way I think about it is the computer [14:41] using agents now since Astra um is now [14:44] capable of doing anything that a person [14:45] can do and uh there are questions around [14:50] I think before of like oh there are some [14:51] like squishiness around like specificity [14:54] and requirements I' I've generally found [14:56] that the model is getting pretty good at [14:58] whenever you're asking it to do [14:59] something and there's some vagueness it [15:01] will actually you know with Astra I [15:02] think we launched this like uh flexible [15:04] input right and so it will actually ask [15:05] you about hey like I want some details [15:07] and if you don't answer in time it'll [15:09] just keep going and then pull the [15:10] context that's necessary I think there [15:12] are some things around you know if you [15:14] have repeatable workflows it's nice to [15:16] be able to kind of define, you know, the [15:19] the requirements early, similar to like [15:20] writing a product spec. Um, but in [15:22] general, I I generally feel like unless [15:26] there is something incredibly sensitive [15:27] that you want to be in the loop for, I I [15:30] just reach for the computer model for [15:31] everything I I do now. I'm curious if [15:33] the same for you. [15:34] >> I Yeah, it is. I'm I'm trying to uh uh, [15:37] you know, build that impulse to to reach [15:39] for it because, you know, it's so good [15:41] and sometimes it's so easy to get locked [15:43] into the old way of doing things. Um I [15:46] um I think the really cool thing about [15:49] computer use as a technology is the [15:51] universality of it. You know it works [15:53] with every application. It's unlike any [15:56] other thing that we've built where you [15:58] know the developer has to do some work [16:00] to adopt it. Someone has to build a [16:01] connector MCP server. You know computer [16:03] use works with everything. And it works [16:06] by using the application the same way a [16:09] human uses it. Um, which I think is [16:11] something worth understanding if you [16:12] haven't spent much time with this [16:13] technology. It like it computer use uses [16:17] computers the same way a human does. It [16:19] makes the agent have all the [16:20] capabilities you do. Um, which I think [16:22] is so powerful. [16:23] >> Yeah. And I guess like adding on to Ari [16:25] saying like part of it is like I think [16:27] over time I've stopped thinking about it [16:29] as like am I using them on like a [16:31] computer use task or a non-computers [16:32] task. Frequently they're like connected, [16:34] right? Like the way we think about [16:36] computer use um is that like computer [16:38] use is not like a thing you pick up and [16:41] just say I want to do something with it. [16:42] It's really like the universal kind of [16:44] API that the model has to do whatever it [16:46] takes. And so if you have an ambitious [16:48] task and you want the agent to do it [16:50] likely the more complex it becomes the [16:53] more likely it will probably touch [16:55] computer use at some point in that [16:57] journey. Um, and so like maybe the only [17:00] time that I I don't reach for computer [17:03] use for some cases is if the task is so [17:06] simple that then it's like easier for me [17:08] to just go and like click two buttons [17:10] and do it. But you know as computer use [17:12] gets faster even faster then the [17:14] question becomes maybe I don't need to [17:15] do that I just you know drive voice and [17:17] it just uses the computer with me. Um so [17:19] those are some interesting thoughts [17:20] there too. Yeah. Do you think there's [17:22] anything you have to share with like an [17:25] enterprise user to get them comfortable [17:27] with that? Obviously, you guys are [17:28] totally pilled on computer use, but you [17:30] know, I think there's obviously like [17:32] hesitance sometimes on enterprise user [17:34] of like, okay, well, yes, the human [17:36] would click through it and probably do [17:37] it worse, but uh there is that feeling [17:40] of like there's accountability and and [17:42] you know, that person's trained and all [17:45] you know, all the classic kind of [17:46] rebuttals. I'm curious [17:48] >> have to convince people around. [17:50] >> For sure. lots lots of conversation with [17:51] enterprises kind of over the last little [17:53] bit and I think the the main thing is we [17:56] think about it as two layers right the [17:57] first layer is like how do we create [17:58] deterministic controls I think that's [18:00] really what gives um you know [18:02] enterprises a lot of feeling of safety [18:04] is when they can even though our model [18:06] is the most aligned ever and it's I [18:09] think better you know better than most [18:11] people at you know protecting against [18:12] prompt injections and things because [18:14] it's like just you know the intelligence [18:16] is so smart but um I think there is just [18:18] a feeling of safety when the enterprise [18:21] admins can basically say for certain [18:22] task or for certain data sources just [18:24] too sensitive to be used. Um and so we [18:27] see kind of enterprise so we launched [18:29] was two things right one is the ability [18:31] to be able to deterministically create a [18:33] sandbox around your agent. So being able [18:35] to say it can use certain apps it can't [18:38] use other apps it can use certain [18:39] websites and not other websites. And [18:41] then two is you know this kind of part [18:42] of a human in the loop like what type of [18:44] actions does it require a human to um be [18:48] interveneed? And so we might say, okay, [18:49] for anytime there's like a download and [18:51] upload, a person has to kind of approve [18:53] it. Anytime it goes to a new website, a [18:55] person has to approve it. Now, again, [18:56] these are all like dials an enterprise [18:58] can kind of go through. And so it's [19:00] really like that's one. And then two is [19:01] really just showing the enterprise [19:03] admins just how you know all the evals [19:06] that we've run internally around how [19:08] aligned these models are even compared [19:09] to like you know both internal and kind [19:11] of third party um evals. I think there's [19:14] also some baseline stuff that we do [19:15] across the board to just try to help [19:18] people build trust with these kinds of [19:20] products. You know, we know it's so new. [19:21] It's such a magical and sometimes alien [19:24] experience for people. And so, um, you [19:26] know, we put a lot of work into the [19:27] design of the product. So, you can [19:29] always see the cursor if you want to [19:31] watch it perform a task and it sort of [19:33] animates in a fun way so it's not boring [19:35] to watch. [19:36] Um, and then like Jame was was saying, [19:39] this is very, um, customizable for [19:41] enterprises, but also for a regular [19:42] user. You know, we ask you for [19:44] permission before using a new [19:45] application or website. Um, uh, [19:47] depending on which sort of modes you're [19:49] in. And, um, I think that helps people [19:51] sort of not feel like it's going to go [19:53] do some runaway stuff on their computer. [19:55] You kind of want to have this trust [19:56] building mode moment where you figure [19:58] out what it feels like and what you um, [20:00] want it to do for you and then um, and [20:02] then you sort of feel more comfortable [20:03] over time. The Codex cursor is so cute. [20:06] I love [20:07] >> it's like such a I know it's a small [20:09] detail, but like watching it use the [20:11] computer gives you like confidence and [20:12] it makes the experience nice. So I think [20:14] there's a kind of like a user experience [20:16] point. Uh that's that's worth uh uh [20:20] that's definitely worth investing on. [20:22] And I guess I'm curious like in in a [20:24] broader sense like okay now we have like [20:26] capable models [20:28] >> they're getting like cheap enough to run [20:30] at scale. [20:32] In your view, what should kind of [20:34] companies think of automating? Like what [20:36] would you what where do you see the most [20:38] promising kind of use cases within [20:42] enterprises? Are there some workflows? [20:44] Maybe something around the ERP, maybe [20:46] something around kind of u how like [20:49] again humans and agents can work [20:50] together to get tasks done. But like if [20:52] you had to like pick where where are you [20:54] most excited to see your models perform [20:57] uh in the world like [20:59] >> Yeah, I guess like from my perspective, [21:00] I'd love to hear ours. It's like two two [21:02] I think of it as in two big categories. [21:04] The first category is like automating [21:06] you know tedious workflows. That's like [21:08] the more boring one and I'll give us [21:09] some boring examples like [21:11] >> boring boring as opportunity. [21:13] >> Opportunity. Yeah, exactly. Um like ERPs [21:15] is a really big one. Um and so all the [21:18] all the acronyms you can think of ERPs, [21:19] CRM, [21:20] >> all the systems of record. [21:21] >> Yeah. Every single system of record, [21:22] right? And so we know we've seen folks [21:24] you know procurement software like you [21:26] know you're going to vendors and every [21:28] year you're ordering a bunch of lab [21:30] equipment uh instead of having this [21:32] giant spreadsheet with like a thousand [21:34] rows that then the ops team so it's like [21:37] you know work companies like normally [21:40] what they have done is you they have [21:41] this giant spreadsheet that gets filled [21:43] out by the teams themselves. Then an ops [21:45] team takes that giant spreadsheet puts [21:46] it into an internal you know procurement [21:49] software and then that gets like you [21:52] know finalized as a bunch of lines and [21:55] then a sales team then goes procurement [21:57] team then takes that giant software [22:00] output and then goes out and reaches out [22:01] to vendors. um computers can automate [22:03] all of that, right? And so like [22:04] procurement is a big one we see. Um [22:06] finance is another really interesting [22:08] category we've seen and so uh you know [22:11] there's a lot of different ledgers [22:13] within a company right and so you have [22:15] like ledgers for expenses ledgers for [22:16] all these things and so at the end of [22:18] the month there's a lot of [22:18] reconciliation and a lot of these recon [22:21] financial software don't have APIs to be [22:23] able to quickly pull things. Um, and so [22:26] the finance team being able to have [22:27] computer use, look through all these [22:29] ledgers, reconcile, pull out anything [22:31] that's like interesting, um, to flag [22:33] either to a person or then do some [22:35] investigation. Also a very boring but [22:37] useful workflow. Um, for our sales team, [22:40] a lot of sales teams uses a lot of [22:42] different software for like transcripts, [22:44] for getting data, all these kind of [22:46] things. And so having computer use go [22:48] and do prospecting and then going and uh [22:52] putting that into a CRM or some kind of [22:55] like you know outreach software and then [22:58] all that. So all the system records is [23:00] the boring part. Um and then like market [23:02] research is another one that we see a [23:04] lot. Um and so you know in my free time [23:06] I've been building this like uh video [23:08] annotation software for for Bmon. I pull [23:10] out of Bmon and like I like to film [23:12] myself playing the game and then uh [23:14] having the agent kind of go and and look [23:16] at stuff and I was like okay how do I [23:18] make this thing really useful there's [23:19] some software out there for like pickle [23:21] ball for squash for tennis and so I just [23:24] had computer use go log in create an [23:26] account on these soft on these softwares [23:28] try it take screenshots give me feedback [23:30] and then try to build it back into the [23:32] app and so these are all kind of like [23:33] tedious workflows that could be done and [23:35] then I think the some of the more like [23:37] interesting creative work of like [23:38] enabling more ambitious work is like, [23:41] you know, we talked about um the model [23:43] now being really good with like visual [23:45] visual design. And so um circuit design, [23:48] you know, being able like architecture [23:50] work, hardware engineering, game design, [23:53] these are also areas that I I think [23:55] we're really excited about where Astro [23:57] can take. Um less, you know, less common [24:00] for enterprises to think about those [24:01] right now because they're thinking about [24:02] the boring opportunities. But I think as [24:04] folks understand the capabilities of [24:06] model more, that's where, um I think [24:07] we'll we'll kind of see more usage next. [24:09] But curious if any thoughts. Sorry. [24:11] >> Yeah, I think kind of similarly on the [24:13] consumer side, there's a lot of things [24:15] where it's sort of automating drudgery [24:17] or just doing things you don't want to [24:18] do. It's filling out DMV forms for you. [24:21] It's helping you uh unsubscribe from a [24:23] bunch of emails in your inbox. Um [24:25] >> and then I think there's also things [24:27] that are more like um I'm I'm hearing [24:29] from a lot of people stories of people [24:31] doing things that they just wouldn't [24:33] have time they wouldn't have had time or [24:34] wouldn't have thought to do otherwise. [24:36] like I don't know for me personally I'm [24:38] um uh maintaining spreadsheets about my [24:41] finance and spreadsheets about meal [24:42] planning and uh I have a friend who's a [24:44] personal trainer who's making customized [24:46] things for his clients and um you know [24:48] some of these things um would just be so [24:50] so time prohibitive to do in your in in [24:53] in your personal life. Um but this makes [24:55] it so easy. Um I use Apple Numbers which [24:57] is like a obscure spreadsheet program [24:59] but the model still knocks it out. [25:00] >> I don't think I've ever heard someone [25:02] use Apple Numbers. [25:03] >> It's a surprisingly great spreadsheet [25:04] software. Um and then I think there's a [25:07] third category of thing which I'm kind [25:09] of the most excited about which is just [25:11] starting and I think you know on the [25:14] horizon you know um which is really just [25:18] having a different way to use your [25:20] entire computer or interface with [25:22] software. I think you saw in our Astra [25:24] ad uh really cool examples of people [25:26] just talk to their computer with their [25:27] voice. It's such a natural way to use [25:29] your computer and I think over time [25:30] we're going to see the form factor of [25:32] software itself change due to this [25:34] technology and I'm I'm really excited [25:35] about that [25:36] >> on that point. Okay, form factor of [25:38] software changing. Um if imagine if [25:41] you're an enterprise user or you've used [25:43] you're used to using whatever clunky ERP [25:47] from you know 30 years ago. Um there's a [25:51] world in which you now, you know, [25:53] unleash uh a set of agents that just [25:55] operate that software and you don't [25:57] really care. Um there's another world in [25:59] which um folks are a new like AI native [26:04] player is using computer using agent and [26:07] I'm curious if do you think the world is [26:09] going to go more in the direction of [26:11] like keep the legacy folks um and just [26:15] have this layer on top or do you think [26:17] you know computer using agents can [26:18] absolutely be part of uh new software [26:20] players and this is something I think a [26:22] lot about so I'm curious to get get your [26:23] guys' thoughts. It is such an [26:25] interesting moment where simultaneously [26:26] the cost and complexity of creating new [26:29] software is so much lower. So you can't [26:31] imagine software changing so much but [26:33] also there exists so much legacy [26:35] software in the world. The computer use [26:36] is so well suited to interact with. So I [26:39] expect both of those things to continue [26:40] for a long time. You know if you look at [26:42] the software that's in production today [26:43] a lot of it was written 10 20 30 40 [26:45] years ago and I I I don't think that [26:47] that stuff's going to go away overnight. [26:49] I do think that in some way the [26:51] existence of computer use um uh puts may [26:54] put more pressure on companies to make [26:58] software work better with agents because [26:59] all the software is going to work with [27:01] agents by default and I think webc is a [27:04] great example of this and I think we'll [27:05] see more stuff like that. [27:06] >> Um so yeah, I I sort of expect both to [27:09] be to be super important. Um but I I do [27:12] think it'll like kind of shift over [27:13] time. Um what do you think, James? [27:15] >> Yeah, I I I agree with you. I think it's [27:17] like um the there the the way there's [27:21] like a lot of arts and craft and you [27:22] understand this because Ari built [27:24] beautiful product with his team. the [27:26] cursor was all in. Um, and so the uh um [27:29] I think there's like a lot of art and [27:30] craft into building like great products [27:33] that people can hold in their hand. And [27:35] um I don't know like for me if we go if [27:38] the internet becomes just this like [27:40] faceless, [27:42] you know, thing that is just agents [27:44] talking to each other and and people are [27:45] not using anything anymore. I would be [27:47] very sad. Um, and and I I don't think it [27:49] will be like that because like there's [27:51] like something about craftsmanship and [27:52] art artisan software that's like just [27:54] feels nice to use because the person who [27:56] designed it really really understands um [27:59] like the experience that the customer [28:03] wants to have. So like my wife loves is [28:06] recently got into playing this like uh [28:08] you know board game. I I forgot the name [28:10] of it. It's where you connect like five [28:11] pebbles to like win. I think it's like [28:13] Omach is the name of it. And so anyway, [28:16] uh so she's trying to get good at it [28:17] because she's playing online with [28:18] friends and it's getting her ass kicked [28:20] and so I was like, "All right, I'm going [28:22] to build you like a tactical trainer for [28:25] um you know, going through different [28:26] gambits so you can like get better at [28:28] it." Like that's for example something [28:30] that like I you know created a first [28:33] version. I was like, "Oh, I know like [28:34] she likes to do it in a certain way. She [28:36] likes these type of uh ways to learn [28:38] more than others." And so I was able to [28:39] like, you know, craft the software to be [28:41] something that feels really good for [28:42] her. And like these kind of personalized [28:44] software, I I know it's like a thing [28:46] maybe today still feels like only really [28:49] tech people will do. But I think if you [28:51] like [28:52] >> think about this a few years out, it's [28:54] like if anybody can build anything and [28:56] they can build these really craft [28:58] software in a very short amount of time [29:00] for like niche use cases that are like [29:02] so well tailored for people, I think [29:04] that's a beautiful world. Um, and again, [29:07] you know, at the end of her training, [29:10] the lessons and mistakes she made, that [29:12] could be an agent that summarizes it and [29:14] gives her in text format, but when she's [29:16] actually playing with the with the AI [29:18] and and and and trying to get better, [29:20] that's something that the person has to [29:21] do. So I do think it's it's um I I'm [29:24] more I'm bullish that I I don't think UI [29:26] is going away and and uh and I think [29:28] kind of a little tangent to your [29:29] original question, but I I do think that [29:31] um the pressure of cheap software will [29:34] make it so that software has to be [29:36] better. Um but I don't think it will [29:38] just be like yeah everything will have [29:39] an MCP and an API and you know comput [29:42] will never be useful. [29:43] >> I love this point so much. I think that [29:45] like humans are so hardwired for direct [29:47] manipulation. you really want to be able [29:48] to, you know, move and and and do things [29:50] with your own hands. Um, and I I think [29:53] that it's really exciting to think about [29:55] how software in the past was so [29:58] oneizefits-all and had to be, you know, [30:00] often uh a lot of the software we [30:02] interact with every day is designed for [30:04] millions, tens of millions, hundreds of [30:06] millions, billions of people to use. And [30:08] I think we will enter an era where any [30:10] piece of software could be designed for [30:11] a hundred people or one person [30:14] >> and um just how much more uh personal [30:16] and and and unique that kind of software [30:19] will be. [30:20] >> Yeah. I my mental model is I think [30:22] there's probably two buckets. One is on [30:24] the consumer side you've got um either [30:27] something you just get you you get joy [30:29] out of you enjoy using or you learn [30:31] from. There is this it's not the utility [30:34] of it. It's the joy of interacting or [30:36] learning or whatever it is and you want [30:37] to spend time on the on the B2B side. [30:41] You you may want your agent to complete [30:43] a bunch of the tedious work, but you [30:44] also there's this concept of like the [30:46] audit log like and you that could be [30:48] personalized in how you want to [30:49] interact, but the product really is me [30:52] as the human user setting the goal and [30:54] then being able to like understand the [30:57] logic but not necessarily need to run [30:59] through like the entire chain of command [31:01] of how it happened. And so like any B2B [31:04] user still needs to figure out okay like [31:05] why did that payment get made and go [31:07] back in and get a summary of what [31:08] happened and be able to say okay pay [31:10] this person at this time um yeah at 30 [31:13] days this this much um discount should [31:16] be applied etc. They want the agent to [31:18] take care of like the execution of it [31:19] but they want to be able to at some [31:20] level like some level of control still [31:23] needs to be put put in there and I think [31:25] that's like something that does require [31:27] a UI of some sort. Um and it has to be I [31:31] think if our it if we're moving more [31:33] towards believing that um AI will do [31:35] more of this that part has to be [31:37] enjoyable to use too, right? Like why [31:39] not? Like why should we settle for [31:41] shitty software? [31:42] >> You know, it's [31:43] >> I do think it'll be really interesting [31:44] to see if we and I I'm optimistic that [31:48] we will see new kinds of software where [31:50] it has traditional user interfaces but [31:53] is also using agents under the hood. And [31:54] then maybe sometimes those agents are [31:56] themselves using computer use to [31:58] automate legacy software by giving you [31:59] some much nicer more purpose-built [32:01] experience. I'm like really excited for [32:03] for that class. [32:05] >> Yeah, it's kind of like the one where we [32:07] like to use internally like the shared [32:08] intelligence layer, right? Like you have [32:10] your agent using legacy software [32:12] somewhere else and doing things other [32:14] places. You like if you're using an [32:17] application, the application wants to [32:19] get access to that context so it's able [32:21] to deliver a better experience. It's [32:22] like I think if we had every website has [32:24] its own agent that doesn't talk to your [32:26] agent, it's it's a very fragmented [32:28] world. And so figuring out like how that [32:30] plays together where the agent can plug [32:32] in and and use your other software, I [32:34] think it's going to be like a really um [32:35] interesting thing to play out over the [32:37] next few months. [32:38] >> I love that you guys Yeah. talk about [32:40] this because I I think it's so [32:41] underappreciated the importance of like [32:43] human AI interaction, especially if you [32:45] want like a product to be successful. So [32:47] like being thoughtful of like how can I [32:49] kind of like humans are used to [32:52] collaborate. So you want any agent to be [32:54] able to like kind of fit into the team. [32:56] I'm just curious like if you apply that [32:58] to a new startup let's say you're like I [32:59] don't know starting a company tomorrow [33:01] and you want to use computer use to like [33:04] um solve a workflow within the [33:06] enterprise. How would you think about [33:08] it? And uh would you be kind of what [33:11] would be the things that jump to mind in [33:12] terms of like oh I want to know when it [33:14] fails well. I want it to be able to [33:17] communicate with people or I want it to [33:18] be like a I don't know a black box just [33:21] like maybe if you had to talk to [33:23] founders right now what advice would you [33:25] give them as they try to use your models [33:27] and capabilities to just solve problems [33:29] for enterprises. In my mind, it depends [33:32] on it depends so much on the modality of [33:36] you know [33:38] >> are you trying to are you a small team [33:41] and you want sort of individual on the [33:44] team to be able to use some piece of [33:46] software that that you're um that that [33:48] your team is using but in a more [33:50] efficient way. Um or are you maybe [33:54] building something that itself wants to [33:56] interact with existing software in which [33:59] case maybe you have some you know [34:01] machine in the cloud somewhere that's uh [34:03] doing computer use in the background I [34:05] think that will really affect the [34:06] modality of how you want the human agent [34:08] collaboration to be right um and um yeah [34:12] you know in the former case I think a [34:13] lot of our products are serving this use [34:15] case um you know where you sit down at [34:17] chatbt and uh you know there's a lot of [34:19] great uh you collaboration you can have [34:21] there. Um and in the other case, you [34:24] know, maybe you're building something [34:25] with with the API. Um you're you're um [34:29] uh doing doing your own computer use [34:30] implementation. And then uh yeah, then [34:32] it really is interesting to figure out [34:34] when when do you want the human to be in [34:35] the loop? Is the agent telling you when [34:37] something's wrong? Is it telling you [34:38] when something went right? Uh you know, [34:40] is something summarizing that so that [34:42] you don't get bombarded with [34:44] notifications saying, "Hey, this [34:45] happened. This happened. This happened." [34:46] Which actually you're going to ignore. [34:48] Um um I don't know. What do you think, [34:50] James? [34:50] >> Yeah, I mean I I you know, both Ari and [34:52] I were startup founders before we joined [34:54] OpenAI and so you know I'm I'm still [34:56] very bullish on startups. You know I [34:57] think the question right kind of comes [34:59] down to like what is there still a mo [35:01] right? Is there something that startups [35:03] can still tackle while labs are are [35:04] doing things? I mean in general I just [35:06] think there's like like two two things [35:08] like one is like I think there's so many [35:09] things to build right and like even [35:11] though we're quite big now thousands of [35:13] people we are still feeling very [35:15] stretched thin internally and like you [35:17] know it's like the teams are not very [35:18] big for all the problems and so if you [35:20] have a problem that you really deeply [35:22] understand you have industry expertise [35:24] you understand how regulatory works the [35:25] go to market motion how to service the [35:28] those customers um it it is still I [35:31] think a very green field out there and [35:33] um the the tools that that you have [35:35] through our APIs like Ari mentioned with [35:37] computer use. Um I think I think we'll [35:40] still let you accelerate and speed [35:41] really I think at the end of the day [35:42] speed and pay is kind of the mode that [35:44] that exists today. Um I think the other [35:46] thing I think about a lot is like there [35:48] are in my opinion still a lot of like [35:49] vertical industries where um the [35:52] diffusion of AI is like much slower. And [35:54] so um you know uh I was talking to a [35:57] company the other day who uh uses [36:00] computer use to help restaurant groups [36:02] to you know take pictures of like their [36:05] back storage you know and then using [36:08] that the the vision looks at kind of [36:10] like what is going to be missing based [36:12] on a projection of tomorrow's traffic [36:13] and then it takes that to then use [36:16] computer use to go to like a shopppee or [36:18] some kind of like procurement software [36:19] to then order your groceries for the [36:21] next day. Um and then based on that it [36:24] will go into your ERP to adjust like the [36:26] timetable for your like employees who [36:29] are um who we might need more people to [36:31] help because in the last year at this [36:33] time of the day um there was a lot more [36:36] traffic coming in from like like a [36:38] holiday or something like that. And so [36:39] like you know those restaurant groups [36:41] are probably not like tech companies [36:43] that are um super you know familiar with [36:47] all the tools are available. And so I [36:49] think like you know living in San [36:50] Francisco we see a lot of thinking wow [36:52] like you know labs can do everything the [36:53] codecs and like you know why doesn't [36:55] everybody just install codecs and just [36:56] have it do stuff but like for the [36:58] average American and the average small [37:00] mid medium business owner the average [37:01] enterprise um even understanding what [37:04] the tools are capable is one connecting [37:06] tools in a way that makes sense is two [37:07] deploying it is three servicing is four [37:10] and so there's still a lot in my opinion [37:11] where if there there are complex [37:12] workflows that require a lot of [37:14] disparate software computer use is a [37:16] great way to join them complete the [37:18] entire intend flow and then building it [37:20] in a format and UI that is feels good to [37:23] hold um for those customers feels still [37:26] like a really really big opportunity [37:27] that um you know we I think is still [37:30] open right tell us about what's next [37:32] what's what are you excited about um [37:34] that's on the frontier that you can [37:35] share around uh around computer using [37:37] agents [37:38] >> well I' I've sort of gestured towards it [37:41] already but we're really excited about [37:43] speed we're really excited about making [37:45] computer use superhuman and Um, we're [37:48] really excited about things that blend [37:50] together different kinds of interfaces [37:52] that bring together sort of multimodal [37:54] inputs and outputs. Um, so you can have [37:57] more kind of fluid experiences with your [37:58] computer. I think right now a lot of our [38:00] computer use product is very background [38:01] focused. It's very much like um, let me [38:04] give the agent a task. So it can do that [38:06] while I do something else, which is [38:07] actually a tremendous strength, right, [38:08] of the product. But I think that in the [38:10] future we'll also see more of this like [38:12] real time collaboration between you and [38:14] the agent. Um, so I'm I'm really excited [38:17] about this area. Um, and um um and other [38:21] otherwise just uh sort of better, [38:24] cheaper, more more available, easier to [38:26] use, [38:26] >> more Yeah, exactly. Being able to [38:28] actually access everything. Yeah. I [38:30] think I think there is, [38:31] >> you know, we with DOTS, right, we're [38:32] we're giving folks these like computer [38:33] in the cloud. Tragic work also has a [38:35] browser in the cloud. I think honestly [38:38] like for for me like one thing I'm [38:40] excited about is just making the [38:41] experience feel a little bit more more [38:42] coherent. Like you know today you can [38:45] install the Chrome extension and it that [38:47] uses your like password manager and [38:48] everything in Chrome and then we have an [38:51] inapp browser in the desktop app which [38:52] has a different set of cookies and [38:54] different passwords. Your computer in [38:56] the cloud also has different things and [38:57] for the average consumer it's confusing. [38:59] It's like why do I have these four [39:00] different browsers that all don't talk [39:02] to each other? If I open up a tab in one [39:04] does it like show up in the other? And [39:06] so like I think a lot of um what I'm [39:08] excited about is how do we make computer [39:10] use go from this product that is used by [39:13] like enterprises and like really [39:15] hardcore tip of the spear kind of users [39:18] to something that's actually [39:19] understandable for for people. It's kind [39:21] of like an oxymoron because I think [39:23] computer use is actually the most easy [39:24] to understand AI product because right [39:26] >> it's an AI using your computer but then [39:28] because of the like uh complexities of [39:30] like the different like states you have [39:32] to manage and the different places [39:33] you're using it the implementation [39:35] details sometimes seep through and then [39:36] it becomes kind of hard to gro and so um [39:39] being able to kind of make that consumer [39:41] grade and easier to understand more more [39:43] coherent um is something I'm really [39:44] excited about next quarter [39:46] >> yeah and I think it it comes back to the [39:47] point that there is still a human that's [39:49] setting off the agent right and they [39:51] have to the the the better the UI and [39:54] controls and the easier and more [39:56] enjoyable it is to use, the more they're [39:58] going to get out of it. And that's [40:00] >> absolutely [40:00] >> principle. On that point, one other [40:02] thing I want to gesture towards is um I [40:05] think another challenge of the product [40:06] today is you you have to put some [40:10] thought into how you're going to use it [40:11] and you have to remember to use it and [40:13] you have to um convince yourself to do [40:15] things that are different from how [40:16] you're used to doing them. Um, and I do [40:19] think, um, dots, like James was talking [40:22] about earlier, um, have this really [40:24] powerful proactivity where they can sort [40:26] of come to you and identify situations [40:28] and and and and, um, and and help you [40:31] solve problems. I think we'll also see a [40:33] lot more proactivity. I'm really excited [40:35] about a world where the agent can sort [40:38] of adapt to you instead of you adapting [40:40] to it. Um, and I think that we're going [40:43] to see a lot of that and I think it will [40:45] um make it a lot easier for people to [40:47] actually like relate to and u and and [40:49] understand and incorporate this [40:50] technology while also um still being the [40:53] ones sort of setting and searing the the [40:54] the goals if that makes sense. [40:56] >> One of the things we haven't talked [40:57] about yet is the momentum behind [40:59] computer use and how much has changed [41:01] and Fitzi and I chatted a little bit [41:03] about this in in our piece. would love [41:05] for you to just kind of talk about where [41:07] it's, you know, how it's the [41:08] trajectories so far and where things are [41:10] headed. [41:11] >> Yeah, I mean James and I have been [41:12] working on computer use for some time [41:14] now [41:15] >> um at OpenAI and it's been an incredibly [41:18] exciting past like six to nine months, [41:21] right? Because we started working on [41:22] these computer use products and [41:24] >> um very few people were using them. um [41:27] the models um all of a sudden it felt to [41:30] me became great at these kinds of tasks [41:33] um earlier this year because of amazing [41:35] work the research team did and then [41:36] we've been working super hard with a [41:39] bunch of teams internally on how to [41:40] create actually good product experiences [41:42] around this and then just in the past [41:44] couple of months all of a sudden you [41:46] know the product's growing like crazy [41:47] and and people are really seeing the [41:49] value in it um but I think it's still so [41:51] early you know it's still the case that [41:53] only um some you know portion of our [41:56] user face is using computer use and um I [41:59] think we're going to see it uh grow [42:01] quite a bit. [42:02] >> Yeah, totally. I mean, you know, I [42:04] joined OpenAI in January and I was I [42:07] joined working on the Atlas product and [42:09] love love Atlas, RIP Atlas, but the uh [42:12] the I think you know, one of the things [42:13] that we realized was um in the Atlas [42:17] product, the agentic experience, I think [42:19] for a few reasons, wasn't actually like [42:20] a big selling point. Not many people [42:22] used it, partly because I think most [42:24] people treated it like a normal browser. [42:26] Um, and over kind of the last few months [42:28] because the model capabilities have [42:29] gotten better. We've now incorporated [42:31] browser and browser use into kind of all [42:33] of our existing products where there was [42:34] already a gentic usage, we've we've just [42:36] seen, you know, it's like YC level of [42:39] growth week over week and then always [42:41] surprised by how fast it can go. And I [42:43] think this next like big inflection [42:44] point is like cloud, right? Like how do [42:47] we like desktop is still very much a [42:51] power user thing. I know we probably use [42:53] our desktop all the time, but like most [42:54] most consumers are on their phones, [42:56] right? And so um how do you give every [42:59] single chat user, how do you give [43:01] everybody who's just using their phone [43:03] for their day-to-day life um a powerful [43:05] computer that they don't even need to [43:07] understand how to use that it just kind [43:08] of makes things work for them. And so [43:10] really excited about how that kind of [43:11] brings it to, you know, billion users. [43:14] >> It feels like computer using agents have [43:16] had a ton of momentum behind them and [43:18] now it's the moment of like, okay, how [43:19] do we get it into more people's hands? [43:21] they trust it, they feel like they can [43:23] really use it. And your experience your [43:25] point earlier about coherent experiences [43:27] like that's that's a real I think once [43:30] as that gets a lot better, I think [43:31] consumers will feel more comfortable [43:33] using it and and it'll feel more [43:34] natural. [43:35] >> I think that's right. I think I think [43:37] we're at the scaling moment of computer [43:39] use both for consumer and enterprise. [43:41] Yeah. At the same time. Um and I'm Yeah. [43:44] really excited to see where it goes and [43:45] what people build with it. [43:47] >> Nice. Us too. [43:48] >> All right. [43:48] >> Well, thank you so much for joining us. [43:50] This was awesome. Um, we love talking [43:52] about this topic and it was really fun [43:53] to hear it from you guys directly. [43:55] >> Thank you. A lot of fun as well. [43:56] >> Fun. Yeah. Thanks. [43:57] >> Thank you. [43:58] >> Thank you guys.