OpenAI Ships Imperfect Agent Harnesses to Fuel 90-Day Horizon

OpenAI推出非完美Agent框架以推动90天规划周期 OpenAI放弃多年期路线图，转向90天模型规划周期，并发布不完美的Agent功能，因实证用户反馈与语音界面的发展已超越传统企业规划。 OpenAI产品负责人Tara Seshan与Nan Yu反对制定多年期路线图，解释称前沿AI团队必须以与即将推出的模型能力相契合的60至90天周期开展工作。为走在企业惯性与能力冗余之前，OpenAI推出了如切换开关等不完美的界面框架，并利用从结构化API到Computer Use的分层备用机制来确保任务完成。演讲者倡导构建适配人类认知带宽的中心化协调Agent，并依托能够从直接用户失败追踪记录中构建严格评估（evals）的高度可接触的产品经理（PM）。展望2027年，他们预测对话式语音与主动型、自驱动的软件体验将取代传统的空白输入框提示词及手动工作流。 OpenAI产品团队放弃多年期路线图，转而采用60至90天的规划周期，专门针对AI模型在两到三个月后的能力水平进行设计，而非受限于眼下的限制。 (19:25) 为了在ChatGPT超过10亿的用户中快速收集实证用户反馈，OpenAI推出了如“切换开关”（the toggle）这类不完美的UI权宜之计作为即时的Agent框架，并在此过程中接受高代码废弃率。 (0:06) 由于人类心理难以同时追踪数十个独立线程，管理大量专用微型Agent的用户不可避免地会在最顶层设置一个单独的“幕僚长”（chief of staff）协调Agent。 (7:35) ChatGPT采用分层能力栈，将结构化第三方集成与底层的Computer Use和Codex后备方案相结合，确保在缺少API或插件的情况下也能完成任务的“最后一公里”。 (12:47) 与AI研究团队合作需要产品经理变得“随时可私信”，以便获取细粒度的会话追踪记录，并撰写严格的评估（evals），从而证明如何将各项能力通过后训练直接融入基础模型。 (18:11)

OpenAI product leaders Tara Seshan and Nan Yu argue that the rapid pace of frontier AI development makes multi-year roadmaps obsolete, requiring product teams to plan in 60-to-90-day horizons and deploy imperfect interface scaffolding to harvest empirical user data. Navigating Velocity with Short Horizons and Scaffolding (2:43) Seshan contrasts the exhaustive design polish standard at companies like Stripe with the speed required in frontier AI, stating that empirical feedback from live deployments disproves abstract theorizing. To deliver agent capabilities to over a billion users without breaking developer workflows, OpenAI deployed temporary interface stopgaps—such as "the toggle"—sacrificing interface purity to gather the usage data needed for future iterations. Yu asserts that users accept product churn and deprecations as long as teams maintain transparency regarding underlying shifts. Because underlying model capabilities evolve so quickly, Seshan targets a planning horizon of only two to three months (60 to 90 days). Multi-year predictions are prone to error in frontier markets, whereas established domains like payments can rely on multi-year scenario modeling. Product teams must balance designing for upcoming model capabilities against the risk of over-indexing on present-day limitations or speculative long-term fantasies. Cognitive Bottlenecks and Multi-Layered Agent Execution (7:35) Addressing debates over single-identity versus multi-identity agent ecosystems, Yu argues that human evolutionary limits prevent people from tracking dozens of distinct conversational threads. Power users who claim to manage swarms of specialized micro-agents inevitably place a single "chief of staff" coordinator agent on top. Effective interfaces must therefore bundle fragmented tasks into cohesive, human-legible workflows rather than forcing users to act as operational dispatchers managing discrete agents, permissions, and partitioned memory contexts. To ensure completion across third-party tools, OpenAI structures its platform with layered execution fallbacks. When structured tool integrations or plugins are unavailable or fail, the platform falls back to direct Computer Use or code execution via Codex. Yu notes that an agent failing on the final step of a complex task creates deep user frustration, making resilient fallback mechanisms essential for solving the "last mile." Grounding Research in Evals and Granular User Traces (18:11) Collaborating with deep research scientists requires product managers to replace traditional feature specifications with concrete session traces, failure analysis, and rigorous evaluations (evals). Seshan explains that when a product manager proves a specific prompt engineering and tooling configuration reliably achieves an outcome, that evaluation benchmark serves as direct evidence for post-training teams to bake the capability natively into future base models. Sourcing these nuanced edge cases requires high-touch user relationships. Both speakers explain that because conversational and agentic breakdowns are subtle and context-dependent—such as an assistant giving unexpected attitude—product managers frequently use direct messaging with users to examine raw transcripts and prompt context. This granular investigation provides far more actionable diagnostic signal for research teams than aggregated usage metrics. Resolving Capability Overhang Through Proactive Interfaces (27:47) Yu highlights an ongoing "capability overhang," in which models possess latent intelligence that users fail to leverage because standard software forces them to confront a blank input box. To lower adoption friction, Yu predicts that software will evolve into "self-driving" applications that proactively guide users and initiate tasks, eliminating traditional onboarding learning curves. Looking toward interface paradigms for 2027, Seshan identifies conversational voice as the dominant primitive, pointing to rollouts like Advanced Voice Mode. Because speech aligns with evolutionary human habits, low-latency voice removes psychological hesitation and technical troubleshooting friction across both non-technical consumers and internal business teams. While core product disciplines like systems thinking and user empathy remain foundational, the speakers conclude that frontier AI demands relentless experimentation and a high tolerance for deprecating code. Significant architectural challenges remain unresolved around how enterprise permissions and partitioned memory should operate, and how business change management will keep pace with continuous model churn.

[0:06] Tara Seshan: Urgency really matters in terms of getting the functionality out there... there's a lot of theorizing that one can do before you ship something, but nothing compares to the actual empirical evidence of seeing users try it, use it, and then iterating from there. [2:43] Tara Seshan: sufficiently AGI-pilled [5:45] Tara Seshan: Lollapalooza of features [5:45] Tara Seshan: If you're not shipping the latest and the frontier to these enterprises, you'll end up getting leapfrogged. [7:35] Nan Yu: Usually what you'll hear is like, 'Well, first I have this chief of staff agent that manages all the other ones for me.' It's like, okay, well, you've just cheated, right? You're not managing seven; you're managing one that's running your team. [12:47] Claire Vo: Don't even give me an MCP. I want to point computer use at your UI. And then, you know, I don't care about the tokens—congratulations on selling me tokens. [15:01] Claire Vo: Well, I have to tell you, every time I go into your home, it tells me it's time to update... It's like I walk in, it's like, 'You want a reno? I got a reno for you.' [19:25] Tara Seshan: I live in the now. [28:31] Claire Vo: a little baby bot over here, a little Codex in a box

[0:06] All right, we're here. And I warned them [0:09] backstage what we were going to start [0:10] this talk about because I know there's a [0:13] lot of questions about OpenAI. There's a [0:15] lot of questions about the future of [0:17] product development. But what I want to [0:19] talk about is [0:21] >> what do you want to talk about, Claire? [0:22] >> The toggle. M I want to talk about the [0:25] fact that every AI product has two [0:30] faces. It has left toggle and right [0:33] toggle. And there's been a lot of [0:34] conversation today about iterative [0:36] product development and shipping things [0:37] that are imperfect and getting things [0:39] into the wild. And so Tara, I just want [0:42] you to talk to me about shipping [0:43] imperfect things instead of a toggle and [0:46] how do you make these decisions about [0:48] putting things in the wild that you know [0:50] may not be the perfect thing right now [0:52] but are the right thing ultimately to [0:54] build the right product. [0:55] >> For sure. I will say this to start like [0:57] you know I'm the daughter of Asian [0:59] parents. I'm not used to doing imperfect [1:01] things. You know you have to output [1:03] something perfect every time. I also [1:05] worked at Stripe for like seven some odd [1:07] years and everything at Stripe is like [1:10] deeply polished and deeply considered [1:11] and did you think through it 100 times [1:13] and did you name this enum the exact [1:15] correct thing and I think what has been [1:17] a big change about shipping things in [1:19] this era is actually urgency really [1:22] matters in terms of being get getting [1:24] the functionality out there and also [1:27] there's a lot of like theorizing that [1:29] one can do before you ship something but [1:31] nothing compares to like the actual [1:33] empirical evidence [1:34] of seeing users try it, use it, and then [1:37] iterating from there. So, I've really [1:39] had to shift my mindset from how do you [1:42] get it perfect a priori versus how do [1:46] you ship something and iterate on it as [1:48] quickly and effectively as possible. So, [1:50] with the toggle like is it the perfect [1:53] solution? No, it is an imperfect [1:55] solution. It is certainly not like the [1:57] ideal thing to ship. But what was really [2:00] important was getting this agentic [2:03] harness or like the ability to use an [2:05] agent to get things done in the hands of [2:07] like the over billion users that use [2:09] chat GBT. And so we needed to ship [2:12] something that allowed us to do that [2:13] without disrupting developers workflows [2:16] >> as such an imperfect solution. And you [2:19] know what I do believe that in a year [2:23] maybe we won't be having the same [2:24] conversation about the struggle and non [2:27] we've had this we've had this chat [2:28] before where we've talked about right [2:29] now you have to build and throw away a [2:32] lot of stuff whether it's code or [2:35] product and I'm curious how you think [2:37] about deciding what to deprecate and [2:41] what to keep around and keep alive. You [2:43] know, the toggle is a good example of [2:45] this where it I think there's an obvious [2:48] sort of next step which is like no [2:49] toggle, right? Like there's a there's a [2:51] story that evolves from step A to step [2:53] B. And if you can take users on the [2:56] journey with you, then they're a lot [2:58] more accepting of like the change and [3:00] the turn and the the the product [3:02] evolution. Um, especially if they [3:04] appreciate that, you know, they can they [3:06] can have an understanding and some [3:07] transparency over like what's, you know, [3:09] what's actually going on behind the [3:10] scenes. So, uh, I think as long as you [3:12] have a coherent story that users can [3:13] follow along with and, you know, with [3:15] the amount of attention that they're [3:16] paying, uh, you know, like there's a [3:18] there's actually a lot that you can do [3:19] and not, uh, you know, and not not [3:21] really burn any kind of bridges with [3:22] them. [3:23] >> Yeah. And I'm I'm curious, you know, [3:25] that the toggle is maybe imperfect and [3:28] we may build a lot of stuff that we have [3:30] to throw away, but you you do exercise [3:35] good judgment and you do exercise [3:37] caution and reason inside the products [3:39] that you've worked on or are about to [3:41] work on. What constraints do you hold [3:44] really strongly when building product [3:47] that's moving so fast? What what what [3:49] keeps that quality bar high for you? I [3:51] think there's a few different elements [3:52] of it. One is certainly are we doing [3:55] something additive for users? Like are [3:57] we unlocking additional value? Is there [3:59] something genuinely [4:01] useful in this product? Um I think the [4:04] second is that we have this internal bar [4:07] of taking a product shipping it [4:09] internally trying it out having everyone [4:11] use it and if there isn't enough uptake [4:15] internally and that's not a specific you [4:17] know number or exact bar but there is is [4:20] it retaining well do people find [4:22] something delightful about it or [4:23] something surprisingly great about it is [4:26] it unlocking a new use case of the [4:27] models like is it you know doing [4:29] something novel I think that is that [4:32] like sense for this is adding to their [4:35] experience. This is something that is um [4:39] unlocking a new set of functionality. [4:41] And then probably most importantly in [4:43] this era is this aiming for 2 to 3 [4:45] months of like where the models will be [4:48] rather than being overly anchored in the [4:50] present or being overly like futuristic [4:52] and unusable. Is this like at that [4:54] perfect zone of being I don't know [4:57] sufficiently AGI pill that I think is [4:59] like the bar here. So is it valuable? Is [5:01] it do users care about it? Is it [5:04] surprisingly great for users? And is it [5:06] like in line with model capabilities? [5:08] And is it getting out of the way of the [5:09] model of doing a great job? [5:10] >> Nan, what about you? What constraints do [5:12] you think help you build great product? [5:15] >> The the biggest constraint uh is [5:18] honestly like people's understanding and [5:21] ability to sort of absorb what you're [5:22] what you're giving them, right? We talk [5:24] a lot about uh like capability overhang [5:26] but like but mods can do so much and [5:28] just like you know people aren't [5:29] necessarily taking advantage of all [5:30] those abilities. I think that's where [5:32] that's where the gap is right. So it [5:34] doesn't matter what you put into it, uh [5:36] if the if the ability for people to [5:38] absorb those changes and absorb that uh [5:40] capability isn't there, then that's the [5:42] that's the real kind of bounding uh like [5:44] kind of process that you're dealing [5:45] with, [5:45] >> you know, and we're talking right now, [5:47] you two are working on codecs and work [5:49] and there's a lot of business users [5:52] including myself that have built [5:53] businesses around relying on this tool [5:55] and I think you know I spent a minute [5:58] and a half in enterprise and a lot of [6:00] people say that B2B customers cannot not [6:03] absorb change. They I I mean like [6:06] customer success people tell me they [6:08] can't absorb change. Sales people tell [6:09] me sales people can't absorb change. [6:11] Like we cannot absorb the rate of [6:13] change. Do you feel like that still [6:14] stands true in the AI era or do you feel [6:17] like that statement is is broken right [6:19] now? Some of it is certainly true in [6:22] that I think the current pace probably [6:24] to most enterprises feels beyond [6:27] breakneck like something beyond [6:28] breakneck like a la palooa of [laughter] [6:31] features and advancements like it [6:33] certainly feels maybe to some [6:35] overwhelming but at the same time like [6:38] if you're not shipping the latest and [6:40] the frontier to these enterprises you'll [6:43] end up getting leaprogged. So a great [6:46] example of this is that most of our [6:48] enterprises were using chat. So they [6:51] were merely you know asking questions [6:53] and getting answers and whilst that was [6:55] happening like the agent revolution was [6:57] going on and we needed to get agents to [7:01] these enterprises as fast as possible [7:02] otherwise they might consider [7:04] alternative products. And so while they [7:06] have their own maybe in the day-to-day [7:08] pace of absorbing changes, we needed to [7:10] ship this giant revolutionary change [7:12] that broke all of their processes and [7:14] broke all of their, you know, [7:16] conceptions about how enterprises should [7:18] receive new updates so that they could [7:21] actually unlock more value. And so it's [7:23] one of those things, I mean, it's like a [7:25] truism of building product, which is [7:26] like, you know, do what your users need, [7:28] not what your users say they need. And I [7:31] think that applies here almost more than [7:33] it ever has before. I want to talk about [7:35] agents since you brought them up because [7:37] I do think we're in the age of agents [7:40] and I have certainly um been in the age [7:43] of agents for a little bit and I want to [7:46] have a debate with you. [7:47] >> Oh gosh. [7:47] >> Which is or I want your opinion. Maybe [7:49] we won't have a debate. Maybe we'll [7:50] agree single identity agents versus [7:54] multi-identity agents. my beloved codecs [7:58] docked on my computer that I talk to all [8:01] day versus my 40 Grock bots that are [8:05] loadbearing across different parts of my [8:07] business. I do think as product builders [8:10] um people who are building agents, we're [8:11] seeing this divergence of do we [8:14] centralize on a central kind of generic [8:16] agent identity that can do anything, let [8:18] the models get, you know, get out of the [8:19] way of the models or do we like [8:21] artisally craft sole MDs and have little [8:24] like micro agents for the job? But I'm [8:26] just curious, you know, coming in fresh [8:28] eyes, where do you land if anywhere on [8:31] that spectrum from a design principles [8:33] perspective? [8:34] >> I I I think when designing this like the [8:36] the most successful um efforts have [8:40] really tried to map on to human nature, [8:42] right? And like we have millions of [8:44] years of evolution that like help us [8:46] understand how to hold different things [8:47] in our heads. And uh you know 40 agents [8:50] is quite a lot, right? And like if you [8:51] asked anybody like would you manage 40 [8:53] direct employees like some you know [8:54] Jensen Huang might do it, right? But [8:55] like like a lot of people would kind of [8:58] look at that and be like I don't know I [8:59] don't know if I could actually keep up [9:01] with that many threads ongoing. Um so I [9:03] I I think there there's there's [9:05] something in there where we need to [9:06] group these things into sort of bundles [9:08] of activity right like if you look at [9:10] like the u you know you ask people hey [9:12] like how do you manage your your agents? [9:14] Usually what you'll hear is like well [9:16] first I have this chief of staff agent [9:17] that manages all the other ones for me. [9:18] It's like okay well like you you've just [9:20] cheated right you're not managing seven [9:21] you're managing one that's like you know [9:23] kind of running your team. So, so I I [9:25] think that that that kind of dynamic [9:26] kind of immediately uh kind of uh you [9:29] know comes comes to the front right as [9:31] soon as you let people do this kind of [9:32] stuff. [9:32] >> What about you? What do you think? You [9:34] know, my mind with this like certainly [9:35] there's a philosophical element to it of [9:37] like you know do you have this one [9:38] unified entity? Do you have these like [9:40] smaller entities? But actually there's a [9:42] lot of really tactical questions that go [9:43] into it like how do you think about data [9:45] access and permissions? Like if you have [9:47] one entity is it like switching between [9:48] using like a service account or your [9:50] account? like how do you manifest that [9:52] to the user in an easy way to understand [9:54] if you have a single agent like does it [9:57] have segmented memory like every times [9:58] it joins a private slack channel is it [10:00] almost a different agent is it like [10:02] severed in the slack channel in like the [10:05] severance sense and so actually to me [10:08] answering this question goes back to [10:10] really specifically the use case and [10:12] probably thinking through all of those [10:13] edge cases of like how should the memory [10:15] behave how should it like how should it [10:18] talk to each person should it be using [10:20] their credentials or should it be using [10:21] a universal set of credentials? It's [10:23] actually like there there are [10:24] philosophical things one could have and [10:26] opine on in this topic but to me there [10:28] are just so many practical product [10:30] considerations like the use case matters [10:32] so much in deciding this. What I love [10:34] about these two answers is I think for [10:37] product folks everybody in the audience [10:39] plus designers there are these two sides [10:41] of the craft that we do. There's this [10:43] very human side where you're trying to [10:45] like in there's inception and severance [10:47] I guess. there's like the inception of [10:49] like how do I get in the mind of a user [10:51] and what are their mental models and how [10:53] do they interface with the world and [10:55] what is the most natural way for our [10:57] products to touch their problems and and [10:59] solve them and then there is like [11:01] systems and architect thinking which is [11:05] in order to make this like manifest this [11:07] into the world in a realistic way I need [11:09] to think through all these systems all [11:12] these platform components all these edge [11:15] cases to make sure that comprehensive [11:17] we've can even deliver this to the user [11:20] in a sensible way. Do you think those [11:21] are the two kind of like skill sets you [11:24] need to design and build great agents in [11:26] this moment? Are there other hard skills [11:27] you feel like you're relying on now more [11:30] than ever as you design or think about [11:32] agentic experiences [11:33] >> you're describing you sort of like [11:35] systems thinking plus you know user [11:37] empathy. These are very classic kind of [11:39] product management types of ideas right [11:42] that you know we we apply to you know a [11:45] different you know universe of [11:46] technology in the before times. Uh but [11:48] now you know we have to kind of start [11:50] from scratch and and you know from those [11:52] principles kind of rethink what does uh [11:54] you know how do these products want to [11:56] be shaped right so I I I think those are [11:58] very important but ultimately they're [11:59] just the old principles in disguise. I [12:01] would maybe add a thing to that which is [12:04] again I mentioned like I used to work at [12:06] stripe which is very academic company [12:08] and my parents are very academic and I [12:10] love thinking about those things in [12:11] abstract but I'd say the third thing [12:13] that you just really need in addition to [12:14] user empathy and systems thinking is [12:16] like a relentlessness to like try it out [12:18] and keep iterating and you're going to [12:20] take a lot of pain while you do that and [12:22] there's just an element of like [12:24] relentlessly trying and trying and [12:26] trying and trying and trying to see what [12:28] works y [12:29] >> that and the ability to learn maybe from [12:31] those feedback loops that certainly has [12:33] always been necessary, but just the [12:34] clock speed has gone way. [12:36] >> Yeah, you're throwing a lot of spaghetti [12:38] against a lot of walls all the time. [12:41] Actually, your chief of staff agent is [12:43] telling all the other agents to fling [12:45] spaghetti at the walls. [12:47] >> You know, talking about systems thinking [12:49] and platform, I am really curious [12:51] because all of us are looking to tools, [12:53] certainly ones that you build for [12:55] productivity and efficiency. And I was [12:59] telling somebody backstage, I just love [13:02] computer use now. Like don't even give [13:03] me an MCP. I want to point computer use [13:06] at your UI. And then, you know, I don't [13:08] care the the token. Congratulations on [13:10] me selling me tokens. Um, but you know, [13:13] I think about what is the intersection [13:15] between the tools we had and the tools [13:17] we have now, what you build, what you [13:20] platform for. I'm curious how you just [13:22] think about the broader ecosystem as [13:24] you're trying to enable productivity for [13:25] people at work. So I think I think we [13:27] think a lot about platforms and chat GBT [13:29] as a platform and what is the [13:31] functionality that like natively goes [13:33] into the platform. What is functionality [13:35] that like we might allow folks to build [13:37] in whether that's like first party or [13:39] third party as as plugins or like [13:41] additional interfaces on top of it and [13:43] that's kind of how that relates to the [13:44] ecosystem. So in that platform what are [13:47] the hooks that we need to expose to a [13:49] third-party like meetings app so that it [13:51] can be most effective for you use in [13:53] codecs using codecs using computer use [13:55] like using all the these tools um but I [13:58] don't think it's so binary as like what [14:00] should be in our platform or in our [14:03] productivity system versus in one of [14:05] these apps. I think we should offer an [14:07] array of different tools like for [14:10] perhaps like a plugin to be able to like [14:12] get all your meetings data and [14:13] understand you know what you need to do [14:15] next. But if that doesn't work and those [14:17] systems don't uh interface well then [14:20] there's always computer use as like a [14:22] next layer or as like a fallback to be [14:24] like well if the user wants to get this [14:25] task done and you know the pre-built [14:28] tools or like ecosystem tools aren't [14:29] available for it then there's like [14:31] always something else that they can fall [14:33] back on. And so I think it's more about [14:35] structuring these capabilities in the [14:37] platform in terms of layers, exposing [14:39] them as like a set of hooks that [14:40] third-party developers and first-party [14:42] developers can like hook into and build [14:44] cohesive experiences, having the right [14:46] system for those experiences to be [14:48] composable with one another. And then [14:50] ultimately all of this is just serving [14:52] the user's goals of as they try to get [14:53] their task done, if like the first layer [14:56] doesn't work, there's like a second [14:58] layer for them to be able to accomplish [14:59] their task as you like traverse all [15:01] those capabilities. What do you think [15:02] about kind of like integrated [15:04] experiences like do it all in one [15:06] platform, let's live in our house versus [15:10] kind of expanding into this ecosystem [15:11] and how do you build I mean the thing [15:13] that I was thinking as you were talking [15:14] is like you have a non-deterministic [15:16] model on top of a like natural language [15:20] interface functionally with a computer [15:23] that can click and do anything with [15:25] third party tools and integrations you [15:27] don't control and you're still trying to [15:29] build like [snorts] a comprehensive of [15:31] highquality [15:33] good branded beloved experience like how [15:37] do you even climb that mountain as a [15:39] product leader? Do you have any what's [15:41] what genius are you going to bring in to [15:42] help us here? [15:44] >> Uh look I I I don't I don't know if this [15:46] is a the a new or unique idea but there [15:49] is a a huge difference between getting [15:52] um like all of the job done versus [15:54] getting everything except for the last [15:56] mile. And the the last mile honestly [15:59] feels sometimes worse than just like [16:00] it's a non-starter. If it's a [16:01] non-starter, you can just like look, I'm [16:02] just going to do it myself or I'll try [16:04] another route or whatever it is. But if [16:05] it gets like 99% there and then it just [16:07] like barfs, then you you're like, well, [16:08] this this is this is actually a worse [16:10] sort of like unfulfilled promise. And I [16:12] think what feels great about computer [16:14] use a lot of times is that it will [16:16] always work, right? It might be a little [16:18] bit like slow or like, you know, it [16:20] might cost a lot of tokens like you were [16:21] talking about, but like it will going to [16:22] get the job done all the way. And you [16:24] know, if as the ecosystem kind of, you [16:27] know, we wait for it to catch up, right, [16:28] and to provide the right hooks and the [16:30] right MCPs and all that kind of stuff, [16:31] it will 100% do the job for you. And I I [16:33] think that ultimately that's that's what [16:35] makes it feel super great. I think the [16:37] right experience amidst all those [16:39] complexities and changing things, the [16:40] ecosystem, the interface, the model [16:43] itself, the like tools like computer [16:45] use, the unpredictability of all of it [16:46] is that like the user should almost feel [16:49] like uh you know like the Charles Emmes [16:51] quote that your that your user should [16:53] almost feel like a guest in your home [16:55] like you've anticipated all the things [16:57] that they want to do and you've like [16:58] provided for them gently. Like so should [17:01] you know our user in our app feel like [17:04] we've anticipated their needs. And in [17:05] some ways it's like the model [17:06] increasingly doing that. [17:08] >> Well, I have to tell you, every time I [17:09] go into your home, it tells me it's time [17:11] to update. [17:12] >> Yeah. Yeah. [laughter] Yeah. Yeah. Get [17:13] get the latest get the latest build. [17:15] That is the welcome, Matt, that I get. [17:17] It's like I walk in, it's like, you want [17:19] a Renault? I got a Renault for you. [17:21] [laughter] [17:22] Well, I I'm curious. You know, you're [17:24] you're designing this product while it's [17:27] flying off the shelf. You know, a lot of [17:29] the things, Nan, as you said, are are [17:31] old things. user empathy, systems [17:32] thinking, but there are some new things, [17:35] right? There are some new skills and t [17:38] I'm curious about working with research [17:41] because this is something I don't think [17:42] a lot of folks in the audience have done [17:45] in their careers because I think it's [17:46] very new for product people or product [17:48] leaders. What have you learned from [17:51] going into this new role where you [17:52] really have to collaborate with [17:53] research? Yeah, I will say for me [17:56] working with research was entirely new [17:59] coming to OpenAI. So I'm certainly not [18:00] the like expert on this topic. I will [18:03] give you like the you know novices's [18:05] view like Connecticut Yankee and King [18:07] Arthur's Court sort of view of this. Um [18:11] my take coming in working with research [18:13] was oh working with research is [18:14] different than working with engineering. [18:15] Like that might be obvious to everyone [18:17] here but I was like oh this you know [18:18] it's it is quite different. Um, in as [18:21] much as like I can add add something to [18:25] uh working with research, it is for me [18:27] to come with very very specific use [18:29] cases, a really clear idea of what like [18:32] users want to do and their goals to have [18:34] done like sample readings. So understand [18:36] like literally what was the like session [18:39] and what did it look like and why did [18:40] the model not do what it needed to do. [18:43] If I can write evals that is like I'm [18:46] there like learning how to write good [18:47] evals and writing evals as much as [18:49] possible is like the most important [18:50] thing and in as much as I can say with [18:53] this prompt and these skills I was able [18:55] to achieve the outcome with the model [18:57] that is like the ideal thing I can take [18:59] to post- training and be like look now [19:02] how can we postrain this capability into [19:03] the model from the beginning but I think [19:06] it starts with first understanding that [19:08] this is like a wholly different way of [19:09] working that I really didn't know how to [19:11] do before and then getting to the point [19:14] where you're good enough at like [19:16] bringing these the specific user data [19:19] and then if they trust you sufficiently [19:21] you can then write the eval to like get [19:23] that iteration loop going. How far in [19:25] the future do you feel like you live as [19:28] a product leader? Like are you you know [19:30] because I I do think there's this time [19:32] horizon of like I could fix what's right [19:34] in front of my face and there's stuff to [19:36] fix. There's I know what's coming next [19:39] and I can anticipate where the product's [19:41] going and then there's like the models [19:43] next year are going to be out here. [19:45] Where do you feel like you have to live [19:48] as a leader building on top of that [19:50] foundation that's moving? You know, as [19:51] soon as you said that like do you live [19:52] in the future? I was like I live in the [19:54] now. Like I don't live in the future. I [19:56] live in the now. Um no, I do. I think I [19:58] have to live a little bit in the future. [20:00] To me it's like the ideal is two to [20:01] three months. can't be too bigrained and [20:03] be like this is the future in years [20:06] because I think that inevitably our [20:08] predictions of that are almost always [20:10] wrong. Like I have a personal dock of in [20:14] incorrectly predicted things that I [20:16] thought were going to happen and didn't [20:17] happen. Um just to remind myself of like [20:20] my inability to predict the future years [20:22] out and then at the same time if you [20:24] build exactly for today you're going to [20:26] get left behind. So it's like I [20:28] constantly repeat this but it's like two [20:29] to three months out. I I do want to just [20:31] let's pull the audience again. It's the [20:32] end of the day. Let's get interactive. [20:34] Who right now is doing 2027 annual [20:37] planning. [20:39] And who just heard her say you can't go [20:41] out more than 60 or 90 days? I think [20:44] this is the fundamental tension. We're [20:46] all It's like so hard to even guess. But [20:49] but it's not the same for every [20:50] business. Y [20:51] >> like again just to use an example I'm [20:53] familiar with like Stripe's business is [20:55] very different than OpenAI's business. [20:57] The dynamics of the payments market look [20:59] largely similar to how they looked [21:01] before. certainly accelerated in other [21:03] ways, but you like know the forces and [21:04] you can kind of model like, oh, here's [21:06] the bull case, barecase, base case in a [21:08] way that for Open AI's business, you [21:10] just like cannot do. Well, maybe Sarah [21:12] Frier can and she's the expert and I'm [21:14] not as good as her for sure. But like I [21:16] it's hard for me to predict OpenAI's [21:18] business the way I I can feel like oh I [21:20] understand the levers and in stripes and [21:22] so maybe it is appropriate to plan you [21:25] know many many months out but I think it [21:27] involves understanding like the pace of [21:28] the market you're in and understanding [21:31] given the pace and the dynamics of that [21:33] market at what you know cadence should I [21:36] be thinking and planning in the future. [21:38] So, so what you're hearing is Stripe [21:39] Stripe PM still have to do their annual [21:41] planning. [21:42] >> Oh, for sure. Good luck to you all. [21:44] Like, yeah, for sure. [laughter] [21:46] >> All right, let's talk a little bit. [21:47] Let's close out with then and now. Um, [21:49] so I want, you know, we've all been in [21:52] product for a minute and a half as fresh [21:54] 26-year-olds. What What still matters [21:57] from a classic PM craft? And what do you [22:00] think is like newly table stakes? you [22:03] know, I I I think the the emphasis uh [22:06] like kind of changes, right? Like for a [22:09] lot of reasons, but you know, we talked [22:10] about, you know, just like absorption [22:12] like rate and like capability overhang. [22:14] So I I I think something that probably [22:16] matters way more now than it's done [22:18] before is is onboarding. I think we've [22:19] like talked about, you know, people [22:20] people have like talked about on board [22:22] is an important thing, but it's also [22:23] something where it's a little bit [22:25] specific and like a lot of engineers [22:26] don't love working on it and things like [22:28] that, but I I think really to to kind of [22:30] get a lot of success like thinking about [22:31] like what is the initial experience? How [22:33] do you get people to really understand [22:35] what your offering is like constantly is [22:38] is probably something that matters way [22:40] more now than it does before. [22:41] >> Tara, what do you think? [22:42] >> Yeah, I think I think users [22:44] understanding like privacy, their data, [22:45] how that's used is like way more [22:47] important than it always has been. like [22:49] users understanding norms of your [22:51] product. Like before any sort of [22:53] unpredictable behavior you could get [22:55] away with because there was some like [22:56] settings menu or some popup that could [22:58] appear. But now if the you know agent is [23:00] doing things semi-autonomously like you [23:02] actually really need to make sure that [23:04] at any given point a user can like [23:06] predict what is probably going to happen [23:07] with a feature or a primitive that [23:09] you're introducing. And in terms of like [23:12] just product thinking or getting up and [23:14] running as like a PM in some ways it's [23:17] going to sound like a really boring [23:19] answer but like all the usual stuff [23:20] still applies except the importance of [23:23] testing things empirically and moving [23:24] faster has only increased. So it's like [23:26] yeah do all the other stuff you used to [23:28] do which is like get in the details have [23:30] the high level and the low level at the [23:31] same time like really understand the [23:33] design really understand the product but [23:35] also do it all faster and and test with [23:37] users has just become way way more [23:39] important. I have one more that um naan [23:42] is going to become a victim of when you [23:44] are in the open slack which is the open [23:47] AAI product and engineering team is [23:49] highly DMable highly accessible highly [23:53] visible. I mean, they're getting like [23:56] feedback UYU IDs from me on the regular [23:58] like I didn't like what Astra said. I [24:01] had an attitude like and so I'm curious, [24:04] you know, certainly I have tried to do [24:06] this in my own life, but do you feel [24:08] like direct user relationships right now [24:12] are becoming table stakes for product [24:14] leaders? Like this idea of the DMable PM [24:17] I think is one that I see a lot out of [24:19] OpenAI and I'm curious like staying [24:21] close to the ground if you feel like [24:23] that's becoming more and more important. [24:25] My background is in like developers and [24:28] enterprise. So I've always been a DMable [24:31] PM in the sense that people could always [24:33] message me and ask for features. Um, but [24:36] I think what kind of feels new is that [24:38] for like a large consumer product or a [24:40] product that everybody uses to do the [24:42] same level accessibility. Um, so in some [24:44] ways like this it doesn't feel normal to [24:47] me like of course you should be in [24:48] direct contact with your users. And as I [24:50] mentioned earlier like your value to [24:52] research is like knowing specifically [24:54] what the users want and what their use [24:56] cases are. And so the more you can be a [24:58] DMable PM or engineer the better. like [25:01] if you didn't like Astra's output. I was [25:02] actually thinking like hm Claire should [25:03] send me those outputs. Like [laughter] [25:05] it would be really interesting to like [25:07] know what's happening there. Like that [25:09] that stuff is like the important thing [25:10] in part that's like how you add value. [25:12] But I'm curious Rene's take. [25:14] >> Yeah. I I I think you know you worked in [25:18] like dev tools, right? And you know [25:20] engineers are extremely picky. They're [25:21] extremely detail oriented. Often you're [25:24] not going to get good feedback until the [25:25] second or third follow-up question. And [25:27] I think that's like what a lot of uh PMs [25:29] are feeling like a change about, right? [25:31] Because if you're designing an agentic [25:33] experience with like natural language [25:34] interfaces, the problems that people [25:36] encounter like Astro Game is attitude [25:38] like they're pretty subtle, right? That [25:39] that is that that bears some explaining [25:41] like what exactly, you know, did it say? [25:43] What was the context? Did you make it [25:44] mad? Like like all sorts of, you know, [25:46] kind of follow-up questions that you [25:48] need to have. So I think the the having [25:50] a a more kind of direct uh deeper [25:52] relationship bears a lot of fruit there, [25:54] right? that that maybe you wouldn't have [25:56] been able to to take advantage of [25:58] before. [25:58] >> You neither of you have gotten my DMs. [26:00] So, I'm very excited for the next 90 [26:03] days for you all. We'll put us in a [26:06] group chat. It'll be really fun. Okay, [26:08] last question. So, when Dan was up here [26:10] with the anthropic team, he asked like, [26:12] you know, what in a year about the [26:14] product craft, about the role will be [26:15] different. I want to I want to talk [26:17] about the stuff. So, like what do you [26:19] feel is going to be different about [26:20] products like the what we touch? what [26:24] the buttons on the site, the [26:25] experiences, like where do you feel like [26:27] this is all going? Because I feel like [26:30] every week, every month, we come up with [26:32] a new form factor, a new thing that [26:34] matters. It's curious as product [26:36] thinkers. What do you think is going to [26:37] be big in 2027? I have an answer for you [26:39] and it's voice. [26:41] >> Voice. [26:41] >> I love voice. You try out GPT live. I [26:45] love voice. Like we're yappers. [26:48] >> Yappers. Hillary Fidley. The yappers API [26:51] genius. Like voice is a like has changed [26:54] the way I work. Like I just want to talk [26:56] to computer. Like I do not like it's so [26:59] much better. It feels so much more [27:00] natural in some ways. Like I'm sure all [27:02] of you have been tech support for your [27:04] parents. Like the existence of voice has [27:07] reduced the amount of like tech support [27:09] I've had to do for my family to like an [27:11] incredible degree. Like I was onboarding [27:14] our own people team at OpenAI onto a new [27:16] product and the existence of voice like [27:20] changed the game like it is such an [27:22] intuitive interface for so many people. [27:24] It's like the natural way they interact [27:26] to the thing Nan said earlier about like [27:28] you kind of have to think about human [27:29] nature and the way we are used to like [27:32] the way we've evolved over you know [27:34] millennia to interact with others like [27:37] speaking feels so natural like voice is [27:40] the models are finally getting good at [27:41] it. It is getting faster. Voice I'm like [27:45] so bullish on voice. Voice. What about [27:47] you? Um I I think I think probably like [27:51] something that looks a lot like [27:52] self-driving is probably going to be the [27:54] the thing that really you know becomes [27:56] much more important and prominent in the [27:58] next year. You know, I think a big [28:00] problem people have is like, you know, [28:01] empty input box problems, right? And and [28:04] it doesn't really matter if you're using [28:06] uh a, you know, general tool like [28:07] ChatGpt or something like more industry [28:09] specific like you you have the same it's [28:11] like, okay, I have the product. Now [28:12] what? Well, now you got to learn how to [28:13] use it, right? And then so having these [28:15] products are smart now. They're like [28:16] literally intelligent. So having them uh [28:19] you know kind of use themselves and sort [28:20] of to help you out and and make that a [28:22] really really gentle on-ramp I think is [28:25] probably gonna going to be a very normal [28:27] thing that we're going to look back on [28:28] be like why why didn't it always work [28:30] that way? [laughter] [28:31] >> Okay we need the notetakers to be our [28:33] accountability buddy. So we got voice, [28:36] we got self-driving. I'm going to say [28:38] hardware that replaces carrying around [28:40] our laptop open. Um so hardware. [28:45] Yeah. Like I want I want like a little [28:47] baby baby bot over here. A little codeex [28:49] in a box where I don't have to do She's [28:51] nodding where I don't have to do this. [28:53] That's my prediction. So whoever took [28:55] notes, we're going to see which of us [28:56] were right on what 2027 hot product of [28:59] the year. Everybody thank these lovely [29:02] leaders from OpenAI for coming. [29:06] Claire, thank you. [applause]