OpenAI Targets AI Software Engineer by Year-End via GPT-5 Codex Agents

OpenAI计划年底前通过GPT-5 Codex智能体打造AI软件工程师 先进模型与自主 harness 的结合使智能体能够执行长达7小时的连续代码重构任务，从而将开发者的重心从语法编写转移到规模化项目监督上。 本期内容追踪了 OpenAI 对 AI 辅助编程的专门聚焦，重点展现了从早期 GitHub Copilot 等简单自动补全功能，向基于 GPT-5 Codex 的完全自主、长时间运行工作流的转变。发言者探讨了诸如“harness”等关键工程概念、其内部高信号代码审查工具的成功，以及向异步云端智能体的架构转变。展望2030年，他们探讨了大规模重构的经济影响、绝对算力短缺的现实，以及为何在“氛围编码”（vibe coding）时代基础工程技能仍然至关重要。 OpenAI 联合创始人 Greg Brockman 与工程负责人 Thibault Sottiaux 指出，将 AI 从被动补全工具转变为主动协作者的关键在于构建围绕模型的“harness”（工具、测试与 Agent 闭环），它充当模型的“身体”以驱动代码在外部环境中运行。 (2:57) 针对自动补全功能严格的 1500 毫秒延迟预算限制，OpenAI 改变了用户交互逻辑，转向由 Agent 自主检索代码库上下文与排错，并确立了年内打造出具备完整自主能力的“Agentic 软件工程师”的目标。 (4:40) 为解决 Agent 缺乏长期记忆的问题，OpenAI 倡导在代码库中引入 `agents.md` 文件，通过结构化总结仓库上下文并明确架构偏好，降低 Agent 探索开销并统一编码风格。 (20:35) 最新优化的 GPT-5 Codex 具备长时程复杂任务处理能力，能够在用户下达任务后自主理清逻辑、执行重构并运行测试排错，内部复杂用例中可持续自主运行长达 7 小时。 (30:29) 展望 2030 年，Brockman 预测世界将走向“物质充裕但算力绝对稀缺”的形态，若要为全球每个人配备全天候专属 Agent，全球算力规模或需达到约 100 亿张 GPU，物理基础设施与软件效率将成为关键决胜点。 (44:00)

OpenAI executives Greg Brockman and Thibault Sottiaux argue that AI-assisted software development is evolving from rigid, low-latency autocomplete tools into persistent, autonomous agents capable of deep refactoring and code review. While models increasingly take over mechanical coding tasks and repository navigation, they assert that architectural fundamentals remain vital as hardware capacity emerges as the primary bottleneck to scaling intelligence. From Latency-Bound Autocomplete to Autonomous Harnesses (2:57) In the early days of language models, OpenAI treated coding as an exception to its broad general-intelligence efforts, establishing dedicated metrics and data pipelines. Early integrations like GitHub Copilot in 2022 operated under a strict 1,500-millisecond latency budget, requiring suggestions to appear almost instantly to prevent developer abandonment. However, as slower reasoning models emerged, OpenAI shifted toward building external "harnesses"—the infrastructure, tooling, and agent loops that Sottiaux describes as a physical body executing the intentions of the model's brain. This harness architecture reverses the traditional workflow dynamic: rather than requiring engineers to manually gather and paste stack traces into chat interfaces, cloud-based agents autonomously inspect codebases, execute tests, and iterate on fixes. Under this architecture, Sottiaux notes that GPT-5 Codex can deliver instant responses for basic requests or work autonomously for up to seven hours on complex, multi-step refactoring operations while systematically running tests to verify its progress. In-Context Integration and High-Signal Code Review (27:06) To embed AI into standard development environments, OpenAI is experimenting with hybrid interfaces spanning terminal CLIs, IDE plugins, GitHub mentions, and standardized context files like agents.md. According to Sottiaux, agents.md compresses repository architectures and records developer style preferences that models cannot infer from raw code, though Brockman acknowledges that multi-session agent memory remains an active research challenge. OpenAI also deployed an internal pull-request review mode focused on high-signal contract verification rather than basic syntax checks. The system traces deep dependency chains and verifies whether code matches the author's underlying intent, sustaining an accuracy rate above 90%. Brockman explains that while early automated review bots were dismissed across the software industry as noise, crossing a critical capability threshold turned the Codex review tool into an essential safety net, exemplified when it caught several critical bugs during an overnight push of 25 PRs before a major IDE extension release. Preserving Engineering Fundamentals Amid Syntactic Automation (32:25) Despite rapid advances in generative code, both Brockman and Sottiaux reject the idea that learning to program is obsolete, urging developers to master both core engineering and AI tooling. Sottiaux notes that members of his own team successfully implemented a core harness in Rust by pairing Codex guidance with mentorship from experienced Rust engineers. Brockman recalls his early self-taught programming projects, where he invented fragile custom delimiters to serialize data without understanding the concept of serialization; modern AI code review, by contrast, immediately steers beginners toward standard libraries like JSON. By delegating mechanical syntax, API lookups, and codebase search to AI systems acting as digital interns, developers can focus on higher-level architecture and system contracts. Brockman reports that models already navigate OpenAI's internal repositories faster than human creators, expanding the scope of what individual software engineers can design and supervise. Compute Bottlenecks, Cost Reductions, and Oversight (41:52) Looking toward 2030, Brockman anticipates that widespread software generation will exist alongside severe compute scarcity. Providing persistent personal agents for every individual globally would require an estimated 10 billion GPUs, making physical compute allocation—rather than software design—the principal ceiling on applied intelligence. Sottiaux adds that multi-step agent workflows executing hundreds of tool calls require placing compute geographically closer to end users to reduce round-trip latency. To expand access under these hardware constraints, OpenAI points to architectural efficiency gains, citing an approximate 80% price reduction on the o3 model alongside packaging GPT-5 Codex within standard Plus and Pro tiers. Amid reports of tenfold usage growth, Sottiaux and Brockman emphasize that scaling autonomous agents safely requires default CLI sandboxing and scalable oversight—governance frameworks that enable humans and weaker models to supervise large populations of cloud-based agents without manually reviewing every line of output. OpenAI projects that programming will increasingly shift toward managing fleets of autonomous, cloud-based agents that execute prolonged refactoring and automated verification. Realizing this ecosystem at scale, however, remains contingent on overcoming physical compute limits, formalizing scalable oversight mechanisms, and solving persistent memory across multi-run workflows.

OpenAI 联合创始人 Greg Brockman 与 Codex 工程主管 Thibault Sottiaux 指出，AI 编程正从被动代码补全转向具备运行环境、能自主探索并执行长达数小时复杂任务的代理软件工程师。他们认为未来的瓶颈不仅在于模型智力，更取决于外部测试治理框架、代码审查可信度以及物理算力供给。 从被动补全到具身代理的框架演进 (2:57) 在早期 GPT-3 时代，模型仅能根据函数名和文档字符串补全代码，将生成一千行连贯代码视为极高门槛；随着模型能力演进，开发重点转向了外围的“harness”（治理框架）。Thibault Sottiaux 将模型比作大脑，而 harness 则如同身体，包含工具链和循环运行机制，使模型能够实际执行代码、运行测试并与环境交互。在产品形态上，早期行内补全存在约 1500 毫秒的严格延迟上限，超过该时间开发者便会放弃使用；而面对更聪明但较慢的模型时，界面必须随之演进，转向异步工作流，让开发者无需自行粘贴长堆栈信息，而是让代理在云端或本地终端自主搜寻上下文并排错。 高信号代码审查与长程重构任务落地 (27:06) 为克服过往自动化工具噪声过大的问题，OpenAI 推出了专注于理解开发者契约与底层意图的 Codex 代码审查模式。该模式能进行多层依赖分析，内部实践显示其准确率超过 90%，并在团队单夜提交 25 个 PR 时成功拦截多项潜在错误；即使审查意见未被完全采纳，其推理过程仍具备参考价值。与此同时，模型告别了早期反复坚持错误答案的顽固表现。针对 GPT-5 Codex，其设计强化了持久性与工具联动能力，面对简单任务反应迅速，而在内部复杂重构任务中，该代理已能持续自主工作长达 7 小时，期间可自行运行测试并逐一修正故障。 混合工程环境部署与上下文记忆挑战 (9:55) 当前软件开发面临本地非标准化配置难以远程容器化扩展的现实阻碍，因此部署路线需兼顾本地终端、IDE 插件与云端异步执行。为解决代理缺乏环境记忆的问题，团队引入了代码库同级的 agents.md 文件，用以压缩项目架构信息并显式标注风格偏好，避免模型重复全盘扫描。但 Greg Brockman 坦言，代理目前仍缺乏长期跨会话记忆，第十次运行无法直接继承前九次吸取的教训。为了安全地迈向由数百万云端代理提供支持的未来，Codex CLI 默认采用安全沙箱机制，团队亦在探索可扩展监督方案，确保人类在不逐行审阅代码的情况下掌控特权提升与系统安全。 算力物理稀缺性与软件工程基本功重塑 (41:52) 面向 2030 年，Brockman 预测数字与物理领域的创造门槛将被极大拉低，但全球将面临绝对的算力稀缺。他估计若实现每人配备专属常驻代理，全球需要约 100 亿张 GPU，远超当前规模；同时为了减少工具调用往返延迟，算力还需在物理层面上更靠近终端用户。尽管工具日益强大且 o3 模型已实现约 80% 的大幅降价，两人仍一致强调学习编程基础的重要性。Sottiaux 指出，理解架构蓝图、系统设计与数据结构的工程师才能真正驾驭 AI；Codex 的核心价值不在于无限膨胀无法理解的代码量，而在于协助修复漏洞、完成遗留系统迁移及实现形式化验证，从而提升软件系统的稳定性与可理解性。 对话表明，GPT-5 结合 Codex 后带动了超过 10 倍的使用量增长，但要实现完全自主的代理软件工程师，仍有赖于解决跨会话长期记忆、复杂本地环境容器化以及极端算力短缺等物理与工程瓶颈。

[0:00] Greg Brockman: As soon as you saw that, you knew this is going to work [2:57] Thibault Sottiaux: think about it a little bit as, you know, the harness being your body and the model being your brain. [4:40] Greg Brockman: just bet that the greater intelligence will pan out in the long run. [16:42] Andrew Mayne: going into a grocery store and refusing to get a cart and just carrying everything to the checkout. [19:05] Thibault Sottiaux: When I collaborate with Greg, you know, I don't complain that sometimes you're on Slack... sometimes we interact like through a GitHub review. [22:25] Greg Brockman: If you've just sort of read the internet... and you're trying to write some code for the first time, you're probably going to have a bad time of it [27:06] Greg Brockman: If something kind of works in AI right now, one year from now, it'll be incredibly reliable, incredibly mission critical. [32:25] Greg Brockman: Why am I typing this thing? [34:31] Thibault Sottiaux: large populations of agents somewhere in the cloud [41:52] Thibault Sottiaux: 100 million lines of code that you don't understand. [46:35] Thibault Sottiaux: I think it's a wonderful time to learn to code [46:35] Greg Brockman: definitely learn to code, but learn to use AI. [49:09] Thibault Sottiaux: I spend more time talking to Codex now than most people.

[0:00] Andrew Mayne: Hello, I'm Andrew Mayne, and this is the OpenAI Podcast. In this episode, we're going to speak [0:05] Andrew Mayne: with OpenAI co-founder and president, Greg Brockman, and Codex engineering lead, Thibault Sottiaux. And we're going to talk about Agenda coding, GPT-5 codex, and where things might be heading in 2030. [0:17] Greg Brockman: Just bet that the greater intelligence will pan out in the long run. [0:25] Thibault Sottiaux: It's just really optimized for what people are using GPT-5 within codex for. [0:31] Greg Brockman: How do you make sure that AI is producing things that are actually correct? [0:39] Andrew Mayne: We're here to talk about codex, which first, I've been using it since actually since I worked here with the first version of this. And then now [0:43] Andrew Mayne: you guys have the new version of this. I was playing with it all weekend long. And I've been [0:47] Andrew Mayne: very, very impressed by this. And it's amazing how far this technology has come in a few years. [0:52] Andrew Mayne: I would love to find out the early story. [0:55] Andrew Mayne: Like, where did the idea of even using a language model to code come from? [0:58] Greg Brockman: Well, I mean, I remember back in the GPT-3 days seeing the very first signs of life of take a doc string, a Python definition of a function name, and then watching the model complete the code. [1:13] Greg Brockman: And as soon as you saw that, you knew this is going to work. [1:16] Greg Brockman: This is going to be big. [1:17] Greg Brockman: And I remember at some point we were talking about these aspirational goals of imagine if you could have a language model. [1:22] Greg Brockman: that would write a thousand lines of coherent code, right? [1:25] Greg Brockman: That was like a big goal for us. [1:26] Greg Brockman: And the thing that's kind of wild [1:28] Greg Brockman: is that that goal has come and passed. [1:30] Greg Brockman: And I think that we don't think twice about it, right? [1:33] Greg Brockman: I think that while you're developing this technology, [1:36] Greg Brockman: you really just see the holes, the flaws, [1:38] Greg Brockman: the things that don't work. [1:39] Greg Brockman: But every so often, it's good to like step back [1:41] Greg Brockman: and realize that like actually things [1:43] Greg Brockman: have just like come so far. [1:45] Thibault Sottiaux: It's incredible how used we get [1:47] Thibault Sottiaux: to things improving all the time [1:49] Thibault Sottiaux: and how to just become like a daily driver [1:52] Thibault Sottiaux: You just use it every day and then you reflect back to like a month ago. [1:55] Thibault Sottiaux: This wasn't even possible. [1:57] Thibault Sottiaux: And this just continues to happen. [1:59] Thibault Sottiaux: I think that's quite fascinating, like how quickly humans adapt to new things. [2:04] Greg Brockman: Now, one of the struggles that we've always had is the question of whether to go deep in a domain. [2:09] Greg Brockman: Right. [2:09] Greg Brockman: Because we're really here for the G, right, for AGI, general intelligence. [2:14] Greg Brockman: And so to first order, our instinct is just push on making all the capabilities better at once. [2:21] Greg Brockman: Coding has always been the exception to that, right? [2:23] Greg Brockman: We really have a very different program that we use to focus on coding data, on code metrics, on trying to really understand how do our models perform on code. [2:33] Greg Brockman: And, you know, we've started to do that in other domains, too. [2:35] Greg Brockman: But for programming and coding, that that's been like a very exceptional focus for us. [2:40] Greg Brockman: And, you know, for GPT-4, we really produced a single model that was just a leap on all fronts. [2:46] Greg Brockman: But we actually had trained, you know, the codex model. [2:49] Greg Brockman: And I remember doing like a Python sort of focused model. [2:52] Greg Brockman: Like we were really, really trying to push the level of coding capability back in 2021 or so. [2:57] Greg Brockman: And I remember when we did the Codex demo, that was maybe the first demonstration of what we'd call Vibe coding today. [3:04] Greg Brockman: I remember building this interface and having this realization that for just standard language model stuff, the interface, the harness is so simple. [3:11] Greg Brockman: You're just completing a thing and maybe there's a follow-up turn or something like that. [3:16] Greg Brockman: But that's it. [3:18] Greg Brockman: for coding that you actually, this text comes to life, right? You need to execute it, that it needs [3:23] Greg Brockman: to be hooked up to tools, all these things. And so you realize that the harness is almost like [3:28] Greg Brockman: equally part of how you make this model usable as the intelligence. And so that is something that I [3:33] Greg Brockman: think we kind of knew from that moment. And it's been interesting to see as we got to more capable [3:38] Greg Brockman: models this year and really started to focus on not just making the raw capability, like how do you [3:44] Greg Brockman: win at programming competitions, but how do you make it useful, right? Training in a diversity [3:48] Greg Brockman: of environments, really connecting to how people are going to use it, and then really building the [3:51] Greg Brockman: harness, which is something that Thibaut and his team have really pushed hard. [3:54] Andrew Mayne: Could you unpack a harness, what that means in simple terms? [3:58] Thibault Sottiaux: Yes, it's quite simple. You have the model, and the model is just capable of input-output. [4:03] Thibault Sottiaux: And what we call the harness is how do we integrate that with the rest of the infrastructure so that [4:07] Thibault Sottiaux: the model can actually act on its environment? So it's the set of tools, it's the way that it's [4:13] Thibault Sottiaux: looping. So the agent loop, as we refer to it as the agent loop. And it's in essence, [4:20] Thibault Sottiaux: it's fairly simple. But when you start to integrate these pieces together and really [4:23] Thibault Sottiaux: train it end to end, you start to see like pretty magical behavior and an ability of the model to [4:29] Thibault Sottiaux: really act and create things on your behalf and be a true collaborator. So think about it a little [4:34] Thibault Sottiaux: bit as, you know, the harness being your body and the model being your brain. [4:40] Andrew Mayne: Okay. It is, It's interesting to see how far it came, like GPT-3 days where you literally had to write commented code and say, [4:48] Andrew Mayne: this function does this with its Python, put your hashtag in front of that, whatever. [4:52] Andrew Mayne: And it's just interesting to see how the models have now become just naturally, intuitively good at coding. [4:57] Andrew Mayne: And you mentioned that trying to determine between a general purpose model or saying how important code is. [5:04] Andrew Mayne: Was it just outside demand, people telling me one of these models is better at code? [5:07] Andrew Mayne: Or was this coming internally because you guys wanted to use this more? [5:11] Greg Brockman: both yeah absolutely both and i remember you know in i think 2022 is when we worked with github to [5:19] Greg Brockman: produce github copilot and the thing that was very interesting there was that for the first time you [5:24] Greg Brockman: really felt what is it like to have an ai in the middle of your coding workflow and how can it [5:29] Greg Brockman: accelerate you and i remember that there were a lot of questions around the exact right interface do [5:33] Greg Brockman: you want ghost text so it just does a completion do you want a little drop down with a bunch of [5:36] Greg Brockman: different possibilities. But one thing that was very clear was latency was a product feature. [5:42] Greg Brockman: And the constraint for something like an autocomplete is that 1500 milliseconds, [5:46] Greg Brockman: right? That's like the time that you have to produce a completion. Anything is slower than [5:50] Greg Brockman: that. It could be incredibly brilliant. No one wants to sit around waiting for it. And so the [5:55] Greg Brockman: mandate that we had, the clear signal we had from users and from the product managers and all the [6:01] Greg Brockman: people thinking about the product side of it is get the smartest model you can subject to the [6:05] Greg Brockman: latency constraint. And then you have something like GPT-4, which much, much smarter, but it's not [6:11] Greg Brockman: going to hit your latency budget. What do you do? Is it a useless model? Like, absolutely not. The [6:15] Greg Brockman: thing you have to do is you change the harness, you change the interface. And I think that that's [6:19] Greg Brockman: like a really important theme is you need to kind of co-evolve the interfaces and the way that you [6:24] Greg Brockman: use the model around its affordances. And so super fast, smart models is going to be great, [6:29] Greg Brockman: but the incredibly smart, but slower models, it's also worth it. And I think that we've always had [6:34] Greg Brockman: a thesis that the, you know, that the returns on that intelligence are worth it. And it's never [6:40] Greg Brockman: obvious in the moment because you're just like, well, it's just going to be too slow. Why would [6:42] Greg Brockman: anyone want to use it? But I think that our approach has very much been to say just that [6:47] Greg Brockman: the greater intelligence will pan out in the long run. [6:54] Andrew Mayne: It was hard for me to wrap my head around where that was all headed back when working on the GitHub copilot, because at that point, [7:00] Andrew Mayne: were used to, like you said, the completion, ask to do a thing, it completes a thing. And I think [7:04] Andrew Mayne: I didn't really understand how much more value you would get out of building a harness, adding all [7:09] Andrew Mayne: these capabilities there. And it just seemed like all you just need is the model. But now you realize [7:13] Andrew Mayne: the tooling, everything else matters can make such a big difference. And you brought up the idea of [7:17] Andrew Mayne: modalities. And now we have CLI, Codex CLI. So I can go in the command line, I can do this. There's [7:24] Andrew Mayne: a plugin for VS Code, so I can go use this there. And then also I can deploy stuff to the web and do [7:29] Andrew Mayne: that. And I don't think I fully kind of comprehend the value of that. And so like, how is this [7:35] Andrew Mayne: something you're using? How are you kind of deploying these things yourself? Like, where are [7:38] Andrew Mayne: you finding the most, you know, utility out of it? [7:43] Thibault Sottiaux: I think just to go back a little bit, like the first signs that we saw is like, we had a lot of developers at the company outside of the company, [7:48] Thibault Sottiaux: like our users use ChatGPT to help them debug like very complex problems. And one thing that we [7:54] Thibault Sottiaux: clearly saw is like, people are trying to get more and more context into ChatGPT. And you're [7:59] Thibault Sottiaux: trying to get bits of your code and stack traces and things. [8:02] Thibault Sottiaux: And then you paste that and you present that to a very smart model to get some help. [8:06] Thibault Sottiaux: And interactions were starting to get more and more complex up to some point where we [8:10] Thibault Sottiaux: realized like, hey, maybe instead of the user driving this thing, maybe let the model actually [8:16] Thibault Sottiaux: drive the interaction and find its own context and then find its way and be able to debug [8:20] Thibault Sottiaux: this hard problem by itself so that you can just sit back and watch the model do the work. [8:25] Thibault Sottiaux: So it's like sort of like reversing that interaction that led to this, I think, thinking a lot more about the harness and giving the model the ability to act. [8:35] Greg Brockman: And we iterated on form factors. [8:37] Greg Brockman: I mean, I remember at the beginning of the year, we had a couple of different approaches. [8:41] Greg Brockman: We had sort of the async agentic harness, but we also had the local experience and a couple of different implementations of it. [8:51] Thibault Sottiaux: We actually started to play a little bit with this idea of running it in the terminal. [8:58] Thibault Sottiaux: And then we felt that was not AGI-pilled enough. [9:00] Thibault Sottiaux: We needed the ability to run this at scale and remotely and just close the laptop and have the agent just continue to do its work. [9:08] Thibault Sottiaux: And then you can maybe follow it on your phone and interact with it there. [9:12] Thibault Sottiaux: That seemed very cool. [9:13] Thibault Sottiaux: So we pushed on that. [9:14] Thibault Sottiaux: But we actually had a prototype of it fully working in a terminal. [9:18] Thibault Sottiaux: And people were using that productively at OpenAI. [9:21] Thibault Sottiaux: We decided to not launch this as a product. [9:23] Thibault Sottiaux: It didn't feel like polished enough. [9:25] Thibault Sottiaux: It was called 10X because we felt like it was giving us this 10X productivity boost. [9:29] Thibault Sottiaux: But then we decided to just experiment with different form factors and really go all in with the async form factor initially. [9:37] Thibault Sottiaux: And now we've kind of gone back a little bit of that and re-evolved and said, hey, actually, this agent, we can bring it back to your terminal. [9:43] Thibault Sottiaux: We can bring it in your IDE. [9:44] Thibault Sottiaux: But the thing that we're really trying to get right is like this entity, this collaborator that's working with you and then bringing that to you in the tools that you're already using as a developer. [9:55] Greg Brockman: And there are other shots on goal as well. [9:57] Greg Brockman: So we had a version where there was a remote daemon that would connect to a local agent. [10:02] Greg Brockman: And so you kind of could get both at once. [10:06] Greg Brockman: And I think that part of the evolution has been that there's almost this matrix of different ways you could try to deploy a tool. [10:14] Greg Brockman: There's this async. [10:15] Greg Brockman: It has its own computer off in the cloud. [10:18] Greg Brockman: There's the local that it's running synchronously there. [10:21] Greg Brockman: You can blend between these. [10:24] Greg Brockman: There's been a question for us of how much do we focus on trying to build something that is externalizable, that is useful in the diversity of environments that people have out there versus really focus on our own environment and try to make it so that things work really well for our internal engineers. [10:37] Greg Brockman: And one of the challenges has been we want to kind of do all of this. [10:40] Greg Brockman: We ultimately want tools that are useful to everyone. [10:43] Greg Brockman: But if you can't even make it useful for yourself, how are you going to make it extremely useful for everyone else? [10:47] Greg Brockman: And so part of the challenge for us has been really figuring out where do we focus and how do we achieve the sort of biggest bang for the buck in terms of our engineering efforts? [10:57] Greg Brockman: And, you know, for me, one of the things that's been an overarching focus has been we know that coding and building very capable agents is one of the most important things that we can do this year. [11:09] Greg Brockman: At the beginning of the year, we set a company goal of an agentic software engineer by the end of the year. [11:15] Greg Brockman: And figuring out exactly what that means and how to substantiate that and how to bring together all the opportunity and all the kind of compute that we have to bear on this problem. [11:24] Greg Brockman: Like that has been a great undertaking for many, many people at OpenAI. [11:29] Andrew Mayne: So you mentioned that you had the tool 10X and that was an internal tool. [11:32] Andrew Mayne: And that seemed to be something at some point you said, oh, this is really useful to other people. [11:36] Andrew Mayne: It's got to be hard to decide when to do that and when not to. [11:39] Andrew Mayne: and how much to sort of prioritize that. [11:41] Andrew Mayne: We've seen cloud code has become extremely powerful, [11:43] Andrew Mayne: which I imagine is probably a similar story [11:45] Andrew Mayne: with something that was used internally [11:46] Andrew Mayne: and then became something deployed. [11:48] Andrew Mayne: When you start to think about next steps of, [11:51] Andrew Mayne: where do you decide to take it next? [11:53] Andrew Mayne: Where do you decide to put the emphasis? [11:54] Andrew Mayne: You mentioned before, I can now run things in the cloud, [11:59] Andrew Mayne: run these web, do these kind of agentic-like tasks [12:01] Andrew Mayne: where I walk away. [12:02] Andrew Mayne: And my problem is just, it's such a new modality. [12:05] Andrew Mayne: It's really, really hard for me to think about. [12:07] Andrew Mayne: But sometimes these things have to sit around for a while and people sort of discover them independently. [12:12] Andrew Mayne: And have you found that internally that somebody says, oh, now I get it? [12:15] Greg Brockman: I'd say absolutely. [12:16] Greg Brockman: Right. [12:16] Greg Brockman: And I think that, you know, my perspective is that we kind of know the shape of the future, right, of the long term. [12:22] Greg Brockman: It is very clear that you're going to want an AI that has its own computer that is able to run, you know, delegate to a fleet of agents and be able to solve multiple tasks in parallel. [12:32] Greg Brockman: You should wake up in the morning, you're sipping your coffee, you know, answering questions for your agent, like providing some review, being like, oh, no, this wasn't quite what I meant. [12:40] Greg Brockman: This workflow clearly needs to happen. [12:43] Greg Brockman: But the models aren't quite smart enough for this to be the way that you interact with them. [12:48] Greg Brockman: And so having an agent that is really there in your terminal, in your editor to help you with the way that you do your work that looks very similar to the way you would have done it a year ago. [13:01] Greg Brockman: That's also the present. [13:02] Greg Brockman: And so I think that the way that we've seen it is almost we're blurring together. Here's what the future looks like, but we also can't abandon the present and thinking about how do you bring AI into code review and how do you make it so that it appears proactively and does work for you that's useful. [13:16] Greg Brockman: And then you have a whole new challenge as well of if you have a lot more PRs, like how do you actually sort through those to the ones that are the ones you actually want to merge? [13:25] Greg Brockman: And so I think we've kind of seen all of this opportunity space and we've seen people start to change how they develop within OpenAI, how they even structure their code bases. [13:34] Thibault Sottiaux: Yeah, I think there are two things to that effect that really combine and mean, you know, this is where we're at today is one infrastructure is hard. [13:44] Thibault Sottiaux: And we would love for, you know, all of everyone's code and like tasks and packages to be like perfectly containerizable. [13:50] Thibault Sottiaux: And so we can run them at scale. [13:53] Thibault Sottiaux: That's not the case. [13:55] Thibault Sottiaux: Like people have very thorough and complex setups that probably only runs on their laptop. [14:00] Thibault Sottiaux: And we want to be able to leverage that and meet, you know, people where they are so that, you know, they don't have to configure things specifically for Codex. [14:06] Thibault Sottiaux: That gives you this very easy entry point into experiencing, you know, what a very powerful coding agent can do for you. [14:14] Thibault Sottiaux: And it's also at the same time, lets us experiment with, you know, what the right interface is. [14:20] Thibault Sottiaux: Six months ago, we weren't playing with these kinds of tools. [14:23] Thibault Sottiaux: And this is all very new and evolving fast. [14:25] Thibault Sottiaux: And we have to continue to iterate here and innovate on like what the right interface and what the right way to collaborate with these agents are. [14:33] Thibault Sottiaux: And we don't feel like we have really nailed that yet. [14:35] Thibault Sottiaux: that's going to continue to evolve, [14:37] Thibault Sottiaux: but bringing it to like a zero setup, [14:41] Thibault Sottiaux: extremely easy to use out of the box, [14:43] Thibault Sottiaux: you know, allows a lot more people to benefit from it [14:46] Thibault Sottiaux: and like play with it. [14:47] Thibault Sottiaux: And for us to get the feedback [14:48] Thibault Sottiaux: so that we can continue to innovate, [14:49] Thibault Sottiaux: that's very important. [14:50] Greg Brockman: I remember at the beginning of the year [14:52] Greg Brockman: talking to one of our engineers, [14:55] Greg Brockman: who I think is really fantastic. [14:56] Greg Brockman: And he was saying that ChatGPT, [14:58] Greg Brockman: we had this integration [15:00] Greg Brockman: where it could automatically see the context [15:03] Greg Brockman: in this terminal. [15:04] Greg Brockman: And he's like, it's transformative [15:05] Greg Brockman: because he doesn't have to like copy paste errors. [15:07] Greg Brockman: He just like can instantly be like, hey, like, you know, what's what's the bug? [15:10] Greg Brockman: And it would just tell him and it was great. [15:13] Greg Brockman: Right. And you realize that it was an integration that we built that was so transformative. [15:18] Greg Brockman: It wasn't about a smarter model. [15:19] Greg Brockman: And I think that one thing that's very easy to get confused by is to really focus on only one of these dimensions and be like, which one matters? [15:27] Greg Brockman: Because the answer is they kind of both matter. [15:29] Greg Brockman: And the way I've always thought about this, I remember when we were originally releasing the API back in 2020, [15:34] Greg Brockman: is there's two dimensions to what makes an AI desirable. [15:38] Greg Brockman: There's intelligence, which you can think of as one axis. [15:41] Greg Brockman: And then there's convenience, which you can think of as latency. [15:44] Greg Brockman: You could think of as cost. [15:45] Greg Brockman: You could think of as the integrations available to it. [15:48] Greg Brockman: And there's some acceptance region, right? [15:50] Greg Brockman: Where it's like, if the model's incredibly smart, [15:53] Greg Brockman: but it takes you like a month to run it or something, [15:55] Greg Brockman: like you still might, right? [15:56] Greg Brockman: If what's going to output is such a valuable piece of code [15:59] Greg Brockman: or cure for a certain disease or something like that, [16:02] Greg Brockman: okay, fine, like it's worthwhile. [16:03] Greg Brockman: If the model is incredibly not that intelligent, not that capable, then all you want to do is autocomplete. [16:10] Greg Brockman: So it has to be incredibly convenient, zero cognitive tax for you to think about what it's suggesting, that kind of thing. [16:16] Greg Brockman: And where we are is, of course, somewhere on the spectrum now. [16:19] Greg Brockman: We now have smarter models that are reasonably less convenient than autocomplete, but still more convenient than you have to sit around and wait for a month for the answer to appear. [16:29] Greg Brockman: And so I think that a lot of our challenge is figuring out when do you invest in pulling that convenience to the left? [16:35] Greg Brockman: When do you invest in pushing the intelligence up? [16:38] Greg Brockman: And it's a massive design space. [16:40] Greg Brockman: It's what makes it fun. [16:41] Andrew Mayne: Yeah. [16:42] Andrew Mayne: I don't know if you remember, but I made an app that was featured on the launch back in 2020, AI channels. [16:48] Andrew Mayne: Uh-huh. [16:48] Greg Brockman: Of course. [16:49] Andrew Mayne: And that was, yeah, the challenge was, GPT-3 app was very capable, but I had to write these like 600 word prompts to get it to do stuff. [16:56] Andrew Mayne: And because it's six cents per thousand tokens in the latency, I'm like, I don't think this is the world for this right now. [17:02] Andrew Mayne: Yes. [17:02] Andrew Mayne: And then GPT-3.5 and GPT-4, then all of a sudden you see all that capabilities. [17:07] Andrew Mayne: And it was hard for me to say why, but then you see that all of a sudden the things that come together. [17:12] Andrew Mayne: And you mentioned, you know, the idea of just having, you know, the model be able to see the context inside of the, you know, where you're working. [17:20] Andrew Mayne: And I remember when I was copy pasting using ChatGPT into my workspace and it reminded me going into a grocery store and refusing to get a cart and just carrying everything to the checkout. [17:28] Andrew Mayne: I'm like, this is terribly inefficient. [17:30] Andrew Mayne: Once you put things on wheels, it works really well. [17:32] Andrew Mayne: And I think we're seeing all kinds of those unlocks now. [17:36] Andrew Mayne: Now the problem I deal with is when I sit down to work on something is, do I go into CLI? [17:42] Andrew Mayne: Do I go use the VS Code plugin? [17:43] Andrew Mayne: Do I go into Cursor? [17:44] Andrew Mayne: Do I use some other tool? [17:46] Andrew Mayne: And how do you guys figure this out? [17:48] Thibault Sottiaux: Right now, we're still at the experimentation phase where we're trying different ways for you to interact with the agent and bring it where you're already productive. [17:57] Thibault Sottiaux: So, for example, Codex is now in GitHub. [17:59] Thibault Sottiaux: You can mention Codex and it will do work for you. [18:02] Thibault Sottiaux: If you do add Codex, fix this bug or move the tests over here, it will go and run off and do it with its own little laptop on our data centers. [18:11] Thibault Sottiaux: And you don't have to think about it. [18:13] Thibault Sottiaux: but if you're working with files in a folder, [18:17] Thibault Sottiaux: you know, then you have that decision that [18:19] Thibault Sottiaux: are you going to do it in your IDE? [18:20] Thibault Sottiaux: Are you going to do it in terminal? [18:22] Thibault Sottiaux: What we're seeing is users are developing, [18:24] Thibault Sottiaux: like power users are developing [18:26] Thibault Sottiaux: very complex workflows with the terminal more. [18:29] Thibault Sottiaux: And then when you're actually working [18:30] Thibault Sottiaux: on a file or a project, [18:31] Thibault Sottiaux: you prefer to do it in the IDE. [18:33] Thibault Sottiaux: It's a bit more of a polished interface. [18:36] Thibault Sottiaux: You can undo things. [18:37] Thibault Sottiaux: You can see the edits. [18:38] Thibault Sottiaux: You know, it's not like just scrolling by you. [18:40] Thibault Sottiaux: And then the terminal is just an amazing, [18:42] Thibault Sottiaux: also vibe coding tool where, you know, [18:44] Thibault Sottiaux: if you don't really care that much [18:45] Thibault Sottiaux: about the code that's being produced, [18:47] Thibault Sottiaux: you know, you can just generate a little app. [18:48] Thibault Sottiaux: It's much more about that interaction. [18:50] Thibault Sottiaux: It elevates the interaction more [18:51] Thibault Sottiaux: instead of focusing on the code. [18:53] Thibault Sottiaux: So it's more focused on the outcome. [18:56] Thibault Sottiaux: And it's just sort of like depends [18:57] Thibault Sottiaux: on what you want to do, [18:58] Thibault Sottiaux: but it's still very much [18:59] Thibault Sottiaux: an experimentation phase right now. [19:01] Thibault Sottiaux: And we're trying different things out. [19:03] Thibault Sottiaux: And, you know, it's going to continue [19:04] Thibault Sottiaux: like that, I think. [19:05] Greg Brockman: Yeah, I really agree with that. [19:07] Greg Brockman: And I also think that a lot of our direction [19:10] Greg Brockman: will be more integration across these things. [19:13] Greg Brockman: I guess people are capable of using multiple tools, right? [19:16] Greg Brockman: You already have your terminal, your browser, [19:17] Greg Brockman: your GitHub web interface, [19:19] Greg Brockman: your repo on your local machine. [19:22] Greg Brockman: Each of these is something people have kind of learned [19:24] Greg Brockman: when it's appropriate to reach for what tool. [19:28] Greg Brockman: And I think that because we're in this experimentation phase, [19:31] Greg Brockman: that these things can feel very disparate [19:32] Greg Brockman: and very different. [19:33] Greg Brockman: And like, you have to kind of learn a new set of skills [19:37] Greg Brockman: and the affordances of the relevant tool. [19:39] Greg Brockman: And I think that a lot of as we're iterating what's on us is to really think about how do these fit together. [19:45] Greg Brockman: And so you can start to see it right with the codex IDE extension being able to run remote codex tasks. [19:52] Greg Brockman: And I think that ultimately our vision is that there should be a AI that has access to its own computer, its own clusters, but is also able to look over your shoulder. [20:03] Greg Brockman: Right. They can also come and help you locally. And these shouldn't be distinct things. [20:07] Thibault Sottiaux: Right. And it's like this one coding entity that is there to help you and collaborate with you. [20:11] Thibault Sottiaux: Like when I collaborate with Greg, you know, I don't complain that sometimes you're on Slack. [20:15] Thibault Sottiaux: Sometimes I talk to you in person. [20:16] Greg Brockman: Sometimes you complain. [20:19] Thibault Sottiaux: Sometimes we interact like through a GitHub review. [20:22] Thibault Sottiaux: Like this seems like very natural when you interact with other humans and collaborators. [20:25] Thibault Sottiaux: And this is also where, you know, how we're thinking about Codex as an agentic like entity that is really meant to just supercharge you when you're trying to achieve things. [20:35] Andrew Mayne: So let's talk about some of the ways of using it, like agents.md. [20:38] Andrew Mayne: Do you want to explain that? [20:40] Thibault Sottiaux: Yeah, agents.md is a set of instructions that you can give to Codex that lives alongside your code. [20:46] Thibault Sottiaux: So that Codex has a little bit more context about how to best navigate the code and accomplish the tasks. [20:53] Thibault Sottiaux: There are two main things that are useful to put in agents.md that we find is like helping with it's like a compression thing where it is a little bit more efficient for the agent to just read codex.md. [21:05] Thibault Sottiaux: instead of like exploring the entire code base. [21:07] Thibault Sottiaux: And then preferences that are actually not clear [21:10] Thibault Sottiaux: in the code base itself, [21:12] Thibault Sottiaux: where you would be like, you know, [21:14] Thibault Sottiaux: actually tests should be over here [21:16] Thibault Sottiaux: or, you know, I like things to be done [21:18] Thibault Sottiaux: in this particular fashion. [21:21] Thibault Sottiaux: And those two things, you know, preferences [21:23] Thibault Sottiaux: and then explaining to the agent [21:25] Thibault Sottiaux: how to navigate the code base effectively [21:27] Thibault Sottiaux: are very useful things to have in agents.mg. [21:29] Thibault Sottiaux: Yep. [21:30] Greg Brockman: And I think that there's something deeply fundamental here [21:33] Greg Brockman: of how do you communicate to an agent [21:35] Greg Brockman: that has no context what you want, [21:38] Greg Brockman: what your preferences are, [21:39] Greg Brockman: and to try to save it a little bit of the kind of spin up [21:42] Greg Brockman: a human would require, right? [21:43] Greg Brockman: We do this for humans, right? [21:44] Greg Brockman: We write readme.mds. [21:47] Greg Brockman: And this is just a convention for a name of a file [21:50] Greg Brockman: that an agent should go look at. [21:52] Greg Brockman: But there's also something that's a little point in time, right? [21:54] Greg Brockman: That the agents right now don't have great memory, right? [21:57] Greg Brockman: It's like if you're running your agent for the 10th time, [22:01] Greg Brockman: Has it really benefited from the nine times that it went and solved a hard problem for you? [22:05] Greg Brockman: And so I think that we have real research to do to think about how do you have memory? [22:08] Greg Brockman: How do you have an agent that really just goes and explores your code base and really deeply understands it and then is able to leverage that knowledge? [22:16] Greg Brockman: And so this is one of the examples, and there are many, where we see great fruit on the horizon for further research progress. [22:25] Andrew Mayne: It's a very competitive landscape now. [22:27] Andrew Mayne: There was a point where, you know, OpenAI kind of came out of nowhere for a lot of people. [22:31] Andrew Mayne: And all of a sudden there was GPT-3, then there was GPT-4. [22:34] Andrew Mayne: And then I think Anthropics building great models. [22:37] Andrew Mayne: And Gemini, you know, from Google has gotten really good. [22:40] Andrew Mayne: How do you guys see the landscape? [22:42] Greg Brockman: How do you see your placement there? [22:44] Greg Brockman: I mean, I think that there's a lot of progress to be had. [22:47] Greg Brockman: I focus a little less on the competition and a little more on the potential, right? [22:53] Greg Brockman: Because we started OpenAI 2015 thinking that AGI is going to be possible, maybe sooner than people think. [23:00] Greg Brockman: And we just want to be a positive force in how it plays out, right? [23:03] Greg Brockman: And really thinking about what does that mean, trying to connect that to practical execution has been a lot of the task. [23:08] Greg Brockman: And so as we started to figure out how to build capable models that are actually useful, right, that can actually help people, actually bringing that to people is this really critical thing. [23:18] Greg Brockman: And you can look at choices that we've made along the way, for example, releasing ChatGPT and making ChatGPT free tier available widely. [23:26] Greg Brockman: That's something that we do because of our mission, because we really think about we want AI to be available and accessible and benefit everyone. [23:33] Greg Brockman: And so in my view, the most important thing is to continue on that exponential progress and really think about how to bring it to people in a positive and useful way. [23:42] Greg Brockman: So I really look at where we're at right now is that these models, like there's the GPT-4 class of pre-trained models. [23:50] Greg Brockman: There's reinforcement learning on top of it to make it just much more reliable and smart. [23:55] Greg Brockman: It's like you think about if you've just sort of read the internet, you've just observed a bunch of sort of human thought. [24:04] Greg Brockman: And you're trying to write some code for the first time, you're probably going to have a bad time of it. [24:07] Greg Brockman: But if you've had the ability to actually try to solve some hard code problems, you have a Python interpreter, you have access to the kinds of tools that humans do, then you're going to be able to become much more robust and refined. [24:23] Greg Brockman: So we now have these pieces working together in concert, but we got to keep pushing them to the next level. [24:27] Greg Brockman: It's very clear that things like being able to refactor massive code bases, like no one's cracked that just yet. [24:35] Greg Brockman: There's no fundamental reason we can't. [24:37] Greg Brockman: And I think the moment you get that, I think refactoring code is one of the killer use cases for enterprise, right? [24:43] Greg Brockman: It's, you know, if you could bring down the cost of code migrations by, you know, 2x, I think you'll end up with 10x more of them happening. [24:51] Greg Brockman: Think about the number of systems that are stuck in COBOL. [24:53] Greg Brockman: And there's no COBOL programmers being trained, right? [24:56] Greg Brockman: It's just like it's strictly like, you know, building liability for the world to have this dependency. [25:02] Greg Brockman: Like the only way through is by building systems that can actually tackle that. [25:05] Greg Brockman: So I just think it's a massive open space. [25:07] Greg Brockman: The exponential continues and we need to stay on that. [25:11] Andrew Mayne: My favorite thing today that happened was there was a tweet from OpenAI, which was showing people how to use the CLI to switch from the completions API to the responses API. [25:22] Thibault Sottiaux: That's a great use. [25:23] Thibault Sottiaux: I expect to see more of that. [25:24] Thibault Sottiaux: You know, where you have special instructions given to Codex in order to go do like refactorings reliably. [25:31] Thibault Sottiaux: And then you just set it off and it does it for you. [25:33] Thibault Sottiaux: That's like a wonderful thing. [25:35] Thibault Sottiaux: migrations are some of the worst things. [25:38] Thibault Sottiaux: Nobody wants to do migrations. [25:40] Thibault Sottiaux: Nobody wants to change from one library to the other [25:44] Thibault Sottiaux: and then make sure that everything still works. [25:46] Thibault Sottiaux: If we can automate most of that, [25:48] Thibault Sottiaux: that's going to be a very beautiful contribution. [25:51] Greg Brockman: I think there's a lot of other ground as well. [25:54] Greg Brockman: I think that security patching is a good example of something [25:57] Greg Brockman: that I think will become very important soon, [26:00] Greg Brockman: and that's something we're being very thoughtful about. [26:02] Greg Brockman: I think that being able to actually have AIs that produce new tools, right? You think about how important the Unix set of standard tools has been and AIs, they're actually able to build their own tools that are useful for you, are useful for themselves. [26:20] Greg Brockman: You can actually build up a ladder of complexity there or utility there to be able to just like continue to improve this flywheel of efficiency. [26:27] Greg Brockman: AIs that are actually really not just writing code, but able to execute their own, be able to administer services or be able to do SRE work and things like that. [26:40] Greg Brockman: I think all of that is on the horizon. [26:42] Greg Brockman: It's like starting to happen, but it's not really happening yet in the way that we would like to see. [26:48] Thibault Sottiaux: One big one that we cracked internally at OpenAI and then we decided to release it as a code review, where we started to notice that the big bottleneck for us was with increased amounts of code needing to be reviewed is like the amount of review simply that people had to do on the teams. [27:06] Thibault Sottiaux: And so we decided to really focus on a very high signal codex mode where it's able to review a PR and really think deeply about the contract and the intention that you were meaning to implement and then look at the code and validate whether that intention is matched and found in that code. [27:28] Thibault Sottiaux: And it's able to go layers deep, look at all the dependencies, think about the contract and really raise things that some of our best employees, some of our best reviewers wouldn't have been able to find unless they were spending hours really deeply thinking about that PR. [27:45] Thibault Sottiaux: And we released this internally first at OpenAI. [27:48] Thibault Sottiaux: It was quite successful and people were upset actually when it broke because they felt like they were losing that safety net. [27:54] Thibault Sottiaux: And it accelerated teams and including the Codex team tremendously. [27:59] Thibault Sottiaux: The night before we released the IDE extension, one of the top engineers on my team was like cranking out 25 PRs. [28:08] Thibault Sottiaux: And we were finding quite a few bugs automatically. [28:11] Thibault Sottiaux: Codex was finding quite a few bugs. [28:13] Thibault Sottiaux: And, you know, we were able to put out an IDE extension that was almost bug free the next day. [28:18] Thibault Sottiaux: So the velocity there is incredible. [28:20] Greg Brockman: And it's very interesting that for the code review tool in particular, people were very nervous about having this enabled because I think our previous experience with every auto code review experiment that we've tried is that it's just noise. [28:36] Greg Brockman: Right. [28:36] Greg Brockman: You just get an email from some bot and you're like, oh, another one of those things. [28:39] Greg Brockman: You ignore it. [28:41] Greg Brockman: And I think we've had kind of the opposite finding from where we are now. [28:45] Greg Brockman: And it really shows you when the capability is below threshold, it just feels like this thing is like totally net negative. [28:52] Greg Brockman: I don't want to hear about it. [28:53] Greg Brockman: I don't want to see it. [28:54] Greg Brockman: Once you kind of crack above some threshold of utility, suddenly people want it, right, and get very upset if it gets taken away. [29:01] Greg Brockman: And I think also our observation is if something kind of works in AI right now, one year from now, it'll be incredibly reliable, incredibly mission critical. [29:10] Greg Brockman: And I think that that's where we're going with code review. [29:12] Thibault Sottiaux: Part of the interesting things there with CodeReview as well is like bringing humans along and really have this be a collaborator, including and review. [29:20] Thibault Sottiaux: And one thing we taught a lot about is like, how can we raise those findings so that you are actually excited to read this finding and you might even learn something, including when it's wrong. [29:32] Thibault Sottiaux: Like, you know, you can actually understand its reasoning. [29:35] Thibault Sottiaux: Most of the time, like actually more than 90% of the time it's right. [29:39] Thibault Sottiaux: And you often learn something as the person who authored the code or someone who is helping review the code. [29:45] Greg Brockman: Yeah, just circling back to what we were saying earlier about the rate of progress and sometimes stepping back and thinking about how things were earlier. [29:52] Greg Brockman: I remember for GPT-3 and for GPT-4 really focusing on the doubling down problem. [29:59] Greg Brockman: Do you remember if the AI would say something wrong and you'd point out the mistake? [30:03] Andrew Mayne: Oh, it would argue with you. [30:04] Greg Brockman: Oh, yeah. [30:05] Greg Brockman: It would try to convince you that it was right. [30:07] Greg Brockman: We're so far past that being the core problem. [30:10] Greg Brockman: I'm sure it happens in some obscure edge cases, just like it does for humans. [30:13] Greg Brockman: But it's really amazing to see that we're at a level where even when it's not quite zeroed in on the right thing, it's highlighting stuff that matters. [30:20] Greg Brockman: It has pretty reasonable thoughts. [30:22] Greg Brockman: And I always walk away from these code reviews thinking like, huh, OK, yeah, that's a good point. [30:28] Greg Brockman: I should be thinking about that. [30:29] Andrew Mayne: We're now just getting to launch a GPT-5. [30:34] Andrew Mayne: And as the recording of this podcast, we now have GPT-5 Codex. [30:39] Thibault Sottiaux: Which we're tremendously excited about. [30:41] Thibault Sottiaux: Very excited. [30:41] Andrew Mayne: Why should I be excited about this, gentlemen? [30:44] Andrew Mayne: Sell me on this. [30:47] Thibault Sottiaux: GPT-5 Codex is a version of GPT-5 that we have optimized for Codex. [30:51] Thibault Sottiaux: And we talked about the harness. [30:52] Thibault Sottiaux: And so it's optimized for the harness. [30:53] Thibault Sottiaux: We really consider it to be like one agent where you couple the model very closely to the set of tools. [30:59] Thibault Sottiaux: And it's able to be even more reliable. [31:01] Thibault Sottiaux: One of the things that this model exhibits is an ability to go on for much longer and to really have that grit that you need on these complex refactoring tasks. [31:10] Thibault Sottiaux: But at the same time, for simple tasks, it actually comes way faster at you and is able to reply without much thinking. [31:17] Thibault Sottiaux: And so it's like this great collaborative where you can ask questions about your code, find where this piece of code is that you need to change or better understand, plan. [31:26] Thibault Sottiaux: But at the same time, once you let it go onto something, it will work for a very, very long period of time. [31:31] Thibault Sottiaux: We've seen it work internally up to seven hours for very complex refactorings. [31:35] Thibault Sottiaux: We haven't seen other models do that before. [31:38] Thibault Sottiaux: And we also have really worked tremendously on code quality. [31:42] Thibault Sottiaux: And it's just really optimized for what people are using GPT-5 within Codex for. [31:47] Andrew Mayne: So when you talk about working longer and you say it worked up to seven hours, you're not just talking about it keeps putting things back into context, [31:55] Andrew Mayne: that it's actually making decisions, [31:57] Andrew Mayne: deciding what's important and moving forward? [32:00] Thibault Sottiaux: Yes. [32:00] Thibault Sottiaux: So imagine like a really tricky refactoring. [32:02] Andrew Mayne: Right. [32:03] Thibault Sottiaux: We've all had to deal with those [32:05] Thibault Sottiaux: where you've decided that your code base is unmaintainable. [32:08] Thibault Sottiaux: You need to make a couple of changes [32:10] Thibault Sottiaux: in order to move forward. [32:11] Thibault Sottiaux: So you make a plan and then you let the model go. [32:14] Thibault Sottiaux: You let Codex, GPT-5 Codex go at it. [32:16] Thibault Sottiaux: And it will just like work its way through all of the issues, [32:19] Thibault Sottiaux: get the test to run, get the test to pass, [32:21] Thibault Sottiaux: and just completely finish the refactoring. [32:23] Thibault Sottiaux: This is like one of the things that we've seen it do seven hours. [32:25] Greg Brockman: Yeah the thing that i find so remarkable is that the core intelligence of these [32:31] Greg Brockman: models is clearly so just like stunning right i think that even three six months ago i think our [32:39] Greg Brockman: models were better than i am at navigating our internal code base right to find a specific piece [32:43] Greg Brockman: of functionality and that requires some really sophisticated [32:49] Andrew Mayne: Are you gonna have to let yourself go? Are you like "Greg, i'm sorry..." [32:53] Greg Brockman: because that's the thing is i get to do more it's like is what i want spend my time doing is what I want people to know me for is like being able to find functionality [32:58] Greg Brockman: code base. Like absolutely not. Right. That's not how I define my value as an engineer, what I want [33:03] Greg Brockman: to spend my time on as an engineer. And now I think that that to me is the core of it, right? [33:08] Greg Brockman: That there's this amazing intelligence and that it can, first of all, suck away all the like kind [33:13] Greg Brockman: of mundane, boring parts. And certainly some of the, there are some fun parts too, right? Like, [33:17] Greg Brockman: you know, I think that really thinking about the architecture of things, it's a great partner, [33:21] Greg Brockman: But I get to choose how I spend my time. Right. And I get to think about how many of these agents you want running on what tasks, how do I break down things? And so I view it as increasing the opportunity surface for programmers. And, you know, I'm an Emacs user through and through. I, you know, I started using, you know, VS Code and Cursor and Windsurf and these things, partly to just just try things out, but partly because I like the diversity of different tools, but it's really hard to get me out of my terminal. [33:47] Greg Brockman: Wow. And so but, you know, I have found that we're now above threshold where I really find myself missing the like I'm like doing some refactor. [33:58] Greg Brockman: I'm like, why am I typing this thing? Right. Like, you know, it's like you're trying to remember exactly the syntax for a specific thing or like trying to, you know, sort of do these very mechanical things. [34:07] Greg Brockman: I'm like, I just want to like have an intern go do the thing. But I have that now in my terminal. [34:12] Greg Brockman: And I think it's really amazing that we're at the point that you have this core intelligence and that you get to pick and choose when and how to use it. [34:19] Andrew Mayne: Please add a whisper to the extension, too, because now I just love to talk to the model and tell it to do things. [34:25] Greg Brockman: Yeah. [34:26] Greg Brockman: Yeah, you should be able to video chat with your model. [34:28] Greg Brockman: I think we're heading towards a real collaborator, a real coworker. [34:31] Andrew Mayne: Well, yeah, let's talk about the future. [34:32] Andrew Mayne: Where do you see this headed? [34:33] Andrew Mayne: Where do you see what's exciting about the agentic future? [34:36] Andrew Mayne: How are we going to be using these systems? [34:38] Thibault Sottiaux: We have strong conviction that the way that this is headed is large populations of agents somewhere in the cloud that we as humanity, as people, teams, organizations supervise and steer in order to produce great economical value. [34:57] Thibault Sottiaux: So if we're going a couple of years from now, this is what it's going to look like. [35:01] Thibault Sottiaux: It's millions of agents working in our and companies' data centers in order to do useful work. [35:09] Thibault Sottiaux: Now the question is, how do we get there gradually? [35:12] Thibault Sottiaux: And how do we get to experiment on the right form factor and the right interaction patterns here? [35:18] Thibault Sottiaux: One of the things that is incredibly important to solve is the safety, security, alignment of all of this. [35:26] Thibault Sottiaux: So that agents can perform useful work, but in a safe way. [35:31] Thibault Sottiaux: And you get to always stay in control as the operator, as a human. [35:36] Thibault Sottiaux: And this is why for Codex CLI, by default, the agent operates in a sandbox and is able to edit files randomly on your computer. [35:46] Thibault Sottiaux: And we're going to be continuing to invest a lot in making basically the environment safe, invest in understanding when humans need to steer, when humans need to approve certain actions, giving more and more permissions so that your agent has its own set of permissions that you allow it to use and maybe escalate permissions when you allow it to do exceptionally more risky things. [36:13] Thibault Sottiaux: And so figuring out this entire system and then making it multi-agent and steerable by individuals, teams, organizations, and aligning that with the whole intent of organizations, this is where it's headed for me. [36:26] Thibault Sottiaux: It's a bit nebulous, but it's also very exciting, I think. [36:29] Greg Brockman: Yeah, I think it's exactly right. [36:31] Greg Brockman: I mean, I think at a zoomed in level, there's a bunch of technical problems that need to be solved. [36:35] Greg Brockman: Like Thibaut is kind of getting at scalable oversight. [36:38] Greg Brockman: Right. How do you as a human manage agents that are out there writing lots of code? [36:43] Greg Brockman: Right. You probably don't want to read every line of code. [36:45] Greg Brockman: Probably right now, most people do not read all the code that comes out of these systems. [36:49] Greg Brockman: But how do you? [36:52] Greg Brockman: But how do you maintain trust? [36:54] Greg Brockman: Right. How do you make sure that that AI is producing things that are actually correct? [36:58] Greg Brockman: And I think that there are technical approaches. [37:00] Greg Brockman: And we've been thinking about these kinds of things since probably 2017 is the first time we published some strategies for how you can have. [37:07] Greg Brockman: humans or weaker AIs start to supervise even stronger AIs and kind of bootstrap your way to [37:12] Greg Brockman: making sure that as they're doing very capable, important tasks, that we can maintain trust and [37:18] Greg Brockman: oversight and really be in the driver's seat. So that's a very important problem. And it really is [37:23] Greg Brockman: exemplified in a very practical way through thinking about more and more capable coding agents. [37:28] Greg Brockman: But I think there's also other dimensions that are very easy to miss because I think at each level [37:33] Greg Brockman: of AI capability, people kind of overfit to what they see and think, oh, this is AI. This is what [37:38] Greg Brockman: AI is going to be. But the thing we haven't quite seen yet is AI is solving really hard novel [37:45] Greg Brockman: problems. Right now you think of it as, okay, I need to do my refactor. You at least have a shape [37:50] Greg Brockman: of what that thing would be. It'll do a lot of the work for you, save a lot of time. But what [37:55] Greg Brockman: about solving problems that are fundamentally unsolvable through any other means? And I think [38:02] Greg Brockman: of this not necessarily just in the coding domain, but think of it in medicine, producing [38:07] Greg Brockman: new drugs. [38:07] Greg Brockman: Think of it in material science, producing new materials that have novel properties. [38:11] Greg Brockman: And I think that there's a lot of new capability coming down the pike that is going to unlock [38:20] Greg Brockman: these kinds of applications. [38:22] Greg Brockman: And so for me, one big milestone is the first time that you have an artifact produced by [38:27] Greg Brockman: an AI that is extremely valuable and interesting unto itself. [38:31] Greg Brockman: Not because it was produced by an AI, not because it was cheaper to produce, but because it's simply like a breakthrough. [38:36] Greg Brockman: It's simply something that is just novel and that the AI, you don't even necessarily have it to be autonomously created by the AI, but just in partnership with humans and that the AI has a critical dependency. [38:46] Greg Brockman: And so I think we're starting to see signs of life on this kind of thing. [38:49] Greg Brockman: We're seeing it in life sciences where humans ask, you know, human experimenters ask o3 for five ideas of experimental, you know, protocols to run. [39:00] Greg Brockman: They try out the five of them. [39:01] Greg Brockman: Four of them don't work. [39:02] Greg Brockman: One of them does. [39:04] Greg Brockman: And the kind of feedback that we've been getting, and this was back in the o3 days, is that it's kind of the results are at the level of what you'd expect from like a third or fourth year PhD student, which is crazy. [39:14] Greg Brockman: Yeah. [39:14] Greg Brockman: Crazy. [39:15] Greg Brockman: And that was o3. [39:17] Greg Brockman: Right? [39:17] Greg Brockman: GPT-5 and GPT-5 Pro. [39:18] Greg Brockman: We're seeing totally different results there. [39:20] Greg Brockman: There we're seeing research scientists saying, okay, yeah, this is doing real novel stuff. [39:25] Greg Brockman: And sometimes it's, again, it's not just on its own solving these grand theories, but it's together in partnership being able to just stretch far beyond where a human unassisted could go. [39:35] Greg Brockman: And that to me is like one of the critical things that we need to continue to push on and get right. [39:40] Andrew Mayne: One of the challenges I have when talking to people kind of about the future, and I want to hear you guys talk about this, is that people tend to imagine the future as kind of the present, but with like shiny clothes and robots. [39:50] Andrew Mayne: And they think about like, well, then what happens when robots do all the code and all that? [39:54] Andrew Mayne: And you brought up a fact that like the things you like to do and the things you don't care to do. [39:59] Andrew Mayne: Where are we in 2030? [40:00] Andrew Mayne: What does it look like? [40:01] Andrew Mayne: It was five years ago, GPT-3. [40:03] Andrew Mayne: Now, five years from now. [40:04] Andrew Mayne: 2030. [40:06] Thibault Sottiaux: We didn't have these still six months ago. [40:09] Thibault Sottiaux: So it's hard to picture exactly what this is going to look like five years from now. [40:15] Thibault Sottiaux: But one thing that is. [40:17] Andrew Mayne: I'm going to pop out of the bushes five years from now with this podcast and be like, you said this. [40:21] Greg Brockman: Well, your agent will do it for you. [40:23] Andrew Mayne: Yeah, it's going to. [40:24] Thibault Sottiaux: So one thing that's important is like the things that are the pieces of code that are critical infrastructure and underpinning society, we need to continue to understand and have the tools to understand. [40:38] Thibault Sottiaux: And this is why also we were thinking about code review is like and code review should help you understand that code and be this teammate that helps you dive into the code written by someone else, potentially help with AI. [40:51] Greg Brockman: And I would actually argue that we already have a problem of there's lots of code out there that is not necessarily secure. [41:00] Greg Brockman: Right. [41:01] Greg Brockman: This happens all the time. [41:02] Greg Brockman: I remember like Heartbleed back, I guess it's almost 12 years ago or something. [41:09] Greg Brockman: Critical vulnerability and a key piece of software used across the Internet. [41:14] Greg Brockman: And you realize that that's not singular, right, that there's lots of vulnerabilities out there that no one has found. [41:20] Andrew Mayne: All these packages and stuff from NPM and all these packages that are just sitting there that people put exploits into. [41:26] Greg Brockman: And the way that it's always worked is that there's a cat and mouse game between attackers getting more sophisticated, defenders getting better. [41:32] Greg Brockman: And I think that with AI, you're like, well, maybe it's going to like which side will advantage the most. [41:38] Greg Brockman: Maybe it'll just sort of accelerate this cat and mouse. [41:40] Greg Brockman: But I think that there's some hope that actually you can unlock fundamental new capabilities through AI. [41:46] Greg Brockman: for example, formal verification that are sort of an end game for defense. [41:52] Greg Brockman: And I think that that to me is very exciting is thinking about not just how do you continue this, [41:58] Greg Brockman: like, you know, sort of never ending rat race, but how do you actually end up with increased [42:03] Greg Brockman: stability, increased understandability? And I think that there's other opportunities like that [42:08] Greg Brockman: for us to really understand our systems in a way that right now it's almost, you know, we're sort [42:12] Greg Brockman: at the edge of human understanding of the traditional software systems that have been built. [42:18] Thibault Sottiaux: One of the reasons we built Codex is to improve the infrastructure and the code out there in the [42:24] Thibault Sottiaux: world, not necessarily to increase the amount of code in the world. And so this is a very important [42:29] Thibault Sottiaux: point where it's also helping find bugs, helping refactor, helping find more elegant, more [42:37] Thibault Sottiaux: performant implementations that achieve the same thing or actually are more general, [42:41] Thibault Sottiaux: but not necessarily ending up with like 100 million lines of code that you don't understand. [42:47] Thibault Sottiaux: One thing that I'm really excited about is how Codex can help teams, individuals, [42:50] Thibault Sottiaux: just write better code, be better software engineers, [42:53] Thibault Sottiaux: and end up with simpler systems that are actually doing more things for us. [42:58] Greg Brockman: I think part of the 2030 outlook is we will be in a world of material abundance. [43:04] Greg Brockman: I think that AI is going to make it much easier than you could almost imagine to create anything you want. [43:10] Greg Brockman: And that will probably be true in the physical world in addition to the digital world in ways that are hard to predict. [43:16] Greg Brockman: But I think it will be a world of absolute compute scarcity. [43:18] Greg Brockman: And we've seen a little bit of what this is like within OpenAI. [43:22] Greg Brockman: The way that different research projects fight over compute or that the success of the research program is determined by the compute allocation is something that is hard to overstate. [43:35] Greg Brockman: And I think that we're going to be in a world where your ability to produce and create whatever you imagine will be limited, partly by your imagination, but partly by the compute power behind it. [43:45] Greg Brockman: And so one thing we think about a lot is how do we increase the supply of compute in the world? [43:49] Greg Brockman: We want to increase the intelligence, but also the availability of that intelligence. [43:52] Greg Brockman: And fundamentally, it is a physical infrastructure problem, not just a software problem. [43:56] Thibault Sottiaux: I know with GPT-5, I think one thing that's quite amazing is we're able to give it as part of the free, the plus plan, the pro plan. [44:05] Thibault Sottiaux: It's like you can use Codex with your plus plan. [44:08] Thibault Sottiaux: You get GPT-5, like the same version that everyone else gets. [44:11] Thibault Sottiaux: And it's like this incredible intelligence. [44:13] Thibault Sottiaux: But the model is also incredibly cost effective in that way. [44:17] Andrew Mayne: I think that was one of the things that really stood out for me was I thought the model was much more capable, but it came out at the same price point or some ways cheaper than the previous model. [44:26] Andrew Mayne: And that was something like, wow, that patterns are great. [44:30] Greg Brockman: I think the degree to which we are improving the intelligence and cutting prices is something that is very easy to miss, take for granted. [44:37] Greg Brockman: But it's actually crazy, right? [44:38] Greg Brockman: I think we did like an 80% price cut on o3 or something like that. [44:43] Greg Brockman: If you just look at your point of like $0.06 per thousand tokens back for GPT-3 level intelligence. [44:49] Andrew Mayne: Yeah. [44:49] Andrew Mayne: There was an article that came out. [44:50] Andrew Mayne: One of the newspapers was complaining that, well, these reasoning models have made it more expensive. [44:54] Andrew Mayne: but they didn't compare reasoning models to reasoning models in like the last six to seven months and how much more efficient they've become. [44:57] Greg Brockman: Yep. And that, that will just continue, [45:02] Greg Brockman: you know, on the compute scarcity point, one thing that I find very sort of suggestive is [45:08] Greg Brockman: thinking about, you know, right now people talk about building big, you know, big fleets of a [45:14] Greg Brockman: million GPUs of millions of GPUs, that level of, of GPUs. But if we reach a point, which is probably [45:21] Greg Brockman: not in that far future where you're going to want agents running on your behalf constantly, [45:26] Greg Brockman: right? Like it's reasonable for every person to want a dedicated GPU just for them running their [45:30] Greg Brockman: agent. And so now you're talking almost 10 billion GPUs that we need. We're orders of magnitude off [45:36] Greg Brockman: of that. And so I think that part of our job is to figure out how to supply that compute, [45:40] Greg Brockman: how to make it exist in the world, but how to make the most out of the like very limited compute that [45:45] Greg Brockman: exists right now. And, you know, that's an efficiency problem. It's also an increase that [45:50] Greg Brockman: intelligence problem. But yeah, I think it's very clear that bringing this to fruition is going to [45:55] Greg Brockman: be just like a lot of work and a lot of building. [46:00] Thibault Sottiaux: So one of the interesting things about agents and the relationship to GPUs and them acting is that it is very beneficial to have a GPU also close to [46:06] Thibault Sottiaux: you. Because, you know, when it's acting and doing 200 tool calls over the span of like a couple of [46:12] Thibault Sottiaux: minutes, it's always is doing this back and forth between the GPU and like your laptop and executing [46:17] Thibault Sottiaux: those two goals, getting that context back, and then continuing to reflect. [46:20] Thibault Sottiaux: And so bringing GPUs close to people is a great contribution there as well. [46:28] Thibault Sottiaux: And it really benefits because it reduces the latency tremendously of the entire interaction [46:33] Thibault Sottiaux: and the entire rollout. [46:35] Andrew Mayne: Gentlemen, we get the question that comes up periodically about the future, about labor, [46:39] Andrew Mayne: about all of this. [46:41] Andrew Mayne: Number one, learn to code, not learn to code. [46:44] Thibault Sottiaux: I think it's a wonderful time to learn to code. [46:46] Greg Brockman: I think, yeah, I agree. Definitely learn to code, but learn to use AI. That to me is the most important thing. [46:53] Thibault Sottiaux: There's something tremendously enjoyable about using Codex to learn about a new programming language. [46:58] Thibault Sottiaux: A lot of people on my team were new to Rust and we decided to build a core harness in Rust. [47:07] Thibault Sottiaux: And it's been really great seeing how quickly they can pick up a new language just by using Codex, asking questions, exploring a code base that they don't know. [47:15] Thibault Sottiaux: and still achieving great results. [47:17] Thibault Sottiaux: Obviously, we also have very experienced REST engineers [47:20] Thibault Sottiaux: to continue to mentor and make sure that we have a high bar. [47:24] Thibault Sottiaux: But it's just a really fun time to learn to code. [47:27] Greg Brockman: I remember the way that I learned to program [47:29] Greg Brockman: was by W3 Schools Tutorials, PHP, JavaScript, HTML, CSS. [47:33] Greg Brockman: And I remember when I was building some of my first applications [47:36] Greg Brockman: and I was trying to figure out how to, [47:37] Greg Brockman: I didn't even know the word for it, serialized data, right? [47:40] Greg Brockman: And I came up with some sort of structure [47:44] Greg Brockman: that had some special sequence of characters [47:46] Greg Brockman: that was serving as a delimiter. [47:48] Greg Brockman: And what would happen if you actually had [47:49] Greg Brockman: that sequence of characters in your data? [47:51] Greg Brockman: Like, let's not talk about that. [47:52] Greg Brockman: So that's why I had to have a very special sequence. [47:54] Greg Brockman: And this is the kind of thing [47:55] Greg Brockman: where you're not gonna have a tutorial [47:58] Greg Brockman: that will flag this kind of issue for you. [48:00] Greg Brockman: But will Codex in its code review be like, [48:02] Greg Brockman: hey, there's JSON serialization, [48:04] Greg Brockman: just use this library? [48:05] Greg Brockman: Absolutely. [48:06] Greg Brockman: And so I think that the potential to accelerate [48:09] Greg Brockman: make it so much easier to code [48:10] Greg Brockman: so you don't have to sort of reinvent all these wheels [48:13] Greg Brockman: and that it can ask the question for you [48:16] Greg Brockman: or answer the question for you [48:17] Greg Brockman: that you don't even know that you needed to ask. [48:20] Greg Brockman: Like that to me is why I think it's like [48:21] Greg Brockman: a better time than ever to build. [48:23] Andrew Mayne: I've learned a lot just by looking at how it solves a problem. [48:26] Andrew Mayne: Found new libraries, found new methods and stuff. [48:28] Andrew Mayne: That's often, I like to sometimes give it like a crazy task. [48:31] Andrew Mayne: Like how would you create your own language model [48:33] Andrew Mayne: with only a thousand lines of code [48:35] Andrew Mayne: and what would you try to do? [48:36] Andrew Mayne: And sometimes it might fail, [48:37] Andrew Mayne: but then you can look at the direction it tried to do it [48:38] Andrew Mayne: and you go, oh, I didn't even know that was a thing. [48:41] Thibault Sottiaux: One of the things as well is that, you know, the people who are most successful coding with AI also have really studied, you know, fundamentals of software engineering and put the right framework in place, right? [48:54] Thibault Sottiaux: Architecture have taught about how to structure their code base and then are getting help from AI, but still, you know, following that general blueprint. [49:01] Thibault Sottiaux: And that, you know, really accelerates you and allows you to go like much further than, you know, you would be able to go if you actually didn't understand the code that's being written. [49:09] Andrew Mayne: Since you've launched this, since you've made this available to GPT-5, since you've been able to deploy things with Codex, what have you seen as usage rates? [49:17] Thibault Sottiaux: Yeah, usage has been exploding. [49:18] Thibault Sottiaux: So we've seen more than 10x growth in usage from across users and the users that were using it already are using it much more as well. [49:25] Thibault Sottiaux: So we're seeing more sophisticated usage and people are using it for longer periods of time as well. [49:32] Thibault Sottiaux: We have now included in the PLUS and PRO plan with generous limits, and that's contributed a lot to being successful. [49:41] Greg Brockman: Yeah, I think that the vibes, I think, also have really started to shift as people, I think, are starting to realize how you need to use GPT-5, right? [49:49] Greg Brockman: I think it's a little bit of a different flavor. [49:51] Greg Brockman: I think that we have our own spin on the right harnesses and tools and the ecosystem of how these things fit together. [49:56] Greg Brockman: And I think that once it clicks for people, then they just go so fast. [50:00] Andrew Mayne: Gentlemen, thank you so much for joining us here and talking about this. Any last thoughts? [50:04] Greg Brockman: Thank you for having us. Yeah, we're really excited about everything that comes next. I think we have so much to build. Progress continues on the exponential. And I think really bringing these tools to be usable and useful by everyone is core to our mission. [50:19] Thibault Sottiaux: Yeah, thanks for having us. I'm also super excited. Now that we have Codex and it keeps improving, we're also getting accelerated and building better Codex every day. And personally, I think I spend more time talking to Codex now than most people. And it's really how I feel the AGI, and I hope more people will be able to benefit from it.