OpenAI: How AI is reshaping the craft of building software - The Pragmatic Summit

OpenAI: How AI is reshaping the craft of building software - The Pragmatic Summit — The Pragmatic Engineer

The Pragmatic Engineer

AI-assisted summary of OpenAI: How AI is reshaping the craft of building software - The Pragmatic Summit — original source. Check the original for full context.

Source published:

Key ideas

  • Internal software engineers at OpenAI routinely consume hundreds of billions of tokens per week by running autonomous coding agents on remote server environments called 'Codex Box.' (0:00)
  • OpenAI continues aggressively hiring early-career software engineers and interns, bringing in a cohort of roughly 100 new grads who operate as 'AI-native' developers alongside flat management structures. (12:07)
  • A single product manager supports the entire Codex engineering organization by using Codex agents to ingest testing feedback, create Notion specs, and automatically assign Linear bug tickets. (19:48)
  • Leadership argues engineering organizations should reframe token consumption from infrastructure overhead into payroll calculations for synthetic 24/7 teammates. (22:52)
  • OpenAI engineering leaders project that within six months, inference speeds will jump by an order of magnitude and multi-agent systems will be able to synthesize millions of lines of code from scratch within 24 hours. (25:45)

Detailed notes

## Background Published on YouTube by The Pragmatic Engineer on February 23, 2026, this recorded session from The Pragmatic Summit features Tibo Sottiaux (Head of Engineering for Codex) and Vijaye Raji (CTO of Applications at OpenAI). The discussion explores how internal engineering workflows, tooling, organizational structures, and daily software practices are being radically transformed by AI within OpenAI.

## TL;DR Vijaye Raji and Tibo Sottiaux detail how OpenAI engineers have transitioned from manual coding to orchestrating autonomous AI teammates across dedicated remote server environments ('Codex Box') that burn hundreds of billions of tokens per week. This massive acceleration has shifted traditional software bottlenecks away from code generation and toward code review, CI/CD pipelines, and high-level architectural guardrails. Rather than eliminating entry-level roles, OpenAI is actively hiring 'AI-native' new graduates while empowering PMs and designers to build production-grade prototypes directly. Ultimately, the speakers urge engineering leaders to view token expenditure as payroll for tireless digital colleagues and predict that multi-agent orchestration swarms and symptom-based debugging will become standard within months.

## Atmosphere & style The discussion carries an energetic, pragmatic, and slightly surreal vibe, as everyday engineering rituals at OpenAI—like dispatching background AI threads during live meetings to resolve data questions before the call ends—feel like scenes lifted straight out of science fiction. Gergely Orosz acts as a grounded, sharp sounding board, posing hard questions about junior skill erosion, gatekeeping, and mounting token expenses. Raji and Sottiaux answer with candid, unvarnished enthusiasm, pairing a twenty-five-year historical perspective on tech cycles with a palpable sense of awe at the blinding speed of the current transformation.

## Bulletin - [00:00](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=0s) Internal software engineers at OpenAI routinely consume hundreds of billions of tokens per week by running autonomous coding agents on remote server environments called 'Codex Box.' - [12:07](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=727s) OpenAI continues aggressively hiring early-career software engineers and interns, bringing in a cohort of roughly 100 new grads who operate as 'AI-native' developers alongside flat management structures. - [19:48](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1188s) A single product manager supports the entire Codex engineering organization by using Codex agents to ingest testing feedback, create Notion specs, and automatically assign Linear bug tickets. - [22:52](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1372s) Leadership argues engineering organizations should reframe token consumption from infrastructure overhead into payroll calculations for synthetic 24/7 teammates. - [25:45](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1545s) OpenAI engineering leaders project that within six months, inference speeds will jump by an order of magnitude and multi-agent systems will be able to synthesize millions of lines of code from scratch within 24 hours.

## Key takeaways - [00:00](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=0s) Coding tools have rapidly evolved from passive autocomplete utilities to autonomous agents and named teammates embedded in developer workflows. - [00:00](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=0s) Remote execution environments like 'Codex Box' decouple long-running agentic coding tasks from local developer machines. - [02:19](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=139s) Engineering bottlenecks continuously shift upstream as code generation accelerates, moving from coding to review, triage, user feedback synthesis, and product strategy. - [06:43](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=403s) Engineers increasingly resolve architectural debates by having agents build multiple working prototypes in parallel rather than writing lengthy speculative design documents. - [06:43](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=403s) Designers at OpenAI are now committing and shipping production code directly without needing dedicated software engineers to rewrite their implementations. - [09:37](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=577s) Codex runs autonomous overnight loops to conduct quality assurance, regression testing, and downstream model training with auto-generated PDF evaluation reports. - [09:37](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=577s) Unanswered telemetry questions during live meetings trigger 20-minute background Codex runs to return data before the discussion ends. - [12:07](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=727s) OpenAI is leaning heavily into hiring new graduates, viewing incoming junior engineers as AI-native talent rather than an obsolete tier of workers. - [12:07](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=727s) Flat team structures, such as a single manager overseeing 33 direct reports, are enabled by leveraging Codex as a primary onboarding mentor and architectural guide. - [15:56](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=956s) Concerns over losing foundational programming skills mirror past transitions from Assembly, C++, and pre-IntelliSense IDEs; higher abstractions shift leverage rather than dilute craftsmanship. - [18:25](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1105s) Human taste, product intuition, and user empathy remain irreplaceable differentiators even as PMs and designers increasingly write their own functional code. - [22:52](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1372s) Token expenditures should be budgeted against the business value of 24/7 digital teammates rather than restricted purely as raw compute costs. - [25:45](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1545s) The velocity and scale of the generative AI transition far outpace prior tech waves, rendering traditional two-year planning cycles unworkable. - [28:06](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1686s) Future engineering will focus heavily on enforcing system guardrails and verifying behavioral properties at the boundary rather than directly inspecting implementation code. - [28:06](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1686s) As system complexity deepens across layers of AI-generated software, debugging will shift toward diagnosing high-level emergent symptoms instead of step-by-step code execution. - [29:41](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1781s) Developer interfaces are expected to consolidate into a single personal assistant that orchestrates and summarizes the work of hundreds of underlying specialized agents.

## Detailed walkthrough ### [00:00](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=0s) Intro: What’s Changing Inside OpenAI

Vijaye Raji (CTO of Applications at OpenAI) reflects on how radically the software engineering environment has transformed over his first six months at the company. He explains that working alongside the frontier research happening internally makes projecting the near-future possibilities of software development truly mind-blowing. Writing code has undergone a fundamental transition: within half a year, tools like Codex have evolved from passive autocomplete helpers into ambient extensions, then into autonomous agents, and now into full-fledged teammates. Raji notes that engineers are even beginning to name their agents and treat them as working colleagues.

```mermaid flowchart LR accTitle: Evolution of Codex in the Developer Workflow accDescr: Progression from simple tooling to agent teammates handling dev work. A["Passive Tool"] --> B["Extension"] B --> C["Autonomous Agent"] C --> D["Teammate (Named Agent)"] D --> E["Codex Box (Remote Dev Box)"] ```

The scale of automated execution has reached unprecedented volumes within OpenAI. Raji shares that internal usage leaderboards reveal individual software engineers regularly burning through hundreds of billions of tokens per week. To support this demand, OpenAI launched an internal system called "Codex Box," which lets developers reserve remote server dev boxes to execute prompts independently. Developers orchestrate runs from their laptops, close the lid, step away to meetings, and return to find the implementation completed on the remote machine. As Raji summarizes this shift: “So I'll tell you this right, the way we write software has fundamentally changed.” He predicts that while this agentic workflow is currently concentrated at the heart of Silicon Valley, it will rapidly diffuse across the industry within months until it becomes the baseline standard for writing software.

- Internal OpenAI engineers routinely consume hundreds of billions of tokens per week executing agentic coding workflows. - "Codex Box" provisions dedicated server-side developer environments, freeing developers from running intense tasks locally on their personal machines.

---

[02:19](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=139s) How the Codex Team Works Day-to-Day

The host notes that while hearing about these workflows sounds like a fairy tale, candid conversations with unfiltered, non-media-trained engineers at OpenAI reveal that adoption varies across teams—not everyone writes 100% of their code with Codex yet. However, the Codex team itself acts as the vanguard, operating at the bleeding edge even compared to the rest of the company.

Tibo Sottiaux (Head of Engineering for Codex) describes the team's culture as a fast-evolving state where practices are reinvented almost week to week. As soon as one part of the workflow is automated, the limiting factor moves down the line:

```mermaid flowchart TD accTitle: Shifting Engineering Bottlenecks accDescr: Sequential movement of workflow bottlenecks from generation to upstream strategy. subgraph Execution ["Execution Stage"] A["Code Generation"] --> B["Code Review"] end subgraph Upstream ["Upstream Discovery Stage"] B --> C["Triage & Bug Tickets"] C --> D["Synthesizing User Feedback<br>(Twitter, Reddit)"] D --> E["Translating Signals into Product Strategy"] end ```

The team now uses agents to ingest sentiment and feedback across external surfaces like X (formerly Twitter) and Reddit, triage incoming tickets, and synthesize that unstructured user data directly into product strategy. Sottiaux recounts a recent hiring interview where a candidate negotiating an offer asked how much personal compute allocation they would receive to build products—a perk traditionally reserved solely for machine learning research scientists training frontier models. As Sottiaux puts it: “So I think there's like this shift there where you know people realize you can hyper leverage yourself in you know all sorts of like novel ways and if you do have great taste, great ideas, you know how to build software, it's like you know what a time to be alive really.”

---

[04:52](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=292s) Product Engineering and the Creative Feedback Loop

Stepping back to examine product engineering across OpenAI as a whole, Raji emphasizes that despite the automation wave, software is still built for human beings. Because of that, core human product intuition remains irreplaceable. Using tools like the 1Password desktop app workflow and new accessible interfaces to experiment, engineers still have to conceive the target product vision and continually iterate on the details. While agents might one day build products for other agents—and perhaps step into the product manager role when that happens—today's engineering value still hinges on human taste.

At the same time, the day-to-day experience of programming has become markedly more rapid and fun because the latency between having an idea and seeing it function has collapsed. Raji shares an anecdote about coding on an airplane without access to remote dev boxes: when flight attendants asked passengers to close laptops for takeoff or landing, he resorted to keeping the screen propped open just a crack under the tray table so the agent's run wouldn't terminate mid-stream. The host jokes that half the tech industry now walks around corridors with laptops propped slightly open to avoid breaking long-running agent threads. For Raji, this rapid cycle is liberating: “I actually think it's more fun now building software because the gratification cycle is so much shorter, and it's so really cool to see the product that you're building, test it, verify it, and then go back to Codex.”

---

[06:43](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=403s) New Engineering Practices and Emerging Bottlenecks

As engineering norms adapt to agentic workflows, traditional development rituals are being replaced by unconventional practices. Sottiaux points out that engineers historically resolved difficult architectural trade-offs through lengthy design documents, debating hypothetical alternatives on paper before discarding them. Today, engineers instruct agents to build multiple distinct architectural implementations simultaneously in parallel, evaluate the concrete working prototypes side by side, and move forward with the proven winner.

Role definitions are also breaking down. Designers are now committing and shipping more production code than traditional engineers did six months prior, because model generation quality has risen to the point where design-driven code can be merged directly as-is without requiring an engineer to rewrite it.

Raji adds that agents have largely eliminated the friction of obscure tooling syntax, such as parsing arcane `ffmpeg` command-line flags when processing media. Yet clearing these micro-hurdles inevitably exposes larger systemic pipeline constraints:

```mermaid flowchart LR accTitle: The Shifting Production Pipeline Bottlenecks accDescr: How accelerating individual coding speeds causes downstream workflow queues. A["5x Individual Coding Velocity"] --> B["Code Review Backlog"] B --> C["Security & Compliance Checks"] C --> D["CI/CD Integration & Deployment Pipeline"] ```

As Raji explains: “Once you solve coding, for example, now you've just made every engineer five times more productive. What's going to happen is like there's going to be more code being written, which means that code reviews will become the bottleneck, and then after code reviews, integrations and deployment CI/CD will become the bottleneck.”

Addressing how the Codex team is moving past the paradigm of short autocomplete snippets, Sottiaux highlights the emergence of autonomous overnight runs. Rather than restricting models to ten-minute micro-tasks, the team provides agents with rich environments and self-testing harnesses, enabling them to execute continuous problem-solving runs for hours on end without human intervention.

[09:37](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=577s) Autonomous Testing, AI Training AI, and "Sci-Fi" Meeting Loops

Tibo Sottiaux explains how Codex has evolved into an engine capable of running long, fully autonomous loops that previously required constant human oversight. For quality assurance, the team runs Codex overnight to autonomously test itself, executing endless QA passes and flagging code regressions before engineers return in the morning. Even more startling is its role in core research: Tibo recounts regular conversations with the team’s model-training researcher, who finds himself continually humbled by the agent's capabilities. When the researcher initially believes a task requires human intervention, he repeatedly discovers that Codex was fully capable all along—it had simply been prompted suboptimally or configured without the right setup. Tibo describes the emotional whiplash of watching an AI independently train downstream models and write structured summary reports, remarking: “And you know this is both exciting and a little bit depressing at the same time.”

Gergely Orosz remarks that the Codex team's internal workflows sound like scenes straight out of science fiction—specifically how live team meetings trigger background AI agents to resolve debates in real time. Tibo confirms this dynamic, pointing to their weekly analytics reviews covering feature adoption, retention curves, and funnel performance. Whenever an open question arises that is not answered by existing telemetry dashboards, the team does not push it to next week's backlog. Instead, the data analyst fires off a background Codex thread right at the start of the session. While the team discusses other topics for twenty minutes, the agent dives into the data and returns with answers in time for the final ten minutes of the meeting. Rather than halting discussions to gather data manually, the team treats Codex as a squad of on-demand analysts working silently in the background, a workflow that also carries over directly to incident response to diagnose outages and identify the fastest path to production recovery.

```mermaid flowchart TD accTitle: Codex In-Meeting Analytics Loop accDescr: Diagram showing how unanswered product analytics questions in team meetings trigger background Codex threads that deliver answers before the meeting concludes.

subgraph Meeting["Weekly Analytics Review"] A["Team analyzes funnel,<br/>adoption, and retention"] --> B{"Unanswered question<br/>not in dashboard?"} end

subgraph CodexBackground["Autonomous Codex Execution (~20 Mins)"] B -->|"Fire off Codex thread"| C["Query raw telemetry<br/>and diagnose data"] C --> D["Synthesize findings<br/>and insights"] end

subgraph WrapUp["Meeting Conclusion (Last 10 Mins)"] D --> E["Present answers to room"] E --> F["Team debates solutions<br/>and decides next steps"] end ```

- **Autonomous Model Training Reports**: The model conducts unsupervised training runs, evaluates performance, and outputs a formatted PDF report containing findings and suggested avenues for human researchers to iterate on. - **On-Call Acceleration**: When an incident fires or an engineer is paged, Codex acts as an interactive co-pilot, surfacing root causes, parsing system state, and generating recovery paths within minutes.

---

[12:07](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=727s) The "AI-Native" New Grads and Flat Team Dynamics

Addressing an ongoing industry debate over whether entry-level roles are being rendered obsolete by senior engineers wielding AI agents, Vijaye Raji reveals that OpenAI is aggressively moving in the opposite direction. OpenAI is actively hiring early-career software engineers straight out of college alongside a rapidly expanding internship program. Vijaye rejects the pessimistic view that juniors are squeezed out, arguing instead that graduates entering the workforce today represent an entirely new paradigm of engineers who think and operate with models intuitively from day one. Looking ahead to welcoming an incoming class of approximately one hundred new grads, Vijaye Raji summarizes his conviction: “I actually truly believe that the new software engineers that are being created are going to be AI native.”

``` "I actually truly believe that the new software engineers that are being created are going to be AI native." — Vijaye Raji ```

On the Codex engineering team specifically, onboarding takes place inside an unusually wide and flat organizational hierarchy. Tibo manages 33 direct reports, deliberately eschewing middle layers so management never becomes an operational choke point for engineers shipping at high velocity. When new hires arrive, their primary onboarding mentor is Codex itself: they query the agent to inspect architectural conventions, navigate unfamiliar repos, trace existing work, and receive daily progress summaries. Crucially, the human side of onboarding is handled by whoever joined the team most recently, ensuring that cultural norms and documentation stay fresh. Tibo highlights a new graduate named Ahmed who joined six months prior and has completely outpaced expectations, playfully joking about his own cognitive decline compared to the unbounded energy and peak mental agility of fresh graduates thriving in an agentic coding environment.

- **OpenAI New Grad Cohort**: OpenAI is welcoming a summer class of roughly 100 new college graduates and interns specifically prepared to work alongside AI tooling. - **Flat Organizational Span**: Tibo maintains 33 direct reports across the Codex team to avoid managerial bottlenecks, betting on high autonomy and automated scaffolding over bureaucratic approvals. - **Self-Directed Onboarding via AI**: New team members ramp up by prompting Codex directly about codebase mechanics and design rationale rather than relying purely on static wiki documents.

---

[15:56](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=956s) Engineering Foundations and the Shifting Stack of Abstractions

When Gergely plays devil's advocate—asking whether new developers who rely heavily on AI generation risk missing out on the core foundational skills that senior engineers spent decades honing—both Tibo and Vijaye push back on the idea that fundamentals are disappearing. Tibo explains that architectural rigor, codebase guardrails, and human code review remain paramount. Codex does not operate with free rein; human engineers establish the broader boundaries, define modular constraints, and scrutinize structural health. When thoughtful software architecture and guardrails are set up in advance, junior engineers absorb the codebase's patterns rapidly and operate with remarkable precision rather than producing chaotic spaghetti code.

Vijaye frames the anxiety around AI assistants as the latest iteration of a cyclical panic he has witnessed throughout his 25-year software career. Recalling his days at Microsoft building the Visual Studio code editor and language services, he remembers when IntelliSense was first introduced and veteran programmers insisted that real developers should never rely on autocomplete popups. The same gatekeeping arguments played out previously when developers transitioned away from Assembly language, moved past manual memory management in C++, or began relying heavily on JavaScript frameworks. In Vijaye's view, higher levels of abstraction do not erode real software craftsmanship; they simply shift where engineering leverage is applied. Emphasizing what actually preserves long-term engineering excellence, Vijaye Raji concludes: “The point is that as long as you have the strong foundations, as long as you have product intuition, know what you're building and be able to like go down up and down the stack to be able to like solve problems, those are going to be the more important ones.”

| Historical Shift | Veteran Skepticism at the Time | Long-Term Reality | |:--- |:--- |:--- | | **Assembly to Higher-Level Languages** | "Real programmers write raw machine instructions." | Higher productivity allowed orders-of-magnitude larger software systems. | | **C++ to Managed Runtimes & JS** | "Abstracted memory and scripting languages aren't serious engineering." | Software eating the world across web, mobile, and distributed cloud systems. | | **Raw Editors to Visual Studio IntelliSense** | "You're not a real developer if you need an autocomplete dot prompt." | Became an indispensable baseline standard across every modern IDE and language server. | | **Manual Coding to AI-Native Agents (Codex)** | "Relying on agents will destroy foundational engineering skills." | Shifts focus toward systems architecture, taste, product intuition, and full-stack problem-solving. |

[18:25](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1105s) The Evolving Roles of PMs and Designers

When asked whether the acceleration of feature development is blurring the lines between engineering, product management, and design, Vijaye Raji dismisses the notion that these disciplines are becoming obsolete. At their core, products are created by humans for human consumption, meaning that human taste, product judgment, and design sensibility cannot be mechanically swapped out for an algorithm. Instead of facing replacement, product managers and designers are experiencing an upward shift in abstraction and productivity. Rather than handing off static Figma files or prose specs, designers and PMs are increasingly writing code themselves, translating interface ideas into interactive, working prototypes to validate assumptions before engineering teams ever get involved.

This shift extends well beyond writing code for the core application. Team members outside of engineering are repurposing tools like Codex to supercharge their day-to-day office workflows, utilizing custom plugins and prompts to generate PowerPoint decks, model data in Excel, and automate operational tasks. Across the board, the baseline of individual productivity is climbing, enabling non-engineers to express their intent through software directly rather than acting purely as coordinators.

As Vijaye Raji summarizes this enduring human element: “I go back to the as long as we're building products for humans to use, we will need human designers, we will need human product managers.”

- Product sense and design sense remain fundamental human differentiators that AI abstractions do not substitute. - Designers and PMs now write code and build production-grade prototypes to validate concepts prior to handoff. - Codex adoption has spread across non-engineering disciplines to automate tasks like assembling presentations and managing spreadsheet workflows.

---

[19:48](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1188s) Rapid Knowledge Diffusion and the Hyper-Leveraged PM

Because OpenAI ships models and tools to the public almost immediately after validating that they work, internal teams have only a fleeting window of foresight compared to the broader tech community. To prevent good practices from siloing, the company prioritizes high-velocity knowledge diffusion through company-wide hackathons, demo days, and active Slack channels dedicated to Codex usage and "hot tips." Tibo Sottiaux emphasizes that there is currently no single dogma or "one true way" to work with generative AI, making active experimentation and cross-team sharing essential.

```mermaid flowchart TD accTitle: Bug bash automation workflow accDescr: Diagram showing how Codex automates Alexander Embiricos's bug bash process from feedback collection to ticket dispatching. subgraph Input ["1. Team Testing"] A["1-Hour Bug Bash"] --> B["Raw Feedback from Testers"] end subgraph Synthesis ["2. AI Aggregation"] B --> C["Codex Agent"] C --> D["Structured Notion Spec"] end subgraph Execution ["3. Ticket Dispatch"] D --> E["Codex Agent"] E --> F["Linear Bug Reports"] E --> G["Feature Improvement Tickets"] F --> H["Assign & Follow Up with Engineers"] G --> H end ```

To demonstrate the extent of leverage possible, Tibo highlights Alexander Embiricos, the sole product manager supporting the entire Codex engineering organization. During a recent pre-release bug bash, Embiricos ran a one-hour live testing session, dispatched Codex agents to collect feedback across participants, compiled the findings into a Notion workspace, and then had Codex automatically file distinct bug reports and improvement tickets in Linear. The agents assigned the issues to the appropriate engineers and handled direct follow-up tracking, allowing a single PM to perform the operational workload of a massive program management office. Vijaye adds that this heightened capability is evident in company demo days: rather than showing superficial prototypes, hackathon participants now present deep, fully realized implementations that comprehensively handle edge cases from day one.

Describing how product managers must scale alongside engineering throughput, Tibo Sottiaux observes: “Your product manager cannot become the bottleneck so it's like you know you need to look at it in a principled way.”

- Internal channels like `#hot-tips` and Codex-focused Slack groups function as rapid clearinghouses for emergent AI workflows. - Alexander Embiricos operates as the lone product manager for the entire Codex team by turning Codex into an autonomous program management assistant. - Internal hackathons and demo days have shifted from proof-of-concept showcases to fully usable applications with comprehensive edge-case handling.

---

[22:52](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1372s) Navigating Token Costs and the Economics of AI Teammates

While OpenAI employees enjoy the privilege of unlimited, cost-free model inference, external engineering organizations must balance their ambitions against steep API bills and subscription limits. When asked how budget-constrained teams should approach adoption, Vijaye argues that the prevailing economic mental model must shift from treating tokens as infrastructure consumables to evaluating them as payroll for synthetic colleagues. If an autonomous agent operates around the clock, accepts Linear or Jira tickets, and reliably executes assigned engineering tasks, the governing calculation is no longer the token price, but what an organization is willing to pay to furnish every engineer with a squad of four or five tireless junior teammates.

Tibo builds on this economic reframing by pointing out the hidden cost displacement that AI enables. Broad operational sweeps—such as conducting exhaustive market research or having a squad of 15 engineers manually comb through an entire product backlog to identify trivial implementations—once required substantial headcount allocation, but can now be completed for a negligible token expenditure. While recognizing that early-stage startups or budget-conscious enterprises cannot always provide unrestricted inference across the board, Tibo cautions against premature austerity, urging leaders to grant their top performers substantial inference allowances so they can uncover the frontier of leverage.

Reframing the debate around model pricing, Vijaye Raji states: “And then the question then becomes like you know how much will you pay this teammate not necessarily like how many tokens are you going to use.”

- Model usage should be calculated against the output value of a 24/7 digital teammate rather than raw compute overhead. - AI drastically reduces the cost of large-scale engineering triage, replacing multi-engineer backlog audits with near-instant automated sweeps. - Prematurely throttling inference budgets for high-leverage engineers poses a strategic risk to organizational velocity.

---

[25:45](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1545s) Unprecedented Speed and Predicting the Next Six Months

Reflecting on his quarter-century career in the technology sector, Vijaye contrasts the current generative AI wave with historical disruptions, including the dot-com crash during his college years, the Y2K scare, the mobile app explosion, and the social networking boom. While each of those milestones brought intense industry realignment, Vijaye stresses that the present shift stands apart due to its sheer scale and velocity, remarking that modern adoption and capability curves represent anomalies where historical benchmarks no longer make sense.

When invited to forecast what software engineering and engineering management will look like two years down the road, Tibo rejects the two-year timeframe entirely, arguing that the industry is moving far too quickly to see that far ahead. Looking out merely six months, however, he expresses strong conviction in two near-term breakthroughs: inference speeds will jump by another order of magnitude, and robust multi-agent orchestration networks will reliably collaborate on massive objectives. Tibo anticipates that in the near future, an engineering team will be able to issue a high-level prompt—such as rebuilding an entire web browser from scratch—and receive millions of lines of functional, complex code back within 24 hours, fundamentally transforming human comprehension and maintenance of software codebases.

Offering his perspective on the historical uniqueness of this transformation, Vijaye Raji remarks: “I don't think I've ever seen anything like this.”

- Vijaye rates the current AI wave as substantially faster and broader in scale than the dot-com era, Y2K, mobile, or social networks. - A two-year planning horizon is considered unworkable given the current acceleration of AI capabilities. - Within six months, model execution speeds are projected to increase by another order of magnitude. - Emerging multi-agent networks are expected to tackle massive architectural goals autonomously, such as synthesizing multi-million-line software projects within a single day.

[28:06](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1686s) Guardrails, Rising Abstractions, and Debugging by Symptoms

As the craft of software development evolves, engineers will spend less time inspecting code directly and far more time defining the boundaries within which systems operate. Tibo Sottiaux explains that the industry will increasingly set guardrails around what gets built, enabling developers to validate correctness and enforce security properties simply by examining inputs and outputs. Code itself will be abstracted away, allowing developers to direct their attention toward high-level system properties and core engineering challenges rather than implementation mechanics. As Tibo describes it: “I think what we'll start seeing is we will set guard rails around you know what is getting built so that you don't actually have to look at the code anymore more and you can sort of either prove that it's correct in some way or the it is constrained in a way where you know it is secure and you can just look at the inputs and outputs and then code will become like abstracted away and it will all become about you know what are the actual challenges and things and you know the the properties of the system.”

Vijaye Raji points out that software engineering has always progressed by layering abstractions, allowing teams to deliver massive amounts of working product while writing relatively little low-level code. What makes the current moment unique is that the rate of change and abstraction has accelerated dramatically. However, this runaway complexity introduces a real engineering risk: when software accumulates too many sophisticated layers, diagnosing bugs by stepping through execution paths becomes impractical. Developers and their tooling will instead need to master the art of diagnosing issues through emergent symptoms. As Vijaye observes: “I think in a few years we'll get to the point where software um is is so complex software has gotten like so many layers in it and we get really good at identifying issues by looking at symptoms and our tools are going to get like really good at that too.” Interpreting these holistic behavioral clues will emerge as an essential, specialized capability for developers in the AI era.

[29:41](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1781s) A Dedicated Personal Assistant to Coordinate Swarms of AI Agents

Before the session concludes, Tibo jumps in with one final prediction regarding how engineers will interface with their automated tools on a day-to-day basis. Instead of forcing developers to individually supervise, monitor, and query 100 or 200 autonomous agents operating across various tasks, workflows will consolidate into a single primary interface. Developers will simply interact with one dedicated assistant that coordinates the entire fleet behind the scenes and reports back on high-level progress. As Tibo puts it: “I think very much you will just be able to call your assistant and check on the work as well and you know you will have like one dedicated sort of like personal assistant that is able to represent the work of like you know all the AI agents that are sort of like doing things for you productively behind the scenes instead of having like to monitor and like you know check in with like a hundred or like you know 200 individual like little agents.” He expects this unified model of agent management to arrive remarkably fast, anticipating its emergence within the year.

The host closes out the conversation by thanking Vijaye and Tibo for offering an authentic look into how OpenAI’s teams operate internally. He notes that the engineering realities and forward-looking practices developing inside OpenAI feel weeks, months, or even further ahead of where the broader industry currently sits, painting a vivid picture of the exciting, rapid transformation reshaping software engineering.

## Notable quotes > [18:25](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=1105s) As long as we're building products for humans to use, we will need human designers, we will need human product managers. — Vijaye Raji

> [12:07](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=727s) I actually truly believe that the new software engineers that are being created are going to be AI native. — Vijaye Raji

> [04:52](https://www.youtube.com/watch?v=Bo6Gtq3nMXc&t=292s) I actually think it's more fun now building software because the gratification cycle is so much shorter. — Vijaye Raji

## Mentioned OpenAI, Codex, Codex Box, Tibo Sottiaux, Vijaye Raji, Gergely Orosz, The Pragmatic Engineer, Alexander Embiricos, Linear, Notion, Microsoft, Visual Studio, IntelliSense, 1Password, Figma, Excel, PowerPoint, Slack, X, Reddit, Jira, ffmpeg

## Video description Tibo Sottiaux (Head of Engineering, Codex, OpenAI) and Vijaye Raji (CTO of Applications, OpenAI) at The Pragmatic Summit: www.pragmaticsummit.com. Watch sessions, with Q&A: https://newsletter.pragmaticengineer.com/p/the-pragmatic-summit-recordings

0:00 Intro: What’s changing inside OpenAI 0:54 Codex as teammate 3:13 How the Codex team works day-to-day 6:43 New engineering practices 12:07 New grads 16:37 The "new" software engineering role 18:25 PMs and designers 22:52 Token costs 26:50 Predicting the future

[Source](https://www.youtube.com/watch?v=Bo6Gtq3nMXc)

Reading summary

OpenAI engineering leaders report that AI tools like Codex have evolved from simple autocomplete assistants into autonomous agents and teammates, transforming development practices while shifting the core bottlenecks of software delivery. Autonomous Agents and Shifting Pipeline Bottlenecks (0:00) At OpenAI, internal developers routinely consume hundreds of billions of tokens weekly, prompting the company to create 'Codex Box' to run agentic coding tasks on dedicated remote servers rather than local machines. As individual coding output accelerates, engineering constraints are migrating further down the delivery lifecycle. Vijaye Raji points out that as engineers produce code up to five times faster, code reviews, security reviews, and CI/CD pipelines quickly become the primary organizational choke points. Similarly, the Codex team has moved from evaluating architectural trade-offs through lengthy design documents to prompting agents to implement multiple working prototypes in parallel. Expanding Automation into Operations, Research, and Meetings (9:37) Rather than being restricted to brief autocomplete suggestions, Codex now runs autonomous overnight testing loops and handles operational workflows. Tibo Sottiaux notes that in weekly analytics meetings, data analysts fire background Codex threads at the start of a session, allowing the agent to parse raw telemetry and return diagnostic findings before the meeting ends. The agent is also leveraged during production incident responses to determine root causes. In research workflows, Codex conducts unsupervised training runs, assesses model performance, and outputs structured PDF summary reports to guide human researchers. Early-Career Onboarding and Shifting Abstraction Layers (12:07) OpenAI is actively hiring early-career software engineers and expanding its internship programs, anticipating a cohort of around 100 new graduates who operate as 'AI-native' practitioners. On the Codex team, which Sottiaux manages through a flat structure of 33 direct reports, new hires use Codex itself as a primary onboarding mentor to navigate codebases and architecture. Addressing concerns that reliance on AI erodes core programming competencies, Raji compares the shift to historical debates surrounding the introductions of high-level languages, managed runtimes, and Visual Studio IntelliSense. Both leaders argue that higher abstraction levels simply shift focus toward system architecture, boundary setting, and product taste. Expanding Leverage for Non-Engineers and Economic Mental Models (18:25) Role boundaries are increasingly fluid: designers and product managers are generating production code, writing interactive prototypes, and using Codex to automate analytical spreadsheets and presentations. On the Codex team, a single product manager runs bug bashes where agents harvest participant feedback into Notion and automatically file and assign Linear issues to engineers. Regarding operational expenses, Raji argues that organizations should evaluate token costs not as infrastructure overhead, but as compensation for 24/7 synthetic junior teammates who conduct broad operational and triage tasks at scale. Projected Paradigms: Symptom-Based Debugging and Fleet Orchestration (25:45) Looking ahead, Sottiaux rejects two-year planning horizons due to the pace of model development, forecasting instead that model execution speeds will jump by an order of magnitude within six months and enable multi-agent systems to generate complex, multi-million-line codebases within a day. As systems gain complexity, developers will increasingly rely on automated guardrails and diagnose issues by analyzing holistic symptoms rather than tracing code line by line. Sottiaux also anticipates that within a year, developers will interact with a single personal assistant that orchestrates and coordinates hundreds of specialized background agents. While these workflows illustrate significant productivity gains, adopting teams face the challenges of downstream CI/CD bottlenecks, managing token expenses, and navigating software architectures whose layered complexity increasingly resists line-by-line human inspection.

Source quotes

[18:25] Vijaye Raji: As long as we're building products for humans to use, we will need human designers, we will need human product managers. [12:07] Vijaye Raji: I actually truly believe that the new software engineers that are being created are going to be AI native. [4:52] Vijaye Raji: I actually think it's more fun now building software because the gratification cycle is so much shorter.

Original transcript

[0:05] So [music] what a time to be alive, EJ. [0:09] >> Yes. [0:11] >> Can you tell us a question that a lot of [0:14] us are asking? What is happening inside [0:17] OpenAI right now? [laughter] [0:21] More specifically, when it comes to [0:23] building software with how engineers are [0:25] are doing stuff and how the whole thing [0:27] is changing. [0:28] I'm glad you clarified that [0:33] lots is hap lots are happening. Um I've [0:35] been there for about six months and one [0:38] of the things that I've learned is uh [0:41] there is so much to learn from uh the [0:44] kind of research that's happening in the [0:46] company. Um and just to project out what [0:49] are the possibilities is just [0:51] mind-blowing. So I'll tell you this [0:52] right um the way we write software um [0:57] has fundamentally changed um it's [0:59] changed so dramatically uh and even in [1:02] the last 6 months I've seen us go from [1:04] um codeex as a tool to an extension uh [1:08] to an agent now to a teammate I fully [1:12] expect engineers to name their agents [1:15] now um and call themselves as their [1:17] teammates and this is happening so past. [1:21] Um I was looking through some of the [1:23] leaderboards of like the people that are [1:25] using um codecs internally and some of [1:28] the engineers routinely hit hundreds of [1:30] billions of um tokens every week. And [1:33] this is not just one agent. We're [1:35] talking about um uh last week we uh [1:38] released uh Codex Box internally which [1:40] is a way for us to like actually reserve [1:43] dev boxes on the server and fire off [1:45] prompts and it's doing the the work. [1:48] doing the job while you're um on your [1:51] laptop orchestrating all of this stuff [1:52] and then people like shut down their [1:54] laptop, go to a meeting, come back and [1:55] then like all of the the work has been [1:57] done. So this is this is happening in [1:59] parallel. This is how fundamentally [2:02] software has changed um internally at [2:04] OpenAI and I can't wait for all of this [2:07] in the kind of like the center of [2:09] Silicon Valley and then to expand [2:11] further and further in a few months. I [2:13] think this will be the norm. Everybody [2:14] is going to be developing software this [2:16] way. Uh, and that's pretty cool. [2:19] >> So, like if I would just take myself [2:22] back, you know, 6 months, even a year, [2:23] and I would hear you you you say this, I [2:26] would think like, oh, it's it's like a [2:27] magical fairy tale. You're making half [2:29] of this up. However, actually like a lot [2:32] of us are using it. I'm using it. I'm [2:33] seeing what's happening. And I've been [2:35] talking with engineers inside of OpenAI. [2:37] I love talking with engineers because [2:38] they have hashtag no filter like like [2:41] this is the secret to part of the prag [2:43] why the pragmatic engine works. I talk [2:44] with engineers who they don't have this [2:46] thing called media training or all that [2:47] and and they just [laughter] [2:50] they they just tell me how it is and [2:52] inside of OpenAI one thing that was [2:54] pretty comforting to me I'll be honest [2:56] is not all engineers are writing 100% of [2:59] their code with with codeex they're all [3:01] using it a lot more but it's there's [3:02] there's lot levels there one team that [3:05] is absolutely on the cutting edge though [3:07] and again I've talked with a bunch of [3:08] engineers is a codeex team and uh [3:10] they're even ahead of others inside open [3:12] AAI so Tibo like you leading the the [3:15] Codex team. Can you tell me how the [3:17] Codex team works today and what the [3:19] typical workflow of an engineer is like [3:21] right now as of like yesterday or this [3:23] morning? [3:24] >> Right. It's a it's a fast evolving [3:26] situation. [3:27] Uh the thing that's delightful about how [3:31] the Codex team operates is that they're [3:32] sort of like constantly reinventing how [3:34] they're working like almost on a week- [3:36] toeek basis. And the thing that we go [3:38] after is like you know we sort of like [3:40] identify every single little bottleneck [3:42] and the bottlenecks keeps shifting. So [3:44] you know it used to be code generation [3:46] and then you know then it moved to like [3:48] code review and then now it's very much [3:50] like hey how do we understand the user [3:52] needs faster? How do we try tickets? [3:54] like how do we like figure out you know [3:57] what everyone is saying on Twitter, [3:58] Reddit, you know, all the important [4:00] surfaces and sort of like synthesize [4:02] that into a strategy and like everyone [4:04] is using you know and trying to like [4:06] leverage agents for that to the very [4:08] best effect and an interesting thing is [4:10] like the other day it was like the first [4:12] time in a negotiation like you know [4:13] someone was trying to join the critics [4:15] team and like this person asked me like [4:16] how much compute am I going to get to [4:18] build products at OpenAI. I was like huh [4:21] that's an interesting question. I mean [4:22] we do have a lot of compute but I [4:23] haven't really thought about you know so [4:25] it's like a compute envelope per [4:26] employee. Um usually that's more like [4:29] reserved to like researchers who are [4:31] actually training like really phenomenal [4:33] models. So I think there's like this [4:35] this shift there where you know people [4:36] realize you can hyper leverage yourself [4:39] in you know all sorts of like novel ways [4:41] and if you do have great taste great [4:43] ideas you know you know how to build [4:45] software it's like you know what a time [4:47] to be alive really like it's just like [4:50] incredible what you can do [4:52] >> and taking a little bit step back [4:53] outside of the codeex team VJ you're you [4:55] you have a lot of visibility inside of [4:57] open AI how is the work of a software [5:00] engineer or should I say a product [5:02] engineer changing open AI has always [5:04] hired software engineers who are product [5:06] engineers very clearly. How is their [5:08] work changing? How how is are things [5:10] morphing with with with product or are [5:12] they not morphing? [5:14] >> Um fundamentally we're still building [5:16] products for humans to use. And so [5:18] there's a lot of like product intuition [5:20] that comes into play even when um so [5:23] I've been messing around with codeex [5:24] thanks to the new onep app that uh makes [5:27] it even more accessible for everyone to [5:29] like um start coding. Um even in the lot [5:32] of cases where we have to imagine what [5:35] the product that we have to build and [5:37] ship and that's where it starts and then [5:39] you have to constantly tweak it to get [5:41] it to the right place. I don't think [5:42] that's going to change. I as long as we [5:44] continue to build software for humans. I [5:47] mean at some point in the future we may [5:48] build software for agents but then maybe [5:50] the agents will become the product [5:52] engineers or product managers at that [5:54] point. Um, but I I think the the uh the [5:57] velocity makes it a lot more appealing [6:00] and compelling and actually more fun to [6:02] be honest. Um, I was coding in on a [6:04] plane. Um, and at that time, you know, I [6:07] didn't have access to the the dev boxes, [6:09] but so you kind of like keep the laptop [6:11] open when the flight attendant comes [6:12] over like you have to shut down your [6:13] laptop. No, no, but I don't want to [6:14] like, you know, have the agent stop, so [6:16] I like keep it slightly open and then [6:18] put it down. [6:19] >> Everyone just runs around with their [6:20] laptop like, you know, half closed right [6:22] now. It's like [6:23] >> yeah what are we doing? I I I think I [6:26] think that's u you know I actually think [6:28] you know it's more fun now uh building [6:30] software is because the the the cycle [6:33] the gratification cycle is so much [6:35] shorter and it's so really cool to see [6:37] the product that you're building test it [6:39] verify it and then go back to codeex [6:42] >> and as as engineers we're engineers what [6:45] are new different or weird engineering [6:48] practices that you're now starting to [6:50] see that kind of you know it starts to [6:52] make sense as weird it [6:54] Um it used to be that you know you had [6:57] like you [6:59] difficult like technical trade-offs and [7:00] you sort of like you know do like a [7:02] design dock and discuss it all and then [7:04] you know maybe you're like oh what are [7:06] the other viable alternatives and then [7:08] you know you sort of like discard that. [7:09] I think a delightful thing is that now I [7:12] see people explore like know multiple [7:14] different implementations like all in [7:16] parallel and then we can like actually [7:18] zoom in on the one that you know we sort [7:20] of like prove to work better. Um the [7:24] other thing is I also see like rules [7:26] like blur. Um so like our designers are [7:29] like shipping more code than like you [7:31] know engineers were shipping like six [7:33] months ago. And that's just also because [7:35] like the models have become like [7:37] sufficiently good that the code that [7:38] they're producing you know is actually [7:40] code that we would want to merge just as [7:42] is. [7:44] >> Do you do you have do you see anything [7:46] else like in the broader OpenAI? Um I [7:48] have like noticed I don't know do you [7:51] all remember like the command line for [7:53] every one of the command line tools you [7:55] use? I I don't want to pick an it's like [7:57] I was talking to Tibo [8:00] >> his team edits um video files and like [8:04] you know f if you know ffmpeg it's like [8:06] I don't think anyone remembers the [8:07] command lines coex is like such a great [8:10] tool for you like okay well I want to do [8:12] this and then craft the command line go [8:14] execute it. Um so those are kind of like [8:17] new ways that we're seeing people use um [8:19] codecs specifically. I also think that [8:22] we've now moved on from just coding um [8:25] to code reviews, security reviews and [8:28] then um as Dibo said we're going to find [8:30] more bottlenecks. So once you solve [8:32] coding for example now you've just made [8:33] every engineer five times more um five [8:36] times more um productive. What's going [8:40] to happen is like there's going to be [8:41] more uh code being written which means [8:43] that code reviews will become the [8:45] bottleneck and then after code reviews [8:47] uh integrations and deployment CI/CD [8:49] will become the bottleneck. So we're [8:51] going to have to constantly go solve the [8:53] next set of problems uh which is really [8:55] exciting actually. Then TB, one really [8:58] interesting thing when we talked about [9:00] what you're doing at Codeex that I've [9:02] never heard before is these overnight [9:05] runs and the self- testing. C can you [9:07] tell us about that because that is like [9:09] net new. Yeah, I I think it's easy to [9:12] sort of get stuck in like, oh, this is, [9:14] you know, autocomplete on steroids and, [9:16] you know, it's just going to implement a [9:18] little feature and sure it will get done [9:20] in like 10 minutes. But what we're sort [9:23] of seeing is that the model is like much [9:25] much more capable actually if you give [9:28] it like a very large task. It's capable [9:30] of like running for multiple hours. So, [9:33] we assemble like we've assembled like [9:35] the environment and the skills so that [9:37] Codex can like fully autonomously test [9:39] itself. uh we run this overnight so that [9:41] you know it just basically performs like [9:43] QA in a loop uh and like flags like [9:45] regressions. The other thing was I keep [9:48] talking to this researcher on the team [9:50] who's actually training the models and [9:52] he's like every time I think I'm more [9:55] capable than codeex is just I figure out [9:58] I'm wrong and I just like didn't prompt [9:59] it right uh or I hadn't set it up in the [10:02] right way. And you know this is both [10:03] exciting and a little bit depressing at [10:06] the same time. Um because he's like oh [10:09] now it's just you know training a model [10:11] fully independently uh and like writing [10:13] a little PDF report at the end you know [10:16] with like its own insights and findings [10:17] and then we just take that and then find [10:20] like you know the most promising things [10:22] to like iterate on and then just like [10:23] reput that into codeex. Um, and so like [10:26] these like very very long running tasks [10:29] and like achievements that you know just [10:31] like it's incredible to see like a model [10:33] do this like independently. [10:35] >> Yeah. And one more thing that we talked [10:37] about that felt to me a bit like from a [10:38] sci-fi is you said that sometimes you [10:40] have meetings you the codec team has [10:42] meetings about codecs and like issues [10:44] that you have and you told me something [10:45] interesting that you know like people [10:47] you know get together in a meeting room [10:49] and then you like fire off codeex [10:52] threads to diagnose stuff with codecs [10:55] can you tell us a little bit of of how [10:56] that's playing because that is like [10:58] really like a loop of itself. [11:00] >> Yeah, there are two big things that we [11:02] do there. So we have this like weekly [11:04] analytics review where we go over you [11:06] know like feature adoption um you know [11:10] retention like you know we analyze our [11:11] funnel and we always start a meeting [11:13] with like questions we have that you [11:15] know just not answered in our dashboards [11:17] or you know we haven't looked into like [11:18] you know we're just like oh this looks [11:20] interesting and then our data analyst is [11:22] just like okay let's just meet let's [11:24] fire off like a little codex thread in [11:26] the background like you know it will [11:27] like come back in 20 minutes we'll have [11:29] the answer like by you know just like [11:30] and we can talk about it like in the [11:31] last 10 minutes of the And then we do [11:33] that you know for five six questions [11:34] that people have in the room and it's [11:36] sort of like this magical experience [11:37] where you know just have like this [11:38] little consultants like you know working [11:40] for us uh in the background and then the [11:42] other thing is like for um whatever um [11:45] you just like we get paid as call is [11:47] like you know just codex is there like [11:48] you know helping figure out like what [11:50] went wrong what is the fastest path to [11:52] recovery um and there it just sort of [11:54] feels like you know so much accelerated [11:56] and like you know how much uh how much [11:58] information we can gather and like how [11:59] quickly can solve for things. So this is [12:02] one and it's absolutely accelerating [12:04] right and we see this we see it [12:05] elsewhere as well. One big question that [12:08] is keeps coming back across the industry [12:10] is what about new grads? What about [12:12] junior engineers? And what I was talking [12:14] with head of engineering at OpenAI uh he [12:17] was saying something interesting that [12:18] you are hiring early career engineers. [12:21] Can you talk a little bit about this is [12:23] great to hear. Can you talk about how [12:25] it's going what you're seeing with them? [12:28] How much are the fears of you know [12:29] juniors are are not great because now [12:32] seniors can just use like an AI agents. [12:34] how how are this founded and you know [12:35] how are they getting up to speed? [12:37] >> Um we are hiring a lot of uh um new grad [12:41] folks uh straight from college. We're [12:43] also having u so this year we have a [12:46] pretty robust internship program um I [12:49] actually truly believe that the new uh [12:52] software engineers that are being [12:53] created are going to be AI native. [12:55] They're going to know these tools in a [12:57] native way. um and they're going to be [12:59] able to leverage um our AI tools uh from [13:03] day one. And I think giving them the [13:05] opportunity is going to be critical and [13:07] important and growing them in this kind [13:09] of like the environment is going to be [13:10] amazing. I can't wait to see this. And [13:12] so this summer um is our kind of like a [13:15] first batch of uh new grads that are [13:17] going to be coming into OpenAI and I'm [13:19] really excited for that. Uh it's going [13:20] to be about 100 people or so. And then [13:23] uh I want to like continue growing our [13:25] internship program uh within OpenAI. So [13:28] yeah, so this is going to be a really [13:30] really cool thing to witness um in this [13:32] age. [13:33] >> And then Tibo, how are you onboarding [13:36] people to the code experience [13:37] specifically? Even within OpenAI, my [13:39] sense is that the Codex team is maybe a [13:41] little bit you know like a few months or [13:42] or weeks or ahead of of how you're [13:44] working. when someone new either from [13:46] the outside or even from OpenAI comes [13:48] like [13:50] how do they get up to speed on how the [13:51] team works? [13:54] >> So we I run the team in a very it's like [13:57] a very flat uh organization like I I [14:00] have 33 direct reports uh on the team [14:04] and they just you know run around and [14:06] like do cool things and uh it's you know [14:08] I don't want to be the bottleneck. I [14:09] think this is like one of the things [14:10] where as leads I think it's very um it's [14:14] it's very tempting to not change [14:16] organizational structure fast enough for [14:17] like you know how quickly people can [14:19] actually build and like a single person [14:21] being the bottleneck on every single [14:22] decision is just like obviously not [14:23] going to work anymore but the first [14:26] thing that people um you know get [14:29] introduced to obviously is like Codex [14:31] itself right so like Codex is [14:32] responsible for the onboarding um you [14:34] know you just like ask codeex questions [14:36] you navigate the codebase like [14:37] understand like what other people are [14:38] doing you receive like you know daily [14:40] reports but then the people who are [14:43] responsible for the onboarding and like [14:45] you know the culture and how we built [14:46] are like also the people that just most [14:48] recently onboarded onto the team. Um and [14:51] I I find that actually like you know [14:53] just talking about the new grats is like [14:54] you know I have this like phenomenal new [14:56] grat joined the team like you know 6 [14:57] months ago and he's absolutely crushing [14:59] it. Um and that was like a little bit of [15:00] a surprise but like I understood you [15:02] know this person has like sort of like [15:03] unbound unbounded energy like much more [15:06] than I do. Um and you know it's just [15:08] like you know super super quick. Uh I [15:10] think you know my my brain is probably [15:12] already in decline. Um you know this [15:14] this person like Ahmed's brain is just [15:16] like absolute peak peak. Um and you know [15:19] just phenomenal person and he's been [15:22] like so successful on the team and [15:23] that's been like really delightful to [15:24] see. Now playing a bit of devil's [15:27] advocate [clears throat] a lot of us [15:29] more experienced folks who have seen [15:31] like you know like new grads grow into [15:33] like really successful professionals we [15:35] have seen that at least up to now [15:38] foundations were so important and so [15:41] what do you think will happen if we have [15:44] news whose foundations are are using AI [15:47] coding and they probably skipped the [15:48] stuff that we did for 10 20 more more [15:50] years. Are they building the right [15:52] foundations or or are are we asking the [15:55] right question here? Even [15:56] >> foundations remain super important, [15:58] right? So we we take great care in like [16:00] designing the overall codebase, you [16:02] know, just like taking care like overall [16:04] architecture. You know, we do code [16:06] review as you said like you know we [16:08] don't fully rely on like you know codeex [16:09] like writing everything and just like [16:11] closing our eyes and like being like [16:12] this is going to be fine. um you know we [16:14] have like the very best engineers like [16:17] working on this as well but I find like [16:19] new grads are able to sort of like [16:20] absorb that and then you know it's like [16:22] if you have like the right structure for [16:23] your codebase and you know you set like [16:25] the right guard rails then you know [16:27] they're like incredibly productive and [16:29] so I think it's just about the [16:30] environment that you're setting up um [16:31] and like you know thinking ahead of time [16:33] of like you know like how is this like [16:34] codebase going to evolve [16:36] >> and how is the role of of starting with [16:39] like software engineers uh changing [16:41] compared to even like six or eight [16:43] months ago. go. What does a software [16:45] engineer do? Like if if you had to [16:46] explain to a new journey what they're [16:48] going to ask like, "Hey, VJ, what am I [16:50] going to do dayto-day?" What are they [16:51] going to do? [16:52] >> Yeah, I think um so the idea of [16:55] foundations, foundations will never go [16:57] out of fashion. So that is going to be [16:59] always important no matter what. Um I [17:01] think we're all here because we have [17:03] strong foundations um that's brought us [17:05] here. Um and then in terms of like you [17:08] know the role of a software engineer, [17:09] it's changed quite a bit. I don't know [17:11] if you I may be dating myself 25 years [17:14] um uh in the industry I've seen so many [17:17] paradigm shifts and uh I actually worked [17:20] on uh developer tools in Microsoft uh [17:23] wrote the editor for visual studio and [17:25] language services. So when first time I [17:28] saw IntelliSense that was kind of like a [17:29] really cool moment where you could kind [17:32] of like type hit the dot and then the [17:35] options showed up. [17:36] >> Yeah. But do you remember I I was [17:38] joining the industry around that time [17:39] and the devs around me were saying like [17:40] you're not a developer if you use [17:41] intellisense. [17:42] >> Yes. [laughter] And I mean I' I've seen [17:45] those I mean like this is probably be [17:47] before my time when people probably saw [17:49] like okay if you're not writing assembly [17:51] um you're not a good um software [17:53] engineer and then C++ and then um you [17:56] know the abstractions kept going up and [17:58] up and then people used to complain [18:00] about JavaScript. Remember those days? [18:02] Um [18:04] I don't think those things actually [18:05] matter. The point is that as long as you [18:07] have the strong foundations, as long as [18:09] you have product intuition, know what [18:11] you're building and be able to like go [18:13] down up and down the stack um to be able [18:16] to like solve problems, those are going [18:18] to be the more important ones. And I [18:20] don't think that'll ever go out of [18:21] fashion. I I feel like that is always [18:23] going to be the case. [18:25] >> We're here between mostly engineers, [18:26] engineering leaders, but let's just [18:29] spare a thought on on product managers [18:31] and designers. How do you see their [18:33] roles changing especially now that both [18:35] engineers and and them can build [18:37] features a lot faster? [18:39] How does it that change their roles or [18:41] are are we getting closer or do they [18:43] still have a distinct role from what you [18:45] see? [18:47] >> Um I go back to the as long as we're [18:51] building products for humans to use, we [18:53] will need human designers, we will need [18:55] human product managers. I think this is [18:57] a you know I don't know um there is a [19:01] substitution for a product sense um or [19:03] design sense those things will evolve [19:05] will get even more productive even more [19:09] um abstractions but um we will continue [19:12] to evolve that the they're they're [19:14] getting more and more productive if [19:16] anything so product managers are writing [19:18] code designers are writing code they're [19:19] taking their pro design um into [19:22] production into um prototypes and [19:25] validating it before they come to [19:27] engineers. So I think those are already [19:29] getting a lot more productive. Um you [19:32] know this may be uh the product managers [19:35] are also using codecs for building [19:37] PowerPoint slides and we have Excel [19:40] plugins and so it's kind of like all [19:42] around it's not just engineers um [19:45] everyone around is getting more [19:46] productive. [19:48] >> One cool thing that you're doing inside [19:49] OpenAI which which I've heard is this [19:51] internal knowledge sharing this show and [19:53] tell where where teams show what they [19:55] do. Can you tell us [19:57] how you came up with it? How you're [19:59] actually doing the mechanics? And can [20:01] you tell us like some cool things that [20:02] you've seen teams show and and like [20:04] maybe other teams adopt? [20:06] >> Yeah, it's um it's interesting because [20:08] we're sort of like discovering the [20:11] technology and evolving it as and we're [20:13] co-evolving with that as well. So like [20:15] you know we're like just as all of you [20:17] are discovering like hey this is what [20:19] you know AI can do for me and this is [20:20] like what it means for the organization [20:22] or this is what it means for my project [20:23] like we're also discovering it you know [20:25] like pretty much at the same time like [20:27] as soon as you know like when we have [20:28] something that so like feels like it's [20:30] starting to work is like you know we [20:31] ship it to the world right so it's like [20:33] that we have like a very small um small [20:36] amount of time where you know we we [20:38] actually are able to like you know have [20:40] like more of the crystal ball than than [20:42] all of you. Um, and it's super important [20:45] that like good ideas diffuse very fast [20:47] through the organization. So like you [20:49] know we we use Slack and like the Codex [20:51] Slack channels and like hot tips are [20:53] like you know two channels that are like [20:55] um super super active and then you know [20:57] we organize like regular hackathons like [20:59] show and tell um we just like try and [21:02] diffuse like you know novel ways of like [21:04] working with AI as fast as possible and [21:06] like it's a highly creative time. So I [21:08] think there's no like one true way to [21:10] use this stuff. It's like you know very [21:12] much still like in discovery and then [21:14] our we have like this phenomenal product [21:17] uh manager on codeex um Alexander [21:20] Emberos and he's just like the single [21:22] pro product manager like for the entire [21:24] codeex team and he hyper leverages [21:26] himself like you know with the help of [21:28] codeex like I like the other day he [21:29] organized this bug bash it was like an [21:31] hour like people were going through like [21:33] you know features that we were about to [21:34] ship and then he sent codeex to collect [21:37] like feedback from everyone this ended [21:39] up in a notion doc and then he [21:40] dispatched Codex to like then file [21:43] feature uh like bug reports and like you [21:45] know feature improvements like tickets [21:48] into linear and then assign it to [21:49] everyone and then follow up with [21:50] everyone on like how it was going and so [21:52] like he's like becoming like a 10x like [21:55] you know 50x like program manager just [21:57] you know by leveraging AI as well and I [22:00] think it's important to so like again [22:01] going to the bottlenecks is like you [22:02] know you need to continue going back [22:04] like you know your product manager [22:05] cannot become the bottleneck so it's [22:06] like you know you need to look at it in [22:08] a principled way [22:10] >> one thing I'll add is like I I've been [22:12] to um these demo days and we've seen a [22:16] whole bunch of these projects being [22:17] demoed. I remember um going to these [22:20] hackathons and looking at like the [22:22] demos. Um one thing I'm noticing is the [22:25] depth of these demos have been [22:26] consistently going up. So it's not just [22:28] like a surface level here here's what is [22:30] possible. Um some of these demos are [22:33] actually like here's what's possible but [22:34] also I've taken care of all of these [22:36] corner cases and actually like a very [22:38] usable product. So the depth um day by [22:41] day of all of these products that people [22:43] are building uh even to just show off uh [22:46] some of the capabilities is definitely [22:48] going down um going up and getting [22:50] deeper. [22:52] >> One kind of disclaimer that we need to [22:55] add is inside OpenAI everyone has access [22:57] to unlimited tokens. there's no cost and [23:01] people are laughing because it's kind of [23:02] a big deal, right? It's a in the outside [23:04] world if if if you may cost is is still [23:07] a problem. You get the max subscription [23:09] and when it runs out you're now on [23:11] credits and you know some some some [23:12] people are cool with it especially [23:13] founders but sometimes people ask [23:15] questions with this in mind that a lot [23:18] of places are constrained with with cost [23:20] just just for practical purposes. What [23:22] suggestions and tactics would you have [23:24] for folks who who w are inspired by how [23:29] the team at OpenAI is is worked but they [23:31] have these constraints/h handcuffs to [23:33] work with. [23:35] Cost is something that we constantly [23:37] think about. Uh one is obviously we want [23:39] to make our models more and more capable [23:41] and offer that um to um our users. Um [23:45] and then the I I also believe that at [23:49] some point the thinking will shift [23:51] because now you should imagine like now [23:54] you have a teammate that is working for [23:57] you 24/7 and you can send instructions [24:00] to your teammate like you know you can [24:02] assign linear tasks or Jira tasks to [24:04] your teammate and then expect um and you [24:07] should fully expect uh your teammate to [24:09] be capable of taking care of those [24:12] things. And then the question then [24:13] becomes like you know how much will you [24:15] pay this teammate not necessarily like [24:16] how many tokens are you going to use. Um [24:18] and so if you start to measure in the [24:20] terms of like productivity of every [24:22] engineer having a team of four or five [24:25] of these teammates then it starts to [24:27] make a lot more sense. Now you should [24:29] like hold us responsible to make these [24:31] agents a lot more capable enough to [24:33] treat them as teammates. And that's kind [24:35] of like you know what we're working on. [24:39] >> Yeah. I I think it also you know is is [24:41] useful to think about you know how it [24:44] displaces costs across uh you know the [24:47] company and there are things that you [24:50] know you can do now that are actually [24:52] like you know it's very cheap for you to [24:53] do so like you know doing like marketing [24:55] research like going over like the [24:57] entirety of like your feature backlog [24:59] and like figuring out like which ones [25:01] are like the ones that you can trivially [25:02] implement. Um you know before that you [25:04] would have needed to allocate like you [25:06] know maybe like 15 engineers to go and [25:08] like look through that uh backlog and [25:09] now it's like you know like almost free. [25:12] Um obviously like not everyone can [25:17] you know provide the perk of like having [25:18] unlimited like inference um you know to [25:21] their employees but I do think limiting [25:25] it prematurely is you know as a risk uh [25:28] as well and we're we're very very early [25:32] stages at like you know how well [25:34] leveraged people can get and so I would [25:36] definitely like sort of be saying like [25:38] hey there's like the best people at your [25:40] company like you know give them like you [25:41] know very very comfortable like large [25:43] amounts of like inference. [25:45] >> Reflecting on the pace of change, we [25:47] know it's fast and and it's getting [25:48] really really fast. It feels like that, [25:50] but taking a step back from your times [25:52] before open AI and and VJ, you you've [25:55] been in in this business for a long [25:56] time, more than 25 years. Looking back, [26:00] what was a time where change also felt [26:03] fast? And did we see anything somewhat [26:06] comparable in the past? [26:08] >> I don't think I've ever seen anything [26:09] like this. Um I can look back in uh in [26:13] the 25 years I've seen the dotcom bubble [26:16] burst and that was during my college [26:18] time and then I remember Y2K I remember [26:22] the mobile uh revolution and I was [26:25] actually part of the social network [26:27] revolution and this one feels very [26:30] different. This one is happening um at a [26:32] massive scale and ma and also happening [26:35] very fast. the speed at which this is [26:37] happening um some of these charts don't [26:40] make sense um and so I do think this is [26:44] something very very special and unique [26:45] and it's also cool to be living in this [26:48] uh period [26:50] now as a closing question it changes [26:52] fast but the two of you have been in [26:54] open AI for for now quite some time so [26:56] I'm going to ask you to make an honest [26:58] prediction in two years time what do you [27:01] think [27:03] software engineering will look like and [27:04] what will engineering management look [27:06] like just knowing what you know [27:12] [laughter] [27:14] >> obviously two years is like way too long [27:16] of a time frame. Uh [laughter] I think [27:19] six six months from now like the things [27:21] that I'm sure you know it's like I feel [27:22] very confident saying is like you know [27:24] we will get maybe another order of [27:27] magnitude on like speed um and that will [27:32] you know change things again and the [27:33] other thing that we will get working is [27:35] like you know large networks of like [27:37] multi- aent uh that can collaborate [27:39] together on like you know very very big [27:40] goals you know for example it should be [27:42] you know within the realm of like [27:44] feasible to say like you know alongside [27:47] in the same team of like you know what [27:48] cursor demonstrated like you know hey [27:49] rebuild a browser from scratch like you [27:51] know just like go and then 24 hours [27:54] later like you know you have this like [27:56] this thing that was built you know like [27:58] two millions of lines of code it's like [27:59] you know pretty much like untractable uh [28:02] to like understand you know like what [28:04] actually is happening under the hood and [28:06] so there I think what we'll start seeing [28:09] is we will set guard rails around you [28:13] know what is getting built so that you [28:15] don't actually have to look at the code [28:16] anymore more and you can sort of either [28:18] prove that it's correct in some way or [28:21] the it is constrained in a way where you [28:24] know it is secure and you can just look [28:26] at the inputs and outputs and then code [28:28] will become like abstracted away and it [28:30] will all become about you know what are [28:32] the actual challenges and things and you [28:33] know the the properties of the system [28:38] software has been increasing in [28:39] abstraction makes it easier for us to go [28:42] build massive amounts of uh product um [28:46] code um with very little code. So it's [28:49] kind of like um over the years that [28:51] abstraction has increased and I feel [28:52] like we're in a time frame where that [28:54] abstraction is increasing the rate of [28:56] change has also increased uh quite [28:58] rapidly. um at some point I worry um I [29:02] I'll say this right there because um any [29:04] sufficiently complex or sophisticated [29:06] system um becomes harder to debug and so [29:09] you rely on symptoms to debug these [29:12] things and so I I I I get I think in a [29:15] few years we'll get to the point where [29:16] software um is is so complex software [29:20] has gotten like so many layers in it and [29:22] we get really good at identifying issues [29:25] by looking at symptoms and our tools are [29:28] going to get like really good at that [29:29] too. Um, and so I I think that will be a [29:31] unique um uh function or I think that [29:35] will be a unique uh ability for software [29:38] developers to pick up. [29:40] >> Well, VJ, [29:41] >> I want to add something to like what the [29:43] future will look like. Um I think very [29:45] much you will just be able to call your [29:47] assistant and check on the work as well [29:49] and you know you will have like one [29:51] dedicated sort of like personal [29:52] assistant that is able to represent the [29:55] work of like you know all the AI agents [29:57] that are sort of like doing things for [29:59] you productively behind the scenes [30:00] instead of having like to monitor and [30:02] like you know check in with like a [30:04] hundred or like you know 200 individual [30:06] like little agents. Uh I think that's [30:08] something that we'll see actually like [30:10] fairly quickly including this year. [30:12] >> Yeah. Well, thanks so much to VJ and [30:14] Tibo for giving us a peak of what is [30:16] actually happening inside and how your [30:18] teams are working, which it feels is is [30:20] either months or weeks or or sometimes [30:21] longer ahead of the curve, but it is [30:23] happening. And also just like what we [30:25] might or might not see in this like [30:27] really exciting time. Thank you so much. [30:28] >> Thank you. Thank you. [music] [30:30] [applause]