Every's Shipper Urges Software Makers to Split Core Teams from AI Labs

Lenny's Podcast

Isolating internal early adopters into two-person research units prevents frontier model shifts from derailing product roadmaps while incubating the tools that will replace them.

AI-assisted summary of How to run your product team like a research lab | Dan Shipper (Every) — original source. Check the original for full context.

Source published: 2026-09-24

Key ideas

Product organizations face severe paralysis when trying to force core teams to simultaneously execute stable roadmaps and track the fast-shifting AI frontier, creating a structural conflict between exploration and exploitation. (2:05)
Modern AI labs do not need large head counts: a nimble 'two-slice team' of one or two people pairing a prototyping 'pirate' with a systems-focused 'architect' can rapidly map frontier capabilities. (7:31)
Discarding 90% of lab prototypes can yield net-positive ROI when organizations publish transparent postmortems as readership content, run early-adopter VIP programs, and educate core engineering roadmaps. (8:51)
OpenAI's small Codex team demonstrated the upside of isolated self-disruption by testing coding form factors until their February 2026 desktop app grew into the foundation of ChatGPT for 800 million daily active users. (17:28)

Detailed notes

## Background Published on YouTube by Lenny's Podcast on September 24, 2026, this keynote features Dan Shipper, co-founder and CEO of Every, speaking live at the Lenny and Friends Summit. The presentation outlines how organizations can build a small research lab to rapidly test frontier AI capabilities in real workflows while the primary product team continues scaling what already works.

## TL;DR Dan Shipper addresses the organizational tension between divergent frontier exploration and convergent roadmap execution in the rapidly shifting AI era. Rather than forcing an entire team to do both, companies should spin up nimble 'two-slice' labs teams—ideally pairing an exploratory 'pirate' with a systems-minded 'architect'—to run disposable prototypes through extreme internal dogfooding. By establishing a structured research pipeline governed by internal adoption, a 10x value filter, and unit-economic viability, organizations can safely discard 90% of experiments while graduating clear winners into the core roadmap. Ultimately, this approach enables deliberate self-disruption without derailment, exemplified by how OpenAI's isolated Codex team eventually reshaped the main ChatGPT application.

## Atmosphere & style Shipper delivers a punchy, energetic conference keynote that balances candid founder humor with actionable operational frameworks. His tone is irreverent and self-aware, drawing genuine laughs with relatable observations about chaotic early adopters and the dizzying disorientation of constant frontier model releases. The presentation follows an emotional arc that moves from technological overwhelm to reassuring clarity, leaving builders with an empowering playbook to welcome future model drops with excitement rather than dread.

## Bulletin - [02:05](https://www.youtube.com/watch?v=DqF08Dz3nok&t=125s) Product organizations face severe paralysis when trying to force core teams to simultaneously execute stable roadmaps and track the fast-shifting AI frontier, creating a structural conflict between exploration and exploitation. - [07:31](https://www.youtube.com/watch?v=DqF08Dz3nok&t=451s) Modern AI labs do not need large head counts: a nimble 'two-slice team' of one or two people pairing a prototyping 'pirate' with a systems-focused 'architect' can rapidly map frontier capabilities. - [08:51](https://www.youtube.com/watch?v=DqF08Dz3nok&t=531s) Discarding 90% of lab prototypes can yield net-positive ROI when organizations publish transparent postmortems as readership content, run early-adopter VIP programs, and educate core engineering roadmaps. - [17:28](https://www.youtube.com/watch?v=DqF08Dz3nok&t=1048s) OpenAI's small Codex team demonstrated the upside of isolated self-disruption by testing coding form factors until their February 2026 desktop app grew into the foundation of ChatGPT for 800 million daily active users.

## Key takeaways - [00:06](https://www.youtube.com/watch?v=DqF08Dz3nok&t=6s) Technological leaps in frontier models cause organizational disorientation, necessitating a rule to avoid major roadmap pivots within 30 days of encountering new AI capabilities. - [02:05](https://www.youtube.com/watch?v=DqF08Dz3nok&t=125s) Exploration (divergent, highly experimental, throwing out ~90% of work) and execution (convergent, reliable roadmap delivery) represent incompatible cognitive postures that should not be forced on the same team. - [03:13](https://www.youtube.com/watch?v=DqF08Dz3nok&t=193s) Internal early adopters can derail roadmap execution with speculative ideas unless their tinkering is isolated and channeled into a dedicated labs initiative. - [03:13](https://www.youtube.com/watch?v=DqF08Dz3nok&t=193s) AI engineering leverage allows companies to start a functioning research lab with a team of one, mirroring how Anthropic Labs created Claude Code, MCP, and Skills from small experiments. - [07:31](https://www.youtube.com/watch?v=DqF08Dz3nok&t=451s) The optimal lab team is a 'two-slice team' of two people: a 'pirate' who generates high-volume, vibe-coded prototypes and an 'architect' who refines promising hacks into coherent software. - [07:31](https://www.youtube.com/watch?v=DqF08Dz3nok&t=451s) The core operational metric of an AI lab is shortening feedback loops, which is achieved most effectively through extreme internal dogfooding on real, substantive problems rather than synthetic sandboxes. - [08:51](https://www.youtube.com/watch?v=DqF08Dz3nok&t=531s) Running parallel, competing bets aimed at the exact same objective helps organizations chart an unpredictable AI frontier faster than enforcing false top-down coherence. - [11:00](https://www.youtube.com/watch?v=DqF08Dz3nok&t=660s) A structured research pipeline progresses initiatives from scrappy lab experiments to internal workflow adoption, trusted customer testing, and formal product roadmap integration. - [12:18](https://www.youtube.com/watch?v=DqF08Dz3nok&t=738s) Every's 'Kate bench' experiment automated editorial taste using three years of editing history, cutting the editor-in-chief's post-agent workload by 12% once an architect established production evaluation pipelines. - [15:17](https://www.youtube.com/watch?v=DqF08Dz3nok&t=917s) Graduating experiments from lab to product requires strict gating criteria: sustained organic internal retention, a 10x workflow improvement past initial novelty, and viable inference unit economics at scale. - [15:17](https://www.youtube.com/watch?v=DqF08Dz3nok&t=917s) Weekly cross-functional reviews of experimental pipelines ensure the broader engineering and product org can prepare for upcoming shifts without losing focus to speculative R&D. - [17:28](https://www.youtube.com/watch?v=DqF08Dz3nok&t=1048s) The true test of a working research lab model is that product organizations greet sudden frontier model releases with excitement rather than dread.

## Detailed walkthrough ### [00:06](https://www.youtube.com/watch?v=DqF08Dz3nok&t=6s) The Moving Frontier and Rule Number One

Building products during an ongoing technological revolution means dealing with what Dan Shipper terms "the unreasonableness of business as usual." The frontier is constantly shifting under product teams' feet: just the week before, cutting-edge models like Fable 5.1 and Astra 6 arrived, unlocking previously unthinkable capabilities overnight. Shipper demonstrates this with wild personal examples: using a single prompt, he generated an accurate 3D historical recreation of the Battle of Waterloo after letting the model churn for four hours. A teammate took an academic paper on agent simulations and, with a few prompts, had a thousand interacting agents simulated and visually rendered locally. Astra has even advanced enough to open Adobe Premiere directly and edit video, and it created the slide animations and substantial parts of Shipper's own summit deck without him ever clicking the "animate" tab.

This pace puts product leaders in an excruciating bind. Sticking strictly to a planned roadmap might work briefly, but the ground is moving beneath them. Asking customers what features they want rarely helps because customers are not familiar with the frontier and look to builders for guidance. Pivoting blindly—such as turning a standard B2B SaaS tool into a 3D multiplayer strategy game—is chaotic. Faced with this disorientation, Shipper establishes rule number one for navigating technology revolutions: "never make any major life decisions within 30 days of a meditation retreat, a psychedelic experience, or your first encounter with a frontier model."

[02:05](https://www.youtube.com/watch?v=DqF08Dz3nok&t=125s) The Structural Dilemma: Exploration vs. Exploitation

The fundamental tension inside modern product organizations stems from conflicting expectations. Companies expect everyone on the product team to simultaneously execute the roadmap at a high level and stay on the cutting edge of the frontier. These two modes represent opposing cognitive and operational postures:

- **Exploration (Divergent):** Trying dozens of new models, spinning up messy demos, testing wild ideas, and accepting that most of the output will be thrown in the trash. - **Exploitation / Execution (Convergent):** Saying "no" to distractions, narrowing focus, optimizing reliability, and delivering predictably against a structured roadmap.

```mermaid flowchart TD accTitle: The Tension Between Exploration and Execution accDescr: Divergent lab exploration pulls opposite to convergent product roadmap execution.

subgraph OrgMode ["Organizational Conflict"] direction LR Explore["Exploration (Divergent) • Test frontier models • Rapid prototypes • Discard ~90%"] Exploit["Execution (Convergent) • Deliver roadmap • Reliability & scaling • Adopt ~10%"] end

Explore <-->|"Fundamental Friction"| Exploit ```

Pulling an entire product team in both directions paralyzes delivery. Shipper proposes that instead of forcing everyone to do both, product organizations should incorporate elements of a dedicated research lab. By splitting responsibilities, companies can preserve focus on their core product while systematically discovering what comes next.

[03:13](https://www.youtube.com/watch?v=DqF08Dz3nok&t=193s) Why Form a Labs Team: Harnessing Early Adopters

In any organization, everyone is nominally juggling execution and exploration, but a handful of individuals naturally spend far more time playing on the frontier. These internal early adopters spend their weekends tinkering with new models, building side projects, and imagining alternate futures. Because they live in the future, they often spot where the industry and the product are heading well before anyone else does.

The dark side of early adopters is that left unchecked, "we can be a massive fucking distraction." They derail roadmap delivery by pitching speculative ideas to engineers trying to ship core work. The answer is to isolate and channel that energy by standing up a specialized **Labs team**. Crucially, in the age of AI, a labs team does not require a massive budget or head count; an organization can start with a "labs team of one." Because AI tooling provides individual engineers with massive leverage, a single person can test fresh model releases, run experiments, and report back on what actually matters without disrupting the core team.

Shipper points to Anthropic Labs as a prime example of this model working at scale. High-impact tools like Claude Code, Model Context Protocol (MCP), Skills, and Claude Design originated from small internal experiments run by a focused team, while hundreds of other tests were discarded before the company committed heavy engineering resources to the winners.

[07:31](https://www.youtube.com/watch?v=DqF08Dz3nok&t=451s) Running a Lab: Two-Slice Teams, Pirates, and Architects

To run an internal lab effectively, Shipper recommends abandoning older management conventions like Jeff Bezos's "two-pizza team" (eight to ten people). In the AI era, coordination overhead kills exploratory momentum, so the optimal unit is a **"two-slice team"** consisting of one or two people at most. Two people operating with AI leverage move faster and with far clearer shared vision than a mid-sized group.

Shipper outlines a complementary two-person team dynamic that has proven successful at Every:

- **The Pirate:** An obsessive "slop cannon" hunting for raw value. The pirate rapidly hacks together messy, fully vibe-coded prototypes, throws out what fails, and discovers hidden utility through sheer experimentation volume. - **The Architect:** A structured builder who inspects the chaotic, vibe-coded system created by the pirate and refines it into something cohesive, extensible, and genuinely valuable.

Pairing a pirate with an architect grounds chaotic experimentation in solid architecture. Beneath team structure lies the core principle of lab velocity: extreme dogfooding. The most important metric for a labs team is shortening the feedback loop between creating a prototype and knowing whether it is actually good. The fastest possible feedback loop occurs when builders create tools to solve their own immediate problems.

[08:51](https://www.youtube.com/watch?v=DqF08Dz3nok&t=531s) Running Competing Experiments and Generating Net-Positive ROI

When testing experimental tooling, the most critical safeguard is using prototypes for actual, substantive work rather than contrived sandboxes. Dan emphasizes that this is the only reliable way to answer an essential question: "is this useful or is it just new?" When internal dogsfooding is impossible, teams must immediately partner with a handful of early adopter customers who can provide rapid, closed-loop feedback. Furthermore, running an AI lab demands a counterintuitive management philosophy: encouraging multiple parallel, competing bets aimed at the exact same objective. While having different people build competing solutions to a single problem may look like a chaotic lack of organizational coherence, moving model capabilities make the frontier fundamentally uncharted. Exploring identical challenges from diverse perspectives helps map that shifting frontier much faster.

Because a true exploratory lab inevitably throws away 90% of what it builds, leadership must actively design mechanisms to ensure that high failure rate is still net-positive in terms of ROI:

- **Publishing "failed" work as content:** Every frequently turns internal experiments—what was built, what succeeded, and what fell flat—into public essays. Audiences love transparent postmortems, and the resulting readership reliably funnels in new subscribers and product users. - **Powering an early adopter VIP program:** Enthusiastic customers who crave proximity to innovation can be invited into an experimental cohort, turning rough prototypes into a high-touch retention and relationship-building channel. - **Educating the core product team:** At minimum, lab members act as scouts who continuously brief product managers and engineers on newly unlocked capabilities, preventing the primary product organization from getting distracted by speculative R&D.

---

[11:00](https://www.youtube.com/watch?v=DqF08Dz3nok&t=660s) Structuring the Research Pipeline

To bridge the gap between speculative prototypes and dependable software, teams need a defined research pipeline where ideas journey progressively from left to right. On the far left, initiatives are strictly confined to the lab, existing as dozens of small, scrappy experiments. Most die quickly, but the few that show genuine promise are pushed into real daily workflows across the company to see if teammates willingly adopt them.

Once an internal team demonstrates organic reliance on a tool, the project is mature enough to show to trusted outside customers and begin formal handoff discussions with product managers. At that juncture, the product team determines roadmap fit, scalability, and long-term viability before any code is officially integrated into customer-facing releases.

```mermaid flowchart LR accTitle: Research Pipeline Flow accDescr: Progression of ideas from early lab experiments to scaled product release.

subgraph Lab ["Lab Exploration"] E1["Experiment A"] E2["Experiment B"] E3["Experiment C"] end

subgraph Validation ["Internal Validation"] IW["Real Internal Work\n(Team Adoption)"] end

subgraph Handoff ["Product Integration"] EA["Early Customers"] PM["Roadmap Planning"] Prod["Scaled Product Release"] end

Lab -->|"Promising signal"| IW IW -->|"Organic usage"| EA EA --> PM PM --> Prod ```

---

[12:18](https://www.youtube.com/watch?v=DqF08Dz3nok&t=738s) Case Study: "Kate Bench" and Automating Taste

To illustrate the pipeline in action, Dan shares the story of attempting to "very lovingly" automate Every’s editor-in-chief, Kate, over a three-year span. Kate possesses exceptional taste for copy editing, but as the company grew to around 30 people, she became a severe operational bottleneck, frequently handling edits late into the night. Hiring another editor of comparable editorial sensibility was prohibitively difficult to recruit and train. For years, Dan experimented with feeding historical editing data into various language models, often sending late-night messages like: *"Not important, but I downloaded every single one of your copy edits over the last 3 years and had Fable try to do a copy edit on the latest piece."*

For a long time, the model outputs fell short, but recent advances in underlying capabilities finally crossed the quality threshold. The internal benchmark—dubbed "Kate bench"—advanced into active internal use powered by the company’s internal assistant infrastructure (`agents@every.to`). Kate can now invoke the agent by asking it to *"do a Kate pass of this draft."* The tool directly inserts suggested changes mimicking her stylistic preferences, freeing her from baseline mechanical edits.

``` +-------------------------------------------------------------------+ | KATE BENCH PIPELINE | +-------------------------------------------------------------------+ | 1. Ingestion: 3 years of Kate's historical copy edit decisions | | 2. Agent Pass: Model suggests tracked inline edits on new drafts | | 3. Architecture: Production dashboard built by Jannik (systems) | | 4. Impact Metric: Accepted suggestions & Kate did 12% less work | +-------------------------------------------------------------------+ ```

Once Kate and the broader editorial staff adopted the hacky prototype, Dan brought in a dedicated systems architect named Jannik to transform the fragile script into an enterprise-grade compounding engine. Jannik constructed an evaluation dashboard tracking how many suggested edits authors accepted and measuring the residual manual editing Kate performed afterward. Within a month, the system reduced Kate's post-agent editing workload by 12% month-over-month. With solid proof of internal value, the team began evaluating how to abstract the workflow for early design partners across non-editorial domains.

---

[15:17](https://www.youtube.com/watch?v=DqF08Dz3nok&t=917s) Pipeline Governance, Decision Criteria, and Self-Disruption

To keep experiments flowing smoothly without drifting into disconnected science projects, organizations must enforce disciplined operational hygiene:

- **Weekly cross-functional pipeline reviews:** Every tracks all initiatives on an internal Notion board and reviews their movement during weekly company-wide all-hands meetings. This guarantees the core product team sees upcoming breakthroughs well in advance and can anticipate architectural implications without conducting redundant investigations. - **Rigorous graduation criteria:** Moving an initiative from one stage to the next must be treated as a major milestone gated by hard requirements: - *Sustained retention:* Are internal users returning to the tool repeatedly of their own volition? - *The 10x filter:* Does the tool provide a ten-fold improvement over existing workflows after the initial novelty wears off a month later? - *Unit economics at scale:* Even if an experimental model interaction looks dazzling, can the company realistically afford the inference costs when serving thousands of paying customers?

The overarching objective of this R&D structure is deliberate self-disruption: merging clear winners back into the core offering before competitors do. While self-disruption has historically been difficult for incumbents, the sheer velocity of AI development makes it uniquely feasible today. Dan points to OpenAI's Codex as a prime model: an isolated, nimble strike team operated completely outside the flagship ChatGPT application, competing directly alongside several parallel internal teams who were simultaneously exploring alternative visions for the future of AI-assisted software development.

[17:28](https://www.youtube.com/watch?v=DqF08Dz3nok&t=1048s) How the Codex Team's Experiments Reshaped ChatGPT, and the Ultimate Payoff of a Research Lab

Dan Shipper recounts how the small Codex team ran numerous iterations across different form factors to figure out what the future of coding would look like—testing whether it belonged inside an IDE, a CLI, or something else entirely. Ultimately, in February 2026, they launched a desktop app that Shipper calls "fantastic." That desktop app grew at such a blistering pace that it was eventually merged directly into ChatGPT, becoming the very foundation of the main product. Because of their success with this rapid way of experimenting, that "small little Codex team" essentially ended up taking over an application with 800 million daily active users.

```mermaid flowchart LR accTitle: Codex Lab Experimentation to ChatGPT Core subgraph Lab["Small Codex Team"] A["Form factor iterations (IDE, CLI)"] --> B["Desktop app launch (Feb 2026)"] end subgraph Main["Core Product Integration"] B -->|"Rapid growth"| C["Merged as foundation of ChatGPT"] C --> D["Leads 800M DAU app"] end ```

For Shipper, this story captures the ultimate promise of running a research lab alongside your core product team: "you can actually build the next version of your product while scaling the one that you already have. And you can do it without going crazy." He leaves the audience with a simple litmus test for how to tell when this organizational model is truly working: "you will welcome moments when new models drop and be excited about them instead of dreading them." Wrapping up to applause, Shipper reintroduces himself as the co-founder and CEO of Every, pitching it as "the only subscription you need to stay at the edge of AI."

- **Form factors explored by the Codex team:** IDE integrations, CLIs, and ultimately a desktop application. - **Desktop app launch date:** February 2026. - **Scale of impact:** The small Codex team's experiment became the foundation of ChatGPT, taking over an 800 million daily active user (DAU) app.

## Notable quotes > [00:06](https://www.youtube.com/watch?v=DqF08Dz3nok&t=6s) never make any major life decisions within 30 days of a meditation retreat, a psychedelic experience, or your first encounter with a frontier model. — Dan Shipper

> [03:13](https://www.youtube.com/watch?v=DqF08Dz3nok&t=193s) we can be a massive fucking distraction. — Dan Shipper

> [08:51](https://www.youtube.com/watch?v=DqF08Dz3nok&t=531s) is this useful or is it just new? — Dan Shipper

> [12:18](https://www.youtube.com/watch?v=DqF08Dz3nok&t=738s) Not important, but I downloaded every single one of your copy edits over the last 3 years and had Fable try to do a copy edit on the latest piece. — Dan Shipper

> [12:18](https://www.youtube.com/watch?v=DqF08Dz3nok&t=738s) do a Kate pass of this draft. — Dan Shipper

> [17:28](https://www.youtube.com/watch?v=DqF08Dz3nok&t=1048s) you can actually build the next version of your product while scaling the one that you already have. And you can do it without going crazy. — Dan Shipper

> [17:28](https://www.youtube.com/watch?v=DqF08Dz3nok&t=1048s) you will welcome moments when new models drop and be excited about them instead of dreading them. — Dan Shipper

> [17:28](https://www.youtube.com/watch?v=DqF08Dz3nok&t=1048s) the only subscription you need to stay at the edge of AI. — Dan Shipper

## Mentioned Dan Shipper, Every, Lenny and Friends Summit, Fable 5.1, Astra 6, Adobe Premiere, Anthropic Labs, Claude Code, Model Context Protocol (MCP), Skills, Claude Design, Jeff Bezos, Kate, Kate bench, Jannik, Notion, OpenAI, Codex, ChatGPT

## Video description How do you keep building a product when AI capabilities change so quickly? At the Lenny and Friends Summit, Every co-founder and CEO Dan Shipper makes the case for a small research lab that can explore new ideas while the product team improves what already works. He explains how to run fast experiments, test them in real work, and bring the most promising results into the main product.

Recorded live at Lenny and Friends Summit on September 10, 2026, in San Francisco.

[Source](https://www.youtube.com/watch?v=DqF08Dz3nok)

中文简报

Every首席执行官Dan Shipper公布应用AI实验室运营手册通过将快速并行的探索与核心产品开发分离开来，软件团队可以在不扰乱日常运营的情况下，可持续地测试前沿AI模型。 Dan Shipper探讨了现代产品团队在平衡严格的路线图执行与前沿AI模型飞速发展之间所面临的困境。为了化解这一矛盾，他提倡建立自主运作的“两人（two-slice）”研究实验室，将高速度原型开发者（“海盗”）与系统思考者（“架构师”）配对，直接针对内部工作流开展并行实验。鉴于大约90%的原型都会失败，团队可以通过公开内容、早期采用者反馈和内部调研来获取价值，同时借助跟踪留存率、经济可行性和10倍实用性的结构化管道来筛选成功的实验。成功的突破性成果——例如Every的“Kate Bench”编辑自动化或OpenAI的Codex桌面应用——最终会从实验实验室毕业，成为核心产品的基础架构。产品团队在收敛性的路线图执行与发散性的AI探索之间面临着不可调和的冲突，这需要设立专门的内部研究实验室来测试前沿模型，同时不干扰核心业务的交付。 (2:05) 高效的AI实验室抛弃了传统的“两张披萨团队”，转而采用仅有1-2人的“两片披萨团队”，通过将高速原型开发者（“海盗”）与系统思考者（“架构师”）配对，最大限度减少沟通成本。 (7:29) 由于实验室必然会淘汰大约90%的原型，团队可以通过将实验转化为公开内容、供早期采用者客户计划使用以及为核心产品团队提供参考，从而在失败中实现净正向投资回报率。 (8:51) 在Every，利用“Kate Bench”自动化编辑工作流在首月将人工文字编辑减少了12%，这证明了内部试用和架构师评估仪表盘如何在更广泛发布之前验证AI实验。 (14:12) 一个探索编程形态的小型自主团队构建了一款桌面应用，该应用迅速扩张，最终被整合进拥有8亿日活跃用户的ChatGPT产品中。 (17:28)

Reading summary

Dan Shipper argues that navigating rapid advances in frontier AI requires product organizations to decouple exploratory research from core roadmap execution. By establishing lightweight, internal research labs that build and discard prototypes quickly, companies can discover future capabilities without paralyzing existing product delivery. Resolving the Clash Between Exploration and Execution (2:05) Shipper contends that product teams face an organizational dilemma between two contradictory postures: exploration (divergent testing of frontier models where roughly 90% of output is discarded) and exploitation (convergent execution focused on reliability and roadmap delivery). Forcing product managers and engineers to handle both simultaneously creates paralysis. Furthermore, customer feedback offers limited guidance when underlying model capabilities change rapidly, and sudden pivots risk destabilizing the entire company. Structuring Lightweight Teams of Pirates and Architects (7:31) To maintain exploratory velocity, Shipper advises replacing Jeff Bezos's eight-to-ten person "two-pizza team" with a "two-slice team" of one or two contributors, arguing that modern AI leverage eliminates the need for large headcounts. Shipper models this on an internal pairing dynamic: a "pirate" who rapidly deploys messy, vibe-coded prototypes to discover latent utility, complemented by an "architect" who inspects the chaotic prototype and restructures it into a cohesive, reliable system. Shipper highlights Anthropic Labs as evidence for this approach, pointing out that tools like Claude Code, Model Context Protocol (MCP), Skills, and Claude Design originated from small internal experiments, while hundreds of other tests were discarded before the company committed heavy engineering resources to the winners. Validating Real Utility and Monetizing Failure (8:51) Shipper stresses that exploratory velocity relies on extreme dogfooding to answer whether a prototype is genuinely useful or merely novel. Builders must use prototypes in substantive internal work rather than synthetic sandboxes, running multiple parallel bets aimed at the same objective. Because exploratory units discard approximately 90% of what they create, Shipper argues leadership must offset this failure rate by publishing transparent postmortems as public content, granting early access to VIP users, and transferring technical insights to the core engineering team. Prototypes that show promise advance through a staged research pipeline: starting as isolated experiments, proving themselves through voluntary internal adoption, and eventually progressing to handoff reviews where product managers evaluate scalability, unit economics, and roadmap fit. Operational Governance and the Codex Benchmark (15:17) Systematic pipeline governance requires strict graduation criteria. At Every, Shipper notes that experiments tracked on Notion boards must demonstrate sustained organic retention, deliver a 10x workflow improvement after initial novelty fades, and maintain viable inference unit economics. In Every's internal "Kate bench" project, three years of editorial data were turned into an automated agent pass, which systems architect Jannik formalized into an evaluation dashboard that reduced the editor-in-chief's manual workload by 12% month-over-month. Shipper cites OpenAI's Codex team as a historical model for internal self-disruption. Operating outside the main ChatGPT application, a small strike team tested various form factors before launching a desktop application in February 2026. The desktop tool grew rapidly, ultimately merging into the core product architecture and underpinning an application serving 800 million daily active users. Shipper concludes that when an internal lab operates effectively, product teams can scale existing offerings while simultaneously building their next iterations, shifting their reaction to frontier model releases from dread to strategic anticipation. The model relies heavily, however, on maintaining disciplined graduation metrics to prevent speculative prototypes from becoming expensive distractions.

Source quotes

[0:06] Dan Shipper: never make any major life decisions within 30 days of a meditation retreat, a psychedelic experience, or your first encounter with a frontier model. [3:13] Dan Shipper: we can be a massive fucking distraction. [8:51] Dan Shipper: is this useful or is it just new? [12:18] Dan Shipper: Not important, but I downloaded every single one of your copy edits over the last 3 years and had Fable try to do a copy edit on the latest piece. [12:18] Dan Shipper: do a Kate pass of this draft. [17:28] Dan Shipper: you can actually build the next version of your product while scaling the one that you already have. And you can do it without going crazy. [17:28] Dan Shipper: you will welcome moments when new models drop and be excited about them instead of dreading them. [17:28] Dan Shipper: the only subscription you need to stay at the edge of AI.

Original transcript

[0:06] Okay, so I'm going to talk about how to [0:08] build products on a moving frontier [0:11] or as I like to call it the [0:13] unreasonableness [0:15] of business as usual during technology [0:17] revolutions and what to do about it. So, [0:19] the world just changed. The world just [0:21] changed again. Uh, we had Fable 5.1 [0:25] launched last week. We had Astra 6 [0:28] launched last week and these are [ __ ] [0:30] sick models. Like look at this. [0:32] This I made this like Battle of Waterloo [0:35] 3D like historical reproduction that's [0:39] historically accurate with like a single [0:41] prompt and it just turned for like 4 [0:43] hours and then I got that. Like that's [0:44] crazy. That is crazy. Someone else on my [0:47] team made this uh, sim agent simulation [0:50] where he fed it a scientific paper about [0:52] how agents interact and and how they get [0:53] simulated together and then he had like [0:55] a thousand agents all running on his [0:57] computer and like visualized by this [0:59] after like just a couple prompts and [1:01] putting a paper in. It's also now [1:03] starting to do our video editing. We do [1:04] a lot of videos and like Astra is [1:06] actually good enough to go into Premiere [1:08] and like do a bunch of editing. [1:10] And uh, also fun fact, the all of the [1:12] animations in this deck and a lot of [1:15] this deck was made by Astra. Like I did [1:16] not touch a lot I didn't touch the [1:18] animate tab at all and we'll get into [1:20] that in a sec. So, this is crazy. [1:23] What should you do? Do you keep your [1:24] head down and focus if you're a product [1:26] leader and you have a product team? [1:27] That might work for a little while. [1:31] Uh, do you ask your customers what they [1:32] want? Like maybe, but most of your [1:34] customers are not familiar with this [1:35] stuff. They're actually looking to you [1:38] uh, to tell them what to do. Do you turn [1:39] your B2B SaaS app into a 3D multiplayer [1:41] strategy game? [1:43] Maybe. Maybe, but it's hard. This is [1:45] This is hard. This is a This is a big [1:47] problem that I think everyone in this [1:48] room faces. I face it. So, the first [1:50] thing to know, rule number one, is never [1:52] make any major life decisions within 30 [1:55] days of a meditation retreat, a [1:57] psychedelic experience, or your first [1:59] encounter with a frontier model. [2:02] All right, we're settled. We're settled. [2:03] That's rule number one. So, let's get [2:05] into the problem, though. So, what's the [2:07] core problem? Why is this so hard? It [2:09] feels like if you're running a product [2:10] team, it feels like it's everyone's job [2:13] to execute the road map at a high level [2:17] and to stay at the frontier at the same [2:18] time. And that's really, really hard [2:20] because these are very opposed ways of [2:22] working. Uh if you want to explore the [2:24] frontier, you're exploring. That's [2:25] divergent. You're doing lots of [2:27] different things. You're probably going [2:28] to throw a lot of this stuff out. You're [2:29] You're trying all these new models. [2:30] You're doing demos. Exploiting is very [2:32] different. You're converging. You are [2:34] focusing. You're saying no to things. [2:35] You're trying to execute against a road [2:37] map that you've like you've planned out. [2:39] And doing both of those uh they're [2:41] You're pulling in the opposite [2:42] direction. Um So, what should we do? [2:45] Well, the thing I want to talk about in [2:47] this talk is I My contention that you [2:51] should run your product org like a [2:54] research lab or you should add some [2:55] research lab lab elements to your [2:57] product org. And we're going to go [2:59] through why you should make a lab, how [3:00] to run a lab, and then what what like [3:03] how to bring what works from the lab [3:05] back into your main product. We're going [3:07] to use examples from stuff we do at [3:08] Every and from some of the uh best [3:10] companies in AI and how they do it. [3:12] So, [3:13] let's start. Why should you make a lab? [3:17] The problem is [3:19] as you said before, right now everyone [3:20] in your product org is doing both [3:22] exploration and execution. The yellow [3:24] circles are the execution. Exploration [3:26] is purple. You can see everyone in your [3:27] org is doing both. [3:29] And uh what what's really interesting, [3:32] though, is there are some people in your [3:34] org, I guarantee it. There are some [3:36] people in your org who are doing way [3:38] more exploring than others. And that's [3:41] they're really important to identify. Um [3:44] because if you identify these people, [3:50] they are your early adopters. We are [3:52] your early adopters. And by the way, [3:53] that animation and this whole all this [3:55] stuff all built by Astra round of [3:57] applause for Astra incredible incredible [4:00] I did not do any of this. So, we are [4:02] your early adopters. [4:04] We love new things. We're already [4:06] running Astra and Fable on the weekends [4:08] to do our own personal projects or do [4:09] things that are similar to [4:11] to to explore things that we might bring [4:13] to work. And because of that, there's a [4:16] lot of things we might be excited about [4:17] that aren't useful, but because of that [4:19] we're going to have a really good sense [4:22] of what might what might your product [4:24] become because we're like living in the [4:26] future already. The problem is that we [4:29] can be a massive [ __ ] distraction. [4:32] And [4:33] the So, the the key question to to ask [4:36] yourself if you're running a product org [4:38] is how do I harness the early adopters [4:41] in my team on my team and maybe even [4:43] some of my some of my customers who are [4:45] early adopters without distracting [4:47] everybody else. And the solution that we [4:51] found at every and that I think a lot of [4:52] big a lot of really great companies are [4:53] trying to do is make a labs team. On a [4:55] labs team what you've started to do is [4:58] separate concerns. Some people are in [5:00] charge of improving and scaling what [5:02] already works. They're they're on the [5:03] product team. And some people are in [5:05] charge of exploring what what what comes [5:08] next exploring the frontier doing all [5:09] these demos trying these new models all [5:10] that kind of stuff. That allows you to [5:12] get the best of both worlds and I'll [5:13] I'll explain how that works and why. And [5:15] if you're looking at this and you're [5:16] thinking, "Oh, that's really expensive [5:17] and I don't have the resources that and [5:18] all that kind of stuff." What is amazing [5:21] about AI is it allows you to have a labs [5:25] team of one. [5:26] You can just have one person on your [5:27] team whose job it is maybe even like [5:29] right when a new model comes out to go [5:31] explore all that stuff and come back and [5:32] tell you what they learned. It actually [5:33] does not require a lot of resources [5:35] because all the people on your team are [5:37] now so they have so many superpowers [5:39] because they can just go off and have [5:40] Astra and Fable do a bunch of stuff that [5:42] it doesn't require a huge investment to [5:44] do this. So again, to sort of pull apart [5:47] the difference between a labs team and a [5:49] product team, on a labs team your job is [5:51] you're going to explore capabilities, [5:52] especially when new models drop, to see [5:54] what's now possible. You're going to run [5:56] a ton of experiments in parallel, and [5:57] we'll get into that. But crucially, the [5:59] expectation for a labs team is very [6:01] different. On a labs team, you are going [6:03] to expect to dispose of like 90% of what [6:07] you make. You try it and then you throw [6:09] it away. Product team's very, very [6:10] different. Your job on a product team is [6:12] you're going to improve and scale the [6:13] product. You want to deliver for [6:15] existing customers. You want to make [6:16] sure that they feel like they understand [6:18] the product, it's coherent. You're not [6:20] just throwing a bunch of garbage at [6:21] them. So, for a product team, you want [6:24] to expect to adopt about 10% of what the [6:28] labs team makes or tries. And this is [6:30] how they sort of start to work together. [6:32] And there's a lot of really good [6:33] examples of this starting to work in AI [6:35] in a way that like we've had labs teams [6:37] for a long time, but it's starting to [6:38] work in a way that it has never, ever [6:40] worked before because of this [6:41] technology. So, really good example is [6:43] Anthropic Labs. Uh I'll be talking with [6:46] uh some people at Anthropic a little bit [6:47] later about this, but and Anthropic [6:49] Labs, Claude Code, like one of the [6:52] biggest, most successful productivity [6:54] products of all time, came out came out [6:56] of Anthropic Labs, a separate little [6:57] group in charge of doing lots of [6:59] experiments. MCPs, skills, Claude [7:01] design, all of these things came from a [7:03] small group of people inside of [7:04] Anthropic experimenting with things, and [7:06] then Anthropic investing more and more [7:08] into the winners and the ones that [7:10] worked. And there's like a thousandth [7:11] experiments that you never you've never [7:13] seen. Let's say you you're convinced [7:15] now that this sort of structure gets the [7:18] best gets you the best of both worlds. [7:19] It helps you to unleash your uh your [7:23] early adopters without distracting the [7:26] people on the product team who need to [7:27] actually deliver on the road map. [7:29] Question is like how do you run a lab [7:31] well? So, the first thing that I've [7:33] started to come to is you really want to [7:35] use very, very, very small teams. For a [7:37] long time the uh the standard for team [7:39] size was the the two-pizza team. So, [7:42] that's like eight to 10 people. That's [7:43] what Jeff Bezos came to. I think in the [7:45] AI age, I I call it a two-slice team. [7:47] Like you want one or two people max. You [7:49] can get so far, so fast with one or two [7:51] people that anything more, there's a lot [7:52] of coordination overhead, and there's [7:54] differing visions, and it just doesn't [7:55] work. One or two people is really, [7:56] really great. [7:57] And I think [snorts] [7:58] the the composition that works for us, [8:01] and I I I think it's starting to work [8:03] for other people, is I I call them [8:04] pirates and architects. So, one person [8:06] on the team is the pirate. They are slop [8:08] cannons who are absolutely obsessed with [8:10] finding value. That's That's me. Like [8:12] I'm just out there just doing lots and [8:13] lots of stuff, and I'm throwing it away, [8:14] and it's all messy, but I'm going to [8:15] find something really interesting. [8:18] And then architects are people who are [8:20] like looking at a messy system, like [8:22] maybe something that I built that's [8:23] completely vibe coded, and trying to [8:25] help shape it into something that's [8:27] valuable and beautiful and extensible. [8:29] And pairing those types of people [8:31] together, I think is very, very powerful [8:33] in AI. [8:35] Um the next best practice that I think [8:37] is really important is dog fooding. The [8:39] biggest thing that that matters on a [8:42] labs team is making the feedback loop as [8:44] tight as possible between making [8:45] something and knowing if it's good. [8:47] And the tightest feedback loop is making [8:49] something for yourself. [8:51] If you can do that, I would do it. If [8:53] you can't, then it's really important to [8:55] get a couple of early customers or early [8:56] adopter type customers [8:59] uh to help help you test with a tight [9:01] feedback loop so that you can iterate [9:02] really, really fast. Um and to use the [9:05] the stuff you're building, all the [9:06] experiments, ideally should be used for [9:09] actual work that you actually have, so [9:11] you can tell is this useful or is it [9:13] just new? It's a big thing you need to [9:15] differentiate. [9:17] Another best practice, which I think is [9:18] quite different from how you should [9:20] probably run a product team, is you [9:22] should be building many experiments in [9:23] parallel, even if those experiments are [9:25] trying to do the same thing. Try [9:27] competing approaches to the problem. Uh [9:30] it might look like a mess and sort of [9:32] like a lack of coherence, but [9:33] everybody's going to have a a bit of a [9:35] different take on how to solve a [9:37] particular problem, and when [9:39] capabilities move, the frontier becomes [9:41] very unknown. And so, having people try [9:44] the same thing from different [9:45] perspectives helps you map that frontier [9:47] and decide what's valuable and what's [9:49] not. And then the last thing that I [9:50] think is really important for doing labs [9:53] is to figure out ways to make your [9:55] experiments net positive in terms of [9:58] ROI. So, we're going to throw out 90% of [10:00] what we make. How do we make even that [10:03] 90%? Even the even the 90% that doesn't [10:05] make it into the product eventually, how [10:06] do we make that actually useful for us? [10:08] So, one thing that we've done a lot at [10:10] every that has worked really well is we [10:12] turn them into external content. People [10:14] love to see here's all the things that [10:16] that we tried, here's what worked, [10:17] here's what didn't, and then that brings [10:19] customers to to use to read our stuff, [10:21] use our products, all that kind of [10:22] stuff. If that's possible, I highly [10:24] recommend it. Another thing to do is use [10:27] these experiments to feed an early [10:28] adopter program. There are probably a [10:30] lot of customers that you have that want [10:33] to be closer, want to be in the fold, [10:35] who are early adopters, and this can be [10:37] a value add for them to get to work with [10:39] you more closely. [10:40] And then the last thing is like at base, [10:43] the labs team should be sharing what [10:44] they learn about capabilities and what's [10:46] now possible with the product team, so [10:48] the product team can take that into [10:49] account [10:51] as they build things without getting [10:53] distracted by having to go, you know, [10:54] explore the frontier themselves. So, [10:56] let's say you've done that, you're [10:59] you've decided to make a lab, you're [11:00] starting to figure out [11:02] like all the all the little best [11:03] practice. Now, how do you get the stuff [11:06] that you're making into from the lab [11:08] into your product? The thing you need to [11:10] do is make a research pipeline where [11:12] ideas start on the left, and they start [11:15] out as lab only, and there are lots of [11:17] little experiments, and then you move [11:19] them progressively from the left to the [11:21] right, and at some point you have a [11:22] handoff with your product team where [11:24] they start to get incorporated into the [11:26] product. So, to give you a sense of what [11:27] this looks like, [11:28] you've got the the labs team, they're [11:30] running lots of lots and lots of [11:31] experiments. Um and most of them are not [11:35] good, but a few of them might work, and [11:38] those become things that you start to [11:39] test in real work. One of the things [11:41] that we do all the time internally is we [11:43] just let other people on the team adopt [11:45] it and see if they like it. Uh there are [11:47] different configurations that work [11:48] depending on what kind of customers you [11:50] serve, but the first step for us is do [11:52] other people on the team start to use [11:53] it? [11:54] If that's the case, it's often then [11:56] ready to put in front of early customers [11:58] and ready for the product teams to like [12:00] really take a look at it and figure out [12:01] where does this fit in our road map and [12:04] um how might we use this? And uh and [12:07] even there not everything makes it, but [12:09] like at at some point uh the thing that [12:11] you do will be ready to scale and [12:12] release to customers. And we'll go [12:14] through some examples of how that works. [12:15] That's the uh the research pipeline, and [12:17] I'll give you like a little bit of a [12:18] concrete example. This is Kate, our [12:20] editor-in-chief, who I've been trying to [12:22] automate for the last 3 years um very [12:24] lovingly. Um and Kate [12:27] Kate is fantastic at a lot of things, [12:29] but one of the things she's really good [12:30] at is she has a very good taste for copy [12:32] edits. [12:33] Um but as the team has grown, we're [12:35] about 30 people now, uh she can't do all [12:38] of the copy edits herself. Um [12:41] and we could hire someone, but it's [12:42] really hard to get someone who has the [12:43] level of taste that she has and train [12:44] them and all that kind of stuff. So, [12:46] what I've been trying to do is like [12:47] figure out how do we make [12:49] how do we make AI like help her with [12:51] this so she can expand her impact in the [12:53] org without having to spend additional [12:55] time doing copy edits really late at [12:56] night. And so, Kate often wakes up to [12:59] messages like this from me saying, "Not [13:01] important, but I downloaded every single [13:03] one of your copy edits over the last 3 [13:04] years and had Fable try to do a copy [13:06] edit on the latest piece." Um [13:08] and I've literally been doing stuff like [13:09] this for a couple years, and it just it [13:12] just started to work where the [13:14] capabilities are good enough that it's [13:15] it's actually like uh Kate ready for [13:18] Kate to actually use. So, we move from [13:21] we we we we call it Kate bench. So, [13:23] we're starting to move from okay, we've [13:25] done a bunch of experiments. Now there's [13:26] one that's like starting to work. Now, [13:28] let's push it out into the organ and see [13:30] if we can get some internal use. So, [13:33] we may we have a we have an every agent. [13:34] It's it's a it's a single agent for your [13:37] entire company. It helps your company [13:38] get AI built and use all all of AI [13:40] workflows. agents@every.to. [13:43] But, we use it internally and what what [13:45] happens now is Kate is starting to be [13:47] like, "Okay, at every do a Kate pass of [13:49] this draft." And it'll go in and it will [13:52] literally file suggested changes like [13:55] she would based on her historical edits [13:57] and then improve over time. And this is [13:59] something that I built just in my spare [14:00] time that just it just sort of works, [14:02] but it's like [14:03] now we're starting to see that there's [14:05] some value for the organization cuz [14:07] she's starting to adopt it and it's [14:08] starting to spread to other people. And [14:10] that's a really good sign. [14:12] So, when we get to this [14:13] internal use stage, this is the time [14:16] where I would then bring in an [14:18] architect. And I have one in particular [14:20] who I work with on my team. His name is [14:21] Yanek. He's fantastic. [14:23] And Yanek is going to Yanek goes and [14:24] takes this like messy thing that kind of [14:26] works and makes it really great. So, now [14:28] we have a whole dashboard of okay, like [14:30] for each document, how many [14:33] how many suggestions were were accepted [14:34] and then how much work is remaining for [14:37] Kate to do after after we go in and do [14:40] it. So, it checks what are the edits she [14:41] does after the the agent goes and does [14:43] the edits and you can see we're [14:45] improving. We did she did 12% less work [14:49] on these types of edits than she did the [14:52] month before. And his job is to make it [14:54] like a real system that we can improve [14:55] and compound [14:57] in an ongoing way. Now, it's starting to [14:59] work well enough that we're starting to [15:00] think about okay, how do we put this in [15:02] front of early customers? Does this sort [15:04] of process work for things beyond copy [15:06] editing? I think it does, but [15:09] you know, we've sort of taken it from [15:10] it's only an experiment in the lab to [15:12] something that we're ready to be like, [15:13] "Okay, I think we can give this to [15:15] customers." So, a few best practices for [15:17] how to continually push things through [15:20] your lab pipeline. [15:22] One thing that we do that I think is [15:23] very important [15:25] is we review the pipeline regularly. We [15:27] have I have a tracker in notion. Every [15:29] week on our all-hands, everyone talks [15:31] about what's going on in our pipeline. [15:34] And I think that's crucially important [15:36] because what you want to be able to do [15:37] is you want your product team, the rest [15:39] of the product team who's not playing [15:40] around with everything, to know here's [15:42] what we think is interesting. Here's [15:44] what's starting to move up the pipeline [15:46] so that you can start to think about, [15:48] okay, if this does actually prove out, [15:50] what are the implications for the [15:51] product without having to then go like [15:53] scramble to like do your own experiments [15:55] and all that kind of stuff. It helps [15:57] keep the lab and the pipe and the [15:59] product [16:01] orgs in sync. Another really important [16:03] thing is defining clear decision [16:05] criteria for moving an an idea through [16:08] the pipeline and really making it an [16:09] event when you do it. So, some of the [16:11] decision criteria, like our big one is [16:13] are people using it and are they coming [16:14] back? Because what you're trying to what [16:17] you're trying to do is use internally [16:19] use as a proxy for value. The second big [16:21] question, is it 10x better than what [16:23] currently exists? And this is actually a [16:25] a really interesting one, a really [16:27] interesting filter because what is new [16:29] feels very exciting, but the question is [16:31] when it's a month later, is it actually [16:33] any better? It's hard to know unless you [16:35] use time and usage over time as a [16:37] filter. [16:39] And then also, is it affordable? Like [16:40] maybe this works now, but is it can we [16:42] actually serve it at scale for our [16:44] customers? Another really big big [16:46] question and and having that clear [16:47] criteria allows you to make sure that [16:49] you're you're being rigorous about how [16:50] you do this. But the ultimate goal is [16:52] that you [16:53] merge the winners into the main product. [16:56] And this is this is this full cycle of [16:59] labs to merging winners is something [17:02] that is starting to happen really, [17:03] really well. Like we've had labs for a [17:05] long time, but the and the idea of doing [17:07] a self-disruption of building something [17:09] that then disrupts your own product has [17:11] been around for a while, but is actually [17:12] starting to work now in AI because you [17:14] can move so fast. I think a really good [17:16] example is Codex. [17:18] So, [17:19] um Codex built by a small team working [17:21] outside of the main app. There were a [17:23] bunch of different teams inside of [17:24] OpenAI who were working on the future of [17:26] coding at the same time as Codex was. [17:28] They had a They did a bunch of different [17:29] iterations on different form factors for [17:31] what the future coding would look like. [17:33] Is it inside an IDE? Is it a CLI? All [17:35] that kind of stuff. Um but they launched [17:37] their desktop app, which uh I think is [17:39] fantastic. They launched it in February [17:40] 2026. And it it grew so fast that they [17:45] eventually just merged it into ChatGPT, [17:48] and it became the foundation of ChatGPT. [17:50] Like they the the small little Codex [17:52] team then just uh took over this 800 [17:55] million daily active user app because [17:58] they were so successful with this this [18:00] way of experimenting. So, I think that's [18:02] the promise of doing something like this [18:04] is that you can actually build the next [18:06] version of your product while scaling [18:09] the one that you already have. And you [18:11] can do it without going crazy. The way [18:13] that you know this is working is you [18:16] will welcome moments when new models [18:18] drop and be excited about them instead [18:20] of dreading them. And that is my talk. [18:22] Uh I'm Dan Shipper. I'm the co-founder [18:24] CEO of Every. You should check out [18:25] Every. We're the only subscription you [18:26] need to stay at the edge of AI. Thank [18:28] you very much. [18:29] >> [applause]