Google Labs Chief Josh Woodward Rejects Design Sprints for AI Bets

Google Labs负责人Josh Woodward在AI押注中弃用设计冲刺 企业创新实验室若隶属于现有业务部门，而非作为追踪临近突破能力并及早砍掉停滞项目的独立前沿团队运作，通常会在四年内瓦解。 Josh Woodward解释了为何模型的快速演进要求企业设立专门的前沿团队，去追踪那些“几近可能”的技术，并通过定性的直觉反应而非标准的虚荣指标来衡量早期兴趣。他反对将不同职能扁平化为通用的“构建者”角色，转而提倡“主修与辅修”技能框架，并配合“爆发式耐力”的季节性节奏。此次访谈详述了运营内部实验室的机制，涵盖管理滚动的100天里程碑周期，以及构建清晰的0到1、1到10和10到100生命周期阶段。Woodward还探讨了软件的未来究竟会倾向于数百万个微型界面还是整合的超级应用，同时介绍了Google Labs内部奖励算力效率和快速解决问题的团队仪式。 Google Labs实时追踪一份“几近可能”的技术阈值清单（如延迟、语言覆盖和多模态），其背后有一份包含82项明确未来预测的内部愿景文档作为支撑，其中预计十有八九会失败。 (7:43) Google Labs并未将团队扁平化为单一通用的“构建者”角色，而是推行大学式的“主修与辅修”模式：专业人员保留其核心技能（产品经理、设计或工程）作为主修，同时将AI原型设计和氛围编程（vibe-coding）作为辅修。 (37:33) 由于AI能力变化过快，无法制定长期路线图，Google Labs将运营规划限制在滚动的六个月窗口期内，并以50至100天的冲刺周期执行，中间穿插季节性的恢复与极客开发期（hacking periods）。 (42:13) 大多数企业研发实验室在3到4年内就会因缺乏独立性或无法将发明商业化而折戟；要想生存下来，需要CEO级别的庇护、文化沟通桥梁的搭建者，并将0到1的探索与1到10、10到100的规模化阶段分离开来。 (43:38)

Josh Woodward, VP at Google Labs, argues that companies must operate dedicated frontier teams to rapidly test emergent AI capabilities, though internal labs risk failure unless granted structural independence, cultural humility, and permission to discard unproductive bets. Tracking the 'Almost Possible' Threshold (7:43) Rather than relying on scheduled brainstorms or design sprints, Woodward finds that breakout products emerge organically from obsessed individuals tinkering on side projects. Google Labs structures ideation around an 'almost possible' framework, maintaining an explicit watchlist of user experiences that fall just short of viability under existing model limits. When a model upgrade lowers latency or unlocks multimodal capabilities, an idea crosses a threshold into technical feasibility, prompting the lab to deploy a small pod immediately. Woodward pairs this technical tracking with broad behavioral research outside Silicon Valley and an internal document of 82 specific predictions across domains like immersive entertainment, software design, and knowledge work. He expects up to 99% of these forecasts may prove incorrect, but considers a concrete point of view necessary to intersect frontier unlocks with genuine, high-value consumer needs. Validation by Physical Signal and Self-Pruning (14:37) In the earliest stages of an AI product, Woodward disregards conventional quantitative metrics like daily active users or D7 retention in favor of qualitative physical reactions. Testing raw prototypes in person to see whether a user leans in and their eyes widen or pupils dilate serves as the core test of real interest. Because reaching viable product-market fit routinely requires three to five pivots, teams are encouraged to stay anchored to the user's problem rather than the specific software solution. Equally critical is a culture that permits teams to kill stalled efforts from the bottom up. Woodward notes that frontline teams almost always recognize a dead end before leadership does. When a product manager canceled a planned Gemini feature after user testing failed to demonstrate value, leadership openly praised the decision, reinforcing psychological safety to abandon sunk costs. Mispriced Skills and Organizational Pods (31:36) As execution costs fall, Google Labs has compressed initial team sizes from five to seven people down to two to four members who operate with fluid, overlapping responsibilities. Woodward rejects the idea that functional disciplines have completely collapsed into a universal 'builder' title. Instead, he advocates a 'major and minor' model where practitioners retain deep craft expertise in disciplines like product management or UX as their major, while treating AI-assisted rapid prototyping as a supportive minor. When evaluating talent for this environment, Woodward looks past standard industry buzzwords to mispriced traits: a candidate's 'unlearning rate' (the speed at which they discard invalidated assumptions), 'explosive endurance' (the capacity to alternate between intense launch sprints and exploratory recovery periods without burning out), and the ability to rapidly establish trust across teams. Protecting Frontier Labs from Corporate Atrophy (46:31) Historical corporate labs routinely fail within three to four years because parent organizations either bury them beneath existing business units or fail to commercialize their prototypes. To survive, Woodward states a lab must remain structurally independent with CEO-level backing, while adopting an explicit service priority: users first, Google second, and the lab product third. This ethos prevents insular fiefdoms and clarifies whether an initiative is designed to graduate into existing flagship products or create standalone commercial categories. To prevent breakout hits from crowding out raw experimentation, Google Labs segments its work into three operational stages: 0-to-1 (discovery and prototype novelty), 1-to-10 (retention and core loop validation), and 10-to-100 (funnel optimization and mainstream scale). Planning horizons are limited to rolling six-month roadmaps with 50-to-100-day execution milestones to stay aligned with unpredictable foundation model releases. An unresolved structural question is whether AI will ultimately drive hyper-fragmentation into thousands of bespoke micro-products or consolidation into a few adaptive super apps. Google Labs currently runs parallel experiments across both models, letting empirical user behaviors and the rise of autonomous agents determine which interface architecture endures.

[0:00] Josh Woodward: They very rarely come from design sprints. They don't come in conventional places. You can't microwave good ideas. [0:00] Josh Woodward: How fast can they learn something and then walk away from it? [0:00] Josh Woodward: When you're showing people early prototypes, you're looking at people's eyes. That is the metric. [0:00] Josh Woodward: The team usually knows before the leader knows. [14:52] Josh Woodward: It is not a DAU, it's not a DAU over MAU, it's not a D7 retention—it's none of that. [14:52] Josh Woodward: I think the future is going to look like this, and I can't explain it. [18:26] Josh Woodward: I don't think we should launch this. It's not good. [27:46] Josh Woodward: Most people in the world are never, ever, ever going to care about an Elo rating. They may not even know how to pronounce Elo. It's like, do you know what? Like it just doesn't matter. [50:26] Josh Woodward: We are obsessed about finding people who just like can't stop themselves from building. [50:26] Josh Woodward: You leave a conversation, you're like, 'I want to be with that person. It's like, I'll run through a wall with that person.' [60:28] Josh Woodward: This is not a bad meeting, this is a good meeting.

[0:00] My thesis is that every company needs to [0:02] start thinking and operating like a labs [0:04] team. [0:05] >> Every company definitely needs a set of [0:07] people, a team. Their job is to be at [0:10] the frontier. They're the people who [0:11] have like eight $200 a month [0:13] subscriptions on everything. [0:14] >> Everyone has to get really good at [0:16] trying a bunch of stuff with the latest [0:17] models, figuring out what is now [0:19] possible before your competition does, [0:21] before a startup comes and eats your [0:22] lunch. [0:23] >> You've got to try to create a space [0:25] where weird things can grow. If you look [0:27] at the history of a large company who [0:29] starts a lab, the lab is usually [0:31] ineffective and fizzles out after about [0:33] 3 to four years. You can't just nestle [0:36] this under an existing business unit. It [0:38] [music] has to have its own kind of [0:40] independence. [0:41] >> You've seen more ideas thrown at the [0:43] wall and tried and prototyped than maybe [0:46] anyone else. What are some patterns of [0:47] where the best ideas come from? [0:49] >> They very rarely come from design [0:51] sprints. They don't come in conventional [0:53] places. You can't microwave good ideas. [0:55] What skills are you finding are trending [0:58] up in value? [1:00] >> I'm actually looking a lot right now for [1:01] people. What's their unlearning rate? [1:03] How fast can they learn something and [1:05] then walk away from it? [1:06] >> What's something that might surprise [1:08] people about what product market fit [1:09] looks like? [1:10] >> When you're showing people early [1:11] prototypes, you're looking at people's [1:13] eyes. That is the metric. [1:16] >> What are signs that it's time to kill an [1:18] idea? [1:18] >> The team usually knows before the leader [1:20] knows. [1:22] >> Today, my guest is Josh Woodward. Josh [1:25] is head of Google Labs, the Gemini app, [1:27] and AI Studio. He's been at Google for [1:30] over 16 years, and his job is to [1:32] experiment with and scale new AI [1:34] products. My premise for this [1:36] conversation, and why I believe it is so [1:38] valuable, is that every company is being [1:40] forced to think and operate like a labs [1:42] team. Because what is now possible is [1:45] changing so quickly. You have to get [1:47] very good at playing with the latest [1:49] technologies, running tons of [1:50] experiments, and uncovering new [1:52] opportunities before your competition or [1:55] startups do. Josh runs the longest [1:57] lasting and biggest labs team in [music] [1:59] the world. And he shares a ton of very [2:01] specific lessons about where the best [2:03] ideas come from, how to know when to [2:05] quit an idea, what it looks like when [2:07] you have product market fit, what skills [2:09] are becoming more valuable in today's [2:11] era, why roles aren't actually blurring [2:13] into one builder role, and so much more, [2:15] including some spicy stuff. A huge thank [2:17] you to Steven Johnson, Sophie Miller, [2:19] Jeff Hong, and Logan Kilpatrick for [2:21] suggesting topics and questions for this [2:23] conversation. Before we get into it, [2:25] I've noticed a lot of people don't [2:26] finish the episodes, which is totally [2:28] fine, but you end up missing the best [2:30] stuff. [music] [2:31] So, to help you, I've created a new [2:33] Lenny's Most Replayed Moments YouTube [2:35] channel that is exactly that, the most [2:38] rewatched and shared segments from our [2:40] long- form episodes. Check it out and [2:42] subscribe at lennispodcast.com/mosre [2:45] replayed moments. There's also a link in [2:47] the description. With that, I bring you [2:49] Josh Woodward. [2:54] Josh, thank you so much for being here. [2:56] Welcome to the podcast. Thank you. So [2:58] great to be here. It's even better to [3:00] have you here. I want to first lay out [3:03] the premise for this conversation and I [3:04] want to get your take if you agree with [3:05] this. [3:06] >> Yeah. Yeah. Sounds good. [3:07] >> Worldview. So my thesis is that every [3:09] company is needs to start thinking and [3:12] operating like a labs team because the [3:16] underlying tech that we're all building [3:18] on is changing so fast. Which means what [3:20] we can do is changing so fast. Which [3:24] means that now everyone has to get [3:25] really good at trying a bunch of stuff [3:27] with the latest models and basically uh [3:30] figuring out what is now possible before [3:32] your competition does, before a startup [3:34] comes and eats your lunch. And so it [3:36] feels like the skills that will matter [3:38] more and more now are things that you [3:40] all have gotten really good at, which is [3:42] finding great ideas, figuring out how to [3:45] run a ton of experiments, figuring out [3:46] what to double down on, uh building [3:48] teams that can operate this way, at at [3:51] the same time not distracting the rest [3:52] of the company from all the shiny stuff [3:53] and letting them focus. So broadly, uh [3:56] how do you does that resonate? [3:58] >> Yeah, I I think your premise is spot on. [4:00] Um I think the devil's in the details [4:03] with all those things you just said. So [4:05] I think what's easy for you know in some [4:08] ways every startup is a lab and the real [4:10] question is like how do you bring some [4:12] of that same methodology ideas energy [4:15] excitement into bigger teams or even [4:18] medium-sized teams. Um but I absolutely [4:21] think it's needed and then the question [4:22] is yeah how do you build that [4:24] environment and nurture and cultivate [4:26] the kind of both ideas but also the [4:28] people uh that can really thrive there. [4:31] >> Awesome. Okay that's exactly what we're [4:32] going to talk about. [4:33] >> Yep. This episode is brought to you by [4:35] our season's presenting sponsor, Work [4:37] OS. What do OpenAI, Anthropic, Cursor, [4:41] Replet, Sierra, Clay, and hundreds of [4:43] other winning companies all have in [4:45] common? They are all powered by work OS. [4:48] If you're building a product for the [4:49] enterprise, you've felt the pain of [4:51] integrating single sign on, SKIM, [4:53] Arbback, audit, logs, and other features [4:55] required by large companies. work OS [4:58] turns those deal blockers into drop-in [5:00] APIs with a modern developer platform [5:02] built specifically for B2B SAS. [5:05] Literally every startup that I'm an [5:07] investor in that starts to expand [5:08] [music] up market ends up working with [5:10] Work OS. And that's because they are the [5:12] best. Whether you are a seedstage [5:14] startup trying to land your first [5:15] enterprise customer or a unicorn [5:17] expanding globally, work OS is the [5:19] fastest path to becoming enterprise [5:21] ready and unblocking growth. It's [5:23] essentially Stripe for enterprise [5:25] features. Visit works.com to get started [5:28] or just hit up their Slack where they [5:29] have actual engineers waiting to answer [5:31] your questions. Workos allows you to [5:34] build faster with delightful APIs, [5:36] comprehensive docs, and a smooth [5:37] developer experience. Go to works.com to [5:40] make your app enterprise ready today. [5:44] So, I'm going to start with coming up [5:45] with ideas, finding great ideas. If you [5:48] were to reflect back on the best ideas [5:51] that emerged out of labs and even in [5:53] Google in general, what are what are [5:55] some patterns of where the best ideas [5:57] come from? [5:58] >> Yeah, it's interesting because a lot of [6:00] them I wish there was an answer in the [6:02] back of the book where there was one [6:04] thing the maybe one observation Lenny is [6:07] there the they come from a variety of [6:09] places. So one lesson I've always felt [6:11] is like you always have to be watching [6:13] for something that catches your eye. [6:15] Maybe we come back to that in a second. [6:17] They very rarely, at least in my [6:18] experience, have come from design [6:20] sprints or times where you set aside on [6:22] a calendar, okay, this is where we're [6:24] going to go come up with our next idea. [6:26] They usually come when people are [6:28] swimming in the hallway, maybe on a [6:31] weekend, oh, often over a break, a lot [6:34] of our labs teams will be hacking and [6:36] building on stuff and they just can't [6:38] help themselves and they'll come back [6:40] and or they'll share a video and be [6:41] like, look at this amazing thing. Um, so [6:44] I think one lesson is like always on [6:46] alert. Second lesson is like they don't [6:48] come in conventional places. You can't [6:50] microwave good ideas. And maybe that [6:53] leads to a third thing I've seen is [6:55] sometimes it's about less about the idea [6:58] and who are the interesting people and [7:00] more importantly what are their [7:01] interesting little side projects and [7:03] passion things and like creating the [7:05] chemistry around that because usually [7:07] what we found at least in labs over the [7:09] last four or five years is that all [7:11] these ideas whether they're notebook or [7:14] flow or AI studio or Google beam or any [7:16] of this stuff they get started like very [7:18] small groups of people who are just [7:21] obsessed and usually very curious about [7:23] something and they just won't stop. Um, [7:26] and so they you kind of like how do you [7:28] kind of create a space where that can [7:30] happen and then kind of be on alert to [7:32] like catch it and try to like you know [7:34] funnel it funnel it forward. [7:36] >> So what I'm hearing here is [7:37] interestingly not like design sprints. [7:38] Uh I didn't hear brainstorm meetings in [7:40] there as a place good good ideas come [7:42] from. [7:43] >> Yeah. Um, is there an example maybe [7:45] because you know like someone listening [7:47] to this I'm I'm trying to think about [7:49] what can they take from this to find to [7:51] help them find great ideas say they're a [7:53] startup founder or at a big company [7:55] playing with the latest models. Um, what [7:57] are some like the conditions that you [7:59] find most help? [8:00] >> Yes. Yes. I mean one thing we try to do [8:02] a lot and I think this would apply in [8:04] any of those situations is like how do [8:06] you get yourself to the frontier and [8:09] that can be both with like playing with [8:11] the models but also just watching what [8:13] are the types of experiences that have [8:16] just become possible where we have a [8:17] phrase on the team we talk about what's [8:19] almost possible and we just have a list [8:21] actually [clears throat] where we're [8:23] trying to track those things and when [8:24] one crosses over it's almost like a [8:27] phase shift when something goes from [8:29] like solid to gas or liquid to gas or [8:31] something. You're like, "Oo, that just [8:33] became possible." I saw one of these two [8:36] days ago and I can't stop thinking about [8:38] it. [8:38] >> Are you able to tell us or it's a [8:40] >> uh maybe soon, [laughter] [8:43] but it's like you you kind of if you're [8:45] at the frontier playing with the models [8:47] watching what's happening and you kind [8:48] of have this list, then when you see [8:51] something, whether it's on X or [8:53] wherever, then you're like, "Okay, that [8:55] just now has become possible." Um, and [8:58] then we try to think about how do we get [8:59] a team around that and just go. Uh, and [9:02] so that may be as you're kind of in [9:03] startup land if you're thinking or maybe [9:05] you're starting to pitch ideas to VCs or [9:08] whatever. That's kind of how I'd be [9:09] trying to think about. And then of [9:10] course tech alone isn't enough, you've [9:12] got to kind of match that to an [9:14] interesting user problem. And ideally, [9:16] if you can put those together in a way [9:17] that's so good, people will pay for it. [9:20] That's kind of the combination of things [9:22] we start looking for uh pretty early. So [9:25] that's an interesting tactic there is [9:26] make a list of what is almost possible [9:30] and then watch for okay wow this is now [9:32] possible let's explore this now [9:34] >> yeah that's right [9:36] >> is there anything on that list you can [9:37] share of things that are not yet [9:39] possible but almost possible that you're [9:40] waiting for watching for [9:41] >> well maybe I I can share a few areas I [9:44] because I think how we think about it in [9:45] labs is we um [9:46] >> we have kind of a kind of a bit of a [9:48] thought experiment we go through where [9:50] we're like what is the future of a [9:52] certain thing going to look like so the [9:53] future of software design and [9:55] development or the future of creativity, [9:57] the future of work or knowledge. Um, one [10:00] area we're watching very close right now [10:02] is just like what is the future of [10:03] entertainment going to look like? And [10:05] this whole idea around immersive [10:08] entertainment is super interesting just [10:11] at a technical level for many reasons, [10:13] but also you think about just kind of [10:14] the user sort of behaviors and like what [10:17] does entertainment look like last year, [10:20] this year versus next year or maybe in [10:22] three years. Um, and so that's one where [10:24] we've got a whole list underneath that [10:26] where we're like, oo, okay, well, if a [10:28] model can get to this kind of latency [10:31] and this amount of languages in this [10:34] amount of modality, you you can start to [10:36] kind of think through what would be [10:38] interesting from the tech side. And then [10:40] we do a lot of work in labs around where [10:43] are just behaviors and trends going. And [10:45] for that, some of it, of course, you can [10:47] kind of study in the Bay Area, but for a [10:49] lot of our teams, I'm always thinking [10:51] about how do we get way outside the Bay [10:54] Area. Um, and thinking about where are [10:57] the other cities, the classrooms, the [11:00] the other places to kind of like like [11:01] almost like mine for ideas and these [11:03] behavioral changes. [11:05] >> So, an interesting takeaway here is [11:07] this. You're not just waiting for [11:08] inspiration to strike. You're doing a [11:10] lot of pre-work to think about here's [11:12] what is almost working. Here's where we [11:14] think the future might be going. Here's [11:16] areas that are going to be ripe for [11:18] disruption. [11:19] >> That's right. Yeah. In this doc I talked [11:21] about, we have um 82 predictions that [11:23] all begin with we believe the future is [11:27] or we predict. Um this is dangerous ever [11:30] to make one prediction, let alone 82. [11:32] So, but I think you have to almost have [11:34] a point of view about the future knowing [11:36] that nine out of 10 or maybe 9.9 out of [11:39] 10 you're going to be wrong. But I think [11:41] that's one of the things we really try [11:42] to cultivate in the team is how do we [11:44] chase the future? That's one of the [11:46] values and sort of trying to chase it. [11:48] You almost have to throw yourself out [11:50] there and try to live in it a little [11:51] bit. [11:52] >> What are signs to you that an idea is [11:54] worth pursuing when you're in the [11:55] shower, you have this idea. What's a [11:57] sign that okay, that's worth betting on, [11:58] at least for to start with. And what's [12:00] often just like a false false signal [12:02] like okay, no, that's actually not going [12:04] to work. [12:04] >> Yeah. I think one thing for me, [12:07] especially when you've got these lists [12:09] going and you're thinking about the [12:10] future, when you see something that [12:12] surprises you or maybe that's very [12:15] interesting to you in a certain way and [12:17] if you can describe it to yourself like [12:19] this thing I saw about 48 hours ago, I [12:23] was I left that sort of demo and I [12:25] thought this is so exciting. [laughter] [12:29] Um, and I think that's maybe a signal I [12:32] use a little bit is like trying to be at [12:34] the frontier and then what makes me [12:36] interested or excited and then feeding [12:38] off that kind of with the team together. [12:40] I feel like that's too that bar is too [12:42] low these days because everything is so [12:43] magical and like or it's the opposite [12:46] now everything okay great this thing is [12:47] doing my taxes for me of course that's [12:49] just the way [12:50] >> yeah yeah [12:51] >> I think this is one of the things [12:52] obviously that excitement is maybe a [12:54] tech thing and yeah there's dazzling [12:56] demos every other day or every other [12:58] hour right now but then I think trying [13:00] to think through where's the user [13:02] behavior maybe that is like universal or [13:05] maybe there is some shift on that side [13:07] and then the way we think about [13:08] monetization too is like can you get to [13:11] a group of people that are just going to [13:13] like can't live without it like they [13:14] literally will pay you because it's so [13:16] valuable to them and then kind of how [13:18] you can kind of put those pieces [13:20] together. But yeah, sometimes it's got [13:22] to come from one of those areas I would [13:23] say. [13:24] >> Is there a great example or story of a [13:26] product that emerged in this way? [13:28] >> Yeah, I mean the fir I remember still [13:30] the first time I heard the first like [13:33] notebook LM podcast exactly where I was [13:35] standing it was right outside this room. [13:37] Um, and it was probably at like 6:30 [13:40] p.m. on a Tuesday or something and there [13:43] were two people on the team who came [13:44] over and were like, "You got to listen [13:45] to this." And it was the proceedings of [13:50] the British Parliament debates. Um, not [13:53] known for its riveting material, but [13:55] these two AI hosts were talking about it [13:57] in a way that was was literally [13:59] interesting. And that that was one where [14:02] we're like, we got to just do something [14:03] with this. And that was one. I remember [14:05] the first time when I saw something come [14:08] out of Google Flow. We had a project [14:10] before that called Whisk where you could [14:12] whisk together multiple images and then [14:14] start to animate them. And it was a [14:17] level of control and creative kind of [14:20] channeling that had never been possible [14:23] that I kind of made it was like that [14:24] accessible. Um so maybe those are two [14:27] examples. We also have other examples [14:28] where we get excited, we try to build [14:30] some product, it's just not good. So, [14:33] um, you know, that's the other thing is [14:34] a little bit of like a game of numbers [14:36] in this as well. [14:37] >> Let's talk about product market fit and [14:39] signs that something's actually working. [14:41] I feel like you've seen more ideas [14:44] thrown at the wall and tried and [14:45] prototype than maybe anyone else. I [14:48] don't know if that's true, but it feels [14:49] true. What are what are signs to you [14:52] that this is something we need to double [14:54] down on? And what's what's something [14:55] that might surprise people about what [14:57] about what product market fit looks [14:59] like? Yeah, I think one of the maybe [15:02] trickiest parts of it is it is way more [15:05] art than science. I wish it was science. [15:08] like we know how to run science [15:10] experiments, make datadriven decisions, [15:12] set OKRs, like all this process that [15:15] helps like um sometimes in the early [15:18] days, Lenny, it's literally a couple of [15:21] people on the team who might just have [15:23] conviction and they're just like, I [15:25] think the future is going to look like [15:27] this and I don't I can't explain it. [15:29] That's what's hard to like that advice [15:31] is just not that useful or helpful. But [15:34] I would say that's some of it. I think [15:36] maybe a little more useful. Uh we talk a [15:39] lot about how do we get outside the [15:41] building and just show people [15:42] prototypes. And when you're showing [15:44] people early prototypes, what I'm always [15:47] doing in those is you're looking at [15:49] people's eyes and like that is the [15:52] metric like uh it is not a Dow is not a [15:55] Dow over a MAU. It's not a D7 retent. [15:57] It's none of that. Um it's literally [15:59] like hey like Lenny did you see this? Uh [16:02] what do you think about this? And like [16:04] if your eyes kind of light up, if you [16:06] lean in, these are the kind of signals [16:08] you're looking for at the early days. [16:10] Maybe the third thing I would say, and [16:12] this one's also maybe on the flip side [16:14] of your question, is like [16:17] the people who get excited about things, [16:19] usually we try to kind of think about [16:22] what's the problem we're falling in love [16:23] with, not the product. Because what I [16:26] found is it takes three, four, five [16:29] pivots before you either got something [16:32] or you're just like that's it, didn't [16:34] work. Um, so I think when people fall in [16:37] love with an actual product solution [16:39] first, it's going to, at least in my [16:41] experience, lead to a bad time. Uh, so [16:43] that's the other thing I think we try to [16:44] think a lot about. And usually people [16:46] talk about PMF, you sort of know it when [16:49] you have it. I think that's true. Um, in [16:52] the last, I don't know, year or so, when [16:54] Nana Banana went viral on Gemini, when [16:57] the Notebook LM podcasts have gone, when [17:00] stuff's happened on Flow or Stitch or [17:02] other kind of projects that have kind of [17:04] had these mini moments, um, you're [17:07] literally just swarming it and holding [17:09] on for dear life. And it is both like so [17:12] exhausting and so fun. Um, but yeah, you [17:16] you almost have to take a lot of shots, [17:17] I would say. And in the early days again [17:19] it's it feels a little more art passion [17:22] driven than than science unfortunately. [17:25] >> Those are actually very concrete [17:26] specific things to look for. Uh the eyes [17:30] what eyes light up. Uh a way I've heard [17:32] someone else describe this is their [17:34] pupils dilate. [17:35] >> Yes. [17:37] That's right. And you know too right [17:38] when you're talking to someone if [17:40] they're having a good time, if they're [17:41] into it or if they're just bored out of [17:42] their mind and you're you know showing [17:44] them some dumb product. So it's it's [17:46] really like can you hit a nerve [17:48] >> with like a pain point in their life [17:50] where they're just like I want that. And [17:52] I mean that's what that's what makes [17:53] this job so fun is trying to figure out [17:55] how how many of those nerves you can [17:57] hit. [17:58] >> So having that context of it's clear [18:01] when things are product market fitting [18:03] and there's eyes are lighting up. What [18:06] are signs that it's time to kill an [18:08] idea? [18:09] >> Yeah. One of the things on this one and [18:11] I talked a lot to the team about this [18:12] too is sometimes a team is looking for [18:16] hey the leader Nita makes makes the [18:18] decision and that's kind of where it [18:19] comes from. What I found is the [18:20] opposite. The team usually knows before [18:23] the leader knows and the signs are like [18:26] does the passion start to run out? Does [18:29] you've tried everything you can think of [18:31] and it just it's just not working. like [18:33] and people know when like a project's [18:36] not working or a relationship's not [18:37] working like you know that's something [18:40] that people see and so I think what as a [18:42] leader what I try to do say in labs or [18:45] even some of our features in Gemini is [18:47] like how do you create an environment [18:49] where people can ask those uncomfortable [18:51] questions and be able to say like hey I [18:52] don't think this is working and [18:54] yesterday just as an example we had a [18:56] feature we were really excited about in [18:57] Gemini we were going to launch and we [18:59] got some of the early data back people [19:01] went out looked in the eyes and it just [19:04] wasn't there. And so I was so proud of [19:06] the PM on the team because she was like, [19:09] I don't think we should launch this. [19:10] It's not good. Um, and so I replied all [19:13] was like, "This is great. Thank you for [19:14] doing this." Uh, and so it's like, how [19:16] do you create a culture where the team [19:19] can call both like good things and bad [19:22] things out? And I guess in that way, as [19:24] a leader, in my case, I I trust the team [19:26] a lot. Um, because they're the ones [19:28] every day trying to make this thing go. [19:30] And I think if they're like, "Hey, it's [19:32] not going." Then it's like, "All right, [19:33] that's okay. Let's pivot. Let's do [19:34] something different." [19:35] >> Uh, I have to ask you this question. I [19:37] was gonna ask it later, but I'm so [19:38] curious. It's so interesting that you [19:40] oversee labs, which is 0ero to1 [19:43] experiments, try stuff that's totally [19:45] new. And then Gemini, the Gemini app, [19:47] which is one of the most popular apps in [19:49] the world with probably a billion users. [19:51] Uh, why how did that come to be? Why is [19:54] that the structure under your team? [19:55] >> Yeah. Yeah. Well, I guess in some ways [19:58] they are very different ends of the [19:59] scale. You're right, Gemini. Over a [20:01] billion users. It's global. Like [20:04] >> labs, you know, we joke on labs. [20:06] Sometimes our early projects, the zero [20:08] to one stuff, you know, we we'll get so [20:10] excited about 10,000 mouse and like most [20:13] other products dashboards don't count [20:15] that low. So this is kind of the joke on [20:17] the team. But I think maybe there's a [20:19] few common pieces though, even though [20:21] the scale is different. I think one the [20:24] type of people that are attracted to [20:26] these things even Gemini despite its [20:29] size is very much got a zero to one [20:32] heartbeat um all over it and so I think [20:35] part of it is like the type of builder [20:37] the type of person you're attracting [20:38] these like blurring lines between what [20:40] what it takes two we're all like AI kind [20:44] of nuts and maniacs we love this stuff [20:46] like we we we can't get enough of it [20:49] we're building all the time um the third [20:51] one though I would say is like a little [20:53] bit of everyone on the team. We talk a [20:56] little bit about like the order of how [20:58] we think about our priorities and we [21:00] like the mantra on the team is like [21:02] users first, Google second and like our [21:05] product third and so like labs third or [21:07] Gemini third and I think that unites us [21:10] too and that's like we don't try to do [21:12] things like for our own team we're very [21:15] much part of team Google but also [21:17] ultimately it's like is someone loving [21:19] this on the end other end of this [21:20] product? Um, and so that does kind of [21:23] tie it all together, but they are on [21:25] Yeah. different ends of the spectrum for [21:26] sure on terms of size like conventional [21:28] numbers. [21:29] >> Yeah, it's quite confusing as an [21:30] outsider, but I get it. Um, is is [21:33] everything you're doing in labs now AI [21:36] based? Is there anything non AI driven [21:39] in labs now? [21:40] >> Yeah, it's um it's all AI driven right [21:42] now for sure. when we started, you know, [21:44] four or five years ago, we had a few [21:46] other areas we were exploring and we [21:47] kind of made the call, at least for the [21:49] foreseeable future, go allin um on AI. [21:52] We we don't just do software stuff [21:55] though. I mean, Google Beam is one of [21:57] the projects we're working on is like [21:58] this incredible kind of magic mirror [22:01] that kind of can create like a 3D [22:02] version. We should get you one. Um I'll [22:04] take where you can kind of talk to [22:06] someone almost like they're a hologram [22:08] on the other side. Yeah. Uh so we are [22:10] we're exploring kind of other adjacent [22:12] things to AI like you know you could [22:14] consider that almost like AI hardware in [22:16] a way. I I will take it. Thank you. I'll [22:19] send you my address. [laughter] [22:22] So it feels like consumer right now is [22:24] in a really fun place where there's [22:26] always been the sense of like consumer's [22:27] dead. There's nothing you can do. [22:29] Facebook [22:30] got Google, you know, like there there's [22:32] no way to compete against these [22:33] juggernauts that have taken the consumer [22:35] world. But AI feels like it's unlocking [22:37] a lot of opportunity in consumer. I'm [22:40] curious what your hypothesis is for [22:41] where the next where a major [22:43] breakthrough might happen in a consumer [22:45] app. [22:45] >> Yeah, I I feel like consumer is one of [22:49] the most like exciting competitive all [22:51] the things happening right now. So it [22:53] feels completely opposite to me uh from [22:55] that kind of view. Um a couple of areas [22:59] I think that are kind of interesting. I [23:00] mean, one, I think there's a big [23:02] question, we talked a little bit earlier [23:04] of just like where's entertainment going [23:06] to go? Um, are we going to be looking at [23:10] like different types of feeds forever or [23:13] are there ways that this technology can [23:16] bring us maybe closer together in the [23:18] real world or do things with other [23:19] people? I mean, there's different views [23:21] on where that could go. Is it like more [23:22] around bots or more around people? Um, [23:25] and so I think that's a very interesting [23:27] space. Um I think we're watching right [23:30] before us now like the entire kind of [23:33] messaging chat space felt like hey that [23:36] was sort of maybe solved. There's lots [23:39] of different experiments happening on [23:41] that right now both inside Google and [23:42] outside. Um so that feels like a whole [23:44] another area that maybe is like no [23:46] longer as fixed as it once felt. Um, and [23:49] then maybe the third one I would throw [23:50] out is I still feel like going back [23:52] again to the problems like what are some [23:55] of the big problems people face in the [23:58] world? And I think like um you know what [24:00] are things that are super scarce? Time [24:02] on earth, how much money people have and [24:06] how they can save money. Um these kind [24:09] of memories or like experiences that [24:11] people can have kind of IRL together and [24:14] that kind of thing. So I do feel like [24:16] there's maybe like new consumer products [24:18] to be built uh that can go after a lot [24:21] of that stuff that you know whether it [24:23] takes the form factor of a a chatbot or [24:26] a personal agent or whatever that's [24:28] going to look like I think is kind of [24:29] yet to be invented in a lot of ways. [24:31] >> Okay. Let me ask you a spicy question [24:33] along these lines. I'm so curious to get [24:35] your take. So, it feels like every every [24:39] uh AI assistant app that's launching [24:41] these days basically sucks in Gmail and [24:44] Google Calendar and Docs, all the Google [24:47] data, all this juicy Google stuff. Uh [24:50] connects and does all these magical [24:52] things with it within Google. I cannot [24:54] do any of these things. It's so hard to [24:56] build anything uh like there's first of [24:59] all, you may be working on AI assistant, [25:01] but there's nothing there yet as far as [25:03] I can see. And then just generally it [25:04] feels like everyone's just taken all [25:05] this juicy data and doing magical things [25:07] with it and Google hasn't. What is [25:10] what's kept you guys from doing [25:12] something like that? Is like privacy [25:14] concerns? Is it works bureaucracy stuff? [25:17] Is it coming? What what can you say? [25:19] >> Yeah. Yeah. Yeah. So um there's parts of [25:21] it out there today like if you're in [25:23] Gemini in Gemini Spark, you can actually [25:25] connect all that stuff and it's quite [25:28] powerful. I think where we're iterating [25:29] on that though is [25:32] we've been in a world where even if you [25:35] look at some of the other competitors [25:37] there different modes almost that you go [25:40] into and I think the lesson even from [25:42] the last couple of weeks is forget the [25:44] modes and toggles just like one prompt [25:47] box tell it what you want and it just [25:49] does stuff for you. So that that's [25:51] definitely where not only where we're [25:52] headed where others are are headed. I [25:55] think that's kind of a broader almost [25:56] like UX simplification happening across [25:59] the industry. Um on the the data bits [26:02] there, um it's interesting because [26:05] actually one of the main things uh [26:08] people will ping me about or others is [26:10] like, "Oh man, Google, I've got all my [26:12] stuff with you. Just make the thing [26:14] work." Um and there's a lot of [26:16] reluctance actually to be connecting it [26:18] to a lot of other sort of competitor [26:21] products. So watch this space. we hear [26:23] you loud and clear. Um, there's a lot of [26:26] work if you go online and just look [26:28] under kind of personal intelligence is [26:30] sort of the way we think about this [26:32] stuff is like you want a Gemini that is [26:36] personal, proactive, powerful um that [26:39] sort of works for you. And so that [26:41] that's been the vision uh for the last [26:43] kind of year or so uh that we're we're [26:46] headed towards. And there's a lot of [26:48] stuff we're playing with right now [26:49] that'll be will be out soon. [26:51] >> All right. I'm excited. It just feels [26:52] like yeah like I have so many little [26:54] schedules and things running within my [26:57] email. I'm just like man why why didn't [26:58] Google just doing this? Uh it does does [27:01] feel that the advantage you all have is [27:02] distribution. So once something happens [27:05] similar to Muse recently it's going to [27:07] be very easy to grow it and convince [27:09] people to use it similar uh to you know [27:12] within like AI mode. So anyway okay I'm [27:14] excited. Things are happening. [27:16] [laughter] Things are definitely [27:17] happening. [27:19] What do you think? So kind of going a [27:20] different direction. What do you think's [27:22] uh underhyped in AI right now and what [27:25] do you think is overhyped if anything is [27:26] overhyped? [27:28] >> Oh yeah. Yeah. Um you know underhyped I [27:31] I was thinking a lot about this over the [27:33] summer. I ended up writing kind of a doc [27:35] internally just around like labs [27:37] principles of like how and almost like [27:41] why are we building what we build? And [27:43] so maybe the underhyped thing is more of [27:45] a discussion around principles, values, [27:49] sort of I I believe very strongly [27:52] whether you're at Google or a startup [27:54] founder or anywhere in between, you you [27:56] speak through your products and your [27:58] products actually represent and almost [28:00] encode those principles. And so I wish [28:03] there was a lot more discussion in the [28:04] industry happening around that. Um and [28:07] sort of what what kinds of things do we [28:09] want to build? what kind of future do we [28:12] want? Uh so anyway, that would be my [28:14] underhyped one. Um because I think uh [28:17] yeah, there could be more of that. I [28:18] think the overhyped one um I think I've [28:21] been saying this maybe for a year now, [28:23] but I still think it's overhyped. Model [28:25] benchmarks. [28:28] And I still uh you know, a lot of [28:30] friends and a lot of labs and everyone's [28:32] still looking at the same set of [28:33] benchmarks for the most part. Um, but I [28:36] think um, you know, where I'm grew up [28:38] and where I'm from and I guess how I [28:40] think about product development is like [28:42] most people in the world are never ever [28:44] ever going to care about an ELO rating. [28:47] They may not even know how to pronounce [28:48] ELO. It's like, do you know what? Like [28:50] it just doesn't matter. And so that's [28:52] where I kind of get back to kind of like [28:54] a lot of I feel like the talk in AI [28:56] right now is about speeds and feeds and [28:58] benchmarks and these are proxies [29:01] obviously for kind of where things are [29:03] and capabilities and but I still feel [29:05] like sometimes as an industry it can [29:08] kind of over over rotate to that and [29:10] kind of lose sight of like why is this [29:12] useful to someone and what's like the [29:14] product or the feature that's going to [29:16] deliver like real value to like help [29:18] people. So anyway, those would be my [29:20] overunders. [29:21] >> Yeah. And it's also along those lines. [29:22] It's always it's also hard to know what [29:25] what what is better. Like how do you [29:27] know what what's going on? Like you have [29:28] this new model. It's the best model [29:30] ever. It's like in practice it's not [29:32] that much different than the previous [29:33] model that also everyone thought was the [29:35] best model ever. We have AGI every every [29:37] time there's a model launch. We have AGI [29:38] now. And so like the benchmarks like are [29:41] trying to show you here's where it's [29:42] better. But but if you didn't like [29:44] you're just sitting there. Okay. This is [29:46] okay. This is better. I like what it's [29:47] doing here. Yeah, I think this is also [29:50] though where um I feel like a lot of [29:52] folks that like follow you and listen to [29:54] you is like please please please think [29:56] of amazing products like because [29:58] ultimately all these model capabilities [30:00] and benchmarks and all the intelligence [30:02] gets translated into just like can you [30:04] build something people want and so to me [30:06] it's like what a time to be alive as a [30:08] product person um because you have kind [30:10] of this like huge menu of things that [30:13] were almost possible that are now [30:15] possible more and more with each passing [30:17] week. The first uh point you made there [30:19] was u something that came up recently [30:21] with a uh in a conversation with Tara [30:23] from OpenAI about how now that we could [30:26] just build everything and AI has so many [30:28] opinions about where to go. What you [30:30] said is exactly right which is what do [30:32] we want the world to look like? What do [30:34] we want the what is the vision of the [30:36] future we want not okay we can do all [30:39] these things that's great let's maybe we [30:40] don't need to do all these things [30:42] >> right [30:42] >> so I think that's a really important [30:44] point there. Yeah, definitely. [30:46] Definitely. [30:46] >> This season's supporting sponsor is DX. [30:49] Now that everybody at your company is [30:52] finally using AI, the question that [30:54] engineering leaders face is no longer [30:56] are we using AI, but is it paying off? [30:59] With DX, you can measure and benchmark [31:01] the impact [music] of AI adoption across [31:04] your full product life cycle. DX shows [31:06] you where AI is helping developers [31:08] [music] and where it's creating new [31:10] bottlenecks. You can also use it to [31:11] evaluate AI vendors, monitor costs and [31:14] licenses, and see exactly what's [31:16] limiting your agents. That's why [31:19] hundreds of enterprises, including [31:21] Snowflake, Sony, and BNY, use the DX [31:24] platform to measure AI's impact on [31:26] developer productivity. Visit [31:28] getdx.com/lenny [31:31] to get a demo of DX. That's [31:34] getddx.com/lenny. [31:36] What skills are you finding are trending [31:40] up in value in the people that you're [31:43] working with and what do you feel are [31:44] trending down when you look at people [31:46] are doing really well now? What are the [31:47] kind of the attributes and skills of [31:49] those people versus things that are [31:51] maybe less important now? [31:53] >> Yeah, I feel like there's a whole lot [31:55] out there on taste and judgment and vibe [31:58] coding and all of that. I guess the [32:00] things I'm looking for I'm always trying [32:01] to find like what are the mispriced [32:02] signals and so like one I think is um [32:07] how we'll hear sometimes people talk [32:10] about like what's someone's like [32:11] learning rate like do they have a growth [32:13] mindset what's their learning rate I'm [32:15] actually looking a lot right now for [32:16] people what's their unlearning rate like [32:19] how fast can they learn something and [32:22] then walk away from it um based on maybe [32:24] what they're seeing when they test their [32:26] product or something. unlearning rate is [32:27] one. Um, another one I'm interested in, [32:31] I read a book a year or so ago, um, [32:34] about Roger Federer, the amazing tennis [32:36] player, and there was this phrase in [32:38] there about Federer had explosive [32:42] endurance. It's just this beautiful turn [32:44] of phrase and uh with the alliteration [32:46] everything but it's like there there's [32:48] something around like how are people [32:51] taking care of themselves and the people [32:53] around them where they can go really [32:56] hard and that's the like explosive part [32:59] but they can do it over a while and they [33:01] find ways to kind of pace themselves and [33:03] labs we have kind of you know we have [33:06] like seasons of labs where we talk about [33:08] this so I think that explosive endurance [33:10] this unlearning rate Um, and then maybe [33:13] another one I would say is like as much [33:16] as you know coding and other things are [33:19] becoming maybe trending towards like you [33:22] being able to direct lots of agents and [33:24] not having to work with as many people. [33:27] I still feel like collaboration and like [33:30] how you work with people matters even [33:33] more now. And um one of the things we [33:36] look a lot for is like can someone kind [33:39] of blend and bridge these worlds where [33:42] they're like incredible collaborator [33:44] with an agent or a swarm of agents but [33:47] also with people. And I guess what we [33:49] believe on the labs team and on the [33:50] Gemini team is that like there's still [33:53] just so much joy in working with people. [33:55] And so like but it's like how do you [33:56] find the right chemistry? And so maybe [33:58] one thing I'd say there is like people [34:00] that can build trust, that can scale [34:03] trust, that that underpins any kind of [34:06] great collaboration. And so that's [34:08] another maybe signal I'm always looking [34:10] for is like how fast can this person [34:13] kind of build trust with people uh and [34:15] sort of create that kind of environment [34:17] if that makes sense. [34:18] >> Yeah, these are so interesting. on that [34:21] last one is that is that kind of a mer a [34:23] growing emerging even more important [34:25] skill you finding with rise of AI and [34:27] I'm curious why [34:28] >> I think so for a couple of reasons I [34:30] mean I would say one maybe is like [34:32] uniquely Google it's a larger [34:34] organization so you've got to be able to [34:35] kind of do that but the ones that don't [34:36] feel uniquely Google um one it feels [34:40] like as the execution cost of making [34:42] products goes down we'll be able to make [34:44] more products which you may make the [34:47] same products with the same people but I [34:49] think there's a lot more you probably [34:50] see this too is just like uh people will [34:53] team up with different people to make [34:54] stuff. And so it's very interesting when [34:56] we're thinking about like almost like [34:58] fielding a new team for a labs project. [35:01] We used to say like it's about five to [35:03] seven people. Now it's about two to [35:06] three people. Uh maybe four. Uh but it's [35:09] kind of like those people just kind of [35:11] almost you can almost imagine like super [35:14] talented musicians who team up with [35:17] different people and then they play [35:18] different songs together. The songs of [35:20] the products but it's like that is like [35:23] those are the MVPs in labs where you can [35:25] be like hey drop this sort of PM but [35:28] that can also code and design because [35:31] all the roles are blurring with kind of [35:33] other people and kind of see how they [35:34] jam on stuff together. So I think that's [35:36] where my prediction is that skill will [35:38] almost increase in importance uh because [35:41] there will be more products and because [35:42] there'll be more of this kind of like [35:44] free agents kind of jamming on stuff [35:45] together. [35:46] >> Along those lines I imagine you have [35:49] been pulling designers, engineers, npms [35:52] into labs and they kind of all kind of [35:54] work together and sometimes different [35:56] functions take leads. Is there has there [35:58] been a shift in who you find is most uh [36:01] impactful and successful in leading [36:05] product ideas labs teams in terms of the [36:08] ratio of PM versus design for NGE for [36:10] example like our PM's becoming more [36:12] valuable or becoming more valuable [36:14] something like that? [36:14] >> Yeah, it's interesting. We've been [36:17] studying this a lot and actually testing [36:18] a lot of this too. Um, I would say in [36:21] general, PM happens to be the function [36:25] maybe by nature because it's a bit more [36:26] horizontal. That feels like it's maybe [36:29] been able to adapt to some of the other [36:30] functions faster. Uh, and we see this [36:33] when we'll run little builder pilots [36:35] across the company. Um, that comes back [36:38] is like, oh yeah, that that's like maybe [36:40] I don't want to say it's easier, but [36:42] maybe it's PMs are more accustomed to [36:44] like, all right, I'm in this meeting. I [36:46] got to simulate something as a lawyer or [36:48] as a BD person or as a sales, do you [36:50] know what I mean? So like I think in [36:51] some way maybe there's some inherentness [36:53] and like who selects into that. So that [36:55] definitely is true. But I would say um [36:58] our best people can kind of come from [37:00] any function. And though I would also [37:02] even widen it, Lenny, we're seeing like [37:05] strategy and ops people and BD people [37:08] and marketing people, you know, kind of [37:10] be like builders in labs and they are [37:12] really good and u I think what maybe [37:16] this is another overhyped underhyped [37:17] thing. I think right now um people are [37:21] maybe overhyping like everybody's a [37:23] builder and it's just this new job [37:24] function and everything's kind of the [37:26] same. I actually feel like well at least [37:28] what we're seeing is that people still [37:31] have their specialty. It's like their [37:33] major or minor in college or something. [37:35] And I feel like there's a risk if [37:38] everybody becomes a builder that you [37:40] lose a lot of respect and appreciation [37:43] and obviously the expertise that comes [37:45] from those specialties. And so I think [37:47] when we set up teams, even though we may [37:49] be like this is more of like a builder [37:51] crew, we're still thinking about like [37:53] where are the specialties and that ends [37:56] up being super important as well. [37:58] >> I completely agree. You know, like Yeah, [38:00] I like that. I like this metaphor of the [38:01] major and minor. That's a really fun way [38:03] to think about it. Like you can be [38:05] better at building, but you're still [38:07] your like your PM skills are still like [38:08] the primary thing you're amazing at. [38:10] >> Definitely. And I think that's the [38:11] advice we give especially to a lot of [38:13] new PMs on the teams like APMs or people [38:15] earlier in their career is it's great [38:17] you're vi coding and you're making all [38:19] these prototypes but do that in service [38:21] of becoming an exceptional extraordinary [38:24] PM and like don't lose that major like [38:27] that's what you need to hone uh [38:28] especially when you're early on. [38:30] >> I love the way you described it where [38:31] PMs are really good at simulating other [38:34] functions [38:35] >> such as like an AI way of thinking about [38:37] >> Yeah. Yeah. collaborating [38:39] >> and this idea of explosive endurance. Uh [38:41] someone tweeted this recently. They're [38:43] like what's what's so called when it's [38:45] both a marathon and a sprint. [38:47] >> It's [laughter] that [38:49] like talk to Roger Federer. Yeah. Yeah. [38:51] >> And I feel like everyone's in that right [38:52] now. That's just like a constant theme [38:54] and trend on this podcast and just in [38:56] tech right now is just everyone's in [38:57] this marathon plus sprint mode. [38:59] >> Yes. Yeah. [39:00] >> Man, I guess is there anything that you [39:02] do to help people deal with that because [39:04] it feels like it's non-stop. [39:06] >> Yeah. Yeah, I mean one of the things we [39:08] try to do and I mentioned it a little [39:09] bit earlier but I can unpack it some is [39:11] like we think of kind of like seasons in [39:13] labs and like can we think about you [39:16] know at any given point there's 20 or 30 [39:19] experiment you know running [39:20] simultaneously but for each individual [39:23] little team trying to kind of create a [39:26] little bit of like milestones or like a [39:28] rhythm to the work and there's sometimes [39:30] like it is just flat out fullon and we [39:33] try to maybe the best lesson I've [39:35] learned as a leader is almost to sort of [39:38] name that, declare that, you know, [39:40] people opt in or opt out of that. Um, [39:42] but also be very clear when a team is [39:44] not in that mode. And I think is like [39:46] how do you try to architect almost like [39:48] a a rhythm to the year where it's like, [39:51] hey, we've got a big model coming. These [39:54] five products are super important to [39:56] showcase this model. We think this model [39:57] will propel them. Like U5 is like it is [40:01] explosive time. Do you know what I mean? [40:03] Um and so it's like trying to create a [40:05] common language around that which is [40:07] sometimes [40:08] in a lot of ways like PMs sometimes can [40:10] be very effective if they can come up [40:12] with like what is the sort of shorthand [40:14] or the common language and so we we [40:16] actually put a lot of thought into that [40:17] too of like all right you all are on [40:20] right now whereas after they go through [40:22] that we may be like say after like a [40:25] Google IO a lot of labs teams will [40:27] sprint after IO June is like hey just go [40:30] hack go rediscover what's new go just go [40:34] build like have fun and then it's so [40:36] interesting Lenny because the instinct [40:38] or the like temptation is like oh man [40:40] it's June it's like unproductive we're [40:42] not shipping and it's like no actually [40:45] those are where all the new seeds are [40:47] coming from for like the next season do [40:49] you know what I mean and so it's like [40:51] trying to think through the rhythm of [40:52] that is is one way we try to do it I [40:54] don't think we always get it right [40:56] though by the way I think and I think [40:58] one of the things we're talking a lot on [41:00] the team is I guess in the past people [41:02] would talk about like work life balance [41:04] and there were other ways this [41:05] conversation got framed. I think what [41:07] we're why I like things like explosive [41:09] endurance or concepts like that is like [41:12] we want to be doing this as a team for [41:14] many many years. And so it's like you [41:16] can't just like empty the tank in 9 [41:18] months and just like kill yourself, [41:20] right? So it's like you have to you have [41:22] to have a view towards the long term but [41:24] also recognize the moment you're in is [41:25] is fast. [41:27] >> I love the tactic of just naming it, [41:28] having a lot of power and just making it [41:30] clear this is the mode we're in now. [41:31] won't be forever. [41:32] >> Right. Right. [41:33] >> And something I found is the opposite [41:35] also is too is dangerous. When you have [41:37] too little to do and too little stress [41:39] and too little pedal on the metal, [41:41] people get bored and start looking for [41:42] other jobs. They're just like this [41:44] sucks. [41:45] >> Yeah. Yeah. Yeah. I think it's um [41:47] there's always something there though. [41:49] You're right. of like we we talk on the [41:51] team a little bit of like never waste an [41:53] event or like a date or you you can [41:56] think about even as a startup like [41:57] where's that moment you want to [41:58] intersect with something in the world [42:01] and um you know whether that for us is a [42:03] Google IO or a model release or whatever [42:06] I think that's always healthy to be [42:07] thinking about I got a question the [42:09] other day on Gemini they're like what [42:11] what's our two-year roadmap and I was [42:13] like uh we have a six-month road mapap [42:15] on this thing like we have a multi-year [42:17] vision but I think they the the world [42:20] has changed so much too. Some of it is [42:21] like the thing I saw two days ago is now [42:23] a project that happened in 48 hours. You [42:26] you can't plan for that in a way and so [42:29] that's the other element you almost have [42:30] to recognize in the moment. [42:32] >> Yeah. So along those lines, how has [42:33] planning changed in the past couple [42:34] years? Like you guys are different from [42:36] the rest of Google because everything is [42:37] new and but just what have you seen in [42:39] terms of timelines and time frames shift [42:42] in planning with the rise of AI? Yeah, [42:44] we usually are thinking, you know, [42:46] somewhere on the order of about six [42:47] months. And usually I say that because [42:50] that's that almost possible window [42:52] where, you know, you assume every lab is [42:54] cranking out some new pre pre-training [42:56] model a couple times a year. There's [42:58] like individual points in between. So [43:00] you kind of have an overall cadence of [43:02] like where the models are and then I [43:03] think we're almost imagining in about a [43:06] hundred days usually we always think [43:08] about where can you move your product [43:10] like meaningfully forward. um our best [43:13] labs team all they all do that they'll [43:15] go from like idea to some meaningful [43:17] milestone in like 50 to 100 days and [43:21] part of that is also like how do you [43:22] design a culture and an environment [43:24] where you unblock a lot of stuff so you [43:27] can just let people cook and so that's [43:29] the that's the other part of is like [43:30] kind of creating that environment but [43:32] that we think about planning is kind of [43:34] almost like rolling six months maybe is [43:36] to answer your question [43:38] >> so I had this uh Lenny and friends [43:39] summit it was uh last week of the today [43:42] we're recording this and Dan Shipper [43:43] gave this really interesting talk about [43:45] how to build where the tech is just [43:47] constantly changing underneath you and [43:49] his advice is to have a frontier team [43:51] that's dedicated to testing out the new [43:53] stuff and basically it's their job try [43:55] all the new stuff report on what we need [43:58] to pay attention to so that the rest of [43:59] the company can not have to be [44:01] distracted and focus on on you know the [44:04] task at hand how do you feel about that [44:06] as a model for companies [44:08] >> I think every company definitely needs a [44:11] set of people, a team, however you want [44:12] to manifest it, that is their job is to [44:16] be at the frontier and they're almost [44:18] like if they haven't heard about the [44:20] latest hot thing viral on X in the last [44:23] 24 hours is like what are you doing at [44:25] your job? You know what I mean? Like so [44:26] I think there's definitely an element of [44:28] like you want people plugged in to like [44:30] the hive mind of kind of how people are [44:32] building and thinking in all the model [44:34] releases. Um they're the people who have [44:36] like eight $200 a month subscriptions on [44:39] everything. Do you know what I like so [44:41] you have to kind of architect the [44:44] organization where that can exist. I [44:46] think the second thing though that's [44:47] just as important is the environment of [44:49] which those people can build. And so [44:51] that's like the other maybe like part of [44:54] that. And then I think the third part [44:56] which is honestly where if you look at [44:57] the history and I've I've read a bunch [44:59] of books on this because most like if [45:03] you just look at the data a large [45:05] company who starts a lab the lab is [45:07] usually ineffective and fizzles out [45:09] after about 3 to four years or they have [45:13] duration but the company can never [45:15] commercialize the things coming out of [45:17] the lab. And so you get things like [45:18] someone comes by and sees the future and [45:21] then they do it as a separate company or [45:22] whatever, right? And there's like famous [45:24] examples throughout history of that. And [45:26] so I think there is this interesting [45:29] balance of like can you have a team of [45:32] people who live at the frontier like Dan [45:34] says who are building frontier things [45:38] but can somehow bridge or connect across [45:41] the bigger company in a way that it's [45:43] not oh that's just some weird R&D group [45:45] always coming up with like gimmicks and [45:47] toys. Do you know what I mean? And so I [45:49] think it takes a certain person or a [45:52] certain set of people who can actually [45:54] be like, "Hey, that thing looks like a [45:55] toy right now. That's going to be [45:56] essential to our future in five years." [45:59] And that's actually where at least [46:01] studying history where sometimes these [46:04] frontier ideas fall short. You can get [46:06] the first two things right and then fail [46:08] on the third one and then the whole [46:10] thing really doesn't matter. Do you know [46:11] what I mean? [46:12] >> Yeah. It's interesting. I was just like [46:14] even Anthropic has a labs team like the [46:17] most cutting edge company they're you [46:20] know just like at the frontier already [46:21] they're like okay but we also need a [46:22] labs team to actually watch the frontier [46:25] frontier. Yeah that's right. Well, one [46:27] thing we joke about too inside as labs [46:29] has grown now and we've had a few [46:31] projects thankfully have kind of hit and [46:33] are you know lots of users now and [46:35] growing we've actually internally to [46:37] this point of like common language what [46:39] we started to find was a bigger project [46:42] like notebook in terms of users or flow [46:45] or AI studio or these things that are [46:47] like kind of grown out of labs um they [46:50] were in some ways there's a risk they [46:52] crowd out the zero to one stuff [46:55] >> and so the terms we've started started [46:56] using is like there's zero to one labs, [46:59] there's one to 10 labs, and there's like [47:00] 10 to 100. And these may not be like [47:03] mean anything to your audience, but to [47:05] us, we actually kind of encoded like [47:08] these are different stages of like the [47:09] lab's life cycle. And we want all of [47:12] these products to have this like [47:13] innovative energy that's common. But [47:16] inside each of those stages, there's [47:18] very different challenges and very [47:19] different things you're optimizing for. [47:21] And in zero to one, you're looking at [47:23] eyeballs like we talked about. and maybe [47:26] 10 to 100 you're looking at like [47:28] waterfalls and CPAs and thinking about [47:30] like how are we actually converting [47:31] people to like these like magic moments [47:34] in the product and it's a scaling game [47:35] in a lot of way someone listening is [47:39] convinced they should set up a frontier [47:41] team a lapse team and it feels like laps [47:43] teams have evolved a little bit used to [47:45] be like build the next big business unit [47:47] is like the idea here it's like okay [47:49] what's like a new technology that is [47:50] unlocking something our core business [47:52] like I I wonder if that's true but [47:54] that's what it feels like to But I guess [47:56] just with that in mind, what are maybe [47:57] three tips for someone that's wants to [47:59] set up a lapse team internally? [48:01] >> Yeah, I think one, you got to try to [48:04] create a space where weird things can [48:07] grow and you can't just nestle this like [48:10] under an existing business unit. it has [48:13] to have its own kind of independence uh [48:16] is one and that in some companies that [48:19] means it needs to report to the CEO or [48:21] it needs to report into enough of a [48:23] distance in a way um and we can go into [48:26] that if you want but that's super [48:27] important I think the second thing is is [48:30] a set of people and in the case of us [48:33] this was this users first Google second [48:35] labs third thinking where the job of the [48:38] lab is to make users successful and make [48:42] Google success successful. It's not to [48:43] make itself successful. [48:46] And so there's a bit of a maybe a [48:48] humility or a collaboration or kind of [48:50] this mindset of like what does success [48:53] look like? And I think for us how we've [48:55] broken that down and how I would [48:56] encourage companies to think about it is [48:58] like is your goal of this lab to [49:00] graduate things to existing products or [49:03] is your goal to go invent new product [49:05] lines and categories? And in our case, [49:07] we started and it was a graduation lab [49:10] and it's expanded over time as we've [49:12] started to realize AI really transforms [49:14] and blurs a lot of categories. But being [49:17] very clear about like what game you're [49:19] playing and what does winning or losing [49:21] look like would be the second thing. So [49:23] isolated in some way but also part of [49:26] the same team. Um and I think the third [49:29] thing and we talk about this a little [49:31] bit on labs is and I could imagine for [49:33] maybe CEOs listening or other kind of [49:35] seauite people use your lab not just for [49:39] product development or technology [49:41] exploration but even for challenging how [49:43] the company works and almost like the [49:46] way things happen. Um and that's [49:49] actually the phrase we use on the team [49:51] is lab sometimes creates good trouble at [49:54] Google. So there's like bad trouble you [49:56] don't want to create. But good trouble, [49:58] give you an example. We were one of the [50:00] first teams that were like, we're seeing [50:02] the job roles blur. We have to have a [50:05] builder job ladder and we need to think [50:08] about it differently and we should use [50:09] labs as the place to pilot it before we [50:13] roll it out across Google. And so that's [50:15] another way you can think of like the [50:16] multi-dimensionality of a lab in a way. [50:19] you just like think about the processes [50:21] and ways of working, not just products [50:24] necessarily. That's really interesting. [50:25] >> Yeah. Yeah. [50:26] >> What about the type of people that you [50:27] would put on this lab team? You [50:29] mentioned this unlearning rate is a [50:30] really interesting way to think about [50:31] what else what do you look for and [50:33] what's like a person you want to kind of [50:34] avoid on the lab team. [50:35] >> Yeah. Yeah. Great. Um man, we are we are [50:39] obsessed about finding people who just [50:40] like can't stop themselves from [50:42] building. I mentioned that earlier. We [50:44] have a whole doc on the team labs in a [50:46] nutshell. Uh we have one on Gemini, too. [50:48] Gemini in a nutshell, the first week [50:50] someone joins the team, I email it to [50:52] them, and most of them have seen it [50:54] before as part of some of our hiring [50:56] screening, too. They build all the time. [50:58] They are intellectually curious. They're [51:00] reading, they're on X, they're listening [51:03] to podcasts. They are obsessed about [51:06] solving people's problems, not about [51:09] their own ego or glory. They are the [51:12] people when you're around them, they add [51:15] energy. you leave a conversation, you're [51:17] like, I want to be with that person. [51:19] It's like, I'll run through a wall with [51:21] that person. Do you know what I mean? [51:22] Um, we've got like 17 or 18 of them. [51:24] Maybe I can send some of them to you [51:25] after if you want if you want to include [51:26] in the show notes. Uh, [51:28] >> yeah, but it's it's um it's such an [51:31] important part. And that third thing I [51:32] talked about as like a tip for setting [51:34] up a lab. It's not just about job [51:36] ladders and stuff. It's literally about [51:38] who are the people of the future that [51:40] you are trying to bring in that are [51:43] going to think different and challenge [51:45] things different and it is it's a [51:47] different type and what we've also found [51:48] as these products have grown the zero to [51:51] one people are different a little bit [51:52] than the one to 10 and the 10 to 100 [51:54] too. So it's like how do you kind of [51:56] create some commonality but I I do think [51:59] like you know when you're talking to [52:00] someone you're looking at like in an [52:02] interview if I'm talking to someone [52:03] they're like hey let me show you this [52:04] thing I just built instant credibility. [52:06] uh or if they're like I'll ask them [52:09] questions about you know unknown [52:12] unstructured dark caves you know these [52:15] events and like where they're like don't [52:17] know what like which ways forward how do [52:19] they figure it out people that are [52:20] energized by that huge credibility boost [52:23] um so yeah there's definitely a certain [52:25] type that you're looking for [52:26] >> in terms of the org structure what's the [52:28] biggest mistake people make [52:30] >> yeah I think you know it's actually I'll [52:31] give you a live one we're talking about [52:32] right now is like we're trying to figure [52:34] out um how much [52:38] uh you kind of isolate different sub [52:42] teams and going after things versus try [52:44] to create groupings of like thematic [52:46] things. For example, we've talked about [52:48] like interactive entertainment as like [52:49] an area or like a pond we're interested [52:51] in fishing in. And it's like how does [52:54] that do you do that as one set of people [52:56] doing that? Is like five people just [52:58] take like five shots at it. Um, so that [53:01] like a mistake sometimes I think is [53:03] people maybe rush to a group and maybe [53:05] the bias is towards like consolidate, [53:08] make it clean. And I think what I've [53:10] learned maybe the hard way over the [53:11] years is like actually sometimes you [53:13] kind of have to live in the messiness a [53:14] little bit and almost like see what [53:16] emerges but still be willing to kind of [53:19] make those kind of changes. Those are [53:20] like org design team kind of things. Um, [53:23] the other change or the other thing I'd [53:25] say sometimes is a common mistake in a [53:27] you know you hire too many people too [53:28] fast. This happens at big companies. It [53:30] happens at startups too. And it's like [53:32] it can be so addicting to like sometimes [53:35] look at like, oh, our team doubled or [53:37] tripled in the last quarter or six [53:39] months or whatever. And these are just [53:41] vanity metrics that are really bad most [53:43] of the time. So I think that's the other [53:45] thing is like a common mistake. Um I [53:47] remember I was on a project years ago [53:49] and we got to like 30 engineers and I [53:52] was so happy and we had zero product [53:54] market fit but a very inspiring vision. [53:57] And you probably know where you know the [53:59] outcome of how that ended basically. [54:01] >> Uh I'm going to uh in our last few [54:03] minutes I'm going to ask you a question [54:04] Logan Logan Kilpatrick suggested to ask [54:07] you. [54:08] >> I think he said of AI Studio, right? I [54:10] forget his current. Okay, cool. [54:11] >> Great. [54:12] >> He's a great former podcast guest. [54:14] >> I don't know exactly where this question [54:15] is coming from, but I'm so curious. Uh [54:18] his question is in five years, do you [54:19] think Google is going to have 10,000 [54:21] products or five? [54:23] >> Oh yes. Okay. This is a longunning [54:25] debate Logan and I have on the team. Um, [54:28] >> but it's funny, at least to me, I keep [54:30] switching my answer. So maybe I'm a bad [54:33] right now. [54:34] >> I'm more in the camp of like I don't [54:37] know if we're even going to think of [54:38] products the same way. So I think one [54:40] framing of it is 10,000 products from [54:42] Google. That's like 10,000 surfaces that [54:45] have their own brands and it's all [54:47] complex and everything. Um, I'm not sure [54:50] that'll be possible to your earlier [54:52] point. what's possible or what should we [54:54] do? Um because you can have a product [54:56] factory that's just going to turn out [54:57] these products. So that's like one one [54:58] thing you could think about. Um I'm more [55:01] and more thinking about like are there [55:03] almost like experiences or things that [55:06] are going to be more bespoke that the [55:08] models might make and that might mean [55:10] there's labs teams behind it making the [55:13] agent that does this thing that you [55:15] experience maybe in a common surface. [55:17] I'm not sure. Um, but I also am not we [55:20] we have a couple of experiments in labs [55:21] going on this right now of like can we [55:24] prove both sides of this debate and [55:26] maybe that's why he threw it in like we [55:29] and sometimes that's what we'll do is [55:30] we'll kind of take both sides of it and [55:32] say like okay if this thing really [55:34] consolidates down and there's a super [55:35] app or a set of super apps what's that [55:38] world going to look like and maybe if [55:41] this thing super fragments what's that [55:43] world going to look like and so right [55:46] now I'm probably more on the camp of [55:47] Like I think super apps have worked in [55:50] some use cases in some parts of the [55:52] world obviously extremely well but I [55:55] think for a lot of people using them [55:58] they can be very complex and it's very [56:00] hard to actually drive kind of the [56:01] behavior that adds value because there's [56:04] so much complexity but this is also [56:06] assuming humans are going to be the ones [56:08] tapping most of these experiences as [56:10] well. And so anyway, this is a fun one [56:12] to debate and uh yeah, one of Logan and [56:14] I's favorite kind of we'll we'll chat on [56:16] this one a lot. [56:17] >> I love that it's just unclear where [56:20] we're going. Like it's just it just [56:21] shows where we're just we're all just [56:22] trying to figure this out. [56:23] >> That's right. [56:24] >> That's such a good uh example of that. [56:26] Okay. Well, before we get to our very [56:28] exciting lightning round, Josh, is there [56:30] anything else you wanted to share? [56:32] Anything else you want to leave [56:33] listeners with? Anything you want to [56:34] double down on? Yeah. I mean, I guess [56:36] one thing I'd think about is like um [56:39] and I I'm always trying to think of this [56:41] too. If people have ideas, please just [56:43] like let me know on X or whatever is [56:45] like I still think there's a lot of [56:47] space right now on how to get feedback [56:50] about the products and experiences [56:52] you're building whether it's in Google [56:54] Labs or as a startup founder or [56:55] whatever. And so that feels like another [56:58] area of like how do you listen and sort [57:01] of aggregate collect get insight [57:03] inspiration from feedback loops. Um that [57:07] I feel like there's a lot of interesting [57:09] stuff happening with AI applied to that [57:10] right now. Um and I'm sure all of you [57:13] probably do too Lenny on your stuff and [57:15] my stuff of you know agents running [57:16] overnight summarizing everything that's [57:18] happening and but I feel like there's an [57:20] interesting like what makes a PM great. [57:23] a great PM has insight about a certain [57:26] type of people and the types of things [57:28] that are going to delight them or solve [57:29] their problem. And I feel like that's an [57:31] interesting like how do you crack that [57:33] feedback loop with AI and these new [57:35] types of products we're building. So [57:37] anyway, that's something also in the [57:38] back of my mind right now. [57:40] >> Yeah. Like so idea the question there is [57:41] just how do you get AI to be as good as [57:43] an amazing top percentile PM at [57:45] identifying opportunities and insights? [57:47] Yeah, I think that's one branch of it. [57:49] And I think the other one is like is [57:50] there a world where [57:52] >> I don't know there's probably some [57:53] people in your life just like they're in [57:54] my life when you're around them [57:56] >> insights seem to flow pretty naturally [57:59] and you kind of get to a lot of [58:00] interesting perspectives and like is [58:02] there a collaboration model with people [58:04] and AI that enables more of that? I [58:07] think it's kind of an interesting prompt [58:09] maybe. [58:09] >> Sounds like we're getting into B2B SAS [58:11] software over here at Google Labs. I [58:12] love it. [laughter] [58:14] I'd buy that. [58:15] >> Okay. Uh, I know you have to run soon, [58:17] so I'm going to skip to just one [58:19] lightning round question. I heard you [58:21] give these hilarious awards to [58:24] teammates. Uh, my favorite is the TPU [58:27] harvester. Uh, but there are others. Uh, [58:29] share share these awards you give and [58:31] what they mean, what they're [58:33] >> Oh, yeah. Yeah. Um, so the TPU [58:34] harvester, um, we just need more and [58:37] more TPUs as our products are growing. [58:39] And I a few of us wanted to figure out a [58:41] way to reward people who do the grungy [58:43] work of efficiency and reclaiming TPUs, [58:47] harvesting TPUs so we can build the next [58:49] features and fuel our growth. So we give [58:51] out a rake to these people uh and a cash [58:54] award [58:54] >> like an actual rake [58:56] >> like a little miniature rake we made on [58:57] Etsy [58:59] and uh that one is really fun and is we [59:01] kind of turned into this big [59:02] celebration. So this is people that save [59:04] TPUs that optimize make them [59:06] >> optimize not use the most tokens. [59:09] >> That's right. That's right. Yeah. Yeah. [59:10] That we talked to them separately. But [59:13] uh you I think that's one's like uh like [59:15] a rake. Another one people that go in [59:17] fix paper cuts these little things that [59:20] add up in a product that annoy people. [59:22] We made like a golden band-aid and we [59:25] give people to that. It's like they put [59:26] it on their desk as a golden band-aid [59:28] for fixing paper cuts. Um we have a [59:30] thing in labs. were really interested of [59:32] like people who can push the model to [59:34] these like frontier ways that like in [59:36] surprising kind of ways. Um we have this [59:40] thing called the LLM whisperer like [59:42] they're able to kind of whisper to the [59:43] model. Uh they get a giant set of ears [59:47] um like because they can hear things [59:48] from the model that others can't and [59:50] like a really nice mechanical keyboard. [59:52] Um, and there's more of these things, [59:54] too. But we're we're trying to find like [59:56] ways to kind of like I don't know encode [59:58] the value in these like dorky symbols, [60:01] but that that mean something like [60:04] becoming more efficient with our TPUs [60:05] enables us to grow and reach more [60:07] people. Paper cuts, these are the what [60:09] make the product delightful and like [60:11] excellent. Uh, and then obviously like [60:13] the whisper with the ears is like these [60:16] lead to kind of breakthroughs and new [60:17] products. So yeah, we're always coming [60:19] up with weird stuff like this. I love [60:21] this. This kind of traditions are the [60:22] best. I heard there's also like a big [60:24] thank you one where you get like talk to [60:26] leadership or something like that. [60:28] >> Yeah, this one is a little bit of kind [60:30] of like a secret society in a way where [60:32] we'll basically like put a put an invite [60:34] on someone's calendar and we'll just, [60:37] you know, it looks kind of scary because [60:39] there's like four or five leaders on the [60:41] team. There's no agenda. Uh and then [60:44] they show up and we take turns. We go [60:46] around the circle and we just tell them [60:48] why we appreciate them so much for being [60:50] on the team. [60:52] >> Um, and it's pretty fun and it totally [60:54] surprises people and I always start the [60:56] meeting with like this is not a bad [60:58] meeting, this is a good meeting. [61:00] [laughter] [61:01] You know, you usually, you know, your [61:02] mind may assume the worst, but we're [61:04] actually doing the opposite. These are [61:05] some of the most most valuable people. [61:08] >> Wow, that is awesome. I love all of [61:10] these. Uh, Josh, how can listeners be [61:13] useful to you? Oh, Miss Cinda's [61:15] feedback. So many of you do already. I [61:17] would [laughter] say u we get a lot of [61:20] um a lot of joy and we're very grateful [61:22] for all the people who use the stuff [61:24] we're trying to build. And um if if we [61:27] can be helpful on the lab side or the [61:28] Gemini side, too, please let's make it [61:30] two-way. Uh and thanks again, Lenny, for [61:33] having me. It's been great great to [61:34] spend some time with you. [61:36] >> I think we uh think we're going to help [61:37] a lot of people figure out how to build [61:40] stay stay at the edge. Uh I I really [61:43] appreciate you being here, Josh. Thank [61:44] you so much. [61:45] >> Yeah, thank you. Have a great day. [61:46] >> Bye everyone. [61:48] Thank you so much for listening. If you [61:50] found this valuable, you can subscribe [61:51] to the show on Apple Podcasts, Spotify, [61:54] or your favorite podcast app. Also, [61:56] please consider giving us a rating or [61:58] leaving a review as that really helps [62:00] other listeners find the podcast. You [62:02] can find all past episodes or learn more [62:04] about the show at lennispodcast.com. [62:08] See you in the next episode.