Tech Chiefs Curb AI Token Costs to Protect Software Margins

科技主管控制AI Token成本以保住软件利润率 在未受监控的编程Agent引发严重开支超支后，企业领导者正在缩减开发团队规模，并将工作负载转向自托管AI模型。 业内高管分享了关于落地AI驱动的编程工具以及去中心化平民开发者项目的实操见解，这些举措旨在精简工作流程并削减软件成本。专家小组讨论了向精简的“单披萨团队”转型、通过两层模型路由管理Token支出，以及依托健全的治理体系攻克流程瓶颈。展望未来，演讲者重点提及了自主缺陷修复循环、“Agent Ops”框架的必要性，并指出在推动业务价值方面，领域专业知识正变得比传统工程背景更为关键。 平民开发者计划让领域专家能够构建量身定制的内部工具，同时使企业得以淘汰每年数百万美元的SaaS软件支出。 (1:40) 设计与生成式AI工具正在将重复、大体量的任务（例如制作多页幻灯片演示文稿）从数周大幅压缩至数分钟。 (1:40) 诸如Quick Protect Agent之类的专用安全Agent，让非安全专业人员也能够加固移动应用和Web端点，以抵御攻击性AI威胁。 (5:17) 为防止算力成本失控，企业应采纳两层模型策略：自托管处理80%的Token，而将昂贵的第三方LLM保留用于10%–20%的工作负载。 (5:17) 企业AI推进受阻的主要原因在于传统人工流程未变，这需要缩小团队规模、更新测试方案并调整治理框架。 (10:00)

Industry technology executives report that coding automation is shifting software development from manual syntax writing to domain-driven, agentic workflows, though realizing business value requires restructuring teams, managing token costs, and enforcing strict governance. Domain Experts Replacing Classical Engineering Bottlenecks (1:40) Accessible AI tools are enabling non-technical staff and domain specialists to build applications directly, bypassing traditional intermediate layers of product managers and business analysts. Seth Siegler notes that eXp Realty partnered with Lovable to launch a citizen developer program, allowing frontline domain experts to create custom internal tools that eliminated several million dollars in annual SaaS expenditures. At Piper Sandler, Kenny Vargas explains that investment banking analysts use design platforms like Gamma and Canva to compress Confidential Information Presentation deck creation from weeks down to minutes. Because coding costs have decreased sharply, organizations are shifting toward embedded pods where engineers and business practitioners collaborate directly. At Piper Sandler, cross-functional pods use tools like Claude or Codeium to prototype live dashboard variations for real-time client feedback. Siegler similarly observes that eXp Realty retooled its international division and its historic SUCCESS magazine brand by pairing subject matter experts directly with forward-deployed engineers, emphasizing that domain context has become more decisive than classical engineering backgrounds. Shrinking Teams and Change Management Friction (10:00) Adopting AI in production exposes organizational bottlenecks rather than model limitations. Derek Holt of Digital.ai notes that developers spend roughly 90% of their time on governance, compliance, security, and decision-making rather than writing code. Paralleling the shift from legacy data centers to cloud microservices, Holt argues that AI requires companies to restructure around smaller 'one-pizza teams' of three to four people. To prevent transformation failures, Digital.ai deploys a three-part team model comprising a forward-deployed AI engineer, a domain subject matter expert, and a change management specialist. Debojyoti Dutta of Nutanix outlines a talent model that couples internal upskilling with hiring new college graduates trained in AI-native paradigms, predicting a fundamentally altered talent pool within two years. Panelists warn that unmanaged organizational resistance remains a primary obstacle, citing Sarah Franklin's recommendation that enterprise leaders must either invest in training or prepare for severance. Economic Realities, Model Routing, and Measurement (23:36) Unmonitored agent usage can cause severe cost inflation without proportional output gains. Siegler highlights that eXp Realty experienced severe bill shock after deploying Claude Code across its engineering team because developers repeatedly rewrote code rather than delegating tasks effectively. Traditional productivity metrics like line count are obsolete; executives are shifting toward software delivery metrics such as pull requests, resolved stories, and Nutanix's baseline factory measures: throughput, quality, and frequency. To balance operational expenses against proprietary IP protection, technical leaders advocate hybrid model routing. Dutta recommends that enterprises 'own your intelligence' by running roughly 80% of workload tokens on local or self-hosted models, reserving expensive third-party frontier models for only 10% to 20% of tokens. Nutanix implements this through an enterprise gateway governing access across both frontier models and Model Context Protocol servers. Autonomous Loops and Agent Ops Governance (28:10) Automated coding workflows are advancing toward autonomous maintenance. Siegler forecasts that fully automated bug-fixing loops—where routines trigger Claude Code to triage incoming tickets, write fixes, and submit pull requests for human review—will become standard practice within six months. Vargas corroborates this trajectory, noting that agent capabilities are compounding quickly toward concurrent multi-workflow execution. However, the proliferation of decentralized application development introduces major governance and security risks. Holt notes that approximately one billion knowledge workers will gain access to automated logic tools compared to 50 million professional developers, necessitating an 'Agent Ops' framework equivalent to DevOps. Furthermore, Holt points out that non-security staff can use tools like Digital.ai's Quick Protect Agent to harden mobile and web endpoints to 90–95% of senior security expert performance, addressing vulnerabilities exposed to offensive AI. Looking out 18 months, panelists predict Jevons' paradox will take hold: dramatically cheaper computational intelligence will increase total software demand rather than decrease it. However, unlocking tangible business value depends on whether enterprises can implement centralized governance strong enough to safeguard security and maintain quality across rapidly multiplying autonomous applications.

来自 Nutanix、Digital.ai、Piper Sandler 和 eXp Realty 的行业技术领袖指出，编程自动化正从辅助生成代码演变为端到端的自主智能体流程，推动软件开发由传统工程主导转向业务专家驱动，并迫使企业在组织架构、成本治理与管控体系上进行深度重构。 业务专家主导开发与内部应用去SaaS化 (1:40) 在实际业务落地中，非技术背景的领域专家正在通过直观的 AI 工具直接构建专属业务系统。eXp Realty 首席创新官 Seth Siegler 分享称，公司支持全球 29 个国家超过 80,000 名经纪人，通过在非技术人员中推广 Lovable 平台和自定义 GPT，一线员工自发开发出管理层未曾预料到的大量内部细分应用；这种去中心化的应用构建方式，使公司得以淘汰数百万美元的年度外部 SaaS 软件支出，将其替换为内置安全护栏的内部自建方案。此外，通过将前瞻部署工程师与业务专家配对，公司成功将历史品牌《SUCCESS》杂志及国际业务重塑为 AI 原生运作模式。 在强监管的金融行业中，Piper Sandler 董事总经理 Kenny Vargas 指出，投资银行家过去需要分析师耗费数周时间在 PowerPoint、Excel 和 Outlook 之间手工制作包含 80 至 100 页幻灯片的保密信息备忘录（CIP）；而通过引入 Gamma 和 Canva 等生成式设计平台，制作周期已从数周压缩至数分钟，银行家得以将精力转回战略构思与叙事表达。同时，由工程师与业务人员构成的嵌入式敏捷小组利用 Claude 和 Codeium 实时生成并迭代多种界面方案，即时获取客户反馈，彻底避免了长周期开发后功能被弃用的风险。 组织重塑：“一盘披萨团队”与三元变革管理 (10:00) 生成式 AI 的普及正在重塑传统研发团队的协作机制与规模。多位专家指出，传统的亚马逊“两盘披萨团队”正在进一步缩减为 3 至 4 人的“一盘披萨团队”。Digital.ai 首席执行官 Derek Holt 直言，当前企业 AI 推广停滞的核心症结在于陈旧的流程与组织架构，而非模型能力不足；这与早期云计算迁移极为相似，若仅仅将数据中心负载重托管到云端会导致严重的账单冲击，唯有重构成微服务架构才能释放红利。同理，AI 驱动的开发要求企业重新设计测试周期、自动化安全审查与治理框架。 在人才战略与组织落地层面，Nutanix 首席 AI 官 Debo Dutta 提出了内部技能重塑与招聘 AI 原生应届毕业生的双轨策略，并预测两年内人才市场将发生本质转变。针对转型阻力，Derek Holt 强调 Digital.ai 内部推行由“前瞻部署 AI 工程师、领域业务专家、变革管理专家”构成的三足鼎立团队架构；他认为非技术层面的组织变革管理往往是决定 AI 项目成败或退回老路的关键所在。 Token成本失控与“拥有智能”的双层架构 (23:36) 随着代码自动化向工程团队深层渗透，算力与 Token 消耗的经济性引发了高度关注。Seth Siegler 举例指出，eXp Realty 在核心工程团队部署 Claude Code 后，首月账单激增却未伴随已解决工作单的相应增长，原因在于开发人员不恰当地重复编写代码或未能有效分配任务。为此，评估指标已从无意义的代码行数转向拉取请求（PR）与解决故事数量等实际开发者交付速度。Digital.ai 的 Derek Holt 补充称，由于首席财务官的严格审查，软件工程智能化度量已成为顶级投资优先级，企业亟需通过智能模型路由在高端模型与低成本本地模型之间取得平衡。 针对避免无上限计费与保护专有数据资产，Nutanix 的 Debo Dutta 主张企业必须建立“拥有智能而非仅租赁智能”的战略。他建议企业采用双层智能架构：仅将 10% 至 20% 的核心负载 Token 分配给昂贵的第三方前沿基础模型，而将剩余 80% 的日常 Token 运行在本地部署或自托管的开源模型上。Nutanix 依托自研的企业级网关与 Model Context Protocol (MCP) 服务器管理访问，并统一通过“数量、质量、频率”三项工厂基准指标对自动化效能进行量化考核。 自主修复闭环与大规模 Agent Ops 治理 (28:10) 研发模式正从人机协同辅助迈向完全自主的闭环流程。Seth Siegler 透露，eXp Realty 正在测试自动化缺陷修复流程：当系统记录 Bug 工单后，自动化脚本将触发 Claude Code 独立分类排查并自动提交 PR，人类工程师仅需负责最终代码审查与批准；他预测这种自主 Bug 修复闭环将在 6 个月内成为行业标配。Kenny Vargas 亦认同智能体发展正呈抛物线加速，即将超越手动提供上下文阶段，演进为并发执行多工作流的高级形态。 然而，自主化规模扩张带来了巨大的安全与治理隐患。Derek Holt 警示称，全球约 5,000 万开发者与近 10 亿知识工人的比例悬殊，当大量知识工作者开始自主构建业务逻辑时，企业软件将面临严重碎片化风险，必须建立类似于 DevOps 的专职 Agent Ops 治理层。此外，针对公开发布的应用与网页端点普遍缺乏防护的问题，Digital.ai 推出了 Quick Protect Agent，使非安全专业人员也能实现资深安全人员 90% 至 95% 的加固水平。Dutta 与 Siegler 总结认为，依据杰文斯悖论，廉价的高性能智能反而会呈指数级拉动软件创造需求；一线业务人员绕过中间层直接消除沟通损耗的同时，企业需要集中部署比以往严格十倍的安全与质量治理机制。 该讨论表明，软件工程自动化的终极检验标准并非技术层面的代码产出速度，而是能否安全地转化为业务战略价值与实际用户问题的解决能力。面对即将在数月内普及的自主修复闭环和非技术人员的大规模构建浪潮，企业亟待解决的核心课题是在释放一线生产力的同时，补齐 Agent Ops、端点安全防护及本地模型路由等系统化底座。

## Background 本视频由 Ai4 于 2026 年 8 月 21 日在 YouTube 平台发布。视频记录了一场由 Olsen Capital 管理合伙人 Joe Olsen 主持的高管研讨会，邀请了来自 eXp Realty、Piper Sandler、Nutanix 和 Digital.ai 的多位企业高管，共同探讨编码自动化浪潮下企业如何重塑软件战略、调整团队角色与交付流程。 ## TL;DR 研讨会围绕企业如何应对编码自动化与 AI 技术的崛起展开。与会高管分享了在私有云与多云环境下部署 AI Agent 的实践，以及如何利用 AI 原生工具提升企业内部生产力。针对人才与团队重塑，专家们提出了“技能重塑（Reskilling）”与精准招聘的双轨策略，旨在打造具备高阶思维能力、能独立承担更多职责的“单披萨团队”。最后，与会者强调虽然软件部署与运维的生产化至关重要，但对业务问题的深刻理解与解决始终是创造核心价值的关键。 ## Atmosphere & style 整场研讨会节奏明快、高效紧凑，高管们在有限的时间内抛出大量干货与行业洞察。对话风格专业且务实，主持人的有效引导与嘉宾们的经验分享形成了良好的互动，展现出面对技术变革时既审慎又前瞻的领导者视角。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=4s) Nutanix 的 AI 策略包含对外构建能够运行 AI Agent 的多云平台，以及对内利用该平台全面提升员工生产力。 - [00:04](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=4s) Digital.ai 专注于代码生成的前后流程管理与软件交付过程中的自动化落地。 - [17:22](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=1042s) 企业应对代码自动化浪潮的核心策略在于重塑现有员工技能（Reskilling），培养具备综合思维能力与独立高阶职责的“单披萨团队”。 - [17:22](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=1042s) 应届毕业生和初级人才正在快速掌握 AI 原生开发模式，企业可以通过精准招聘引入适应新型 AI 工具的新鲜血液。 - [17:22](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=1042s) 高校针对 AI 时代软件工程的课程体系改革尚处于起步阶段，预计将在未来两年内输出更多全新的 AI 原生人才。 - [34:57](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=2097s) 在推动软件自动化与生产化（productionizing）的过程中，审慎落地的同时必须将焦点放在理解与解决实际业务问题上。 - [34:57](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=2097s) 编码自动化时代真正的核心价值与竞争壁垒在于深入解决业务需求，而非单纯的代码生成。 ## Detailed walkthrough 这里是为你整理的视频第 1 部分（00:04 至 17:20）的详细实录与内容梳理： ### [00:04](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=4s) 研讨会开场与嘉宾自我介绍 会议由主持人 Joe Olsen 主持，共有 30 分钟的时间供 4 位行业领袖分享他们在软件开发自动化策略方面的经验。与会嘉宾依次进行了自我介绍： * **Debojyoti (Debo) Dutta**：Nutanix 首席 AI 官（Chief AI Officer）。他负责多项业务：一方面主导构建 Nutanix 的多云平台，以高效运行 AI 智能体（AI agents），并确保其在私有云和公有云等各种环境下的一致性；另一方面，利用该平台推动内部员工生产力的提升。 * **Derek Holt**：Digital.ai 的 CEO。Digital.ai 专注于在代码生成上下游### 研讨会开场与嘉宾介绍 研讨会主持人 Joe Olsen（Olsen Capital 管理合伙人）拉开序幕，指出本次研讨会时间紧凑，仅有 30 分钟，却聚集了四位重量级嘉宾。为了让每位嘉宾能够分享尽可能多的干货，他引导大家快速进行自我介绍。 首先发言的是 Nutanix 的首席 AI 官（Chief AI Officer）Debojyoti Dutta（简称 Debo）。他表示自己的职责涵盖多个维度：一方面，团队致力于打造 Nutanix 的下一代多云平台，以极高的性能运行 AI Agent，并保证其在私有云、公有云等各种环境下的一致性；另一方面，他们也在企业内部使用同一套平台，全面推动员工生产力的提升。 接着是 Digital.ai 的 CEO Derek Holt。他介绍 ### [017:22] 重塑人才与团队：重塑技能、精准招聘与“三足鼎立”转型模式 在探讨如何应对代码自动化浪潮时，针对“如何重塑团队与技能”这一核心问题，Debojyoti Dutta (Nutanix) 提出了两手抓的双轨策略：一方面通过重塑现有员工技能（Reskilling），让他们掌握 AI 原生工具，从而打造具备更高阶思维能力、能独立承担更多职责的“单披萨团队”（One-pizza team）；另一方面，进行精准的高质量招聘。Dutta 特别提到，如今的应届毕业生在大学阶段就已经开始学习并掌握这些 AI 构建软件的新模式，这与两年前大不相同，这让企业能从应届生和初级人才中挖掘出非常优秀的人选。他透露，自己与多所大学教授交流时发现，高校### AI时代的企业人才重塑与“三足鼎立”转型团队 Debojyoti Dutta（Nutanix）指出，公司正在采取“双管齐下”的策略来推进自动化：一方面对现有员工进行技能重塑（Reskilling），提升其综合思考能力以打造能够承担更多高阶职责的“单披靡团队”（One-Pizza Team）；另一方面进行精准招聘。有趣的是，两年前的新毕业生可能还不懂这些 AI 工具，但如今的应届生在校期间就已经掌握了 AI 原生开发方式。然而，与高校教授交流会发现，大学课程体系的改革才刚刚起步，预计两年后市场上才会涌现出全新的 AI 由于本次提交的转录文本片段极短（仅包含视频尾声的最后两句总结与对提问者的致谢），面板讨论的主体内容已在前两部分结束。 ### [34:57](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=2097s) 生产化与业务价值的平衡 在研讨会的尾声，发言者对软件工程的未来走向作出了总结性的升华。发言者强调，虽然团队必须对软件的“生产化”（productionizing，即部署、运维与上线落地）保持高度审慎与关切，但整个过程中最核心、最具有“金子般价值”的部分，始终在于对业务问题的理解与解决。这才是编码自动化时代里真正带来奇迹与竞争壁垒的“魔力”所在。发言者随后向提问者表达了诚挚的感谢，为整场关于编码自动化与软件策略重塑的讨论画上了句号。 ## Notable quotes > [34:57](https://www.youtube.com/watch?v=7d0z8wWH_hg&t=2097s) 整个过程中最核心、最具有“金子般价值”的部分，始终在于对业务问题的理解与解决。这才是编码自动化时代里真正带来奇迹与竞争壁垒的“魔力”所在。 — 发言者 ## Mentioned Seth Siegler, Kenny Vargas, Debojyoti (Debo) Dutta, Derek Holt, Joe Olsen, eXp Realty, Piper Sandler, Nutanix, Digital.ai, Olsen Capital, Ai4 ## Video description Seth Siegler - Chief Innovation Officer, eXp Realty Kenny Vargas - Managing Director, Head of Software and Data Engineering, Piper Sandler Debojyoti Dutta - Chief AI Officer, Nutanix Derek Holt - CEO, Digital.ai Software, Inc. Joe Olsen - Managing Partner, Olsen Capital The rapid rise of coding automation is forcing business and technology leaders to reimagine their software strategies. In this panel, senior executives from top enterprises will share how they are navigating this shift—adopting AI tools, rethinking team roles, and redesigning development workflows. Learn how they’re prioritizing investments, measuring outcomes, and preparing their organizations for the next generation of software development. Whether you're piloting AI copilots or scaling automated delivery, this panel will offer strategic insights on how to lead through change and build a more agile, efficient engineering organization. [Source](https://www.youtube.com/watch?v=7d0z8wWH_hg)

[34:57] 发言者: 整个过程中最核心、最具有“金子般价值”的部分，始终在于对业务问题的理解与解决。这才是编码自动化时代里真正带来奇迹与竞争壁垒的“魔力”所在。

[5:17] Debojyoti Dutta: Own your intelligence, don't just rent all of it. [17:22] Sarah Franklin (referenced by Joe Olsen): either invest in training or invest in severance. [23:36] Seth Siegler: That's a robot's problem [23:36] Debojyoti Dutta: how much, how good, and how often

[0:04] Well, thank you guys for for joining us this  afternoon. We've we've got 30 minutes and [0:08] uh and four people. So, I'm going to try to uh  keep this moving so that these folks can share as [0:14] much knowledge as possible with you guys. Um why  don't we start with quick intros? Uh do maybe you [0:19] want to start and we'll just run down the line.  So, hi, I'm Debo Duta, chief officer at Nutanix, [0:24] which means I do multiple things. One of them  is we build the cloud platform next multi cloud [0:29] platform to run AI agents very well and this  can it's a and it runs consistently across all [0:36] um you know private cloud public cloud elsewhere  and all the on the other side I you we use the [0:43] same platform to drive down productivity increases  for our employees right uh Derrick Holtz CEO [0:50] of digital.ai uh digital.ai AI focuses on uh  bringing AI solutions upstream and downstream [0:56] from coding agents to unlock the true value  of accelerated delivery particular in the in [1:02] the enterprise working with a little over half  of the Fortune 100 to help them take advantage [1:06] of this new opportunity. Uh Kitty Vargas, I'm  a managing director at Piper Sandler. My role [1:13] entails deploying AI, data, any type of solution  across our investment banking sectors. and uh [1:20] Seth Seagler from a company called EXP Realy, the  largest single real estate brokerage in the world [1:26] uh with more than 80,000 real estate agents in 29  countries. Um yeah, and my role is uh is focused [1:33] on our our AI transformation um finding ways any  way possible for it to help us. Yeah. Awesome. [1:40] Thanks, guys. So, uh, we're here to talk about  automation as it relates to software development [1:46] strategy or, um, and so maybe Seth, why don't we  start with you and we'll work our back or work our [1:51] way back the other way. Maybe walk through one  specific AI assisted coding initiative that you [1:56] guys have deployed or actively building. Um,  and maybe start with the business problem you [2:01] guys were solving and what the measurable  impact was on the end. Yeah, I think um, [2:06] I think our most interesting one is uh, what we  call our citizen developer program. Um, so what [2:11] we did was partner uh with a company called  Lovable uh a couple year and a half ago now. [2:17] Tiny little company. Yeah, tin. Well, they were  tiny when we getting any press at all. So, yeah, [2:22] we're our first enterprise customer. Um, but uh  but we extended access to that that platform. It's [2:29] a you know what's known as a vibe coding platform.  We extended access to that to our non-technical [2:34] staff, anybody that wanted um to get in on it. And  um and it was sort of a continuation of what we [2:40] were doing with OpenAI and uh enterprise chat GBT  and custom GBTs where we incentivized people to [2:47] um nontechnical staff to leverage their subject  matter expertise to build solutions for just [2:54] exactly what they were doing dayto-day or what  their team does day-to-day. So what we saw was [3:01] um an explosion of solutions built that solved  really focused problems that our upper management [3:11] uh seuite anybody in leadership wouldn't even have  known about in many cases but was an actual pain [3:17] point for the team and the people doing the  work in the trenches. Um not only that we saw [3:22] uh a replacement of a huge number of uh of  SAS solutions. So, we ended up being able [3:28] to sunset several million dollars in SAS spend  in favor of uh of in-housebuilt non-technical [3:37] staff subject matter expertise leveraged  decentralized innovation uh is what we saw. So, [3:42] we built a whole process around that to actually  productionize uh ensure safety, security, privacy, [3:50] everything else. Um and it's been a huge success.  Cool. Okay. So my example is similar but not quite [3:58] the same. Right? So if you look at the life of a  banker, their life is spent in PowerPoint, Excel, [4:05] Outlook, right? So a lot of it is there's a  lot of churn in their process or day-to-day [4:10] workflows. So one of the items that we built to  unblock bankers because if you think of a it's [4:16] called a confidential information presentation. So  it's a deck that's like 80 to 100 slides. You know [4:20] bankers spent significant amount of time putting  them together and where AI is today if you look at [4:27] cloud design as a good example it's a beautiful or  an amazing tool to build slides or to build decks [4:32] right so it takes away the process of identifying  which images or which icons or what's the flow of [4:39] my presentation and as a banker you can focus more  on the storytelling positioning compon you know [4:45] component of it which eventually compresses  cycles so for us it's been transformational [4:50] We piloted Gamma, Canva, everything that's in the  marketplace to identify what was the best tool, [4:55] right? Because it allowed allowed us to take weeks  from, you know, the analysts and associate classes [5:01] that would spend, you know, putting together these  decks, going back to their MDs to ultimately just [5:06] hitting a button, getting like four or five  different versions of the deck, and then saying [5:11] this is the go forward and ultimately mechanize  the whole process. That's great. So one uh example [5:17] that I'll give so as I mentioned we uh help build  tools and agents sort of upstream from coding and [5:21] downstream from from coding uh in in the delivery  space. And the main reason for that is in most [5:26] enterprises about 90% of the time is actually  spent not coding. It's around decision-m and then [5:31] all the governance and compliance and security  and and quality work that needs to be done. And [5:35] so in some ways writing more code was not ever  their bottleneck. A little different story for a [5:40] startup but but but that that is uh certainly the  case. And so um one that I'll focus on and this [5:45] is timely given the the news over the last few  weeks. Um if you look at mobile apps and websites [5:50] the overwhelming majority are not protected.  Those endpoints are not protected. Sure the the [5:55] uh infrastructure side is for the most part  protected. But if you go out to the app store and [5:59] download an app, most of those are not protected.  And thus they are a new attack vector. And if we [6:03] were any everybody's reading the news, uh there is  a massive increase in in in obviously offensive AI [6:10] um attacking applications. So, one of the big  bottlenecks, we have a product that allows you [6:14] to harden mobile apps and websites, right? Uh, one  of the bottlenecks has been that there's not as [6:19] many skills as you think there there needs to be  in order to protect every app that is out there. [6:24] And it's required a level of sophistication um and  training to be able to use the tools, ours, ours [6:29] included historically. And so, what we introduced  was what we call the quick protect agent. [6:34] This allows anybody, even if you're not a security  expert, to take your app, run it through an agent, [6:39] and we will protect it nearly to about 90 to 95%  of what we see most security professionals able [6:44] to do. And we'll continue to raise that raise  that bar. So, I think it's a good example of [6:48] like not necessarily replacing a job, but in fact  doing work that wasn't able to be done because of [6:53] of lack of security expertise, right? Yep. Yep.  Yeah. So um I think uh last year or um I last [7:02] two years we were in AI assist mode. So we across  the company we've helped build and also use agents [7:10] to help a particular persona do their job better  whether it's customer support agent whether it's [7:16] software engineers we're a technology company and  we saw the huge wave in since cloud code basically [7:22] uh about last year where things were becoming  quite autonomous and of course with open cloud [7:28] it's gone crazy so what we've we are now in  the middle of that journey from to go from AI [7:34] ist to like agentic with a little with human  intervention. So in in order to do that so [7:41] what we've done is of course we we've looked at  every I'll just talk about our engineering for [7:46] now because the topic is coding automation. So  again we saw that uh to the point uh that coding [7:53] is not the most important I mean it's not the most  uh intense uh part of the journey it's like 7030 [7:59] or something right so we are looking at every uh  part of the software development life cycle and [8:05] we're trying to see how we can make it agentic so  that's point one point two um now when we look at [8:12] uh coding tools I mean everybody has a choice on  you know which coding tool they like developers [8:18] you know it's back in my school days like it was  Vim or Emac. So it's still the same and it's not [8:23] going to change. What we realized is that we need  to make sure this agentic SDLC or whatever we want [8:29] to call it eventually will need to be guardrail  governed and co and costgoverned actually. So in [8:37] order to do that so we all built an in just to  help ourselves we built our nutanics enterprise [8:43] AI part of our nutanics agent uh AI solution which  actually does again two things one is it helps you [8:50] run local models locally so you can run LLMs near  frontier LLMs locally the second thing is it helps [8:57] you we have a gateway which helps you protect both  model access and MCP server access so that you [9:04] can actually run a coding agent without having  to worry about bringing the house down. Uh and [9:12] with this uh internal innovation so basically  drinking the champagne while bottling it, [9:17] we can I feel we are very confident we are making  strong progress in our agentic AI journey rather [9:23] AI assist is done. It's already a done deal. We  we've gotten the similar kinds of benefits that [9:29] all of us are talking about. And so my strategy  is is that you know own your own intelligence. [9:35] Don't just rent it. Don't rent all of it. And  think about a two-tier intelligence where you [9:40] have to use your foundation models for about 20%  10 20 dependence on your use case and there's no [9:47] way around it but 80% of your tokens should  come from something that you own and perhaps [9:53] not per token not price per token. Well, uh, you  know, I think when we think about, uh, Agentic, [10:00] uh, or AI supported coding or automation, I  think one of the things we think about is you, [10:05] it's a lot of trial and error. Um, and so maybe  Derek, you can start with you, but where where [10:10] have you seen, you know, an instance where you've  pushed something down the line and decided, hey, [10:16] we're running into roadblocks. This isn't working.  We need to go backwards. um and how do you handle [10:21] um those initiatives when you have to decide you  know what to kill, what to roll back etc. I mean [10:26] I think in in the context of the enterprise um uh  you find that very quickly the moment you go from [10:31] pilot to like trying to do something in production  right because there are so many requirements from [10:36] a regulatory perspective from a compliance  perspective um etc. I think one of the things [10:40] that's probably and I'm as guilty as this as  anybody is we like to talk about what's the next [10:44] version of the model and how much better that next  version of the model but I've been through enough [10:48] of these big transformations to kind of always  fall back to the fact that the people and process [10:53] part is often overlooked and so one of the things  that I think is really compelling and there's [10:57] been some really good discussions here which is  what is the optimal team size for a development [11:01] team in the AI age right I think most are saying  much much smaller than what we're we're used to [11:06] uh number two is what are the right processes  and I think that's the number one thing that [11:10] I think is holding a lot of organizations  back. Um there's a lot of differences here, [11:13] but like I remember the early days of cloud and  the first move was like take what we used to do in [11:18] our data center and move it to the cloud. And that  failed really badly, right? The bills came in, [11:22] they were really high and we all looked at each  other like why did we do this? And it wasn't [11:26] until we changed the architecture. We we changed  uh more kind of microservices and the way that we [11:31] would deliver things that we started to see the  benefit. I think we're right at that injunction [11:34] point with with with uh AIdriven coding. We need  to rethink how the teams are organized. We need to [11:40] rethink the processes and that includes when  we test, how we test, when we secure, how we [11:45] secure and what does governance look in a world  where uh it's a combination of humans and agents, [11:50] right? Um, and so it's probably not a direct  answer, but I I really do think this next wave [11:56] will be maybe a little bit less about uh the the  models getting incrementally better and a little [12:00] bit more about how do we do this at scale uh in in  the realities of of banks and insurance companies [12:05] and governments and and others. Yeah. You know,  real quick, you you hit on uh I've heard uh [12:10] Bezos's two pizza team come up quite a bit during  this event. Um which I think is just interesting [12:15] is we're going back. We're in like a three to four  person team right now. One pizza team. Yeah. to be [12:20] team. Yeah, it's a one piece of team to uh and we  have some that are still legacy, but but but um I [12:25] think it's a lot of experimentation right now. And  the good news is we can do quick iterations and [12:28] determine what works and what doesn't work. And  microservices, it's funny that you bring that up. [12:31] You know, you hear this word agent thrown around  and you'll see it all over the expo hall, too. But [12:35] as an engineer, you look at it and you're like,  it's just it's microservices calling code calling [12:38] code, which is just a funny thing to think about.  But Seth, what what do you think? When have you [12:42] had to roll something back or um we had to pump  the brakes a little bit uh on that same program? [12:47] Um once we started seeing just like an explosion  of of pieces of software of web applications [12:52] getting launched um and then by the way it grew  past that too grew we uh started working with a [12:58] company called SIM AI uh we're using it for all  different things and you know building workflows [13:02] and and different uh even smaller orchestrated  aentic systems. All of that stuff had to flow into [13:09] our review process um to ensure we're a publicly  traded company. uh we have quite a bit to ensure [13:16] uh is is good to go before it goes into production  and our review process became the bottleneck. So [13:22] uh we started seeing at one point we had like  a hundred submissions just waiting for me to go [13:26] through and do a code review and uh our security  people our privacy people everything else to kind [13:32] of like jump in there and review it. So we had  to go about audit, you know, automating that with [13:37] um with cloud routine, cloud code routines and  yet another sim uh automation um for auto um [13:44] initial code reviews to bounce back just real the  the sort of the lowerhanging fruit right off the [13:49] bat to the builder um and just saved us a ton of  time. And then but as we've done that right as the [13:55] velocity is otherworldly now for creating code and  creating software it just moves like we never had [14:02] to we never had a bottleneck with DevOps before.  We never had a bottleneck with like security being [14:07] able to take a look at a piece of software but  we have that and we had to kind of keep coming up [14:12] with new solutions in this totally new paradigm.  But uh yeah, a few times we had to you know to [14:17] pause things you know communicate new ways of uh  of you know of what to expect in the review portal [14:23] and and things like that and even think about yeah  the who is creating software now like you said is [14:31] very very different you know what does that team  look like the roles you know are collapsing some [14:37] new roles are being created right um and maybe it  makes sense to yeah I'll invest a little bit more [14:42] now in infrastructure you know few people all  kinds of different things like that. It's a new [14:46] way of thinking about it though and we've run into  a ton of pitfalls and totally uncharted territory [14:51] but yeah um each time we apply AI to fix it to  be honest. Kenny in your business there's got [14:57] to just be a ton of regulation wrapped around this  stuff. So I would imagine you're constantly trying [15:01] to figure out frameworks that you can try to do  to scale to solve some of those DevOps problems [15:05] that Seth talked about. Maybe maybe talk to us a  little bit about your guys' process. That's a good [15:11] one. I wasn't ready for that question, but I'll do  my best to answer it. Organic response. Exactly. [15:17] No. So the the the theme here is is speed or time  to market, right? So all that has compressed. So [15:23] in a regulated industry, you have all these gates  and all these mechanisms and audits and controls, [15:28] right? That ensure that software is deployed  especially if it's client-f facing in a secure and [15:33] governed way. Right? when you have AI and to your  point you have people in the business that are bip [15:38] coding apps you know people producing artifacts  distributing to clients that whole paradigm shifts [15:43] completely right so ultimately what we had to  build and I I have to smile or laugh a little bit [15:48] because it was an interesting approach was to use  AI to basically aggregate all the telemetry from [15:54] all of our systems everything that's available  firmwide to have a clear view not only as to what [16:00] people are doing on their workstations doing with  our market data with their emails etc. but also [16:06] how models are being utilized, right? Because  you have the notion of people believing that [16:11] an Opus 5 can do everything or why can't I use  Fable, right? And at the end of the day, it comes [16:16] down to right model, context, skills, you know,  utilizing the constructs that are present for you [16:21] to be effective at your job. But to your point, we  can't give everything to everybody because skills [16:26] is a is a good pattern, right? If you think about  making AI work for you, but then you start into [16:32] people wanting to share them, right? and people  having 30 skills, you know, and that explosion of [16:38] 30 times 5,000 bankers, you know, what does that  mean? So, there's some interesting problems. Yeah. [16:43] Um I think Demo maybe maybe just to shift out of  that for a second and talk about talent because [16:48] I think that's um one of the most interesting  trends over the last 12 months even has been, [16:53] you know, two and a half years ago, people were  afraid AI was going to take all of their jobs. [16:56] And now I think we're realizing that human beings  are quite important in the process and most of the [17:02] enterprises I talk to spend more money on the  business problem and trying to figure out the [17:07] cultural adoption all these other things than they  actually do writing code and all those things. So [17:10] I'm curious um in in in you know at Nanix are you  guys are you guys hiring less people? Do you share [17:18] the same view? How do you look at kind of the  human process laid out over the top of of software [17:22] automation? Yeah. So that's a great question and  we've um I mean there's no right answer but I can [17:27] I'll tell you we have a two-prong strategy. One is  we upscale our people. We give them the AI tools [17:33] uh to basically be AI native in order to be a one  pizza team because we were just talking about one [17:40] pizza teams. Each team member needs to do um  more higher level thinking. So that can be done [17:47] through resklling. And what we've noticed is that  our new college grads, they are kind of learning [17:52] this in school right now. So there was maybe  two years ago it was not the case. But I think [17:57] uh so we've had amazing uh luck finding really  good talent from new college grads and junior [18:06] candidates who actually know this how to do this.  So we are doing a mix of both reskilling and smart [18:13] hiring and actually I'm hiring. Yeah. I mean the  hard part is to find uh people who understand this [18:20] new way of building software and when I talk to  uh professors in different universities I mean [18:26] everybody's trying to retrain to change the  curriculum and train people how they actually [18:31] do this but so I think in about a couple of  years you'll see a very new talent pool than [18:36] you see today. I thought uh Sarah Franklin,  you know, who's a former CMO of Salesforce, [18:40] uh likes to post uh little kind of one or two  liners on LinkedIn and she had one about a month [18:45] ago that was to enterprise executives that said  either invest in training or invest in severance [18:50] and I thought that that was a really interesting  sort of HR perspective, but um Derrick, [18:55] you got anything to add or I mean we're hiring as  well. I mean it's evolving, but the reality is it [19:00] was evolving before AI as well. I mean this is  not the first time we've evolved as an industry. [19:04] Um I think the other thing that maybe I would just  throw out is hopefully this is like interesting is [19:09] um we like every other company yes we build  software to help people use AI to build [19:13] software but we also run a company and so we  have been making the same transformation a lot [19:17] of companies have had around how we do billing  how we do collections how we do marketing right [19:22] a lot of the kind of inside elements and one  of the things we found that has worked is we've [19:27] actually formed kind of a three-legged stool team  anytime we're driving an AI transformation so one [19:31] is we have internal forward deploy engineers  that are bringing the the the AI expertise, [19:37] right? Instead of blindly training everybody.  Of course, we've done that for like the basics, [19:42] but bring a forward deploy engineer, bring a  subject matter expert. So, somebody who knows [19:47] billing inside and out, but maybe not as as as  expert at AI. And then the one thing I think [19:52] folks miss is bring a a person who's an expert at  change management because like that element which [19:57] is very not technical is the thing that will bury  these things is you've got to have somebody that [20:02] understands how to get the organization from  A to B not just one or two people but like the [20:06] organization and we have found that those kind of  three-legged stool teams have driven a lot more [20:12] result for us versus hey we tried something and  then we reverted back to the old way of doing it. [20:16] They are in it through the entire transformation.  Yeah. Seth Seth, when you're rolling out some of [20:22] the solutions that you're doing, are you are you  finding the same thing that there's, you know, [20:25] more people wrapped around these things than  most people might think? Like more people wrapped [20:31] around like a more more people wrapped around  solving business problems and things like that, [20:35] you know, as opposed to writing code. Yeah. Yeah.  I think it it's just it's just interesting that [20:41] everybody can be involved in solving the problem.  uh yeah you know not I think the most valuable [20:48] thing is the subject matter expertise you know  like we are seeing um it's the rise of the of the [20:55] generalist you know it's the rise of the subject  matter expert who's interested in learning about [21:00] you know how they can leverage AI and in a lot of  cases it is so accessible for them to solve their [21:06] own problems but um but yeah we've seen like in  our international division um which is everything [21:11] outside of North America for us um they've  built an entire uh AI first uh fully or you [21:18] know orchestrated operating system by uh pairing  what is essentially a forward deployed engineer [21:26] you know with a with subject matter expertise  there um in our success magazine division um we [21:33] have um an amazing person leading that uh Rachel  she's actually right here and um yeah that's [21:40] that's now fully uh fully transformed as well from  one of the oldest companies in existence actually [21:47] uh brandwise um to be you know really AI first now  and and um by subject matter experts people that [21:54] have trained themselves like like Rachel and even  our founder and CEO Glenn Sanford um and uh and [22:01] it's just interesting yeah who those people are  yes there's there's a number of different people [22:06] that need to be involved but who they are and  their background is the most important thing now [22:12] uh versus, you know, I think I'm preaching to  the choir probably in this room versus like [22:17] the classical uh engineering training, you know.  Yeah. Um Yeah. Kenny, in your business, again, [22:22] just because it's so regulated and so interesting  and nuanced, are you uh pulling people in from the [22:28] business and helping them get closer to this new  type of way of working? Are you going out trying [22:33] to bring them in and teach them the business?  Like what do you find is the right mix for that? [22:38] So the right mix today is bringing them to similar  approaches as you know has been described o over [22:44] and over which is pairing engineers or creating  pods if you will you know that notion of the two [22:50] pizza team but the business is embedded in it  because the price of building software has gone [22:54] down significantly right we can quickly iterate  and have three you know claude or codec pick your [23:01] tool of choice produce three artifacts in real  time and you can tell your client or your banker [23:06] at that point you know, do you like this dashboard  better? Is this one hitting the mark? Which one [23:12] do you want? Right? And kind of ideulate at that  point in time and ultimately build a solution. So [23:17] that has been very valuable rather than us going  away building something that we think it's going [23:22] to be useful to them and they ultimately say, you  know what, I don't like it. And bankers are very [23:26] fickle, too. So if you put something in front of  them that doesn't hit the mark, you're never going [23:30] to hear from them. So yeah. So maybe I'll combine  a couple questions just to hit a couple hot topics [23:36] when it comes to automation um and AI written  code and things like that. So obviously there's [23:40] a lot of talk about um tokens and the expense of  tokens and you know some of the conferences I've [23:46] been to recently there's even discussion about  where the cost of that sits. Do we put it into [23:50] employees costs and things of that nature? Um, and  then on the other side you have like kind of the [23:54] ROI of of of of of automated software, you know,  and in the old days you you measured code output [24:00] and obviously now you're not you're not doing  that and in some instances it's like it's like [24:04] uh conflicting with spending tokens, right? So h  how do you uh how do you kind of manage that topic [24:10] internally when it when it relates to metrics and  software automation and how you talk about it to [24:15] the business ROI really? Yeah. Yeah. So uh it's a  hot topic that's for sure. Yeah. Uh so we just uh [24:22] we just deployed uh cloud code a little bit more  widely into our our main traditional engineering [24:27] org actually and because we had seen so much  success with it in our you know non-technical [24:34] uh users and and sort of uh smaller targeted teams  of technical users uh were like let's give it to [24:40] everybody and then um immediately saw the first  month's bill and we're like we need to uh we need [24:45] to work with them to to use it properly uh because  yeah we weren't seeing a huge amount of output [24:51] um increased um but we were seeing a huge amount  of spend so that we just tacked on expense for [24:56] ourselves. But yeah uh it is about uh it is about  you know seeing uh you know 5xing 10xing their [25:04] actual output of like you know tickets and and  pull requests and stories and stuff like that. So [25:10] um that's what we're looking at yet certainly not  the amount of code because I don't care about that [25:15] at all anymore. That's a robot's problem. But um  you know um yeah it's it does take a bit of of [25:21] training on how to use it properly. And with  traditional engineers I think the big thing [25:25] is like letting it do the work. Let it write the  code for you. Don't write the code and then use it [25:30] again. Right. And then we're just not getting any  velocity but we are gaining the cost. Yeah. I I I [25:36] would say the reason this is such a hard question  to ask to answer is um even before AI most large [25:43] scale organizations could not measure the business  process of building and delivering software right [25:48] we've seen waves of this through the history of  the industry maybe most recently with software [25:52] engineering intelligence was this going to be  a platform layer we have some products in that [25:56] space and I will tell you um prior to AI it felt  like it was a nice to have now that AI has started [26:02] to proliferate and CFOs are knocking on doors and  asking hey what are we getting for uh it's one of [26:07] our fastest growing areas, right, that people  are paying attention to. Um it's um you know [26:11] the thing about software has always been a little  bit of an art and a little bit of a science and I [26:15] think engineers and I'm an engineer by the way  so I'm not pointing fingers. Um uh I would say [26:19] the same thing like hey you can't measure this is  like you could measure a supply chain or you could [26:23] measure a a um a finance organization but the  reality is we got to figure that that out and and [26:29] uh start one number one is the benchmark today  right because I think sometimes we're not even [26:33] having a reasonable comparison about what it was  before um and then number two is um I think to the [26:39] to the earlier comments like it is very clear  that we are using a Ferrari in many many cases [26:43] like when a bicycle would be just fine and um  uh this idea of model routing and being smarter [26:49] about what you keep on uh on prem versus what  you're using Frontier Labs for um uh both in the [26:55] you know hot topic of preserving your alpha but  also in the topic of preserving your bottom line [27:00] um I think is going to be really critical in this  next chapter. It's still very very early and these [27:04] things tend to sort themselves out and the CFOs  are really good at helping you sort them out. It's [27:09] a good one. So um I'll actually chime in a little  more. So if you really think about metrics right [27:16] he said very hard to define but I think we should  not I mean I we don't promote token maxing so we [27:23] obviously started off with measuring how well  the AI adoption is going so we had to see how [27:28] much code was being generated but that's it our  metrics for software delivery haven't changed and [27:34] in general uh if you think about for any industry  not just software engineering if you think of [27:40] any industry as a factory So you there are three  metrics that I I personally care about. How much, [27:48] how good, and how often. And that and  whether you apply to iPhones or software, [27:54] it really doesn't matter. And AI is a tool to get  to the how much and how often and how good. I like [28:01] that. Yeah. Um I guess uh we've got a few more  minutes here. I guess maybe maybe let's try to [28:10] close this out looking at um you know getting past  kind of agents well maybe agents would be a part [28:16] of this but getting past co-pilots and things like  that if we're here in the same stage next year [28:20] uh having a similar conversation about software  automation and AI and those things what are we [28:26] talking about in your guys's point of view I think  um I think something that you know that and just [28:32] to use Derek real quick sorry what should CFOs be  afraid that they're going to have to pay for that [28:37] No, go ahead. Tiny joke. It's a whole separate  thing. Yeah. The uh No, but something we're [28:44] seeing I I think a lot of success in it is like  real automation of the um AI generated, you know, [28:52] coding, you know, loops. I think um that's a big  thing that we're just dipping our toes into where [28:59] uh for every uh let's say bug story that gets  created um you know for a given piece of software [29:07] it kicks off a loop now uh that can pull in a um  a cloud code uh routine to to triage it and and a [29:16] lot of times um it's very straightforward  where the bug story was written properly [29:23] um it's triaged and solution is pretty clear and  in those cases we are experimenting with having [29:29] cloud code just fix it and open a PR. Um that  way it's it doesn't take anybody's time except [29:36] to look at the the PR um and then give that a  go. So uh that's just like really uh I I you [29:42] know maybe people are further along in that than  I than I think but we're just dipping our toes [29:46] into it. I think that will be uh table stakes uh  within six months for everybody. Uh but um but I [29:54] think that that's kind of an interesting thing.  Uh for a year from now, I cannot even imagine [29:59] uh to be honest. What do you think, Kenny? No, I  agree. I think um the progression of of agents, [30:06] if you will, has been parabolic, right? So  what we're doing today, which is, you know, [30:10] feeding context and babying agents, I don't think  that's going to exist in a couple of months, [30:16] if not a year. Yeah. You're going to have your  engineers basically running multiple workflows [30:21] at the same time. and outcomes are going to be  significant. So, I agree with both of those. [30:26] The the um maybe a little bit of a outside of the  business process of building delivering software. [30:32] One of the things we're thinking a lot about is  there's about 50 million professional developers [30:35] in the world give or take. Um there's about a  billion people that are in white collar jobs [30:40] and knowledge working jobs. Um we are in the midst  right now of uh even beyond writing code just the [30:46] ability to take business processes and create  an agent around it. And in most organizations, [30:52] they are reasonably governing the business process  delivering software. Um early days, they are not [30:57] governing agents, particularly agents that get  deployed internally, right? Um so you've got [31:02] 50 agents that do the exact same thing but in a  different way. You have no real way of determining [31:06] whether or not it's been tested. If you find a  defect in an internal agent, like who picks up [31:10] the ticket? It's a bit of a free-for-all in a  lot of organizations. I think a year from now, [31:15] agent ops as an answer to what DevOps did for,  you know, traditional software development and [31:21] delivery. Agent ops is going to be the next big  um I think the big topic. It's already happening, [31:26] but um we're about ready to give, you know, a  billion people the ability to create automated [31:31] business logic. Yep. That's exciting and scary  all at the exact same time. Yeah. I always uh my [31:37] said on uh a panel a couple days ago like if you  could look into lovable database even and look [31:41] at how many websites are doing the exact same  thing it's probably 75% of those are overlapped [31:46] so it's yeah same thing yeah interesting yeah so  I mean I will just kind of summarize everybody [31:52] and say that there are three dimensions 18  months from now right the people side we've [31:56] already talked about one pizza teams um people  getting much more qualified um either through [32:03] retraining or res-killing in the universities  ities. So that's going to happen. Processes, [32:07] we all know that there'll be loops. I'll just  call call it loops because that's the hottest [32:12] topic today. Yesterday at least it was. Uh so so  loop engineering and all of that processes will [32:17] get transformed and then tech tech there'll be  Javon's paradox will take over the intelligence [32:24] will become a couple of orders of magnitude  cheaper. Again not going into details it will get [32:30] cheaper. So silicon intelligence will be cheap.  Now the thing is we will uh we as humans will [32:36] still need to have the right governance to make  sure that we can have these AI software factories [32:43] that everybody's talking about where people write  specs what they get a um as an output is a u I [32:49] mean is a feature or a product but we have we as  humans we have a much bigger role to play in that [32:56] future where we manage the entire software factory  from spec to owning the final output. So I think [33:05] that's my prediction. It's interesting how much  of a lot of the things you guys talked about were [33:09] kind of on similar wavelengths around just putting  more orchestration around this, more governance, [33:13] more control and those kind of things. So  I was just uh wondering like you're talking [33:17] a lot about empowering um the non-technical  people in the business to create applications. [33:26] Doesn't that does that create a concern for you  about the business process? Like are are they [33:33] speeding up the wrong things potentially? Uh  no. So it's a good question, but the answer is [33:40] uh no because they know the business better than  any of the engineering org knows the business, [33:45] right? They're doing the business. They're the  ones doing, you know, doing the work. And in a [33:49] lot of cases, it's not even the managers of  the teams that are building the software. [33:53] It's the people you know the frontline people that  are that are doing it. Um so they know it better [33:59] than anyone and you know I think uh in traditional  engineering again that yeah I also am an engineer [34:06] um but you know so I've been guilty of this too  is it gets lost in translation to some extent when [34:11] that person tells their manager and then that  manager tells a you know business analyst the [34:15] business analyst works with the product manager  and product manager you know what I mean they made [34:18] a movie about it office space right that's exactly  right and actually it's a great question which we [34:23] didn't cover in the panel on the panel that what  what happens how Do you manage the development [34:29] done by non-engineers? I think again in 18 months  you'll see a lot of non-engineers uh using their [34:36] own creativity to come up with apps. But the main  thing is what we just discussed that governance [34:40] is going to be key. You can build your app that  where you are a subject matter expert which was [34:45] covered right but subject matter experts don't  need to be engineers but the governance need [34:50] to be actually needs to be rock solid. In fact  10x more important. Yeah. Yeah. Yeah. for sure [34:57] concerned and careful about the productionizing  of the software, but the business part of it is [35:03] the is the part that is the gold. That's the  magic of it. That's great question. Thank you.