CloudZero, Clayton Homes Target AI Unit Costs as GPU Usage Fails

GPU利用率指标失效，CloudZero与Clayton Homes聚焦AI单位成本 传统云效率指标在生成式AI时代失效，若脱离明确的业务成果，高GPU利用率可能会演变成彻底的财务浪费。 该研讨小组探讨了企业AI从最初的探索试验向长期财务可持续性及严格单位经济效益的转变。行业专家指出，服务器或GPU达到100%利用率等传统云指标已不再适用，因为高算力使用率若与实际业务成果脱钩，极易沦为纯粹的财务浪费。为应对持续上涨的Token成本并防止预算超支，企业管理者讨论了构建集中式内部治理平台、评估定制软件开发的真实成本，并从传统的FinOps模式转向衡量产出每项具体业务成果所需的成本。 传统云端指标如 100% 的 GPU 利用率在 AI 时代已然过时，若缺乏明确的业务产出对应，满载运算可能等同于 100% 的财务浪费。 (2:25) AI 工具的普及大幅降低了软件构建成本，正在颠覆传统的“外购还是自研（buy vs build）”决策矩阵，促使非科技类传统企业转向自建定制化工具。 (5:28) 随着思维过程通过 Token 被直接货币化，撰写邮件或制作 PPT 等日常动作均成为采购决策，企业须建立中央 AI 平台以兼顾安全治理并防范影子 AI。 (10:08) 受杰文斯悖论影响，监控“单 Token 成本”容易产生误导；企业成本结构已倒置为 80%–85% 由 Token 消耗主导，应转而追踪“单次业务产出成本”。 (15:41) 盲目自建开源大模型容易忽视后续运维与工程维护的隐性成本，企业应在需求达到足够业务规模后再考虑自托管专用基础设施。 (38:35)

Enterprise AI economics requires replacing legacy cloud utilization metrics with outcome-based unit economics, as variable token consumption and architectural complexity dominate operational spending. Industry practitioners emphasize that controlling infrastructure costs depends on evaluating true business outcomes, establishing real-time cost visibility, and avoiding premature self-hosting or over-engineering. The Obsolescence of Traditional Utilization Metrics (2:25) Legacy FinOps frameworks prioritized server and GPU capacity utilization to eliminate idle hardware. In on-demand generative AI, Erik Peterson of CloudZero argues that high utilization is no longer a proxy for efficiency, warning that 100% compute utilization can represent 100% financial waste if workloads fail to produce business value. Executive teams and corporate boards increasingly evaluate AI investments on cost per business outcome or transaction rather than raw infrastructure metrics. The 80/20 Cost Inversion and Outcome Measurement (22:08) Generative AI flips traditional cloud cost models: fixed hosting infrastructure accounts for only 15% to 20% of spending, while variable token consumption drives 80% to 85%. Because providers bill for raw token volume without business context, enterprise leaders must evaluate expenditures against tangible results rather than activity metrics. For instance, evaluating an AI customer service workflow strictly by volume—such as 20,000 interactions at $0.03 per chat—obscures its true unit cost. If the system only successfully deflects 100 human support tickets, the entire cost of the workflow must be measured against those 100 resolved cases to determine actual return on investment. The In-House Build Paradox and Central Governance (5:28) Lower software development costs tempt enterprises to build custom tools in-house rather than buy commercial software. While this empowers domain experts to act as builders, Peterson cautions that bespoke applications require ongoing maintenance, operational overhead, and data verification for which executive leadership remains accountable. To balance agility with security, Abhinav Mahajan explains that Clayton Homes implemented a centralized internal platform. This central control plane manages the policy engine, identity chain, and lineage tracking for autonomous agents, preventing shadow AI and redundant compliance hurdles while allowing operational units to securely deploy tools against shared corporate data. Hidden Architectural Overhead and the Open-Source Trap (31:06) Traditional financial forecasting breaks down when engineering teams can generate substantial token spend within days, outpacing quarterly budget reviews. Furthermore, modern multi-step reasoning frameworks introduce hidden costs, executing dozens of unmonitored sub-queries, vector lookups, and inter-cloud data transfers for a single user prompt. Attempts to optimize spending can also misfire. Peterson cites an optimization project where running dynamic model-routing evaluation logic cost more than simply serving the original prompt. Similarly, panelists caution that self-hosting open-source large language models introduces unexpected maintenance and engineering overhead, advising organizations to validate demand on managed gateway services before investing in dedicated infrastructure. As token prices decline, panelists anticipate Jevons' paradox will take hold: lower unit prices will spur higher aggregate usage, compounding financial risk. Organizations must balance developer speed with continuous, minute-by-minute unit-cost tracking, ensuring that architectural choices prove their financial viability before being scaled.

在 Ai4 研讨会上，来自 TDK Ventures、CloudZero 与 Clayton Homes 的与会专家指出，企业级 AI 已从概念验证转向长期财务可持续性阶段；企业必须破除对传统算力利用率指标的迷信，转向以具体业务结果为锚点的单元经济学，并警惕隐藏的多跳推理开销与过早自建开源基础设施的财务陷阱。 传统云优化失效与业务成果导向的重构 (2:25) Erik Peterson 指出，传统 FinOps 依赖追踪硬件规格并淘汰闲置算力的运维逻辑在生成式 AI 时代已经失效。在按需付费的 AI 基础设施中，100% 的 GPU 利用率并不代表高效；若算力未与具体业务产出挂钩，全负荷运转反而可能造成 100% 的财务浪费。企业董事会与高管层关注的根本不是底层算力指标，而是每个业务成果或具体行动的成本回报。 Abhinav Mahajan 指出，企业过去引入大型企业级软件时，往往缺乏对每个席位投资回报率的精确衡量，习惯将其视作理所当然的机会成本。针对这一现象，Peterson 警告称仅监控“每 Token 成本”是误导性的虚荣指标。根据杰文斯悖论（Jevons' paradox），随着模型效率提高和 Token 单价下降，企业整体的消耗总量反而会激增。因此，企业度量标准必须回归真实业务单元，例如每建造一栋房屋、每处理一次客户交互或每完成一个业务目标的边际成本。 成本结构的二八倒置与预算观测滞后 (19:23) 研讨会讨论显示，生成式 AI 的支出结构与传统云计算相比发生了根本性倒置。传统云环境中，主机计算与存储等固定成本占据总支出的 80% 至 85%，可变运维仅占 15% 至 20%；然而在生成式 AI 部署中，固定支撑架构仅占 15% 至 20%，高达 80% 至 85% 的开销由模型 Token 消耗这类动态变量主导。Raghavendra Bagalkoti 举例说明，若客服智能体处理了 20,000 次对话（单次成本 0.03 美元），但最终仅成功拦截并解决了 100 起原本需要人工介入的技术支持工单，则该系统的真实投资回报必须完全由这 100 起有效拦截来衡量，而非表面的对话总量。 由于支出极易随着调用量快速膨胀，传统的季度预算审查周期出现了严重的观测滞后。Peterson 强调，工程团队在数天测试中消耗的 Token 即可达数十万美元，企业全年的 AI 预算很容易在短时间内由 10 万美元飙升至 1000 万美元，而现行财务报销制度往往对小额差旅层层把关，却对开发者的实时 API 消耗缺乏实时洞察。为此，企业需要推行结合预期业务量的单元指标预测模型，并为技术团队提供分钟级的成本可观测性。 自研决策风险与隐藏的多跳推理复杂度 (5:28) Mahajan 表示，AI 编程工具降低了定制软件的生成成本，推动非技术类企业重新评估“采购还是自建”的决策，使业务专家能跨越传统产品经理的隔阂直接构建工具。但 Peterson 警告，软件具备终身的维护成本与准确性要求，“能够构建”绝不意味着“应当构建”。如果内部构建的系统产生错误数据，高管仍须向董事会承担完全责任，AI 无法免除企业的运营问责。 在模型架构层面，单次用户查询的真实成本往往因复杂的内部推理链路而失真。Mahajan 阐述称，估算全天候运行的云服务器固定成本相对简单，但在多步推理智能体（Agent）中，单次用户交互会在后台触发数十甚至上百次隐蔽的子请求、向量检索、网络搜索及评估校验循环。Peterson 分享案例指出，其团队曾尝试构建能查询外部定价表以动态挑选最优模型的智能路由，但在实际运行中，执行路由选择本身的计算开销反而超过了直接调用标准模型的费用。 集中式治理架构与开源自建模型的陷阱 (10:08) 为防止“影子 AI”扩散并消除分散治理带来的官僚审批阻碍，Clayton Homes 采取了自研集中式内部 AI 平台的方案。Mahajan 介绍，该中心化平台内嵌了统一的安全策略引擎、智能体身份链跟踪（Identity Lineage）与可观测性，使不同业务部门能够直接复用统一的架构基座，在确保核心敏感数据合规的前提下安全接入企业专有数据，免去了重复安全审批与技术栈割裂的内耗。 关于底层模型部署，与会者警示了盲目采用开源模型的隐藏成本。部分团队过度自信于工程能力，认为自行托管开源大语言模型在成本上优于商业模型 API，却被后续繁重的底层基础设施调优、GPU 算力锁定以及持续的运维开销打乱节奏。专家建议，在业务规模尚未达到足够大的临界点之前，团队切忌过早投入自托管开源模型，应优先采用成熟的即插即用型托管网关服务验证需求。 总体而言，与会专家达成共识：企业 AI 的核心瓶颈已由算法可用性转向单位经济学与平台工程治理。然而，随着多智能体自动化复杂度的增加与跨系统集成的深入，企业如何在灵活性与持续维护开销之间划定明确的“终止或规模化扩张（kill or scale）”界限，仍是长期落地中有待持续权衡的核心议题。

## Background 本内容由 YouTube 频道 Ai4 于 2026 年 8 月 21 日发布，题为《AI 云经济学：控制基础设施成本与最大化 ROI》。视频汇聚了来自 TDK Ventures、CloudZero 及 Clayton Homes 的行业领袖与技术高管，深入探讨企业 AI 从实验阶段走向生产落地过程中，如何在推进 AI 创新的同时进行科学的云基础设施成本管控并证明业务投资回报。 ## TL;DR 圆桌讨论围绕企业 AI 落地中的单位经济学（Unit Economics）展开，强调在按 Token 计费的生成式 AI 时代，传统的 GPU 利用率指标已彻底失效，100% 的硬件满载完全可能是 100% 的纯粹浪费。嘉宾们深入剖析了企业自研统一平台与外购 SaaS 的权衡，警惕单 Token 成本下降导致的杰文斯悖论，并指出多轮调用与 Agent 架构中存在的隐性变动成本。团队必须将算力开销与真实的业务交付成果强力绑定，建立分钟级的成本遥测反馈，避免陷入动态模型路由成本倒挂及开源模型盲目自建的维护泥潭。 ## Atmosphere & style 整场对话务实、犀利且充满来自工程一线的实战反思。嘉宾们摒弃了对大模型技术的盲目狂热，以清醒的财务视角和自嘲式的案例（如为了选便宜模型反而花掉更多 Token、评估成本反噬系统本身）拆解 AI 落地陷阱。现场互动节奏紧凑、共鸣强烈，展现了技术决策者在面对 C-Suite 问责与真实业务交付压力时，对“算清算力账”这一命题的严谨思考与迫切诉求。 ## Key takeaways - [02:25](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=145s) AI 时代效能评估发生根本范式转移，传统的 100% 硬件算力利用率已不再代表业务价值，甚至可能是 100% 的纯粹浪费，企业必须转向衡量“单位成果成本”。 - [04:39](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=279s) 软件生成成本的大幅暴跌逆转了传统的“能买就买”外购逻辑，非纯科技企业开始走向自研优先，具备深厚业务领域知识的员工皆可直接构建 AI 应用。 - [04:39](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=279s) AI 时代企业员工的日常操作（如写邮件、做 PPT）都在调用 Token，思考本身已被货币化，因此企业需要将成本责任下放并建立精细的单位经济核算。 - [10:08](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=608s) 破解“影子 AI”泛滥的核心在于自研内部统一基础平台，通过一次性打通安全边界、统一身份认证与全链路可观测性，消除各部门的重复建设与审批摩擦。 - [13:14](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=794s) 自建系统需警惕盲目自研的长期代价，“软件是一生的承诺”，AI 无法承担数据出错的商业问责，自研必须聚焦核心业务成果并赋能一线业务专家。 - [17:25](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1045s) AI 单 Token 价格下降会触发“杰文斯悖论”，引发调用量指数级暴增，因此成本管理不能依赖每 Token 单价，而必须锚定企业的核心业务交付单元。 - [22:15](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1335s) AI 账单结构发生根本变化，周边工程架构的固定成本仅占小部分，80% 至 85% 的实际开销集中在黑盒化的模型 Token 消耗与上下文膨胀上。 - [23:24](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1404s) 评估 AI 交互不能只看活动量指标，而要看最终转化的有效业务成果（如工单真实分流率），避免因大量低效闲聊掩盖了高昂的单次有效交付成本。 - [26:28](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1588s) 传统的固定硬件预测方式已失效，企业应基于“单次交付成本 × 业务增长预期”构建动态单位经济学预测模型，并配合准实时的分钟级成本观测监控。 - [31:06](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1866s) 复杂的模型推理可能在幕后触发数百次子请求与数据流转，AI 项目立项阶段必须严格将固定算力成本与动态推理步数、往返请求等变动成本彻底拆解。 - [34:49](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=2089s) 警惕元成本倒挂陷阱：为了寻找最优模型而发起的自动化路由或评估测试（Evals），其消耗的 Token 和调用开销可能直接超过直接调用高规格模型或生产系统本身的成本。 - [34:49](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=2089s) 高能效架构应将系统中所有可预测、可重复的确定性业务逻辑坚决移出 LLM，转由常规代码实现，大模型仅保留处理非确定性核心。 - [37:43](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=2263s) 开源与自建模型并非“完全免费”，其后续的安全合规审查、架构审计及运维成本极其沉重；在达到极大规模前切勿盲目自建，应优先选用全托管服务。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=4s) 圆桌开场：从实验走向生产中的 AI 经济学 主持人 Ankur Saxena（TDK Ventures 投资总监）首先开场，介绍了基金对前沿深度科技（Deep Tech）以及 AI 与计算领域的投资聚焦。随后，两位嘉宾依次进行自我介绍。Erik Peterson 介绍自己是 CloudZero 的创始人兼首席技术官（CTO），并将 CloudZero 定位为 AI 与云原生环境的“财务控制平面”（Financial Control Plane），其核心使命是帮助企业理清构建 AI Agent、AI 系统及工具采纳背后的经济学问题，最大化投资回报率。紧接着，Abhinav Mahajan 介绍了自己就职于总部位于田纳西州的预制房屋制造巨头 Clayton Homes，担任 AI 技术交付负责人。Clayton Homes 是一家垂直整合型企业，业务涵盖金融、零售、供应链、制造以及保险等诸多领域。这种高度集成的企业架构，使他们在过去几年推进企业级 AI 实际落地时，能够全方位洞察不同业务板块所面临的摩擦与挑战。 Ankur 指出，在过去几年中，企业 AI 已经完成了从概念验证、实验到生产环境部署的转变。如今的核心疑问已经不再是“AI 能否创造价值”，而是“如何以财务上可持续的方式交付价值”。企业高管层目前都在努力平衡两个核心诉求：一方面必须争分夺秒地捕捉 AI 带来的竞争红利，另一方面又绝不能在 AI 项目上盲目挥霍巨额预算。 ```mermaid flowchart LR accTitle: AI投资决策的双重平衡 subgraph Balance["高管层的核心考量"] Speed["快速推进<br/>抢占 AI 业务红利"] <--> Budget["成本可控<br/>避免巨额财务失控"] end Balance --> Core["核心目标：单位经济效益与可持续 ROI"] ``` --- ### [02:25](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=145s) AI 时代效能定义的重构：从 GPU 利用率到业务产出 当 Ankur 询问当前最优先考虑的 AI 基础设施议题时，Erik Peterson 直言当前的节奏已经从“按季度规划路线图”被强行压缩到“一周内交付一整年的工作成果”。管理层要求在极短时间内证明投资回报。在云计算时代，基础设施优化的黄金法则是追求 100% 的资源利用率（Utilization）；然而在按需消耗、按 Token 计费的 AI 时代，“利用率”这一指标已然失效——“100% 的利用率完全可能是 100% 的浪费”。 董事会和管理层根本不在乎底层 GPU 的利用率高低，他们唯一关心的指标是：每次产出成果的成本是多少（cost per outcome）、每项业务举措的成本是多少，以及 AI 究竟让哪些业务跑得更快、交付了哪些真正独特的新价值，而非仅仅停留在“每天节省几美元或几小时”。AI 领域浪费的隐蔽之处在于，任何人都可以随意发起极其昂贵、吞噬 Token 的实验，但并非所有实验都具有商业价值。企业必须将每一笔 Token 支出与实际的商业产出紧密锚定，而非关注基础设施本身的账单。 - **云时代优化指标**：追求服务器与计算资源的 100% 饱和利用率。 - **AI 时代效能指标**：关注单位产出成本（Cost per outcome）与业务回报率，而非硬件占用率。 - **实验浪费风险**：门槛降低导致昂贵的 AI 实验泛滥，缺乏商业价值转化的实验即为纯粹的成本损耗。 --- ### [04:39](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=279s) “自研还是外购”之争的逆转与全员工程师时代 Abhinav Mahajan 对 Erik 的观点表示强烈赞同，并指出传统企业软件（如 Salesforce 等大型平台）往往从未真正衡量过单座位或单许可证的实际 ROI，过去大家仅仅将外购视为节省自研机会成本的手段。但随着软件生成成本的大幅暴跌，非纯技术驱动型企业的核心逻辑已经发生根本性逆转：从过去能买就买，转变为现在的**自研优先（Build vs. Buy）**。 Abhinav 强调，自己的职责是推动 AI 从“Demo 演示”走向“生产部署”（Demo to Deployment），在真实生产中通过“关停还是扩展（Kill or Scale）”的指标来验证单位经济效益（Unit Economics）。过去软件工程师常因隔着产品组合团队而与业务利益相关者脱节，但现在这种模式被彻底颠覆：只要拥有深厚的领域业务知识（Domain Knowledge），无论是工程师、产品经理还是业务分析师，都能凭借自身的业务判断力构建壁垒，借助 AI 快速交付价值，实现“全员皆为构建者（Everyone is a builder）”。 ```mermaid flowchart TD accTitle: 软件研发模式转变：传统外购对比 AI 时代自研 subgraph Traditional["传统 IT 模式"] A["采购现成 SaaS 平台"] --> B["按席位/许可证付费"] B --> C["难以精确衡量单点 ROI"] end subgraph Modern["AI 时代模式"] D["软件生成成本骤降"] --> E["企业自研内部 AI 平台"] E --> F["人人皆是构建者<br/>领域知识直接转化为应用"] F --> G["按业务成果衡量单位经济效益"] end ``` Erik 对此回应称自己“极其强烈地赞同”，并风趣地提到自己公司连人力与文化负责人（Head of People and Culture）都找他申请 GitHub 账号来自己写软件了。如今整个软件工程的定义已经重写： - **云时代**：每一次工程技术架构决策，本质上都是一次“采购决策”。 - **AI 时代**：员工的每一个日常动作——无论是写邮件还是做 PPT——背后都在调用模型消耗 Token，相当于“每一次决策与思考都在直接花钱”。思考本身已被货币化，因此企业更需要建立单位经济效益的精细核算意识。 --- ### [10:08](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=608s) 破解影子 AI：构建兼顾安全、身份与可观测性的统一平台 针对主持人关于实际落地中成本优化与价值落地的案例提问，Abhinav Mahajan 分享了 Clayton Homes 当前的战略举措。目前几乎所有大型企业面临的最大痛点是**安全治理与“影子 AI（Shadow AI）”的泛滥**，首席信息安全官（CISO）对此极其担忧。 为了从根本上解决这一难题，Clayton Homes 正在自研内部统一平台，旨在一次性解决三大核心挑战： 1. **安全挑战（Security）**：确立合规边界，阻止未受监管的第三方模型调用。 2. **身份认证挑战（Identity）**：统一权限管理与用户/Agent 身份鉴权。 3. **可观测性挑战（Observability）**：全面监控调用成本、Token 消耗与性能瓶颈。 Abhinav 说明，这并非晦涩高深的高阶技术，而是企业软件工程的民主化实践。通过自建统一的基础平台，企业技术人员可以直接在受控基座上快速搭建各类业务 Agent。这种模式从根本上消除了传统大企业由于官僚审批流程导致的重复建设问题——团队只需要通过一次严格的平台级安全与治理合规审批，后续所有构建在平台之上的 AI 应用即可直接继承其合规性与治理能力，从而实现规模化的高效交付。 ### [11:51](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=711s) 构建企业统一 AI 平台：数据治理与身份链路控制 在企业内部推进 AI 时，避免各部门各自为战是控制成本与风险的前提。如果能够建立一个共享的企业级平台，所有人就可以复用同一套技术栈，而无需在孤岛中重复造轮子。在这种共享架构下，任何业务团队都可以快速参考现有标准来构建自己的 Agent、工具以及接入数据源。毕竟“没有数据，AI 本质上一无是处”（AI is actually nothing without data），该平台的核心使命，正是解决如何安全地将业务数据和上下文注入给 AI Agent。 在过去的传统软件采购思维中，企业往往倾向于直接外购成熟平台（比如直接买 Claude 或各类商业 SaaS），认为自研基础平台并不构成自身的核心护城河。然而自建统一底座的价值在于掌控力：企业能够完全掌控并治理 AI，建立专属的策略引擎（Policy Engine），并构建端到端的身份链（Identity Chain）。通过身份链，系统可以清晰追踪代表用户执行操作的 AI Agent 的全链路数据血缘（Lineage）。尽管这是一个在技术上极具挑战的复杂问题，但它彻底颠覆了企业治理 AI 的底层逻辑。 --- ### [13:14](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=794s) 自研的诱惑与代价：“软件是一生的承诺” 拥有构建能力并不等同于应该去构建一切。“能力越大，责任越大”（With great power comes great responsibility），很多技术团队往往难以抵挡“重新造一套属于自己的 CRM”的诱惑。但核心问题始终在于：团队追求的具体业务成果是什么？为了这个成果愿意付出多大代价？ ```mermaid flowchart TD accTitle: 软件自研的隐性成本与治理风险 accContext: 展示从盲目自研到背负长期维护与数据责任的成本传导路径 subgraph BuildRisk ["自研诱惑与现实代价"] A["自研系统冲动<br/>(例如自己造 CRM)"] --> B["耗费数万至数十万美元<br/>开发与维护成本"] B --> C["系统产生错误数据<br/>或半成品搁置"] C --> D["董事会/管理层问责<br/>声誉与财务双重损失"] end subgraph BestPractice ["正确使能路径"] E["业务专家<br/>(懂真实痛点)"] --> F["提供核心原语与准确数据<br/>(Primitives & Context)"] F --> G["聚焦业务价值与确定性回报"] end ``` 无论代码是由 AI 编写还是工程师手写，“优秀的软件都是一生的承诺”（Great software is a lifetime commitment），后续需要持续的维护。如果花费数十万美元自研系统，最终站在董事会和 C-Suite 高管面前汇报时却发现数据漏洞百出，任何人都无法将这种问责制推卸给 AI。缺乏准确数据流动的半成品系统不仅极其昂贵，还会重创团队声誉。因此，企业必须时刻评估时间与资金的投产比，防止陷入项目半生不熟却耗资不菲的泥潭。 - **问责不可外包**：AI 可以辅助编写系统，但系统产生错误数据的业务责任无法转移给机器。 - **赋能业务专家**：与其让技术人员闭门造车，不如为最懂业务痛点的一线专家提供核心技术原语（Primitives）与精准数据支撑。 - **隐性维护成本**：自建系统的长期运维、数据校验与重构开销，往往远超初期的开发预算。 --- ### [15:36](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=936s) 业务差异化与 AI 缝合能力：向“单一管理入口”演进 企业必须清醒认识到自身的商业差异化所在。以 Clayton Homes 为例，其核心业务是“建造和销售房屋”，而非成为一家基础软件公司。过去，企业即便要在 AWS 或 Azure 等主流公有云上将不同服务拼接整合，也必须重金寻找具备特定领域专长的高级工程师；如今，AI 降低了技术门槛，团队可以直接借助 AI 自动化缝合各种云服务与工具，大幅削减了对单点专业技能的依赖。 从长远来看，AI 的终极价值在于对异构数据与分散平台的深度统一。未来的工作形态将不再需要员工在数十个复杂的独立数据库或 SaaS 系统之间频繁切换，而是演变为通过一个统一的平台或界面完成所有业务交互。当上下文切换（Context Switching）带来的损耗被消除，AI 将作为中间层自动向底层系统写入和提取数据，这甚至可能引发行业重新思考：企业未来是否还需要为昂贵且复杂的专用数据库软件持续支付高额授权费用。 --- ### [17:25](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1045s) 重新定义单位经济学：警惕“每 Token 成本”与杰文斯悖论 在评估 AI 基础设施投资时，仅看总成本几乎毫无意义，必须引入单位经济学（Unit Economics）思维。然而，紧盯“每 Token 成本”（Cost per Token）是一种危险的误导指标。 ```mermaid flowchart LR accTitle: 杰文斯悖论在 AI 算力中的表现 accContext: 展示单价下降反而导致总体消耗与成本激增的逻辑 A["模型推理效率提升<br/>单 Token 成本持续下降"] --> B["应用门槛降低<br/>业务调用量指数级激增 (杰文斯悖论)"] B --> C["系统复杂度提升<br/>返工与重构成本增加"] C --> D["必须转向业务产出指标<br/>(例如：每建一套房的 AI 成本)"] ``` “杰文斯悖论”（Jevons Paradox）在当前的 AI 浪潮中表现得淋漓尽致：随着模型推理效率的提高和单 Token 成本的下降，Token 的整体消耗量不仅没有缩减，反而呈爆炸式增长。如果将优化重心放在单 Token 价格上，只会让团队误入歧途。 真正的单位经济学必须回归业务本质，直接与企业交付的最终成果挂钩： 1. **明确核心业务单元**：必须进行直击灵魂的审视——“我们公司到底是靠什么赚钱的？”例如 Clayton Homes 的衡量基准是“每建造一栋房屋所需的计算成本是多少”，客户服务团队应衡量“每处理一次客户对话的成本是多少”。 2. **追踪边际成本趋势**：监控这些业务层面的单位成本是随着规模化逐步递减，还是因为系统架构日益臃肿复杂而不断攀升。 3. **纳入返工与重构开销**：软件永远不会真正“完工”，只会被废弃或重构；架构迭代、代码返工所消耗的人力与算力必须全额计入单位业务成本中。 --- ### [19:43](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1183s) 云计算经济学范式转移：100% 算力利用率可能是 100% 的浪费 过去数年支撑企业 IT 运转的传统云财务运营（FinOps）策略在生成式 AI 时代已经失灵。传统的云成本优化逻辑专注于寻找低利用率资源、清理无用闲置实例（即“没人在房间时把灯关掉”以节省开支）。但在大模型时代，这种基于硬件资源指标的方法论遭遇了根本性瓶颈。 在生成式 AI 场景下，如果企业采购了一批极为昂贵的 GPU 服务器，并成功将其计算利用率跑满至 100%，但若这些算力产出的内容未与具体的商业成果或业务价值相绑定，那么这“100% 的满载利用率本质上就是 100% 的纯粹浪费”。 传统 FinOps 团队习惯于紧盯服务器规格、存储容量与网络吞吐（Speeds and Feeds），但在 AI 时代，企业必须将**资金消耗数据**与**业务遥测数据（Telemetry）**深度结合。通过将成本与团队实际想要达成的业务上下文相链接，企业才能准确判断某项算力开销是否值得发生，彻底摆脱单看硬件利用率的传统误区。 --- ### [22:15](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1335s) 企业账单的现实困境：从结构化明细到黑盒 Token 消耗 从传统云计算转向生成式 AI 后，企业在实际财务核算上面临着巨大的账单黑盒挑战： | 维度 | 传统云计算时代 (Cloud 1.0) | 生成式 AI 时代 (GenAI Era) | |:--- |:--- |:--- | | **账单透明度** | 高度明细化（Itemized Bill），清晰列出计算实例、存储、网络等具体事务开销 | 极其抽象，供应商账单通常仅显示一行“消耗了 100 万个 Token”，缺乏业务上下文 | | **成本可预测性** | 随着业务规模化扩张，单位资源采购成本具有确定性的下行趋势 | 受到上下文窗口膨胀及 Agent 多轮调用的影响，调用量难以精确预估 | | **成本结构重心** | 传统计算实例与底层存储等固定/半固定基础设施占据支出大头 | 周边外围架构（Harness）的固定基建成本极低，**80% 至 85% 的实际支出完全集中在模型 Token 消耗上** | 尽管企业在工程架构上可以将模型外围的调度层、防护栏（Harness）优化得极其高效，但模型本身吞吐的 Token 才是决定账单生死的绝对主力。这种成本结构的剧变，迫使技术管理者必须将治理焦点从底层基础设施全面转向模型调用逻辑与上下文精简。 ### [23:24](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1404s) 衡量商业价值：利用率不再等于价值（Utilization is Not a Proxy for Value） 在评估 AI 架构的工作流时，团队必须基于具体的业务结果进行基准测算（Baseline），而不是仅仅看活动量（Activity）。Abhinav Mahajan 指出，传统思维中“80/20 分割”的成本分布往往让人震惊：如果构建一个聊天机器人，它回答了 20,000 条消息，每次交互平均花费 3 美分，但最终只成功分流（Deflect）了 100 个原本需要人工客服处理的工单，那么在计算真实投资回报率（ROI）时，这笔总成本只能与那 100 个有效分流的工单进行比对，而不是这 20,000 条无意义的闲聊互动。只有通过界定业务能够承受的“变动成本范围”，才能设计出真正具备性价比的未来系统架构。不管选择哪家云服务商或推理模型 API，“掌控未来系统王国大门钥匙的，永远是你自己的架构设计”。 Erik Peterson 对此深表赞同，并给出了一个极具警示意义的核心论断："Utilization is no longer a proxy for value"（利用率不再是价值的代名词）。在传统基础设施中，资源被充分利用可能意味着高产出，但在生成式 AI 时代，这种假设已经破裂。例如，软件研发团队可能因为引入 AI 编码助手而使得代码提交量（Commits）和拉取请求（PRs）激增 10 倍，但如果这些代码没有转化为切实的业务成果，甚至只是生成了企业根本不需要的代码，那么这种所谓的“高利用率”实际上只是虚假的繁荣。传统软件工程所追踪的开发指标全面攀升，但真实的商业价值却并未同步增长。 ```mermaid flowchart TD accTitle: 聊天机器人成本与价值转化漏斗 subgraph Cost["成本产生端 (Activity)"] A["20,000 条对话消息\n(平均 $0.03 / 次)"] --> B["总基础设施/API成本: $600"] end subgraph Outcome["实际价值转化端 (Value)"] A -.-> C{"有效分流人工工单?"} C -- "99.5% 闲聊/无效交互" --> D["无实际业务减负"] C -- "0.5% 真正分流" --> E["100 个工单被分流\n(实际单工单解决成本: $6)"] end ``` --- ### [26:28](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1588s) 基于单位经济学（Unit Economics）重塑 AI 成本预测体系 面对 AI 基础设施成本极难预测的痛点，Erik Peterson 指出，过去那种“预估需要多少服务器和硬件支出”的传统预测方式已经彻底失效，取而代之的必须是**单位经济学**（Unit Economics）： 1. **明确单次交付成本（Unit Cost）**：例如构建一个处理系统，测算出交付每个业务部件（Widget）的单价为 20 美分。 2. **结合业务端增长预期**：从业务部门获取真实的业务量增长模型（如第一季度预计新增 1,000 家客户或实现 30% 增长）。 3. **推导动态预测**：将单价乘以业务量即可得出高准确度的预算模型，而非死板地按固定硬件资源做预算。 这种方法贯穿了 CloudZero 成立十多年来的底层实践，在过去几年被直接套用到 AI 领域。依赖传统季度历史支出做线性推断的企业往往遭遇严重的预算失控（例如 AI 支出在一年内从 10 万美元暴增至 1,000 万美元）。由于 CFO 和 CIO 习惯于按季度（Quarterly）节奏进行财务审视，而 AI 支出的飙升速度远超企业的观测窗口期，往往等季度结束账单出来时才惊呼失控。 与企业内部对出差吃牛排大餐的报销单进行严苛审查形成鲜明讽刺的是，如今任何一名工程师都有权限在几分钟内通过调用 Token 和云资源花掉成千上万美元。Erik 呼吁企业必须实现准实时的监控与观测，同时将“决策权与成本责任下放到真正在一线做技术选型和架构设计的人手中”。 --- ### [31:06](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1866s) 拆解 AI 工作流的固定成本与推理变动成本 Abhinav Mahajan 针对预测 AI 成本分享了 Clayton Homes 的实操落地方案。他们为每个试点的 AI 用例建立结构化评估表，将成本严格拆解为**固定部分**与**变动部分**： | 成本构成维度 | 包含的具体考量因素 | 关键技术陷阱与风险 | |:--- |:--- |:--- | | **固定成本 (Fixed Cost)** | • 支撑基础服务的云实例（如全天候运行的 2 台 EC2 实例）<br>• 常驻后台的基础设施支出 | 相对容易测算，按 7×24 小时运行时间进行线性估算即可。 | | **变动成本 (Variable Cost)** | • 推理步数（Reasoning Steps）与决策调用链<br>• 单次交互后端的往返请求次数（Round Trips）<br>• 数据出向流量（Egress）模式与智能层依赖 | 许多人误以为调用模型只是单次请求与响应，但像 Claude 等平台以及复杂的 Agent 架构在幕后可能发起数十甚至数百次子请求以确保结果质量与准确性。 | Abhinav 强调，AI 与传统云资源的自动伸缩完全不同，模型自身的智能层复杂度以及跨平台的数据流转（Egress）都会带来巨大的成本不确定性。因此在项目立项需求准入（Intake）阶段，就必须深入剖析这些架构细节。 --- ### [33:53](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=2033s) 当项目偏离轨道时的成本反思 主持人 Ankur Saxena 指出，尽管有详尽的预测体系，但 AI 项目依然极容易发生“脱轨”和成本失控。Erik Peterson 坦言这是一个充满教训的“目标密集区”（Target-rich environment）。在早期项目中，团队最容易犯的错误就是没有彻底算清乘法数学题：看似单次调用仅花费 20 美分的轻量任务，一旦在线上被调用 100 万次，就会迅速变成吞噬数万甚至数十万美元的昂贵实验。如果没有建立端到端的成本跟踪与调用频次预警机制，创新实验就会迅速演变为财务危机。 ### [34:49](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=2089s) 动态模型路由与自动化评估中的“成本倒挂”陷阱 在构建 AI 应用时，工程师极易忽视单次调用的微小开销在规模化之后的“乘数效应”（Multiplicative Effect）。Erik Peterson 分享了一个团队踩过的真实案例：为了在推理时选择最划算、最高效的模型，团队专门开发了一项自动化决策能力，通过调用外部系统实时查询各个模型的报价与性能参数，试图在请求级实现动态最优路由。在单次请求的微观视角下，这套机制看起来精妙且极具成本效益；然而一旦接入生产流量，团队惊愕地发现：为了搞清楚“究竟该用哪个模型回答问题”所消耗的往返开销与 Prompt Token 成本，竟然直接超过了直接调用一个高规格模型来作答的费用。单次事务的综合成本瞬间翻倍，这种隐藏的系统性开销如果不在微观单位成本（Unit Cost）层面严密监控，一旦放量就会引发毁灭性的账单灾难。 ```mermaid flowchart TD accTitle: AI系统中的成本倒挂与优化决策流 subgraph RoutingTrap ["模型路由中的元成本陷阱"] A["用户请求接入"] --> B["调用外部系统查询各模型价格与指标"] B --> C{"元决策耗费的 Token 与算力成本"} C -->|"元决策成本 > 模型直接推理成本"| D["成本倒挂：优化决策本身成为最大支出"] end subgraph ArchitectureDesign ["正确的高能效架构范式"] E["解耦业务逻辑"] --> F["可预测/重复模块：移出 LLM，转为确定性代码"] E --> G["非确定性核心：保留由 LLM 处理"] H["构建实时成本遥测反馈"] --> I["开发者以分钟级敏捷开展 A/B 测试"] end ``` 这种成本失控同样在开发者日常迭代与模型升级中频繁上演。Erik 强调，绝不能等到月末才把云账单甩给开发团队，而必须把成本遥测做成分钟级的实时反馈，让开发者在编码与调试当下就建立起 A/B 测试的经济直觉。当前大模型迭代速度极快，“就在我们坐在这里开会的当下，可能就有新版本在发布”，许多团队往往下意识认为版本号更高就代表更好、必须无脑升级，但这其实是一个巨大的误区。盲目追新不仅未必带来业务收益，还会引发自动化评估（Evals）的成本黑洞。有团队为了验证新版本是否正常工作而部署了大量评估测试，直到有人突然核算开销才发现：光是跑这些自动化 Evals 所花掉的钱，就已经远远超过了被测试的生产系统本身的运行成本。Erik 指出，这种荒谬现象的根源在于工程师试图用一层“令人安心的确定性外壳”去包裹本质上不可预测的大模型。真正明智的做法是将系统中所有可重复的确定性逻辑坚决移出 LLM，改用常规代码实现，并在项目初期严密测算基于调用量放大的单位经济模型，否则“迟早会有人来替你算这笔账，而那将是一个极其痛苦的过程”。 - **模型路由悖论**：为寻找最便宜模型而发起的元决策（Meta-reasoning）调用成本，往往高过直接使用单一模型的推理开销。 - **实时成本反馈**：开发者需要分钟级（Same-minute）的成本可见性，以此驱动理性的 A/B 测试与版本选型，而非被动等待月末账单。 - **评估成本超支**：盲目追求确定性保障会导致 Evals 阶段的算力支出反噬生产系统本身的预算。 --- ### [37:43](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=2263s) 开源网关与自建模型的隐性维护成本 团队在探索前沿技术架构时，极易被开源方案“免费且自主可控”的表象所误导，从而陷入沉重的运维与合规泥潭。分享者结合自身团队的经历指出，他们此前曾尝试采用开源网关来统一纳管与治理 MCP（Model Context Protocol）服务器，构建企业内部的模型上下文注册中心。最初团队认为开源方案唾手可得、开箱即用，但在实际落地中，这套开源底座带来的隐性维护成本迅速暴露——由于涉及开源组件的底层安全与代码变动，系统每一次微小的功能调整或依赖升级，都必须重新触发漫长严苛的企业安全审查（Security Reviews）与架构合规评审（Architectural Reviews）。 面对开源自建所带来的巨大工程摩擦与流程内耗，团队最终果断止损，调转方向转向即插即用的全托管（Managed）网关服务。分享者坦言，技术团队往往容易陷入“技术上我们全都能自己做”的过度兴奋状态，进而对非代码层面的维护成本视而不见。这种教训在当前大热的开源大模型自建部署（Self-hosting）中同样适用：在企业没有达到足够的调用规模（Scale）之前，千万不要盲目自建托管开源模型；必须先在托管和商业 API 阶段彻底验证业务价值与规模真实性，再去评估自建基础设施的投资回报率。 - **开源维护黑洞**：开源软件的免费仅体现在代码获取层面，后续的安全合规审查、架构审计及日常运维会带来高昂的企业流程成本。 - **架构决策转向**：在非核心差异化环节优先选择全托管即插即用方案，避免团队精力被基础设施合规问题过度分散。 - **自建模型红线**：“如果没有达到真正的业务规模，连想都不要去想自建托管开源模型”，先跑通业务规模是探索自建算力的唯一前提。 ## Notable quotes > [02:25](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=145s) 100% 的利用率完全可能是 100% 的浪费 — Erik Peterson > [04:39](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=279s) 软件生成成本的大幅暴跌，非纯技术驱动型企业的核心逻辑已经发生根本性逆转：从过去能买就买，转变为现在的自研优先 — Abhinav Mahajan > [04:39](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=279s) 全员皆为构建者（Everyone is a builder） — Abhinav Mahajan > [11:51](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=711s) 没有数据，AI 本质上一无是处（AI is actually nothing without data） — Abhinav Mahajan > [13:14](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=794s) 优秀的软件都是一生的承诺（Great software is a lifetime commitment） — Abhinav Mahajan > [23:24](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1404s) Utilization is no longer a proxy for value（利用率不再是价值的代名词） — Erik Peterson > [23:24](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1404s) 掌控未来系统王国大门钥匙的，永远是你自己的架构设计 — Abhinav Mahajan > [26:28](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=1588s) 决策权与成本责任下放到真正在一线做技术选型和架构设计的人手中 — Erik Peterson > [37:43](https://www.youtube.com/watch?v=ujLiNFEqyXE&t=2263s) 如果没有达到真正的业务规模，连想都不要去想自建托管开源模型 — Erik Peterson ## Mentioned Ankur Saxena, Erik Peterson, Abhinav Mahajan, Raghavendra Bagalkoti, TDK Ventures, CloudZero, Clayton Homes, Barclays, Salesforce, GitHub, AWS, Azure, Claude, EC2, MCP (Model Context Protocol) ## Video description Ankur Saxena - Investment Director, TDK Ventures Raghavendra Bagalkoti - Vice President - Capital Markets Data Platform, Barclays Abhinav Mahajan - Technical Delivery Leader - AI, Clayton Homes Erik Peterson - Founder & CTO, CloudZero As enterprise AI deployments mature from experimental pilots to mission-critical applications, managing the associated cloud infrastructure costs has become a top priority for the C-suite. This panel explores how organizations are balancing aggressive AI innovation with sustainable cloud spending. Enterprise leaders will share their strategies for forecasting compute costs, optimizing cloud resource allocation, and proving the business ROI of their infrastructure investments. Discover how to architect a cloud strategy that scales your AI capabilities without breaking your budget. [Source](https://www.youtube.com/watch?v=ujLiNFEqyXE)

[2:25] Erik Peterson: 100% 的利用率完全可能是 100% 的浪费 [4:39] Abhinav Mahajan: 软件生成成本的大幅暴跌，非纯技术驱动型企业的核心逻辑已经发生根本性逆转：从过去能买就买，转变为现在的自研优先 [4:39] Abhinav Mahajan: 全员皆为构建者（Everyone is a builder） [11:51] Abhinav Mahajan: 没有数据，AI 本质上一无是处（AI is actually nothing without data） [13:14] Abhinav Mahajan: 优秀的软件都是一生的承诺（Great software is a lifetime commitment） [23:24] Erik Peterson: Utilization is no longer a proxy for value（利用率不再是价值的代名词） [23:24] Abhinav Mahajan: 掌控未来系统王国大门钥匙的，永远是你自己的架构设计 [26:28] Erik Peterson: 决策权与成本责任下放到真正在一线做技术选型和架构设计的人手中 [37:43] Erik Peterson: 如果没有达到真正的业务规模，连想都不要去想自建托管开源模型

[2:25] Erik Peterson: 100% utilization could be 100% waste. [5:28] Erik Peterson: My head of people and culture is building software... they came to me and asked me for a GitHub account and I looked at them and said, 'Really?' [10:08] Erik Peterson: Just because you can build a thing doesn't necessarily mean you should. [19:23] Erik Peterson: Utilization is no longer a proxy for value.

[0:04] Let me start with a quick introduction. I'm  Ankur Sakenna. I'm an investment director [0:07] at TDK Ventures. We are a deep tech focused  venture capital fund. Uh and I lead our AI and [0:15] uh computing initiatives in terms of venture  investments. Uh I'll turn it over to Eric to [0:21] introduce himself briefly. Yeah, thank you Anker.  So I'm the CTO and founder of CloudZero. Cloud [0:26] Zero is the financial control plane for  AI and cloud. And uh the thing that we're [0:32] focusing on today is is quite simple. What are  the economics around building AI agents, systems, [0:39] adopting these tools? How can we help people  maximize their return on that investment? And [0:45] uh we're going to get into that topic in a big way  today. Well, about me, uh, my name is Abhinav and, [0:52] uh, I work for, uh, a manufactured  housing company called Clayton Homes. Uh, [0:59] we are based in Tennessee and we are actually  going through this journey. um you know probably [1:07] from last couple of years like how do we actually  do practical enterprise implementation of AI and [1:15] like we have vertical integrated organizations.  So we have finance, retail, supply, manufacturing, [1:23] uh insurance you know all under one umbrella.  So technically we see the friction you know of [1:30] actually implementing a use case through all  uh these different enterprise you know uh AI [1:36] lens and that's exactly what I'm actually going  to share today with you guys. Thank you. So over [1:43] the last few years we've seen that AI has moved  from production to experimentation. Um today the [1:49] question isn't whether AI can deliver you know  value but the question is how can it do it in a [1:54] very sustainable way financially sustainable way  right so every executive that I know is balancing [2:00] two questions uh one is how how fast can they move  to capture the benefits of AI and then balance [2:08] that with how can they not spend a ton of money  while trying to extract value out of AI projects. [2:14] So this this panel will talk about how to kind  of strike the right balance between pursuing AI [2:19] initiatives but in a cost-effective way. With that  let me start by asking the panelists to share what [2:25] is their topmost AI infrastructure priority  today. Well, there you know what what isn't a [2:33] priority today, right? We went from having kind  of structured thoughts about what we were going [2:38] to work on quarter by quarter to let's see if we  can compress all of that into a week. And uh uh [2:45] the demand from the seauite is is essentially if  we're going to show return on this investment, [2:51] we've got to figure out how we take that year of  work and maybe compress it into that week. So for [2:56] for us it's really trying to better understand  what actually defines optimization in this [3:04] era because in the in the cloud infrastructure  era optimization was uh pursuing 100% utilization. [3:12] Utilization is not a metric that makes any sense  in the AI era. It's all consumed on demand. 100% [3:17] utilization could be 100% waste, right? And your  board doesn't care about uh GPU utilization or or [3:24] things like that. What they really care about  is what's the return on this investment as [3:29] it relates to you know the cost per outcome,  the cost per uh you know uh initiative taken [3:37] um what part of the business is now moving faster  or delivering more value um and not just you know [3:46] saving a few bucks or saving a few hours in the  day but actually delivering something unique. So, [3:52] we're driving into a new world where we have to  think completely differently about what it means [3:56] to be efficient. We're driving in a new world  where we have to think completely differently [4:01] about how um the um work that we start makes  sense. You know, there's a lot of very expensive [4:08] uh experiments that are that are happening. And  um that's the kind of other scary side of the the [4:15] waste equation I think for AI is because you can  start literally any experiment that you want but [4:21] not all of them are worth it right so at the end  of the day you really have to think about what [4:25] the worth of that you know token spend or that  you know that that dollar you spend in terms of [4:32] the business outcome not the infrastructure that  you spent the money on. Yeah, I I would actually [4:39] second that like it's very difficult to generally  you know measure value from technology in general [4:46] like I have a good use case. Think about like  any platform implementation that we have done [4:52] with technology. I don't think anybody has  probably baselined a metric of like if I'm going [4:59] to implement Salesforce or I'm sorry I'm taking  names but there are so many platforms like that [5:05] but when they actually get implemented I I'm not  sure how many companies actually measure ROI that [5:11] they're getting you know per license or per seat  they basically look at like implementation and [5:18] they think it is opportunity cost that they did  not had to build a platform and they could use it [5:23] and you know move the business forward. forward.  Similarly, if I talk about you know my priority, [5:28] I think the equation that has changed probably  you know uh for companies that are not technology [5:34] first is actually the buy versus build mindset.  Before companies were thinking that you know we [5:41] should actually buy a product as much as possible  because that's the opportunity cost. Um but I [5:47] think now the cost to produce software has gone  down significantly. So now the question becomes [5:53] like what can we build and as you said like how  quickly can we build this is the only question [5:59] that probably I am being asked and you know that  is exactly what I do. um I deliver AI and again [6:08] I don't talk about demos I talk about how to  go from demo to deployment and that is where [6:13] basically rubber meets the road and that's where  you decide you know kill or scale metric right [6:19] um so for us you know that's the priority it's  all about unit economics how do I actually measure [6:24] the business outcome do I really understand the  business workflow half of the challenge is that [6:30] you know uh this is what they say about software  engineers that they are usually disconnected [6:35] from their stakeholders because uh it's actually  a portfolio team you know that is interacting with [6:41] the stakeholders and then you have software  engineers who are basically delivering value [6:46] or wishes of the business you know through that  lens but I think that equation is flipping on [6:52] its head because the more domain knowledge you  have doesn't matter you're an engineer a product [6:58] owner you know a business analyst I think you  have your own judgment remote and you could [7:05] actually use that to leverage AI and deliver  value quickly. It's just the mindset of like [7:12] how quickly can we adopt adapt and you know move  the needle forward. Um and it's it's a very common [7:20] theme probably you would have heard you know from  others as well like everyone is a builder. There [7:25] is like no doubt about it. It's it's just about  you know the question is actually about like are [7:32] we actually willing to apply ourselves and if we  do that right like nothing is there that we cannot [7:38] do. So from our perspective you know what we are  trying to do is we are building platform you know [7:44] inhouse so that we can actually move from pilots  you know to production uh and that is exactly you [7:52] know where we stand. I was gonna I was going to  ask uh well earlier when I got up here I was like [7:58] how controversial are we going to make this this  panel and I'm I'm almost ridiculously violent I [8:04] mean so violently in agreement with what you just  said there you know one the the world of AI it's [8:10] all it's a unit economics problem right you you  have to think of it that way you can't think about [8:14] the total spend you need to think about the cost  per unit of value and for many organizations they [8:21] haven't thought about it that way Um the the other  question about engineers I was going to ask was, [8:26] you know, well, who is an engineer these days? I  woke up, you know, one day and realized the whole [8:31] company is is nothing but engineers. My head of  people and culture is building software. They've [8:38] got a get they came to me and asked me for a  GitHub account and I I I looked at them and said, [8:42] "Really?" Um and um everybody in the company is  building software. So everyone's an engineer. [8:48] uh what it means to build software is completely  changed in its entirety and um we need people [8:54] thinking and seeing more importantly just you know  being conscious of what they're spending because [8:59] it used to be in the cloud era every engineering  decision was a buying decision in the AI era it's [9:06] really every decision is a buying decision right  you sit down to um write an email you're probably [9:12] spending a few pennies you sit down to um go  create a PowerPoint presentation you're spending [9:18] some money, right? And uh and literally everything  is spending money. So, it is a it is a truly [9:24] unique time where almost like thought has been  monetized, but it's super powerful in the sense [9:30] that what you're able to accomplish often exceeds  what you could have dreamed of doing in years, [9:36] right? Um but you have to measure that against the  unit, you know, the outcome that you're shooting [9:41] for and use economics, I I believe very strongly  as a as a path there. Speaking of the the build [9:49] part, the deployment part, let's get specific,  right? Can you both share any examples of real [9:55] deployments uh in the recent history where you've  faced a lot of cost optimization challenges, [10:02] how you've thought about delivering business value  and this whole unit economics kind of balance. [10:08] Yeah. So, you know, talking about I'll give you an  example. So, this is uh this is what we are doing, [10:14] you know, at the moment. So we are actually so so  the biggest thing you know that probably everyone [10:19] is struggling is like the big question is security  and governance. How are we actually going to [10:25] govern this? How can we stop shadow AI because it  is happening at scale. Um and usually like if you [10:33] have your CISOs they are really worried about  you know the security aspect of it. So what we [10:38] are actually trying to do at this point in time is  we are building a platform and again we are trying [10:44] to solve the security challenge, the identity  challenge and the observability challenge all [10:49] at the same time by building that platform. Now at  the same time like when we say you know all these [10:56] words in one sentence it seems very technical. It  is actually not. This is how software has operated [11:03] probably all the while. It's just that we are  democrat uh you know it's more democratization [11:09] of the software that probably we are handling  ourselves. So instead of actually buying a [11:15] platform it's more like building a platform and  going through that experience of actually using [11:19] AI to do that and then enabling other you know  again I'm not going to say that business you know [11:27] stakeholders but it's more about technology  stakeholders who can actually come and build [11:32] agents on the platform. The challenge this solves  like it's a repeatability issue. So today what [11:38] happens is you know on the name of governance we  have lot of you know bureaucracy everywhere right [11:43] in in the corporate sector. What this does is like  if you get one approval for one good platform what [11:51] will happen is that everybody can actually use the  same platform. They don't have to create or build [11:57] their own technical stack in isolation. then you  actually have shared architecture that anybody can [12:03] basically reference and build their agents,  build their tools, build their data sources [12:09] because at the end of the day you know um AI is  actually nothing without data. So how do you give [12:16] context or data securely to your AI agent is what  we are solving you know with this platform. Now in [12:23] conventional world you know probably nobody would  actually try to do you know a product like that [12:30] they would rather just think about like that's not  our mode let's just go and buy a platform right [12:35] like buy claude buy codeex or whatever right  whatever makes sense uh but in this scenario [12:41] what you can do is you can actually control AI  govern AI you can create your own policy engine [12:47] you can create your own identity chain where you  can actually track the lineage you know of uh [12:54] the user technically an agent which is working on  behalf of the user end to end uh it's a technical [13:01] complex you know problem to solve but this is how  it has flipped or changed the dynamic you know in [13:08] my perspective you know with it's a lot of convers  with great power comes great responsibility right [13:14] let's I'll I'll go there you know just because  you can build a thing doesn't necessarily mean [13:18] you should And I mean, how many people here in the  room have been tempted to go build their own CRM? [13:25] I know I have, right? And and the question is,  well, what is the, you know, this this notion, [13:30] I'm going to repeat it a lot, like what is the  outcome that you're pursuing? And and what are [13:35] you willing to spend to achieve that outcome?  And remember, you know, all software is um is a [13:41] lifetime. You know, great software is a lifetime  commitment whether AI built it or you built it [13:46] yourself. And you have to maintain that. So are  you ready to to stand in front of uh your seauite [13:53] your board and take accountability for the data  that's coming out of those systems? Uh I think a [13:59] lot of people don't understand the the difficulty  in maintaining the accuracy of that and the costs [14:04] to maintain that right. You can easily spend tens  of thousands of dollars or even you know hundreds [14:09] of thousands of dollars trying to build your  own CRM and then stand in front of your board [14:13] and realize the numbers are wrong. Right? And  you can have systems building things for you but [14:19] you cannot hand off the accountability. So what is  the the the cost of pursuing that when the the the [14:28] data is wrong. So what I see software transforming  itself when I see people building things I think [14:35] um there's a great power into being uh unlocking  the experts in your in your own organization [14:41] who understand the business problem that they're  trying to solve. You need to enable them with the [14:48] right primitives, core components, the accurate  data, because if that stuff isn't flowing, [14:54] then you're going to have a very expensive outcome  that is unfortunately inaccurate and it's going [14:59] to cost you uh reputation as well as money, right?  Um, but the at the end of the day, what um I think [15:09] people really deep need to think about is where  are you willing to spend your time and what is it [15:15] worth and how am I going to maximize the value or  the return on that time because it's still going [15:20] to take time, energy, and money. And if you're  not tracking it, you're quite quite frankly you're [15:29] going to find yourself with a lot of projects that  are halfbuilt, halfbaked, and uh and unfortunately [15:36] very expensive. Well, yes, absolutely. We  need to basically know, you know, what is our [15:41] differentiation, right? Like I can give you our  differentiation. We sell homes. We don't need to [15:47] build technology, right? I'm uh all I'm trying to  basically say is like if even if you had to stitch [15:53] a platform together in the past let's say AWS or  Azure you know any cloud platform that you want [15:59] to use or services that you want to use we always  looked for expertise now we don't need to look for [16:05] that expertise because we can actually leverage  AI to build you know or stitching those services [16:12] together it's that's exactly what my premise was  uh so it's it's not about that yes we should build [16:18] a CRM but if in long term if you think about  the value of AI it's actually unification of [16:27] your data and of the platforms eventually like  there would be a world where we are just inter [16:35] interacting you know with one platform that is  actually doing everything for us so technically [16:41] writing data into systems or platforms you know  that exist and probably that time you know the [16:46] question would become like do we really need to  pay for those sophisticated databases that we are [16:50] actually investing in today. Maybe yes, maybe not.  Nobody knows the answer. But I think like the way [16:57] we are heading towards things, you can see like  you can connect your clot to different platforms, [17:02] right? And then you are able to basically unleash  productivity that is exactly where we are heading [17:08] where probably context shifting will become uh  annoyance you know to humans and it's like okay [17:14] I just want a single pane of glass to do all  my work. Not sure when that's going to happen, [17:20] but but but that's it's definitely going to happen  pretty soon. Yeah. You know, so what I would add [17:25] to that is um you know, when you think about uh  [snorts] how you're going to measure these things [17:32] practically like in real world like things that  we've actually built when we um when we sit down [17:37] and just try to look at the total cost, that's not  a very useful number. When you get at the unique [17:42] economics of it, it's a useful number. But there's  two different types of of ways of thinking about [17:47] unit economics. And I want to I want to kind of  equip everyone here to think about it what I think [17:51] is the right way. You could you could walk out  of this and go, "Oh, all right. Let me go figure [17:56] out what my cost per token is." And that's not a  very practical number, right? We know already the [18:01] cost per token is decreasing. It will decrease  over time. Jeban's paradox is in full effect. [18:07] And what that means is as efficiency improves, as  the costs improve, consumption is going to go up. [18:12] And we've already seen that happen over a couple  of, you know, years now. It's hard to say like [18:18] it feels like we've all aged like a million  years probably just in this in this last month, [18:24] right? But the tracking a uneconomic like cost per  token is going to send you down the wrong path. [18:31] Tracking a cost to do what it is that you do as a  business and sometimes that's a very soularching [18:37] conversation, but you're going to have to have  that conversation. What is it that we do around [18:42] here? What does it cost to actually do that? You  know, we build homes. Okay, what does it cost to [18:47] do that? Right? Uh we process these conversations.  Okay, what does it cost to do that? Are those [18:52] costs at a unit level going down over time or are  they becoming more expensive because we're making [18:57] the systems more complex along the way? And how  much time was spent in rework or re, you know, [19:04] re, you know, architecting the whole thing?  Those have to be included as well. And this is a [19:10] ongoing kind of like optimization. Again, all all  software is uh is a lifetime commitment. You know, [19:17] it's never finished. It's abandoned. And um and  you've got to be really uh thoughtful about that [19:23] with everybody building software now. We need  to encourage our teams to really be thoughtful [19:29] around that. And uh and that's where the uni  economic conversation around value versus cost [19:35] per infrastructure token or something like that is  unbelievably uh critical. I think we'll talk about [19:43] unit economics a bit more later in the panel,  but I do want us to to recognize that the cloud [19:50] playbook that a lot of organizations depended on  for the last several years does not apply to the [19:56] world of generative AI very cleanly. Right? So  let's let's talk about where is it broken, right? [20:02] What's what's maybe one piece of conventional  wisdom from the cloud economics days that does [20:07] not apply to the generative AI AI era. I I love  this question because the you know when you sat [20:14] down to try to optimize a cloud environment, you  would go looking for areas of of load utilization. [20:19] You'd go looking for waste. You you the the  recommended advice was can I turn the lights [20:24] off when nobody's in the room? Can I can I save a  few bucks that way and optimize my workloads? And [20:30] um and the the reality is um I could pursue  100% optimization. Let's say I bought a bunch [20:37] of really expensive GPU systems and went out  to make sure that they are being used 100%. [20:44] If I am not actually thinking about the outcome  I'm trying to pursue, the value, the thing that [20:50] I'm trying to create, um I could have my entire  estate of all the compute, all the cloud resources [20:56] I have at 100% max utilization and it could be  all waste, right? It has to be tied to value. It [21:04] has to be tied to a business outcome. That's  the big big shift. and and so it's no longer [21:10] an infrastructure uh conversation. And everyone  who's been in the PHOps world or the cloud world [21:16] for a long time, this is entirely new to them.  They they're used to looking at the speeds and [21:21] feeds and the servers and the the storage and  they're not used to getting into well should [21:26] we even be doing this right now? If you have the  people who are actually, you know, engaged in the [21:34] business close to the customer building some of  these things, hopefully they can articulate what [21:40] those things are. That has to get into the system.  You have to combine the costs with the usage. We [21:47] Clazor thinks about this in terms of how much  money you're spending and then what is it being [21:51] used for the telemetry that drives that. So you  have the context of what your teams are actually [21:57] trying to achieve. Without that, you don't have  that that conversation. You're just comparing [22:03] speeds and feeds and that'll send you down the  wrong path. So the that conventional wisdom is [22:08] is is so fundamentally changed forever now cuz  100% utilization is not going to necessarily [22:15] save you money. Yeah. I'll I'll actually give  you a practical example because you know we are [22:22] a heavy cloud user, right? And again the premise  you know in the past basically was the cost of [22:30] the cloud is actually going to basically go down  with scale right and also it was easy to measure [22:39] because it was uh more transactional right you can  actually measure compute storage you know all the [22:47] things that you talked about. Yeah. So while you  had an itemized bill from your cloud provider, [22:52] now I have a bill from my uh AI provider  just says you consumed a million tokens, [22:56] right? What does that mean, right? Nobody knows.  Yes. And also like from our perspective like when [23:02] we now like put things in perspective um our like  fixed cost for infrastructure is actually really [23:09] low especially around AI workloads. The real cost  like 80 to 85% is actually tokens. So we are very [23:18] efficient in architecting you know the harness you  know around the model but the model itself is what [23:24] actually causes you know our build to actually  go up. So like that 80/20 split you know is what [23:30] blew my mind to begin with and that's where you  know it becomes more important to actually follow [23:36] the principle of like for example let's say you  have because since we're talking about value and [23:41] outcome right we really have to baseline you know  what outcome this workflow is actually generating [23:47] what is the cost you know associated with that  but also like what value is actually going to get [23:53] generated for example if we are building a chatbot  yeah it is going to actually answer questions, [23:59] but are we actually tracking like how many tickets  requests are actually going to get deflected? That [24:06] is going to basically tell us that if I had  20,000 chat messages coming back out of it, [24:12] it costed me 3 cents per chat on average,  but it only deflected 100, you know, [24:17] requests that probably would have gone to someone.  Then technically those 100 requests is what you [24:22] have to compare against the cost, the overall  cost, not 20,000. And then you can probably [24:28] figure out okay is this activity or is there is  this value. Um and this is how you know we have [24:35] to think about different workflows like whenever  you look at a workflow you have to figure out a [24:40] range like what is the range of variability that  I'm comfortable spending you know my money on [24:47] because that will actually help you architect  the systems of the future and today like I can [24:54] confidently say the magic is actually going to  be in that architecture that you are going to [25:00] build doesn't matter which cloud provider it is  does doesn't matter you know which AI inference [25:05] provider it is at the end of the day you control  the keys you know to the kingdom and in how you [25:11] architect you know those systems of the future  I mean if I could you sum it up in in kind of [25:18] one sentence I'd say if you take nothing away  take this that utilization is no longer a proxy [25:24] for value and you cannot um look across your your  your state and go okay everything's is being used, [25:33] I must be delivering immense amounts of value.  And that applies not just to infrastructure, [25:38] but anyone who's been writing code has almost  certainly seen this explosion. Oh, we're we're [25:44] doing 10x number of commits and PRs than we've  ever done. Well, clearly something amazing must [25:51] be happening. No, utilization is not a proxy for  value. So if your entire team is now writing, [25:58] you know, 10 10 times more code, well, is that  good code? Are they creating things that we even [26:07] need? What's the actual business outcome of that?  because every team I've talked to that that used [26:14] to track the classic kind of developer software  metrics is seeing all those things go up but the [26:21] actual outcomes not moving commensurate with that  utilization is not a proxy for value makes sense [26:28] very well said so we've talked a lot about the  business value the outcomes that we need to aim [26:34] for we've briefly touched upon the cost as well  but let's delve a little deeper into the cost side [26:41] um predicting the costs of running AI models and  just the whole AI deployment piece, you know, the [26:46] cost side is very very difficult to predict. So  would love to hear your you know best practices, [26:52] any battle scars in terms of how you went about  predicting costs of your AI deployments and and [26:58] obviously the challenges along the way you  faced. Um so I would the old way of building [27:03] a forecast looking at your you know what kind of  infrastructure are we going to need? What's the [27:07] outlay on this? Um doesn't work anymore. You've  got to um you I'm going to tap tie back to to unit [27:15] economics again here. You have to actually really  think about what is the unit cost to deliver that [27:20] value. What is it going to be and then track that?  You have to build a forecast around that. So first [27:26] okay we're building a let's say we're building a  system to um deliver widgets right you know it's [27:32] like all right so then what's my cost per widget  for this system? Um 20 cents. Fantastic. Okay. So, [27:39] what's the volume that I'm expecting over time?  You know, what's what is the business expecting? [27:44] I guarantee you someone in the business has  an idea. Well, this should grow 30% or we're [27:47] expecting massive, you know, customer growth in  the first quarter, thousand new customers. Okay, [27:52] multiply those things. That's your forecast.  Don't build a forecast off of, oh, we're going [27:58] to spend this much on infrastructure. Everything  is on demand. Um, and that path will give you an [28:05] actual forecast. In fact, this is in many ways  I'll give you a very practical example tied to [28:10] even cloud zero's like founding journey from the  very beginning when I started the company you [28:17] know over 10 years ago we actually started with a  problem that we had to forecast how much a system [28:22] was going to cost before we had any customers for  it and we needed to do something that was going [28:26] to require massive scale and we had never built  anything like that. This is over a decade ago. So, [28:33] we were looking at that problem going, well, all  right, well, how are we going to figure this out? [28:36] We decided, well, I'm going to take this unit  metric and then I'm going to forecast it against [28:42] what I think is the volume and then from there  I'll have a number. And it turned out that was [28:47] wildly more accurate than anything else. And we've  been applying that to cloud for the last decade. [28:52] Now, we're applying it to AI in just the last  few years. And every single one of our customers [28:57] has seen the a more predictable more um you know  understandable defensible story when they sit down [29:04] in front of their board or their seauite and say  well this is what we think it's going to cost for [29:08] folks who just looked at what they spent last  quarter and tried to build an estimate. Well, [29:13] those are the people who last year spent $100,000  in AI and this year spent 10 million. And the pace [29:20] of change exceeds our ability to observe it. Most  CFOs, CIOS operate on like a quarterly basis. All [29:31] these budgets that exploded happened too too  quickly for them to even observe that it was [29:38] happening until the quarter was over. That's why  it's become such a hot button button topic because [29:44] they're like, "All right, well, we're looking  at the budget in Q4, everything looks good, [29:48] and then Q1 comes around. They go, what happened?  Oh my god, how did this where did this even come [29:53] from?" And it's because our ability to observe is  now the the window's too big. You have to really [30:00] think about this almost in real time. And that  is also relatively new. Even cloud expenditures [30:06] didn't grow like that. Um but it is very easy  for an organization really of any size to go out [30:13] and spend millions of dollars right now. Think  about everyone of us when we get back from this [30:18] conference like maybe our stake dinner uh receipts  are going to be scrutinized probably rather [30:24] extremely by you know somebody in accounting but  what we spend on tokens and AI and even in cloud [30:30] like how many people are wonder is that actually  being scrutinized the same no absolutely not right [30:35] every single one of us in this room has the power  to go spend tens of thousands if not hundreds [30:40] of thousands right now which is kind of an  amazing power But like I said, like great power, [30:46] great responsibility. So we've got to increase the  pace that we observe these things. And then we've [30:52] got to push that decision-making power down to the  people making the decisions, the buying decisions, [30:57] which are all of us building things so that we  know what we're getting for that for that cost. [31:06] Well, I second everything you know you just said  because there's where's the controversy here? We [31:10] got to there's no controversy. [laughter] Yeah. So  it's it's it's actually you know very interesting [31:16] problem to have but the way you know we have  solved for it like practically what we did is from [31:21] forecasting perspective right um for every use  case that we probably started you know piloting [31:28] or building we actually created a sheet you know  to basically list that workflow and then what we [31:35] did is we actually did the fixed you know aspect  of that workflow of like okay if I'm going to spin [31:40] up you know two EC2s you Oh to basically you know  deploy uh my you know work workload you know in in [31:48] cloud okay I can basically predict that it's going  to run 24 by7 fixed cost right now when I go into [31:54] the variability of it I am actually going to ask  some questions like how many reasoning steps do [32:01] I need to make so when I ask that question the  question is how many model calls are we talking [32:06] about here like how many times I'm invoking that  and based on whatever The quality expectation is [32:14] how many round trips I may have to do in order to  provide that quality threshold because sometimes [32:20] you know people just think oh it's just one  request that goes out to a model and you get [32:24] a clean answer back it does not happen that way  there is so much orchestration even happening [32:30] behind claude and all these platforms that they  are doing probably sometimes you know hundreds of [32:36] requests behind the scenes to actually get to the  truthful like the right answer right so all these [32:42] are actually buried in your architecture and if  you are able to figure out okay how many reasoning [32:46] steps is this a decision-m uh you know bot agent  that you're trying to you know build uh some of [32:54] the other things that you should basically ask  yourself is like um if I'm building this thing and [33:00] if I build it this way is it actually going to be  the right way to get to the answer because there [33:07] are so many variable parameters that exist today  with uncert certainity that did not exist in the [33:13] past as you talked about utilization right that's  the fallacy that you just have cloud and you know [33:19] it scales but AI is not like that so your model  your intelligence layer is you know a dependent [33:27] factor your egress to your data because every  platform you know has different egress patterns [33:33] right so these are some real questions that you  have to incorporate you know into your workflow [33:39] when you're actually doing intake take about that  and again from forecasting perspective whatever [33:47] you said you know I I actually agree with from  the you know bigger perspective despite all those [33:53] robust forecasts there are situations where the  AI projects kind of go off the rails the costs [34:00] spiral you know and you're back to forecasting or  reforcasting or whatever right and or or you're [34:06] back to kind of reing in the project and its scope  so maybe share share an a story or anecdote with [34:14] the audience about when this happened in your  organizations and how you went about doing sure [34:19] the damage control. I mean, there's so many  it's a target-rich environment. Um, you know, [34:26] we we learned very early on with some of the  projects that we're working on that if we um don't [34:32] really follow through on the math, and what I mean  by that is, oh, look, I have a thing that costs 20 [34:38] cents, but how often do I expect that to be used?  Oh, it's going to be called a million times. Well, [34:44] now it starts to become expensive. Um, and it's  very easy. you start these experiments and you [34:49] don't think about the math the multiplicative  effect of that and um and so some you know [34:54] some very real examples for us we were building  things to try to figure out what would be the [34:59] most efficient way to choose which model to use  and um built a skill around this created a couple [35:06] of capabilities things that would call out to uh  other systems look up the price go try to figure [35:11] out what the optimization is and on a on a single  you know kind of request scale So well that was [35:18] really efficient. Um but when we actually started  to use it we realized the whole conversation [35:25] around um understanding which model to use costs  more than just using a model to do to answer the [35:32] question. And um and that those kinds of things  are are easy to miss if you're not looking at the [35:41] um the you know individual kind of total cost  of that transaction. So you add something to it. [35:46] Wait, our transaction cost just doubled. Well,  at scale, that's going to be a pretty serious [35:51] problem. [snorts] The the other piece was to  the builders when they were doing this, it's not [35:56] enough to like show the bill to them at the end of  the month. You have to actually be showing it to [36:01] them that same minute so that as they're testing  this stuff, you really want to get them in an [36:08] AB testing mode. And with the pace at which the  models are are getting, you know, improved, new [36:14] versions coming out, you know, there's probably  a new one shipping right now while we're sitting [36:18] here. Um, you know, most people's inclination is  to think of these things as, oh, there's a new [36:23] version. New versions are good, right? I upgrade  to the new thing, it's better. That's not true at [36:28] all. I think the versioning numbers are really a  misnomer, right? You have to look at the outcome, [36:34] but that gets you into trouble, too. So, you know,  this is another very real cautionary tale for us, [36:39] which is, well, we're doing a lot of evals to go  figure out um is the system working, you know, [36:45] and then somebody had the bright idea to ask the  question, well, how much money are we spending to [36:50] do all these evals? It was more than the system  that we were, you know, testing cost, right? [36:56] So that is happening actually um in a lot of  places where people are so they're and you get [37:03] yourself into this trouble because you're trying  to wrap a the a comforting deterministic wrapper [37:08] around something that is just never going to be  deterministic uh in that way uh when really what [37:14] you should be thinking about is well what are  the repeatable parts of the system that I can [37:18] move out of the LM and into something a little  bit more deterministic. Um, all those things [37:24] are decisions that you wouldn't make if you if you  were just if you weren't looking at the unit cost [37:30] and then multiplying that based on what you think  the volume or the usage of this will be. And if [37:35] you don't do that, sooner or later somebody will  do it for you and it will be a painful experience. [37:43] Yeah, I'm actually going to answer this like  from experience but like I'm just going to [37:47] keep it very short. So, we looked into,  you know, actually using an open-source [37:54] gateway to govern our MCP servers. Um,  it's more like, you know, a registry [38:00] uh that everyone probably is actually, you know,  trying to build or register somewhere, right? [38:05] we uh so to Eric's point you know like that's  where uh we were actually marching down towards [38:12] you know that path and we realized that oh we did  we actually missed the cost of maintaining that [38:18] platform because since it's open source it seems  you know to be a myth like it's so easy right but [38:24] in reality like every time we change something we  have to go through security reviews we have to go [38:29] through architectural reviews because it's open  source so those were the challenges that we did [38:35] not think about and at that point in time you know  what we did is we said okay we are actually going [38:40] to you know use managed version of a ser like of  you know one of the gateways that we can just plug [38:47] and play. Um but that's you know one path where  we were like okay we were blindsided just because [38:53] we thought we we were too excited that we can  technically do everything right but it still does [38:58] not hold right because there is cost associated  you know with open source as well and similarly [39:04] you know open source models like everybody talks  about hosting their own open source if you don't [39:09] have the scale don't even think about it first  prove the scale and then start thinking about