FanDuel AI Chief Macher Overhauls Dev Lifecycle to Cut Token Waste

FanDuel AI主管Macher重构开发生命周期以削减Token浪费 FanDuel在发现模型推理仅占总开发成本的一小部分、而缺乏引导的推理循环会损害代码质量后，围绕上下文工程和平台控制重构了其软件交付流水线。 Bryce Macher阐述了FanDuel整合AI的方法，将重心从单纯的代码生成转向以“决策判断速度”（speed to judgment）为核心的软件全生命周期变革。他详细介绍了其架构的四大核心支柱——代码前（Left of Code）上下文工程、代码层（Code Layer）、代码后（Right of Code）对抗性测试以及底层基础设施（Underneath），这些支柱协同实现了降低成本与提升质量。演讲还探讨了模型过度推理等现实挑战、企业内部评估相较于公开基准的必要性，以及标准化的Agent定义如何借助Prompt缓存来优化经济效益。 FanDuel 将 AI 软件开发生命周期拆分为四个层面：代码左侧（上下文与信息架构）、代码层、代码右侧（对抗性 QA/红队）以及底层基础设施（网关、编排与缓存），强调真正的成本与质量控制取决于代码之外与底层架构。 (4:35) 团队将“判断速度（speed to judgment）”置于单纯的代码生成速度之上，以衡量工程师和产品团队做出关键技术与架构决策的效率。 (2:27) 通过集中式 AI 网关、不可知编排（agnostic orchestration）以及语义缓存等平台控制手段，企业能够削减高达 80% 的推理成本，并规避全量实时调用的开销。 (11:18) 无节制地为简单任务默认选用顶级推理模型会导致“偏执推理（paranoid inference）”、降低代码生成质量，并引发工程师技能退化；约 80% 的常规负载更适合交由小型或前代模型处理。 (14:28) 公共基准（如 SWE-bench）无法准确预测企业私有代码库的表现，FanDuel 将编写模型评估（evals）视为像单元测试和 TDD 一样不可或缺的现代核心工程能力。 (18:59)

FanDuel Head of AI Bryce Macher argues that integrating AI into software development requires shifting focus away from raw code generation toward platform infrastructure, structured context engineering, and organizational discipline. Real cost control and quality improvements depend on central orchestration and internal evaluation rather than deploying frontier models alone. Prioritizing Speed to Judgment and Cost Visibility (2:27) Rather than judging success by conventional coding speedups—such as accelerating pull request merge times by 80%—FanDuel tracks "speed to judgment." This metric measures how rapidly product and engineering teams can reach sound architectural and technical decisions on trade-offs that models cannot navigate independently. To manage its AI platform as an internal product, the enterprise monitors three levers: Speed, Value, and Quality. Engineering the Architecture Outside the Code Layer (4:35) FanDuel divides its software lifecycle transformation into four tiers: context and information architecture ("left of the code"), code generation, adversarial testing and red teaming ("right of the code"), and foundational infrastructure underneath. Macher stresses that the code generation layer is labor-cheap, whereas context discovery and foundational infrastructure dictate final quality and operational spend. Providing incorrect or overly broad context risks hallucinated edge cases, unneeded code, and wasted context windows. To mitigate this, FanDuel structures raw logs, telemetry, and documentation to differentiate live runtime dialogue from static history. Rating information quality prevents models from relying on obsolete signatures or inaccurate human memories. Platform Controls, Semantic Caching, and Prompt Standardization (11:18) FanDuel operates a centralized, tool-agnostic platform perimeter featuring an AI Gateway, agnostic orchestration, safety layers, and semantic caching. Fine-tuning these infrastructure controls has yielded up to an 80% inference cost reduction among partner organizations. The platform handles developer workflows across tools like Cursor, OpenAI, Databricks Genie, Salesforce Agentforce, or local open-weight models. Macher also notes an early miscalculation regarding agent sprawl: duplicate agent markdown files and fragmented prompts across teams prevented prompt caching from triggering. Standardizing prompt templates across teams replaced expensive, fresh inference calls with cached responses, yielding substantial latency and compute savings. Mitigating Over-Reasoning and Developing Internal Evals (14:28) Unmanaged developers frequently default to frontier models with maximum reasoning parameters, triggering "paranoid inference" that devises absurd edge cases for basic tasks, degrades output quality, and risks engineering deskilling. Macher notes roughly 80% of enterprise workloads can run on smaller models, provided tasks are planned explicitly. Because public benchmarks like SWE-bench carry margins of error around ±4% and fail to mirror proprietary enterprise codebases, FanDuel relies on internal ModelOps benchmarking against actual production workloads. Macher views writing model evaluations (evals) as an essential engineering competency akin to the rise of unit testing and Test-Driven Development. Transforming the software development lifecycle requires treating AI tooling as a governed software factory rather than isolated pilot projects. Sustainable adoption hinges on educating engineers on model economics, enforcing standardized context architectures, and using historical prompts to preserve architectural intuition among junior staff.

FanDuel 人工智能负责人 Bryce Macher 提出，企业软件生命周期的 AI 转型本质是人、流程与技术的深层重构，其重点不在于追求代码生成的单点速率，而在于依托平台基础设施、上下文工程与内部评测体系加速工程决策并控制运行成本。 重新定义评估指标与四层生命周期架构 (2:27) Macher 指出，FanDuel 并没有将重点放在 Pull Request（PR）合并速度等表面速率上，而是将“决策速度”（Speed to judgment）作为核心度量，即工程与产品团队在面对 AI 缺乏全局视角的复杂技术权衡时做出正确判断的效率。根据演讲引用的行业数据，约 35% 的企业年度推理支出属于 Token 浪费造成的超支，且 71% 的工程团队无法准确量化单个 PR 产生的财务成本。为此，FanDuel 将内部 AI 系统作为产品运营，围绕速度、价值与质量三大指标进行追踪，要求所有技术与流程变革都必须切实驱动这三项杠杆。 在架构设计上，FanDuel 将软件生命周期拆分为四个互相关联的层面：代码左侧（上下文工程与信息架构）、代码层（代码生成与模型应用）、代码右侧（对抗性测试与红队安全）以及底层基础设施（编排、AI 网关、语义缓存与安全层）。Macher 强调，行业目光普遍集中在中间的代码生成模块，但就实际研发开销而言，模型生成只是最廉价的劳动力环节；真正决定最终账单和交付质量的，是代码左侧的上下文发现能力以及底层的系统化治理。 上下文工程防范低效输入与幻觉风险 (8:04) Macher 认为，向 Agent 提供错误的上下文是 AI 系统中最昂贵的失误。过于宽泛的上下文不仅白白消耗上下文窗口容量，还会诱发模型臆测不存在的边缘情况并生成多余代码。为此，FanDuel 致力于将企业底层数据转化为结构化情报，包括将 Splunk 或数据湖中的日志与遥测数据提炼为可复用的人类工作流经验，并为技术文档建立严格的时效与关联结构，防止模型因无法识别当前有效的函数签名而调用过时代码指针付费推理。 此外，系统架构必须严格区分实时运行上下文（如 Slack 即时消息、站会讨论）与静态历史记录。Macher 指出，人类记忆本身存在局限，团队成员常会记错以往的技术决策；因此系统必须建立信息质量评分与分级机制，防止劣质输入污染 Agent。很多时候系统出现的输出偏差并非模型固有的幻觉，而是因为系统输入了错误信息。 底层平台治理与提示词缓存经济效益 (11:18) FanDuel 搭建了一套独立于具体开发工具的平台边界，集成了基于 Kong 的 AI 网关、Initialise 编排层、语义缓存、合规安全管控与评估引擎。无论是工程师在 Cursor 中编写 Agent，还是调用 Salesforce Agentforce、Databricks Genie、OpenAI 接口或本地 DGX 硬件上的开源模型，均受该中央系统的统一编排与安全管控。通过拦截重复查询并提供语义缓存响应，配合精细化的平台治理，合作机构实现了高达 80% 的推理成本降幅。 在运营实践中，Macher 坦承团队早期曾犯过忽视 Agent 冗余的错误，误以为 Markdown 格式的 Agent 定义文件存储成本低廉故无需限制复用。但随着规模扩大，团队发现提示词缓存机制（Prompt Caching）对经济效益有决定性影响：当不同团队为相同任务编写碎片化或存在细微差异的提示词时，会导致缓存失效并触发全额的实时推理。通过标准化与合并 Agent 提示词定义，团队有效激活了缓存机制，从而大幅削减了延迟和计算费用。 规避过度推理陷阱与构建内部评测体系 (14:28) Macher 警告称，工程师普遍存在默认调用顶级大模型并拉满推理参数的文化倾向，这会导致“偏执推理”——模型因过度思考而推演极度荒谬的边缘场景（例如假想某个偏远地区断网断电），反而导致简单任务的输出质量下降。这种过度依赖还可能引发工程师的“隐性去技能化”（Quiet deskilling），使其放弃系统的架构设计和测试思考。实际上，约 80% 的日常业务工作负载在配合清晰的前期规划后，完全可以由 Claude Sonnet 或低规格 OpenAI 等更经济的模型高效承载。为帮助初级工程师建立核心技术底蕴，FanDuel 探索利用早期编写的详尽历史提示词作为学习材料，让年轻工程师理解资深人员的架构逻辑与思考过程。 在模型引入与升级方面，FanDuel 坚决脱离公开发布的通用基准与厂商宣传噱头。Macher 指出，公开测试（如 SWE-bench）往往带有 ±4% 的误差范围，难以反映专有生产环境代码库的真实情况。FanDuel 要求 ModelOps 团队直接基于内部生产任务进行模型实测与版本迭代（如对比 4.8 至 5.6 各版本）。FanDuel 将编写模型评测（Evals）视为与十年前普及单元测试和测试驱动开发（TDD）同等重要的软件工程核心能力，并将其列为组织转型的首要培训重点。 Macher 总结指出，企业软件转型的最低标准在于各项变革必须切实加快工程人员的专业判断能力并体现可量化的业务推进。然而，由于当前高等教育体系尚未涵盖模型评测（Evals）的设计规范，工程团队能否快速补齐该项能力缺口，并抵御因过度依赖生成工具而引发的团队去技能化风险，依然是企业在持续落地 Agent 体系时面临的核心考验。

## Background 该视频由 Ai4 频道于 2026 年 8 月 15 日发布在 YouTube 上，演讲嘉宾为 FanDuel 的 AI 负责人 Bryce Macher。简介指出内容主要探讨如何通过以 Agent 为核心的软件开发生命周期（SDLC）重塑，将 AI 转化为持久的交付优势，并涵盖从孤立试点走向可规模化生产级能力的架构演进与实战教训。 ## TL;DR FanDuel 提出 AI 转型的北极星指标是“加速人类判断”（Speed to Judgment）而非盲目的代码合并速度，强调模型本身反而是最便宜的部分，代码左侧的上下文工程与底层基础设施才决定账单与质量。团队构建了涵盖无感编排、AI 网关、红队测试与语义缓存的四大演进支柱，指出行业中 80% 的研发负载可由良好规划的中小模型承接。同时，演讲警示了工程师依赖顶配大模型导致的“偏执推理”与技能退化风险，并用亲身踩坑经验揭示了通过标准化 Agent 技能提高 Prompt 缓存命中率对削减推理开销的关键作用。 ## Atmosphere & style 整场演讲务实冷静且干货密度极高，完全摒弃了空洞的 AI 炒作，洋溢着一线技术领袖在生产环境中踩坑摸爬后的硬核实战质感。主讲人风格犀利而坦诚，在调侃“大模型偏执推理”与承认自身早期“存储便宜无所谓”的认知错误时引人发笑又发人深省。听众能强烈感受到一种从浮躁的技术狂热回归到系统工程底座、成本经济学与扎实基本功的清醒与力量。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=eBpHJH714q0&t=4s) AI 转型不仅是引入新工具，而是对人、流程与技术的全面重构，传统平台工程与软件工程边界正在消解，AI 工厂必须作为正统软件工程体系来建设。 - [02:27](https://www.youtube.com/watch?v=eBpHJH714q0&t=147s) 核心度量指标应定为“决策速度”而非单纯的“代码合并速度”；如果架构与业务方向错误，盲目提升代码交付效率毫无意义。 - [02:27](https://www.youtube.com/watch?v=eBpHJH714q0&t=147s) 当前企业推进 AI 工程化面临巨大资源浪费，行业数据显示高达 35% 的推理预算属于无效损耗，且 71% 的团队无法量化单次 PR 的 AI 成本。 - [04:35](https://www.youtube.com/watch?v=eBpHJH714q0&t=275s) FanDuel 将 AI 驱动的软件生命周期划分为四大支柱：代码左侧（上下文与信息架构）、代码本身、代码右侧（对抗性测试与安全红队）以及底层基础设施。 - [07:16](https://www.youtube.com/watch?v=eBpHJH714q0&t=436s) 从计算成本来看“模型其实最便宜”，代码左侧的上下文治理与底层基础设施才是决定最终交付质量与财务账单的核心关键。 - [07:16](https://www.youtube.com/watch?v=eBpHJH714q0&t=436s) 大模型天然缺乏时间感知与推理能力，企业必须重构技术资产的信息架构，避免模型因时效混乱而频繁调用过时 API 与废弃代码。 - [10:10](https://www.youtube.com/watch?v=eBpHJH714q0&t=610s) 明确区分运行时上下文与历史上下文，通过校验机制解决人类工程师在会议或沟通中由于遗忘、直觉等导致的输入端信息失真。 - [11:18](https://www.youtube.com/watch?v=eBpHJH714q0&t=678s) 平台构建了与具体工具解耦的统一编排层与 API 网关，既在安全合规边界内控制流量，又允许开发者自由接入 Cursor、Genie 或私有 DGX 开源模型。 - [11:18](https://www.youtube.com/watch?v=eBpHJH714q0&t=678s) 将幻觉视为信息密度缺失引发的问题并进行闭环富化，同时借助语义缓存复用先验模式，可将端到端推理成本最高削减 80%。 - [14:28](https://www.youtube.com/watch?v=eBpHJH714q0&t=868s) 必须破除“唯大模型与顶配推理”的迷思，防范在简单任务上过度推演导致的“偏执推理”，并警惕工程师依赖一键生成而引发的“无声去技能化”。 - [17:26](https://www.youtube.com/watch?v=eBpHJH714q0&t=1046s) 显式 Prompt 规划成为沉淀资深架构师隐性思维的教学资产，业内约 80% 的实际常规研发负载完全可由中小型工作模型高效承接。 - [17:26](https://www.youtube.com/watch?v=eBpHJH714q0&t=1046s) SWE-bench 等公开榜单得分无法代表生产环境实际表现，企业应像编写单元测试一样构建专有业务评测集（Evals）以评估真实投资回报率。 - [20:28](https://www.youtube.com/watch?v=eBpHJH714q0&t=1228s) Agent 交互与提示词编排正在成为软件工程的核心素养，鉴于高校教育滞后，企业需要像当年普及 TDD 一样主动承担工程师技能重塑。 - [20:58](https://www.youtube.com/watch?v=eBpHJH714q0&t=1258s) 碎片化且重复定义的 Agent 技能会导致 Prompt Cache 击穿并产生昂贵的实时推理开销，必须建立统一标准化的技能库以最大化复用缓存。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=eBpHJH714q0&t=4s) 重塑软件开发生命周期：人、流程与技术的重构 在 FanDuel 推进软件开发方式转型的过程中，团队深刻体会到经典维度——“人、流程与技术”（People, Process, Technology）都需要被重新审视与定义。传统开发所依赖的 Jira、GitHub 以及既有的代码管理工具与流程，在面对 AI 带来的极致交付加速度时，形态上已经显得格格不入。团队协作模式也经历了颠覆：过去仅仅是人类工程师之间的协作，如今必须融合物理上的“真实团队成员”与数字化、智能化的“Agent 协作者”。与此同时，传统的平台工程（Platform Engineering）、软件工程与产品工程之间的边界正在迅速消解，迫使团队将底层基础设施视为最核心的交互界面与产品能力。 在这一演进过程中，FanDuel 总结出了贯穿整体转型的关键经验： - **平台中的上下文（Context in the Platform）：** Agent 固然重要，但成本与质量的决定性因素往往深藏在底层平台及上下文管理中。 - **模型选型（Model Selection）：** 尽管动态模型路由（Model Routing）方案正在涌现，但针对工程场景科学选型仍然是一项能产生长期复利的核心能力。 - **评测与基准测试（Evals & Benchmarking）：** 在底层前沿大模型每 3 到 6 周就迭代一次的大环境下，持续评估与基准测试是保障系统稳定加速的前提。 - **一致性（Consistency）：** 保持体系结构与数据定义的一致性至关重要，这是踩坑后迅速调整战略得出的教训。 - **AI 工厂即软件工厂（Staffing the AI Factory as a Software Factory）：** 许多团队将 AI 转型误当成“数据转型”而困在数据结构和术语里，实际上必须将其作为正统的“软件生命周期转型”与基础设施重构来对待。 ```mermaid flowchart TD accTitle: 软件转型核心要素 subgraph Core["转型核心认知与杠杆"] direction TB A["传统边界消解\n(Platform/SE/PE 融合)"] --> B["协作范式跃迁\n(人类与 Agent 协同)"] B --> C["加速判断\n(Speed to Judgment)"] C --> D["三大支柱度量\n(Speed, Value, Quality)"] end ``` --- ### [02:27](https://www.youtube.com/watch?v=eBpHJH714q0&t=147s) 核心度量体系：追求“决策速度”而非单纯的“代码速度” 转型的第一准则是：任何改动必须“加速人类的判断并展现实质进展”（"accelerate judgment and show movement"）。FanDuel 强调，转型的目标不是单纯缩短写代码或合并分支的时间。即便能达成“代码合并速度提升 80%”的指标，若合并进来的代码本身方向错误，也毫无意义。因此，团队将北极星指标定为“决策速度”（Speed to Judgment）——即多快能做出正确的架构决策、技术权衡与产品取舍，尤其是在 Agent 缺乏足够业务上下文而无法自主决策的复杂地带。 团队采用“平台即产品”（Platform as a Product）的理念，设立了贯穿速度（Speed）、价值（Value）与质量（Quality）三大维度的度量指标体系。这一布局的背后，是整个行业在 AI 工程化推进中面临的两大痛点数据： - **35% 的年度企业推理超支：** 合作伙伴与行业独立研究表明，企业高达 35% 的 Token 预算纯属无效浪费。 - **71% 的工程团队无法量化 PR 成本：** 绝大多数团队在引入 Agent 编写代码、执行安全扫描与代码评审后，完全算不清一次 Pull Request（PR）背后的真实财务与资源消耗。 --- ### [04:35](https://www.youtube.com/watch?v=eBpHJH714q0&t=275s) 四大工程支柱：左移上下文与全栈基础设施 FanDuel 将整个 AI 驱动的软件开发转型划分为四个紧密交织的板块，每个板块都拥有独立演进的路线图： ```mermaid flowchart LR accTitle: FanDuel 四大软件演进支柱 subgraph Left["1. 代码左侧 (Left of Code)"] L1["上下文工程 Context Engineering"] L2["信息架构 Information Architecture"] L3["协调知识生成 Knowledge Generation"] end subgraph Center["2. 代码构建 (Code)"] C1["前沿实验室模型 Frontier Labs"] C2["开源权重模型 Open-Weight Strategy"] C3["代码生成与审查 Agent"] end subgraph Right["3. 代码右侧 (Right of Code)"] R1["对抗性 AI 测试 Adversarial Testing"] R2["红队演练 Red Teaming"] R3["自动化 QA"] end subgraph Bottom["4. 底层基础设施 (Underneath / Infrastructure)"] B1["编排层 Orchestration (Initialise)"] B2["AI 网关 Gateway (Kong)"] B3["语义/Prompt 缓存 Semantic Caching"] B4["信任、安全与合规 Trust & Safety"] end Left --> Center Center --> Right Bottom -.-> Left Bottom -.-> Center Bottom -.-> Right ``` 1. **代码左侧（Left of the Code）：** 聚焦成本与质量的根源，涵盖上下文工程（Context Engineering）、信息架构以及协调知识生成。 2. **代码本身（The Code）：** 涵盖前沿模型（Frontier Models）与开源权重模型（Open-Weight Models）的混合应用，以及编码 Agent 的研发。 3. **代码右侧（Right of the Code）：** 聚焦对抗性 AI 测试（Adversarial AI Testing）与红队演练（Red Teaming）。随着 Agent 生成的代码量剧增，潜在攻击面与系统风险显著扩大，必须重塑 QA 与安全体系。 4. **底层基础设施（Underneath Infrastructure）：** 涵盖 Agent 编排层（与设计伙伴 Initialise 合作）、AI 网关（基于 Kong 构建）、语义缓存（Semantic Caching）、Prompt 缓存、安全防护以及信任与安全（Trust & Safety）平台。 各团队紧密联动：软件工程团队构建 Agent 交互逻辑，平台团队提供网关与缓存等支撑，产品工程团队则在“代码左侧”优化历史上下文与运行时上下文的配比，从根源上避免无意义的工具调用（Tool Calls）、冗余 MCP（Model Context Protocol）追踪以及过宽过深的上下文消耗。 --- ### [07:16](https://www.youtube.com/watch?v=eBpHJH714q0&t=436s) “模型其实最便宜”：上下文与信息架构的工程实践 在实际落地中，FanDuel 认清了一个关键事实：“从算力劳动力成本来看，模型反而是最便宜的部分，代码左侧和底层设施才真正决定最终的账单与交付质量”（"The model's actually the cheap part... Everything to the left of the code and underneath it decides your bill and your quality"）。单纯教工程师怎么写出更好的 Prompt 带来的收益极其有限，真正的杠杆在基础设施与上下文治理层。 在“代码左侧”，核心教训在于“要么喂给模型精准的数据，要么为混乱买单”（"Feeding the model or paying for the mess"）： - **上下文工程（Context Engineering）：** 必须在精准的时间节点向 Agent 提供精准的系统切片。错误的上下文是最昂贵的输入，会导致 Agent 产生幻觉、思考根本不存在的边缘用例，进而造成严重的算力与 Token 浪费。 - **信息架构（Information Architecture）：** 原始轨迹与日志（如 Splunk、缓存、分析数据湖中的数据）必须经过可靠流程沉淀并转化为结构化“信息”。当前大模型天然缺乏对时间的感知与推理能力，无法分辨哪一份文档或 API 是最新的。若企业内部技术资产没有清晰的时效结构，模型就会频繁调用废弃的函数、过期的 API 和旧代码，导致系统输出过时且昂贵的结果。 - **协调知识生成（Coordinated Knowledge Generation）：** 必须建立机制，将过去散落的知识和新生成的知识体系化组织，使 Agent 与人类能够持续获取一致、高信噪比的技术上下文。 ### [10:10](https://www.youtube.com/watch?v=eBpHJH714q0&t=610s) 运行时与历史上下文的分离：应对人类信息的不确定性 在构建 AI 赋能的软件生命周期时，信息科学专家面临的最大挑战之一是如何捕捉并利用“即时生成的信息”（Information generated on the fly）。这不仅包括正式文档，还涵盖会议讨论、Slack 频道沟通、走廊即兴交流以及代码审查中突发的灵感与共识。为了系统化协调这些动态信息，FanDuel 将信息架构明确划分为“运行时上下文”（Runtime Context）与“历史上下文”（Historical Context）。运行时上下文指的是在特定的协作节点上，AI 系统必须实时获取并对照历史数据进行加权评估的动态背景。 人类开发者天然存在局限——会遗忘、记错细节、凭直觉行事，或者因为某位核心成员休假而缺失上次决策的完整背景。在这种情况下，系统面临的问题往往不是大模型的“幻觉”，而是输入端的“错误人类信息”。因此，平台必须建立一套机制，依据既有的信息架构对运行时输入的信息质量与一致性进行评级和校验，确保进入交付流水线的上下文具备足够的准确性与可信度。 --- ### [11:18](https://www.youtube.com/watch?v=eBpHJH714q0&t=678s) AI 平台底座的五大支柱：编排、网关、缓存、安全与可信 为了将上下文高效转化为生产力，FanDuel 搭建了一套平台级基础设施，由五大核心支柱构成。首要是“平台无关的编排能力”（Agnostic Orchestration）：无论工程师选择在 Cursor 中构建 Agent、使用 Salesforce Agentforce、Databricks Genie、云端商业 Agent，还是在本地 DGX Spark 集群上运行开源权重模型，“只要能接入系统的统一编排，开发者尽可放手去探索”。这种设计实现了“在边界内自由实验”，由平台掌控安全外围，同时最大化赋予开发者创新自由。网关（Gateway）负责严格控制进出边界的全部流量，贯彻“设计即安全”（Secure by Design）的原则。 ```mermaid flowchart TD accTitle: FanDuel AI Platform Infrastructure Architecture accDescription: Architectural diagram showing the five pillars: Gateway, Orchestration, Semantic Caching, Security, and Trust & Safety. subgraph Boundary ["受控外围 (Secure Perimeter)"] GW["API 网关 (Gateway)"] ORCH["多平台编排层 (Orchestration Layer)"] subgraph Compute ["计算与模型端"] AG1["IDE / 商业 Agent (Cursor / Genie / Agentforce)"] AG2["开源权重模型 (DGX Spark / 本地集群)"] end subgraph OptimAndTrust ["优化与可信保障"] CACHE["语义缓存 (Semantic Caching)"] SEC["安全与隐私合规 (Security / PII)"] EVAL["可信与安全 (Trust & Safety / Evals)"] end end GW --> ORCH ORCH --> AG1 ORCH --> AG2 ORCH <--> CACHE ORCH <--> SEC ORCH <--> EVAL ``` 在优化与安全保障层面，语义缓存（Semantic Caching）发挥了决定性作用。代码生成过程中存在大量重复模式，从缓存检索先验知识的成本远低于实时推理；结合开源权重模型共享上下文后，语义缓存能将重复任务的边际推理成本降至近乎为零。作为强监管行业，安全层着重监控敏感隐私信息（PII）被非人类 Agent 处理时的合规风险。可信与安全（Trust & Safety）模块则涵盖两项关键职能：一是通过评估（Evals）验证生成内容的真实性、速度与准确性；二是进行幻觉检测。在本质上，幻觉可以视为“低信息密度”引发的数学问题，系统通过定位代码中信息密度匮乏的盲区，调用协调 Agent 回流富化上下文，从而根除幻觉风险。通过精细调优这些基础设施，行业伙伴与客户实现了高达 80% 的推理成本降幅。 - **多模型与工具自由**：支持 Cursor、Agentforce、Databricks Genie 及私有 DGX 部署的开源权重模型无缝接入。 - **降本增效**：借助语义缓存与架构调优，推理成本最高可降低约 80%。 - **幻觉治理**：将幻觉视为信息密度缺失问题，通过 Agent 闭环进行上下文主动富化。 --- ### [14:28](https://www.youtube.com/watch?v=eBpHJH714q0&t=868s) 破解“唯大模型论”迷思：防范偏执推理与工程技能退化 技术体系就绪后，真正的挑战转向了组织文化变革。许多团队面临的最大认知陷阱是误以为“模型越大、推理算力堆得越多就越好”。未经系统培训的工程师往往会习惯性调用顶配大模型，并将推理参数拉到上限。这种做法不仅极其昂贵，而且往往适得其反——大模型会在简单任务上过度推演，产生令人哭笑不得的“偏执推理”（Paranoid Inference），例如莫名其妙地去推演“罗德岛某个小镇停电、用户手机恰好插电且路由器断网跳出 Wi-Fi”等极端边缘异常。 更为隐秘且危险的风险是工程师团队的“无声去技能化”（Quiet Deskilling）。部分工程师试图用顶配模型一键单发（One-shot）搞定一切，放弃了深入推敲实现路径、设计边界和代码鲁棒性的思考过程。他们误以为“只要写两句话丢给 36 个 Agent 碰运气，自己就能掌控全局”，导致基础工程能力的全面滑坡。FanDuel 强调，智能路由与技能培养的核心在于守住工程基本功：产品工程师必须懂产品构建逻辑，后端工程师必须精通 API 与架构设计，绝不能因依赖大模型的一键生成而丧失核心上下文掌控力。 > "大模型不仅会带来昂贵的推理账单，更会带来昂贵且偏执的推理（expensive paranoid inference）。" > > "你不能作为一个产品工程师却不知道如何构建产品，也不能作为一个后端工程师却不知道如何搭建 API 和后端架构。" --- ### [17:26](https://www.youtube.com/watch?v=eBpHJH714q0&t=1046s) 模型分级、显式思维传承与实战评测（Evals）体系 早期大模型（如早期 Claude Sonnet 提示工程）要求开发者极度清晰、显式地写明步骤、指导约束与参考示例。FanDuel 发现，这类高要求的 Prompt 资产反倒成为了培训初级工程师与平台工程师的绝佳教材——它前所未有地迫使资深开发者将原本藏在脑海里的完整系统思考链路具象化、白纸黑字地展现出来。 数据表明，业内 80% 的实际工作负载完全可以由经过良好规划（Planning）的中小型模型高效完成（如 Claude Sonnet 系列或轻量级 OpenAI 模型）。然而，在“生成方案”与“落地执行”之间往往存在鸿沟，需要前沿大模型在关键决策节点把控方向。 ```mermaid flowchart LR accTitle: Task Routing and Benchmark Alignment accDescription: Flow from prompt planning to right-sized model execution and real-world eval benchmarking. A["显式规划 (Explicit Planning)"] --> B{"任务复杂度评估"} B -->|"80% 常规负载"| C["中小型工作模型 (如 Claude Sonnet / 轻量级模型)"] B -->|"方向校准 / 复杂推演"| D["前沿模型 (Frontier Models)"] C & D --> E["生产真实任务评测 (In-House Evals)"] E -.->|"对照 SWE-bench 纠偏"| A ``` 在工程师技能进阶中，FanDuel 将评测（Evals）能力的构建置于核心地位。针对开源社区频繁宣称新模型“性能提升 2%”的狂热现象，团队指出这类测试通常存在 ±4% 的误差区间，且 SWE-bench 等公开基准测试的跑分往往无法真实反映企业生产代码环境下的实际表现。为此，FanDuel 转向基于内部真实业务场景构建专有评测集，用来精准衡量模型升级或跨代迁移（如 4.8、4.9 到 5.x 系列）是否真正具备投资回报率。 - **工作负载配比**：行业内约 80% 的研发负载可由中小型模型配合前置规划稳定承接。 - **公开基准脱节**：SWE-bench 等公开榜单得分无法直接等同于生产环境效能，需警惕小幅跑分提升背后的统计误差。 - **评测即测试**：“编写模型评测（Evals）本质上就如同编写单元测试”，是现代 AI 工程师必须掌握的基础工程素养。 ### [20:28](https://www.youtube.com/watch?v=eBpHJH714q0&t=1228s) 软件工程新范式的演进与技能重塑 在探讨现代软件研发流程时，Bryce Macher 强调，基于 Agent 和提示词工程的开发模式正在迅速演变成为构建软件的核心与关键步骤。然而，这种能力并非工程师天生具备的技能，目前从大学刚毕业的初级工程师也并未在校园中系统学习过这些知识。这与五到十年前行业推广测试驱动开发（TDD, Test-Driven Development）时的境遇如出一辙——当时高校课程同样没有涵盖 TDD，企业必须在工程实践中完成对新人的培训与技能重塑。因此，当前围绕 Agent 工具与开发模式的探索，正是整个软件工程团队经历新一轮能力转型的必经阶段。 - **工程素养断层**：高校计算机教育在新型 AI 研发模式（如 Agent 工作流设计、提示词编排等）上存在滞后，企业需要像当年普及 TDD 一样承担起培训职责。 - **流程重心转移**：人机协同的 Agent 交互与评估机制正从外围辅助手段转变为软件生命周期（SDLC）的核心支柱。 --- ### [20:58](https://www.youtube.com/watch?v=eBpHJH714q0&t=1258s) 技能重复与成本陷阱：Prompt 缓存与实时推理的经济账 Bryce Macher 坦率回顾了团队在一年前犯下的一个重大认知错误——关于 Agent 技能与配置文件的重复性问题。起初他的想法非常直接：“我当时根本不在乎外面写了多少个重复的技能，或者建了多少个 Agent 专用的 Markdown 文件（如 `AGENTS.md`），毕竟存储成本那么便宜，谁在乎呢？”（"storage is cheap, who cares?"）。但随着系统规模不断扩大、成熟度不断提升，团队在运营成本上遭遇了现实的敲打：这种碎片化、重复化的做法在经济模型上是完全错误的。 问题的核心在于大语言模型推理的缓存机制（Prompt Caching）。当团队中的不同工程师针对同一个任务编写了五花八门、各不相同的 Agent 指令与技能定义时，模型调用就无法命中预缓存的上下文，必须每一次都执行昂贵的实时推理（Live Inference），导致企业持续支付高额的推理溢价。意识到这一点后，团队在内部展开了深度的认知转型与工程规范教育，让全员彻底理解“命中缓存的推理”与“未缓存的独立实时推理”在成本和性能上的巨大鸿沟，从而推动了全局技能库与上下文的标准化整合。 ```mermaid flowchart TD accTitle: Agent Prompt Caching vs Live Inference subgraph Fragmented["碎片化/重复定义（错误模式）"] A1["工程师 A 自定义 Agent Prompt"] A2["工程师 B 自定义 Agent Prompt"] A1 --> M1["无法命中缓存 -> 全额实时推理（高成本/高延迟）"] A2 --> M1 end subgraph Standardized["标准化 Agent 技能库（优化模式）"] B1["统一标准 Agent 技能定义"] B2["团队共享复用上下文"] B1 --> C1{"Prompt 缓存检查"} B2 --> C1 C1 -- "命中缓存" --> M2["缓存推理（低成本/低延迟）"] C1 -- "未命中" --> M3["首次实时推理并写入缓存"] end ``` - **认知转变**：从“存储成本低廉所以无所谓重复”，转变为“重复提示词会导致缓存击穿并带来巨额推理账单”。 - **缓存经济学**：统一、标准化的 Agent 技能与指令模板能够最大化 Prompt Cache 命中率，大幅削减大模型推理开销并降低响应延迟。 ## Notable quotes > [02:27](https://www.youtube.com/watch?v=eBpHJH714q0&t=147s) accelerate judgment and show movement — Bryce Macher > [07:16](https://www.youtube.com/watch?v=eBpHJH714q0&t=436s) The model's actually the cheap part... Everything to the left of the code and underneath it decides your bill and your quality — Bryce Macher > [07:16](https://www.youtube.com/watch?v=eBpHJH714q0&t=436s) Feeding the model or paying for the mess — Bryce Macher > [10:10](https://www.youtube.com/watch?v=eBpHJH714q0&t=610s) Information generated on the fly — Bryce Macher > [14:28](https://www.youtube.com/watch?v=eBpHJH714q0&t=868s) 大模型不仅会带来昂贵的推理账单，更会带来昂贵且偏执的推理（expensive paranoid inference）。 — Bryce Macher > [14:28](https://www.youtube.com/watch?v=eBpHJH714q0&t=868s) 你不能作为一个产品工程师却不知道如何构建产品，也不能作为一个后端工程师却不知道如何搭建 API 和后端架构。 — Bryce Macher > [17:26](https://www.youtube.com/watch?v=eBpHJH714q0&t=1046s) 编写模型评测（Evals）本质上就如同编写单元测试 — Bryce Macher > [20:58](https://www.youtube.com/watch?v=eBpHJH714q0&t=1258s) storage is cheap, who cares? — Bryce Macher ## Mentioned Bryce Macher, FanDuel, Jira, GitHub, Initialise, Kong, Splunk, Cursor, Salesforce Agentforce, Databricks Genie, DGX Spark, Claude Sonnet, OpenAI, SWE-bench ## Video description Bryce Macher - Head of AI, Fan Duel What if the highest-leverage AI investment is a rebuilt software development lifecycle? I’ll show how we used agent-centric dev lifecycle patterns to compress the path from concept to prototype to production, redefining many stages and enabling engineering, product, and marketing teams to operate on the same accelerated delivery loop. I’ll cover put maturity arc to incorporating and sustaining agentic tooling, code generation, evaluation frameworks, and continuous learning pipelines the improved experimentation velocity and customer relevance - including the many missteps. I’ll explain the technical and operating model changes required to move from isolated pilots to scalable, production-grade AI capabilities. Attend if you want to learn how leading organizations are turning AI into a durable software delivery advantage. [Source](https://www.youtube.com/watch?v=eBpHJH714q0)

[2:27] Bryce Macher: accelerate judgment and show movement [7:16] Bryce Macher: The model's actually the cheap part... Everything to the left of the code and underneath it decides your bill and your quality [7:16] Bryce Macher: Feeding the model or paying for the mess [10:10] Bryce Macher: Information generated on the fly [14:28] Bryce Macher: 大模型不仅会带来昂贵的推理账单，更会带来昂贵且偏执的推理（expensive paranoid inference）。 [14:28] Bryce Macher: 你不能作为一个产品工程师却不知道如何构建产品，也不能作为一个后端工程师却不知道如何搭建 API 和后端架构。 [17:26] Bryce Macher: 编写模型评测（Evals）本质上就如同编写单元测试 [20:58] Bryce Macher: storage is cheap, who cares?

[0:04] Bryce Macher: Every one of these must accelerate judgment and show movement—that's the minimum bar as we think about our transformation. [4:35] Bryce Macher: the model is actually the cheap part in terms of labor. Everything to the left of the code and underneath it decides your bill and your quality at the end of the day. [8:04] Bryce Macher: It's not necessarily a hallucination; it's bad information. [14:28] Bryce Macher: They get to a point where they're like, 'Well, I can manage 36 agents at the same time because I can write two sentences and send it off... That results in an engineering organization that starts to slide backwards. [21:05] Bryce Macher: I don't care how many skills are out there, how many agent MDs and their markdown files. Like, storage is cheap, who cares? As we matured our system and as we scaled, one of the interesting things we discovered is that it's wrong on the economics.

[0:04] Good afternoon, everyone. I'm Bryce Mocker.  Uh the focus this afternoon is really on the [0:10] lessons learned from transforming the way that we  develop software at FanDuel. Uh lessons learned [0:16] cross a lot of different domains. It's It's  the classic people, process, and technology. [0:21] What's interesting about this space is it requires  you to reformulate your understanding of all of [0:25] those. We encounter a lot of problems, things  that, you know, folks are talking about now, [0:29] uh Jira and GitHub and code code management being  the wrong shape for the acceleration that we're [0:34] experiencing. The way that we think about teams  collaborating, how they collaborate both with [0:39] physical collaborators, but also agentic and  digital collaborators. Focuses on forces us [0:45] to think a lot about infrastructure as part of  the surface levels. There's breakdowns between [0:50] platform engineering, software engineering,  product engineering falling apart in many ways. Uh [0:55] and so that's the area that I want to share with  you, just kind of putting the the big points up [1:01] These are the the big lessons that that we  learned that I'll give you a little bit more [1:04] flavor and context before behind. But briefly,  it's it's finding context in the platform. [1:10] Uh there's a lot of focus on agents, and that's  great. It's an It's an important part of this [1:14] system, but we're going to talk about what we  found diving a little bit deeper where cost and [1:19] quality actually live. Um model selection, it's,  you know, there there are solutions emerging for [1:25] model routing as we learn more about how engineers  work, but thinking about how you choose a model [1:30] is a skill that pays dividends throughout the  entire process. Um evals are absolutely critical [1:37] uh as we try and accelerate in this world that's  also accelerating underneath us. New model [1:41] releases every uh 3 to 6 weeks uh require you  to start thinking about evals and benchmarking. [1:48] Consistency becomes absolutely critical.  We'll talk about a place where I had one [1:52] opinion coming into this year and quickly changed  it from the lessons learned. Uh and then finally, [1:58] staffing the AI factory as a software factory,  thinking from software models where a lot of teams [2:04] are getting stuck in kind of data terms and and  data structures rather than thinking about the [2:10] the underlying infrastructure and transformation  is really a software transformation. Every one [2:16] of these must accelerate accelerate judgment and  show movement. That's where we start. That's the [2:21] minimum bar as we think about our transformation.  And I want to share with you what that looks like. [2:27] Um so for us, the the kind of key focus was the  speed to a judgment, not necessarily speed to [2:33] code. You can set a benchmark of something  like a, you know, 80% acceleration in merge. [2:40] That's great, but what are you merging? Is it the  right stuff to merge? So for us, we really thought [2:45] about this as making a speed to judgment, speed to  technical decisions, speed to product decisions, [2:50] especially those places where both of those impact  each other and you have to make a trade-off that [2:54] an agent doesn't necessarily have your context to  make. We also thought a lot about the different [3:01] dimensions of that transformation, transcending  three categories. It was speed, it was value, [3:07] and it was quality. And we were setting  metrics as we go through this transformation [3:11] for each of those categories. So treating it like  a real product development process, platform as [3:16] a product approach to making sure that every  modification we make to our platform, every change [3:21] we make to people, process, or technology drives  either speed, value, or quality in a measurable [3:26] way that helped us meet our benchmarks, which  I just put some placeholder variables here for. [3:33] This is against the backdrop that's really  interesting as we were looking through some [3:36] of our partners' independent research. There  were two numbers that really stood out to us [3:42] at the beginning of this year. It was 35% annual  enterprise inference overspend. So 35% of your [3:48] token budget is absolute waste. And so how do you  approach that? That was an area of benchmark that [3:54] we've seen in a couple of different places, from  our frontier labs, from our inference providers, [3:59] uh etc. And then, 71% of engineering teams can't  tell you the cost of their pull requests. So, [4:04] as we go through the agentic transformation,  as agents are merging more code together, [4:10] as they're going through code reviews, as  security agents scan, as context agents [4:14] bring into a discovery process, it's really  difficult for engineering teams to tell you [4:18] that as they go through this transformation, this  is the financial impact of that transformation. [4:24] So, that's how we ultimately approach this  problem is understanding our metrics and [4:28] where we wanted to be against a backdrop of what  we think are are the industry best practices. [4:35] In that transformation, our transformation really  lives in four places. It's what we call to the [4:40] left of the code, which is really about cost and  quality. That's your context engineering, your [4:44] information architecture, coordinated knowledge  generation. We'll come back to information versus [4:49] data versus code, uh here momentarily. But,  left of the code is one of the most critical and [4:55] crucial places that that we're seeing a lot of uh  kind of skipping the early steps and foundations. [5:01] Um one of the things that we discovered,  we'll talk about it here in a moment, [5:04] is that things like orchestration, context spines,  runtime versus uh versus historical context, [5:12] managing those, balancing them critically, working  with our partners across our orchestration layer, [5:18] uh Initialise, who's a who's a design partner  of ours, working with Kong as our AI gateway, we [5:23] started to really uncover that the acceleration,  the transformation, actually doesn't start with [5:28] the agents or the code. It actually starts left of  that area. So, then we have the code. This is the [5:34] place where a lot of folks have been accelerating  and focusing. We we we use multiple frontier labs. [5:40] We have a an open weight model strategy that we're  we're working through. And this is the piece where [5:46] we focus on how do we write code, how do we build  agents that can write code. On the far right side, [5:52] adversarial AI testing becoming a key part  how you QA, but also how do you red team. [5:58] That's an important breaking area that we are  very focused on as we have more agents write code. [6:04] You know, substantially increasing what could be  a risk surface. And then finally, the other area [6:09] I want to focus on here in the remainder of the  talk is the underneath that infrastructure for how [6:14] you do this process. Orchestration, gateway,  caching, semantic caching, prompt caching, [6:20] the security layer, and then also  your trust and safety platform [6:23] are absolutely critical. So when we look at our  portfolio, we're talking about a transformation. [6:28] That transformation is really happening in one  of these four buckets. And each of these four [6:32] buckets has their own road map for how we evolve  in tandem together. They're deeply intertwined [6:39] as our software engineering team builds out our  agentic layers and how we do software engineering, [6:44] the platform team is building out the platform  pieces that support them. As that platform team [6:48] builds that out, our product teams and our product  engineering teams are working to the left on how [6:53] do we do collaboration? How do we do coordination?  How do we think about the relationship between [6:57] those two things in terms of historical  context and runtime context in a way that [7:02] could be optimized by the platform underneath  so we're not burning tokens on useless tool [7:07] calls, useless MCP traces, and context that's  too deep for the the task at hand for an agent. [7:16] So all of that to say, if you you know, if you're  paying attention is when we're going through that [7:20] chart, the thing one of the big lessons we  learned is the model's actually the cheap part [7:24] in terms of labor. Everything to the left of the  code and underneath it decides your bill and your [7:29] quality at the end of the day. Prompting  is becoming a thing that we can actually [7:32] automatically do as part of the infrastructure to  change that prompt so it's optimized for the the [7:38] frontier lab it's going to or to the open weight  model it's going to. It requires infrastructure [7:43] and context to do it well. So, if you optimize  that box in the middle, that code box, [7:48] you've ultimately optimized nothing in the entire  system. So, as you dig deeper into how do we [7:53] teach people to write better prompts, you're not  actually seeing the gains that you would see if [7:57] that energy is directed at an infrastructure level  or a discovery and uh context management level. [8:04] So, these are some of the big lessons we learned  to the left of the code. It's It's really about [8:08] feeding the model or paying for the mess  ultimately. So, context engineering means [8:13] giving the agent the right slice of the system at  the right moment. So, if you give wrong context, [8:19] it's actually the most expensive input. It  starts generating things you didn't ask for. It [8:22] It has too much depth. It's thinking through edge  cases that don't even won't even actually appear [8:27] if you're overshooting the the model complexity as  well. Information architecture is another piece. [8:33] This is an area where we found is absolutely  critical to think through. And And what we [8:38] mean by information architecture is if you think  about the evolution of of all that traceability uh [8:44] and a signal that's coming in. You can  take that traceability and turn it into [8:48] data that usually lives in your various  data layers, Splunk, uh your caches, your [8:54] uh your other areas, your analytics data lake,  etc. But, in order for that to become valuable, it [9:00] actually has to get translated into information.  In order for this system to understand how to [9:05] translate your data and your your kind of audit  trace into information, it actually has to seen [9:10] good processes by humans that have turned it into  information reliable ways. That then requires you [9:15] to step back and think about how do we structure  the information in our enterprise environment, [9:20] specifically from a technology perspective, to  make that context discoverable and consistent. [9:26] It's a very large problem that reckons back  to maybe, you know, a decade or two decades [9:31] ago when we were trying to figure out how to  structure SharePoint's. Um because ultimately, [9:36] the models don't have a consistent uh concept of  time. It can't tell you which of these documents [9:41] is the most recent unless you're very explicit  about how you structure that information for it [9:46] to consume. Give it old context, old code,  out-of-date functions, out-of-date calls, [9:51] out-of-date pointers, and you're just paying  for out-of-date information from a model. [9:56] Finally, it's coordinated knowledge  generation is absolutely critical for us. [10:00] As we started down this pathway, we we figured  out ways to organize information that had been [10:05] created previously. It was actually pretty easy.  Bringing in some, you know, actually library and [10:10] information science experts to think through  how we would do that. It's the information [10:14] that's generated on the fly. It's It's the  meetings. It's the Slack conversations. It's the [10:20] hallway conversations. It's the uh aha  moment that happened in a code review. [10:26] That's the piece that we really started to  think through how we coordinate, and it's a [10:30] absolutely critical piece of this process to  figure out. So, this is where we split apart [10:36] runtime context from historical context. By  runtime context, we're really referring to these [10:41] coordinated moments where we need to consume  that context and weight it against history. [10:47] One of the interesting problems that  you run into is humans are, you know, [10:51] have faults. We forget things. We misremember  things. We shoot from the hip a little bit, maybe [10:56] not know the full context of what happened last  time because that person's, you know, on vacation. [11:02] That's one of the critical pieces in that runtime  coordination that you actually have to figure out [11:06] because it's not necessarily a hallucination.  It's bad information. So, how do you then rate [11:10] and grade the quality of the information in terms  of consistency with that information architecture? [11:18] Then we talk a little bit about the platform.  So, as we're generating that context, how do we [11:23] actually mobilize that into a transformation? For  us, this is five categories. It's orchestration, [11:28] like I mentioned, we're we're working with  a design partner on how we orchestrate [11:32] in a way that's agnostic to platforms. Our vision  is ultimately you want to you know build agents in [11:39] cursor, you want to build things in agent force,  you want to use data bricks genie, you want to use [11:43] cloud agents, open AI agents. You've got this  random server running in a DGX spark box that [11:50] you got agents on an open weight model. Great.  As long as it is orchestrated by my system, [11:55] beautiful. Go for it. Knock yourselves out.  That really gives us the freedom to start [12:00] thinking about experimentation within  boundaries. So we control the perimeter [12:05] and ultimately we empower software engineers  to find better and more interesting ways [12:09] and more impactful ways to do the work that they  need to do. That's also powered by a gateway. [12:14] That gateway controls all the ins and outs of that  perimeter. It's a it's a layer that allows us to [12:20] secure by design and then also follow through in  the orchestration layer with that security carry. [12:27] Caching becomes absolutely critical in the  system because as you hit more and more code [12:32] getting generated, you know, a lot of the stuff  that we actually generate pulls on things we've [12:36] already know. It's much easier to infer from  cache than it is to infer on on the fly. And so [12:42] semantic caching allows us to turn turn  repeated work into almost near zero cost. [12:48] That becomes even more true as you layer in  open weight models and you share your context [12:52] across them. That ultimately gets handled by the  orchestration which goes through the gateway. [12:58] The security layer is also an area that we pay  particular attention to as a regulated industry. [13:03] So we will focus on things like how do we manage  private information? What do we count as private [13:08] information in our industry? How do we think  about private information and the risk it [13:12] incurs being handled by a non human individual or  agent and how do we think about it differently in [13:20] that context? And then finally trust and safety  is absolutely critical to us. In this trust and [13:24] safety bucket we put two types of work. We have  our eval work which tells us whether or not the [13:29] work being done is actually has veracity, speed  and truthfulness. And then trust and safety also [13:35] focuses on our hallucination detection. If you  think about a hallucination is actually a fairly [13:41] easily solved mathematical problem where it's  low information density problems. Can we use the [13:47] trust and safety system to identify places in our  code where we lack a density of information that's [13:52] needed to go through the coordination agent come  back into the pipeline and enrich that so you can [13:57] reduce the risk of the hallucination process. So,  these are not necessarily overhead. There's cost [14:03] to them, but they're where your cost and quality  get one. As you tune each of these pieces of that [14:08] infrastructural system, there are in drastic  drops in the cost of inference. Usually to to [14:15] pretty significant savings. Looking at some of our  partners, they've they've quoted other customers [14:21] are seeing upward near 80% cost reduction  as they tune different levers in this space. [14:28] So, all of that's great. We can build technology  all day long, but the problem is is we also have [14:33] these humans working in this and they're fallible  and they're hungry to learn and they're excited [14:37] and they want to do work. So, part of our  cultural transformation is not just putting these [14:41] infrastructural pieces in place, it's explaining  why those infrastructural pieces are in place [14:46] and helping people embrace that logic so that they  can first figure out how to do it on their own, [14:51] but then second, the most valuable thing software  engineers ever do is break your platform and [14:56] tell you why their broken version of it is  better. Like that is an amazing experience for [15:00] those of us who work in this space. And so, we  do that cultural transformation to focus teams [15:06] on why we do some of the things. So, one of the  big cultural transformations and in in fact one [15:11] of our biggest risks is the misplaced notion that  a bigger model and more effort is the default. [15:18] That is the wrong default. So, left untrained  engineers reach for that largest model every time [15:24] and then they bump that that reasoning up to max  and it becomes a very expensive process if it does [15:30] it correctly. We're seeing instances where it  doesn't even do it correctly, right? It starts [15:34] to try and find and generate edge cases that you  know, what if there's an electrical outage in this [15:40] Rhode Island town and somebody's cell phone is  plugged in and it skips off Wi-Fi because their [15:46] router went down and there's a weird electrical  poles in your like what like calm down, right? So, [15:52] that big model can really run up those costs  not just from being an expensive inference, [15:57] but being for an expensive paranoid inference. So,  the result of that is really a bigger bill on what [16:04] work on work that never needed the horsepower to  do it in the first place. Sometimes it comes back [16:08] with a great answer which is very expensive. Over  time, we see worse quality over reasoned output [16:14] on simple well-scoped task. And the social part  of this that that really kind of worries us and [16:20] resonates with us as a as an AI team at FanDuel is  the quiet deskilling that happens. And what I mean [16:26] by that is engineers start you know, focusing on  other things they find interesting which is cool. [16:32] But they're trying to one-shot everything  with an over-the-top top-end model. [16:36] They stop thinking through the process. They  stop thinking how do I break this? How do I how [16:39] do I make sure this is right? How do I how do I  really push the envelope of this? And they get to [16:44] a point where they're like, well, I can manage  36 agents at the same time cuz I can write two [16:49] sentences and send it off to do a thing and hope  it hits the right context layer. That's the wrong [16:54] level. That results in an engineering organization  that starts to slide backwards. And so, we think [16:59] a lot about model routing and model skilling  is really about pushing the envelope on the [17:04] next generation of what engineering looks like.  You can't be a product engineer and not know how [17:09] to build products. Can't be a back-end engineer  and not know how to structure APIs and back-ends. [17:16] Those are absolutely critical. The one-shot risk  of these large models is that we start to see [17:20] engineering talent, especially junior engineering  talent, losing that absolutely critical context. [17:26] One of the things that's really interesting that  we we hadn't expected that I think it has been [17:31] fascinating is when we look at junior engineers  and we share with them some of the prompts from [17:36] back in the day of things like Sonic where you  had to be very highly, you know, explicit about [17:41] the instructions and what you wanted it to do and  how to do it and what samples you're including. [17:46] Those are incredible teaching materials for junior  folks, especially for platform engineers, because [17:52] you can you not just you don't only see the code  that comes out of it, but now finally an engineer [17:57] had to make every step of the way they thought  about this absolutely crystal clear explicit. [18:03] And so they become very good training materials  for this is how we do platform engineering. This [18:07] is how we build products or this is how we build  data pipelines. So, one of the thing that's one of [18:13] the the stats that we find really interesting  in this space also to share here is that 80% [18:19] of industry workloads can run on these smaller  models. Sonic is a great workhorse, uh, especially [18:26] if you're using planning and other approaches in  the Claude ecosystem. We're seeing the lower end [18:32] open AI AI model, same thing. If you're explicit  and you've got that great plan up front, you're [18:37] good. The missing layer that we're experimenting  with that I think is is actually very interesting [18:43] is on the plan. A plan is great, but it's never  actually executed the plan. You actually need [18:48] something between the plan and the execution to  show that you're headed in the right direction. [18:52] And that's a place where we see a lot of frontier  models playing an absolutely critical role. [18:59] To that, as we think about upskilling engineers  and where we upskill them to, one of the critical [19:04] skills we're putting on the plate and and I was  I was talking about this just a few minutes ago [19:09] with a with one of our partners is benchmarking  the emails. The problem is as a new model drops [19:15] and they're like it's 2% better. Everybody's  like it's 2% better, I want it. It's 2% better. [19:21] And you have to kind of step back and be like,  all right, I don't want to be I don't want to be [19:24] that stats guy. But it's a plus or minus 4% margin  of error. So the 2% doesn't really mean anything. [19:33] But one of the things we've also learned  from our own benchmarking experience as [19:36] we build out our model ops teams is that those  benchmarks applied to software, you know, the [19:41] SWE bench and everything else don't necessarily  translate to your production environment and your [19:46] production code. So we've started actually  benchmarking against real jobs that we see [19:50] inside to understand is this model worth the  cost? Is that Is it a real upgrade for us or [19:57] are we better off continuing on the pathway we  are? And this is both in terms of model families [20:02] moving up and down the the size of the model, but  also across the generations of those model 4.8, [20:07] 4.9, 5.3, 5.4, 5.6. Um all of those are really  interesting to look at outside of of the public [20:16] benchmarks in your own benchmarks. So one of  the areas we've focused on helping engineers [20:20] uh get upskilled is really in this email domain.  Um writing emails is just like writing unit tests. [20:28] It's going to become the centerpiece and critical  step in the process of developing software. [20:34] Um it's it's a skill that isn't innate though.  It's not one we're seeing uh junior engineers [20:40] coming out of college having learned. In  the same way that I think if we went back, [20:44] you know, 5 10 years, test-driven development  also wasn't something we were seeing [20:48] kids coming out of college with. So it's just  another one of those moments where we're focused [20:52] on how thinking about how these things work as  part of that software engineering transformation. [20:58] So uh let me go back to this one point that  I think is actually very interesting. Uh [21:05] There's one place where we were absolutely  wrong coming into this a year ago, [21:08] and it was around duplication. I was like,  I don't care how many skills are out there, [21:12] how many agent MDs and their markdown  files. Like, storage is cheap, who cares? Uh [21:18] as we matured our system and as we scaled,  one of the interesting things we discovered [21:22] is that it's wrong on the economics. And that's  because each one of these can either pull from [21:27] cash or can inference live. If your agents are  all written differently for the same tasks, they [21:33] always inference live. You're paying a premium  on this. They're never hitting your cash. And so, [21:38] that was an area that we really transformed  the organization in terms of educating of like, [21:43] what's the difference between a cached inference  and a developed enter- inference. So, that's [21:49] that's the total. There's There's more here, but I  will just leave us on this slide and close there.