Data Teams Shift Focus to Context Curation as AI Autonomous Agents Overload Systems

AI 自主智能体引发系统过载，数据团队重心转向上下文管护 企业级 AI 部署受困于实时流处理与 Token 开销，迫使数据工程团队优先进行上下文管护与智能体细粒度访问控制，而非手动构建数据管道。 面对 AI 带来的系统负载与成本压力，企业数据团队正从手动管道构建和过度追求复杂实时流处理，转向上下文管护、特征库分区与评估基准（evals）建设。随着模型上下文协议（MCP）等工具使非技术人员能够直接使用 AI 查询内部数据，数据工程的核心职责已演变为确保数据质量与编排声明式工作流。展望 2027 年，自主数据智能体的激增、细粒度身份与访问控制（IAM）以及算力和能耗限制，将成为企业数据架构面临的主要瓶颈。 通过优化模型外围的确定性系统与上下文管理，并依据更新频率对特征库进行分区，能够有效避免智能体冗余重试，从而降低 Token 消耗与计算成本。 随着模型上下文协议（MCP）让非技术员工能够直接使用大模型查询内部业务数据，数据工程团队正从重复制作报表转向上下文管护、数据清洗与评估基准构建。 预计到 2027 年，大量并发运行的自主数据智能体可能造成底层基础设施过载，智能体的细粒度身份与访问控制（IAM）以及能耗与算力上限将构成核心挑战。

Building an effective data stack for AI requires moving away from speculative infrastructure hype and premature scaling toward pragmatic engineering, deterministic context management, and strict evaluation frameworks. Pragmatic Architecture Over Streaming and Tooling Hype (18:12) The panel emphasizes that infrastructure choices must target concrete business problems rather than technological novelty. Startups are warned against implementing Google-scale architectures before achieving product-market fit. Large-scale enterprise deployments encounter structural bottlenecks that simple architectures cannot handle; for example, basic retrieval-augmented generation (RAG) pipelines that succeed on several thousand documents completely fail when deployed across 500 million documents. Simultaneously, speakers caution against the default adoption of real-time streaming architectures. While complex streaming pipelines are intellectually engaging, they create immense operational friction. Outside microsecond-sensitive environments like high-frequency trading, incremental micro-batching is easier to operate and provides sufficient data freshness. Teams also face severe productivity drains from evaluating dozens of nearly identical niche tools, such as automated business development bots, instead of deploying deterministic logic. Controlling Costs Through Deterministic Harnesses and Context (23:36) While industry discussions often center on token pricing and model weights, the panelists observe that significant cost and performance optimizations originate in the deterministic infrastructure surrounding the model. Investing in structured context management and domain-specific harnesses reduces token waste by preventing repetitive agentic retry loops and synthesis cycles. In data pipelines, modern feature stores partition features by update velocity—separating static attributes like nationality from real-time activity metrics—to avoid re-evaluating invariant data during inference. Token reduction directly improves latency and model accuracy. Clear, structured technical documentation serves as an essential prompt input, allowing models to retrieve precise facts without expensive broad context scans. In regulated domains such as finance, where customer data cannot be used for training or testing, algorithmic synthetic data provides a privacy-compliant proxy for evaluating real-world behavior. From Pipeline Coding to Context Curation and Evals (30:53) The data engineering role is transitioning from manual processing tasks, such as tuning Spark jobs, toward declarative workflows and agent-assisted operations. Engineering teams increasingly deploy RAG-based monitoring agents to analyze system logs and operational runbooks to diagnose pipeline failures and data contamination. Emanuel Bettelheim observes that navigating this transition across Bloomberg's 27,000 employees poses a much larger cultural challenge than deploying the technology itself. Jordan Tigani notes that integrating Model Context Protocol (MCP) servers connected to models like Claude and ChatGPT at MotherDuck allowed non-technical staff to query internal data autonomously. Rather than eliminating data teams, this self-service shift frees engineers from repetitive dashboard generation. Instead, data engineers are pivoting toward context curation, data sanitization, and the construction of rigorous evaluation benchmarks (evals) to ensure automated queries remain consistent and accurate. Forecasted 2027 Bottlenecks: Agent Sprawl and Compute Looking ahead to 2027, the panelists predict major architectural pressure points driven by autonomous systems and resource constraints. Jordan Tigani forecasts that organizations will deploy dozens of autonomous data agents for cleaning, ingestion, and monitoring. Without strict boundaries, this sprawl risks overwhelming legacy infrastructure. Tigani also highlights fine-grained identity and access management (IAM) as an acute operational challenge, as enterprises must establish strict permission boundaries for autonomous agents acting across corporate databases. Bhanu Prakash Reddy Rella projects that compute ceilings, budget restrictions, and energy limits will become the dominant bottlenecks. As organizations shift away from unconstrained experimentation, data teams are facing strict caps on token usage and infrastructure spending. Balancing model capabilities with energy efficiency and sustainability constraints will force engineering teams to prioritize lightweight, highly optimized architectures. While AI democratizes data querying across non-technical teams, organizations face unresolved friction in securing autonomous agent permissions, adjusting large corporate workforces to declarative workflows, and operating within strict compute and energy budgets.

来自Meta、Bloomberg与MotherDuck的技术专家在Ai4研讨中指出，AI驱动的数据架构正在摒弃过度追求实时流处理与盲目堆叠工具的炒作，转向以业务问题为核心的实用主义路线。现代数据栈的关键演进方向在于通过确定性系统与特征工程优化算力能耗，并推动数据工程从底层的管道编写转向上下文治理、评估体系构建与自主智能体编排。 避免过度工程与反思实时流处理 (18:12) 专家组强调，工程决策必须植根于解决真实的客户问题，而非盲目追逐新颖技术。初创公司常常陷入在验证产品与市场匹配度（PMF）之前就构建Google级超大规模架构的陷阱，甚至耗费大量周期在数十家同质化的AI商业开发代表（BDR）工具中做无谓评估。与此同时，企业级规模存在独特的扩展瓶颈：一套能轻松处理数千份文档的简单RAG检索增强生成系统，在扩展至5亿份文档的企业级场景时往往会彻底瘫痪。 讨论还质疑了行业对实时流式处理架构的执念。实时流处理虽然技术上极具挑战性，但会引入巨大的运维复杂度。除高频交易等微秒级需求外，绝大多数企业用例采用更易理解、运维负担更低的增量微批处理（micro-batching），即可满足业务对数据新鲜度的实际需求。 确定性架构优化与能耗成本控制 (23:36) 控制AI工作负载成本不能仅依赖模型路由或开源与闭源权重的选型，更需要优化围绕模型运行的确定性系统。投资于领域专属基础设施和上下文管理，能够从源头减少智能体重复重试与合成循环，从而直接削减Token消耗。此外，优质且结构清晰的技术文档被视为核心AI输入，能让模型直接定位关键事实，避免大范围高成本的上下文检索。 在底层数据架构中，现代特征存储（Feature Store）已演进为连接离线训练与在线推理的一致性边界。工程团队开始按数据更新频率对特征进行分桶管理：将国籍、出生日期等静态低频元数据与实时行为遥测等高频特征物理隔离，在线推理时即可避免对静态特征重复计算。针对敏感数据，金融等强合规行业严格禁止使用真实客户数据进行训练或评测，转而采用依据底层规律生成的算法合成数据作为隐私合规的代理方案；科技企业则更多利用合成数据生成动态销售演示和模拟仓储环境。 数据团队职能向上下文治理与评测演进 (30:53) AI的普及正在重构数据团队的日常权责。数据工程师以往大量时间用于调优Spark作业和维护底层重型处理框架，如今正转向构建元数据驱动的自动化管道与AI辅助运维工作流。通过RAG驱动的AI智能体解析系统运行时日志和运维手册，团队能够自动化诊断数据污染与管道故障。数据管理正走向声明式模式，工程师的日常工作从手写管道代码转变为编排自主智能体集群并设定测试目标。 在组织协同层面，MotherDuck通过部署连接Claude与ChatGPT的模型上下文协议（MCP）服务器，使非技术人员能够自主跨工单、用量指标与新闻流进行复杂查询。这一转变将数据团队从繁杂的临时报表开发中解放出来，其核心职能全面转向数据质量管理、上下文策展（Context Curation）以及建立严格的评测基准（Evals），确保模型在计算关键指标（例如第三季度营收）时能够稳定输出准确、一致的结果。 2027年架构瓶颈预测与安全治理挑战 展望2027年，专家们预测企业将面临智能体蔓延与算力能耗的双重瓶颈。Jordan Tigani指出，随着数据团队像软件开发者运行编程智能体一样部署数十个自动化数据智能体来执行清洗、处理与监控任务，缺乏管控的自主智能体极易导致现有基础设施过载瘫痪。与之伴随的是细粒度访问控制与身份权限管理（IAM）危机，如何在企业环境中定义智能体代表谁并精准限制其查询权限将成为极其复杂的安全挑战。 Bhanu Prakash Reddy Rella则预测算力上限、能耗约束与模型执行效率将成为决定性瓶颈。随着企业从无限制的AI技术投入转向严格设定Token消耗上限与控制算力预算，构建更经济、更绿色的低能耗模型执行生态系统与轻量级评测机制将成为未来架构的核心诉求。 本场讨论表明，现代AI数据栈的成熟标志是从盲目追求技术前沿转向对业务价值、算力能耗与确定性工程的审慎权衡。然而，如何在成规模的企业（如拥有27,000名员工的Bloomberg）中完成组织流程与信任边界的文化再造，并在2027年可能到来的智能体泛滥与权限失控面前维持数据安全与治理标准，依然是业界有待解决的长期挑战。

## Background 本视频由科技会议频道 Ai4 于 2026 年 8 月 21 日发布在 YouTube 平台上。内容围绕‘构建面向 AI 的现代数据栈：哪些奏效，哪些失效’展开，邀请了来自 Meta、彭博社（Bloomberg）与 MotherDuck 的一线技术领袖，深入探讨数据团队在支持复杂 AI 工作负载时的管道设计、数据新鲜度治理、成本优化及底层工具选型的硬核实战经验。 ## TL;DR 圆桌深入剖析了传统数据基础设施在现代 AI 工作负载与智能体（Agent）冲击下的崩溃点与重塑方案。嘉宾们指出‘智能体投机’正在击穿传统的冷热数据分层，警惕团队盲目追求硬实时流处理的技术虚荣，并分享了稀疏路由分流、以 CPU 确定性系统削减 Token 支出以及金融级合成数据等核心实践。讨论进一步揭示了 MCP 协议推动下的‘全民数据交互’趋势，明确数据团队的角色正从做报表看板彻底转向上下文治理与评估体系构建。最后，专家们预警了并发智能体对底层架构的冲击、Agent 权限控制的复杂性以及未来算力与能耗紧缩带来的严苛约束。 ## Atmosphere & style 整场对谈务实、辛辣且充满工程一线的真实感，主持人的巧妙穿针引线与嘉宾们的坦率交锋碰撞出高密度的思想火花。台上既有 MotherDuck CEO 幽默自嘲过去曾误判 AI 影响的松弛感，也有彭博与 Meta 面对数亿级复杂生产环境时的严谨与审慎。对话毫无空洞的技术兜售，而是以批判性思维穿透行业炒作泡沫，展现出一种面对剧烈技术变革时兼具敬畏与敏锐的技术掌舵者风范。 ## Key takeaways - [03:04](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=184s) Meta 推荐系统通过 Kafka/Spark 流式打标与稀疏网络路由分流，将 90% 的简单请求导向轻量模型，降低了高达 40% 的算力开销。 - [03:04](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=184s) 彭博终端推出对话式 AI，为打破异构数据孤岛并确保溯源，专门训练 LLM 生成并执行专有的彭博查询语言（BQL）。 - [11:35](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=695s) AI 智能体的“投机性探索（Agentic Speculation）”打破了传统的冷热数据分层与长尾假设，可能瞬时冲击工程师从未优化的冷门数据链路。 - [11:35](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=695s) 为防止多智能体并发冲击底层数据仓库，MotherDuck 采用独立的云端计算实例隔离方案，保障各智能体高频调用的弹性与稳定性。 - [16:09](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=969s) 企业在架构选型中应警惕盲目追求微秒级“实时流处理”，绝大多数非高频交易场景通过增量微批处理（Micro-batch）即可兼顾成本与性能。 - [21:23](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1283s) 评估 AI 工具应坚持以业务痛点为中心倒推架构，避免把精力过度消耗在评测同质化 AI 工具和无谓的技术内耗上。 - [23:36](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1416s) 特征存储（Feature Store）应建立基于数据新鲜度（Freshness）的分区分层治理策略，避免对静态特征进行重复计算以减少推理流水线开销。 - [25:07](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1507s) 在 AI 任务中削减 Token 消耗不仅能降低成本与延迟，还能通过减少噪声提升准确率，打破了传统数据工程中“降本必降速”的权衡定律。 - [28:38](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1718s) 在金融等强合规场景下，彭博采用保留真实统计偏差与分布的高保真合成数据进行系统评估与基准测试，解决真实数据无法直接用于训练的合规难题。 - [31:33](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1893s) 数据工程运维开始深度结合 RAG 智能体，通过整合日志与操作手册实现自动化故障根因定位与排错建议生成。 - [35:08](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=2108s) 随着 MCP 架构与对话式查询的普及，数据团队的核心职责从制作低效仪表盘转变为“上下文治理（Context Curation）”与“评估体系构建（Evals）”。 - [38:21](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=2301s) 展望 2027 年，企业级 AI 将面临大规模数据智能体并发过载、智能体身份与访问权限控制（Access Control）以及严格的算力能效限制等多重架构瓶颈。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=4s) 嘉宾介绍与 AI 时代现代数据栈的重塑挑战 主持人 Emil Protalinski 率先调动全场气氛，向台下抛出谁对探讨数据栈与数据管道感兴趣的问题。他指出，数据领域一直在适应快速变化，而 AI 的到来让这一切变得更加迅猛。随后，三位来自顶尖科技公司与初创企业的专家依次登台自我介绍，分享各自在 AI 与数据基础设施领域的从业背景与核心聚焦点。 来自 Meta 的高级数据工程师 Bhanu Prakash Reddy Rella 拥有超过 12 年的从业经历，近期主要专注于构建高能效的 AI 数据管道，以支撑机器学习推理生态系统，并将探索更廉价、更环保的推理模型部署方式作为首要任务。彭博社（Bloomberg）数据与分析主管 Emanuel Bettelheim 则拥有超过 25 年的技术产品经验，此前曾在初创公司及 Google 任职；他在彭博的核心使命是彻底重塑金融与 AI 的未来结合形态，既要通过 AI 强化彭博内部的系统能力，也要面向外部金融市场客户输出 AI 赋能。MotherDuck 联合创始人兼 CEO Jordan Tigani 则深耕数据分析系统 15 年，曾参与创办 Google BigQuery 并担任工程负责人，后任 SingleStore 首席产品官。Jordan 幽默地指出，台上的同僚们专注于“使用数据”，而他则专注于“打造工具来帮大家使用数据”。在他看来，过去可以清晰看清技术演进的列车开往何方并顺势跟进，但如今 AI 就像当年的云计算转型一样充满了不确定性，作为初创公司的核心挑战就在于精准预测未来的技术走向并抢先布局。 - **Bhanu Prakash Reddy Rella**：Meta 高级数据工程师，专注高能效 AI 数据管道与更环保低成本的 ML 推理架构。 - **Emanuel Bettelheim**：彭博社数据与分析主管，专注金融大模型应用与跨多源异构数据的统一赋能。 - **Jordan Tigani**：MotherDuck 联合创始人兼 CEO，曾任 Google BigQuery 工程创始成员与 SingleStore CPO。 --- ### [03:04](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=184s) 重新设计数据栈与产品创新实践 主持人请三位嘉宾分享近期对哪些产品或架构进行了大刀阔斧的重新设计。Jordan Tigani 首先介绍了 MotherDuck 的实践。他们注意到现代数据栈中部分领域存在代码化与管道迁移的需求，因此构建了一项基于 Python 的数据管道运行时能力。这一设计让系统在接到触发器或定时调度时，能灵活借助 Claude 或 ChatGPT 等模型快速拉取 dbt 管道、执行数据转换并调用开源连接器提取数据，同时无缝连接 MotherDuck 的上下文层、可视化层与数据仓库。 ```mermaid flowchart TD accTitle: Meta 推荐平台推理管道改造流程 subgraph DataEngineering["数据工程层 (Kafka / Spark Streaming)"] A["用户请求流"] --> B["元数据打标与复杂度特征提取"] end subgraph Routing["稀疏路由层 (Sparse Network)"] B --> C{"请求难度判定"} C -- "90% 简单/可预测请求" --> D["轻量化模型 (Lighter Model)"] C -- "10% 复杂/长尾请求" --> E["全量深度模型 (Dense NN / LLM)"] end subgraph ComputeImpact["计算优化效果"] D --> F["大幅降低 GPU 算力消耗"] E --> F F --> G["整体算力成本降低达 40%"] end ``` Emanuel Bettelheim 随后分享了彭博社的重大变革。作为金融界的“中枢神经系统”，彭博终端（Bloomberg Terminal）沉淀了 45 年的行业工作流，积累了数千个功能组件。今年 1 月，彭博正式公开推出了对话式 AI 功能，支持用户直接与海量非结构化数据（如企业财报、业绩电话会议记录、宏观经济数据）和结构化数据进行对话交互。为了在几秒钟内从数百个针对各自领域深度优化了数十年的异构数据库中获取精准答案，彭博不得不打破烟囱式架构，在顶层依赖专有的“彭博查询语言”（BQL，Bloomberg Query Language）。为了保证模型回答的可解释性与数据溯源，彭博团队投入巨大精力训练 LLM 理解并生成这一内部独有的 BQL 查询语句，将自然语言转化为精准的底层查询。 Bhanu Prakash Reddy Rella 则深入剖析了 Meta 推荐平台推理架构的优化历程。随着流量增长，Meta 面临推理成本与基础设施开销增速远超业务流量的严峻挑战。在传统大厂思维中，最直接的应对方案往往是“无脑加 GPU”，但团队通过深入分析发现：高达 90% 的用户请求非常简单且高度可预测，仅有 10% 的复杂请求才真正需要调用高算力消耗的稠密神经网络（Dense Neural Network）或大模型微调。为此，Meta 在数据工程链路中利用 Kafka 和 Spark Streaming 在流经的数据上打上元数据指针，结合稀疏神经网络（Sparse Neural Network）进行动态路由分流，让 90% 的简单请求流向轻量模型，从而一举削减了高达 40% 的算力消耗。 - **MotherDuck 管道扩展**：集成 Python 运行时，支持 AI 辅助生成代码、dbt 管道转换与统一上下文管理。 - **彭博对话式 AI 与 BQL**：克服 45 年历史积累的异构数据库孤岛，训练 LLM 熟练使用专有 BQL 语言实现数据溯源。 - **Meta 推荐推理优化**：通过稀疏路由与元数据标记将请求按 90%/10% 分流，算力开销降低 40%。 --- ### [10:14](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=614s) 传统数据架构在现代 AI 工作负载下的崩溃临界点 主持人进一步提问：面对现代 AI 工作负载的冲击，传统 legacy 数据基础设施中最先发生“崩溃”的薄弱环节究竟在哪里？Bhanu Prakash Reddy Rella 指出，传统系统在设计之初只针对标准机制和单一数据类型进行了优化。一旦企业尝试在跨多系统的大规模场景下引入多模态、多样化的复杂数据类型，旧系统的扩展能力就会迅速见顶并出现瓶颈。在向 AI 架构转型的过程中，企业必须投入大量精力进行数据审计与管道重构，以确保整套基础设施真正具备兼容多源异构数据类型的大规模扩展能力。 Emanuel Bettelheim 对此深表赞同，并补充道，彭博系统在过去 45 年里高度针对人类专家既定的交互工作流进行了极致优化；然而在引入 AI 代理（Agents）等新型工作负载后，调用模式和系统吞吐的动态性发生了颠覆性变化，对底层数据栈的弹性和统一抽象能力提出了前所未有的要求。 ### [11:35](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=695s) 智能体投机性探索打破“冷热数据”边界与计算隔离架构 随着 AI 智能体（Agent）在底层数据基础设施中的深度应用，其行为模式与人类用户展现出了根本性的差异。人类的探索方式通常具有规律性，而伯克利近期的一篇论文将其描述为“智能体投机（Agentic Speculation）”。智能体为了综合与归纳海量信息，往往会进行高度投机性的发散探索，直接涌向那些工程师从未进行过性能优化的冷门路径。 这种机制直接冲击了数据架构中的长尾理论。在以往的设计中，处于长尾区的数据访问频率极低，仅需维持基本的可用性；但在智能体眼中，某个长尾节点可能突然被判定为极其关键的信息，从而遭到集中且密集的调用。传统的“冷热数据（Hot/Cold Data）”分层逻辑与长尾假设在智能体时代开始失效，迫使数据系统的底层设计必须彻底重构。 ```mermaid flowchart TD accTitle: Traditional vs Agentic Data Access Patterns subgraph Traditional["传统人类访问模式"] H["人类用户"] -->|规律访问| Hot["热数据层（高频优化）"] H -.->|偶发低频| Cold["冷数据长尾（低成本存储）"] end subgraph Agentic["AI 智能体访问模式"] A["智能体集群"] -->|投机性探索| Spec["全量数据穿透探索"] Spec -->|瞬时高频冲击| Cold Spec -->|并行并发调用| Hot end Cold --> Break["传统长尾未优化链路面临崩溃风险"] ``` 针对智能体高并发访问引发的系统崩溃风险，计算层面的架构隔离成为了关键解法。人类对数据库的并发冲击力存在天然上限，但大量智能体如果同时“在同一份数据上狂欢”，很容易瞬间击垮数据系统。MotherDuck 通过云端计算隔离来应对这一挑战：为每个用户及每个智能体分配独立的云端计算实例，各自按需弹性伸缩与关停。这一机制最初是为多租户 SaaS 应用独立访问而设计的，但在面对海量智能体独立、高频轰击数据的场景时展现出了极强的适应性。 - 伯克利研究提出的“智能体投机（Agentic Speculation）”揭示了 AI 探索数据路径的高度不可预测性。 - 传统的冷热数据分层模型在智能体高强度穿透长尾数据时出现失效。 - 采用每个智能体独占独立计算实例的架构，可有效避免并发流量相互干扰与集群过载。 --- ### [14:11](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=851s) 流量毛刺与新型非结构化数据倒逼底层管线重构 在实际落地场景中，数据格式的复杂化与智能体放大的流量毛刺正直接导致原有数据管道的破裂。Meta 在数据接入层面面临着前所未有的挑战：来自多种数据源的流式非结构化数据与异构事件格式不断涌入，直接击穿了传统的处理管道。面对这种结构性变化，单纯在原有链路上进行微调或局部修补已无济于事，必须从零设计全新的处理架构来适配多模态与流式数据。 彭博（Bloomberg）作为金融行业的核心基础设施，虽然历经数十年构建了高容错系统，能够对外屏蔽大部分故障，但其后台同样承受着前所未有的压力。智能体的介入在过去几乎没有流量的冷门业务区域引发了前所未有的流量剧烈波动（Spikiness）。这促使彭博从最底层的数据摄入机制开始重新评估整个数据流向。 过去的金融数据清洗与质量保障高度依赖自动化确定性系统结合大量人工维护，系统对微小变更极度敏感；而在 AI 时代，团队正在全面重塑从底层数据流到顶层彭博终端（Bloomberg Terminal）展示的完整链路，重新思考 AI 原生环境下的数据交互形态。 - Meta 通过重构全新架构以应对流式、非结构化事件与复杂数据格式对传统管线的冲击。 - 彭博后台面临智能体在冷门业务路径上引发的异常流量毛刺（Spikiness）。 - 彭博正逐步淘汰传统依赖大量人工修复的确定性数据管线，转向 AI 原生的端到端数据流。 --- ### [16:09](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=969s) 警惕架构陷阱：避免过度追求实时性与技术虚荣 在构建 AI 项目时，最普遍的架构失误往往来自于对“实时性（Real-Time）”的盲目执念以及脱离业务场景的技术崇拜。许多团队在项目启动初期就默认所有业务需求都必须以超低延迟的实时流处理实现，从而在算力和 GPU 上投入巨额预算。 合理的做法是在立项阶段进行严格的需求分级，明确区分真正需要微秒级响应的核心场景与可承受数秒甚至批处理的次要场景。在盲目采购昂贵算力之前，应当优先通过数据工程手段进行优化，例如引入文本解析分流（分别交由轻量级模型和重型模型处理）以及构建特征库（Feature Store）等。 ```mermaid flowchart LR accTitle: AI Architecture Evaluation Matrix subgraph EnterpriseStage["企业生命周期考量"] direction TB Startup["初创团队 / 早期阶段"] -->|核心重点| PMF["聚焦 PMF 与客户真实问题<br/>避免过早搭建复杂基础设施"] Scale["成熟企业 / 规模化阶段"] -->|核心重点| Complexity["重视超大规模落地复杂度<br/>5亿文档 RAG ≠ 几千文档 Demo"] end subgraph LatencyChoice["时延架构选型"] direction TB RealTimeCheck{"是否为高频交易等<br/>微秒级硬实时场景？"} RealTimeCheck -- 否 --> MicroBatch["增量微批处理（Micro-batch）<br/>（降低复杂度与推理成本）"] RealTimeCheck -- 是 --> ComplexStream["复杂流式架构（Streaming）<br/>（专属算力与超低延迟）"] end ``` 企业在评估 AI 架构时还需结合自身的成熟度阶段，避免陷入技术偏见： - **初创企业阶段**：首要目标是寻找产品市场匹配度（PMF），切忌在验证商业模式前就盲目照搬 Google 级别的复杂基建，“不要让完美成为优秀的敌人”。 - **大型成熟企业阶段**：不能低估 AI 从原型到生产环境的复杂度。在几千份文档上通过简单向量嵌入（Embeddings）和检索增强生成（RAG）构建聊天机器人轻而易举，但当数据规模跃升至 5 亿份文档时，原有方案将彻底失效。 - **流式处理与批处理的边界**：很多团队所定义的“实时”往往只是 5 到 10 分钟级别的延迟，这完全可以通过增量微批处理（Micro-batch）高效完成。流式处理系统（如 Dataflow 类架构）虽然工程精妙，但带来了极高的系统复杂性与理解成本；除非涉及高频交易等微秒级应用，否则增量微批处理已能覆盖绝大多数应用场景。 --- ### [20:57](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1257s) 评估新兴 AI 工具：在泡沫与实用价值之间保持清醒 面对当前 AI 生态中层出不穷的新工具与技术框架，技术决策者亟需建立审慎的评估机制。面对海量涌现的技术，最理性的第一反应应当是“先默认其属于市场炒作（Hype）”，进而通过严格的业务场景测试与 ROI 评估，筛选出真正具备实际生产价值的工具，避免被概念泡沫裹挟。 ### [21:23](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1283s) 避免工具内耗：从业务问题而非 AI 炒作出发 在团队引入前沿 AI 工具时，如何在敏捷创新与安全治理之间取得平衡成为讨论焦点。Jordan Tigani 指出，当前的 AI 技术正在经历巨大的阶梯式飞跃，企业绝不能直接下令禁止团队尝试新事物；但另一方面，数据安全与权限合规问题又如影随形。初创公司在试用新工具时虽然更具灵活性，但面对员工随手添加的新型 Slackbot，也必须警惕其背后的数据流向与潜在风险。此外，市场上涌现出大量同质化 AI 工具（例如针对销售与业务拓展的各类 AI BDR 工具），如果团队将精力全部耗费在“评估哪款工具最好”上，就会陷入本末倒置的陷阱，变成“为了工具做优化，而非为了解决问题做优化”。 ```mermaid flowchart LR accTitle: 业务驱动与工具驱动的决策路径对比 subgraph S1["工具驱动 (陷入内耗)"] A1["AI 炒作 / 新工具涌现"] --> B1["盲目试用 / 评测工具"] B1 --> C1["忽视真实业务场景"] end subgraph S2["业务驱动 (健康模式)"] A2["锁定客户核心痛点"] --> B2["确定业务算力需求"] B2 --> C2["选择极简匹配工具"] end ``` Bhanu Prakash Reddy Rella 补充道，工程实践中并不存在所谓的“魔法级降本工具”，只要现有的计算资源恰好能够满足业务需求，这本身就是最极致的优化。Emanuel Bettelheim 则从产品经理的底层思维出发，强调必须“爱上问题本身，而不是爱上解决方案”（"fall in love with the problem not with the solution"）。虽然 AI 是不可否认的颠覆性技术，但绝不能把 AI 硬塞进所有流程中，团队应当始终以终为始，从客户痛点倒推架构选型。 - **安全治理与创新的张力**：初创团队虽具灵活性，但对随意接入的未知插件（如 Slackbot）必须严格把控数据暴露风险。 - **避免工具选型内耗**：将时间耗费在对比市面上无数同质化的 AI 辅助软件上，往往会分散解决实际业务问题的精力。 - **以问题为中心逆向推导**：无论底层模型迭代多快，软件工程的核心原则依然是从客户需求倒推技术架构。 --- ### [23:36](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1416s) 特征存储与数据新鲜度策略：工程层面的降本增效 针对规模化扩张时如何在控制成本的同时保持模型性能的问题，Bhanu Prakash Reddy Rella 深入剖析了现代数据工程体系中的特征存储（Feature Store）机制。特征存储的角色早已从单纯的“存储仓库”升级为贯穿“模型训练（Training）”与“在线推理（Inference）”之间的高一致性服务层。 ```mermaid flowchart TD accTitle: 特征存储的数据新鲜度分层与推理架构 subgraph Ingestion["特征分流输入"] F1["静态特征 (国籍 / 出生地)"] F2["高频动态特征 (实时行为 / 状态)"] end subgraph FeatureStore["特征存储治理层"] S1["低更新频率分区 (长效缓存 / 静态元数据)"] S2["高更新频率分区 (实时计算 / 特征版本化)"] end subgraph Serving["模型服务层"] M1["在线推理流水线 (Inference Pipeline)"] end F1 --> S1 F2 --> S2 S1 --> M1 S2 --> M1 ``` 在特征治理层面，Bhanu 强调必须建立基于“数据新鲜度（Freshness）”的分层策略。用户的静态属性（如出生国家、国籍等基础标签）在几个月甚至数年内都不会发生改变，如果每次模型推理都对全量特征进行重新计算与刷新，会造成巨大的算力浪费。合理的方案是将特征库按新鲜度需求进行切分，仅对需要高频变动的动态特征执行高频更新，并配合特征版本控制（Feature Versioning），从而显著减轻推理流水线（Inference Pipelines）的算力开销与执行延迟。 - **特征存储的角色定位**：作为消除训练与推理数据偏差（Train-Serve Skew）的一致性中枢，而非单纯的数据存储介质。 - **新鲜度驱动的算力分配**：区分静态元数据与高频动态指标，避免在无变化的特征上重复消耗流水线算力。 - **特征版本化管理**：为在线推理流水线提供可追溯的特征版本支持，保障模型结果的稳定性与审计需求。 --- ### [25:07](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1507s) 算力与 Token 经济学：确定性系统优于单纯的模型调优 在行业普遍焦虑于 Token 成本、纠结于选择大模型还是小模型、开源权重还是闭源权重、路由调度（Routers）时，Emanuel Bettelheim 提出了不同的见解。他指出，大模型本身大概每三周就会迭代升级一次，团队过度针对单一模型做微调往往得不偿失。真正的降本增效突破口在于围绕模型构建的“确定性系统（Deterministic System）”和运行环境。以彭博的 BQL（Bloomberg Query Language）为例，这是一套运行在常规 CPU 集群上的领域特定语言，负责精确执行获取国家 GDP 并进行对比等结构化计算任务。在 Agent 架构中，如果确定性系统能在初次请求时为大模型提供精准剪裁的上下文，就能避免让模型陷入多达 15 次的重试与反复交互，从源头上削减庞大的 Token 支出。 ```mermaid flowchart LR accTitle: Agent 交互中确定性系统对 Token 消耗的影响 subgraph BadPattern["上下文未优化 (重试循环)"] A1["模糊请求"] --> B1["LLM 试错推理"] B1 -->|数据不匹配 / 缺字段| C1["反复重试 (多达15次)"] C1 --> D1["Token 成本激增 / 高延迟"] end subgraph GoodPattern["确定性基础设施赋能 (单次精准)"] A2["精准 BQL 查询"] --> B2["确定性 CPU 平台计算"] B2 --> C2["结构化上下文注入 LLM"] C2 --> D2["单次生成精准结果"] end ``` Jordan Tigani 进一步探讨了 Token 经济学带来的底层逻辑反转。传统数据工程中，成本与性能通常是一对权衡取舍（Trade-off）——使用更小、更便宜的计算实例往往意味着需要耗费更长的运行时间。但在 AI 任务中，削减 Token 数量不仅能直接降低费用，还能同步降低推理延迟，并由于减少了无关噪声的干扰而显著提升模型的回答准确率。为此，MotherDuck 在辅助用户构建数据管道和可视化时，将“技术文档本身视作产品的一部分”。清晰结构化的文档能够让 AI 迅速定位关键信息，避免因盲目检索大量无关文档而烧掉不必要的 Token。 - **确定性基础设施护航**：依赖 CPU 端成熟的结构化查询与领域语言，减少大模型在数据检索阶段的算力消耗。 - **Agent 上下文精简**：通过精准的首轮上下文注入杜绝多次交互重试，降低错误率与接口调用成本。 - **Token 优化颠覆传统权衡**：在 AI 场景下，降低 Token 消耗实现了“更便宜、更快速、更准确”的统一，打破了传统软硬件“降本必降速”的权衡定律。 - **文档即 AI 上下文产品**：将技术文档编写纳入系统工程，降低 AI Agent 理解系统的认知负荷。 --- ### [28:38](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1718s) 合规约束下的合成数据实践与销售赋能 关于合成数据（Synthetic Data）的应用落地，两位嘉宾展示了截然不同但各自契合业务场景的实践路径。Jordan Tigani 表示，MotherDuck 目前主要将 AI 生成的合成数据应用于销售演示环节。销售团队在与潜在客户沟通时，可以根据对方所处的行业与业务特征，实时生成一套高度拟真的数据集和可视化看板，向客户展示其数据仓库在真实运行场景下的效果，但后端核心系统并未深度依赖合成数据。 ```mermaid flowchart TD accTitle: 彭博金融场景下的合成数据生成与基准评测流程 subgraph PrivacyBoundary["客户隐私红线 (禁止直接使用)"] RawData["真实客户数据 / 邮件 / 交易记录"] end subgraph SynthesisEngine["算法脱敏与合成引擎"] Algo["统计分布拟合 / 偏差保留算法"] end subgraph BenchmarkSystem["下游无风险评估"] SynData["高保真合成数据集 (保留真实偏差 / 无隐私风险)"] Eval["基准测试集 (Benchmark Sets)"] Test["真实场景应用效果验证"] end RawData --> Algo Algo --> SynData SynData --> Eval Eval --> Test ``` 与此相对，彭博则在合成数据领域进行了极高强度的投入。Emanuel Bettelheim 解释称，金融领域受到极其严苛的合规监管，客户对数据隐私有着严密要求，彭博被严格禁止将客户的实际生产数据直接用于模型分析、训练或后训练（Post-training）。为了评估自研系统在真实金融场景下的表现，彭博必须依托底层客户数据的统计特征运行复杂算法，生成完全脱敏、不包含任何特定客户具体言论、但完整保留了真实世界偏差分布的高保真合成数据。这些数据被广泛应用于邮件处理与各类基准测试集（Benchmark Sets），成为彭博在无法直接接触真实世界数据的前提下，模拟并验证真实业务表现的核心支柱。 - **销售场景的拟真演示**：利用合成数据即时构建客户专属的数仓业务示例，提高销售转化率。 - **强监管下的合规必然**：金融机构无法直接使用客户数据进行训练或评测，合成数据是唯一的合规替代方案。 - **保留统计偏差与分布**：优秀的合成数据需要精准复刻真实世界的业务分布特征，以确保基准测试的有效性。 --- ### [30:53](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1853s) 数据工程角色的演进：从 Spark 底层调优到快速迭代 在谈到 AI 对数据工程团队架构与个人技术职责的重塑时，Bhanu Prakash Reddy Rella 结合自己长达 12 年的数据工程从业经历给出了观察。过去，数据工程师的核心精力主要集中在底层分布式计算框架的精细化调优上，例如持续进行 Apache Spark 参数调优、底层任务执行优化以及各类数据处理策略的打磨。然而，伴随着 AI 体系的深度介入与技术架构的重构，整个行业的技术迭代节奏正经历前所未有的加速，工程师的关注重心开始从单纯的底层算子调优向更高层面的智能化数据流与流水线协同演进。 - **底层性能优化重心的转移**：传统以 Spark 深度调优为核心的工程日常正在逐步转向更高维度的 AI 流水线设计。 - **技术迭代周期急剧缩短**：相较于以往十数年的渐进式演变，AI 驱动下的数据工程体系演进呈现出显著加速的态势。 ### [31:33](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1893s) 利用 AI 优化数据工程：自动化运维与 RAG 故障排查 来自 Meta 的 Bhanu Prakash Reddy Rella 探讨了如何借助 AI 进一步优化和自动化数据工程领域的工作。在数据工程中，最核心的一环是建立对数据的信任（Trust in data），因此业内通常会部署数据质量框架（Data Quality Frameworks）和验证机制，以极低的延迟来处理数据。如今，团队正积极探索如何利用 AI Agent 来扩展元数据驱动的流水线（Metadata-driven pipeline），实现只需编写一条流水线定义，即可通过 AI Agent 规模化扩展到任意操作级别。 此外，数据工程中最大的痛点之一是流水线故障与突发事件（Incidents）。每次流水线出现问题，都会耗费大量研发人员和技术支持团队的时间去排查根本原因（Root Cause）。特别是在广告技术（AdTech）等复杂业务场景中，由于用户在多设备之间频繁切换，经常会出现数据污染问题，追踪异常链路极其困难。为了缩短调查和排障时间，许多数据工程项目开始构建基于 RAG（检索增强生成）的智能运维系统，整合运行日志、操作手册（Runbooks）等文档，让智能体不仅能自动定位故障根因，还能直接生成具体的调试排错步骤。 ```mermaid flowchart LR accTitle: RAG 驱动的数据工程故障自动排查架构 subgraph DataSources["上下文与日志输入"] A["运行日志 (Logs)"] B["运维手册 (Runbooks)"] C["元数据与流水线状态"] end subgraph AgentEngine["RAG 智能体引擎"] D["RAG 检索与分析"] E["AI 诊断智能体"] end subgraph Output["故障治理输出"] F["根本原因定位 (Root Cause)"] G["生成修复与调试步骤"] end A --> D B --> D C --> D D --> E E --> F E --> G ``` --- ### [33:07](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1987s) 组织架构与工作模式的深远重塑 彭博社（Bloomberg）的数据与分析主管 Emanuel Bettelheim 强调，尽管外界对 AI 技术的讨论非常热烈，但他认为**人员、流程和运营模式（Operating Model）所经历的变革，远比技术本身的变化更加剧烈**。AI 是一种通用技术（General-purpose technology），正在深度渗透到彭博社的每一个系统和产品之中。Emanuel 认为数据管理和数据工程正在沿着软件工程的发展路径演进——最终将全面走向声明式（Declarative），工程师只需设定目标，具体的执行则由系统在一定距离外自动运行。 这一范式转移给实际工作流程带来了巨大冲击： * **瓶颈的转移**：如同前线软件工程师现在可能同时驱动多个 Agent、不再亲自写代码，开发瓶颈迅速从“编写代码”转移到“审查代码输出”；而当代码生成量过大导致连审查都无法人工完成时，工作焦点进一步转变为“制定自动化测试目标并验证正确性”。 * **信任与合规挑战**：数据管理同样面临这一转变。在彭博社这样拥有 27,000 名员工且处于强监管行业（Highly Regulated Industry）的企业中，如何在全面放开 AI 工具的同时保证数据输出一如既往的可靠、符合合规要求，并向客户交付最高品质的产品，是一个极具挑战性的难题。 * **人的协同重于技术调整**：计算机和底层 LLM 只要给出指令最终都能被约束执行，但协调庞大组织内成千上万员工的协作方式，则需要自上而下进行深层次的重新思考。 --- ### [35:08](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=2108s) 全民数据交互：MCP 架构与数据团队角色的蜕变 MotherDuck 首席执行官 Jordan Tigani 开玩笑地自嘲道，就在去年 11 月，他还在类似的小组讨论中公开宣称“AI 不会对数据领域产生太大影响”，结果事实证明他“错得相当离谱”，现实迅速推翻了他的预测。MotherDuck 在内部部署了 MCP（Model Context Protocol）服务器，使全体员工能够直接通过 Claude 或 ChatGPT 等模型无缝查询公司内部数据。 这一改变彻底打破了数据使用的边界： * **业务人员的自主探索**：一名身穿 MotherDuck 工装裤的销售人员成为最早的重度用户之一，他每天会直接向 AI 提问：“我今天应该和哪个客户沟通？”这种问题并非传统数据分析师会提出的标准指标，但它深度结合了客户支持工单、使用量变化趋势甚至是最新行业新闻等综合信息。 * **数据团队从“做看板”转向“语境与评估治理”**：数据不再是数据团队的专属领地，全员都能从中获取价值。但这并不意味着数据团队不再重要，反而让他们走向了更前沿的核心位置。数据团队的工作重心从疲于应付“帮我做个仪表盘（Dashboard）”等低效取数需求，转变为**上下文治理（Context Curation）**与**评估体系构建（Evals）**——确保当任何人询问“第三季度营收是多少”时，底层计算逻辑严密无误，保证每次输出结果的确定性与一致性。 --- ### [38:21](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=2301s) 展望 2027：架构瓶颈、访问控制与算力限制 主持人 Emil Protalinski 请嘉宾们对 2027 年可能出现的重大架构瓶颈做出预测，并调侃鉴于 Jordan 之前的“打脸记录”，先请 Jordan 发表预测。 针对未来的核心技术瓶颈，嘉宾们给出了不同视角的判断： * **数据智能体过载与系统稳定性**：Jordan Tigani 预测，正如现在软件工程师习惯同时运行 37 个编码 Agent 一样，到明年人们很可能会并发运行几十个用于数据清洗和处理的 Data Agent，这些智能体的大规模并发将以全新且意想不到的方式击垮现有的底层系统。 * **智能体权限与访问控制（Access Control）**：Jordan 补充指出，一个被忽视的关键瓶颈是权限管理。当自主 Agent 在企业系统中无处不在时，系统必须明确“这个 Agent 到底代表谁的身份与权限”，这在大型企业环境中将变得极其棘手。 * **算力开销与模型能效限制**：Bhanu Prakash Reddy Rella 指出，各组织此前都在激进投资，但目前已经进入紧缩阶段，开始对 Token 使用进行严格限制。随着能耗与算力成本上升，企业无法再无节制地运行大模型，未来架构必须在极其严苛的算力约束下，通过更高能效、高度优化的方式来运行和评估模型。 在热烈的讨论中，主持人宣布圆桌讨论结束，并邀请现场观众在会后与嘉宾进一步交流。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=4s) 打造工具来帮大家使用数据 — Jordan Tigani > [11:35](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=695s) 智能体投机（Agentic Speculation） — Jordan Tigani > [16:09](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=969s) 不要让完美成为优秀的敌人 — Emanuel Bettelheim > [21:23](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1283s) 为了工具做优化，而非为了解决问题做优化 — Jordan Tigani > [21:23](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1283s) fall in love with the problem not with the solution — Emanuel Bettelheim > [33:07](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=1987s) 人员、流程和运营模式（Operating Model）所经历的变革，远比技术本身的变化更加剧烈 — Emanuel Bettelheim > [35:08](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=2108s) AI 不会对数据领域产生太大影响 — Jordan Tigani > [35:08](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=2108s) 我今天应该和哪个客户沟通？ — Jordan Tigani > [38:21](https://www.youtube.com/watch?v=-JyOqc6pmBA&t=2301s) 这个 Agent 到底代表谁的身份与权限 — Jordan Tigani ## Mentioned Emil Protalinski, Bhanu Prakash Reddy Rella, Emanuel Bettelheim, Jordan Tigani, Meta, Bloomberg, MotherDuck, Techmeme, EPro Strategies, Google, Google BigQuery, SingleStore, Claude, ChatGPT, dbt, Kafka, Spark Streaming, Apache Spark, Bloomberg Query Language (BQL), Bloomberg Terminal, Model Context Protocol (MCP), Slack, Dataflow ## Video description Emil Protalinski - Editor and Consultant, Techmeme and EPro Strategies Bhanu Prakash Reddy Rella - Senior Data Engineer, Meta Emanuel Bettelheim - Head of Data & Analytics, Bloomberg Jordan Tigani - CEO, MotherDuck Data engineering is under pressure to support increasingly complex AI workflows—and fast. This panel dives into how leading teams are building robust, scalable infrastructure to power real-time AI applications. Experts will share hard-won lessons on pipeline design, data freshness, cost optimization, and tooling selection. Expect candid insights on where legacy architectures are breaking down, how vector and streaming solutions are evolving, and what’s actually working at scale. [Source](https://www.youtube.com/watch?v=-JyOqc6pmBA)

[0:04] Jordan Tigani: 打造工具来帮大家使用数据 [11:35] Jordan Tigani: 智能体投机（Agentic Speculation） [16:09] Emanuel Bettelheim: 不要让完美成为优秀的敌人 [21:23] Jordan Tigani: 为了工具做优化，而非为了解决问题做优化 [21:23] Emanuel Bettelheim: fall in love with the problem not with the solution [33:07] Emanuel Bettelheim: 人员、流程和运营模式（Operating Model）所经历的变革，远比技术本身的变化更加剧烈 [35:08] Jordan Tigani: AI 不会对数据领域产生太大影响 [35:08] Jordan Tigani: 我今天应该和哪个客户沟通？ [38:21] Jordan Tigani: 这个 Agent 到底代表谁的身份与权限

[0:04] Emil Protalinski: data stacks and pipelines are adapting to change faster than ever. [0:04] Bhanu Prakash Reddy Rella: building energy-efficient AI data pipelines to power machine learning inference ecosystems, specifically working on cheaper, greener inference models. [0:04] Emanuel Bettelheim: rethink finance and AI—both for internal operations and for external market customers. [18:12] engineering decisions must be rooted in solving real customer problems rather than falling in love with shiny new tech. [18:12] Unless an organization is engaged in microsecond-level activities like high-frequency trading, incremental micro-batch processing is far easier to reason about and provides sufficient freshness for the vast majority of enterprise use cases. [21:03] optimizing compute and tooling to meet specific business outcomes is the truest form of cost optimization. [27:06] Cutting unnecessary tokens makes requests faster, more accurate, and cheaper. [30:53] Engineers are transitioning from manually writing code to orchestrating fleets of autonomous agents and establishing test objectives. [34:49] Jordan Tigani: spectacularly wrong [34:49] Jordan Tigani: What customer should I talk to today? [38:21] Jordan Tigani: knock over systems in new and exciting ways.

[0:04] Right. Jere who's excited to talk  about data stacks and data pipelines. [0:20] the, you know, the challenge.  They're always adapting to change, [0:24] but I feel like they got a way that's made  things even faster. Um, so today we're going [0:29] to hear from some three experts to my left,  what they are doing, what they are not doing, [0:36] and what they would. But let's start there.  Why don't you introduce yourselves one by one [0:40] and kind of give a little bit of a taste  of the audience of why you're up here, [0:44] why you're experts in the space. Yes. My  name is so I have over 12 years completely [0:54] recently I've been working on building energy  efficient AI data pipelines to power the ML [1:00] ML inference kind of ecosystem as well as one  of the top priority in my list is working on [1:08] uh like inference models which usually comes  up with a more cheaper and more green ways. Hi, [1:15] my name is Emanuel. I'm a data analytics at  Bloomberg. Um, my background is actually mostly [1:21] in the tech industry and kind of technical  product. I've been doing this for 25 plus [1:27] years now. I've been startups and all the way  off to large companies. Most recently was at [1:33] Google before I jumped to work. Um our biggest  priority I would say is rethinking entirely what [1:40] uh finance and AI looks like in the future and  how to apply AI both uh internally for us to [1:47] kind of power the capabilities of Bloomberg  uh as well as how to uh give AI capabilities [1:54] and the applicability to our customers um in the  the markets. I'm Jeremi uh co-founder and CEO of [2:03] Malak. I've been working on data analytics systems  for for 15 years. I helped start Google BigQuery [2:10] uh that engineering product at BigQuery and  was chief product officer single store before [2:16] starting a few years ago. So because these  guys are on sort of the you know using data [2:22] side and I'm on the sort of like help building  tools to help people help people use data side. [2:27] uh kind of the biggest the biggest thing for  us right now is is we're trying to react to how [2:33] the world's changing way is uh with AI and I think  you know the old days you can sort of like see the [2:40] direction that you know the train was going and  you can get on that train and now uh AI just sort [2:46] of as cloud has sort of moved into it's very hard  to know what the next next steps are going to be I [2:51] think one of the key things for us as a startup is  really sort of like figuring out where you Where [2:57] is the world going? How do we get there before  you know before something else for realizations? [3:04] Uh let's start with you Gary. Can you give us an  example of specific push product you recently your [3:10] active redesign and feel free to hear? Sure. So  um you know one of the things we sort of realized [3:19] uh in uh was that some of the areas of the modern  data stack are fairly by a portable uh and so we [3:27] really recently added this feature called client  which is um by coded data pipelines and so [3:34] uh and we realized hey using cloud uh and you  point cloud Salesforce uh is going to figure out [3:40] how to get that get that data into your into your  so we thought what if we just sort of factor that [3:46] uh and made it easy for people to sort of use my  encoded pipelines. We had uh we we bothered. So [3:52] what it turned out to be is really it's just a a  Python runtime uh that uh that runs you know runs [3:59] individually a trigger or or on a schedule and  it turns out to be incredibly powerful because [4:04] like if you give that to call or chat GPT or  whatever it can turn that into you know it can [4:11] pull in your your DBT pipeline. It can it can  do the transformations. It can use open source [4:17] uh open source connectors do the pull data from  wherever and um and so you have this sort of this [4:24] integrated runner. It integrates with the context  layer that we that we have. It integrates with you [4:29] know the visualization layer. It also relates  [clears throat] to the uh wind warehouse. Very [4:34] cool. Um so Bloomberg for folks who don't know is  is kind of the central nervous system of of the [4:40] financial industry and um one of our key products  that's been around for 45 years is called the [4:47] terminal. Um if you ever watch movies think of  the trailer sitting in front you know the big [4:53] screen that often will be the terminal. Um over  the years the terminal added more and more data [5:00] and more capabilities. Um and those were targeted  workflows for the specific workflows that happen [5:07] within the financial industry. Um and as you can  imagine there's a tremendous amount of workflows [5:14] capabilities that they can be built. There's  literally thousands of what we call functional [5:17] applications essentially that you can do that  very specific analytics provide specific data. [5:23] uh with AI the biggest change [clears throat] for  us now the biggest move that we've got publicly [5:28] um is introduce a conversational AI into um so  the ability to literally have a conversation [5:34] with all that data have a conversation with both  the unstructured data company filings transcripts [5:40] from the earnings calls for instance screening  uh any kind of mac economic data about anything [5:45] in the world prices straight up any of that is is  all available uh as well as structured data Um and [5:53] uh we launched that in January and after we  built that, one of the biggest things we had [5:57] to change was going from very kind of siloed uh  purpose-built specific uh end toend uh data t [6:06] stacks uh for each of the workflows into a unified  system. How do you get within a few seconds the [6:13] right answers um from you know hundreds of  different databases that were optimized over [6:20] many many many years um to answer only questions  within their domain. Um and so we essentially had [6:27] to completely rethink every layer of the stack. Um  the most interesting one probably to talk about is [6:33] something um we have on the top we have we  have something called the BQL which stands [6:37] for Bloomberg wear language. It's a unified data  access and analytics layer that essentially goes [6:42] to the heterogeneous federated uh data systems  and pulls this all together and then layers on [6:48] top of it its own query language. Um it's great.  It's been actually used by our customers. We use [6:53] it internally a lot. Uh except now we wanted it  exposed within ASB because we want to provide the [6:58] transparency of how we actually pulled the data,  the grounded data for the answers. And so we had [7:04] to for instance work through how do you take  natural language and teach an LLM to speak BQL [7:10] uh when BQL isn't something that is kind of  generally known to the world. It is more or [7:15] less an internal thing. Um and there's there's  really a lot of learnings that come out of that [7:19] and there's a tremendous amount of investment that  went into that. So I would like to uh pick one of [7:26] the interesting case like which I have come  across one of the most interesting challenge [7:30] uh where we have tackled. So uh this is around the  uh like inference kind of architecture where like [7:37] uh it supports the recommendation platform.  So often times what we initially did is as a [7:42] part of PC so we reached out to a point where  all the companies reach where like our traffic [7:48] is growing but the inference costs and are  like infrastructure costs are going growing [7:52] very rapidly and way too fast than your traffic.  So eventually uh what happened is whenever you [7:58] are uh you know insert like feeding the inference  kind of pipelines what happens is each and every [8:04] user request or when you're for working for  recommendation platform all the user request [8:09] triggers each and every user request triggers this  dense neural network again and again repeatatively [8:14] that's you know either it's really useful for  uh the inference or not. whether it's necessary [8:19] or not. So the obvious answer here is like you  know in big tech companies they will just add [8:25] more GPUs. That's the obvious answer. But what we  did is we thought in a different way what exactly [8:30] why do we need the like you know uh to run the  dense neural networks for each and every request [8:36] or the user request. That's where we have did  did some kind of profiling where we have come [8:41] up with some analysis where we have seen if you  look at the users so a lot of these requests can [8:47] be categorized into two different ways one  is like majority of the 90% of the requests [8:51] are kind of very easily predictable and you know  very easily like very simple straightforward but [8:57] 10% of the requests are really hard which needs  a full LM tuning now the question is why do 90% [9:03] of the requests which are easily predictable still  uses the same dense neural network model which is [9:08] very compute intensive. So that's where we have  come up with we that's where we thought like we [9:13] have to redesign this whole pipeline end to end  pipeline. So what we did is on a high level we [9:18] have uh implemented sparse sparity kind of  uh uh like method where we implemented the [9:24] sparse neural network. This actually activates  whatever is needed. So the way it works is you [9:29] have hundreds of requests. So the thing is you  need to know what requests are simple and what [9:33] requests are complex. So in order to determine  that at the data engineering perspective the you [9:38] know the pipelines feed from the Kafka and spark  streaming where uh this adds some additional kind [9:45] of uh you know pointers towards the metadata  which usually tells the sparse sparse method [9:51] like you know whether it's easy or hard and then  this parts will direct this uh request towards [9:56] the lighter model and the complex model. that way  you know deviating and dividing the requests helps [10:02] helped us in you know reducing more compute. So  that is one of the main eventually we could reduce [10:09] up to 40% of the compute down there. Perfect. Um  would you be able to actually all three of you [10:16] but starting with you pinpoint where in legacy  data infrastructure the the breaking point is [10:24] happening the fastest when it comes to modern AI  workloads. So in my opinion it's all about like [10:29] the traditional legacy systems were built for kind  of standard mechanisms or standard data types but [10:34] when you start implementing or incorporating this  data types from you know across multiple sources [10:39] like you know multiple systems that's where it  starts breaking because it's unable to scale. So [10:45] uh when you are trying to switch from legacy  towards AI adoption the main thing that you are [10:49] going to do is your compatibility of your current  uh architecture whether it really scales or not [10:54] whether it's compatible with the AI uh you know  implementation or incorporation that's where the [10:59] lot of investment goes you need to data author to  see how the things are going and you know whether [11:04] the you need to uh like rescale all the pipelines  where it has to you know accommodate all different [11:12] types of data types. And I could say that that  where you're seeing it too, Emanuel. Yeah, [11:16] I think certainly the the scale I think is is is  a is a challenge and I think one of the things as [11:22] I mentioned earlier is like we were very optimized  for the workflows that we had seen for you know 45 [11:27] years of users and we've adjusted over the years  obviously and built out the infrastructure. Um, [11:32] but one of the things that's interesting  is that agents, especially if you give [11:35] them if you use them really extensively in your  infrastructure, um, they don't behave like people, [11:40] right? They don't they're they explore in ways  that people actually wouldn't explore. Um, there [11:46] was a paper from Berkeley a couple years that came  out that that phrased it like agentic speculation, [11:51] which I really kind of like because it's they're  often very speculative in their exploration. [11:55] um which allows them to synthesize a lot of data  and like get a lot of things together that would [12:01] have been really hard otherwise um but as an end  result on your infrastructure it means they go [12:05] to places that you've never optimized to go for  um and so certainly I think one of the breaking [12:11] points that that are very clearly visible is the  places that were sort of in the long tail that you [12:17] mostly I don't want to say didn't care about you  cared about it but it wasn't that critical because [12:21] it was in a long tail every now and then somebody  would hit it you'd optimize enough that it could [12:26] serve that some agent might suddenly think that  this is the most important thing and hit it like [12:30] crazy. Um and and figuring out how to suddenly  like what even the curve looks like the whole [12:36] concept of like hot and cold data for instance  that the idea of a long tail doesn't quite apply [12:41] anymore as much in a in an agentic world as it did  before and that obviously changes your the way you [12:48] design your data systems. I I think these that's  exactly I think what we're seeing is that you know [12:53] agents are are using data differently. Agents are  hitting hitting the system differently and you [12:59] have to build differently in order to be able to  handle that. And you know one of the things that [13:03] you know that we're doing for uh for to solve that  particular problem is you know in um uh you know [13:11] humans can hit a data database you know only only  so hard. um agents can can basically hammer it and [13:18] I think you know we were seeing you know systems  get systems get knocked over when you have a bunch [13:22] of agents trying to basically party on the same  on the same thing. Uh and so the architecture [13:27] that we built you know where a a data warehouse  but was basically that each user and each agent [13:33] gets its own instance in the cloud but basically  its own its own compute instance. So, you know, [13:38] they can scale up, they can scale down, they can  they can they can shut down. But if you've got [13:42] hundreds, thousands of agents, uh, you know, they  each, you know, they're not going to interfere [13:47] with each other, not they're not going to knock  any knock anything over. Uh, you know, we built [13:51] it for a different purpose. We built it to enable,  you know, SAS apps to have lots of different users [13:55] kind of all independently hitting the system. But  it turned out to be super convenient for you know [14:00] when all of a sudden agents are are are partying  on the data they have they have sort of similar [14:05] similar behavior to sort of to to these sort of  broad broader SAS apps. Bonu or Emanuel do you [14:11] guys have an example of a story where something  was breaking and you had to rebuild your data [14:15] pipelines as a result? Um as I said like you know  it's all about the data you know data types and [14:21] what you know having different types of data types  in the picture. right now we are going to source [14:25] like different events you're getting streaming  unstructured data and like different formats of [14:30] data right now. So that that was breaking all our  pipelines. So technically we have to you know try [14:36] to see what exactly is breaking and um instead of  uh like you know fine-tuning the pipelines or you [14:42] know working or as an enhancements we have to come  up with brand new kind of uh architecture in order [14:47] to tackle that. What about you Emanuel? Yeah, I  mean I think you know Bloomberg is the critical [14:54] sort of infrastructure for the financial industry.  So we've spent many many decades to actually [14:58] harden the infrastructure in a way that it doesn't  just break. Uh the breakages are invisible in a [15:03] way um to the public or to our customers um at  least hopefully. Um that being said certainly [15:10] behind the scenes I think there's a lot of areas  where we're seeing the pressure build up and we're [15:15] we're seeing for instance spikiness in areas there  was never spikiness before. Um and and because [15:21] again the agents kind of amplify the behavior  as well and and kind of go to places where there [15:25] maybe wasn't as much traffic before. And so those  are for us the places where we're looking okay [15:30] what do we need to rearchitect? How do we need to  rethink literally all the way from the bottom up? [15:35] um how do we even rethink the the data flow  into the system in the first place which [15:39] um obviously is fairly it's kind of a mix of an  automated deterministic systems but they were very [15:45] sensitive to changes all the time and so there was  a lot of human labor that was actually put into um [15:50] fixing that all the time and and keeping in touch  with that um and making sure that the quality [15:55] stays where we needed to be um and now you know we  can completely re reevaluate that and think about [16:00] like hey what does this look like in an in an AI  world how do you build that for an AI all the way [16:05] again to the front where we then present it to  the customer. What does the terminal look like in [16:09] the age of AI? Some of the people in the audience  might be just building out their first, you know, [16:14] first project for real-time AI application or  another AI application. What is a what is the most [16:21] common architectural mistake that you want to warn  them about so they can skip that whole ordeal? Uh [16:28] common architectural mistake? Um uh that's good  good question. Um uh I can you come back to me? [16:41] Sure. I mean if none of you have a maybe it's  not there are no common ones. Definitely. So [16:46] what in my opinion what I think is a lot of these  organizations whoever starts building the first [16:51] projects is like you know often they think every  team has everything as a real time like you know [16:56] real time they have to go by default real time but  which it rarely you know really useful to go real [17:02] time. So um before investing lot into the GPU or  lot into the compute or you know investing lot of [17:08] money into it. So what we need to think really  think is like whether all the requirements or [17:12] all the requests we are having can be segregated  into you know what is what can bear like you know [17:17] ultra low latency or you know what is what has to  be very instant and then even we have to divide [17:22] from for a second how many seconds it it can last  longer or can be a batch or something. this kind [17:27] of analysis and you know prioritizing only on  the you know focusing only on the ones like [17:33] uh uh you know that are really need the real-time  processing is something which we need to do that [17:38] uh because this often like reduce a lot of compute  cost so uh that that's really one of the important [17:43] thing I could call about when it comes to the even  real-time processing where you are looking for [17:48] uh you know there like you don't really need  to invest a lot of compute or GPUs there you [17:52] can always come up with the engineering kind of  optimization techniques so as I said like you [17:56] know we introduces parsing which usually helps us  to try you know compete I mean use two different [18:02] models for lighters and heavier ones as well as  you can come up with the feature enhancements [18:07] feature stores or there a lot of things you can do  in the data engineering optimization side rather [18:12] than you know directly getting into the comput  side yeah so basically plan first before you dive [18:18] in and start spending the budget manual Jordan  do you want to add to either I think I mean I [18:23] would almost say my advice would be the same even  before AI right which is it very much depends on [18:28] the scale of the operation that you have like the  company if you're a small startup I think that one [18:33] of the famous advice was like don't try to build  out before you actually have product market fit [18:38] right and and and don't let perfect be the enemy  of the good often you would see startups build [18:43] essentially the Google like environment and fall  into love with the technology rather than actually [18:47] solving the customer problem I think if you're  a larger company I would go the the other way [18:51] actually and it's like assuming that a lot of the  things that are being talked about in the industry [18:56] for like well a chatbot. How hard can that  be? Right? Just like give it some documents, [19:00] make some embeddings, do some rack, boom, you  have a chatbot, right? That works when you have [19:05] a few thousand documents. It doesn't work when  you have 500 million documents. Um, and so like [19:10] really thinking about like what stage of maturity  are you in for your company and what are your [19:14] needs for AI and what does it mean to actually  fulfill your customer needs to me is you know [19:20] I'm a product manager by heart and so in the end  it always goes back to that like actually try try [19:25] to solve customer problems don't just fall in love  with the technology. Yeah. So, I I I do think that [19:30] um there's there's something that I've changed  my mind about a couple times during my during [19:35] my career, and that's that's really about um uh  you know, in in the realms of of of real time is [19:41] uh is is streaming. I think one of the things  that's good to ask yourself is like what does real [19:46] time actually mean to you? And when I was working  on BigQuery, we said, "Oh, we need real-time data. [19:50] We need we need this immediately." And like what  do you mean? And they they say, "Well, it's got [19:54] to be within like five or 10 minutes." And for  us, that wasn't real time at all. like that's [19:58] you know you could run a pretty coarse grain batch  batch operation and so I think a lot of people you [20:04] know assume they need streaming assume that they  need uh you know these sort of elaborate streaming [20:09] systems and I I love streaming stuff I I like you  know it's like it's a great puzzle and to figure [20:16] out okay how does how does this stuff work stuff  work and like data flow systems um but it adds a [20:22] lot of complexity it's harder to reason about and  I think for the most most of the time like you So [20:28] unless you're doing like high frequency trading  where you kind of you do need like microscond [20:32] resolution like um you know microbatch is as  long as you can do it incrementally like you know [20:37] obviously if you're blowing everything away every  time like that's probably the easy anti-attern but [20:44] like um you know microbatch can get you very  very far and then you know you may only need [20:51] streaming for the some of the the more esoteric  use cases. Yes. as you're as you're figuring out [20:57] which use case deserves what. I mean, there's  so many new tools out there. Kind of curious, [21:03] do you guys have a framework or what questions do  you ask yourselves when you're determining whether [21:07] a new tool is genuinely useful and going  to make a difference or if it's just hype? [21:13] I assume that it's assume that it's just hype, but  like you know I think that the tricky thing with [21:18] with AI is like there's just so many new things  coming out and some of them many of them like have [21:23] these massive step-wise you know improvements and  so you don't want to you don't want to lock that [21:28] down and tell the team like no no no you're not  allowed to try anything new. Uh on the other hand [21:32] there are a lot of like security and governance  you know and controls uh you know problems with [21:38] that and so it is it is tricky to balance the you  know the the the riskreward um you know as a small [21:45] startup we can be a little bit more cavalier about  letting people you know letting people try try new [21:51] things that but every once in a while there's like  so and so wants to add this you know Slackbot and [21:55] it's like what is that actually doing and and  uh so you have to you know be a little bit be [22:00] a little careful. Um but uh you know there are  just so many so many things out there that can [22:06] that can that can help. Uh and it it also can be  a huge time sync because you know for every new [22:14] um I don't want to you know criticize AIBDR  but for every new like you know type type of [22:19] thing like that there's a bunch of different  companies and if you're like I we need to find [22:22] the best one you can spend all your time you know  just trying to find the best of these these new [22:27] AI tools. Yeah. You're optimizing for the tool  rather than the problem. Yeah, technically in [22:32] my opinion what I feel like is uh you know we  don't really have any kind of optimization or [22:38] cost optimization for kind of like magic tooling  kind of thing. So uh what I would highlight is if [22:44] the level of compute meets your business needs  uh it's always like you know it's the best kind [22:49] of optimization that ever possible you can make  any of the project as you have any any tools for [22:56] or or framework for deciding what's what's hyping  I think I mean I again my my nature as a product [23:02] manager comes out in some ways and I'm I'm almost  always going back to just like actually go back [23:06] to the customer backwards from what it is you're  actually trying to solve fall in love with the [23:11] problem not with the solution right and that's  I think that's fundamentally that has been true [23:16] you know for a really really long time in in  software technology now um it still holds true [23:20] in AI and like I think AI is extremely powerful  it's a transformative technology unquestionably [23:25] but don't just stuff AI into everything you need  to you want to actually figure out what you're [23:30] trying to solve and then work backwards from there  and then hopefully scale it right hopefully yes so [23:36] speaking of scale scale scaling costs money right  curious what tools tools, strategies, optimization [23:43] techniques you guys have seen that are actually  effective uh for scaling without impacting cost [23:49] but also while keeping model performance. Yeah,  definitely I would um I would go a little deeper [23:54] into the data engineering tech stack here. So  when we are trying to implementing this kind of [23:59] pipeline so what we really can leverage is kind  of feature store. So we have long moved away from [24:05] feature store as a storage. we have come up to  a level of level to think that as a consistent [24:10] layer between the training as well as inference  kind of systems. So that's how this feature store [24:16] uh you know evolved as as important in this world.  So within this feature store so we need to come up [24:22] with some kind of freshness kind of strategy  because uh you know feature store is all all [24:26] about like you have an easer all the fe user and  all the features corresponding to user stores [24:30] within the feature store but whether and you can  divide the feature stores into based on freshness. [24:36] a a user tag to a country. So it's often doesn't  change it like birth, country or nationality. [24:40] Why do you always need to run the model on these  features which are consistent across like you know [24:45] several months. So that's where you have to come  up divide the feature store into like you know [24:49] the freshness kind of strategy and then you have  to work on like whatever the feature stores need [24:54] like more frequent updates and then see how it  goes and we can even evolve into the future vers [25:00] versioning right now. So which is really uh useful  while building any kind of inference pipelines. [25:07] I think one of the things like there's a lot of  conversation about token economics and and kind of [25:12] like oh how do I reduce my token cost and I think  there's a lot of focus on oh which model do I pick [25:17] and I should pick a smaller model larger model  and you know open weights close weights routers [25:21] and harnesses. Um I I think in my view certainly  these things do matter to a certain degree. Um, [25:27] I don't want to say that they don't, but I  I actually think what's often forgotten is [25:31] that there's still the deterministic side of the  world and actually the the surrounding ecosystem [25:36] of the LLM you're running, the model that you're  running, and you can actually probably drive a lot [25:41] more cost savings or at least get efficiencies  if you actually invest in the surroundings, [25:46] which are also probably more specific to your  infrastructure than the model which will kind [25:50] of upgrade, you know, every 3 weeks it seems like  right now. Um and so in our case for instance as [25:56] I mentioned PQL earlier right like the BQL is  a deterministic infrastructure piece it does [26:01] uh it's a it's a domain specific language you can  say hey give me the GDP of a country and compare [26:07] it to this country and you know things like that  uh and that executes almost all of that executes [26:12] in our regular CPU kind of deterministic platform  uh and is tied to the model though and so one of [26:21] my things would be like all don't just look at the  model and try to optimize your model performance. [26:26] Look at the system surrounding it, the harness,  governance models, etc. and how much efficiency [26:31] you can drive there. U it will often make the  model performance itself also better because [26:37] especially in aentic systems for instance, if  I uh optimize the context on the first request [26:43] just right, I don't have to do retries. I don't  have to have it sync as much. I simply get the [26:48] answer and it synthesizes that. if it needs to  do 15 retries to get to the data, of course, [26:53] it's going to cost you more tokens. Um, so it's  really the don't forget the determin deterministic [26:59] system would be like my biggest advice probably.  I I think you know uh specifically in terms of [27:06] token economics there's something that I'm finding  particularly interesting which is in the old days [27:11] uh efficiency and performance were were sort of  trade-offs like or like cost efficiency is I could [27:18] you know I can run a smaller instance uh for less  expensive but it's going to take me it's going to [27:24] take me longer and it's all really about where  your your threshold is. I think with uh with a [27:29] lot of AI tasks that you're doing, if you can  cut down on the number of tokens you need, um [27:34] not only is it less expensive, but probably it's  going to be more performant and it's going to be [27:38] more right. And so a lot of what we do is we focus  on you know in building these sort of AI enabled [27:44] systems kind of around the around the warehouse  whether it's building building visualizations [27:49] building building pipelines is like how can  we make it token efficient for our customers [27:54] to build these things because that's going to be  actually more likely what they what they really [27:59] want and some of the things to do to do around  that is like you know your your documentation [28:04] becomes part of the product because the as  part of what's happening is the AI is reading [28:09] your documentation. And so if it's clear for a  human to read, it's probably also clear for an AI [28:13] to read. But there's, you know, obviously there's  ways of of of optimizing that or being able to [28:18] sort of have let the AI sort of jump to what does  it really need to know about about the system that [28:23] it's solving. So it doesn't have to go and you  know, search everything that it can possibly uh [28:28] it can possibly look at. And as you get better and  better at that, you find you also get better and [28:32] better better and better performance, better, you  know, better better outcomes. Have either of you [28:38] experimented with synthetic data and how's that  impacted if if at all impacted you know quality? [28:46] We're not we're not really doing much on  the synthetic synthetic data other than [28:50] actually one of the I mean cool things that  we can do is we we generate synthetic data [28:55] uh on sales calls and we basically you know we'll  talk to a customer and say hey here's a here's a [29:00] data set looks just like what we from what we know  about your business uh and we build visualizations [29:06] and you know we basically can kind of show them a  running a running warehouse uh environment um by [29:12] by using you know AI generated synthetic synthetic  data But we haven't really incorporated much of [29:18] that into our own our own systems and I would  assume Bloomberg's not doing anything since [29:22] actually we we've invested a decent amount in  it. Um and the reason being is we're in a highly [29:27] regulated industry and actually our customers  essentially they expect privacy for their data. So [29:34] uh we can't use any of our customers data more  or less for almost any analysis for any kind of [29:41] training postraining for anything. And so we're  essentially forced to to create synthetic data. [29:48] Often that synthetic data is driven off of the  original data that comes from the customer. So [29:53] you take the customer data, you run a lot of  algorithms on it. You come out with synthetic [29:57] data that represents with the correct biases  etc. exactly what the customer data was like but [30:02] without being tied to the customer without being  exactly what people said etc. And then working off [30:07] of that data. Um, a lot of that falls into, for  instance, our email sets and our benchmark sets, [30:12] right? Where you can benchmark against the real  customers, but you still want to know like, well, [30:16] how does this thing that I built work in the  real world, but I don't have a real world to [30:22] work against? Because everything that's in the  real world is basically untouchable. And so [30:26] the synthetic data really helps us represent  uh a reflection of the real world that isn't [30:32] actually the customer's data but a version of the  customer's data that is completely synthetically [30:37] generated. Have you done anything? No. Okay.  I was just curious but it seems like anything [30:44] that is being done is really just present things  to the customer and not actually for training or [30:49] fine-tuning or anything. So that's interesting.  Um can we talk a little bit more just going back [30:53] to data engineering in general? How have you  seen whether it's the last few months or last [30:59] few years, how has AI's role impacted or how  has AI impacted the roles and responsibilities [31:04] on your various teams like data infrastructure and  architecture? How are individual roles changing? [31:12] How are teams being reconfigured? I'd love to hear  a little bit about that. So, uh in my project, [31:17] so majority of the time, so I was at like data  since 12 years. So it was it kept evolving but [31:23] this is a faster pace where I know previously I  used to work like lot on the spark optimizations [31:28] or you know finetunings or how we can leverage  the data processing kind of strategies but right [31:33] now it's all about how you can you know better  optimize the sol like automation or automate the [31:38] things in the data engineering space like  something's really important in the data [31:42] engineering is all about like you know the trust  in the data. So that's where you usually have the [31:46] data quality frameworks or you have validation  mechanisms or some kind of you know like systems [31:52] that build around the data engineering space where  which helps to process the data like you know with [31:58] with very less latency but right now using AI  so we have been you know looking into automate [32:03] a lot of these things. So how you can scale  a metadata driven pipeline like you know just [32:07] write one pipeline and how you can scale it to any  any level of operations using an AI agent right [32:13] and also like one of the pain points in the data  engineering is all about the incidents right like [32:17] whenever there is a failure in any of the pipeline  technically it eats up a lot of developers or you [32:23] know support guys time like you know in order to  figure out the root cause or and in in spaces like [32:28] atex or attex systems where a lot of time this  contamination happens and it's really hard to [32:34] trace out you know what's going to happen within  the user switching to multiple devices. So uh so [32:39] eventually like you know in order to um figure  out and and trim down the investigation time. So [32:45] technically even all the data engineering projects  are trying to come up with the rag the rag kind of [32:50] you know systems where they are running logs and  you know run books and everything and they are [32:54] creating some agent which usually tells us show  throws you the root cause and it helps you with [33:00] the debugging steps as well. So that's how  it's getting evolved around. I think for us, [33:07] you know, there's a lot of conversation about the  technology. I actually think that the the people, [33:11] the processes, the operating model is probably  what changes even more than the technology. Um [33:16] particularly because AI is such a general purpose  technology in a way and it really touches for us [33:22] every aspect of um the systems that we have and  and and the products that we have. Um I I I am [33:29] one of the people who generally believes that um  data management broadly uh data engineering will [33:35] follow software engineering's path uh which means  eventually we'll all be completely declarative [33:40] right you just give it an objective and it will  be run at arms length. Um but if you start with [33:45] that premise that you're like hey you think that's  going to go somewhere then you have to think about [33:50] exactly like software engineering thought about  like you talk to a software engineer at the front [33:53] line right now they might run many many agents  right um they're hurting those agents they're [33:58] never writing code anymore uh and the bottleneck  for instance move to reviewing the code that comes [34:03] out right and then it's like well we don't even  can review the code anymore we actually need just [34:07] objectives that you test against how do you make  sure that that's right the same thing shifts into [34:11] into data management and so it's a fairly dramatic  shift I think for all teams on like what does it [34:17] mean even if you give them all the AI tools and  the freedom how are you supposed to operate in [34:23] that world what are the what are the models how do  you still stay as trustworthy as they always were [34:29] um how do you trust in the outputs yourself um in  again in our case we're highly regulated industry [34:34] how do you make sure you stay within compliant  and you make sure that your customers still get [34:38] the highest quality products that you that they're  used to um so this it's a it's a fairly dramatic [34:43] change and and um it's a harder change I think in  some ways than the technology because the computer [34:49] you can just instruct it to do what to do and LLMs  are sometimes tricky too but in the end you get [34:54] forced to do whatever you want you can't do that  with a person right like a person you actually [34:58] have to figure out like well how do we all work  together and the larger organization is Bloomberg [35:02] is 27,000 people it's a pretty big change right  it's a pretty large rethink of like how do we [35:08] operate as a company um I think as recently as  Last November, I was on a panel um like this [35:16] one talking about how data was or AI wasn't going  to really impact data. Um so I'm on the record as [35:23] being spectacularly wrong and I think I was proven  pretty wrong, you know, very very quickly a after [35:29] that. But one of the things that we started to  notice you know internally uh when we started you [35:34] know adding an MCP server and making it so that  people could use claude or chatgbt or whatever to [35:40] uh to basically query our query our data uh and  opening it up to the team is that uh virtually the [35:46] whole company was getting u was getting value out  of it. In fact, uh, one of our sales people who's [35:51] actually in the duck overalls over there, uh, was  one of the first people who was like started to [35:56] just ask questions like, "What customer should  I talk to today?" And those are not the kinds of [36:01] questions that a that an analyst would ask or you  know typically a data person would ask but it was [36:06] a a very very it was like the the question that's  important to him and that it that you know and it [36:12] you know digs into the data about you know about  support tickets and you know customer customer [36:18] usage and what's changing and but could also  even look at you know what what's in the news [36:23] and what's actually what's actually happening and  combining combining those kinds of things and so [36:27] like um I think that the job of of uh of data is  no longer just data teams. Uh I think everybody [36:37] everybody can get um can get can get value out of  out of that. Um and that that's super exciting but [36:45] it doesn't mean that like the data team is no is  not is not important anymore because uh first of [36:50] all like the cleaner and better the data actually  is uh and the more context and the better the cont [36:56] the context is the better these you know the  rest of the company the people that are asking [37:00] these questions are going to actually be able to  answer the right questions answer those correctly [37:06] uh answer them quickly and um and so then the job  of like the data team does change because it's you [37:13] know it's a lot of it's a lot of curation it's  context curation uh it's like writing evals so [37:18] to make sure that like when somebody asks about  you know what was revenue in Q3 like you know [37:25] uh that number should be calculated correctly and  you kind of need to make sure that the systems you [37:29] have are going to are going to be doing are going  to be doing the right things should be the same [37:33] every time yes exactly oh sorry I didn't mean to  yeah no I mean but that that um it ends up being [37:39] being you know that, you know, our data team is  now more is now more important and uh and more [37:45] at the forefront of kind of because because now  that everybody's using data um you know there's [37:51] more like hey well we need this or we want to be  able to do this or like or this is not working [37:57] um and uh and so kind of I think they're less  they're less overstressed like because they have [38:03] like because they're actually able to build you  know when somebody asks them something is they're [38:07] able to use AI enabled tools to kind uh to get  things done faster and get and get more done. [38:13] Uh and they don't have to do as much as many  like, hey, can you build me this dashboard? [38:17] Because people can can generally build build up  themselves now. Yeah. No, I think that's that's [38:21] very insightful. I know we're reaching the end of  the time. So, just maybe one final question and [38:28] it's more of a prediction, so I think we should  start with you, Jordan, given your track record. [38:32] What is the uh what would you say is the biggest  architectural bottleneck that you expect for 2027? [38:40] um biggest architectural bottleneck. I mean, I  think the the the rise of agents and and I think [38:47] there's going to be agents that are going  to be knocking over systems and people are [38:50] going to be I think agents and data are just  really starting to sort of take take off like [38:55] agents and software engineering. Like everybody's  like, "Yeah, of course I run like 37, you know, [38:59] cloud code instances." Um, I think that, you know,  the next the next year we're likely going to see [39:05] people running 37 data agents that are cleaning  data and doing all sorts of things and then [39:10] that's going to, you know, knock over systems  in new and exciting exciting ways. I think in [39:17] addition to that, I don't think there's just one  thing, there's going to be a number of things, [39:20] but if I had to pull out one for instance that's  not talked about as much is access control. [39:25] I think with agents getting getting going going  everywhere um figuring out access control and [39:30] and kind of even the thought of like well what's  what does that agent represent who does that agent [39:35] represent will become increasingly challenging in  large environments particularly uh in my opinion [39:43] so I would think the major bottleneck is going  to be the compute and the efficiency of these [39:47] models whatever we build so till now all the  organizations were like you know a aggressive and [39:53] they they are making lot of investments Now but we  are at a point now organizations are coming back [39:58] you know restricting the tokens. You can see like  you know restricting the tokens and then they are [40:02] completely into the compute uh you know compute  intensive kind of world where they were uh putting [40:07] some restrictions like you know you have to use  this much of compute or you have to restrict or [40:11] you have to make your models more efficient. So  that's way like we are losing that kind of you [40:16] know flexibility of running the models as flexible  as we did. So we need to run the models and [40:22] evaluate the evaluate the things right now in very  optimized and more more kind of energy efficient [40:28] way. So that's something is a bottleneck which I  feel like you know we'll be facing now in future. [40:34] Okay. So definitely multiple bott bottlenecks.  Let's end it there. Thank you so much. If you [40:38] have questions for them please come see them  right after the show. Thank you. Thank you.