CIOs Shift to On-Premises GPU Racks Over Cloud Data Sovereignty Fears

出于云端数据主权担忧，CIO转向本地部署GPU机架 除非企业以实时事件流取代批处理ETL管道，并将原始数据库隔离在受版本控制的语义层之后，否则企业AI举措将面临失败。 技术领袖们探讨了将传统企业系统转型为就绪于AI的现代化平台的实际现实。专家小组强调，干净的实时事件流和结构化数据治理——而非仅仅是模型选型——才是企业AI取得成功的关键基石。讨论涵盖了多项架构策略，例如用于安全原型的只读复制、利用模型上下文协议（MCP）防止幻觉的语义数据层，以及解决高密度GPU集群中入包拥塞（incast congestion）等网络瓶颈。此外，演讲者还强调了对本地主权模型、细粒度Token可观测性以及严格的模式（schema）版本控制日益增长的需求，以推动可衡量的业务成果。 企业AI现代化需要在网络架构（从InfiniBand转向AI以太网）、从传统报表系统中释放数据，以及为智能体（agent）可观测性配置技术栈等方面实现底层基础设施的转变。 (0:04) AI战略必须锚定核心业务目标和高管KPI——例如在Snowflake中构建集中式上下文引擎以压缩项目周期——而非追求脱节的投机性单点解决方案。 (4:40) AI系统的主要界面已从传统图形UI转向结构化API、CLI和SDK；非标准化的遗留集成点会直接导致数据漂移和幻觉。 (8:28) 细粒度的事件驱动架构和精确的时间编码是必不可少的基础；业务事件的遗漏会造成系统性盲区（类似于生理学上的扫视），从而削弱预测性智能。 (12:33) 利用只读复制数据隔离实验性AI工作负载并暂存写入操作，可在现代化过程中保护核心事务记录系统免遭损坏。 (15:03)

Deploying generative AI within established enterprises requires overhauling underlying infrastructure, shifting from batch ETL to real-time event-driven data, and enforcing strict semantic governance to prevent probabilistic systems from failing. Bridging Legacy Systems Through Event Capture and Read-Only Layers (12:33) Integrating AI into older enterprise stacks requires modernizing data ingestion without risking core systems of record. Billy Boozer argues that event visibility with precise time-coding is the baseline requirement; failing to record micro-events creates blind spots—analogous to physiological saccades—that deprive predictive models of necessary context. To protect foundational operations during modernizations, teams should isolate prototypes using read-only replicated data and route new AI-generated writes to staging tables managed by modernized middleware. Jesse Sanders notes that surfacing rapid insights from trusted legacy data shifts user culture. At Clayco, unifying operational records into a central Snowflake repository and adapting dashboards to role-specific requirements turned field workers from skeptics into collaborators who actively work to refine upstream source data. Transitioning to Real-Time Streaming and Observability Batch processing paradigms, such as nightly ETL jobs or morning imports, are insufficient for modern AI agents designed to execute decisions synchronously. Boozer explains that organizations must push data capture toward real-time streaming at the network edge, integrating real-time telemetry from video feeds alongside trace tracking from frameworks like LangChain and Langfuse. Sanders warns that platforms lacking open APIs for direct ingestion represent compounding technical debt that must be abandoned. Unmonitored LLM usage introduces hidden organizational costs and operational drift. Dualboot Partners open-sourced Axel Insights, using TimescaleDB hypertables to log prompts, model invocations, and token volumes across 500 engineers. This telemetry exposed waste, including developers spending costly LLM tokens on basic terminal commands such as creating git commits and merging pull requests. Physical Cluster Constraints and On-Premises Repatriation (28:07) Modern AI clusters introduce physical demands vastly distinct from traditional data centers. Sani Ronen notes that an AI data center housing several hundred GPUs requires 100× to 1,000× the scale of legacy facilities, generating thermal output that necessitates direct-to-chip liquid cooling. Furthermore, high-concurrency tasks are vulnerable to incast congestion, where simultaneous multi-node transmissions to a single GPU cause switch buffer exhaustion, packet loss, and tail-latency stalls across the distributed cluster. Concurrently, sensitive regulatory obligations and high data volumes are driving a shift toward dedicated on-premises infrastructure. Sanders explains that corporate skepticism over third-party cloud data segregation led Clayco to deploy internal LLMs and small language models (SLMs) locally. Boozer adds that large, proprietary binary assets—such as CAD files used by large-scale manufacturers—suffer from unacceptable network latency when routed to multi-tenant clouds, necessitating local GPU clusters and formal AI disaster recovery plans. Semantic Context Layers and Schema Governance (40:10) Delegating enterprise authority to probabilistic AI models without triggering operational failures demands classical software engineering discipline. Rather than passing raw SQL database tables directly to AI agents—which regularly induces hallucinations—Sanders describes structuring dynamic tables layered with semantic views in Snowflake, which are then exposed to models via Model Context Protocol (MCP) servers and custom connectors. Boozer cautions against dumping unstructured documents into databases without rigid schemas, comparing untyped AI data repositories to the legacy failure mode of burying business logic across hundreds of unmanaged stored procedures. Establishing type safety, cataloged ontologies, and version-controlled schema contracts ensures that chained models receive structured, predictable inputs. While enterprises face pressure to adopt generative AI, success hinges on whether organizations can reconcile outdated data architectures, manage physical cluster networking constraints, and maintain strict schema controls before automating operational decisions.

在 Ai4 研讨会上，来自企业 IT、智算网络与高校的多位技术负责人指出，传统 IT 架构无法直接支撑现代人工智能需求；企业必须从底层数据治理、事件驱动流式管道、语义层封装及高密度物理网络进行全栈重构，并建立私有化数据主权与严格的观测审计机制。 业务驱动的数据统一与语义层治理 (4:40) Jesse Sanders 强调，推进 AI 不应从“模型能做什么”出发，而必须锚定核心业务目标与关键绩效指标（KPI）。Clayco 通过梳理高管与各业务单元目标，将历史招投标记录、CRM 数据及外部情报集中整合至 Snowflake 构建中央智能层，成功将复杂工程的 80 周施工周期压缩至 60 周。直接向大模型暴露海量原始数据库表极易诱发幻觉，因此团队在动态表之上构建语义视图，并通过模型上下文协议（MCP）服务器与标准化连接器接入大语言模型，从而显著提升生成结果的可信度。 Billy Boozer 进一步补充，现代 AI 架构必须将数据模式（schema）与本体设计纳入版本控制，避免重蹈过去业务逻辑散落在数百个难以维护的数据库存储过程中的技术债陷阱。针对非结构化数据，若缺乏标准化模式定义和严格的数据类型约束，下游应用便无法可靠预测并解析模型链传递的输入与输出。 事件驱动流式管道与安全集成范式 (12:33) 传统 IT 依赖的夜间批处理与定时报表正在失效，无法匹配未来 AI 智能体微秒级的决策节奏。Boozer 指出，系统若无法完整捕捉并标记带有精确时间戳的微观事件，就会产生类似于人类眼跳（saccade）的认知盲区，导致预测模型因信息断层而失真。每个被记录的交互事件都是模型介入赋能的操作节点，因此企业数据流必须向接近实时的流式传输演进。 为确保试验期不破坏核心业务系统的运行稳定性，小组提出的安全演进模式是对生产数据库的副本提供只读权限，将所有 AI 生成的写入操作路由至外部暂存区并通过受控中间件调和。此外，多模态模型的发展带来了成本套利空间，例如由于云厂商对视觉 Token 的计费优惠，直接将含文字图像输入视觉模型在效率和成本上已优于独立的 OCR 预处理流程。Sanders 观察到，一旦统一平台展现出业务价值，一线工程团队会自发主动改进源头数据质量以获得更精准的洞察。 接口演进、Token 观测与研发效能评估 (26:12) 面向 AI 的系统交互正从图形用户界面转向命令行界面（CLI）、软件开发工具包（SDK）和结构化 API。针对企业在接入大模型后普遍存在的无序调用，Boozer 警告必须对所有提示词、模型响应及 Token 消耗进行全量遥测。Dualboot Partners 基于 TimescaleDB 超表架构开源了监控工具 Axel Insights，在其 500 名工程师的使用分析中，发现开发者频繁将高昂的 Token 浪费在生成基础 git 提交说明等简单任务上；Lilian Ajayi Ore 援引安永研究指出，缺乏对 Token 消耗模式的监控将导致企业向超大规模云厂商持续流失巨额成本。 关于 AI 项目成功落地的量化标准，各机构给出了具体路径：DriveNets 的 Sani Ronen 透露，团队全员采用 AI 工具后达成了研发功能吞吐量翻倍或客户需求周转时间减半的年度目标；Michigan Tech 的 Josh Olson 则表示，针对建校初期遗留下来的老旧教务系统，其开发的课程规划智能体正逐步脱离人工复核，目标是在新生入学首日实现 90% 至 100% 的自主排课准确率。 智算基础设施重塑与私有主权回流 (28:07) Ronen 对比指出，AI 数据中心的能耗与计算规模是传统 IT 设施的 100 至 1000 倍。仅包含数百张 GPU 的集群所消耗的网络带宽即可相当于整座城市，极高的热密度迫使机房必须转向芯片级液冷。传统网络架构主要服务于以秒为单位的人类响应和南北向流量，而分布式 AI 训练依赖成千上万张 GPU 的东西向全同步计算，系统算力受制于最慢节点的尾部延迟；当多个 GPU 同时向单一节点发送数据时造成的入包拥塞（Incast），会导致交换机缓冲区耗尽与丢包，进而令高成本 GPU 算力陷入停滞。 面对算力成本与数据隔离要求，企业部署模式正从公有云向本地专属集群（如 NVIDIA NVL72、AMD Helios）回流。Sanders 表示，对于涉密与强监管业务，客户对公有云多租户隔离的承诺持审慎怀疑态度，推动企业将私有部署视为参与竞争的底线；Boozer 列举大型混凝土制造案例说明，海量建筑 CAD 图纸的极高吞吐量与极低推理延迟要求，使得本地部署成为必然，企业在重构硬件的同时必须建立针对模型状态与非结构化资产的长期灾备机制。 本场研讨会表明，传统 IT 面向 AI 时代的改造是一项涉及业务语义抽象、流式管道构建、Token 级成本管控及微秒级物理网络部署的系统性工程。随着企业对数据主权与低延迟确定性输出的要求提高，单纯调用公有云通用模型已无法满足合规与核心业务需求；如何在本地基础设施与海量非结构化数据增长的双重压力下建立可验证的灾备和治理体系，仍是企业技术管理层亟待解决的长期议题。

## Background 该内容由 Ai4 频道于 2026 年 8 月 22 日发布在 YouTube 平台上。视频记录了一场关于在 AI 时代重构传统 IT 系统的专题圆桌讨论，汇集了来自 Dualboot Partners、Clayco、密歇根理工大学、哥伦比亚大学及 DriveNets 的多位技术决策者，共同探讨遗留架构现代化、数据管道与网络升级、治理及组织变革等核心议题。 ## TL;DR 圆桌讨论围绕传统 IT 系统如何平稳过渡至支持现代 AI 工作负载展开，嘉宾们分享了各自的落地经验与核心优先级。讨论重点涵盖了从批处理向实时流式计算与边缘决策的演进、通过只读复制与解耦写入防范核心记录系统污染，以及利用 Snowflake 动态表、语义视图与 MCP 协议降低模型幻觉。此外，专家们深入剖析了 AI 智算中心在网络吞吐与液冷层面对传统机房的重构，强调了企业级数据主权、严谨软件工程及 Schema 版本控制对建立概率性系统决策信任的基石作用。 ## Atmosphere & style 整场圆桌讨论务实而兼具前瞻性，充满了来自大型总承包商、高校与技术供应商一线实践者的工程智慧与真实洞见。主持人开场幽默风趣并有效穿针引线，嘉宾们则直言不讳地指出数据未准备好、盲目追求模型黑盒以及缺乏 API 的封闭系统等痛点，打破了诸多浮夸的 AI 营销神话。交流过程中充满着架构层面的深度碰撞与跨行业的经验共鸣，既有从底层网络微秒级抖动到顶层业务战略的宏观推演，又洋溢着严谨工程师文化下脚踏实地解决技术债务的求真氛围。 ## Key takeaways - [04:40](https://www.youtube.com/watch?v=bADZcKI_8CE&t=280s) 企业推进 AI 的首要前提是梳理全局业务战略与深度上下文，避免陷入“AI 能做什么”的功能陷阱，而是聚焦于业务成果来构建企业智能层。 - [08:28](https://www.youtube.com/watch?v=bADZcKI_8CE&t=508s) AI 时代面向系统的新用户界面（New UI）已从传统前端页面转变为 CLI、SDK 和 API；缺乏标准开放 API 的系统应被视为直接的技术债务。 - [12:33](https://www.youtube.com/watch?v=bADZcKI_8CE&t=753s) 针对传统遗留系统的改造，应采用事件驱动架构与只读复制（Read-only access）机制，避免 AI 幻觉或写入操作污染核心记录系统（SoR）。 - [16:43](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1003s) AI 落地应顺应一线员工的实际工作流与需求，以切实的使用成效倒逼业务人员自发改善底层数据质量，而非自上而下强推变革流程。 - [21:21](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1281s) 企业数据流水线正在从延迟的传统批处理（Batch）模式全面转向逼近毫秒级的实时流式计算，以支撑未来边缘智能体的即时决策。 - [26:19](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1579s) 企业必须对提示词（Prompt）、工具调用与 Token 消耗进行全量审计和日志监控，以消除工程师不当使用导致的成本浪费并沉淀流程资产。 - [28:14](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1694s) AI 数据中心因超高带宽、微秒级尾延迟敏感性及高密度液冷需求，与传统数据中心存在物理和网络架构上的本质差异，机架级集成方案正成为主流。 - [34:12](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2052s) 由于高敏感行业客户对数据隐私与主权的严苛要求，企业正从公有云大模型黑盒逐步转向在本地环境部署私有化 GPU 集群与自主模型。 - [38:08](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2288s) 从确定性系统向概率性系统过渡时，软件工程的严谨性更为关键，需要通过安全类型约束与可信主数据记录来建立输出的确定性。 - [40:10](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2410s) 直接向大模型输入原始宽表极易引发幻觉，应通过构建动态表、业务语义视图并结合 MCP 协议进行降维约束，同时将数据 Schema 纳入严格的版本控制。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=bADZcKI_8CE&t=4s) 开场介绍与嘉宾的核心 AI 优先事项 主持人 Lilian Ajayi Ore（哥伦比亚大学专业研究学院 AI 首席导师）热情开场，并幽默地提醒台下观众如果往前坐会有“特别惊喜”。她向全场介绍了来自技术领域的四位行业领袖，指出当前传统 IT 系统如何适应 AI 时代的议题正是每位技术管理者与决策团队必须面对的核心挑战。随后，主持人要求每位嘉宾用两句话作快速自我介绍，并阐明当前办公桌上与遗留系统相关的“头号 AI 优先级任务”。 ```mermaid flowchart LR accTitle: 嘉宾核心关注领域 subgraph S1["Billy Boozer (Dualboot Partners)"] A["遗留系统改造"] --> B["AI 决策监控与可追溯性"] end subgraph S2["Sani Ronen (DriveNets)"] C["淘汰专有网络协议"] --> D["推广 AI 以太网方案"] end subgraph S3["Josh Olson (Michigan Tech)"] E["报表级数据抽取"] --> F["构建正式 AI 治理框架"] end subgraph S4["Jesse Sanders (Clayco)"] G["打通业务战略上下文"] --> H["构建企业智能层与 Agent"] end ``` - **Billy Boozer（Dualboot Partners CTO）**：专注于利用开源软件构建大规模系统（曾任 Truth Social 首席产品官并为特朗普总统搭建该平台）。当前的核心重点是帮助企业在遗留系统与现代化系统之间平稳过渡，让架构深入理解 AI 的核心原则，并具备对 AI Agent 决策过程进行监控与审计报告的能力。 - **Sani Ronen（DriveNets AI 网络总监）**：DriveNets 最初专注于服务提供商的网络解决方案，过去三年全面进军 AI 网络。他当前的首要目标是推动行业摆脱 InfiniBand、Slingshot 和 Omni-Path 等传统高性能计算（HPC）专有遗留协议，转向专为 AI 优化的新型以太网（Ethernet）解决方案。 - **Josh Olson（密歇根理工大学 CIO）**：几乎整个职业生涯都在高等教育领域度过。当前头号优先级是在大学原有的正式数据治理框架之上，搭建一套正式的 AI 治理框架。核心痛点在于如何从那些“原本只为了打印报表而设计”的古老遗留系统中抽取数据，并安全地喂给现代 AI 系统。 - **Jesse Sanders（Clayco IT 卓越运营总监）**：Clayco 作为大型总承包商，承接包括超大规模数据中心、先进制造、生命科学及关键任务设施在内的复杂建设项目。他当前的重点是将现有技术生态转化为“企业智能层（Enterprise Intelligence Layer）”。例如在建筑行业，施工工期延误是严重的“利润杀手”，通过引入智能层优化排期，成功将长达 80 周的工程周期缩短至 60 周。 --- ### [04:40](https://www.youtube.com/watch?v=bADZcKI_8CE&t=280s) Clayco 的业务战略驱动：如何打造企业智能层与销售 Agent 主持人抛出核心问题：各位目前正在主导推进哪些具体的 AI 首创项目（Initiative）？又如何界定项目是“跨过了终点线（成功落地）”还是“仍在研发中”？Jesse Sanders 率先给出了极具方法论价值的回答。他强调，AI 绝不应该仅仅是一个简单的问答机器，它必须具备对企业全局业务的“深度上下文（Context）”。技术人员往往容易陷入“AI 或技术能为你做什么”的功能陷阱，但真正正确且有力的问题应当是：“企业的战略目标与业务成果究竟是什么？” ```mermaid flowchart TD accTitle: Clayco 企业智能层落地架构 subgraph 战略与目标收集 A["高管愿景 (创始人 / CEO / CIO)"] B["各业务单元目标 & KPI"] end subgraph 数据与智能层构建 A & B --> C["企业业务上下文数据集 (Clayco 之脑)"] C --> D["接入 Snowflake 数据平台"] D --> E["生成高关联度业务 Use Cases"] end subgraph 智能化交付 E --> F["构建专属 Agentic 解决方案"] F --> G["建筑销售 Agent (整合 CRM、外部追踪与政务数据)"] end ``` Clayco 在推进实践时，首先联合创始人 Bob Clark、CEO Anthony Johnson 以及 CIO Jeff Miller 梳理顶层战略；接着深入每一个业务单元，细化各部门的目标、KPI 及其优先级权重。团队将这些目标数据化，汇聚成 Clayco 的“业务大脑”，并将其作为智能层注入 Snowflake。有了全量上下文后，业务用例的生成便水到渠成且高度契合战略目标。借助基于 Snowflake 的 Agentic 方案，Clayco 构建了一款建筑行业销售 Agent： - **整合全维度数据**：统一汇集内部 CRM 中的存量管线、商机线索、历史输赢记录，并引入外部行业追踪数据与政府公开项目信息等数百、数万条复杂记录。 - **实时秒级交互**：颠覆了传统人工翻查或仪表盘（Dashboard）的低效模式，使销售团队能够在数秒内跨海量记录精准检索潜在目标项目的地理分布、联系人详情与竞标契机。 --- ### [08:28](https://www.youtube.com/watch?v=bADZcKI_8CE&t=508s) 数据就绪度、新型交互接口（CLI/SDK/API）与决策信任 Billy Boozer 从数字化咨询与 AI 工作流落地角度切入，指出了多数企业普遍存在的底层软肋：绝大多数客户在找上门时，其“数据根本没有准备好”，甚至完全不清楚哪些数据与最终用户的业务行为存在相关性。因此，首要工作往往是协助客户采集并梳理多模态数据（视觉图像、文本数据），以及由大语言模型（LLM）在运行时产生的新型数据资产（如 Token 消耗、决策路径论证与系统链路 Trace）。 ```mermaid flowchart LR accTitle: 现代 AI 交互体系与信任机制 subgraph 传统交付 vs 新型接口 A["传统 UI / 前端展示"] -.->|演进| B["新型 UI: CLI / SDK / API"] end subgraph 规范约束与输出质量 B --> C{"技术栈是否符合 AI 规范?"} C -- 否 --> D["模型漂移 (Drift)<br/>内容幻觉 (Hallucinations)<br/>不可信数据"] C -- 是 --> E["高置信度决策输出"] end ``` Boozer 提出了一个颠覆性的技术视角：**AI 时代面向客户的“新用户界面（New UI）”不再是前端可视化页面，而是 CLI（命令行工具）、SDK 和 API**。如果技术栈内部的数据源与功能组件无法严格契合 AI 的交互规范，系统就会出现模型漂移（Drift）、幻觉（Hallucinations）和不可信数据。在当前的技术转型过渡期，确保输出结果的“绝对可信”是企业赖以做出关键商业决策的唯一基石。 --- ### [10:03](https://www.youtube.com/watch?v=bADZcKI_8CE&t=603s) 落地里程碑定义：从高校排课 Agent 到企业研发效率翻倍 在探讨“如何定义项目冲过终点线（Crossing the Finish Line）”时，Josh Olson 与 Sani Ronen 分别从高校业务落地与技术厂商内部提效两个不同维度展开说明： - **高校新生智能排课 Agent（Josh Olson）**：密歇根理工大学正在研发一款辅助大一新生的选课与排课 Agent。针对学校那套“宛如诞生于 20 世纪（1900s）般笨重古老”的教务遗留系统，Agent 会结合学生的过往学术成绩、测试分数、所选专业等多重因子，为新生定制贯穿其大学生涯的最优课表。 - **终点线标准**：目前该方案仍依赖大量人工干预、复核与审核来确保课表的合规性；只有当 Agent 能够在开学第一天“100%（或稳定在 90% 以上的高成功率）自主生成精准适用的课表”且无需人工兜底时，才算真正冲过终点线。 - **厂商内部生产力翻倍（Sani Ronen）**：作为底层技术供应商，DriveNets 实现了全员 100% 使用协同/AI 工具（如 Co-work 工具）来提升研发生产力。 - **终点线标准**：最初设定的目标是在相同研发周期内交付两倍的功能特性，或者将客户功能需求的响应交付时间缩短 50%（耗时减半）。目前团队在当年已成功达成这一生产力翻倍的里程碑，并持续加速迭代。 ### [11:43](https://www.youtube.com/watch?v=bADZcKI_8CE&t=703s) 四大支柱与核心命题：传统系统与现代 AI 的桥接挑战 各部门正在跨领域构建并共享能力，从营销技巧、演示文稿制作到软件开发技能，这种跨部门协作带来了显著的成效。主持人 Lilian Ajayi Ore 借此梳理并总结了企业推进 AI 时的四大核心支柱：战略（Strategy）、信任（Trust）、可用性（Usability，即明确谁在使用产品和工具）以及不可或缺的治理（Governance）。 随后，主持人提出了整场讨论中最核心的技术难题：传统遗留系统（Legacy Systems）在设计之初并非为了承载现代 AI 工作负载。面对这一现实，企业该如何制定最有效的策略，在不中断核心业务流程与日常运营的前提下，将过时的技术架构与前沿的 AI 能力无缝桥接？ --- ### [12:33](https://www.youtube.com/watch?v=bADZcKI_8CE&t=753s) 事件驱动与只读复制：构建不破坏核心业务的智能层 针对传统系统的改造，Billy Boozer 率先给出了极其明确的架构判断：如果遗留系统无法输出事件，或者企业无法明确界定系统所输出事件的属性，那么该系统根本还没有准备好在其上叠加 AI 智能层。捕获事件并确保其带有精准的时间戳编码至关重要，只有这样才能为 AI 提供最新、不陈旧的数据，避免基于不可信的数据做出决策。每一次事件的触发，都代表着内部组织或用户行为的一次真实活动，而每一次活动都是系统进行智能介入的契机。 Billy 以 Josh Olson 先前提到的大学新生入职与入学注册（Onboarding）为例：学生从刷门禁卡、课程排期到财务状态变更，会产生海量的事件流。如果能完整捕捉这些事件，就能为学生创造成功的支持环境；但如果遗失了任何事件，系统就会出现认知断层。Billy 将这种断层比作视觉认知中的“扫视”（Saccade）现象——当视线从一处跳跃到另一处时，大脑无法插值中间缺失的信息，从而导致数据层面的盲区。 ```mermaid flowchart TD accTitle: 遗留系统安全接入 AI 架构图 accDescription: 展示传统记录系统通过只读副本与中间件安全连接到 AI 智能层的架构 subgraph LegacyLayer["传统系统层 (Legacy Systems)"] A["核心记录系统 (SoR)"] B["数据副本 (Replicated Data)"] C["独立暂存库 (Alternate Location)"] end subgraph ProcessingLayer["中间件与管道 (Middleware & ETL)"] D["事件捕获与时间戳规范化"] E["中间件提取工作流 (ETL Pipelines)"] end subgraph AILayer["智能应用层 (AI Intelligence Layer)"] F["LLM / 多模态推理引擎"] G["智能应用与业务决策"] end A -->|"只读同步 (Read-only)"| B B -->|"读取数据 (防止数据污染)"| D D --> E E --> F F --> G G -.->|"写入需求 (避免直接回写 SoR)"| C C -->|"校验后回流"| A ``` 为了确保在试点阶段不破坏传统系统的稳定性，团队采取了严格的数据隔离策略： - **只读复制机制**：初期试点仅向 AI 提供复制数据的只读访问权限（Read-only access）。这样最坏的情况也只是模型产生幻觉，而绝不会污染核心记录系统（System of Record, SoR），避免对遗留系统造成不可逆的中长期下游故障。 - **解耦写入路径**：当 AI 必须执行写入操作时，数据会被写入到独立的备用存储位置，随后通过中间件修改提取工作流（ETL）来对接这些新数据源。 - **成本与架构套利**：当前正是现代化重构 ETL 流水线的黄金时机。例如前几周发现将图文混合数据输入大语言模型时，由于底层 OCR 成本折扣，Token 的使用效率反而更高。这表明在智能系统的支出控制、事件规范化以及数据结构标准化方面存在巨大的套利与优化空间。 > “AI 应当适应你的业务，而不是让业务去生搬硬套 AI（AI should adapt to your business, not the other way around）。” --- ### [16:43](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1003s) 施工现场的智能化落地：以用户需求驱动数据治理 Jesse Sanders 结合建筑施工行业的实际场景分享了实践经验。在建筑工程中，企业内部往往分散着各类已被业务部门认可的受信数据源。技术团队的核心任务在于推动文化转变，建立最终用户对 AI 工具的信任。只要这些系统具备 API 接口，就可以将数据汇聚到统一的智能层（如 Snowflake 数据仓库），从而加速原型（Prototype）向生产产品（Product）的转化。 Jesse 强调，当用户体验到 AI 带来的切实效率后，他们的行为模式会自发改变。很多时候企业的数据质量并不理想，但一旦业务人员看到了高质量数据配合 AI 所能释放的巨大能量，他们就会主动改进自身的工作流程，确保录入准确的数据。 - **智慧工地看板案例（Job Site Intelligence Dashboard）**：团队在施工现场的项目拖车（Trailers）中部署了项目状态可视化大屏。最初技术团队以为这是一套“通用型”（One-size-fits-all）方案，上线后才发现，项目经理（Project Manager）和项目总监（Project Director）基于同一份数据所关注的核心问题完全不同。 - **顺应用户习惯而非强推流程**：技术团队没有去批评一线人员“数据录入有误”或“使用方式不对”，而是选择“在用户所在的地方与他们会合”（Meet people where they're at），根据不同角色调整洞察维度。一线人员感受到价值后，便会主动提问“我们如何改进数据与业务流程”，这彻底逆转了以往 IT 部门自上而下强推变革却屡遭抵抗的被动局面。 --- ### [19:00](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1140s) 多模态新数据源与 LangChain 追踪：赋能高自主性员工 Billy Boozer 进一步指出，现代智能化转型正在引入大量前所未有的新型数据资产： - **现场视觉数据**：施工现场安装的摄像头不再仅用于安防监控或资产保护，它们已经转变为推动流程自动化（Process Automation）的核心多模态新数据集。 - **新型数据架构与追踪**：传统的记录系统（如 GitHub、Blackboard、Procore 或网络协议日志）只存储静态主数据，缺乏现代 AI 运行产生的动态数据——例如来自 Langfuse 或 LangChain 的链路追踪记录（Traces），以及将多模态图像解析后存入哈希数据库（Hash-based databases）以供高效检索的数据。 将这些多维的新数据源与系统深度整合，能够让组织清晰洞察每日待办事项（Work to be done）。当企业中具备高度主观能动性（High Agency）的员工能够顺畅调用 AI 系统和精准数据时，他们便能在极短时间内完成多轮迭代，这直接构成了企业在 AI 时代的核心超级竞争力（Superpower）。 主持人 Lilian 总结强调，系统建设必须保持一致性，关注每位员工的具体工作流，并通过持续的反馈闭环（Feedback Loop）不断打磨系统，打破“AI 系统一经建成即可自动运转”的错误假设。 --- ### [21:21](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1281s) 重塑数据流水线：从传统批处理转向实时流式计算 在进入下一个讨论环节时，Lilian 针对底层数据基础设施提出了进阶问题：AI 固然无所不能，但维护和管理数据需要专业技能。企业在融合新型多模态数据与陈旧历史数据时，应如何重塑自身的数据流水线（Data Pipelines）与存储架构？ Billy Boozer 从数据交付模式的演进切入分析：过去企业运营严重依赖批处理模式（Batch Processing），例如设定固定的清晨数据导入或午后数据转储。然而，现代业务对数据时效性的容忍窗口正在急剧收窄。这就如同现代劳动力对薪酬支付方式的诉求转变一样——人们越来越倾向于“即时薪酬”（Earned Wage Access），要求按劳即时结算，而不是等待周期性的批量发薪。企业数据流同样正在从迟滞的批处理彻底走向实时化。 ### [22:39](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1359s) 实时数据管道与边缘决策：从批处理转向即时响应 企业数据管道必须制定严格的策略，让数据尽可能逼近实时流动，因为未来的智能体（Agents）将以微秒级的速度做出决策。随着模型变得更加高效轻量，决策引擎将直接下沉部署到网络边缘。在建筑施工现场等真实物理场景中，社会上已经普及了类似 Flock Safety 的边缘安防，企业的日常闭路监控（CCTV）也完全有能力在数秒内识别异常并触发行动。然而，企业当前普遍缺乏将这些边缘数据快速同步的核心系统。大量高价值数据往往沉睡在边缘设备中，只有将它们汇聚到 Snowflake 等核心数据湖与聚合工具中并叠加智能层，才能真正发挥作用。如果企业仍停留在传统的“批处理（Batch）”思维模式中，就会在组织内部不断加剧异步作业的滞后，而当今业务的核心诉求早已转向高频同步。 这种节奏变化直接体现在日常的工程与开发交付中。以快速搭建营销网站为例，依托大量 Markdown 文档和信息提炼，开发者仅用两三个小时就能完成以往需要数天的建站任务。然而，当营销负责人首次目睹这种极速交付后，紧接着提出的要求就是：“下周我们能不能一口气再上 15 个项目？”这说明技术加速不仅改变了交付速度，更彻底拉高了业务部门的预期——人们不会因为任务提前完成而停下休息，反而会迅速加码，试图无限放大其潜在产出。 ```mermaid flowchart TD accTitle: 边缘数据到核心智能决策的流转链路 subgraph Edge["边缘网络 (Edge)"] Sensors["CCTV / 传感器 / 现场设备"] EdgeEngine["轻量级边缘决策引擎"] Sensors --> EdgeEngine end subgraph Core["核心数据与智能层 (Core)"] Pipeline["实时数据同步管道"] Lake["集中式数据湖 (Snowflake / Databricks)"] AI["企业级 AI / 智能体决策层"] Pipeline --> Lake Lake --> AI end EdgeEngine -->|"毫秒/微秒级响应"| Pipeline AI -->|"闭环指令"| Sensors ``` --- ### [24:47](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1487s) 数据才是真正的基石：拒绝无 API 的技术债务 回顾过去三年大语言模型（LLM）刚兴起时的狂热，业界曾普遍宣称“大语言模型是企业的新底座，它将颠覆一切”；但现实很快给出了清醒的答案——数据才是且永远是企业技术架构的真正基石，大模型和 AI 只不过是一面放大镜，将底层数据的优劣无情地暴露出来。因此，企业在评估引入任何内外部解决方案时，如果该系统不提供开放 API 或无法支持灵活的调用与数据摄取，就必须果断放弃，因为这种封闭系统“就是直挺挺摆在面前的技术债务”。 企业若想重塑 IT 环境，就必须对那些缺乏开放性的传统供应商施加压力，或者重构数据链路。无论底层平台选用 Databricks、Snowflake 还是其他集中化方案，核心原则只有一个：必须将分散在各处的数据集中到一个统一的数据层中，让团队能够基于全局数据提出具备洞察力的智能化问题，进而激发更多业务创新。 - **大模型与数据的关系**：大模型只是表现层与分析工具，数据质量与流动性决定了 AI 能发挥的上限。 - **选型硬性红线**：缺乏标准化 Open API 的遗留软件和新采购系统，在 AI 时代均应被视为重大技术债务。 - **数据中心化底座**：必须建立跨业务线的统一数据访问层，方可支撑高阶智能分析与自动化决策。 --- ### [26:19](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1579s) 提示词与 Token 消耗审计：避免工程团队的隐性成本 对于企业在使用大语言模型时忽视输出与调用数据沉淀的现状，必须保持高度警惕。企业应该对每一次 Prompt 输入、助手回复、工具调用（Tool Calls）以及工程师使用的具体工具链进行全量日志记录，因为这些交互数据本身就是极具价值的新型资产，能够为未来的流程优化与模型训练提供支撑。例如，团队开源的 Axel Insights 工具便通过将 500 名内部工程师的每一次提示词、模型版本及 Token 消耗实时写入 TimescaleDB 的超表（Hypertable）中进行全量监控。 这种全维度的可观测性直接揭示了许多意想不到的效率黑洞。日志分析显示，不少工程师甚至会消耗昂贵的 Token 让大模型去生成 Git Commit 提交信息、推送 PR 或合并分支——而这些操作在终端中仅仅需要一两条简单命令即可完成。当乘以 500 名工程师的规模时，这种由“工程惰性”引发的 Token 开销就会演变成一笔巨大的财务浪费。安永（EY）的研究也明确指出了企业在 Token 使用中面临的成本效益失衡问题；如果企业不主动捕获和沉淀这些调用模式数据，不仅会遭受资金损失，还会将最有价值的流程洞察拱手让给云厂商。 --- ### [28:14](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1694s) AI 集群对传统数据中心的重构：算力、网络与机架级方案 传统的企业数据中心与专为 AI 设计的数据中心完全是“截然不同的物种”，二者在规模和物理架构上存在数百到数千倍的巨大差距。AI 集群需要承载自主模型的研发与训练，传统数据中心在机房空间、供电容量、散热设计和网络拓扑上均无法直接承载 AI 负载。以规模为例，一个配备数百颗顶级 GPU 的 AI 数据中心，其内部网络带宽需求甚至能匹敌整座拉斯维加斯市的带宽总量；在散热方面，高密度的计算发热也迫使机房必须从传统风冷全面转向液冷架构。 更根本的差异在于网络传输逻辑与容错机制。传统 IT 应用面向的是秒级的人类交互响应，而 AI 训练则是数千颗 GPU 协同进行分布式矩阵计算，其计算进度取决于“最慢的那颗 GPU”（尾延迟效应），因此微秒级的网络抖动都至关重要。单卡 800 Gbps 的带宽本身并不足以保证性能，当多颗 GPU 同时向单一点发送数据形成微突发（Incast）时，极易引发网络丢包，导致整个集群的计算陷入停滞。在成本结构上，传统机房的重心是服务器与通用通信设备，而 AI 机房中 GPU 算力占据了绝对的成本大头；此时针对网络与调度层面的优化投入，相比于昂贵 GPU 闲置带来的折损而言微不足道。 | 架构维度 | 传统企业级数据中心 (Legacy DC) | 现代 AI 算力中心 (AI Cluster) | |:--- |:--- |:--- | | **核心业务诉求** | 面向人类响应时间（秒级/亚秒级） | 分布式并行计算（微秒级同步，消除尾延迟） | | **网络主要挑战** | 通用路由吞吐、常规负载均衡 | Incast 拥塞控制、高吞吐无损网络、消除丢包 | | **散热与电力配置** | 传统风冷、中低机架功率密度 | 高密度液冷（Liquid Cooling）、超大规模供电 | | **核心成本构成** | 服务器硬件、交换机及基础通信设施 | GPU 算力集群占据绝对大头，网络属于配套杠杆 | | **典型交付形态** | 分立式服务器堆叠、公有云虚拟化租用 | 机架级集成方案（如 NVL72、AMD Helios） | 随着金融、科研等行业对敏感数据合规性要求的提高，企业自建本地（On-prem）AI 集群的需求正在迅速攀升。为降低部署门槛，以 NVIDIA NVL72（72 颗 GPU 的机架级系统）以及 AMD Helios 为代表的整机柜交付方案正在成为主流，厂商通过把高密算力、液冷管路与网络交换深度集成在单个机架内，简化了企业落地 AI 基础设施的复杂度。 --- ### [32:20](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1940s) 从“黑盒”到掌控底层：本地部署与 AI 投资回报率的重构 算力架构在公有云租用与本地自建之间的往复循环，标志着企业认知正在迈入成熟阶段——技术团队不再把 AI 当作一个盲目输入提示词的“黑盒子”，而是开始像钻研 Linux 内核底层机制一样，去深刻理解算力、网络与数据的运行机理。只有掌握这种底层逻辑，企业才能将技术能力与真实的财务回报精准对齐。 将高带宽、低延迟的 AI 决策能力直接赋能给关键业务人员，能够形成强大的组织协同效应。过去，企业向公有云大模型黑盒中倾倒了堆积如山的资金，却往往很难算清投入产出比（ROI）；而当下通过合理调配本地算力与 Kimi、Qwen（通义千问）等先进开源与商用模型，深入理解系统的成本构成与能力边界，企业才真正迎来了看清并实现 AI 商业化 ROI 的拐点时刻。随后，论坛正式进入现场观众互动提问环节。 ### [34:12](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2052s) 数据主权与本地算力：企业大模型落地的现实考量 讨论围绕企业在推进 AI 基础设施时所面临的资源约束、预算编制以及人才专业知识展开。来自 Clayco 的 Jesse Sanders 指出，数据主权正在成为企业级建设的“新前沿”（the new sovereignty frontier）。在承接高度受监管、敏感度极高的未来建筑工程项目时，客户对数据安全提出了严苛的要求。最初，团队向客户推介企业级大型语言模型时承诺系统是完全隔离的、绝不会使用私有数据进行训练；但如今客户的态度已经转变为“我听到了你的承诺，但我并不相信你（I hear you, but I don't believe you）”。为了赢得客户信任，企业必须转向在内部环境中构建和运行大模型（LLM）或小型语言模型（SLM）。Jesse 强调，企业目前的重心是培养内部构建此类模型的能力，并将其转化为核心竞争力与业务增值服务，因为对某些重点客户而言，这是达成合作的硬性先决条件。在预算与投资决策上，必须紧密挂钩业务成效，评估其究竟是直接创造营收、带来利润率扩张，还是提升运营效率或客户满意度。 Dualboot Partners 的 CTO Billy Boozer 紧接着从数据增长与存储的角度进行了补充。他半开玩笑地声明“这绝非投资建议”，但如果现在要投资一家企业，他一定会投数据库，因为从此刻起到未来的无尽岁月里，数据只会呈爆炸式增长。人类在过去 12 个月内生成的文本量，将远超以往任何时期，而且这种增长节奏在接下来的 6 个月、24 个月里会不断成倍刷新。这就要求企业必须拥有可靠的存储载体。鉴于存储成本低廉且访问速度前所未有地快，企业应当持续将数据存入仓库以备未来之用。 Billy 分享了一个近期在佛罗里达州大型混凝土企业的实际案例。该企业专门建造巨型立体停车楼，其庞大的 CAD 工程图纸和项目文件产生海量数据。出于低延迟提取智能信息的需求，管理层正面临一项重大决策：是否在本地机房进行大额投资，部署 GPU 计算集群和私有数据存储。他们已经果断决定淘汰那些无法提供低延迟响应和封闭智能的外部系统，转向能够完全掌控底层数据访问权限的基础设施。Billy 建议，如果企业正在评估私有化托管（On-prem）与采购硬件，当下正是引入此类基础设施进行验证测试的极佳窗口期；企业必须建立起针对 AI 关联数据的容灾恢复策略（Disaster Recovery Policy），以确保对底层硬件拥有确信掌控，并永久保有自身的数据主权。 - **客户信任门槛的演变**：从接受公有云企业版“不训练数据”的合规声明，倒逼转向要求完全本地化、私有化部署的 LLM/SLM。 - **业务投资回报驱动（ROI）**：私有化算力与模型环境的立项，严格绑定于新增收入、利润率提升、运营提效或关键客户满意度。 - **制造业本地 GPU 集群趋势**：以建筑与重工设计为代表的行业因 CAD 等超大文件与毫秒级延迟诉求，正从云端回撤至本地高性能存储与算力平台。 --- ### [38:08](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2288s) 提问环节：从确定性系统向概率性系统安全过渡 现场观众提出了关于系统本质转变的深度疑问：过去几十年来，传统 IT 系统始终是完全确定性的（deterministic），架构师对系统如何运转、产生何种输出拥有绝对的控制权与权威；然而现在技术体系正全面转向概率性系统（probabilistic systems），企业实际上在将控制权让渡给具有不确定性的 AI 模型。那么在系统重构过程中，如何以最安全的方式完成控制权的委派，避免造成不可承受的业务后果？ Billy Boozer 对此回应指出，核心命题在于企业如何在无法完全搞清楚内部组件黑盒运作的情况下，依然能够安全地过渡并信任概率性系统所输出的决策。他强调，解决之道绝非认为“工程学已经过时了”。恰恰相反，严谨的软件工程比以往任何时候都更加关键。系统内部的“安全类型约束”（safe typing）是建立可信数据的根本基石；企业必须在数据仓库中沉淀出经过清洗、高度可信的“主数据记录”（Master Records），并围绕这些记录在跨业务与客户关系之间建立确凿的数据关联。 对于所有企业来说，核心任务是将底层数据规范地塑造成一致的结构形态，确保在严格的身份认证（Authentication）与权限控制（Authorization）协议框架下，使组织内的任何人都能顺畅访问这些数据，并安全地将其转化为日常工作所需的生产力智能。 --- ### [40:10](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2410s) 构建语义层与模式版本控制：让 AI 理解数据背后的业务语境 另一位来自现场的观众提到了数据质量向语义层（Semantic Layer）演进的痛点。提问者指出，传统机器学习界自 2012 年起就在与数据质量问题斗争，如今这项基础工作终于得到了普遍重视；但在清洗干净的数据之上，企业内部通常沉淀了大量特定指标、专业术语与自定义定义，这些概念并未硬编码进模型权重中，必须依赖覆盖在数据层之上的语义层来传递上下文。 Jesse Sanders 详细拆解了 Clayco 在 Snowflake 上的工程落地经验。他们的数据管道初期将原始数据摄入至基础表格与数据列中，随后基于这些底表构建“动态表”（Dynamic Tables），并在动态表之上封装出精巧的“语义视图”（Semantic Views）。这一设计为 AI Agent 与数据的交互提供了既优雅又兼具成本效益的途径。在语义层之上，他们部署了 MCP（Model Context Protocol）服务器，向下连接底层视图，向上对接 Claude、OpenAI 等大模型连接器。Jesse 直言这一架构是决定性的突破，因为如果只是把结构化表格一股脑塞给 AI，即便为字段标注了 ID 和标签，一旦数据规模扩大，模型依旧会严重幻觉、吐出非确定性的混乱结果；而依托语义视图与动态表，交付给模型的信息精准度大幅跃升，最终赢得了终端业务用户的信任。 ```mermaid flowchart TD accTitle: Data Semantic Layer Architecture for AI Integration subgraph DataPlatform["Snowflake 数据底座"] Raw["原始数据列 / 基础表<br/>(Ingested Columns & Tables)"] DT["动态表<br/>(Dynamic Tables)"] SV["业务语义视图<br/>(Semantic Views)"] Raw --> DT DT --> SV end subgraph IntegrationLayer["上下文与模型协议层"] MCP["MCP 服务器<br/>(Model Context Protocol)"] SV --> MCP end subgraph ConsumerLayer["大语言模型与 Agent"] Claude["Claude Connectors"] OpenAI["OpenAI Connectors"] MCP --> Claude MCP --> OpenAI end ``` Billy Boozer 从数据模式版本控制（Schema Version Control）与系统演进历史给出了强有力的警示。他回忆道，过去企业常常把存储过程（Stored Procedures）写满整个数据库，将业务逻辑深锁在动辄累积了 500 个、每个长达 4000 行代码的巨型存储过程中，最终导致整个架构固步自封、完全无法解耦重构。要打破这种泥潭，第一步就是将代码放入版本控制系统中；这一原则完全适用于新型记录系统或现代数据仓库中的数据 Schema 与业务本体（Ontology）。统一的数据协议和结构化输出（如 OpenAI 定义的标准结构规范）使得各类型模型（如 LLaMA、LM Studio 等）都能标准化地解析并消费数据。Billy 告诫开发者，切忌贪图一时省事将未经治理的非结构化数据直接甩进 MongoDB 或 Elasticsearch 中，妄想日后从里面捞出 JSONB 文档时能自然搞清其背后的业务逻辑；只有严密把控数据 Schema 的版本演进、结合静态或动态类型约束，才能对系统的输入输出建立明确预期，使多智能体管道协同决策成为可能。 - **表格数据向大模型输入时的幻觉风险**：大宽表即便具备完备的列 ID 映射，在海量数据注入场景下仍极易引发 LLM 幻觉；必须通过抽象动态表与语义视图进行降维约束。 - **存储过程的历史教训重演**：业务逻辑不可深埋于数据库黑盒中，Schema 定义、本体设计与数据转换逻辑必须像常规代码一样进入严格的 Git 版本控制。 - **数据结构化规范**：反对无节制的非结构化 NoSQL 存储，倡导规范的数据模式约束与严格类型控制，保证各模型输入输出的确定性。 --- ### [43:55](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2635s) 闭幕致谢与总结 主持人向现场所有提出精彩问题的与会者表示感谢，并建议未能在有限时间内发言的参会者在会后直接与各位嘉宾建立联系、展开深入技术探讨。在全场观众的热烈掌声中，主持人对各领域专家的精彩洞见表示了高度肯定，本场关于传统 IT 架构面向 AI 时代大修的专题圆桌讨论圆满落幕。 ## Notable quotes > [08:28](https://www.youtube.com/watch?v=bADZcKI_8CE&t=508s) AI 时代面向客户的“新用户界面（New UI）”不再是前端可视化页面，而是 CLI（命令行工具）、SDK 和 API。 — Billy Boozer > [12:33](https://www.youtube.com/watch?v=bADZcKI_8CE&t=753s) AI 应当适应你的业务，而不是让业务去生搬硬套 AI（AI should adapt to your business, not the other way around）。 — Billy Boozer > [16:43](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1003s) 在用户所在的地方与他们会合（Meet people where they're at）。 — Jesse Sanders > [24:47](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1487s) 数据才是且永远是企业技术架构的真正基石，大模型和 AI 只不过是一面放大镜，将底层数据的优劣无情地暴露出来。 > [24:47](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1487s) 如果该系统不提供开放 API 或无法支持灵活的调用与数据摄取，就必须果断放弃，因为这种封闭系统就是直挺挺摆在面前的技术债务。 > [28:14](https://www.youtube.com/watch?v=bADZcKI_8CE&t=1694s) 传统的企业数据中心与专为 AI 设计的数据中心完全是“截然不同的物种”，二者在规模和物理架构上存在数百到数千倍的巨大差距。 — Sani Ronen > [34:12](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2052s) 数据主权正在成为企业级建设的“新前沿”（the new sovereignty frontier）。 — Jesse Sanders > [34:12](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2052s) 我听到了你的承诺，但我并不相信你（I hear you, but I don't believe you）。 — Jesse Sanders > [34:12](https://www.youtube.com/watch?v=bADZcKI_8CE&t=2052s) 如果现在要投资一家企业，他一定会投数据库，因为从此刻起到未来的无尽岁月里，数据只会呈爆炸式增长。 — Billy Boozer ## Mentioned Billy Boozer, Jesse Sanders, Josh Olson, Lilian Ajayi Ore, Sani Ronen, Bob Clark, Anthony Johnson, Jeff Miller, Dualboot Partners, Clayco, Michigan Technology University, Columbia University SPS, DriveNets, Truth Social, Snowflake, Databricks, Langfuse, LangChain, Axel Insights, TimescaleDB, EY, NVIDIA, NVL72, AMD Helios, Kimi, Qwen, Model Context Protocol, Claude, OpenAI, LLaMA, LM Studio, MongoDB, Elasticsearch, Blackboard, Procore, Flock Safety, InfiniBand, Slingshot, Omni-Path ## Video description Billy Boozer — CTO, Dualboot Partners Jesse Sanders — Director of IT Operational Excellence, Clayco Josh Olson — Chief Information Officer, Michigan Technology University Lilian Ajayi Ore — AI Lead Faculty, Columbia University SPS Bestselling Author, The Power of Learning Mindset, Columbia University Sani Ronen — Director of AI Networking, DriveNets Legacy IT systems were not designed with AI in mind—yet they still power critical enterprise functions. This panel will explore how IT leaders are modernizing legacy infrastructure to support AI workloads, from upgrading data pipelines and storage to rethinking networking, middleware, and governance layers. Panelists will discuss approaches to phasing out outdated systems, bridging old and new technologies, and ensuring business continuity during large-scale modernization efforts. Expect candid lessons on budget allocation, organizational change management, and vendor selection. Learn how enterprises are re-architecting their IT foundations to stay competitive in an AI-first world. [Source](https://www.youtube.com/watch?v=bADZcKI_8CE)

[8:28] Billy Boozer: AI 时代面向客户的“新用户界面（New UI）”不再是前端可视化页面，而是 CLI（命令行工具）、SDK 和 API。 [12:33] Billy Boozer: AI 应当适应你的业务，而不是让业务去生搬硬套 AI（AI should adapt to your business, not the other way around）。 [16:43] Jesse Sanders: 在用户所在的地方与他们会合（Meet people where they're at）。 [24:47] 数据才是且永远是企业技术架构的真正基石，大模型和 AI 只不过是一面放大镜，将底层数据的优劣无情地暴露出来。 [24:47] 如果该系统不提供开放 API 或无法支持灵活的调用与数据摄取，就必须果断放弃，因为这种封闭系统就是直挺挺摆在面前的技术债务。 [28:14] Sani Ronen: 传统的企业数据中心与专为 AI 设计的数据中心完全是“截然不同的物种”，二者在规模和物理架构上存在数百到数千倍的巨大差距。 [34:12] Jesse Sanders: 数据主权正在成为企业级建设的“新前沿”（the new sovereignty frontier）。 [34:12] Jesse Sanders: 我听到了你的承诺，但我并不相信你（I hear you, but I don't believe you）。 [34:12] Billy Boozer: 如果现在要投资一家企业，他一定会投数据库，因为从此刻起到未来的无尽岁月里，数据只会呈爆炸式增长。

[12:33] Billy Boozer: If you drop any of those events, what you leave is a gap in your knowledge. It's basically like... the concept of saccade. It's basically when you look from one place to the next, your mind can't interpolate these things. You're losing some layer of data that you have the opportunity to gather, but you are not codifying that within a data system within your organization. [16:30] Jesse Sanders: AI should adapt to your business, not the other way around... We had to meet those individuals where they're at rather than trying to change their mindset about, 'No, you're not using this the right way. This data is wrong.' It's more or less, 'How do we get them to get the information that they're looking for, providing that insight?' And then culturally, now they're asking, 'How do we make this data better?' [22:39] Billy Boozer: Hey, can we push 15 more things? [24:47] Jesse Sanders: run, because that's all it is is just technical debt looking you straight in the face. [34:12] Jesse Sanders: It's okay, I hear you, but I don't believe you. [40:10] Billy Boozer: You remember the time in which we put stored procedures into every database and locked ourselves into data structures that were impossible to get out of as we accumulated 500 stored procedures with 4,000 lines of code that housed all of our business logic. The easiest step to get away from that is put the stored procedures into version control first... The same applies to the schema and the ontology that you create within your new systems of record.

[0:04] Good morning everyone. Thank you for choosing  to be here today. Just want to share that if [0:09] you sit closer to the front, we have a special  surprise for you. So in case you want to move [0:14] your seats later on, you can. Um I am joined  by four fantastic leaders in the tech space. [0:22] And today's conversation is on every single one of  us's mind. Some of you may have the responsibility [0:28] of addressing this problem. Some of you guys may  be involved in a team that's making decision about [0:33] this problem, but they're going to give us  some key insights on how to navigate this new [0:38] era that we're all embarking on. All right, so  I'm going to jump right to it. Welcome everyone. [0:43] Great seeing you all today. Let's start with a  rapid fire introduction. Give us two sentence [0:49] on your background and share your number one AI  priority related to legacy AI systems on your desk [0:57] right now. Uh my name is Billy Boozer. I'm the  CTO of Dual Boot Partners. Um uh my background [1:05] is building largecaled systems, especially with  open-source software. I was previously the chief [1:10] product officer at Truth Social, so I built that  for President Trump. Um and our our number one [1:15] focus today is um really helping people transition  between uh systems, legacy systems and modernized [1:24] systems that understand the core tenants of AI  and the requirements for monitoring applications [1:30] and reporting against the decisions that are  being made by agents. Thank you. Good morning. [1:37] My name is Sani Ronin. I'm director of AI  networking in Drivets and Drivets develops [1:43] AI networking solution. Historically we started  with networking for service providers and in the [1:49] last three years we are doing AI networking.  So this is part of the discussion today. The [1:54] the difference is quite big between the two  different product line. But basically what [1:58] we are trying today is to help the industry get  away from legacy protocol and legacy product. In [2:04] the AI networking historically for HPC there were  infiniband and slingshot and omnipad and today we [2:12] try to help the industry to make sure internet  or a special version of internet will be good [2:17] fit for AI networking. Thank you. Thank you. Good  morning. My name is Josh Olsen and I am the chief [2:25] information officer for Michigan Technological  University. Um, I've spent almost my entire [2:31] career in higher education. Our top priority  with regard to AI right now is introducing a [2:38] formalized AI governance framework on top of our  university formalized data governance framework. [2:44] We're basically trying to figure out how to  extract data from our legacy systems that were [2:49] designed to basically print reports into systems  that can uh be utilized to safely feed uh AI AI [2:58] systems. Thank you. Thanks for the question and uh  I am excited to be here at this AI4 conference. It [3:07] is amazing. The uh this morning's keynote was  incredible, wasn't it? Yeah, it was amazing. [3:13] And so my name is Jesse Sanders. I'm the director  of IT operational excellence at Clico. My focus [3:19] is connecting people, data, and technology to  help improve our construction teams and our [3:25] business leaders to make better decisions. And  so at Cleico, we are a general contractor and we [3:32] focus on building the world's some of the world's  most complex construction projects. your hypers [3:37] scale data centers, your advanced manufacturing,  u life sciences and mission critical facilities. [3:45] And so because of that operational excellence  and trusted data, wink wink, is extremely [3:52] important for our success for our clients. My  priority right now is transforming our existing [3:58] um technology ecosystem and turning into an  intelligence uh enterprise intelligence layer [4:04] where we're leveraging our current investments  and AI to help improve the way we build projects [4:11] for our clients. Why is that important? So, one  of the examples is scheduling. Scheduling in [4:16] construction is a margin killer. So many things  happen at a construction site that throws things [4:22] off track. And so we're using technology like  this to help improve the way we build buildings [4:28] for our clients. And take for example a project  that takes 80 weeks down to scale to uh 60 weeks [4:34] is what what we do. Excellent. Thank you. All  right. Um, this goes really well with our next [4:40] question for my panel. And the question is, what  AI initiative are you championing right now? And [4:48] how do you define crossing the finish line versus  still in development? Whoever wants to take this. [4:59] So, um, the one thing that I I think about  is that AI shouldn't just answer questions, [5:04] right? It should have context about your business.  And one of the things that we learned at Clayco [5:11] um and you guys have all heard this as well. Um  technology is a strategic enabler. And we have [5:16] a tendency sometimes as technology people what  can AI do for you? What can technology do for [5:20] you? And that is a thought that is an approach.  But the power in the right question is what is [5:26] the strategic objectives? What is your business  outcomes? You you heard all those things and and [5:31] what we done at Clico is that we worked with our  uh executive u founder Bob Clark, our CEO Anthony [5:39] Johnson and our CIO who's Jeff Miller who's in a  competing uh conversation right now downstairs in [5:45] another panel. But uh but we asked them what are  your strategic objectives? What are those things [5:50] that that that KCO wants to drive towards? And  then we talked to all of the business leaders of [5:55] each of the business units. What are your business  goals? What are your outcomes? What are your [6:00] technology um uh uh target objectives and your  KPIs? And what's the priority of those of those [6:07] objectives and those uh and those KPIs you're  trying to look for? And we turned that into a data [6:12] set. That data set essentially came a brain of  Clico where that became a part of our intelligence [6:19] layer that we tied into Snowflake. And so as a  result of that once we created all that context [6:25] once we created all that information then what  came next it was the use cases and the use cases [6:31] were more u intertwined and tied to those outcomes  than we ever been able to see before as far as [6:37] doing it as separate one-offs. And once we had  the use cases, then we were able to ask Snowflake [6:43] in our particular case, coco, how many people  use Coco and Snowflake or co-work any co-work [6:48] users here? Right. And so we use that information  now to have those use cases to build us agentic [6:56] solutions that are more aligned with the business  outcomes. And it's just like a light bulb went off [7:01] when that happened. And this is something there's  a difference between know and actually executing. [7:06] But once you identify those business goals, um,  we know that this works every time. I've done [7:11] this for entrepreneurs, for political groups,  for nonprofit organizations, school systems, [7:17] churches, startup companies, you name it. And  it's always been the same response. It's like, [7:22] wow, now the whole world, the light bulb just  just goes off at that particular point. And so [7:27] now once we had that information, it changed the  way we worked. And one of the solutions that came [7:33] after that is that we built a sales agent um for  our construction industry. So it identified once [7:39] we created it, it identified all the potential  pursuits that are in our pipeline today using CRM, [7:46] right? and they identify all the contacts, wins  and losses. And then we brought in all this other [7:51] external information for other pursuits that are  in the industry that we were considering. Um, [7:56] government information that we were bringing  in, all types of other intelligence all went [8:00] into this intelligence layer. So now our sales  teams were allowed to be able to ask questions [8:06] across hundreds and thousands of records, right,  that no one could potentially do on their own or [8:11] in a dashboard. But with the intelligence layer  that we created and the agentic solution that [8:16] we created, it allowed them to be able to ask  questions about contacts, potential pursuits, [8:22] where they're located in minutes and in seconds.  And it was just dramatic dramatic shift as far [8:28] as how how we do things. So anyway, when  we're thinking about these initiatives, [8:33] um most of the time customers come to us, we we  do digital consulting and AI workflows for our [8:39] customers and the vast majority of the time the  data is not ready or they don't understand what [8:45] data correlates with what activity for their end  user. And so oftent times our initial job is just [8:51] helping them figure out how to gather multi  multimodal data such that they can begin to [8:56] consume it whether it's visual data textbased data  or it's actually the new data sets that they're [9:02] uh generating as a function of things like uh AI  large language models tokens uh decision making [9:10] justifications along the entire path and the  traces that exist within these new paradigms. And [9:16] what we've realized is that the new user interface  for our customers is not necessarily something on [9:22] the front end where you uh see something visually.  The new user interface is a CLI. The new user [9:28] interface is an SDK. The new user interface is  your uh APIs. And if those data sources and those [9:35] pieces of functionality within your stack don't  conform to the expectations of AI, what you end [9:42] up with is drift. You end up with hallucinations.  you end up with data that you cannot trust. And [9:47] in a situation of transient technology or a  transformation of technology, the ability to [9:54] trust the output is absolutely integral to being  able to make decisions within your business. [10:03] So to add on to that, so we're in the early stages  of creating an agent to assist our first year [10:08] students with uh course scheduling. So, students  that are showing up, uh, first college experience, [10:15] they've got a major picked out, but we're  running a system that, as my son likes to say, [10:20] is from the 1900s. So, it's pretty clunky.  They It's their first go around with it. [10:26] What what we're trying to figure out is based on  their academic record, based on their test scores, [10:30] based on their major, all these other factors,  how can we help them uh have the most optimal [10:38] schedule throughout the course of their career  at the institution in a way that they day one [10:44] they're ready to go. Um it's crossing the finish  line, I think was part of your question was like [10:49] how do we know when we're when we're there? Um  right now there's a lot of human intervention [10:53] and human review and human moderation to ensure  that the schedules that are being produced are [10:59] appropriate and in line for first year students.  Success what I would I really categorizes success [11:05] is when when those agents are producing these  schedules 100% of the time or or in the high 90s [11:10] for our students out of the gate on day one. Yeah.  And if I can look at it from a different angle, [11:17] we as a vendor when we implement AI, we have  100% of the employee using co-work for example. [11:25] Uh improving the productivity. Our initial goal  was to double the productivity either to have the [11:31] double amount of features in the same time or  being able to serve our customers with feature [11:36] request with in 50% of the time. And this year we  achieved that. We keep developing more and more [11:43] agent more and more skills. Every department is  working on marketing skills, presentation skills, [11:49] development skills that are being shared across  the company. And I think the result is amazing. [11:55] Excellent. So, so far we've gotten key things.  Strategy is very important. Trust is really [12:01] important as well. Usability. Who's using the  out product? Who's using the tools? And then [12:06] the last thing is governance. Um, that part  cannot be omitted. So I'm going to transition [12:11] to my third question to my panel. Legacy system  weren't built for modern AI workloads. So the big [12:18] question for you now is must what is the most  effective strategy for bridging outdated tech [12:24] with new AI capabilities without disrupting core  business flows and operations. So Billy, Josh, [12:33] and Jesse and you're welcome to join in. Danny,  well, I I'd love to start this one because I think [12:39] we've got a pretty clear perspective on this. If  your legacy system does not output events or you [12:46] cannot um qualify what the events your system  uh outputs, then you are not ready to begin to [12:53] layer intelligence on top of your systems. uh  gathering the events [snorts] making sure that [12:58] you understand the um time coding of those events  such that you can give AI the most up-to-date d [13:05] uh data that is not stale so that you're not  again making decisions on untrustworthy data [13:11] is absolutely integral to you being able  to create systems that not only leverage [13:17] this intelligence in the appropriate way for  your business but also create velocity and [13:22] efficiency within your organization. Because  each time one of these events is fired, it's [13:28] an activity that happened with your user. It's  an activity that happened within your internal [13:32] organization. And that activity is an opportunity  to intervene. I think the really salient uh point [13:39] and you forgive my Josh made just a minute ago  was that um you know they're they're focusing on [13:44] onboarding and onboarding for students creates a  huge amount of events that are required for them [13:52] to feel comfortable in those organizations and in  the situation environments that they're in. And [13:57] if you can capture those events from the tags that  are scanned in and doors to the schedule that they [14:04] have to their financial status to each and every  one of these elements, you can create a situation [14:10] for them to succeed as end users for your  institutions for your companies. But if you drop [14:17] any of those events, what you leave is a gap in  your knowledge. Uh it's uh it's basically like uh [14:23] I don't know if you're familiar with the concept  of sakad. It's basically when you look from one [14:27] place to the next, your mind can't interpolate  these things. You're losing some layer of data [14:32] that you uh have the opportunity to gather, but  you uh are not codifying that within a data system [14:38] within your organization. And so gathering those  events, understanding what happens at those events [14:44] so that you can sequence opportunities within  your uh intelligence systems that you want to [14:49] add to your organization is absolutely probably  one of the most important things that you can do [14:53] in the early phases of evaluating where you  are as an organization and how you'd like to [14:59] layer in intelligence to your organization. And  with regard to the gathering of those events, [15:04] um, our early use case is for for new pilot  initiatives. We we're providing readonly [15:09] access to replicated data because at worst then  you're producing hallucinations on on data but [15:14] but not writing to systems of record especially  with legacy systems where writing to systems of [15:20] record can create downstream problems that you  might may or may not uh foresee in the immediate [15:24] to mid midterm. There you do need to write to  systems of record. We're writing them to alternate [15:30] locations and then modifying our our extraction  workflows to to rely upon those other data sources [15:36] uh through middleware. There's never been a better  opportunity to modernize those ETL uh pipelines, [15:43] right? the [clears throat] we we found what was  it a few few weeks back that uh if you pass image [15:49] based data with text into a lot of these LLMs you  get more efficiency in your token usage primarily [15:54] because they were discounting the underlying cost  for OCR right and so that just tells me there's a [16:00] lot of opportunities and gaps to create arbitrage  not only within the spend that your organization [16:05] is outputting for creating intelligent systems  within your organization but also there's a ton [16:10] of opportunity and really recognizing what those  events are how to create standardization in your [16:16] data structures and then how to leverage that  standardization to create consistency throughout [16:21] the output and determination in the output of your  your LLMs or your AI integrations. Um AI should [16:30] adapt to your business, not the other way around.  And what we've seen a lot of times our case [16:43] We're gonna we're going to share because we're  buddies. Um, so thank you. So what what we've [16:49] seen and we got a lot of different trusted data  sources um in the construction industry and I'm [16:54] pretty sure a lot of you guys have that are  already embedded within your organization. [16:58] So it's about that cultural shift too as far as  how do you trade trust with your end users. And [17:04] so if you have that information already and you  can identify what those are that are trusted by [17:09] the business ingesting that information into some  type of intelligence layer in our case again I'll [17:15] reference Snowflake as long as it has an API  of some sort that you're able to collect that [17:20] information into a central location that's the key  to start to kind of figure out again going from [17:25] prototype to to product. How do you get the best  information out quickly? And then once you do that [17:32] then your end users are are like okay well oh wow  now if you can do that maybe you can do this as [17:39] well and then the behavior starts changing. So if  you got bad data sometimes and we do have bad data [17:44] um it incentivizes them to know the power that AI  provides now that you have better data now I can [17:51] go ahead and improve my business process to make  sure I'm putting in the right information as well. [17:56] And those are some of the things that we see a lot  of is that even though the information is there, [18:00] the data is not right. It's not accurate.  We're not putting it in correctly. So we [18:03] try to meet people where they're at. And one of  the examples, we built a job site intelligence [18:08] dashboard. So at job sites, we have all of these  trailers and we we have information like project [18:14] status updates that we try to visualize that  information for our end users in the trailer to [18:20] try to get a status of what the job sites are  doing. And when we started rolling this out, [18:24] we thought it was a one-sizefits-all. And it  wasn't. It was the same data, but it was a [18:29] different question depending on the PM, which is a  project manager or project director. So, we had to [18:34] meet those individuals where they're at rather  than trying to change their mindset about, no, [18:38] you're not using this the right way. This data is  wrong. It's more or less know how do we get them [18:43] to get the information that they're looking for,  providing that insight, and then they can see the [18:47] power behind it. And then culturally, now they're  asking, okay, how do we make this data better? how [18:52] do we make this business process better rather  than from technology folks the other way around [18:56] we're pushing trying to tell they to change their  process and we've seen that that doesn't work very [19:00] well I think one really interesting thing that  you mention is um basically new data sources [19:08] right like observing a job site with a camera is  not simply for the purposes of uh security at this [19:16] point right or for asset protection it's also for  process automation it's also a new data set that [19:22] should you should be rolling into your systems. If  you think about the systems of record for each one [19:27] of us up here, you know, we have agency systems  of record. We have GitHub as a system of record. [19:32] Uh you probably have some networking protocols as  systems of record, Blackboard, things like that. [19:37] You've got uh Procore and other types of systems  that can feed in master records into systems, [19:43] but they actually don't have the new data, which  is your traces in your uh Langfuse or Lang chain, [19:49] right? your ability to consume uh multimodal  data through images and parse those images [19:55] into things like hashbased databases so that you  can actually efficiently query them. And so if [20:00] you think consistently about each one of these  new data sets, what you have is an opportunity [20:05] to create more consistency within the output of  your systems, right? You get more intelligence [20:11] out of what you need to do and what the work to be  done is on a given day and the iterative loop of [20:18] what you can do with that information with a high  agency individual within your your organization. [20:24] Someone that feels as though that they can take  action with AI systems and access the appropriate [20:29] information basically becomes your superpower as  an organization. It gives you enablement because [20:35] individuals can take more activity in a shorter  amount of time and create more iterations to [20:40] create you a better product for your business.  Exceptional. Thank you for that. I think the [20:46] biggest takeaway is consistency is key and being  mindful of what the workflow looks like for every [20:52] single individuals in your organization. Setting  priorities, making sure that everyone has some [20:59] kind of key benefit. And there is also a feedback  look. So you're taking back the feedback of [21:04] whatever it is that they're confronting because I  think we don't talk enough about feedback because [21:09] there is an assumption that AI systems are just  based are build and people start using it but [21:15] we need to remember that feedback is a way to  create consistency for a lifetime. All right. [21:21] So you mentioned data and data has been coming up  a lot. Um, we know that AI is good at everything, [21:28] but keeping data requires a special skill set.  And when you think about the adoption process, [21:34] we're not only merging new data that we're  collecting, we're also trying to merge it into [21:40] our outdated data or even other data sets that  the organizations keep in as well. So the big [21:46] question for my panel now is how do you rethink  your data pipelines and your storage when it comes [21:53] to creating this enterprise ready AI capabilities?  What are you doing differently with your data [21:59] collection and then how are you managing legacy  data as well? I can start that if you like. Um um [22:08] I I think there's an expectation previously that a  significant portion of data that you operated your [22:14] business on was done in batches, right? Uh you  would have a morning import or you would have an [22:19] afternoon dump or something like that. Um I think  those expectations are narrowing and narrowing and [22:26] you can see this in the heristics of people.  Uh the way people want to get paid today is [22:30] uh what is it? Earned wage access, right? They  they want to get paid immediately and they don't [22:35] want to get paid in batches. they want to get  paid as they earn those things. But the same [22:39] thing applies to your data pipelines. You want to  be able to have a set of uh policies within your [22:45] organization that move data as close as it can to  real time as a function of how fast you can make [22:52] decisions today because agents in the future will  make decisions in micros secondsonds, right? uh as [22:58] models get more efficient, as they get lighter  weight, you'll have the opportunity to deploy [23:02] decision-making engines all the way to the edge  of your networks. Think in a construction site, [23:08] uh you know, we've got organizations like Flock  Safety kind of perpetuated throughout society, [23:13] but we also have opportunities within our  day-to-day CCTV systems to actually watch [23:19] and observe things and make take activity in  seconds, but we actually don't have the systems to [23:24] sync that information at the moment. And so oftent  times they sit in utilities at the edge where you [23:29] can bring them into the core like you mentioned  earlier with Snowflake or other decision- making [23:34] or other data aggregation tools and then  layer intelligence on top of it so you can [23:39] actually take action against this data. But if we  continue to think in this concept of batch data, [23:44] we'll continue to um perpetuate a pattern within  our organizations of asynchronous activity where [23:53] synchronous activity is one of the expectations  and requirements. I mean, every single day when [23:58] I'm writing code or I'm instructing my teams, the  expectations of how quick I can take action and [24:05] take activity within my organizations continue to  shrink. Even though we're getting more and more [24:10] and more output, we recently just like I'll put  like a marketing site and I think I built it in [24:15] like a couple hours off of a ton of markdown  documents and the distillation of a bunch of [24:20] information. And my marketing uh uh director  had never had that opportunity to see someone [24:27] take action that quick. But then, you know, a  week later he's like, "Hey, can we push 15 more [24:32] things?" Right? The expectation isn't, "Oh, we did  that fast. Now we've got opportunity to do other [24:38] things. It's can we pile into this activity and  can we exaggerate its potential outcomes? Um yeah, [24:47] I totally agree with Billy saying. So how many of  you guys have been using AI more than more than [24:51] three years? Show hands. And and do you guys  remember when when it first came out and you [24:57] were told that that your large language models  are your foundation and it's going to change [25:03] everything? And then you quickly found out that  no that's not true. Data is the foundation and it [25:09] always has and it always will be and and and the  LLM's AI exposes that data good or bad. But it [25:16] it is the key to all of this. And so whatever  solutions you have today or in the future, if [25:23] if you're looking to on board and bring that into  your organization, if it doesn't have an open API [25:28] or some kind of API to where you can do calls and  ingest that information, run because that's all [25:33] it is is just technical debt looking you straight  in the face, right? And so how do you continue to [25:40] kind of re-equip your environment or partner with  your existing vendors that haven't necessarily [25:46] given you that flexibility? But no matter what to  what Billy's point, you have to get access to that [25:51] data and you have to centralize it into some type  of layer. Doesn't matter what the platform is. It [25:56] can be data bricks, it can be snowflake, it can  be some other type of solution that centralizes [26:01] all that information that allows you to ask  intelligent, insightful questions into that [26:06] information to give you some insight about what's  possible. Then all the other ideas will continue [26:11] to come. Make sense? Yes. Thank you for that. It  scares me when people do not consume the data that [26:19] they're outputting into LLM today. You should be  logging every prompt, every assistant response, [26:25] every single tool call, every tool that your  underlying engineers or team is using because that [26:31] data set is a new data set that will help you make  decisions in the future. For instance, we're open [26:37] sourcing an application called Axel Insights over  the next week or so, and it logs off of our 500 [26:44] engineers in our team every single prompt, every  model, every token that's input into a system into [26:50] a set of hyper tables in time scale DB. And the  reason why that's important for us is that if [26:55] we were not able to observe the heristics of our  engineering teams, there's zero chance that we're [27:00] going to create efficiency for our organization.  Uh one of the things that we realized was just by [27:06] logging this information, we see engineers  using tokens to input commit messages and [27:13] uh push PRs and merge PRs. The reality is that's  like one or two commands in a terminal across [27:20] 500 engineers. The token spend to do that becomes  actually substantive. And if we were not observing [27:26] those patterns and bringing in every one of those  prompts, we would not know that that's one of the [27:31] kind of I wouldn't say abusive tactics, but one  of the things that is creating a little bit of [27:36] a laziness within engineering organizations.  And it's super important that we continue to [27:41] push to keep that data and not just give it to  all the hyperscalers. Yeah. No, you're right. [27:45] Because there was a research study that was done  by EY and he talked about the cost effectiveness [27:51] and the cost ineffectiveness of tokens. And  I think if you're not capturing the data, [27:56] understanding the partner, the patents and the  partnership that you need to maintain consistency, [28:02] you are going to lose money. And so I want to  direct the next question to S and give him a [28:07] chance to talk to us as well. What is the missing  functionality of legacy AI equipment compared to [28:14] the needs of AI clusters? Thank you. Right.  So when we look on the infrastructure itself, [28:22] there is a data center today in an organization  and there is an AI data center. So both of them [28:28] have the data center but the AI is a different  animal at all. Big difference is scale. Again the [28:35] need for that comes in order to optimize the AI.  As mentioned before you should not work for AI, AI [28:42] work for you. So enterprises would need not just  rely on LLM, develop their own model and train [28:48] their own model. In order to do that, you need  to build an AI cluster that you can train it. And [28:55] again looking at traditional data center does not  fit in any aspect. Not in the space, not in the [29:02] power, not in the cooling, not in the networking.  The scale is is huge. The enormous delta is 100 [29:10] 1,000 times bigger when you're talking about AI.  I think one AI data center with a few hundreds of [29:16] GPUs has the bandwidth of the entire city of Las  Vegas, right? And also from power point of view, [29:22] cooling, you need to do liquid cooling in order  to make sure your GPUs are functioning in order [29:28] to make sure your networking is functioning.  Um so in order to optimize that right you need [29:34] to actually make a big change it's not evolution  of the legacy IT equipment you need to design it [29:42] specifically for AI just rely again as you look at  it as mentioned before the legacy IT applications [29:51] are more human response time you send something  you get it within a second you're happy in AI it's [29:57] a different story you have thousands or hundreds  of GPUs doing the same calculation and the key [30:04] part is when you get the data from all of them  you need to wait for the last one and saving every [30:10] microscond counts. So uh in order to optimize that  you need to make sure not only you have the right [30:18] infrastructure not only you have the best GPUs  but you need to make sure that uh when you look [30:23] at it on a cluster level everything is optimized  and the network itself for example just having [30:29] bandwidth of 800 gigs from a GPU doesn't mean it's  enough because you have the 800 gig you have the [30:36] tunnel but you have many GPUs talking to to one  specific GPUs with a task what's called incast [30:43] then you have packet drops you have packet drops  it means all your infrastructure is not working [30:48] so when we look at it from spending point of view  in legacy IT equipment when you have a data center [30:55] the equipment the servers the communication is the  major cost when you look at AI the GPUs are the [31:01] major cost so if you invest in the right equipment  to make sure the GPUs are better utilized that's a [31:07] neglectable cost compared to the GPU cost so I  think this is what organization building their [31:13] own GPUs should be focused on. If we look at it  from an industry point of view and traditionally [31:20] hyperscaler were building their own GPU clusters  and neoclouds are doing that and enterprises [31:26] normally are renting and using cloud services.  However, I think the industry is expecting [31:31] enterprises to build more and more onrem clusters.  Specifically if we look on research institute we [31:38] look on financial institute they have sensitive  data that they cannot use in the cloud and they [31:45] need to do it on prem. So the vendors the big  vendors are working on offering a solution for [31:51] for such application like the Rex scale solution  the the NV72 from Nvidia or recently announced [31:58] Elios by AMD that it's a Rex solution with 72 GPUs  and you have everything in the rack and I think [32:05] they will go and offer more and more bigger racks  for enterprises to make the life of the enterprise [32:11] easier and simpler to implement AI. Yeah, I I  I can't uh impress upon anyone enough that it [32:20] it's an exciting opportunity when you see the es  and flows of going on prim versus cloud then back [32:27] on prim because what you're actually observing is  people beginning to understand the underlying core [32:33] technology as a as opposed to looking at it as a  black box where you throw prompts into. And when [32:39] you begin to do that, it's like understanding a  core Linux system or a kernel level understanding [32:45] of an environment. You begin to understand the  fundamentals of what creates value within your [32:51] business and organization and you begin to map  that to not only the financial functionality of [32:56] your business, but the actual technical capability  of your business. So being able to uh put [33:02] something or a utility next to an individual that  has high bandwidth needs for AI decisionmaking is [33:08] a superpower for not only that individual but the  people that are adjacent to them. It's only going [33:13] to exaggerate their capabilities and their ability  to be able to leverage these tools like Kimmy and [33:20] uh uh Quinn in ways that u we are just spending  piles of money into these black boxes that in [33:27] the end we don't get as much value as we'd  like to see out of it. So I think this is [33:32] an opportunity and a moment to really begin  to realize the ROI as it relates to AI. Yeah. [33:38] Thank you so much for all your contributions.  I kind of want to turn it over to the room and [33:42] give you guys a chance to ask your questions.  If you do have a question, raise your hand and [33:47] I'll have Ebony bring the mic. Any questions  in the room? Okay, we have one. All right. Uh, [33:54] so question for you guys. You'd mentioned GPUs  as a resource to building out enterprise data [34:00] centers. As enterprises start thinking about  owning their own and building out and weaning [34:06] off of public cloud and then eventually off of  neo clouds, can you talk a little bit about other [34:12] resource constraints or other things that they  should be taking into consideration moving forward [34:18] uh both from resources, budgeting, um personnel  expertise? Yeah, I'll I'll take the first part [34:26] of that question. So at KLCO, we know that that's  the new sovereignty frontier that we have clients [34:33] that have regulated, highly sensitive requirements  when it comes to building construction projects of [34:38] the future. And when we first started out, we had  these LLMs that we said that were enterprise and [34:43] they were separate. They won't be trained on any  information that could could become public data. [34:49] And and now our clients have moved away from that.  It's okay, I hear you, but I I don't believe you. [34:55] And so how do I come up with large language models  or small language models that are internal to [35:02] um to us that our clients will feel comfortable  with? So that next step is how do we build the [35:09] competency in building those things and getting  good at it so we can offer it as a value ad to [35:15] our customers who require that as as a must-have  in order to do business together. So those are the [35:21] things that we're thinking about as far as how  do we how do we create those? How do we how do [35:25] we stand up in our own environment and then start  to budget for that based on the clients we know [35:30] in our pipeline and based on our goals as well  that this is a priority that we want to pursue. [35:36] Is it revenue generating? Is it just margin margin  expansion? Is it operational efficiency or is it [35:42] just customer satisfaction? But it's always tied  back to those business outcomes that help drive [35:47] the next decision on investing in an environment  that's going to provide us that capability. [35:53] Yeah, as you think about data as well, like  if if I'm investing in a business, this is [35:58] not investing advice by the way, but uh that if  I'm investing in a business, I'm thinking about [36:02] databases. Uh like honestly, databases are only  going to grow. Uh I think in the last 12 months [36:08] and I I imagine this is 12 months in infinity from  this point forward continuing to roll forward. We [36:14] will create more text in a 12 month span this 12  months than we know the next 6 months and the next [36:20] uh uh 24 months over and over and over and over  again. And all that says to me is uh we need to [36:26] have places to store that information. And when  you think about on-prem or sovereignty in your [36:31] cloud environment, there's an opportunity. Storage  is cheap, right? Keep putting things in storage [36:37] and it's faster than it's ever been because it  gives you the opportunity to leverage that in [36:41] the future. We were at a large concrete company  in Florida that, you know, builds these giant like [36:47] uh what do you call these uh parking decks the  other day and they're thinking about the decision [36:52] on whether to put a significant investment in  GPU clusters and data storage on prim in their [36:58] environment primarily because their CAD files and  their pro project files generate massive amounts [37:04] of data and they need to be able to have low  latency and accessing that intelligence quickly, [37:10] right? and they're already making decisions  to move off of systems that do not give them [37:14] that latency, do not give them that  intelligence to systems that they can [37:18] layer that intelligence on themselves because  they open access to the underlying information [37:24] in a more palatable way. So, if if I'm someone  thinking about on prim hosting, grabbing hardware, [37:30] I think it's a great opportunity right now to  pick up one of these utilities and begin to [37:35] test with it, begin to store data, come up with  a disaster recovery policy for your AI related [37:41] data so that you know that you've got assurance  within your underlying hardware and that you [37:45] own that data in perpetuity. Yeah. Thank you.  Anybody wants to add before we move on? Okay, [37:52] any final questions? Yes, we have a couple. Um,  we'll take two questions and then if we can limit [37:59] the responses to about a minute, we'll wrap this  up and everyone will be ready to go. All right. [38:08] Uh, so traditional IT system been for decades  been very deterministic, right? So we knew the [38:16] output and we knew how the system going  to work and we had the full authority. [38:22] Now we are moving towards probabilistic  systems and we are delegating our authority [38:28] to a probabilistic system. Right? So in if we look  at overhauling what's the safest way to delegate [38:38] our authority to a probabilistic system uh that  won't have consequences that we can't tolerate. [38:50] I think uh the in my mind the question is how do  we safely transition to an environment in which [38:57] u we can trust the underlying probabilistic  decisions that a system that we are not [39:03] necessarily sure of its component parts actually  outputs. Um the the quickest way to that is [39:09] uh not thinking that engineering has gone away.  Um actually safe typing within uh um uh systems [39:16] is really important because that creates uh data  that you can actually trust and then aggregating [39:22] that data, the trusted data, your master records  in systems and data warehouses that you can [39:29] then pull that information off of and create the  correlations between that data that are required [39:34] to um understand the relationships between  your customers and the rest of your business. [39:40] um is the actual like core thing that everyone  should be working on. Shaping that data into [39:45] a system that anyone within your organization uh  depending on the authorization and authentication [39:51] protocols within your company um can access  and then leverage for the intelligence of the [39:56] work they do every single day. Thank you.  All right, we'll take one more over here. [40:10] So you all mentioned data quality is probably one  of the largest issues that people are facing right [40:15] now and in traditional machine learning we have  been fighting this since like 2012 and it's nice [40:22] that it's now a broader known topic and people  are willing to put time and energy into solving [40:26] this problem now. But one thing that's be coming  as the next natural step of this is the semantic [40:34] meaning of those metrics of that data. And it  almost requires you to have some kind of semantic [40:40] layer on top of your data layer. And so I was  curious how you guys are handling that because [40:44] there are key metrics, key terms, things that we  create in our companies that are not inherently [40:51] baked into those models. And so you need that  extra context about the data not just that the [40:56] data is clean itself. Yeah, that's a really  good question. So at least in our case when [41:02] we when we built these semantic views in and  snowflake it started out as we first ingested [41:08] the data into these tables into these columns  and then we created dynamic tables [snorts] and [41:12] on top of those dynamic tables we created these  semantic views which allowed a more elegant way [41:18] costefficient way for the agent to interact  with that and on top of that we created MCP [41:23] servers that created claw and and connectors and  and open AI ones if you wanted to have that but [41:29] That was the way and a game changer for us because  if you know when you give it a AI a lot of data in [41:37] table structured format it still hallucinates you  can you can tag the rows you can you can identify [41:44] ids on these columns and but if that data set gets  large enough you're going to get non-deterministic [41:50] information and it's just like h you know I got  to start all over again change for us was the [41:55] the semantic views and the dynamic tables it  it really boosted the confidence level of the [42:00] information that we're giving to be more accurate  um that the users were able to trust. Yeah, I mean [42:07] I think versioning schemas uh thinking about uh  managing schema and version control uh for your [42:15] underlying data systems is extremely important.  Um, I mean, you you remember the time in which we [42:21] put uh store procedures into every database and  locked ourselves into data structures that were [42:26] impossible to get out of as we accumulated 500  store procedures with 4,000 lines of code that [42:32] house all of our business logic. Um, the biggest  the easiest step to get away from that is put [42:38] the store procedures into version control first,  right? And if you don't have in version control, [42:42] there's no way you're actually going to be able to  mutate that data. The same applies to the schema [42:46] that you and the anttology that you uh create  within your new kind of systems of record or [42:51] your new uh data warehouses. And I think if  you can focus on the structure of the data, [42:56] you can see how open AI has created a protocol  or a structured data uh set that almost everyone [43:02] apply abides by such that things like uh LM  studio and llama and each one of these underlying [43:08] models knows how to consume that information. And  without the normalization of that information, the [43:14] understanding of the version of the structure of  the data and things like dynamic or static typing, [43:20] um you really don't have the opportunity to create  a clear understanding of what you should expect [43:26] the data from a system to output and then what  you expect for that data to from that system [43:30] to input into the next model that's going to  be making a decision for you. So I would just [43:36] encourage you to make sure that you really have  an understanding and grasp of the schema of your [43:40] data and you're not throwing unstructured data  into things like MongoDB and elastic and hoping [43:46] one day you're just going to get that JSON B uh  document out and all of a sudden you're going to [43:50] be able to mutate it and understand what's  in that system. Well ladies and gentlemen, [43:55] thank you all for your questions. Feel free to  reach out to our panelists directly with your [44:00] question. Let's give them a round of applause. You  guys were exceptional. Thank you for your time.