Enterprises Shift to Open-Weight AI as Frontier Costs, IP Risks Mount

前沿模型成本与知识产权风险攀升，企业转向开放权重AI 飙升的推理成本以及基础模型实验室在垂直领域的竞争，正推动企业放弃单一前沿模型，转向基于自主可控开放权重模型的多智能体路由。 专家组探讨了在医疗健康、生物技术、金融和能源等强监管垂直领域，从实验性LLM沙盒向生产级自主智能体部署的技术与运营转变。嘉宾们重点指出了关键的基础设施瓶颈，包括传统数据库在智能体高频交互查询模式下不堪重负、临床数据中PII泄露风险，以及前沿模型推理成本的失控。为应对这些难题，发言者主张采用自主物理隔离数据中心、治理优先的控制平面、开放权重小模型与前沿推理模型之间的多智能体路由，以及专用的垂直领域智能体运行套件。最终，他们预计企业AI将在GPU持续短缺的背景下，依托数十亿美元的资产负债表级投入，从任务自动化迈向深度的业务优化。 企业 AI 的瓶颈已从算法突破转向基础设施与治理架构，在医疗、金融等强监管垂直领域安全部署自主 Agent 成为核心挑战。 (0:04) 自主 Agent 具备高频调用的“chatty”特征，其持续而密集的程序化查询极易压垮为人类操作节奏设计的传统数据库与数据管道。 (3:37) 针对包含敏感患者信息的临床试验场景，架构设计需将数据治理与策略控制置于控制平面的最顶层，防止 Agent 抓取错位的 PII 导致违反 HIPAA 合规要求。 (10:44) Agent 工作流直接与底层数据交互且需要即时状态同步，正在推动基础设施重心从传统的隔夜批处理数仓转向超低延迟的实时流式数据引擎。 (15:55) 面对指数级增长的 Token 成本，企业正采用多 Agent 路由架构，将边界明确的常规子任务分流至开源/开放权重模型，仅保留前沿大模型处理复杂认知。 (36:32)

Deploying autonomous AI agents at enterprise scale requires fundamentally re-engineering data, compute, and governance architectures rather than relying on raw model capability. Panelists across venture, cloud, and life sciences argue that high-frequency agent queries, intellectual property liabilities, and inference economics are forcing regulated industries toward sovereign infrastructure, real-time data engines, and tiered model routing. Governance Inversion and Sovereign Infrastructure (1:20) In regulated environments like healthcare and clinical trials, standard application-driven security models fail because autonomous agents continuously issue programmatic queries that can traverse misfiled protected health information (PHI). Mark McNeill of OneSource Cloud notes that after initial penetration tests failed due to misfiled data slipping into outputs, the platform had to invert its architecture. They elevated external data governance controllers above compute and platform layers to act as primary gatekeepers for authorization, audit trails, and credentialing before workloads reach GPU clusters. Data Infrastructure Strains and Defensibility (3:37) Legacy data systems tailored for human query rates or nightly batch ETL runs—such as traditional Snowflake warehouses—collapse under the continuous, 'chatty' query loads of autonomous agents. Ilya Kirnos of SignalFire emphasizes that agents demand real-time state synchronization, reducing traditional Retrieval-Augmented Generation (RAG) to a specialized workflow rather than an end-to-end architecture. In biotechnology, Hok Hei Tam of Montai Therapeutics stresses that agent data manipulation requires verifiable provenance to raw laboratory readouts to satisfy FDA audits and protect high-margin patent monopolies. Inference Economics and Multi-Agent Routing (31:29) Unmanaged agent usage can cause enterprise inference costs to surge unsustainably. David Casem recounts how revised terms nearly caused Telnyx's monthly bill to scale from $80,000 to roughly $3 million; the company mitigated this by shifting bounded tasks to high-performance open-weight models (such as DeepSeek and Kimi) alongside specialized inference hosts like Together AI and Fireworks AI. Kirnos describes an industry-wide transition where complex agent architectures decompose workflows: expensive frontier tokens are reserved for core cognitive orchestration, while cheaper non-frontier tokens handle bounded sub-agent actions like search, extraction, and SQL querying. Open Weights Versus Proprietary Models (39:57) Enterprise adoption of open weights is uneven. McNeill reports that enterprise clients in energy, finance, and healthcare still routinely select closed, domain-specialized models despite free open-weight availability on OneSource Cloud's infrastructure, preferring turnkey regulatory compliance. However, foundation model providers expanding directly into vertical industries like drug discovery have introduced severe counterparty risk. This risk is driving institutional biopharma entities like Flagship Pioneering to fine-tune self-hosted open-weight models to ensure strict data sovereignty. Enterprise ROI and Sector Trajectories (42:44) While robotics faces near-term market vulnerability due to high valuations, lack of production deployments, and the absence of a standardized physical training commons, software agents are demonstrating tangible returns. McNeill highlights energy operators as unexpected leaders in AI monetization, using exploration insights to develop green initiatives and local infrastructure. Tam and Lu Zhang observe that as coding agents compound software engineering efficiency, enterprise adoption is maturing from task automation toward full business optimization, with Global 1000 CTOs committing dedicated multi-billion-dollar balance-sheet budgets to AI infrastructure. A persistent tension remains between enterprise demand for turnkey compliance—which currently favors proprietary models—and the mounting counterparty risk of model vendors entering vertical enterprise domains, leaving open-weight sovereign governance as a critical, evolving frontier.

Fusion Fund创始合伙人张璐与业界专家指出，企业级自主智能体落地的核心瓶颈已从基础模型算法转向底层基础设施。在金融、医疗等强监管领域，企业必须针对数据治理、实时流水线、推理成本控制以及数据主权进行全面的架构重构，才能实现从实验沙盒向生产级规模化部署的平稳跨越。 治理架构前置与强合规数据隔离 (10:44) OneSource Cloud副总裁Mark McNeill指出，当自主智能体直接接入受保护健康信息（PHI）等受监管数据时，传统的应用驱动型静态权限控制迅速失效。在实际医疗数据集中，患者身份信息（PII）经常误入非标数据列，自主智能体在进行全库扫描检索时极易触发HIPAA合规违规，导致常规渗透测试失败。为此，基础设施必须推行架构倒置：将数据治理与合规策略控制层提升至控制平面的最顶层作为核心网关，全权接管鉴权、隔离与审计日志，而GPU计算平台则退至侧边，专注于算力监控与能效调度。 Montai Therapeutics的谭学曦（Hok Hei Tam）同样强调，在生物制药等强监管流程中，必须建立涵盖实验溯源与强制人工签署的“人类基础设施”。由于医药企业核心资产是具有长期专利壁垒的药物分子，所有由智能体自动化执行的质控与数据处理步骤，都必须保持对原始实验室数据的端到端溯源链路，以满足FDA等监管机构严苛的合规审查。 高频查询冲击与实时数据底座转型 (3:37) SignalFire联合创始人Ilya Kirnos指出，传统企业数据底座在应对人类查询时已显脆弱，而自主智能体具有固有的高频程序化交互特性，其持续、密集发起的机器查询会直接击垮脆弱的数据管道。随着检索增强生成（RAG）逐渐演变为特定复杂检索的辅助组件，企业数据的直接消费者正从查看UI界面的人类分析师转变为直接调用底层库的自主智能体。 这种交互范式的转移彻底打破了传统的批处理模式。以往依靠Snowflake等数据仓库进行隔夜ETL批量更新的机制足以应付商业智能仪表板，但智能体在代替人类执行动作时需要毫秒级的即时状态感知，数据延迟将直接引发工作流瘫痪。同时，Telnyx首席执行官David Casem指出，智能体规模化运行还需要配备涵盖电话、短信、WhatsApp等数十种真实世界物理通信原语，且企业级协同必须脱离Slack等非正式聊天工具，建立专用的智能体间编排协议。 推理成本危机与双层模型路由架构 (31:29) David Casem透露，Telnyx在扩大内部智能体规模时，前沿模型提供商调整条款曾导致其推理支出面临从每月8万美元骤增至每日10万美元的巨大压力。为确保商业可持续性，Telnyx内部每日约300亿Token的推理负荷转向了开源及权重开放模型（如DeepSeek、MiniMax、Kimi K3）以及专业推理平台（Together AI、Fireworks AI）；若同等工作量全部运行在Claude Opus上，日均成本将高达150万美元。 Ilya Kirnos将主流工程方案概括为双层Token分流体系：在概念验证阶段，团队优先调用顶尖前沿模型验证能力；一旦进入规模化生产，则必须拆解任务架构。工程团队通过模型路由器将庞杂任务解耦为多个垂直子智能体，由低成本的开源模型承担信息提取、搜索或SQL生成等边界清晰的任务，仅在复杂高阶推理节点调用昂贵的前沿模型。此外，专家们一致告诫企业避免开发庞大统摄的通用单一聊天机器人，因为上下文窗口过载会严重削弱决策精度，专用的垂直业务智能体才是落地正解。 专有数据壁垒与前沿厂商纵向竞争 (21:24) 针对初创企业的护城河问题，业内观点出现明显分化。David Casem提出了激进的反共识判断，认为模型蒸馏技术使后发者能极低成本复制数据集，纯粹的数据壁垒在2026至2027年将不复存在，团队唯一的护城河在于避开官僚流程快速执行。然而，Ilya Kirnos以法律科技公司EvenUp为例指出，通过构建聚合多家律所匿名诉讼数据与医疗账单的“数据联盟”（Data Co-op），可以解决长尾案情定价的冷启动难题并形成网络飞轮，专有数据飞轮依然构成无法被平台轻易商品化的防御壁垒。 与此同时，前沿基础模型厂商向应用层纵向渗透的趋势引发了高价值资产行业的深层焦虑。谭学曦与张璐指出，生物医药企业极度担忧专有分子结构和查询记录被商业API服务商在底层留存或用于模型学习。这种信任壁垒促使Flagship Pioneering等顶尖机构与大型药企开始大力投入自研机器学习团队，转向对开源模型进行本地化适配与深度微调。在商业化模式上，生物医药初创团队也倾向于直接授权候选药物资产而非售卖SaaS工具，以契合药企数十亿美元规模的传统交易框架。 商业化落地分化与长期ROI路径 (42:44) 在资本市场高度审视AI投资回报率的背景下，各垂直领域的商业化速度呈现出显著分化。Mark McNeill指出，能源行业在AI基础设施上的变现速度出人意料地超越了医疗和金融；中东及北非的能源运营商不仅将AI用于油气勘探，更将其与绿色能源、碳中和倡议及本地就业深度绑定，直接贡献于区域经济增长。与此相对，具身智能与机器人领域被视为短期市场风险最高的板块，核心症结在于缺乏类似互联网量级的物理交互训练数据，且实际工业部署量极低。 关于企业级ROI的显现周期，谭学曦认为编码智能体在过去半年的成熟是关键先行指标，其工程效能正加速向工业和科研领域渗透。张璐总结指出，企业AI落地正遵循从“业务自动化”（任务替代）迈向“业务优化”（驱动营收增长）的演进轨迹；根据Fusion Fund涵盖全球1000强CTO的网络数据显示，非科技巨头的AI专项预算正普遍跃升至十亿美元级别，表明企业核心预算向生产级智能体环境的倾斜正在实质性发生。 当前物理数据中心建设与算力供给短缺仍制约着智能体的扩展节奏，且大型受监管企业在生产环境中对开源治理工具的成熟度仍持审慎态度。然而，企业级AI的竞争重心已彻底告别基准测试跑分，转向了以合规门槛、动态治理、推理成本分流以及实时数据流水线为核心的底层工程落地竞赛。

## Background 该视频由科技峰会频道 Ai4 于 2026 年 8 月 21 日发布在 YouTube 平台。页面简介显示，本场圆桌汇集了来自 Fusion Fund、Telnyx、SignalFire、OneSource Cloud 和 Montai Therapeutics 的顶尖投资人与技术高管，重点探讨如何为企业级自主智能体（Agentic AI）设计涵盖算力、数据存储、治理隔离与编排协作的底层基础设施技术栈。 ## TL;DR 圆桌嘉宾深入探讨了 AI 智能体从前沿探索迈向金融、医疗及能源等强监管行业时面临的基础设施重塑挑战。面对高频自主查询对老旧系统的冲击，企业必须将数据治理与合规审计提权至架构顶层，并建立完备的人在回路（Human-in-the-loop）追溯机制。针对高昂的推理账单与专有数据泄露隐忧，行业正加速从全能单一模型转向由模型路由驱动的专用 Agent 体系，结合开源模型实现大幅降本增效。尽管具身智能等局部赛道受制于物理数据短缺而引发泡沫争议，但能源与生物医药等领域的实战成效已推动跨国企业 AI 预算跨入十亿美元时代，产业正从业务自动化稳步迈向业务优化。 ## Atmosphere & style 整场圆桌对谈务实精悍且直击工程痛点，褪去了早期 AI 行业峰会的概念狂热，洋溢着一线技术操盘手与硬科技投资人碰撞出的硬核思辨。嘉宾们毫不避讳地拆解了数百万美元账单冲击、渗透测试越权失控等真实的落地“踩坑”教训，言辞犀利幽默且充满洞见。在主持人对监管合规、推理成本优化与华尔街 ROI 拷问的层层推进下，现场展现出一种既保持冷静清醒、又对企业级 Agent 颠覆性未来充满笃定的务实张力。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=4s) AI 行业正经历从单一对话语言模型向各行业 Agentic AI（智能体 AI）的关键范式转移，金融、医疗、能源等强监管行业对底层基础设施的重构提出了紧迫需求。 - [01:07](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=67s) 在强监管行业部署主权 AI 和私有 AI 时，物理隔离（Air-gapped）环境与严格的安全护栏（Guardrails）是防范智能体越权访问敏感数据（如 HIPAA 医疗档案）的核心前提。 - [03:37](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=217s) AI 智能体的高频、繁冗查询（chatty queries）特性对底层数据基础设施构成了严峻考验，会瞬间放大并击垮传统人类交互下勉强维持的架构缺陷。 - [06:45](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=405s) 智能体正从调用物理交互原语（电话、短信、邮件等）演进到原生 Agent-to-Agent (A2A) 通信协议，旨在摆脱依赖人类即时通讯工具（如 Slack）的落后交互现状。 - [10:44](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=644s) 面对自主运行的 AI Agent，传统应用驱动的技术架构极易在渗透测试中崩溃，必须将数据治理与访问控制层提权至整个架构最顶层（Top of Stack）实施统一调度与审计。 - [14:34](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=874s) 在强监管行业中，完备的数据治理与合规能力不再是研发枷锁，反而能够消除客户对数据泄露与监管重罚的后顾之忧，直接转化为初创企业的核心竞争壁垒。 - [15:22](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=922s) 传统“数据湖+数仓”的批量更新模式因高延迟无法支撑依赖全局最新状态决策的 Agent，推动基础设施向支持超高频读写的实时数据系统演进。 - [21:24](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1284s) 单纯的技术单点差距极易被巨头追平，通过“数据合作社”（Data Co-op）等机制汇聚专有行业数据、沉淀网络效应飞轮，是应用层企业建立可防御护城河的关键途径。 - [24:18](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1458s) 在制药等涉及核心专利与高额排他权益的领域，“数据孤岛”是保障合规的必要屏障，同时必须建立严密的“人在回路”（Human-in-the-loop）全流程审计追踪链条。 - [27:04](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1624s) 向制药巨头销售纯 Agentic SaaS 软件阻力重重，利用 AI 加速研发并直接向药企交付经过验证的“候选药物资产”（Drug Candidates），能精准切入数十亿美元的商业化甜区。 - [29:10](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1750s) 随着模型蒸馏普及与大厂跨界竞争，单纯通用数据壁垒快速贬值，大企业的繁琐合规流程反成小团队的不对称优势，初创公司应凭借极速执行力压制对手。 - [31:29](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1889s) 前沿模型高昂的 Token 账单迫使企业引入模型路由器（Model Router）和任务拆分策略，将大量非核心子任务路由至高性价比的开源/开源权重模型以控制推理成本。 - [33:55](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2035s) 针对特定业务功能与 API 端点封装上下文环境的“垂直专用 Agent”，在实际落地效果和上下文窗口利用效率上远优于通用的全能型公司问答机器人。 - [41:30](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2490s) 对专有分子数据外泄及闭源模型厂商跨界竞争的顾虑，促使头部生物医药孵化机构和大型药企大力布局开源模型的科研化微调，以建立自主可控的科研 Agent。 - [43:14](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2594s) 能源行业在 AI 基础设施投资的商业化变现速度上表现抢眼，通过将传统勘探分析与绿色碳中和转型相结合，实现了显著的直接盈利与经济拉动。 - [44:57](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2697s) AI 泡沫呈现局部细分赛道分化，具身智能与机器人领域因极度缺乏互联网级的物理世界交互训练数据，若短期内无法实现落地或将面临估值挤压；算力短缺在宏观层面仍将持续。 - [46:07](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2767s) 全球跨国企业的 AI 预算已攀升至十亿美元级，企业引入 AI 正在从第一阶段的“业务自动化”（降本）跨越至第二阶段的“业务优化”（重塑决策并驱动真正营收增长）。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=4s) 开场介绍与行业范式转移：从语言模型走向企业级智能体 圆桌主持人张璐（Lu Zhang，Fusion Fund 创始人兼管理合伙人）首先开场。她介绍道，Fusion Fund 在过去 11 年间一直深耕企业级 AI、医疗 AI 以及前沿硬科技领域的投资，经历了从早年被称为“机器学习”（Machine Learning）至今的完整周期。近两年来，整个 AI 产业正在以令人惊叹的速度演进——仅仅在过去的一周里，开源社区就发布了大量重磅成果，关于物理智能体（Physical Agent）与具身智能的讨论不仅席卷了科技界，更成为了各行各业领军人物关注的焦点。 张璐指出，当下的行业叙事正在发生关键转变：AI 不再局限于科技行业内部对大型语言模型的自娱自乐，而是演进为面向全行业的 Agentic AI（智能体 AI）。金融、保险、医疗等高度受监管的行业占据了美国国内生产总值（GDP）的巨大比重，这些行业亟需解决基础设施层的核心难题，才能支撑 AI 的大规模落地部署。本次圆桌正是要深入探讨如何通过重构底层基础设施，为企业级智能体应用铺平道路。 ```mermaid flowchart TD accTitle: AI Paradigm Shift to Agentic Workloads subgraph S1["传统阶段 (Traditional AI / LLM)"] A["机器学习与通用语言模型"] --> B["科技行业内部探索与独立对话应用"] end subgraph S2["智能体落地阶段 (Enterprise Agentic AI)"] C["高度自治的 Agentic AI"] --> D["深度融入金融、医疗、能源等强监管行业"] D --> E["底层重构：算力、物理通信、数据治理与隔离护栏"] end S1 ==> S2 ``` --- ### [01:07](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=67s) Mark McNeill：主权 AI、物理隔离与高监管行业的安全护栏 OneSource Cloud Corporation 的全球销售副总裁 Mark McNeill 率先分享了他们在基础设施层面的探索。OneSource Cloud 是一家专注于为客户提供私有 AI（Private AI）及主权 AI（Sovereign AI）开发与部署的基础设施提供商，总部位于达拉斯-沃斯堡大都会区，在敏捷调度下于全球 55 个国家和地区运营着 96 个数据中心。他们根据客户的具体合规要求提供高度定制化的交付模式：既可以在其自有数据中心内构建完全物理隔离（Air-gapped）的容器化环境，也可以直接在客户内网防火墙之后完成本地化部署，并通过安全通道实施远程运维。 ```mermaid flowchart LR accTitle: Sovereign AI Deployment Options subgraph Local["客户侧本地部署"] A["客户内网防火墙"] --- B["本地私有基础设施"] C["远程集中运维管控"] -.-> B end subgraph DataCenter["OneSource Cloud 数据中心"] D["全球 96 个数据中心"] --- E["物理隔离 (Air-Gapped) 环境"] end ``` Mark 提到，公司的早期客户主要集中在对数据安全极度严苛的行业，例如医疗健康、金融服务、政府机构，以及出人意料成为主要需求方之一的能源行业。过去几年里，客户大多处于在隔离环境中训练模型、搭建内部应用的研发阶段。然而，随着这批客户正式将产品推向市场并全面引入 Agentic AI，安全风险急剧升级。当自主智能体被赋予直接访问高度敏感、受严密法律监管的数据（如患者健康医疗档案）的权限时，基础设施必须构筑严格的“安全护栏”（Guardrails），以杜绝任何因智能体越权调取数据而引发的 HIPAA 等合规与隐私灾难。 - **OneSource Cloud 基础设施规模**：在 55 个国家运营 96 个数据中心。 - **目标强监管行业**：医疗保健、金融服务、政府机构以及能源行业。 - **核心风险场景**：智能体访问敏感患者信息（PHI）时的越权与 HIPAA 违规风险。 --- ### [03:37](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=217s) Ilya Kirnos：智能体高频交互对数据基础设施的严峻考验 SignalFire 联合创始人兼首席技术官 Ilya Kirnos 从工程与投资的双重视角进行了剖析。Ilya 拥有计算机科学与数学背景，早期在 Oracle 数据库团队担任软件工程师（他幽默地调侃道：“如今 Oracle 更出名的身份似乎是数据中心公司，但在当时它只是一家纯粹的数据库公司”）。2003 年他加入 Google 并工作了 9 年，历任 AdWords 底层架构、Gmail 以及 Gmail 商业化变现等核心业务线。离开 Google 后，他经历了一次初创公司被 Twitter 收购的历程，随后联合创办了 SignalFire。目前 SignalFire 管理资产规模约 30 亿美元，正通过一期约 10 亿美元的基金重点布局 Seed 轮到 Series B、C 轮的 AI 创新企业。与此同时，SignalFire 内部自建了专属的机器学习与工程团队，亲自下场构建产品工具。 结合第一线工程经验与被投企业的客户反馈，Ilya 强调**数据基础设施（Data Infrastructure）**正面临前所未有的压力。以往面向人类用户的数据系统即便设计不够完善，往往还能勉强维持运转；但引入 AI 智能体后，由于其交互行为极其高频、繁冗（"AI agents tend to be pretty chatty, issue lots of queries"），会瞬间产生海量查询请求。这导致原本在人类负载下只是“勉强支撑”的基础架构，在接入智能体后会立刻崩溃，从而将底层的架构缺陷成倍放大。 --- ### [05:24](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=324s) Hok Hei Tam：AI 驱动药物研发与数据治理的零幻觉底线 Montai Therapeutics 首席技术官兼 Flagship Pioneering 科学合伙人 Hok Hei Tam 探讨了智能体在生物医药前沿的应用。Flagship Pioneering 作为专注于生命科学领域的孵化先锋，曾孵化出 Moderna 等知名企业。Montai 致力于利用小分子技术将发现的化合物转化为口服药物，用于治疗溃疡性结肠炎（Ulcerative Colitis）等慢性自身免疫性疾病，并计划近期在澳大利亚正式启动人体临床试验。 在制药研发流程中，虽然 Montai 长期利用经典机器学习来加速药物筛选，但当前正在全面引入 AI 智能体，以帮助科学家更快速、精准地发现候选药物。Hok Hei 指出，药物研发的核心资产就是海量实验与分子数据。引入智能体协助科学家进行复杂因果推理的同时，必须在数据治理与底层管道上确立严苛标准，确保智能体处理数据的绝对准确性，坚决防止智能体产生数据幻觉（Hallucination）或生成错误的科学假设。 --- ### [06:45](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=405s) David Casem：物理交互基元、GPU 算力短缺与 A2A 通信协议 Telnyx 联合创始人兼首席执行官 David Casem 阐述了为实时 AI 智能体（Real-Time AI Agents）提供底层基础设施的实践。Telnyx 的平台涵盖了智能体赖以生存的完整生命周期底座——从计算、存储、推理，延伸至智能体与物理世界交互所需的各种通道。通过其提供的各类原语（Primitives），智能体能够直接拨打电话、发送 WhatsApp 信息、短信以及电子邮件。目前该平台已上线多达 44 种基础交互原语，用户甚至可以让自己的智能体直接访问其官网，由智能体全自动完成产品了解并注册开户。 在落地痛点方面，David 指出当前面临着 GPU 算力获取困难、算力集群交付部署周期长以及部分地区的出口管制等硬件挑战。在软件与协作层面，Telnyx 内部已在业务中实际部署了大量智能体，当前的下一代攻坚目标是打通高效的“智能体对智能体通信”（Agent-to-Agent，简称 A2A）。他笑称，目前人们和智能体交流、甚至智能体之间互相交流，竟然还得依赖 Slack 这种人类聊天工具，Telnyx 正致力于彻底改变这种落后的交互现状。 ```mermaid flowchart TD accTitle: Telnyx Agent Infrastructure and Communication subgraph AgentCore["智能体核心底座 (Telnyx)"] Compute["计算 / 存储"] Inference["实时推理引擎"] end subgraph Primitives["物理世界交互原语 (44+ Primitives)"] Voice["电话语音"] SMS["短信"] WA["WhatsApp"] Email["电子邮件"] end subgraph Interaction["通信演进目标"] Legacy["现状：依赖 Slack 进行人机 / 代理通信"] A2A["目标：原生 Agent-to-Agent (A2A) 协议"] end AgentCore --> Primitives AgentCore -.-> Interaction ``` --- ### [07:44](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=464s) 主持人梳理三大维度：安全治理、私有化小模型与算力能耗效率 张璐随后总结了各位嘉宾发言中的核心关键词，并提炼出企业级 AI 与消费级 AI 的三大本质差异： 1. **AI 治理与低幻觉容忍度（AI Governance & Zero Tolerance for Hallucinations）**：不同于面向 C 端的 ChatGPT 对偶尔的错误回答具备较高包容度，企业级应用对幻觉的容忍度极低。在严苛的行业标准下，上线前的合规治理层是绝对的前置条件。 2. **小语言模型与私有化部署（SLMs on Private Networks）**：出于数据合规和隐私考虑，大多数强监管企业不愿将全部数据上传至公有云。通过在私有网络上部署小语言模型（SLMs），既满足了数据隔离要求，又倒逼网络与算力基础设施提供专项支持。 3. **成本与能耗效率（Compute & Energy Efficiency）**：当前的 AI 竞赛已不仅是单纯的模型性能比拼，更是推理成本、计算消耗与能耗效率的综合较量。只有大幅压降单次运行成本，才能实现真正的大规模生产级部署，这也是开源生态在产业界愈发受到推崇的根本原因。 --- ### [08:43](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=523s) 医疗临床试验案例：从离线研发到跨机构智能体的数据安全攻坚 张璐向 Mark McNeill 进一步提问：高监管行业占据美国 GDP 的 40% 以上，拥有最庞大的高质量数据，具备孕育最佳 AI 用例的土壤，但在数据层和基础设施层却尚未为 AI 大规模落地做好准备。她邀请 Mark 结合实际案例，拆解如何在强监管领域重构架构并建立治理层。 Mark 详细分享了一家部署在其全球多地物理隔离数据中心内的客户案例。该客户是一家为医疗生态提供软件支撑的研发企业，其团队并不直接提供诊疗服务，对 HIPAA 细节规范也知之甚少。他们开发的核心技术与智能体平台，旨在帮助医疗机构、跨国制药企业以及耐用医疗设备（DME）制造商加速推进食品药品监督管理局（FDA）的临床试验审批流程。 在实际业务中，该系统构建的智能体需要调取并解析来自多中心临床试验的海量真实患者病历数据，涵盖“患者表现出的具体症状、所接受的实验药物或安慰剂对照治疗方案，以及最终的病情转归结果”。在早期的研发测试阶段（长达两年时间），OneSource Cloud 提供的物理隔离架构、基于角色的凭证权限体系以及数据科学家模型拉取鉴权运行良好。然而，随着该产品正式推向商业化市场，基础设施必须同时承载这套系统与其终端客户（各大医疗服务机构），智能体跨组织访问真实患者敏感数据的安全风险瞬间呈现出全新的复杂度。 ### [10:44](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=644s) 医疗合规与数据治理：Agent 时代的架构倒置与控制重构 在处理高度敏感的医疗与临床试验数据时，企业面临着极其严苛的 HIPAA（健康保险可携性与责任法案）合规与隐私保护红线。Mark McNeill 指出，许多人直觉上认为只要简单剥离患者身份标识信息（PII/PHI）即可，但在真实复杂的业务场景中这一设想往往会破灭——海量数据中常有大量标识信息被错误地录入到非标准字段或混乱的列中。一旦自主运行的 AI Agent 在分析这些海量数据集时将这些残留敏感信息抓取并写入最终的对外报告或公开文档，就会引发灾难性的违规风险。由于 OneSource Cloud 自身的定位是专注于算力规格配置、GPU 集群设计与部署运维，“做好自己的本职工作（stick to our knitting）”，既非网络安全公司也不做数据预处理与治理，因此他们决定引入专业的数据治理合作伙伴。在评估了多家厂商后，他们选定了一家在临床试验领域拥有深厚经验、具备成熟授权认证机制、细粒度数据访问策略以及完备审计日志（audit trails）追踪能力的合规治理平台。 然而，在将该治理平台集成进既有架构时，团队遭遇了严重的架构冲突。原有的技术架构是完全“应用驱动”的：数据科学家登录平台后自行挑选 GPU 算力与存储组件，随后系统直接放行他们访问底层基础设施与数据集。但这套逻辑在面对“在环境中肆意穿梭运行（running wild）”的 AI Agent 时彻底崩溃——在随后的渗透测试（penetration testing）中，系统遭遇了惨败，Agent 频频越界抓取不该访问的数据并生成报告。为了彻底根治这一漏洞，团队在其合作伙伴的建议下完成了颠覆性的架构重构：将数据治理应用层直接提权至整个控制菜单的最顶层，由治理层接管所有对底层基础设施与数据集的访问调度指令；而原有的管理平台则被“退居侧翼”，仅专门负责 GPU 利用率监控、集群算力优化及机房温控等底层物理指标。这种架构重构虽然颠覆了以往的运维习惯，却成功阻断了 Agent 的违规数据外泄，成为项目在客户临床试验中得以顺利落地的决定性支柱。 ```mermaid flowchart TD accTitle: AI Agent 基础设施的控制架构重构 accDescription: 展示从传统的应用驱动架构向数据治理优先的控制架构演进过程 subgraph Legacy["传统应用驱动架构（测试失败）"] L_User["数据科学家 / AI Agent"] --> L_App["应用管理平台 (直接控制)"] L_App --> L_Data["敏感临床数据集 (HIPAA 泄露风险)"] L_App --> L_GPU["底层 GPU / 存储资源"] end subgraph Refactored["重构后治理优先架构（成功落地）"] R_Agent["自主运行 AI Agent"] --> R_Gov["数据治理控制层 (Top of Stack)<br/>• 身份授权与凭证管理<br/>• 访问策略与阻断引擎<br/>• 行为日志与全流程审计"] R_Gov --> R_SecureData["合规过滤后的安全数据流"] R_Gov --> R_Compute["经审计授权的算力调度"] R_Mon["平台监控层 (Sidecar/侧翼)<br/>• 算力利用率优化<br/>• 硬件健康与温度监控"] -.-> R_Compute end ``` - **数据错误录入风险**：敏感标识信息经常散落在非标准列中，传统静态脱敏脚本极易遗漏，导致自主 Agent 误采。 - **渗透测试红线**：原有架构在 Agent 渗透测试中暴露出大量越权读取数据的红线，迫使团队彻底重构权限控制流。 - **职责解耦**：治理平台全权接管策略阻断、授权、审计与访问入口，底层云平台退守纯硬件维度的性能监控与利用率调优。 --- ### [14:34](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=874s) 投资人视角：AI 治理从“合规枷锁”转向“竞争壁垒” 主持人 Lu Zhang 对 Mark 分享的临床试验案例表示高度认可，并由此引申出对 AI 治理（AI Governance）认知维度的探讨。在传统的行业认知中，治理往往与繁琐的监管、苛刻的限制以及对研发速度的束缚联系在一起；然而创新从来无法在纯粹的限制中繁荣，创新需要正向激励。Mark 团队的实践恰恰证明：在金融、医疗等强监管行业（highly regulated industries），完备的 AI 数据治理与安全管控能力不再是绊脚石，反而能够直接转化为初创企业的核心竞争优势（competitive advantages）。 对于身处强监管赛道的创业者而言，将合规与安全防护内嵌到技术方案的最底层，能够直接击中企业级客户最核心的合规痛点，从而消除客户在数据泄露与监管重罚层面的后顾之忧。这种确定性将转变为客户加速采购和规模化部署其解决方案的强力催化剂。Lu Zhang 强调，现场汇聚了大量创业者，理解强监管行业客户的真实诉求与优先级清单，是将前沿技术转化为商业化成果的关键。 --- ### [15:22](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=922s) AI 技术栈的演进：从向量检索（RAG）热潮到 Agent 驱动的实时数据基础设施 Lu Zhang 将话题转向 SignalFire 联合创始人兼 CTO Ilya Kirnos。作为兼具深厚工程背景与投资经验的技术原生型基金合伙人，Ilya 将庞大的 AI 技术图谱划分为四大层次： 1. **底层硬件与数据中心**：芯片、服务器与算力基础设施； 2. **模型层**：以 OpenAI、Anthropic 为代表的基础大语言模型； 3. **基础设施与数据层（Infra Layer）**：涵盖向量数据库（如 Pinecone）、Agent 编排框架（如 LangChain）以及各类数据管道与存储组件； 4. **应用层（Application Layer）**：面向终端用户的“最后一公里”垂直场景产品。 ```mermaid flowchart TD accTitle: AI 整体技术栈的分层结构 accDescription: 展示从底层硬件到顶层应用的四层 AI 技术架构 App["4. 应用层 (Application Layer)<br/>医疗健康 / 企业安全 / 消费端垂直场景"] Infra["3. 基础设施与数据层 (Infra Layer)<br/>Agent 编排 (LangChain) / 向量库 (Pinecone) / 实时数据基础设施"] Model["2. 模型层 (Modeling Layer)<br/>OpenAI / Anthropic 等基础大模型"] HW["1. 底层数据中心与硬件 (Hardware & Data Centers)<br/>GPU 集群 / 算力设施 / 存储硬件"] HW --> Model Model --> Infra Infra --> App ``` Ilya 指出，在经历了几年前的狂热洗礼后，AI 领域的“岩浆正在逐渐冷却（lava is starting to cool a little bit）”，技术格局开始沉淀。以一两年前在行业峰会上备受追捧的检索增强生成（RAG）与向量数据库为例，虽然 RAG 仍有其应用场景，但在 Agent 架构与自主搜索（Agentic Search）技术演进的挤压下，其纯粹的市场份额与叙事热度正在收缩。 当前基础设施层最严峻的瓶颈在于**数据消费主体的更替**。过去的数据消费端是人类用户与分析师，企业通常采用“数据湖 + 数据仓库（如 Snowflake）”的传统架构，依赖夜间批量更新（batch updates）即可满足次日人类在 BI 报表与可视化看板中的离线查询需求。但在当下，AI Agent 已经绕过 UI 界面，直接对底层数据层进行高频查询并自主执行行动。Agent 依赖全局最新状态进行推理决策，批处理数据仓库数十小时的延迟直接导致 Agent 决策失效。因此，传统的批量数仓模式面临严峻挑战，市场迫切需要能够支持超高频更新、甚至具备端到端实时读写能力的下一代实时数据基础设施。 - **应用层投资广度**：SignalFire 正广泛布局医疗、网络安全及消费级应用等垂直落地场景。 - **数据孤岛的放大效应**：无论是成熟大企业还是 AI 原生独角兽，数据孤岛现象普遍存在，Agent 的直接介入将老旧架构的数据滞后问题放大了数倍。 - **基础设施代际更迭**：构建支持 Agent 秒级状态同步的实时数据仓库与存算系统，已成为 Infra 赛道的核心投资热点。 --- ### [20:12](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1212s) 巨头竞合与初创企业护城河：大厂自研还是 Venture-Backable 独立业务？ 针对 AI 基础设施领域的并购潮与巨头扩张，Lu Zhang 提出了一个尖锐的追问：过去一年中，大量从事 RAG 检索及 GPU 底层优化的初创企业在极短时间内被英伟达（NVIDIA）、MongoDB 等科技巨头快速并购，同时甲骨文（Oracle）等传统大型科技巨头也在不遗余力地自研内部 AI 基础设施。在巨头大举布局与并购的双重挤压下，投资机构如何筛选出那些具备独立发展为大规模风险投资标的（Venture-Backable Business）潜力的初创公司，而非沦为被巨头轻易自研替代或低价收购的技术构件？ Ilya Kirnos 直言，这一抉择的根本在于**商业护城河（moats）与可防御性（defensibility）**。仅仅拥有出色的单点技术并不足以构筑壁垒，因为“在你的公司四面墙壁之外，同样存在着大量绝顶聪明的工程师”，技术差距随时可能被巨头或竞争对手通过研发追平。要打造具备独立生存能力的 Infra 企业，必须在底层技术之上寻找更深层的数据网络效应、客户迁移壁垒与数据护城河（Data Moats）。 ### [21:24](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1284s) 法律 AI 案例：Even Up 如何通过“数据合作社”构建护城河 在探讨应用层公司的护城河时，Ilya Kirnos 分享了一个生动的投资案例——专注于法律科技的初创公司 Even Up。该公司主要服务于人身伤害（Personal Injury）律师事务所。大家走在路上常能看到各种吸睛的广告牌，写着“受伤了吗？请联系某某律师”（就像美剧《风骚律师》里经典的“Better Call Saul”）。这个行业规模庞大且极度分散，律所的核心运作逻辑是在客户遭遇车祸等事故登门时，根据案件事实、警局报告以及极为复杂的医疗代码（Medical Codes），为案子估算一个和解金额（Settlement Value）。输入的数据既有结构化的代码，也有非结构化的事故经过叙述。由于这些律所通常采取胜诉提成分成制，如果评估胜算不高或金额过低，他们就不会接案。因此，对案件进行快速分诊和定价，成为了 AI 落地的绝佳应用场景。 然而，Even Up 的杀手锏不仅在于替律所预测和解金额，更在于它构建了一套“数据合作社”（Data Co-op）机制。单独一家律所平时接触到的案件样本非常有限，Even Up 建议他们以匿名方式贡献自己的历史数据，换取访问平台所有客户汇聚而成的匿名行业数据库。这样一来，即使某家律所此前从未处理过摩托车事故索赔，一旦有类似客户上门，他们也能依托全平台的综合数据给出极为精准的索赔估值。当 Even Up 积累了数千家律所客户后，新进入的竞争对手几乎无法匹敌，因为后来者根本拿不到同等体量和深度的专有数据。每一个新客户的加入都在持续反哺这个数据飞轮，让产品对所有人变得更加好用。主持人 Lu Zhang 随后对此表示赞同，强调“没有数据就没有 AI”，专有数据获取能力对于构建特定垂直行业的 AI 至关重要，尤其是在多模态数据清洗（Multimodal Curation）仍是智能体瓶颈的当下，这为底层基础设施投资带来了巨大的机遇。 ```mermaid flowchart LR accTitle: Even Up 数据合作社飞轮模型 subgraph Input["数据汇集"] A["分散的人身伤害律所"] B["结构化医疗代码与案情报告"] end subgraph Platform["Even Up 平台"] C["匿名化处理与数据合作社"] D["AI 案件评估与索赔定价引擎"] end subgraph Output["业务价值"] E["高精度和解金额预测"] F["跨机构数据飞轮效应"] end A --> B B --> C C --> D D --> E E --> F F --> A ``` --- ### [24:18](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1458s) 制药业中的 Agent 落地：严格监管、数据孤岛与人类审查闭环 针对制药行业（Pharma）这一公认的“硬骨头”，Hok Hei Tam 结合自身在生物医药领域的孵化经验，深入剖析了 AI Agent 在面对高敏、高质量生物医疗数据时所遭遇的独特挑战。他幽默地调侃道：“监管越严格的行业，律师就越多；而律师越多，只要你想推进合作或交易，他们就越喜欢制造海量的法务文件。”制药行业的核心生命线是知识产权（IP）与药物专利，这保障了药企在长达 20 年左右的专利期内享有极高的独占利润。正因如此，药企对数据治理的要求近乎严苛，绝不允许任何侵权或泄漏风险。 在基础设施与架构设计上，Hok Hei Tam 指出这带来了两个核心要求：第一，数据隔离。在许多软件领域大家都在努力打破孤岛，但在跨药企合作场景中，“数据孤岛反而是件大好事”，必须确保各家客户的数据在物理或逻辑上绝对隔离，严禁跨机构混合与访问。第二，常被忽视的“人类基础设施”（Human Infrastructure）。AI Agent 的数据底座归根结底是为了辅助科学家和合作伙伴更高效地决策，系统不仅要提供低延迟的实时数据流，更必须建立“人在回路”（Human-in-the-loop）机制。面对美国食品药品监督管理局（FDA）以及药企高层的严格审计，每一项由 Agent 推导出的结论或清洗的数据，都必须有一张清晰、可追溯（Traceable）的审计追踪（Paper Trail），确保从原始实验数据、数据清洗加工到最终分析的每一步骤都有人类签字确认。Lu Zhang 也补充道，业内已有企业通过部署联邦计算（Federated Computing）技术，让药企无需将原始数据进行物理传输和搬运，便能安全接入 AI 能力。 - **监管与法务壁垒**：制药业核心资产为药品专利及长达 20 年的排他性权益，法务合规审核极度繁复。 - **孤岛价值重估**：在跨药企合作中，严格的数据隔离保障是合规前提，数据孤岛在此被视为积极的安全防线。 - **追溯与审计闭环**：无论是 FDA 监管还是药企实验核查，Agent 实施的数据质控（QC）和处理过程必须完整保留源头追踪链条，并配备人工签批。 --- ### [27:04](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1624s) 自研还是采购？制药巨头的买单逻辑与商业化“甜区” 谈及制药巨头在面对敏感数据时“自研（Build）还是外购（Buy）”的抉择，Hok Hei Tam 坦言药企在技术采购上极具保守色彩，受制于美国健康保险可携性与责任准则（HIPAA）及患者隐私法规，它们对直接购买外部软件非常谨慎。Montai Therapeutics 在摸索中发现，顺应药企传统的业务拓展（Business Development, BD）逻辑，往往比硬推纯软件工具要有效得多。 如果把 Agentic 软件当作单纯的 SaaS 工具来卖，潜在客户数量其实非常有限，因为全球大型药企数量屈指可数，难以形成庞大的用户规模。然而，如果团队利用内部的 AI Agent 加速研发，直接向药企交付经过前期临床验证的“候选药物资产”（Drug Candidates），交易逻辑就发生了质的飞跃。在生物医药行业，围绕候选药物达成的合作与授权协议动辄高达数十亿美元。这种模式完美契合了药企现有的风险决策框架与庞大律师团队的审批流程，同时也让初创公司能够绕开软件采购的漫长阻力，精准切入利润最为丰厚的商业化“甜区”（Sweet Spot）。 --- ### [29:10](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1750s) 推理成本、去隐私化与初创公司的“执行力破局” 主持人随后转向 Telnyx 联合创始人兼 CEO David Casem，探讨 AI 基础设施从模型训练转向推理（Inference）过程中企业面临的算力瓶颈与 Token 消耗挑战。在正式回答前，David Casem 先对前面嘉宾的观点给出了直截了当且具有实操维度的回应：针对 HIPAA 和患者隐私（PII）问题，他指出“清洗个人身份信息（Scrubbing PII）在工程上基本已经是一个彻底解决的问题”。企业完全无需为了脱敏而盲目采购部署私有 GPU 集群，甚至可以直接调用大语言模型（LLM）来完成这项任务——只需牢记“大模型输入的内容应被视为建议，而非绝对指令”。他更公开承诺，只要客户有 HIPAA 合规诉求，Telnyx 愿意随时签署业务合作协议（BAA）。同时，针对前面关于律师团队的讨论，他打趣道：“我完全理解为什么法务如此紧张，毕竟像 Anthropic 这类大模型巨头自己都快变成制药公司了。” 在数据与算力生态层面，David 强调了“零日志”（Zero Logging）推理的重要性。企业必须审慎考察推理服务商是否在后台记录数据或使用客户数据训练其模型，而 Telnyx 的原则便是不记录任何客户数据。关于专有数据壁垒，David 提出了更为激进的观点：随着模型蒸馏（Distillation）技术的普及，单纯数据的护城河正在快速贬值——“别人去千辛万苦搜集数据，而你可以直接通过蒸馏复制过来”。从科技史来看，互相模仿产品本就是常态。 由此，David 为初创公司创始人提供了一条反直觉的建议：大企业所陷入的数据治理、海量律师和合规泥潭，恰恰是小团队巨大的不对称优势。在 2026 年甚至即将到来的 2027 年，“没有任何东西是真正坚不可摧的壁垒（Literally nothing is defensible）”，小公司无须理会大企业的条条框框，直接将 Agent 接入所有现成的 API 即可全速奔跑，“唯一的生存之道就是跑得比所有人都快（Run like hell），用极速执行力压制一切”。在谈到企业自身的推理成本控制时，David 回顾道，Telnyx 最开始也经历过极其简单粗暴的阶段：直接为全公司员工采购 Claude 等服务顶配订阅（Max Subscription），让他们尽情放开手脚去探索。 ### [31:29](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1889s) 从 Anthropic 账单冲击到开源模型：Telnyx 的推理成本自救 在谈及模型推理的实际成本时，David Casem 分享了一段令 Telnyx 刻骨铭心的经历。早期公司依靠云厂商补贴 token 费用，然而到了 4 月复活节周末前夕，Anthropic 突然依据服务条款（T&Cs）严格收紧政策，令团队措手不及。当时 Telnyx 仅有约 400 名员工，模型开销原本是每月 8 万美元，新规则下瞬间飙升至每天近 10 万美元，相当于每月需支出 300 万美元。面对这笔完全无法承受且仍处于起步阶段的开销，团队被迫全力攻克推理成本难题。 幸运的是，以 DeepSeek、MiniMax、零一万物（01.AI）以及月之暗面（Moonshot AI / Kimi，特别是其 K3 模型）为代表的开源与高性价比模型迅速崛起并彻底改变了行业格局。与此同时，Together AI 和 Fireworks 等初创公司凭借扎实的 GPU 推理工程能力，迅速成长为估值数百亿美元的基础设施平台。目前 Telnyx 内部每天在员工间消耗约 300 亿个 token；若全量换算为 Anthropic Claude Opus 的调用成本，每天需花费约 150 万美元。David 强调，如果不转向开源模型，公司早就迅速破产了。大语言模型虽然几乎无所不能，但绝不意味着企业应该盲目烧 token，必须对真正输入进模型的内容保持克制与精细追踪。 - **Anthropic 费用剧增**：400 人团队的月度支出面临从 8 万美元骤增至 300 万美元（单日近 10 万美元）的危机。 - **内部 token 消耗量**：Telnyx 内部每日处理约 300 亿 tokens，若采用 Opus 模型单日成本高达 150 万美元。 - **开源/中国模型与推理新贵**：DeepSeek、MiniMax、Kimi (K3) 以及 Together AI、Fireworks 的崛起，大幅平民化了推理算力。 ```mermaid flowchart TD accTitle: LLM Inference Cost and Routing Strategy accDescription: Flow showing how enterprises handle initial smart-model prototyping versus scaled specialized routing. subgraph Stage1["阶段一：PoC 探索与原型构建"] A["前沿大模型 (如 Claude Opus / GPT-4)"] --> B["验证业务可行性与 PMF"] end subgraph Stage2["阶段二：规模化与推理优化"] B --> C{"模型路由与任务拆分 (Model Router)"} C -->|"高复杂度决策 / 核心推理"| D["前沿模型 (Frontier Tokens)"] C -->|"垂直子任务 / 结构化处理"| E["开源及开源权重模型<br/>(DeepSeek, Kimi, LLaMA 等)"] end subgraph Stage3["阶段三：专用化代理落地"] E --> F["专用 Agent (封装特定端点与上下文)"] D --> F F --> G["企业级降本增效交付"] end ``` --- ### [33:55](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2035s) 企业推理成本优化：全员普及与专用 Agent 策略 主持人 Lu Zhang 随后提问：当大型企业将 AI Agent 从单一业务部门扩展到全公司规模时，面临高昂的推理账单应如何应对？David Casem 给出的建议是——初期不要因噎废食，应当全力推动全员“吃下 AI 药丸”（AI-pilled），即使初期需要支付高昂的推理费用也值得去做，因为 AI 带来的生产力重构是颠覆性的。 针对长期的高昂成本，David 认为“解决高价的良药本身就是高价”——高昂的账单自然会倒逼企业采用模型路由器（Model Router）和更经济的模型组合，必要时也可以设置部门预算。此外，硬件层面的演进也在持续拉低单 token 成本，英伟达（NVIDIA）每年推出的新芯片效能都在大幅提升。 谈及架构设计时，David 明确反对打造所谓的“万能公司级 Agent”（例如在 Slack 或 Teams 里挂一个通用的问答机器人），这种做法往往因上下文窗口（Context Window）的限制而导致生成质量低下；真正的最佳实践是构建垂直化、专职化的 Agent（Specialized Agents），为特定的 API 端点与业务功能封装专门的上下文治理环境。 - **全员渗透优先**：企业应优先实现 AI 普及，随之引入模型路由（Model Routers）与预算配额机制。 - **硬件降本红利**：英伟达芯片代际迭代带来的算力与能效提升，将持续压低单位 token 的推理成本。 - **垂直 Agent 优于全能 Agent**：避免通用型聊天机器人，采用针对特定功能封装的垂直智能体以克服上下文窗口瓶颈。 --- ### [36:33](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2193s) 前沿模型与开源阵营的博弈：非前沿 Token 的替代路径 针对开源与开源权重（Open-weight）模型在企业中的地位，Ilya Kirnos 结合 SignalFire 投资早期创业公司的经验阐述了典型的技术演进路径。初创公司在起步阶段的第一目标是“把功能做通”，因此会不计成本地直接选用最聪明的前沿模型（Frontier Models）；一旦验证了产品市场契合度（PMF）且调用量激增，成本控制便成为首要任务。 Ilya 指出，当前市场上的 token 明显分化为两类： 1. **前沿 Token（Frontier Tokens）**：价格昂贵，由 Anthropic、OpenAI 等闭源巨头提供； 2. **非前沿 Token（Non-frontier Tokens）**：由快速迭代的开源模型提供，成本大幅降低。 企业通常会将复杂问题拆解为多个子智能体（Sub-agents），将不需要前沿通用智能的子任务分配给低成本的开源模型处理。目前开源与闭源前沿模型的技术差距已缩短至 3 到 6 个月（甚至随每周发布不断波动），这也促使闭源大厂试图借助监管手段建立护城河。不过由于闭源厂商目前的推理利润率极高，面临开源的定价压力，它们同样具备充足的降价空间。 --- ### [39:58](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2398s) 强监管行业的落地现实：开源权重与定制闭源模型的取舍 Lu Zhang 补充指出，未来的企业架构极有可能是开源与闭源并存的混合模式，特别是在医疗健康、生物制药等强监管行业，开源模型绝非开箱即用，企业必须在其之上构建大量的微调、数据治理与合规层。 Mark McNeill 结合 OneSource Cloud 的企业级服务经验对此表示赞同。尽管平台免费提供了大量优秀的开源及开源权重模型，但在医疗健康、金融服务、能源等特定垂直领域，企业客户对开源模型的实际采用率远低于预期。这些大客户更倾向于采购针对特定行业数据集训练成熟的专用闭源模型，因为这类模型已经内置了垂直领域的专业细微差别（nuances）与合规要求，企业无需从头自行训练或微调。随后，Montai Therapeutics 的 Hok Hei Tam 也准备从生物医药研发的具体实践角度切入探讨这一议题。 - **混合部署范式**：企业未来将广泛采用开源模型与闭源商业模型的组合架构。 - **垂直行业的特殊偏好**：医疗、金融与能源行业更青睐预置行业专有数据的成熟模型，而非直接采用通用开源权重进行二次开发。 ### [41:30](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2490s) 闭源模型的 IP 隐忧与开源在生物制药中的崛起 随着近期关于大模型厂商进军制药领域的讨论不断升温，现场嘉宾打趣道，像 Anthropic 这样的头部公司靠卖 Token 赚取的充裕利润，俨然已经让自己变成了“制药公司”。这种跨界引发了整个生物医药行业的强烈担忧与竞争警惕。虽然各大闭源模型提供商普遍承诺“不使用客户数据进行模型训练”，但在实际运作中，这种政策到底有多强的约束力，企业内心始终存疑。生物医药本质上是一个高度依赖知识产权（IP）的行业，如果药企科学家搜索的分子结构和核心研发数据留存在闭源模型供应商的数据库中，其内部的 AI Agent 完全有可能接触甚至利用这些数据。台上嘉宾调侃道：“Cursor、Figma，现在又是制药公司，接下来会是谁？所以当你把核心数据发送给 Anthropic 时，真的要三思。” 这种对专有数据泄露和潜在商业竞争的恐惧，促使开源大模型迅速成为极具吸引力的替代方案。特别是对于那些资金雄厚、有能力组建专业团队并对开源模型进行精细微调（Fine-tuning）以适应科学研究的机构——例如大型药企以及像 Flagship Pioneering 这样的顶级生物孵化机构，开源路线正在成为主流。相比之下，多数小型生物科技公司往往缺乏专门的机器学习团队，面临更高的技术门槛。从风险投资的角度来看，开源社区的繁荣极大降低了初创企业的准入门槛，让创始人能够基于开源基础模型针对垂直领域进行微调，构建专属的企业级 AI Agent。同时，这也凸显了构建严格的数据治理层（Governance Layer）的重要性，必须确保在规模化部署中充分满足企业客户对数据隐私与合规的严苛诉求。 - **IP 敏感性**：生物医药的核心资产是分子结构与专利数据，药企极度担忧将专有研发数据传入可能成为潜在竞争对手的闭源模型厂商系统中。 - **开源微调生态**：以 Flagship 为代表的头部机构正在大力投资开源模型科学化微调，以建立自主可控的垂直领域科研 Agent。 - **初创企业路径**：开源社区大幅削减了底层研发成本，使创业团队能聚焦于垂直业务逻辑与特定场景治理层的开发。 --- ### [43:14](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2594s) 华尔街拷问 ROI：能源行业出人意料的 AI 变现先锋 主持人将讨论引向当下行业叙事的快速转变：去年的核心话题是模型评测与基准测试（Benchmarks），而今年则全面转向企业级 AI Agent 的安全部署与规模化落地。与此同时，来自华尔街的拷问也接踵而至——企业在算力与 Token 上投入了巨额资金，究竟何时能看到真正的投资回报率（ROI）？ 在对未来一年的落地预测中，嘉宾们指出，能源行业的表现令人大为震撼。尽管医疗和科技行业通常被视为新技术的早期拥趸，但能源板块在 AI 基础设施投资的商业变现速度上，实际上已经超越了绝大多数传统行业。能源企业不仅利用 AI 数据分析去勘探新的能源储备或优化钻井作业，更创造性地将传统勘探与绿色环保倡议相结合。以中东和北非的多国政府能源委员会为例，他们通过 AI 模型将能源勘探过程中产生的衍生数据转化为推动碳中和与绿色能源转型的核心动力，不仅实现了可观的直接盈利，还带动了当地的就业增长、孵化了新兴业务并直接推升了 GDP，成为 AI 变现最具实效的范例之一。 ```mermaid flowchart TD accTitle: 能源行业 AI 基础设施商业变现路径 subgraph S1["底层投入与勘探"] A["AI 基础设施与算力投入"] --> B["传统能源勘探与精准分析"] end subgraph S2["绿色转型与数据复用"] B --> C["勘探衍生副产物与环境数据"] C --> D["碳中和与绿色能源举措"] end subgraph S3["综合经济回报"] D --> E["新增就业机会与新兴业态"] D --> F["直接商业盈利与 GDP 增长"] end ``` --- ### [44:57](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2697s) AI 泡沫论、算力短缺与机器人领域的潜在危机 面对“ROI 在哪里”的质疑，嘉宾们将其反转为一个更深刻的问题：“我们是否正处于 AI 泡沫之中？如果看不到 ROI，市场泡沫破裂会怎样？”对于这一疑问，专家们普遍认为，虽然整个 AI 领域不至于全面崩塌，但某些细分赛道确实存在估值过高、过度炒作的风险，其技术承诺与实际落地能力之间存在明显脱节。 其中，具身智能与机器人领域被特别点名。尽管该赛道吸纳了巨额风险投资，但目前真正在生产环境中稳定运行的案例极少。与文本或图像大模型不同，机器人面临着底层维度的巨大挑战——“机器人的互联网级训练数据在哪里？”缺乏海量、标准化的物理交互数据，构成了难以逾越的瓶颈。如果未来一两年内无法交出具有实际商业价值的成果，机器人领域可能会迎来一轮估值挤压与行业困境。然而从宏观基础设施层面来看，算力短缺（Compute Shortage）在短期内依然无法根本解决，硬件基础设施的扩建周期漫长，算力需求的持续旺盛注定会推动企业级 AI 会议与产业生态在未来一年进一步扩大规模。 - **泡沫分化**：并非全局性泡沫，而是局部过度炒作；技术无法履约的细分赛道将面临出清。 - **机器人数据困境**：严重缺乏高质量物理世界交互数据集，落地生产环境面临硬核工程与数据双重瓶颈。 - **算力长周期性**：计算资源紧缺是物理基础设施建设周期所决定的，短期内仍将是限制与驱动行业发展的核心变量。 --- ### [46:07](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2767s) 从自动化到优化：十亿美元级企业预算与未来展望 对于 AI Agent 的未来轨迹，专家给出了“充满信心，但注定是一段颠簸旅程”（"Bullish but a bumpy ride"）的生动总结。代码生成 Agent 真正发挥颠覆性作用也不过是最近半年到一年的事情；同样的生产力突破正在加速向科学研究、能源勘探等实体行业渗透，更多经得起检验的商业落地案例将在未来一年集中涌现。 从产业演进逻辑来看，企业引入 AI 必将经历两个关键阶段：第一阶段是**业务自动化（Business Automation）**，即通过 AI 取代重复劳动并提升运营效率；第二阶段则是**业务优化（Business Optimization）**，即依托 AI 重塑决策与核心流程，从而释放由 AI 直接驱动的真正营收增长。资本市场的短期波动和局部泡沫是每一轮重大科技革命的必经之路。 ```mermaid flowchart LR accTitle: 企业 AI 落地两阶段演进 A["阶段一：业务自动化<br/>(Business Automation)"] -->|"替代重复劳动 / 降低运营成本"| B["阶段二：业务优化<br/>(Business Optimization)"] B -->|"重塑决策流程 / 挖掘新商业模式"| C["真正的营收增长<br/>(AI-Powered Revenue)"] ``` 作为佐证，来自拥有 45 位 Global 1000 强企业 CTO 网络的最新调研显示，过去两年间，跨国企业的 AI 预算已迅速攀升至“十亿美元级”（Billion-dollar level）。这种庞大的资本支出不仅局限于传统科技大厂，更广泛覆盖了制药、医疗健康和金融服务等深水区行业。面对企业从基础实验全面迈向大规模落地的历史窗口，AI Agent 基础设施与垂直应用正迎来前所未有的广阔机遇。 ## Notable quotes > [03:37](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=217s) AI agents tend to be pretty chatty, issue lots of queries — Ilya Kirnos > [10:44](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=644s) stick to our knitting — Mark McNeill > [15:22](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=922s) lava is starting to cool a little bit — Ilya Kirnos > [20:12](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1212s) 在你的公司四面墙壁之外，同样存在着大量绝顶聪明的工程师 — Ilya Kirnos > [21:24](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1284s) 没有数据就没有 AI — Lu Zhang > [24:18](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1458s) 监管越严格的行业，律师就越多；而律师越多，只要你想推进合作或交易，他们就越喜欢制造海量的法务文件。 — Hok Hei Tam > [24:18](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1458s) 数据孤岛反而是件大好事 — Hok Hei Tam > [29:10](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1750s) 清洗个人身份信息（Scrubbing PII）在工程上基本已经是一个彻底解决的问题 — David Casem > [29:10](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1750s) 大模型输入的内容应被视为建议，而非绝对指令 — David Casem > [29:10](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1750s) Literally nothing is defensible — David Casem > [29:10](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=1750s) 唯一的生存之道就是跑得比所有人都快（Run like hell），用极速执行力压制一切 — David Casem > [33:55](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2035s) 解决高价的良药本身就是高价 — David Casem > [41:30](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2490s) Cursor、Figma，现在又是制药公司，接下来会是谁？所以当你把核心数据发送给 Anthropic 时，真的要三思。 — David Casem > [44:57](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2697s) 机器人的互联网级训练数据在哪里？ > [46:07](https://www.youtube.com/watch?v=oxuMrtjl9yw&t=2767s) Bullish but a bumpy ride ## Mentioned Lu Zhang, Fusion Fund, David Casem, Telnyx, Ilya Kirnos, SignalFire, Mark McNeill, OneSource Cloud, Hok Hei Tam, Montai Therapeutics, Flagship Pioneering, Moderna, Oracle, Google, Twitter, Even Up, FDA, HIPAA, Pinecone, LangChain, Snowflake, NVIDIA, MongoDB, Slack, WhatsApp, Anthropic, OpenAI, DeepSeek, MiniMax, 零一万物 (01.AI), 月之暗面 (Moonshot AI / Kimi), Together AI, Fireworks, Cursor, Figma ## Video description Lu Zhang - Founder & Managing Partner, Fusion Fund David Casem - CEO & Cofounder, Telnyx Ilya Kirnos - Co-Founder & CTO, SignalFire Mark McNeill - Vice President of Global Sales, OneSource Cloud Hok Hei Tam - CTO, Montai Therapeutics AI agents bring new levels of autonomy, but also new infrastructure complexity. This panel unpacks how leading organizations are designing and adopting robust infrastructure stacks specifically for agentic workloads. From compute and hardware to databases and memory layers to software and orchestration layers, panelists will share the building blocks that support scalable, secure, and observable agent use cases. They’ll also explore strategies for integrating agent frameworks with existing cloud and compute systems, balancing innovation with reliability. Whether you’re scaling customer support agents or internal copilots, this session offers an inside look at the technical foundations powering the next wave of enterprise AI. [Source](https://www.youtube.com/watch?v=oxuMrtjl9yw)

[3:37] Ilya Kirnos: AI agents tend to be pretty chatty, issue lots of queries [10:44] Mark McNeill: stick to our knitting [15:22] Ilya Kirnos: lava is starting to cool a little bit [20:12] Ilya Kirnos: 在你的公司四面墙壁之外，同样存在着大量绝顶聪明的工程师 [21:24] Lu Zhang: 没有数据就没有 AI [24:18] Hok Hei Tam: 监管越严格的行业，律师就越多；而律师越多，只要你想推进合作或交易，他们就越喜欢制造海量的法务文件。 [24:18] Hok Hei Tam: 数据孤岛反而是件大好事 [29:10] David Casem: 清洗个人身份信息（Scrubbing PII）在工程上基本已经是一个彻底解决的问题 [29:10] David Casem: 大模型输入的内容应被视为建议，而非绝对指令 [29:10] David Casem: Literally nothing is defensible [29:10] David Casem: 唯一的生存之道就是跑得比所有人都快（Run like hell），用极速执行力压制一切 [33:55] David Casem: 解决高价的良药本身就是高价 [41:30] David Casem: Cursor、Figma，现在又是制药公司，接下来会是谁？所以当你把核心数据发送给 Anthropic 时，真的要三思。 [44:57] 机器人的互联网级训练数据在哪里？ [46:07] Bullish but a bumpy ride

[0:04] Lu Zhang: when AI was called machine learning [6:45] David Casem: Believe it or not, Slack is the place that people talk to agents and agents talk to agents, so we'd like to change that. [10:44] Mark McNeill: stick to our knitting [10:44] Mark McNeill: failed miserably [15:55] the lava is starting to cool a little bit [23:50] Hok Hei Tam: the more regulated an industry is, the more lawyers they have, and the more lawyers they have, the more paperwork they like to construct. [28:33] David Casem: We don't log anything... You should be asking your partners if they're logging or using your data to train their models [28:33] David Casem: Nothing is defensible in 2026, soon to be 2027. Literally nothing. So the only thing that you can do is out-execute everybody else... you just need to run like hell. [33:55] David Casem: the solution to high prices is high prices [41:30] Cursor, Figma, and now the pharma companies. Who's next? [44:57] Hok Hei Tam: bullish but a bumpy ride

[0:04] Uh I'm the moderator today. I'm Lou. I'm the  founder, managing partner of Fusion Fund. So in [0:09] the past 11 years, we've been investing enterprise  AI, healthcare AI and the deep tech. Especially [0:14] one AI was called machine learning. So in the past  couple years, we can see the whole AI industry is [0:19] evolving in a amazing speed. Even in the past  week, you know, all this launches open source [0:25] during the weekend and also all this discussion  about physical agent AI has been really on the top [0:32] of not only tech industry but for all different  industry leader that's also the main topic I [0:38] want to discuss today right we're shifting the  narrative this year is not only just a AI language [0:43] model from the tax sector now is a gentech AI not  only for tax sector but also across all different [0:50] industry financial show insurance the healthcare  sector highly regulated the industry how to really [0:56] solve all this challenge of the infrastructure  layer be able to enable the large scale deployment [1:03] of AI so we're going to deep dive into all  this topic and take it from there so to get [1:07] started I would like to invite all the panelists  to quickly introduce yourself and also maybe just [1:13] uh think about the one critical challenge you are  solving on top of your mind for uh infrastructure [1:20] layer for agent deployment I'll start with  you, Mark. Okay. Yeah. Thanks, Luke. Uh, hi, [1:24] I'm Mark McNeel from One Source Cloud Corporation.  Um, we are an AI infrastructure provider. We're [1:30] based in the Dallas Fort Worth area. We actually  operate 96 data centers in 55 countries around the [1:37] world and our specialty is private or sovereign AI  development and deployment for our customers. Um, [1:45] so we will do that in various methods depending  on the customer requirements. After we design and [1:52] build the deployment and the configuration, um we  will either uh deploy that airgapped in one of our [1:58] data centers or we'll actually build it behind the  customer's firewall on their prem and then manage [2:03] it remotely for them. So we target initially  early adopters who were some of the more highly [2:10] regulated industries that needed extra security of  of an airgapped kind of containerized environment. [2:16] So you think of healthcare, financial services,  government was another big one. Energy was sort [2:22] of a little bit of a surprise to us, but that  was one of our our main uh our main targets as [2:27] well. um Aentic AI now as a lot of our initial  customers who've been in our data centers or [2:35] in their AI deployments for a couple of years you  know training models uh building applications and [2:41] everything they're now in market and they've now  moved to Agentic AI for their customers which we [2:47] now need to support and that has created some  major security risk challenges especially when [2:53] these agents now have access to highly sensitive  very private and usually highly regulated data [3:01] like patient healthcare information. And that's  our main challenge is to put the proper guard [3:06] rails into our infrastructure that prevents those  kinds of HIPPA security violations that could [3:12] easily happen if an agent gets into some data  that they should not be able to access. Yeah, [3:16] 100%. I think now we're really focused  on what is the security deployment of AI, [3:21] right? for the highly regulated industry which is  also very big percentage of the US GDP about the [3:27] data layer the info layer they're not ready for  AI deployment but we cannot wait anymore is the [3:32] precursor to have the governance layer before the  deployment to happen in zero great so over to you [3:37] IO hi my name is IA Kos I'm a software engineer  by background computer science math undergrad [3:43] came out to Silicon Valley right after college and  began my career as a software engineer at Oracle [3:49] I guess today Oracle is better known as a data  center company or something but back then it was a [3:55] just a database company. I worked in the database  group and um after a few years there I moved to [4:00] Google in 2003. Spent 9 years there worked on a  variety of things AdWords infrastructure, Gmail, [4:06] Gmail monetization. Uh left Google in mid 2012  to join a startup that was acquired by Twitter [4:11] pretty quickly and I left before the acquisition  and co-founded SignalFire which is a venture [4:15] capital firm. uh we have about 3 billion under  management uh currently investing out of a fund [4:20] uh that's about a billion in total uh seed through  series B and C something like that those are the [4:26] stages we focus on invest in tons of AI companies  um that I can talk about a little bit more and uh [4:33] also we have our own internal engineering effort  so we have our own ML team our own engineering [4:37] team so we're building the stuff ourselves have  some experience they're hands-on as well and you [4:42] know when I think about the kinds of problems that  we are experiencing and what we're hearing from [4:47] our company's customers. Um, data infrastructure  looms large to me. In some ways, it's always been [4:54] a problem even with human users. Now, you have  AI users as well, but it just magnifies all the [5:00] problems that you already had. Uh, you know, AI  agents tend to be pretty chatty, issue lots of [5:07] queries. And um so what you see is, you know, if  you kind of had something that was just barely [5:12] working with human users, all of a sudden starts  to fall apart when you add AI agents to the mix. [5:17] Um so maybe I'll pause there and hand it over to  the next speaker. Yeah, thank you. Awesome. I'm [5:24] Hake Tam. I'm the chief technology officer at  Monty Therapeutics and also a science partner [5:30] at Flagship Pioneering. Flagship is a company that  creates new companies mostly in biotechnology. Our [5:37] most famous product is Madna and we start a lot  of these small companies that then grow up and [5:42] create therapeutics. So uh Montai is one that's  focused on using basically small molecules that [5:49] turn compounds that we find into pills in order  to treat chronic immune diseases like ulcerative [5:55] colitis for example and we're starting to go into  uh human trials actually hopefully next month in [6:00] Australia. So when it comes to AI agents, right,  we've been using a lot of AI. We I my background [6:06] is in using what's classically known as machine  learning to improve the way we search for drugs. [6:11] Uh and instead of doing that, we're adding to  that AI agents to help us make drug discovery [6:16] more efficient to help us find better drugs faster  for patients. Some of the challenges that includes [6:21] include essentially data governance and data  infrastructure. As you can imagine, the data [6:26] behind these drugs is our most valuable asset.  making sure that that data as it gets handled [6:32] by agents so that we can make our scientists do  much more of the thinking around why these are [6:36] the great drugs that we want to build. Making sure  that data is correct. Making sure that the agents [6:40] aren't hallucinating any data and coming up with  wrong hypotheses around why things are happening [6:45] the way they are. Hi folks, David Casem here,  co-founder CEO of Telnix. We sell infrastructure [6:50] for real-time AI agents. So um think about it as  everything from where your agent has to live. So, [6:57] compute, storage, inference, uh, to everything  that your agent needs to be able to interact with [7:02] the physical world. So, make phone calls, send  WhatsApp messages, text messages, email. Now, [7:08] uh, I think we're up to 44 primitives. You  can check them out at telmix.com or point [7:12] your agent at our website and the agent will  tell you everything that you need to know, [7:16] including signing you up for an account. uh big  challenges today getting enough GPUs deploying [7:22] them uh export controls in certain places um and  then as it relates to agents we have a ton of [7:28] them deployed in the business and I think now the  next big level up for us is getting A2A to work [7:34] um because believe it or not Slack is the place  that uh people talk to agents and agents talk to [7:40] agents so we'd like to change that yes thank you  very much you can see that across the panelist [7:46] there's a couple keywords we always uh mention  And the one thing is AI governance right because [7:50] we talk about uh from the consumer uh chat GBT app  to enterprise level the expectation for AI is very [7:57] different and the hallucination tolerance is much  lower in lots of the industry don't uh application [8:03] another thing is what a large large language or  small language model because most of the industry [8:08] we're discussing here they actually don't want to  upload everything to the cloud of the compliance [8:13] and data data privacy. So small language model  will be able to deploy on the private network [8:19] which also requires certain infrastructure support  like David just mentioned and also another thing [8:23] is efficiency right now the AI competition is  not only the competition of the performance but [8:28] also competition of the cost consumption of the  compute consumption of the energy they will make [8:33] it easier for the large scale deployment that's  also reason I'm has been a very big supporter to [8:39] open source since many years ago and very happy  to see now the industry realize the importance [8:43] of the open source So coming back to Mark uh so  you mentioned about your past experience working [8:49] with highly regulated industry uh and we think  about the US GDP healthcare financial insurance [8:54] industry probably in total over 40% of US GDP is  highly regulated but there are also the industry [9:00] with a huge amount of data ready for AI deployment  probably create best use cases so can you share [9:06] some case study of how you deploy AI in the highly  regulated industry how to make infrastructure work [9:12] how to redesign the architecture to really  building the governance layer. Can you give [9:16] us some good example? Uh yeah, sure I can. Um  there's a a customer that we've had in one of [9:22] our airgapped deployments in multiple data centers  across the the world. They're actually a developer [9:28] that supports healthcare. So they're not a  healthcare provider. They really couldn't tell you [9:32] anything about HIPPA. They sort of know it exists,  but they don't really care that much about that. [9:37] They've developed a technology actually that  basically the bottom line is it will help health [9:44] care providers, pharmaceutical companies, durable  medical equipment manufacturers who need food and [9:50] drug administration approvals to go through  clinical trials. and their technology along [9:55] with the agents that they've developed on our  platform um go through massive amounts of data, [10:01] patient record data that was collected through  these trials of you know patient presented this [10:06] symptom, received this treatment either placebo  or the medication and here was the outcome right [10:12] so it's massive amounts of data that that they are  now accessing through these agents. So initially [10:19] our security protocols and infrastructure was  more than adequate. The air gapping obviously [10:24] creates some security levels right and we  would do things like you know credentiing [10:30] and authorizations for the data scientists who  wanted to access various data sets or pull in [10:34] models or whatever it may be. That worked well  for 2 years. Now they're in market. So now we're [10:39] not only supporting them we're supporting their  customers. These are the healthcare providers. [10:44] These are the ones who are very strictly  adhering to HIPPA guidelines and data privacy [10:49] and everything. So we quickly looked at what  our customers customers were going to be doing [10:55] and said okay our security infrastructure is not  sufficient to handle what an agent may do as it [11:03] accesses these massive data sets. You think that  okay well you can strip out things like patient [11:09] identification information. It sounds good.  doesn't always work that way because ultimately [11:14] a lot of patient identifier information is is  logged into the wrong column, the wrong space, [11:18] whatever it may be. The risk of an agent going in  and finding that, pulling it into their analysis [11:25] and then reporting on something that may become  a public document is a severe HIPPO violation [11:31] and something that our customers customers cannot  cannot uh cannot deal with. So we had to look at a [11:38] totally new data governance and and security  infrastructure and that was something that [11:44] we don't do. We like to stick to our knitting.  We're really good at tell me your requirements, [11:48] what are your goals, what's your model, how big it  is, what are you trying to use it for. We'll come [11:52] up with the right configuration. We'll design that  and we'll deploy it and we'll manage it for you. [11:57] We are not a cyber security company. We're not  a data management company. We don't do data prep [12:01] for entry into GPUs and we certainly don't do data  governance. We interviewed a number of significant [12:08] partners. We found a couple that we really liked.  We're actually exhibiting here right very near [12:12] where we're exhibiting uh in in the hall there  which you know I'll I'll be happy to introduce you [12:16] to some of them. The one we chose for this project  is not here but they were able to put together a [12:23] data governance platform that did everything  you would expect of it. Okay. So it it it [12:27] did authorizations, it did credentiing, it it it  embedded the policies and guidelines for accessing [12:34] data or where data access was prohibited.  And then it also did a really good job of [12:39] um uh logging activities and then audit trails and  they had some experience in clinical trials that [12:45] we really liked and that differentiated them from  the others we interviewed and that's the partner [12:49] that we decided to to employ. Now here's where  the rub came and this is where we had to totally [12:55] redesign our architecture to fit that governance  layer in. We were very applicationdriven. A data [13:00] scientist would come into our infrastructure  and our platform, select various components, [13:06] select their GPU infrastructure, their storage,  whatever it may be, and then we would admit them [13:11] through you know if as they after they logged  in everything we would let them access the data [13:16] as well as the infrastructure. Okay, that was not  what happened here. Okay, so now we've got agents [13:23] that are running wild in in these environments.  So what we did is applicationdriven. We tried [13:30] it. When we did penetration testing, it failed  miserably. We noted so many red lines where data [13:37] that should not have been accessed and reported on  inadvertently got into the report. So we knew that [13:42] the way we had architected things was not going to  work. So on the advice of our partner, we actually [13:49] promoted the data governance application layer to  the actual top of the control menu. So they were [13:55] actually directing everything that eventually  was able to access infrastructure, data sets, [14:02] etc. Our platform was still usable, but we moved  it to the side and all it did was specifically [14:08] monitor the GPU and other infrastructure for,  you know, utilization, optimizing it, you know, [14:14] temperature controls, all the things that we would  normally report on. That was a huge paradigm shift [14:19] for us. But letting the data governance layer be  the controller in this particular environment was [14:25] hugely successful and that's the only thing that  allowed us to move forward with our customers [14:29] customers for these clinical trials. Yeah,  that's terrific example of the use cases and [14:34] also actually that's bring an interesting topic  how we really look at AI governance right when [14:40] people think about governance always think about  regulation and restriction but innovation never [14:45] works well with restriction innovation works well  with incentive the example you just gave can show [14:50] that air governance can be a potential competitive  advantages for a startup when you offer solution [14:56] to highly regulated the industry and eventually  become incentive for for the customer to support [15:02] you to deploy your solution much much faster.  I know we have lots of founder here today. I [15:06] think this is really unique opportunity to also  hear what's truly matters from the industry and [15:11] what really the priority list for the industry  especially highly regulated industry that they [15:17] wanted to see the feature from the technology  provider. Thank you very much Mark. So Idia I [15:22] come to you and uh you guys are very technology  native. We also have a AI analyst ADA since 2018. [15:27] But I know you guys since you founded uh single  affair, you've been build up such a strong data [15:31] science team and leverage AI supporting your  founder sourcing and you invest a lot on the [15:37] AI infrastructure especially as a former engineer  operation background. So can you share a little [15:42] bit more on what are exciting uh opportunity  infrastructure trend you are looking at right [15:48] now and also what are the missing opportunity that  the capital is not tackling enough? Yeah, I mean [15:55] that's um it's a broad question. You know, when we  think about AI, we it's such a large surface area. [16:01] I try to break it down into kind of layers of the  AI stack, if you will. Uh so I guess start with, [16:07] you know, data centers and hardware at the bottom,  right? Uh then you go up to the modeling layer, [16:12] you know, so you know the open AIs, the  anthropics, etc. Uh then you have the info layer [16:17] which covers a wide variety of things. everything  from like vector databases to agentic frameworks. [16:24] So like you know on the vector DB side you know  like pine cone or whatever or you know on the [16:31] um agent framework side it would be you know lang  chain or whatever you're using there and lots [16:36] and lots of other things um and I think that's  probably the place where also like the data infra [16:42] uh plays as well and then on top of that of course  you have the application layer like the last mile [16:47] to the user and um you know we've spent quite a  bit of time at the app layer and are now looking [16:53] at the infra layer more deeply now that the dust  has somewhat settled. Things are still moving [16:59] super fast, but I feel like in the last few years  they moved even faster now. I think we're lava is [17:05] starting to cool a little bit. Um and I think you  saw that with you know a lot of these um early [17:10] infrastructure projects like vector databases etc.  You know rag was kind of all the rage if you came [17:16] to this conference a year or two ago like that's  all people wanted to talk about. I think it still [17:21] has a place today, but you know, I think the kind  of the market share has has shrunk for rag. It has [17:28] its use cases, but you know, agentic search,  all of that stuff has taken away some of the [17:32] use cases. And so when I think about kind of where  we're focused on now maybe versus a year ago, it's [17:40] what I mentioned. I think this question around,  you know, used to have lots of kind of human users [17:45] going in and using your UI or whatever. I have  kind of agents that are bypassing that layer and [17:51] going straight to the data layer, quering your  data directly and it's just magnifying all the [17:56] problems that you used to have even before. Just  imagine, you know, so one issue that's kind of a [18:02] perennial thing is data silos, right? Uh whether  you're a corporate or even within kind of fast [18:09] growing tech companies that are more AI native or  technative, it happens that you know you have data [18:15] living in lots of different places. I think when  it came time to kind of surfacing that data to [18:21] humans, a typical architecture would be, you know,  I don't know, a data lake, you put it in a data [18:26] warehouse, you know, you put it all in snowflake  or something like that and that maybe works okay [18:31] for BI use cases where it doesn't have to be super  real time, right? You do a batch update, you know, [18:37] nightly or whatever. Then your analysts can go  query that data in their dashboards or their [18:42] favorite BI tool or whatever the next morning. And  that's okay. It's not perfect, but it's okay. Um, [18:47] what's happening now is, you know, as agents  become the users, they need the most up-to-date [18:54] data to take action on on behalf of the humans or  autonomously. And what's happening is anytime you [19:00] have an update, it needs to be reflected in the  state so that the agent can do its thing. And [19:05] you know, data warehouses tend to be more batch  operated. Um, updates take time. And so that's [19:11] breaking down, right? And so we're looking at  what is the new architecture, what is the new data [19:17] infra that enables agents to act and read the most  up-to-date data. And that in some ways looks like, [19:25] you know, can you make a faster data warehouse,  something that updates much more frequently, [19:31] ideally in real time versus what's out there  today, these batch processing things. Um, [19:36] so that's one area we've been looking at pretty  closely. And then of course like the application [19:40] layer is like the product surface area is vast.  Every industry, every use case, every vertical is [19:47] up for grabs and there are entrepreneurs, founders  applying AI to everything you can imagine. Um, [19:55] and we've been actively investing in those areas  including healthcare, uh, including consumer use [20:02] cases, um, including cyber security. Um and so  kind of that's where we've been spending a lot [20:08] of our time. Yeah. So thank you for sharing.  So coming back to AI infrastructure trend, [20:12] I have a little bit sensitive questions. Want to  follow up with you. I agree with you. For example, [20:17] we have company like V AI doing the rack and lab  doing GP optimization. They were acquired very [20:22] quickly by Nvidia by MongoDB. Last year we got  like five merge acquisition. This all happened [20:27] very quickly for this type of AI info company with  the large tech company. And you mentioned you are [20:32] working at Oracle and lots of this traditional  large tech company are also building up their [20:37] AI infrastructure layer aggressively and  uh they're competing collaborating with [20:42] the startup ecosystem. So how you make the pick  like which type of the AI info opportunity will [20:49] be better suit for large tech company to building  house versus the startup be able to really expand [20:54] it from the same layer become a venture backable  business. Yeah. Yeah. I mean it comes down to sort [20:59] of like modes and defensibility. Obviously having  great technology is a good start. Um but in some [21:06] ways you know it's maybe not enough uh because you  know smart people exist outside the four walls of [21:13] your company and so it's always possible somebody  can catch up you know and you know we we look for [21:19] various kinds of data modes uh so various kind  of modes and one is maybe data mode you know as [21:24] you're I think it's uh probably most applicable to  application layer companies um and maybe to give [21:32] a concrete example so you know we're investors in  this company in the AI legal space called Even Up. [21:37] Uh, and they sell to personal injury law firms.  Um, you you've probably seen all these billboards, [21:42] you know, injured, question mark, you know, call,  you know, Saul, better call Saul. And, um, that [21:49] industry is a large industry, very fragmented. And  what happens is, I don't know if any of you have [21:53] had this experience, hopefully not, but if you  get injured, let's say you walk into one of these [21:57] firms, they basically put a dollar figure on you  based on the facts and figures of your case and [22:03] like the medical codes that you have, uh, if it's  a, let's say, a car accident, the police report, [22:07] all of that stuff. So, like data goes in, [snorts]  some of it is structured like medical codes, [22:12] some of it is unstructured like the description  of what happened. And then they have to assign a [22:17] value to the potential settlement. they only get  paid as a percentage of the settlement, right? So, [22:21] if they don't think you're likely to succeed, they  won't take you on and they have to triage kind of [22:27] a perfect case uh case study for applying AI. Um,  and but what Even Up has offered is not just, hey, [22:34] give us all your data, we'll give you the expected  value of the settlement. What they've done is [22:38] they've created almost like a data co-op. They go  to all these firms and say you see a tiny sliver [22:43] of all we how about you on um anonymized basis  agree to give your data in order to get every one [22:52] of our customers data back in an anonymized format  and now all of a sudden you can make a much more [22:59] precise estimate of the value of a case. Let's  say you've seen uh you've never seen a motorcycle [23:04] accident before. Now somebody walks in, they've  had a motorcycle accident. How do you put a value [23:09] on that case? Even up enables you to do that.  So, that's a good example to me of, you know, [23:14] if you're a competitor, it's hard now that they  have thousands of law firms as customers, it's [23:20] hard to break in and deliver as good a service  because you don't have all the data that they have [23:25] and every new customer they sign up contributes  to this data co-op, makes the product better [23:30] for everybody. That to me is a good example. Uh,  so that's kind of the sorts of things we try to [23:34] look for. Yeah. Yeah. preparatory data access  is really important for building the industry [23:39] specific there's no AI without data yeah and also  look at the meta AI agent physical the bottleneck [23:44] is still data curation multimod curation so that's  an interesting AI infrastructure opportunity for [23:50] fun to consider okay thank you and come to you  Hokkai uh we all know that pharma industry is not [23:56] the easiest industry to work with you mentioned  about you've been working with them you'll be [24:00] incubating company in that sector and can you  uh share some uh insights And also the use cases [24:06] how you work with the pharma sector in terms of  deploying AI and also update and upgrade their [24:13] AI infrastructure to make it ready especially  consider their data is very sensitive high quality [24:18] but sensitive. Yeah. So I hope uh there aren't too  many lawyers who will be upset by this statement [24:25] but the more regulated an industry is the more  lawyers they have and the more lawyers they have [24:29] the more paperwork they like to construct when  you're trying to do a partnership or some kind [24:33] of deal with them. And that extends especially uh  in this space into the data primarily because the [24:39] main asset in the pharma space is actually  intellectual property right it's it's the [24:44] patent that protects the drug that gives them  an exclusive license to sell it for very high [24:48] prices for 20 years or so. Uh and so as a result  one of the challenges that we have is making [24:54] sure that when we have these agents that our data  governance is very strict right and that includes [24:59] two elements. The first is that if you're working  with one company versus another, make making sure [25:05] that all of the data is siloed. In this case,  silos are a good thing, right? So that you're [25:08] not accessing data across companies. Uh and also  uh when it comes to governance, the the piece of [25:14] infrastructure that people sometimes forget about  is actually the human infrastructure here, right? [25:20] Because it's it's really about building your data  infrastructure to support the decision making in [25:24] this case of the scientists of your partners as  well and making sure that your data infrastructure [25:30] enables that. For example, when it comes to uh  getting making sure you have real-time data, [25:35] right? So that the scientists can actually get  to that data make better decisions faster. But [25:40] then also when it comes to building these highly  regulated relationships, making sure that there [25:44] is a human that signs off on every single decision  on every single choice that these agents make with [25:50] the backing of the data that is used to make that  decision. Uh in order to have a paper trail when [25:56] it comes to the FDA, for example, uh they want  to see every single piece of data that supports [26:02] the statements that you're making to them. Pharma  partners, same thing. They want to see and make [26:07] sure that every experiment was run, here was the  the data, right? And here's what the analysis was [26:12] and making sure all that is traceable. And  if you're having that work done by agents, [26:16] which uh in our case has been really helpful,  doing a lot of the quality control, doing a lot [26:21] of the data manipulation with agents, making sure  that all of those are traceable from the raw data [26:27] so that we can actually go back and cross check  against these. Uh these are the types of ways you [26:32] have to work from a data governance perspective  to make sure that you're not breaking any laws and [26:37] you're not ma you're not uh you're ensuring that  your partners can trust you to accelerate their [26:43] own efforts uh to develop therapies for patients  as well. Nice. And also when you look at the some [26:50] companies expectation that uh when they're  looking at working with a startup company how [26:54] do they make the decision between build versus  buy because lots of the solution they do want [26:59] to build inhouse with very sensitive data and  what is the main kind of the guideline for them [27:04] to make that decision. Yeah this has been a very  interesting uh journey. There's lots of different [27:10] uh attitudes out there but in general pharma is  very conservative again about the data you know [27:16] due to some of the uh limitations around HIPPA and  around patient data especially uh what we found to [27:22] be really helpful is to work with their standard  business development structures you know instead [27:27] of selling them the Agentic software as a tool and  in some cases for applications where that's less [27:34] of a concern uh that works well but for the more  sensitive applications they want to have standard [27:40] products. Basically, these drug candidates  sold to them and it's a lot more valuable [27:44] from their perspective, right? If you look at  the deals that pharma does for therapeutics, [27:49] they're typically in the billions of dollars,  right? After a small amount of clinical trials [27:54] and this is the sweet spot for us when it comes to  doing these partnerships because software uh there [27:59] there's there's a very limited number of pharma  companies. So, it's not like you have a giant [28:03] user base for this type of software. uh whereas  if you do a couple of these deals right that's [28:09] much much more profitable from our perspective  uh and it fits within their decision-making [28:14] frameworks which is something that uh works much  better for them especially with the armies of [28:18] lawyers involved. Yeah. Yeah. We also have a  company been doing a lot of federal computing [28:23] deployment across the pharmaceutical industry. I  think with that technology implement a farmer a [28:28] farmer company doesn't doesn't need to physically  transport move their data make them ready equipped [28:33] with AI deployment. Thank you. So David come to  you. You mentioned early on that you've see the [28:38] challenge of the access to compute the computer  consumption. I think lots of the customer you are [28:43] working with probably facing the similar challenge  and there are lots of discussion about the token [28:48] usage right when they deploy AI agent and also  now there's another discussion about used to [28:53] be lots of compute is doing the training but now  more compute is doing inference and inference is [28:58] also related to agent deployment. So can you also  share a little bit more on how you really help uh [29:04] help the customer you're working with to optimize  the inference and also influence the uh cost for [29:10] the agent deployment. Yeah, I mean um first I just  want to sort of respond to a couple of things that [29:16] uh the other panelist said. Um so as it relates  to HIPPA um p like scrubbing PII is a relatively [29:25] solved problem. So you should be able to tackle  that for the most part without having to deploy [29:28] your own GPUs. Um and obviously like you can even  use LLMs to do it. Remember like anything that you [29:35] put into LM is a suggestion, not an instruction.  Uh but nevertheless, like it's generally a solved [29:40] problem. Um we'll sign a BAA for anyone that needs  one if you need any help from from a from a HIPPA [29:46] perspective. Um I don't blame you, by the way,  on the lawyer stuff given that Anthropic is, you [29:51] know, about to become a pharma company themselves.  Yes. Um, so I I do think it's sort of important [29:56] that when you think about who you're using from an  inference standpoint that you're thoughtful about [30:01] their ability to train on your data and what they  do with your data. So we don't log anything. Uh, [30:07] we think that's pretty important. You should be  asking your partners if they're logging or using [30:10] your data to train uh their models. Um, the data  stuff is obviously particularly interesting. Um, [30:16] I think data is less important now because people  are doing distillation. Uh so somebody else can go [30:22] get the data and then you can just copy it from  from them. Um look, I mean it's been happening [30:28] since the beginning of time. Uh people have been  copying each other's products for a long time. [30:31] I don't think there's anything wrong with that.  Um and frankly encourage folks to continue to do [30:37] that. Um, and since we're doing founder advice,  um, I think it's also worth noting that, um, [30:44] the all the things that you're hearing here about  data governance and lawyers and everything else, [30:49] uh, just suggests how much of a tremendous  advantage you have, uh, as a small company, uh, [30:55] because you don't have to deal with any of this  muck and you can just plug your agents in and give [30:58] it access to all your APIs and you're off to the  races. Uh and nothing is defensible uh in 2026 uh [31:06] soon to be 2027. Literally nothing. Uh so the only  thing that you can do is out execute everybody [31:11] else. Uh so you just need to run like hell. Uh  okay. And now back to inference costs. So we were [31:17] uh spending so we we first started by using  clause subscriptions. I just give everybody [31:23] a Mac subscription and told them to go wild. Uh  thank you Anthropic and Open AP uh OpenAI on the [31:29] record uh for subsidizing our token spend early  on. Uh we couldn't have done it without you. Um [31:36] when Anthropic pulled the rug on us uh going into  Easter weekend in April and said that they were, [31:42] you know, they sort of had put it in their T's  and C's, but they never really enforced it. Um we [31:47] obviously had to get pretty serious about solving  the inference cost problem. Um because at the time [31:53] uh we would have gone from spending I think we're  at 400 people then 80 grand a month to close to [32:00] $100,000 a day. So $3 million a month which  obviously wasn't um palatable and we were just [32:06] getting started. So I you know we were thinking  that the the numbers were going to go up. Uh [32:11] thankfully the Chinese showed up. So, shout out  to Zai, uh, Miniax, uh, and, uh, Deep Seek, and, [32:20] uh, you know, all Kimmy, obviously. Kimmy, thank  you, Kimmy. K3 is awesome. Uh, could it they, uh, [32:25] they changed the game, uh, in a meaningful  way, and you can sort of see what's happened [32:29] with folks like together and fireworks who are  relatively small players and now are trading at, [32:34] uh, tens of billions of dollars uh, simply  because they know how to run inference on GPUs. [32:40] There isn't really think anything defensible  about that, but uh they were there, they put [32:43] in the hard work and uh they're benefiting from  it today. Uh so today we we do about 30 billion [32:50] tokens a day internally at Telmix uh just among  our people. Um and so the equivalent opus cost I [32:58] calculated it was like $1.5 million per day. Uh so  definitely would not have been able to afford that [33:04] on Anthropic or OpenAI. We would have been way  out of business uh very quick. So um that's I I [33:12] mean I think what we sort of encourage customers  to do is leverage open source models and I think [33:18] I mean they're already doing it to a large extent.  People sort of can figure out the math themselves [33:21] pretty quickly. Uh and then obviously you know  like you have to be thoughtful. There's a bunch [33:26] of stuff that you don't need to put into an like  LLM to some extent are a silver bullet believe it [33:31] or not. Like you can literally do anything with  an LLM. Uh that doesn't mean that you should burn [33:35] tokens doing it. Uh sometimes I even have fun,  you know, flipping on Fable to see what it can [33:41] do with, you know, something ridiculous. Uh but  nevertheless, like you just need to be thoughtful [33:45] about what you actually put into an LLM. And  if you're if you sort of start to like just [33:50] track what you're doing, you should be in pretty  a pretty good place. So, so to follow up on that, [33:55] you know, lots of the large enterprise uh when  the initial deploy agent is within one division, [34:00] right? But later on they want to expand it to  the large scale across the whole organization. [34:04] That's when they look at their token bills and  also consumption the inference cost is really [34:08] high. Do you have any kind of a best practice  or suggestion on how corporate should think [34:14] about optimization of their inference when they  doing the large scale deployment any preparation [34:19] work they need to do? How they select the right  vendor to really help them uh thinking through [34:24] the process? Uh I don't I my view is like you  should try to give like you need to get everybody [34:30] AI pled in the organization. If you're going to  end up paying a pretty high inference bill in [34:34] the beginning to achieve that then like go for it.  Uh because it's it's fundamentally transformative [34:40] uh for for folks in the organization. And I  feel like you know you really like obviously [34:44] engineering has been using it pretty much  universally now I think at this point but it [34:50] has use cases pretty much everywhere. Uh and so uh  you know the the solution to high prices is high [34:57] prices. Uh and so like uh people will sort of find  their way toward um better models. model routers [35:04] are sort of emerging. There's a lot of like  different things that will come into play and then [35:09] obviously like worst comes to worse you can put a  budget on it and like have people operate within [35:13] said budget. Um but I mean I think the I think  you you probably would rather be spending more [35:18] on AI now than less. Yeah. And also we saw some  of the interesting trend of the more specialized [35:25] agent like vertical focus the agent that they are  more efficient in terms of the how many inquiry [35:30] they're making for certain use cases and make it  more uh efficient in terms of the answer the same [35:35] questions and memory infrastructure. I think lots  of the evolution on the AI info is trying to make [35:40] inference cheaper and make it more sustainable to  for the large corporate. Yeah, I think inference [35:44] is just going to get cheaper because Nvidia's  kicking ass. I mean, every every year they release [35:50] a new chip that's 10 times better. So, like the  inference cost will just go down on a per token [35:55] basis simply because Nvidia is going to put out  a new chip that's more efficient and can crank on [35:58] more tokens. Um, I do agree with the specialized  agents thing. I I think you know, you need to [36:03] be thoughtful about context windows and um, you  know, giving essentially wrapping a harness around [36:10] particular endpoints and particular functions is  the way to go. I think we've seen a lot of folks [36:15] to try to create like a universal company agent  um like oh here's your AI agent everybody use you [36:21] know Mr. AI agent in Slack or Teams or whatever  to like answer your questions. Uh and I think [36:26] that actually you end up uh with wor worse results  because you know context windows are are still a [36:31] limiting factor. Nice. Great. So next question  is for everyone because David already briefly [36:36] mentioned about this amazing open-source uh uh  innovation happening across US and China. It's [36:42] not only in China at US side there's also a new  open source model coming out open weight model. If [36:48] you look at Nvidia, we'll mention Nvidia a couple  times, they actually have the full open source, [36:52] not only open with even open data to really  help support their enterprise deployment, [36:57] their customer to deploy AI agent in large  scale. So what do you guys think about uh the the [37:02] potential changes that bring in by the open source  or the agent deployment in the large corporate for [37:08] the industry you're investing? Do you think open  source will be one of the top choice for them for [37:13] deployment? are they still wanted to work with uh  more established like anthorpic open AI Microsoft [37:18] of the world to deploy their AI agent and also if  they choose open source what are the main kind of [37:25] the challenge or maybe the thing they need to  really think through before they actually start [37:30] working on it yeah I mean I'm happy to go first  just because maybe we see it a little bit um [37:36] sooner through our portfolio companies that tend  to be earlier stage and kind of more nimble and [37:41] all this stuff and what we're seeing is usually  the pattern is something like this. First you just [37:46] got to get it to work period and you so you start  with the smartest model. You don't care about [37:49] cost. Once you get it to work um and the usage  scales now costs become much more top of mind and [37:59] then people explore these other approaches other  models etc. And I think what you're seeing right [38:04] now is like there are two types of tokens. There's  frontier tokens that are expensive like Anthropic, [38:10] OpenAI, whatever. And then there are the  non-frontier tokens that are getting cheaper [38:16] um and the cost uh difference is huge. And so what  you quickly see is people will start experimenting [38:24] with these open source models uh for subtasks,  maybe not for the whole thing. They will split the [38:29] problem up into sub aents and parcel out parts  of the problem that are maybe that don't need [38:34] frontier intelligence. the open source models  and I think what you're seeing is kind of the [38:39] frontier keeps us expanding out uh and the open  source models kind of the non-frontier stuff is [38:45] kind of quickly giving chase and you know I don't  know it's like what are they six months behind now [38:51] 9 month like depending who you ask or 3 months  you know it it changes week to week right with [38:56] every model release but the the point is they're  not that far behind and I think what you're seeing [39:01] is you know openthropic trying to use regulation  to stifle these guys cuz they understand the [39:07] threat them competitively. There's a lot of  pricing pressure that that um comes into play. [39:12] Uh hopefully that doesn't happen. And so I think  that's the way it's going to go. I think people [39:17] will maybe start with the smartest model, the  most expensive one. Once they find uh something [39:23] that works that has kind of product market fit  within an organization or out with customers, [39:28] then they optimize for cost. And I do think,  you know, the anthropics, the open AIS probably [39:33] will have their share of the market as well. And  they're reducing prices too. Right now they're [39:38] making a lot of margin on their inference. So they  have room to drop prices as well. I think in the [39:44] near term there's so much demand for this stuff. I  think everybody is going to do fine. If you ask me [39:49] to look out, I don't know, 10 years or whatever  what happens. I'm not that's about my pager. I [39:54] don't know. I'm not smart enough to predict that  far out. Yeah. At least I think based on your [39:58] share which I also believe it will be eventually  a hybrid mode right open source close source can [40:03] work together and also depends on the access of  the data and also specific use cases especially [40:08] for some highly regulated industry like healthcare  pharma industry open source is not the plug and [40:13] play you still need to do lots of fine tuning and  also be able to put on the governance layer on top [40:18] of that what do you think and I think I agree with  you on that one that's what we're seeing in our [40:22] environment we obviously have exposure to dozens  and dozens of different enterpris rises, but very [40:27] specifically in those industries where they really  do require closed models. So, they're looking for [40:32] specialties in in developed models, right? So,  they're not actually and we do offer them in [40:37] our platform. They're not really um accessing the  open source models as much as we thought. However, [40:42] but do they think open because open weight has a  narrative that open weight you can add it, you can [40:47] fine-tune it, you have better control if you use  open weight and make it yourself. Yeah, I mean I [40:53] I think that's going to eventually happen. And  I think what Ilia said is is that is eventually [40:57] going to happen. We just haven't seen as much of  that as we thought. So we we're offering all those [41:01] openweight models and we're not getting a whole  lot of traction on that. So they're importing [41:05] their own. They're looking for closed models with  specific data sets that they've been trained with. [41:10] you know, in as you can imagine in specific areas  of health care or financial services or energy, [41:16] there's certain nuances of the model that have to  be embedded in it and that's why they're not doing [41:22] some of the open source models that we offer to  them obviously free of charge on our platform. So, [41:26] oh that's interesting. Yeah. And then from  from our perspective, right, especially with [41:30] the recent news around Anthropic becoming a drug  company as well with the spare profits they have [41:35] from all these tokens, it's really all the farmer  company, right? Yeah. Exactly. Exactly. Uh people [41:41] are worried about the competition that Anthropic  actually provides and is there data even though [41:45] we have these policies around no training right  how much does that actually hold when it comes to [41:50] what's going on inside Anthropic and again this is  an IP driven business right if there's molecules [41:56] that are just search terms that are somewhere  in anthropics database and their agents go [42:00] get a hold of it so there's lots of concerns  around this type of approach and so now the [42:06] open source models are fast becoming a a much  more attractive opt especially for players who [42:11] have enough backing uh to hire the people who  can understand how to fine-tune these models, [42:16] how to make them usable for science. A lot of  these smaller biotech companies don't really have [42:21] machine learning teams uh but some of the groups  like Flagship uh and then the pharma companies of [42:26] course are looking more into this. Yeah. Cursor,  Figma, and now the pharma companies. Who's next? [42:36] So think about uh think about that when you send  your data anthropic. Nice. So I think uh also was [42:44] showing that open source of course as a venture  capitalist we love open source uh community [42:49] because it provide uh such a strong foundation  for startup founder have much lower entry barrier [42:54] to build their own application and fine-tuning the  open source model and build a specialized vertical [42:59] application AI agent move from there and also  another thing we discuss a lot across the panel [43:05] is really have this governance layer and make sure  you really understand what customer care and when [43:10] they're doing the scale deployment we A couple  minutes left. One last question for the panel. You [43:15] know, now we're talking about the narrative shift  very fast in AI, right? Last year is all about [43:19] benchmark data. This year is about large scale  deployment, security deployment for enterprise [43:23] AI agent and the the Wall Street always ask when  we're going to see the ROI, right? Uh when we're [43:29] going to see this return of investment when we  spend so much money on the AI on the token. So I [43:34] want to ask you to give us a prediction next year  if we come back to AI4 again. What is market going [43:40] to looks like? what are your expectation in terms  of the enterprise deployment of AI agent? I I have [43:46] to say I'm very impressed with one sector that we  work with in energy. Okay. Um I am ultra surprised [43:54] because the early adopters were in healthcare and  other industries that energy has done some things [43:58] that I think are monetizing their investments in  AI infrastructure faster than any other industry [44:04] that we've seen. And it's not just what you would  expect ident you leveraging analytics from from [44:11] AI to find new sources of energy you know where  am I drilling where am I finding energy they are [44:16] leveraging them to counterbalance that with green  energy initiatives so that they're looking at [44:21] carbon neutrality we support many um government  energy commissions in North Africa Middle East [44:28] etc. That's critical to them and they have  figured out how to monetize that by taking [44:33] the byproduct of let's find where the energy  is to create these green energy initiatives [44:38] that are then driving job growth, new businesses,  increasing GDP. They're profiting off of that. So, [44:45] I was really amazed by what the energy sector  was doing and we did not expect that. Wow, [44:50] that's a surprise. New uh information as well.  Amazing. Okay. So, maybe another way to phrase [44:57] what you asked is is you know, is there ROI? Maybe  the way to flip it is are we in an AI bubble? If [45:03] there's no ROI, what happens to all those bubble?  Yeah. Um, you know, a year from now, I tend to [45:11] think that we're not I obviously there are some  sub areas where I think there is a lot of hype, [45:17] a lot of funding. Um but the you know it's sort of  TBD whether the technology can deliver you know in [45:25] particular I'm thinking things like robotics you  know I think there's a lot of funding um there's [45:29] not that much that's in production and it's a  fundamentally hard problem um as you mentioned [45:34] data is a big problem there yeah for training uh  there's like what's the internet for robotics um [45:40] so that's a sector that I think could probably you  know have some trouble some distress uh in I don't [45:48] know if it's next year, but you know, unless they  deliver something, but we're at large, I think if [45:52] we're here a year from now, I think we're still  in a compute shortage. I mean, these things take [45:56] a long time. They do. And so, I don't think that's  going to resolve itself quickly. And um, you know, [46:01] if I were to bet, I think this conference is even  bigger this next year. That's for sure. Thank you. [46:07] Now I expect we'll see a lot more ROI at case  studies especially next year as it's really only [46:13] been the last year right or half a year where  these coding agents have actually been able to [46:17] do what they are doing now maybe we'll see the  same thing in science maybe we'll see the same [46:22] thing in many other industries like energy as well  and there the use cases will only become apparent [46:27] next year bullish but a bumpy ride [laughter] yeah  so I think uh we have lots of discussion about AI [46:36] I for enterprise uh I also same as you guys very  bullish about the future AI will go through the [46:42] two phases the the business automation business  optimization once we go through automation to [46:46] the optimization we'll see the true revenue  increasing powered by AI so I still hope we [46:51] see more talents capital resources investing  into this technology trend while we have some [46:56] bubble on the capital market it's okay for every  technology trend one last piece of information [47:01] I want to share with you we do have this saxo  network with 45 CTO from global 1000 companies [47:07] we organized since 2018. In the past two years,  I heard many of them told me their AI budget [47:12] is increasing to billion dollar level which is  quite impressive. It's not only just tax sector, [47:18] it's industry like pharmaceutical, healthcare,  financial which is amazing opportunity for startup [47:23] founder. So thank you very much for spending  time with us today to talk about the future [47:27] of enterprise AI agent deployment. really thank  you uh to all of our panelists that share your [47:33] perspective and insight and uh look forward to  next year and the bigger conference. Thank you.