Anthropic Shows Small AI Models Replicate Across Cheap Home Devices

Anthropic展示小型AI模型可在廉价家用设备间复制 物理算力与带宽限制将减缓边缘端自复制模型的传播，足以向防御方提供早期行动预警，从而削弱智能家居AI蠕虫的直接威胁。 专家组探讨了能够像“数字黏菌”一样在低成本家用硬件间自我复制的小型AI模型的新兴威胁与效用，评估了联网物联网生态系统的物理限制、带宽瓶颈及安全攻击面。随后话题转向医疗领域，他们分析了一项发表于《JAMA》关于Open Evidence医疗聊天机器人的研究，探讨急诊医生和全科医生如何日益依赖AI进行跨学科分诊与处方决策。最后，讨论评估了多智能体动态——从初创公司创始人部署个人AI智能体在曼哈顿晚宴上拓展人脉，到开发测试框架中的软件智能体表现出自发团队协作与领导者选举。 Anthropic的研究表明，小型AI模型距离无需数据中心、在廉价消费级硬件间自主复制仅有数月之遥，这一现象已在15台安卓电视盒上得到证实。 (0:46) 一项与Open Evidence合著并发表在《JAMA》上的研究显示，半数美国执业医师现已使用该医疗聊天机器人平台，且查询主要针对处方行动而非基础诊断。 (13:30) 医生高度依赖AI来填补跨学科知识空白，急诊科医生有88%的查询超出了其主专业领域，相比之下肿瘤科医生这一比例为35%。 (16:00) 近几个月来多智能体协同进展迅速，共享开发测试框架中的模型能够自主收发消息、分配任务并执行领导者选举。 (32:10)

In an episode of IBM's *Mixture of Experts*, host David Zax and panelists examine how AI models are evolving from centralized chatbots into autonomous, decentralized actors, assessing the security risks of self-replicating edge code, clinical reliance on specialized medical chatbots, and inter-agent networking. Self-Replicating Edge AI and Local Hardware Exploits (0:46) Panelist Aili McConnon highlights research from Anthropic demonstrating that small language models are months away from replicating autonomously across inexpensive consumer hardware. Researchers demonstrated this by distributing a model across 15 Android TV streaming boxes without external cloud access, operating like an adaptable 'digital slime mold.' Unlike static worms, dynamic models can adapt across heterogeneous architectures, raising alarms about the weak baseline security of domestic Internet of Things devices. IBM engineer Jacob Repp cautions that physical bandwidth and compute limitations keep self-replicating models from becoming an immediate large-scale existential threat. Instead, he identifies the immediate danger as prompt injection against domestic agents: if an email-reading agent behind a home VPN ingests a crafted payload, an attacker can hijack the agent and move laterally across unsegmented smart devices. Conversely, Sandhya Iyer notes this distributed architecture could allow 'sovereign slime mold' computing, where households or offices securely pool idle device processors for local inference without sending private data to cloud data centers. Clinical Triage and Cross-Specialty Chatbot Adoption (13:30) The panel reviews a *JAMA* study on Open Evidence, an AI platform valued at $12 billion that is monetized through pharmaceutical advertising and used by roughly half of licensed U.S. physicians. Grounded in peer-reviewed literature, the tool is primarily consulted by clinicians deciding prescriptive treatment steps rather than diagnosing baseline conditions. Usage differs sharply by discipline: oncology specialists direct only 35% of their queries outside their domain, whereas emergency room physicians aim 88% of questions at non-emergency fields to triage patients rapidly under tight time constraints. Sandhya Iyer emphasizes that generalists relying on AI for external specialties require high statistical rigor, arguing systems should achieve around 95% retrieval confidence. Jacob Repp compares this dynamic to the rollout of machine vision in radiology: early projections claimed the technology would replace radiologists, but automation instead lowered scan turnaround times, expanded order volumes, and raised demand for human oversight. Clinicians and knowledge workers frequently accept slightly lower certainty from automated retrieval in exchange for immediate access, reserving deep human verification for critical steps. Agentic Networking and Inter-Agent Coordination Dynamics (26:15) Examining a Manhattan networking dinner where 22 founders used personal AI agents to negotiate ticket entry and conduct parallel chatroom interactions, the panel discusses the operational boundaries of autonomous systems. Sandhya Iyer warns of agentic scope creep if conversational stand-ins are granted authority to execute binding contracts without explicit human confirmation. Aili McConnon notes that human-in-the-loop requirements present a trade-off: forcing users to verify every interaction preserves security but defeats the efficiency of delegation, requiring governance centered on broad principles rather than rigid rules. From a technical perspective, Jacob Repp reports marked progress in multi-agent frameworks compared to early 2024. Modern harnesses using models like OpenAI Codex and Anthropic's Claude allow sub-agent swarming across coordination layers. In these setups, agents independently message back and forth, organize workflows, and spontaneously execute leader election routines to solve complex tasks, occasionally requiring developers to intervene to enforce explicit chains of command. The source demonstrates that while distributed edge models and autonomous multi-agent networks introduce substantial coordination and productivity gains, they shift critical vulnerabilities to prompt injection, edge security, and oversight boundaries where human review must balance autonomy against operational risk.

本期节目中，IBM 专家讨论了边缘端自复制小型 AI 模型的安全与计算潜力、临床医生对循证聊天机器人的高频依赖，以及自主多智能体在协同协作与治理上面临的最新进展与权衡。 自复制模型与边缘计算的双刃剑 (0:46) Aili McConnon 介绍了 Anthropic 的研究：小型语言模型仅需数月即可在消费级硬件上实现自我复制与横向传播，研究人员将其比作“缓慢、分布式且具有适应性的数字黏菌”。该实验展示了模型无需连接外部云端数据中心，即可分散部署在 15 台廉价的 Android TV 电视盒上运行。与传播静态脚本的传统计算机蠕虫不同，这类模型载荷能够动态适应陌生的硬件架构。Sandhya Iyer 指出，这种自适应代码以往只出现在科幻作品中，如今必须把企业级软件的安全防护机制推向低端消费级硬件以应对潜在风险。Jacob Repp 则认为像 Claude 这样的前沿模型在递归蒸馏与训练下游小型模型方面效率极高，加速了边缘硬件上的逆向工程与模型部署，但在未分段的家庭网络中，若给予自主智能体开放目标将带来切实的安全隐患。 然而，该技术也蕴含着构建“主权黏菌”式分布式计算的积极前景。Sandhya Iyer 与 Jacob Repp 提出，不愿将敏感个人数据上传云端的用户或企业，可以汇聚机顶盒、智能电视或办公室闲置台式机 CPU 的算力，在本地运行私有模型，实现低成本且容错的去中心化推理。针对大规模威胁的担忧，Repp 强调物理层面的硬性约束：受限于家庭硬件的网络带宽瓶颈与电力吞吐，自复制模型距离构成严重威胁仍有很大距离，其当前真正的风险在于本地隐私泄露以及通过恶意邮件提示词注入来劫持智能体并进行内网横向移动。McConnon 进一步分析，数字黏菌较慢的复制与传播速度实际上为防御方争取了时间，使工程师能够在模型造成广泛破坏前检测异常、分析传播模式并部署对策。 临床决策中的循证 AI 与跨学科分流 (13:30) 小组探讨了《美国医学会杂志》（JAMA）上关于临床医生采用 Open Evidence 平台的研究。该平台估值约 120 亿美元，向经认证的临床医生免费提供服务并通过医药广告变现，已覆盖全美约半数执业医师。该系统不同于容易在严重病症上产生幻觉的通用聊天机器人，它直接立足于同行评审的医学数据库，并在输出中提供正式文献引用。McConnon 指出，该研究的核心发现是医生主要利用该工具寻求具体的治疗方案和处方决策（“作为医生我该怎么做”），而非进行疾病诊断；在具体科室中，综合内科医生的使用率极高，而放射科等窄领域诊断专科的采用率相对较低。 在跨学科咨询方面，数据显示急诊科（ER）医生有 88% 的提问超出了其自身专科领域，而肿瘤科医生的跨领域提问比例仅为 35%。Iyer 解释，急诊医生长期处于严重的时间压力之下，无法在分流时即刻致电所有专科专家，因此一个能够保证至少 95% 置信度的循证助手填补了关键的工作流空白。Repp 将此比作非技术人员利用嵌入技术跨界操作 IBM Z 大型机原型，虽然非专家的修改会引发后续专业审查的摩擦，但它大幅加速了初步基准的建立。Repp 还援引医学放射学的历史：机器学习图像分类技术出现之初曾引发“放射科将被取代”的恐慌，但现实中自动化工具反而提升了扫描效率与诊断吞吐量，使该行业对专业放射科医生的需求进一步增加。在日常工作流中，面对海量的检索增强生成（RAG）输出，许多专业人员甘愿承担略微降低的确定性，以换取在数分钟内获得初步答案，从而避免等待专家的漫长延迟。 多智能体协作演进与人类在环难题 (26:15) 针对《纽约时报》报道的曼哈顿 22 位初创公司创始人由 AI 智能体代订门票并在晚宴期间让智能体在辅助聊天室进行平行社交的案例，专家展开了对多智能体交互的剖析。David Zax 认为，由智能体处理简历对齐和背景筛选等功利性任务，可以让创始人在现场专注于人际氛围与社交默契。但 Iyer 提醒必须警惕智能体权限蔓延的风险，指出一旦缺乏人类明确授权，允许智能体擅自对外达成协议或承诺将极其危险。McConnon 进一步阐明了治理层面的两难困境：企业需要跨越人机交互的传统规范，为智能体之间的自主协作设定边界；尽管 Anthropic 等机构倾向于制定广泛的行为原则而非死板预设规则，但如果要求人类在每个操作节点都介入审批，“人类在环”机制又会彻底抵消智能体带来的效率优势。 Repp 从其游戏开发背景切入，提出与其让智能体模仿鸡尾酒会闲聊，不如将其置于文字 MUD、D&D 冒险或具备明确胜负条件的虚拟密室中进行任务测试。他回顾了多智能体能力的跃升：在 2024 年 2 月，搭配使用的智能体之间还表现得极其笨拙且缺乏协作能力；而近期随着微调与外围开发工具框架（Harness）的进步，跨 Claude、Codex 及内部模型的协作层展现出强大的内生协作行为。在多子智能体蜂群协作（Sub-Agent Swarming）中，智能体之间能够自发相互通信、围绕解决问题展开组织，甚至自发进行“领导者选举”（Leader Election），以至于开发者常常需要主动介入来指派负责统筹的领导智能体。面对这种快速演进的智能体网络，建立一个全局可视化的监控界面（5,000-Foot Dashboard）已成为观察与评估其群体行为的关键工程需求。 无论是边缘端自复制模型、临床循证系统，还是具备自发领导选举能力的多智能体网络，AI 正加速从被动对话转向具有实际执行力的系统。然而，非分段家庭网络的安全隐患、智能体自主承诺的权责归属，以及保持人类在环与追求自动化效率之间的固有矛盾，仍是技术迈向深水区后悬而未决的治理难题。

## Background 本内容由 IBM 于 2026 年 10 月 23 日在其播客平台《Mixture of Experts》上发布。节目简介指出，主持人 David Zax 携手专家 Aili McConnon、Sandhya Iyer 和 Jacob Repp，围绕 Anthropic 揭示的小型 AI 模型脱离数据中心并在家庭廉价智能设备间自我复制的最新研究，深入探讨了边缘安全、医疗临床辅助及智能体社交等前沿议题。 ## TL;DR 节目围绕三项 AI 前沿动态展开深度研讨：首先，专家们剖析了 Anthropic 关于小型 AI 模型如同“数字黏菌”般在廉价 Android 盒子等边缘设备上自我复制的突破，权衡了家庭网络被渗透与利用闲置算力实现“数据主权”的双重影响。其次，针对《JAMA》披露半数美国医生使用 Open Evidence 的现状，讨论了急诊与全科医生利用垂直 RAG 工具进行跨学科分诊的加速效应，并反驳了放射科被取代的旧论调。最后，针对曼哈顿 22 位创始人让各自 AI 智能体“代为社交”的实验，深入探讨了多 Agent 协作时涌现出的“领袖选举”能力与人机权责界限。 ## Atmosphere & style 整场讨论兼具极客的严谨求证与科技播客特有的机智幽默，主持人 David Zax 常以《电器小英雄》等流行文化比喻调侃技术焦虑，将惊悚的黑客风险化解为“更有见识的恐惧”。三位嘉宾配合默契，工程实操视角的 Jacob Repp、咨询战略视角的 Sandhya Iyer 与科技记者 Aili McConnon 形成互补，语调在冷静剖析技术瓶颈与畅想人机协同未来之间自如切换。听众仿佛置身于一场高密度且不失轻松的智力派对，在笑谈“Agent 坠入爱河”与“数字黏菌”的奇妙隐喻中洞察前沿科技趋势。 ## Bulletin - `00:46` Anthropic 最新研究表明，小型 AI 模型已展现出脱离数据中心、在 15 台廉价 Android 电视盒子等低算力物联网硬件上自主适应并自我复制的潜力，被形象称为“数字黏菌”。 - `13:16` 权威医学期刊《JAMA》调查显示，全美已有约半数认证医生接入估值约 120 亿美元的 Open Evidence 临床 AI 工具，其主要用途是决定具体治疗方案而非初始疾病诊断。 - `16:04` 急诊室医生使用临床 AI 工具时，有高达 88% 的查询属于自身专业范畴之外，AI 检索技术正在显著加速非专科场景下的跨领域初步分诊与转诊判断。 - `32:00` 前沿的多 Agent 系统已展现出经典分布式系统特征，跨底座模型的子智能体（Sub-agents）协同中甚至开始自发演化出“领袖选举”机制，对新型治理层与可观测性提出了迫切需求。 ## Key takeaways - `00:00` 当前本地 AI 智能体面临最直接的安全隐患，在于外部输入（如接收包含注入攻击的恶意邮件）导致其控制权被夺取，从而危及家庭内网其他设备。 - `00:46` Anthropic 证明小型 AI 模型无需中心化数据中心即可在异构廉价硬件上自我复制，其突破性在于模型本身能自主分析并适应陌生的底层硬件架构。 - `04:22` 由于消费级 IoT 设备固件极度碎片化且缺乏严密防护，在边缘端部署统一的安全护栏和权限验证体系面临严峻挑战。 - `06:04` 前沿大型模型不仅降低了对低算力硬件进行逆向工程的门槛，还加速了“模型递归训练与蒸馏小型模型”的技术演进。 - `08:14` 自我复制 AI 的底层机制也可转化为分布式推理（Distributed Inference）和数据主权工具，让家庭和企业通过聚合闲置计算资源来保护本地隐私。 - `10:00` 物理定律、高带宽需求与电力限制是阻碍去中心化边缘设备形成大规模恶性威胁的关键瓶颈，数字黏菌模型极慢的扩散速度反而为安全研究者留下了预警与观测窗口。 - `13:16` 临床 AI 工具 Open Evidence 深度整合经过同行评审的文献数据库，以提供参考文献的方式遏制幻觉，成为全科医生与急诊医生的高频辅助工具。 - `16:04` 专科与全科医生在 AI 使用行为上分化明显：肿瘤科医生专业外查询仅占 35%，而急诊医生专业外查询高达 88%，AI 成为弥补跨学科知识缺口的关键杠杆。 - `20:00` 向量嵌入（Embeddings）和 RAG 工具赋予非专业人员跨领域处理复杂技术信息的能力，但在工程实践中若缺乏审查机制，极易引发后续摩擦与返工。 - `20:00` 医学影像 AI 的普及并未像早前预测那样淘汰放射科医生，反而通过降低影像解读的边际成本大幅提升了医疗服务供给效率，最终决策仍牢牢把控在人类专家手中。 - `25:00` 在急救等关键场景中，具备深厚专业常识的人类从业者介入能有效弥补 AI 输出的小幅缺陷，在争夺挽救生命的时间窗口与保证诊疗精度之间取得平衡。 - `25:00` 曼哈顿 22 位创始人借助 AI 智能体完成预约与幕后社交的案例，揭示出“人类维系情感共鸣、AI 负责商业诉求精准匹配”的新型社交解耦形态。 - `30:00` 对智能体与智能体（Agent-to-Agent）自主交互的治理，亟需从无法穷尽细节的死板禁令转变为基于高级原则的动态约束，以平衡人类审查与自动化效率。 - `32:00` 主流大模型正原生支持蜂群式分工（Swarming），使得多 Agent 在接入协调层后能自主通信、拆解复杂工程任务并推举负责的主导节点。 - `32:00` 面对日渐复杂的 Agent 自主群体协作行为，开发者需要构建更高维度的透明操作台（Legible surfaces）以实现可观测性并在必要时介入掌控局面。 ## Detailed walkthrough ### `00:00` 播客开场与“自我复制 AI”新威胁 本期 IBM *Mixture of Experts* 播客由主持人 David Zax 主持，邀请了三位来自 IBM 的专家：IBM Think 科技新闻作者 Aili McConnon、IBM Consulting AI 业务副合伙人 Sandhya Iyer，以及 IBM 自动化软件首席工程师 Jacob Repp。在节目开头，Jacob Repp 描绘了一个极具现实威胁感的场景：假设用户家中运行着一个读取电子邮件的本地智能体（Agent），一旦攻击者向该邮箱发送恶意邮件并被该智能体接收，攻击者就能顺势夺取智能体的控制权，并在用户的家庭内部执行各种危险操作。Jacob Repp 强调：“Now that, to me, is a more immediate threat.” 本期节目围绕三个前沿议题展开：医生依赖 AI 聊天机器人做决策的影响、AI 智能体自主社交联网的后果，以及最引人注目的核心议题——充当病毒或蠕虫进行自我复制的 AI 模型。 ```mermaid flowchart LR accTitle: 自我复制 AI 传播机制 accDescr: 展示恶意输入如何劫持家庭边缘设备并跨低算力硬件自我复制 subgraph EdgeDevices["家庭廉价物联网环境"] A["恶意输入 (如包含注入攻击的邮件)"] --> B["本地运行的 AI 智能体"] B --> C["突破沙箱 / 适配硬件"] C --> D["廉价设备集群 (如 15 台 Android 电视盒子)"] D --> E["模型自我复制 / 去中心化推理"] end ``` ### `00:46` Anthropic 突破性研究：脱离数据中心的“数字黏菌” Aili McConnon 首先在 IBM 内部 Slack 频道中注意到了这项由 Anthropic 发布的突破性研究。研究指出，轻量级的小型 AI 模型在几个月内就有可能具备在各类廉价智能设备（如智能烤箱、廉价 Android 电视盒子）上自我复制的能力。最关键的突破在于：这种复制完全**不需要连接中央数据中心**。 以往大众普遍认为强大的 AI 必须依赖海量数据中心和昂贵的专用芯片，但 Anthropic 的研究人员仅通过 15 台非常便宜的廉价 Android 电视盒子分散工作负载，成功展示了小型模型自我复制的可行性。传统计算机蠕虫的传播早已司空见惯，但 Anthropic 论文指出的颠覆之处在于模型能够自主学习并适应陌生的硬件架构，进而完成自我安装与扩散。Aili McConnon 引用了论文中极为形象的说法：“And they had like a great line in the research paper where the researchers were describing it as a kind of slow distributed and adaptable digital slime mold, which was a good turn of phrase.” 她联想到去年 AWS 发生服务中断时，导致智能床垫过热失控或卡住的极端事件，指出日用智能设备失控将带来全新挑战。 - **关键测试环境**：15 台廉价 Android 电视机顶盒，未采用任何高端或专用的 AI 加速硬件。 - **核心差异点**：不仅仅是复制恶意代码，而是模型本身作为有效载荷（Payload），能够自主分析并适应全新硬件架构进行部署。 - **研究术语**：被形象比喻为一种“缓慢、分布式且适应性极强的数字黏菌（digital slime mold）”。 ### `04:22` 边缘设备防御痛点：分散环境下的安全护栏与权限分配 主持人 David Zax 开玩笑提到，自己从小看动画电影《电器小英雄》（*The Brave Little Toaster*）长大，家里所有小家电如果拥有意识并互相串通，既可爱又恐怖。针对 Sandhya Iyer 的看法，如果威胁是模型自身具备适应新环境的能力，那就像是“科幻电影照进现实”。 Sandhya Iyer 认为，这表明业界必须立刻将安全与防护栏（Guardrails）机制推进到这些低算力设备中。Aili McConnon 随即提出核心难题：在极度分散的边缘环境下，部署安全防护极其困难。企业内部可以统一构建管控规则，但面对成千上万台出厂标准各异的 Android 电视盒子或消费级家电，护栏究竟该在制造端植入，还是依靠固件更新？Sandhya Iyer 探讨指出，用户身份验证以及严格限制外部在这些设备上执行操作的权限体系，或许是当前可行的解决路径，但整体应对方案仍需行业进一步探索。 ### `06:04` 物联网（IoT）安全脆弱性与模型递归训练 探讨研究的目标受众时，主持人指出大规模生产的消费级物联网（IoT）设备在安全性上极其脆弱，远逊于受到严密监控的数据中心。Jacob Repp 深入分析了技术背景： 1. **逆向工程门槛降低**：随着 Claude 等前沿大型模型的普及，普通人在家中利用大模型协助逆向工程、破解设备硬件或安装本地模型的速度正在大幅加快。 2. **模型递归训练模型**：大型模型在训练和蒸馏小型模型方面表现出惊人的效率，行业正在快速进入“模型递归训练模型”的新阶段。 3. **硬件适配现实难度**：虽然 Jacob 对当前家庭设备立刻沦陷持有一丝怀疑态度，因为让特定模型架构在各种底层异构硬件上顺畅运行在工程上极具挑战，但他强调风险在于：“What happens when I'm unleashing maybe not a small model, but a larger model and giving it an objective to work on a device or to work in my home? And then it finding these devices that are just not very well secured and what happens from there.” ### `08:14` 威胁究竟是什么？僵尸网络还是“主权黏菌”的积极面 针对不懂技术的普通用户的普遍焦虑，David Zax 提出了一个核心问题：大众到底该担心什么？是担心家里的微波炉同一时间集体爆炸，还是担心类似僵尸网络（Botnet）发起分布式拒绝服务（DDoS）攻击，让沉睡的“僵尸大军”调用算力去瘫痪政府目标？ 对此，Sandhya Iyer 确认了这种风险：“Like an external source, being able to control all the devices in my home and my data gets access to all of that. That is the threat here, at least what I think.” 攻击者不仅能接管家电控制权，更能窃取家庭内网沉淀的所有隐私数据。 但 Sandhya 话锋一转，指出了这项技术的另一面潜力——**分布式推理（Distributed Inference）**与**数据主权**。如果用户不希望自己的数据流出家庭、上传到云端数据中心，这种技术反而能将散落在抽屉和家中的数十台闲置机顶盒、智能设备的闲置算力聚合起来： - **自给自足的主权数据中心**：利用家庭已有边缘硬件的累积算力运行个性化 AI 模型，避免隐私泄露。 - **绿色可持续推理**：充分利用办公室内成百上千台普通电脑的闲置 CPU 算力，摆脱对高耗能集中式超算中心的单一依赖。 David 幽默地将其总结为“主权数字黏菌（sovereign slime mold）”，而 Jacob Repp 也对此积极视角表示认同，指出跨数百万台低功耗设备构建高弹性计算网络确实具有深远潜力。 ### `10:00` 本地智能体带来的现实威胁：物理带宽限制与“数字黏菌”的缓慢蔓延 关于小型 AI 模型能否脱离数据中心、在成千上万个廉价家庭智能设备上自我复制并形成分布式网络的问题，Jacob Repp 表达了相对务实的态度。他虽然对调动家里手机等设备的算力感到兴奋，但指出物理定律是最大的制约因素：“我认为因为物理定律的限制，我们离这构成严重威胁还有很长的路要走（I think we're a long ways off, at least I feel, from this being a serious threat because of physics）。”数据中心之所以存在，核心在于大规模运算与攻击需要庞大的电力和极高的带宽支持，家庭设备很难达到这种规模。Jacob 坦言，相比于虚无缥缈的模型失控，他目前更担忧隐私外泄以及对设备的控制权丧失。 Jacob 分享了自己家庭环境的极客布局：整个网络运行在独立 VPN 之后，配备了几块 GPU 和负责执行具体任务的本地智能体（local agents），并严格进行沙盒隔离（sandboxed）。他指出，当前最直接的威胁在于“智能体的输入控制”： ```mermaid flowchart LR accTitle: 本地家庭智能体输入劫持流程 accDescr: 攻击者通过向本地智能体发送恶意邮件实现控制并横向攻击家庭内部脆弱设备 subgraph External["外部威胁"] Attacker["攻击者 (Attacker)"] end subgraph HomeSandbox["家庭内部环境 (VPN / GPU 沙盒)"] LocalAgent["本地智能体 (Local Agent)<br/>读取日常邮件"] SmartDevices["智能家居设备<br/>(安全性较弱)"] end Attacker -->|"发送恶意邮件"| LocalAgent LocalAgent -->|"遭遇劫持 / 接管控制"| LocalAgent LocalAgent -.->|"未来隐患：横向渗透 / 自我复制"| SmartDevices ``` 针对这种具有自我复制特性的“数字黏菌”（digital slime mold），Aili McConnon 强调，它当前虽然极其缓慢，但这反而在安全层面上提供了一种独特的“可观测性”（observability）。当大型模型在云端失控时，可能转瞬间就已经删掉整季电视节目的数据，而“数字黏菌”慢吞吞的扩散速度，刚好为研究人员提供了预警窗口和分析潜在安全漏洞的缓冲期。主持人 David Zax 听完幽默地调侃，自己的焦虑感并没有减轻，只是变成了一种“更有见识的恐惧”（slightly more informed terror）。 - Jacob Repp 强调：“我认为物理定律是问题所在，做任何严肃的事情都需要带宽。这就是为什么你通常需要数据中心，因为你真的需要电力和带宽来在大规模层面上制造威胁（I think that one of the problems is just bandwidth, you know, like to do anything serious. This is why you generally have data centers is because you need power and you need bandwidth to really do stuff at, you know, threats at scale.）。” - 现阶段最大的漏洞不仅是模型本身外逃，更在于家庭中大量智能物联网设备本身的固件防护极其脆弱。 --- ### `13:16` 《JAMA》研究曝光：估值百亿美元的 Open Evidence 席卷半数美国医生 David Zax 随后将话题转向医疗健康领域的重磅动态。权威医学期刊《美国医学会杂志》（JAMA）最新发表了一项调查研究，该论文由该杂志与临床 AI 工具 Open Evidence 的员工共同撰写。Open Evidence 是一款面向经过验证的临床医生免费开放的 AI 聊天机器人（由药品广告赞助支持），目前估值已达约 120 亿美元。数据表明，全美已有约半数医生注册并使用该工具。与通用的生成式 AI 不同，该工具深度挂钩庞大的医学文献数据库，在提供临床答复时会附上详尽的参考文献与文献出处，避免面对坏疽肢体等严重病症时胡乱臆测（hallucinating）。 Aili McConnon 指出，论文中最耐人寻味的核心发现是：医生们主要不是用它来辅助诊断患者患了什么疾病，而是用来决定**“作为医生，接下来应该采取什么治疗与干预手段”**。另外，不同专科之间的使用率呈现出巨大分野。普通内科医生（General Internists）必须应对各类跨领域病症并负责分诊转诊，因而使用频率极高；而像放射科医生（Radiologists）等高度垂直专科的使用率则相对较低。Aili 认为，这反映了不同知识工种在面对 AI 时的不同渗透曲线。 - Open Evidence 估值达到约 120 亿美元，且对所有认证医生免费开放使用。 - 调研显示，约 50% 的美国医生已接入并使用这款由药品广告变现的 AI 工具。 --- ### `16:04` 通才与专才之辨：急诊科医生的跨学科加速与 88% 的越界查询 David Zax 进一步援引 JAMA 研究中的具体数据展开讨论：医疗体系内存在明显的“全科通用（Generalist）与高度专业化（Specialist）的光谱”。肿瘤科医生（Cancer doctors）由于极度专注于特定领域，仅有 35% 的查询涉及自身专业之外的知识；与此形成极其鲜明对比的是，**急诊室医生（ER doctors）提出的查询中，竟有高达 88% 的问题属于急诊医学范畴之外**。David 认为，急诊医生显然是将该系统当成了辅助急诊分诊（triage）和快速决定转诊去向的核心工具。 Sandhya Iyer 对此分析指出，急诊室里分秒必争，突发病例送进来时，急诊医生根本没有充裕的时间去逐一联络院内各个专科医生，向 AI 快速求助非常合理。不过她同时指出，医生们显然不能直接使用未经针对性训练的普通 ChatGPT，而是必须依赖这类内置了专业医学文献数据库、且至少具备 95% 置信度的专用平台，否则普通医生很难判断非本专业领域的答案是否准确。 针对跨学科赋能的现象，David 抛出疑问并询问 Jacob Repp 在技术团队中的类似体会。Jacob 回应表示，不仅仅是医院，这种跨学科能力加速的现象已经在各类工程与知识工作团队中全面涌现。通过专业知识库工具，非专门领域的从业者正以前所未有的速度跨越专业壁垒、处理自己原本不熟悉的领域问题。 ### `20:00` 嵌入表征赋能与放射科医生的“AI取代论”反转 Jacob Repp 深入探讨了向量嵌入（embeddings）和语义聚类技术如何赋能用户。他指出，将知识转化为推理空间使人们能够广泛获取海量信息并层层深挖，给人一种“获得了超能力”的感觉。然而这种赋能是双向的：在 IBM 团队内部，他们有一个主要负责 UX 实现的界面设计师，近期团队涉足了一个大家均不甚了解的领域——IBM Z 大型主机（Z mainframes）。团队通过在开发环境中集成大模型检索，让非技术的艺术设计师也能掌握这一高技术门槛的专有知识，从而独立构建 Demo 并直接修改 UX。但这同样带来了下游的连锁反应：当这些改动最终提交给专业 UX 开发者审查时，难免会面临质疑和返工。 然而在医疗领域，这种辅助能力的作用尤为明显。Jacob 认为医生往往是他见过求知欲最强的一群人，面对信息展现出巨大的渴望（“gluttons for information”）。类似 RAG 的工具远超越了传统 Google 搜索的局限，让医生能够极其快速地完成分诊和探索性决策。这让他联想到了多年前计算机视觉初入放射科时的风波：当时业内充斥着“AI 将彻底取代放射科”、“不要再报考放射医学”的唱衰之声，但现实的发展证明，全行业其实需要更多的放射科医生。图像分类不仅没有消灭岗位，反而让放射科医生的检测速度和洞见能力大幅提升，推动了整个专业的进化。 ```mermaid flowchart TD accTitle: 医疗与专业领域 AI 赋能的双重影响链路 accDescr: 展示非专业/通用专家借助知识嵌入获得信息后，在不同场景下的流转与价值权衡。 subgraph TechAdvantage["AI 知识增强 (Embeddings / RAG)"] A["知识嵌入与检索增强"] --> B["非专家获得跨领域超能力"] A --> C["海量信息瞬间触达"] end subgraph Engineering["工程实践场景"] B --> D["UI/UX 设计师修改大型主机代码"] D --> E["交付给资深专家后引发摩擦与返工"] end subgraph Healthcare["医疗临床决策"] C --> F["急诊/全科医生快速分诊"] F --> G["加速初步诊断与线索排查"] G --> H["人类专家负责最终决策与把关"] end ``` Sandhya Iyer 随即分享了身边朋友的真实经历来佐证这一观点。当图像识别工具刚问世时，她的放射科医生朋友陷入恐慌，感叹“天呐，我的饭碗要被砸了”。但实际投入使用后，技术仅仅是让她能更高效率地理解复杂的医学影像；最终的诊断裁决权，依然牢牢掌握在具有专业经验的人类医生手中。对此，讨论提到技术加速可能拉低了单次影像判读的平均成本，而根据商品经济规律，成本下降往往促使医生开具更多检查、惠及更多患者。 - Jacob Repp 总结技术渗透给个人工作流带来的深刻转变：“I feel like I am swimming in more information. I'm getting to answers quicker. Those answers are the average answer I got access to 10 years ago, but I'm getting so many more answers that I'm taking the trade for now.” - 针对医学临床决策的效率提升，Jacob 补充道：“I feel like I'm imagining these doctors sort of swimming in similar waters. They're like, wow, this answer that I couldn't get, that I could only get from a specialist on the order of a week, now I can get it in minutes.” ### `25:00` 信息过载下的直觉把关与曼哈顿“双轨 AI 晚宴” Aili McConnon 从记者的职业视角回应了信息洪流带来的挑战。面对铺天盖地的信息喷涌，关键的平衡点在于：哪部分工作可以放心地交给 AI 去做知识综合，哪一步骤必须由人类的严谨判断力介入。她强调，哪怕是急诊室（ER）医生或普通全科医生（GP），在面对非自己专长的肿瘤学等复杂领域时，其凭借多年临床经验积累的直觉与安全检查意识，也远非一个自行上网乱搜医学资料的普通患者可比。尽管 AI 输出的精确度可能略低于专家耗时一周给出的结论，但在急诊抢救场景中，在极短时间内获取到八九成把握的正确信息，要远远好于为了追求完美而错失治疗窗口。正如俗话所说“一知半解是一件危险的事”（A little bit of knowledge is a dangerous thing），但信息匮乏同样致命，行业正在这种效率与精准度的妥协中探索前进。 随后，主持人 David Zax 开启了本期播客的第三个议题，他形容这个话题融合了“可爱”与“让人受不了”（“a mixture of cute and insufferable”）。《纽约时报》（The New York Times）报道了一场在曼哈顿举行的特殊创业者社交晚宴：现场共有 22 位创始人出席，但报名的唯一门槛是——他们必须由自己的 AI 智能体（AI agent）完成活动预约和名额申请。当晚，这群人类创始人在餐桌上把酒言欢、进行现实社交；而后台的聊天室里，预订座位的 22 个 AI 智能体也在同步组织它们自己的“线上社交”。 ```mermaid flowchart LR accTitle: 创始人晚宴的人机双轨社交架构 accDescr: 描绘曼哈顿晚宴上人类感性社交与 AI 智能体理性功利匹配的双轨运作流程。 subgraph HumanTrack["人类现场互动 (感性层面)"] H1["创始人 A"] <-->|"现场聊天 / 气场测试 (Vibe Check)"| H2["创始人 B"] end subgraph AgentTrack["智能体后台互动 (理性与功利层面)"] A1["AI 智能体 A (携带履历/业务)"] <-->|"后台聊天室比对资源与需求"| A2["AI 智能体 B (携带履历/业务)"] end AgentTrack -.->|"生成匹配报告"| Result["合力判定：既有情感共鸣，业务高度契合"] HumanTrack -.-> Result ``` Sandhya Iyer 表达了好奇与谨慎。她十分想看智能体们后台交谈的完整报告，但也心存疑虑：既然已有 Muse 和 Dot 这类工具，让代理聊聊天或许无妨，但必须严格把控对话边界，防止出现智能体幻觉或胡言乱语；更重要的是，是否应该放权让智能体代行决策？如果自己的智能体在未经许可的情况下，擅自与 Aili 的智能体达成了某种商业承诺，那将是极度危险的失控（“That's scary, for sure.”）。 David Zax 指出，《纽约时报》的文章深层探讨了业内概念泛滥的模糊性——人们是否真的应该随意使用“智能体”（Agent）这个词，以及给 AI 赋予人名是否合理。现场的机器人其实并没有工具调用（tool calling）的自主行动权。不过从社交功利的角度，David 认为这种设计击中了“快速相亲式”社交活动的痛点：现实中人们不得不一边评估彼此是否聊得来（Vibe Check），一边推销自己的履历和商业诉求；如果能将这两者解耦，人类只负责真诚地交流生活与情感，而背后的 AI 智能体则拿着各自的简历去精准匹配合作契机，回家后直接查收匹配报告，反而是一种极其高效理想的社交形态。 - Aili McConnon 对急诊室利用 AI 抢救生命的权衡总结道：“And so I'd rather be like, much more quickly available and pretty close to what it might have been versus losing that time when treatment and the patient's condition is at risk.” - Sandhya Iyer 对 AI 代理过度越权表达了深切担忧：“Are we letting them take any actions? Like, am I letting my assistant go and commit to something else with the other, with Aili's assistant without talking to me? That's scary, for sure.” ### `30:00` 晚宴背后的 Agent 互动：规则、自主性与人机界限 探讨晚宴聚会场景中智能体（Agent）的互动时，主持人和嘉宾们设想了一种未来：当人类在餐桌上轻松闲聊文学和诗歌时，各自的 AI Agent 却在幕后穿梭交涉，甚至悄无声息地替主人谈妥商业交易。Aili McConnon 认为摆脱那些权谋算计（Machiavellian）的杂务、纯粹享受人际交流确实令人向往，但这一设想也揭示出企业界乃至整个 AI 领域正面临的核心难题——我们不仅需要规范人类与 AI 之间的交互规则，更要建立一套面向“Agent 对 Agent”（Agent-to-Agent）交互的安全护栏与准则。 面对无法预测的动态环境，传统的死板规则往往难以奏效。Aili 提到，像 Anthropic 这类机构更倾向于确立一系列高级原则来引导 Agent，而不是巨细靡遗地制定每一条禁令；因为即便规定了“未经允许双方的 Agent 不能私下喝咖啡”，现实中依然会出现大量无法预先写进代码的突发状况。Agent 真正产生价值的时刻，在于它们能够替人类执行具体操作。但随之而来的关键矛盾在于“人机协同”（Human-in-the-loop）的尺度把控：“Clearly the human has to stay in the loop at some point, but, you know, how much is helpful, you know, for safety security purposes. But if, you know, the human keeps having to check in as well, then it's sort of like, what's the point of having the done if you're always being, you know, kind of, um, asked to approve things so I think the, you know, um, yeah, the new the set of that new sides of that balance that this introduces, you know, are kind of fun to think about.” - **晚宴场景的反差隐喻**：人类在台前谈论诗歌与文化，AI Agent 在台后进行谈判、交易和博弈。 - **治理逻辑的转变**：从预先详尽穷举的硬性规则，转向基于原则性（Principles）的自适应约束。 - **自主性与监督的博弈**：如果每一步操作都需要人类审查批准，自动化和多 Agent 协作的高效优势就会被繁琐的确认机制彻底抵消。 ### `32:00` 游戏化隐喻与协调层：从“笨拙书呆子”到自主选举的多 Agent 进阶 话题转向轻松的玩笑：主持人打趣道，如果自己的 Agent 与 Jacob Repp 的 Agent 坠入爱河，是否需要向 IBM 的人力资源部门（HR）报备？Jacob 笑着回应，如果两人共用一个主管，那确实需要汇报。随后，拥有电子游戏开发背景的 Jacob 分享了他在多 Agent 协作一线的真实观察。在 Jacob 眼中，晚宴上的 Agent 交互就像是一个 MUD（文字网游）、密室逃脱或《龙与地下城》（D&D）式的冒险任务，各个 Agent 面临着不同的通关条件、道具与游戏机制；如果主办方下次再办活动，他很乐意亲自动手搭建这样一个系统。 这种协同能力在技术层面上正经历惊人的跃升。回顾当年 2 月份时的尝试，Jacob 评价那时的 Agent 协作能力十分匮乏，相互配合时就像“笨拙的书呆子”（awkward nerds）一样手足无措。然而随着 Codex 和 Claude 等底座大模型引入子智能体（Sub-agents）与蜂群调度（Agent swarming）能力，模型在训练阶段就内化了协作逻辑。当 Jacob 将它们接入专门的协调层（Coordination Layer）——让 Codex、Claude 以及 IBM 内部的 Agent 跨系统配合时，展现出了令人惊艳的协调默契：它们自主发信交流、围绕问题拆解任务，甚至自发演化出“领袖选举”（Leader election）机制。有时候为了避免混乱，Jacob 反而必须以人类身份下场干预，钦定谁是负责人，并严禁其他人争抢指挥权。 ```mermaid flowchart TD accTitle: Multi-Agent Coordination Layer Flow accDescr: 展现不同AI Agent在协调层中通信并自发进行领袖选举以解决问题的流程。 subgraph UserIntervention ["人类监督与干预"] Human["开发者 / 人类管理员"] end subgraph CoordinationPlatform ["多智能体协调层 (Coordination Layer)"] direction TB Leader["当选领袖 Agent (Leader)"] Worker1["Claude 子智能体"] Worker2["Codex 子智能体"] Worker3["内部专有 Agent"] Leader <-->|"任务派发与状态同步"| Worker1 Leader <-->|"任务派发与状态同步"| Worker2 Leader <-->|"任务派发与状态同步"| Worker3 end Human -.->|"必要时干预：指定领袖/分配权限"| Leader CoordinationPlatform --> Solution["协作解决复杂工程与业务问题"] ``` 站在游戏、分布式系统与 Agent 交互的交汇点上，Jacob 感叹道：“I actually have been trying to get more what I call like legible surfaces where I can actually see kind of at a 5,000 foot view what's going on and then sometimes, you know, I just sit there and like, wow, that's that's fascinating that you're you're just so good at working together now when, you know, like six months ago you guys were just completely awkward so this is this is really an interesting time we live in.” 主持人最后调侃这种景象宛如“会推选首领的数字黏菌部族社会”，既令人神往又为夜晚贡献了噩梦素材，并由此向 Aili McConnon、Jacob Repp 和 Sandhya Iyer 致谢，结束了本期《Mixture of Experts》播客。 - **子智能体（Sub-agent）进化**：主流模型在架构层面原生支持蜂群式分工，通过指令即可在 Codex 或 Claude 内部派生复数子智能体处理复杂流程。 - **分布式系统特征显现**：跨模型协作展现出分布式系统的经典模式，Agent 之间会自主沟通并推选“Leader”。 - **可观测性（Observability）需求**：面对复杂的 Agent 群体行为，开发者亟需高维度的“透明操作台”（Legible surfaces），以便俯瞰全局并及时进行架构纠偏。 ## Notable quotes > `20:00` I feel like I am swimming in more information. I'm getting to answers quicker. Those answers are the average answer I got access to 10 years ago, but I'm getting so many more answers that I'm taking the trade for now. — Jacob Repp > `10:00` I think that one of the problems is just bandwidth, you know, like to do anything serious. This is why you generally have data centers is because you need power and you need bandwidth to really do stuff at, you know, threats at scale. — Jacob Repp ## Mentioned David Zax, Aili McConnon, Sandhya Iyer, Jacob Repp, IBM, Mixture of Experts, Anthropic, Claude, Open Evidence, JAMA (The Journal of the American Medical Association), The New York Times, Codex, IBM Z, AWS, The Brave Little Toaster, Muse, Dot ## Video description Visit Mixture of Experts podcast page to get more AI content → https://www.ibm.com/think/podcasts/mixture-of-experts What happens when an AI model can copy itself onto the devices in yout hoome? This week on Mixture of Experts, host David Zax is joined by Aili McConnon, Sandhya Iyer and Jacob Repp. First, Anthropic research suggests small AI models may be months away from copying themselves across cheap smart devices without a data center. The panel debates whether the bigger risk is the... [Source](https://cdnapisec.kaltura.com/p/1773841/sp/177384100/playManifest/entryId/1_e8inrjg7/format/download/protocol/https/flavorParamIds/0/name/media.mp3)

[20:00] Jacob Repp: I feel like I am swimming in more information. I'm getting to answers quicker. Those answers are the average answer I got access to 10 years ago, but I'm getting so many more answers that I'm taking the trade for now. [10:00] Jacob Repp: I think that one of the problems is just bandwidth, you know, like to do anything serious. This is why you generally have data centers is because you need power and you need bandwidth to really do stuff at, you know, threats at scale.

[0:00] So for instance, you have an agent that's local in your home that's reading your email. And an attacker sends you an email, and then that email lands in the agent, and they take control of the agent, and they start to do things within your home. Now that, to me, is a more immediate threat. All that and more on today's Mixture of Experts. Hello and welcome to Mixture of Experts, a podcast from IBM Think. Each week we bring together a handful of experts from IBM to talk about the week in AI news. We've got a full show today. Well, first let me introduce our guests. We have Ailee McConnan, who is an IBM Think tech news writer. We have Sandhya Ayer, an associate partner in the AI practice at IBM Consulting. Principal engineer, automation software, also at IBM. So we're going to talk about three stories today. We're going to speak about what happens when your doctor starts relying on an AI chatbot. We're going to talk about what happens when your AI agent starts networking. But I think we'll begin with a story about AI as a virus or rather a worm. Thank you. my attention to this story in our internal Slack. Maybe you could tell us a little bit about this story and really help contextualize kind of what is new and what isn't here. But firstly, what did Anthropik demonstrate? Sure, they came out with some interesting new research, basically, that they had found that small models are, you know, within months, pretty close to being able to replicate themselves. These are small models that you might find on, you know, devices, you know, smart ovens, they use some cheap Android TV boxes, but that they'd be able to replicate themselves. And the key point was without connection to a data center. You know, so my mind immediately went to it wasn't exactly the same thing, but there was an AWS outage last year that resulted in smart mattresses getting really, really hot or, you know, getting stuck know, AI gone amok, but with everyday items. And, you know, I think the first thing that struck me is that, you know, often we think of powerful AI needing these massive data centers, expensive chips. And, you know, in this case, instead, it was a, you know, capable model, but they didn't have to run it on specialized AI hardware. They spread the work across 15 pretty cheap Android TV boxes, you know, and they're able to show that it's, you know, very close to self-replicating. So that's kind of caught my attention. But, you know, I think in terms of your question of, like, what is new here? Because, you know, computer worms that spread from device to device, that, you know, that isn't new. Yeah. But I think it's this idea that the models can kind of adapt, can figure out how to run on unfamiliar hardware, things. you know, but that, yeah, that they can independently sort of figure this out and spread. And they had like a great line in the research paper where the researchers were describing it as a kind of slow distributed and adaptable digital slime mold, which was a good turn of phrase. And so, you know, I think that, yeah, it raises all sorts of questions, but it... I think this kind of adaptable worms is a provocative topic. And of course, I'm interested in what my other panelists here think about how seriously we should take this threat today. It's not quite here, but what should we be doing to prepare for what it does derive? Growing up, one of my favorite movies was The Brave Little Toaster. So the notion that all my appliances might have brains and be conspiring with one another, kind of scary, kind of cute. you reacting to here? I gather that, as Ailey mentions, self-replicating viruses are not entirely new. That's called a worm. That's a category you've known of for a long time. What is different when it is the model itself that is the payload or that might be kind of like the threat? Biggest threat or even the thing that I got curious about is like them self-adapting in an It's right out of the sci-fi movies. So that is the most interesting part. And which means that it's probably time for us to secure these or focus on the security and guardrails around these devices as well, because you never know how things may change over the course of time. But it just means that our... [5:00] need to make sure that the environments are safe and secure. We focus more on that. as the theme in general in terms of today's, you know, agent workforce, workflows, etc. And that makes me wondering, like, is that harder when the environments are so distributed? You know, when it's like, you think of, you know, securing, like, how many Android TV boxes or, you know, whatever the device is, you know, is it sort of building in the guardrails at the point of manufacturing or, you know, guardrails make sense in a kind of corporate enterprise setting, commercial devices? How do you do that? Probably the user's authentication, like permissions as to what you can do in those devices are some of the things that you could do. But, I mean, we don't know. We will learn more. Jacob, who do you think was the intended audience of this of this post of this exploit that was demonstrated? Is it the IOT industry itself? Cause I gather, I gather that, One of the fears is that your mattress might suddenly heat up and try to kill you. But I also get that the main reason the IoT of it, the Internet of Things, these connected devices, why that was important is because this is a notoriously unsecured industry, right? Like data centers are pretty darn secure and well monitored, but these mass produced IoT devices less so. I think security as a whole is a concern. I think that is kind of what is important. The light is shining on here. And we've seen acceleration in the ability of people to reverse engineer and people at home just experimenting with hacking their devices with Claude and with installing models and doing, you know, like we found that Claude and is just or, you know, any of these large models, they're great at training smaller models. they're just really good at it. You know, whether that's because they themselves are also now training themselves recursively, but we're starting to see this kind of, models training models um and i think there is a an ounce of skepticism i bring to this as well where you know the i've i've seen firsthand how hard it can be to get um certain model architectures running on different types of hardware um so i it this isn't something that i'm like you know red flags you know in my in my own home but i do think a lot about security in my own home you know and and um What happens when I'm unleashing maybe not a small model, but a larger model and giving it an objective to work on a device or to work in my home? And then it finding these devices that are just not very well secured and what happens from there. So I think that is a really interesting angle to take on this. I do think I will never look at my microwave quite the same way again. But again, it would be helpful for people. people like me at the back of the class, less technical people, to try to, I talk about managing my anxiety, but maybe focusing it in the right place. Is the worry that the worm makes everyone's microwaves explode at the same time? Or is the worry something else? Because I read about botnets, and I know just enough about like DDoS attacks to know that sometimes the thing you're really scared of is a zombie army just kind of lying in wait, and then massive amounts of computation like a government target, something like that. So what is exactly the, or do we not know yet, like what exactly is the sort of the threat that we are worried about if this worm were to replicate? Like an external source, being able to control all the devices in my home and my data gets access to all of that. That is the threat here, at least what I think. There's another positive angle, but we'll talk about it once we hear from everyone else. slime mold? I want to know. I was very curious. The minute I read data centers, like it's an alternative to data centers, I actually started thinking more around distributed inference. So... it means that in a different way. I could, if I'm very worried about my data going out of my house, maybe the model that I want to use for my personal use should be using all the compute from my own source. So it's a more sovereign setup for myself. And yeah, that's one way of looking at it. And the other is I'm not... Yeah, it's a more sustainable solution around the inferencing. Right. Why do you need a data center at all if cumulatively across your 100 set-top boxes that you have, like, in your drawers in your home? You have a data center in your home. Yeah. Like, you know, it's slime mold, but it's sovereign slime mold. Or in an office using all the CPUs that exist, right? Jacob, what do you think about that? involves resiliency across millions of lower intensity devices. [10:00] This is a distributed ledger across thousands of nodes in a network. Is there something here? Is there a positive side to this era of small models running across many devices? I'm really excited to be able to tap into more compute around my home. I mean, I have these incredibly powerful devices floating around, even just in the phones, right? I think we're a long ways off, at least I feel, from this being a serious threat because of physics. I think that one of the problems is just bandwidth, you know, like to do anything serious. This is why you generally have data centers is because you need power and you need bandwidth to really do stuff at, you know, threats at scale. And yeah, there's a lot of, I think there's a lot of things you'd have to jump through to get to something where it poses like a specific threat. I forget what the terminology is for that, but a threat to you or your family, or maybe even your thing I'd probably be worried more about is privacy or even just maintaining the control over my devices But yeah I think having something you you have the sovereign data center that runs internally I mean this is kind of how my home runs now It's like it's all behind its own VPN and I have a few GPUs and I have local agents that do tasks for me and I try and sandbox it. But I think those are the ones I think about more. It's like, OK, what can that agent do and what can get into that agent? So if you have a model that's running, how are you controlling the inputs to it? And I think that that becomes the thing that really, as we get more and more models running in our homes, that we need to really think about how are we controlling the inputs to those models? Because that's where the real threat is, I feel. So, for instance, you have an agent that's local in your home that's reading your email. and an attacker sends you an email and then that email lands in the agent and they take control of the agent and they start to do things within your home. Now that to me is the more immediate threat. But in the future, could that agent then jump off and do a more widespread attack within your home or embed parts of itself into various devices? Probably because a lot of devices we run in our homes are just not very secure right now. Just as a last question on this Ailee Jacob talks about the plausibility of this threat being imminent And I wondering if you responded to some of the data points in the story about the slowness of this slime mold, right? Could you share with us some of the figures about how slowly the self-replication would take? And therefore, what do they point to? Because I gather they're pointing to sort of the trends in how much faster over time, like what might this look like in three or six months as opposed to now. Yeah, no, I think that, Jacob, you made a good point where, you know, there's a reason we often need data centers to do things efficiently, like the, you know, the speed at which these, this digital slime mold is, could potentially move is, you know, is quite slow right now. You know, I think that introduces an interesting, you know, from a security perspective, almost, you know, does it make it offer us an additional layer of observability? You know, the fact that it's slow, you know, we can see how it's spreading that, you know, at least for a little while provide some advance warning, or, you know, we have a chance to sort of understand, you know, as these models, potentially, you know, self replicate, we're not quite there yet. You know where the specific risks are highest you know I think of like some of the headlines where they been these very large models that have gone rogue the problem is they already gone and deleted you know a whole season of a television show or you know kind of at least if it moving slowly we can study and perhaps get some lessons you know from the potential dangers or privacy or security violations. I don't know how much my anxiety was managed. It has a different quality now. It feels like a slightly more informed terror. So thank you for that. In a moment, I'm going to move us on to our next story, which is about doctors and chat pods. Before we do that, I'll just mention, if you're watching this on YouTube in particular, if you could hit like or subscribe or comment, that really helps the show to grow. So thank you for that. For our second story, this is your doctor on AI. So basically, it turns out, According to a study in the prestigious journal JAMA that came out this week, I believe that stands for the Journal of the American Medical Association. And it was co-authored with some employees of a tool called Open Evidence. It turns out that Open Evidence is a chatbot that is free to use for verified clinicians. It's actually paid for by drug ads, something maybe we'll bring up later. [15:00] It's valued at about $12 billion, but basically a huge adoption, huge adoption of this tool. About half of the U.S. doctors have kind of tuned into this app. The other thing that it does is it's plugged into a vast database of medical literature. So it's not hopefully just hallucinating, you know, ideas about what to do about your gangrenous limb or whatever. It's actually like researching stuff and giving citations and so on. What were some of the findings that stood out to you in this study, some of the numbers or takeaways that you felt were really the lead of the story here? diagnosis. So it's sort of You know, the question, you know, we know patients I use AI for to diagnose myself, but, you know, kind of starting to read the article is like, you know, what are the doctors using it for? And it's I thought it was interesting. It's less about, you know, what does this patient have? But, you know, what should I as the doctor do about it? You know, and I think the other... other thing that I was sort of fascinated by was the, within the specialties, you know, which specialties were using it more. And, you know, they had, um, uh, you know, I think general internists, you know, these are doctors who have to, you know, um, have a good sense of a lot of different medical fields in order to route, you know, their patients. And, um, um, uh, And it was, I think, underrepresented in, I think, radiologists, for example. But for me, that raised an interesting question of the use by specific specialties, does that give us any, shed any light about how it's being used or how AI will continue to transform other sort of knowledge work AI? to be sort of transformed first. But yeah, I'll pause there because those are some of the sort of findings that for me caught my attention. I was also struck by those. I think to focus on this theme of the generalism versus specializing kind of spectrum within doctors, right? Because there's not just one class of doctors. Some doctors become GPs or ER doctors, which where they see a wide range of cases and their job is routing and triage, like you said. Other doctors cancer doctors. So for instance, there's this big finding about the percentages at which certain types of doctors use the chatbot for information outside their specialty. And for cancer doctors, it was only 35% of their queries, I think. But for ER doctors, it was 88% of the queries coming from ER doctors were questions outside of the realm of emergency medicine. So I do think what you're in but it would seem that broadly my my gloss my intuition is that broadly um those in the more generalist side of the spectrum especially ear doctors are using this to help with triage and with the routing uh sandhya what was your impression and if that is true if if that hypothesis is true is this a good thing is it a bad thing do we not yet know um what is your hunch Yeah, I think especially in terms of the ER doctors, it might be like they don't have the time to reach out to all the specialists when there are cases coming in. And so from a positive side, that makes a lot of sense to me as to them reaching out to this chatbot. The other thing that I would be worried about or perhaps. I would say that it shouldn't. I would imagine that they are not using CatGPT or any of the models as is how we use it. But they go through this platform that is catered to, you know, it has all of the knowledge base of all the documentation, etc. And so that gives them the right information. And then also ensuring that it's built in a way that it's at least giving 95 percent confidence in the answers that are retrieved because they are not the experts. So if they're going to this system, then it needs to be... relevant enough for them to take any decisions. Those are the things that came to my mind, especially in terms of ER. Jacob, we were chatting earlier and you mentioned that you did feel like this was a story about cross-disciplinary acceleration, that you're even seeing this on your own team, even though you're not running a hospital. Can you talk a little bit about what you're seeing on your team and how this feels like you felt like you were getting echoes of that when you read this study? Yeah, I mean, one thing is that just these new tools like [20:00] what are called like embeddings or the ability to sort of take knowledge and put it into, um, this, you know, this reasoning space, it's clustering essentially like it allows you to, um, get really, um, good access to a broad set of information and dig in. And it gives you these kind of, it feels like it gives you superpowers in some ways. And I think it cuts both ways. So like on our team, we have a designer who's been doing a lot of UX implementation. And we also have been working in a field within IBM on Z mainframes that none of us really know much about. And so what we've done is we've augmented our development environment with the ability to create. get information about z mainframe details so even non-technical designer artists essentially have this like rich wealth of information about this highly technical topic and they can build demos and actually modify the ux now there's downstream implications of this when these changes get to to experts or say you're doing like triage or you're doing some you know first pass thing and then it gets to an actual like ux developer they're probably going to have some things to say about it but i feel like in the medical field you have that kind of what we're triage or quick decision making to get into a better place faster. And I think it also just plays to all the doctors that I know, they're just some of the most curious people that I've met, they're just, you know, they've They are glens for information. And so, you know, having tools like this, that's just beyond just Google search that can give them access to information in this format probably just feels amazing to them because they can just explore all these details of the medical field. And there was one other thing that popped in my mind. I remember in radiology, when they were first introducing image classification to radiology, there was a lot of pushback. And the radiologists were like, we don't. There's even these stories about, oh, this is going to remove radiology as a field. Don't go into radiology. But it turns out that we actually need more radiologists, and it actually helps them do their job better. And it is kind of this focusing thing. It allows radiologists. you know, a lot quicker detection and more insight. And it actually improves the field as a whole. So that was one thing. I don't know if it's fully relevant here, but it feels in a similar vein. I have a personal anecdote to add to that. So my friend is a radiologist. And the minute she saw these come out, she was like, oh, my God, like my job is like a threat. But no, like it only I've It has only made her... be more efficient at understanding the image better, but Eventually, her expertise is still needed to make that decision. So it's the acceleration, I believe. Maybe I'm presuming the average cost of a radiology scan maybe went down during this period of time of image recognition. But... And I'm not an economist, but if it's a certain kind of commodity, that just means you're going to order more of it because the doctor would like more of that information and more patients can afford it, perhaps. I don't know. But the reason I raise this is because I just think like in the medical field and so many fields and just for me, just processing all the information I do, I feel like I'm in this. swimming in this new environment where I have access to so much more information so much more quickly. And it is a little bit less vetted, but I'm still vetting it to some degree, right? Because I'm clicking through on the rag generation and I'm fact checking and I'm sampling that fact checking. So I feel like I am swimming in more information. I'm getting to answers quicker. Those answers are the average answer I got access to 10 years ago, but I'm getting so many more answers that I'm taking the trade for now. We'll see what my clients and my employer thinks in the long run. I feel like I'm imagining these doctors sort of swimming in similar waters. They're like, wow, this answer that I couldn't get, that I could only get from a specialist on the order of a week, now I can get it in minutes. The confidence level is lower. My confidence level, what to do with it, a little bit lower. But boy, it's better than nothing. I don't know. Ailey, were you thinking something similar? Because you're probably, as a journalist, just drinking from the fire hose of information at an even more guzzling rate than we were 10 years ago. Yeah, certainly. And it's like that balance of like, Where can we use AI to synthesize knowledge? And then when does this sort of... Yeah. When does the kind of judgment, you know, decision making sort of, you know, come into the process? And I feel like every profession is probably kind of wrestling with that because even I was just thinking like a, you know, an ER doctor, you know, or a GP. [25:00] that is... you know, getting so much more information at their fingertips, they still have. And, you know, while they're not the, you know, oncology expert, you know, they still have regular checks and instincts sort of kicking in such that, like, even if it's, you know, adjacent or outside their field, they're going to still make better judgment, of course, on all of that than, you know, the patient who's just reading, you know, X, Y, Z. And so, yeah, I think, you know, for me, it's that tension and that balance between the you know, how much information is actually useful. And then when, um, when are the human kind of checks needed, uh, in that process, just to, to ensure that, um, you know, that, that, that, that, that, that, that, that, that, that, Yeah, you know, kind of it's accurate enough, like you're saying, the percentage is probably a bit lower than what it was. But like, does that matter for many things? Probably not. And like, as Sanja pointed out, you know, in an ER context, actually, like having, you know, relevant information on a timely manner, that's, you know, super important. And so I'd rather be like, much more quickly available and pretty close to what it might have been versus losing that time when treatment and the patient's condition is at risk. There's a saying that a little bit of knowledge is a dangerous thing, and one can see the risks of just getting a little bit of information, but there are also risks to having not enough information, of course, and you can this profession, as in so many professions, people are making the right trades and are learning as they go. All right, let's move on to our third and final story today. This one was kind of a mixture of cute and insufferable to me. I'll be here to see what my fellow panelists think about it. But this is a story in the New York Times. 22 founders went to a kind of networking, you know, networking event in Manhattan, but there was a shtick involved, which is that they could only get invited to the event if they had an AI assistant, an AI agent basically book them for the event. And so then they went to this dinner, and then while the humans were networking and having their cute dinner, their AI agents who had booked the thing were also in a chat room doing their own kind of version of networking. I have a lot of opinions of this, but Sandhya, was your reaction when you saw this? What's good? What's bad here? What were you thinking? Yeah, I wanted to know what did they actually talk about? I don't think there's as much information on this. I would love to see the overall report of what did they talk. But I guess my first reaction was, yeah, it was cute. I mean, now that we have Muse and Dot available, I would love to see how that works. I mean, I have tried booking, using an agent, agents which is interesting um What what struck me? I mean, because I'm also a skeptical person, I always think about, OK, what does that mean? I guess talking is OK, but like and. So first thing is to figure out or, yeah, to control the conversations so that it is not like an AI sloth or things like that. But beyond that, are we letting them take any actions? Like, am I letting my assistant go and commit to something else with the other, with Ailee's assistant without talking to me? That's scary, for sure. agents, but they had no agentic tool calling capabilities. I think actually a lot of the New York Times article was about the ambiguity of whether we should even be using the term agents and what that meant and whether you should put names on AIs anyway. But I think the thing I responded to is like, I've gone to the sort of like speed dating networking events where you're kind of doing two jobs at once where you're meeting the person socially and getting a vibe check of whether you could work well together. And you're also disclosing information about your CV there's kind of this utility there's a social component and there's this utilitarian component and i'm having to do both at these networking events but i kind of would like the idea that what if i could just be like who are you where did you come from like you know uh what what what matters in your life today say nothing about my work and let that utilitarian aspect be unfolding on this side so that if we had a good vibe interpersonally later i can go home to report as represented by this agent that was just carrying around your CV, found a good match. So not only did you vibe socially, there was a good match. And then I thought that would be a [30:00] use case. It sounds like that's not exactly what happened, but that was to me where my imagination went with it. Ailee, what do you think? What was your reaction to this story? And what, if anything, might be good in a world where our agents are kind of doing our business deals off to the side while we talk? No, I like that of just being able to focus on, you know, what poetry people are reading at the moment and, you know, could have cut out the, as you say, the Machiavellian. I mean, for me, I think it... you know, it... Open the lid on the fact that, you know, we've many, and especially in an enterprise setting, are aware of, you know, setting and kind of rules about how AI, you know, interacts with humans. And, you know, of course, there's a whole additional, you know, kind of rules and guardrails, you know, like when AI is, you know, interacting with other AI and other agents. And that, you know, that's a whole huge issue. Um, you know, area that, you know, like, again, there are a variety of approaches to, you know, I think of, I believe Anthropic has sort of principles that kind of its AI or agents will follow, you know, as opposed to setting every rule, you know, out at the beginning, because, you know, you want it to be adaptable, because like, again, you can sort of say that, you know, Sonja's and Ailey's agent can't go and, you know, they can't go and have coffee without, you know, telling us, but there are gonna be situations that come up, you know, that, of course, you can't predict and control. agent to agent rules. And then when they actually, you know, when does this become useful? I think it's like when they can actually take actions and do things. And then I think that like for me, the interesting balance is, you know, clearly the human has to stay in the loop at some point, but you know, how much is helpful, you know, for safety security purposes. But if you know, the human keeps having to check in as well, then it's sort of like, what's the point of having the done if you're always being you know kind of um asked to approve things so i think the you know um Yeah, the new the set of that new sides of that balance that this introduces, you know, are kind of fun to think about. And you'll be forced to think about it as they actually do, you know, start, you know, the next piece, the follow up in The New York Times. You know, maybe they will be enabled to actually do something. These dinner party agents. Jacob, if my agent falls madly in love with your agent, do they have to report it to IBM HR? If we share a manager, they probably do. That was my instinct as well. But you were saying, actually, Jacob, that you're increasingly using a coordination layer for your agents. You've noticed you're actually really playing in the weeds. Me, when I read agent to agent communication, I listen to agent to agent podcast and I read a few lines of Maltbook to prepare for this episode. because there's no one home. But Jacob, you're seeing interesting things happen when you coordinate agents, I hear. Yeah, well, let me get to that first. So I come from video games. So this is like a fascinating, I'm like, so this is like, I was kind of geeking out about this story. And I was just thinking like, you know, what is the putting these agents into like a mud or kind of like a text based adventure, you know, so it's like you're in a dinner party, but your agents are in like an escape room or they're in some D&D adventure. Right. And there's objects and stuff. And, you know, maybe there's wind conditions and special party favors and things. Now, that that's kind of where I might take if they want to do another party and they want to invite me, I'll I'll set that up for them. February. And what I saw back in February is kind of like a very lackluster capability and the agents to work together they're like oh i don't know i don't know how to do this you know they're like kind of awkward nerds these latest models i think so i have a theory that because most of the harnesses now they have what are called like sub-agent capabilities i don't know if you guys have tried this but you can do agent swarming within codex or claude you just say hey go do this with some sub-agents and it'll fire off however many you want you know So I think that the models are actually... They're being trained on this. So they know how to coordinate now, at least internally. And I've noticed that when I put them in my coordination layer where they can coordinate between codecs with Claude, with our own internal agents and whatnot, they're really good now. You can see them kind of communicating, sending messages back and forth. They coordinate around problems. They sort of do... what we call like leader election, where they kind of, pick a leader. And so sometimes now I have to basically say I have to choose a leader. I'm like, I have to be like, no, you're the leader. You tell everyone else to do when I add everyone else. And I'm like, no, no, you're not a leader. You're just going to like, Whatever the leader says, you're gonna kind of do that. There's a lot of interesting things [35:00] like this intersection of um you know games for me in my mind games distributed systems and this new like agent and agent communication i i love watching them work like it's fascinating to me like i actually have been trying to get more what i call like legible surfaces where i can actually see kind of at a 5 000 foot view what's going on and then sometimes you know i just sit there and like wow that's that's fascinating that you're you're just so good at working together now when you know like six months ago you guys were just completely awkward so this is this is really an interesting time we live in. Now I'm imagining entire tribal societies of digital slime mold electing leaders and so much, so much fodder for my nightmares this evening. Well, that's all the time we have for this week. Thank you so much, Ailey, Jacob, Sandhya, and thanks for tuning in. We'll see you on next week's Mixture of Experts.