Synchron Trains Generative AI on Blood-Vessel BCI to Restore Autonomy

Synchron基于血管脑机接口训练生成式AI以恢复自主能力 Synchron认为，直接基于血管内神经遥测数据训练自回归基础模型，可以捕捉未压缩的认知意图，并在无需开颅手术的情况下实现可扩展的运动功能恢复。 Peter Yoo介绍了Synchron的血管内脑机接口（Stentrode），该技术通过在脑血管内部记录高保真颅内信号，从而避免了侵入性开颅手术。Synchron并未依赖文本或语音等有损的下游人类产物，而是将神经振荡动力学进行词元化，以训练能够捕捉底层认知与运动基元的自回归基础模型。通过将这些预训练的Transformer表征与轻量级实时解码器及多模态AI系统相结合，瘫痪的临床试验参与者能够在具备强抗伪影能力的情况下自主操作消费级电子设备。本次演讲还探讨了经验性神经缩放定律，并阐释了Synchron为何优先选择单向传感而非神经刺激，以安全地扩大对患者的救助规模。 Synchron 致力于直接基于上游大脑神经动力学训练认知基础模型，绕过文本和语音等下游、有损的人类表征，在生理源头捕捉人类意图与思维。 (0:04) 其微创血管内 Stentrode 阵列经由颈静脉植入上矢状窦，无需高风险的开颅手术或留置经皮导线，即可高保真采集皮层电生理信号。 (12:46) 团队将高频神经频谱脉冲离散化为 Token 并利用 Transformer 进行自回归预训练，证实了神经动力学同样遵循 AI 的模型缩放定律（Scaling Laws）。 (23:28) 通过仅需 5 分钟现场校准的 75k 参数轻量解码头，系统可在抑制谈话和环境动作杂波的同时精准识别脑电点击，并结合 VLM 与 LLM 实现对日常电子设备的独立控制。 (26:49)

Synchron argues that existing artificial intelligence models are fundamentally constrained by training on lossy, downstream human artifacts such as text and audio, proposing instead to build foundation models directly from the biological source of cognition using scalable endovascular brain-computer interfaces. Bypassing the Skull Barrier via Endovascular Sensing (6:31) Capturing high-fidelity human neural data historically forced a severe trade-off between invasiveness and signal quality. Scalp electroencephalography (EEG) avoids surgical risk, but the cranium acts as a physical low-pass filter, permanently attenuating the information-rich, high-frequency dynamics. Conversely, recording single-unit action potentials via penetrating cortical microarrays across the brain's 86 billion neurons presents intractable thermal, power, and surgical safety barriers. Citing speech decoding breakthroughs from Edward Chang’s laboratory and the BrainGate consortium, Synchron asserts that recording single-neuron spikes is unnecessary to extract complex intent. To bypass open-brain craniotomies, Synchron developed the Stentrode, an electrode array delivered through a catheter up the jugular vein and seated inside the superior sagittal sinus adjacent to the primary motor cortex. The implant senses direct local electrical field potentials—rather than hemodynamic blood flow—through the vessel wall and relays data subcutaneously to a wireless chest transmitter. Synchron maintains a strictly unidirectional sensing architecture, setting aside therapeutic stimulation to minimize clinical and regulatory hurdles while restoring motor agency to patients with paralysis and amyotrophic lateral sclerosis (ALS). Discrete Neural Tokenization and Autoregressive Pre-Training (23:28) To evaluate whether generative scaling laws apply to neural electrophysiology, Synchron tested autoregressive pre-training against traditional supervised decoders. Rather than predicting raw continuous waveforms, which allows models to exploit trivial smoothing shortcuts, the engineering pipeline routes signals through high-frequency filter banks and a proprietary burst-detection algorithm to yield discrete token sequences. The team trained a roughly 500,000-parameter transformer on 47 hours of clean clinical data comprising approximately 17 million tokens. Training followed a curriculum schedule, scaling from single-token predictions at a 10-millisecond horizon up to ten tokens across 100 milliseconds. Despite initial loss spikes at each horizon expansion, the model's loss trended consistently downward, demonstrating generalized learning of neural state transitions. Without task-specific fine-tuning, the pre-trained embedding space cleanly separated intended neural clicks from resting baseline states. Coupled with a 75,000-parameter decoder head calibrated on five minutes of field data, the system executed real-time commands on a commercial iPad, successfully distinguishing intended motor actions from environmental distractions, speech, and movement artifacts. Hybrid AI Workflows and the Boundaries of Cognition (20:02) Traditional BCI calibration relies on labor-intensive supervision by engineering teams, creating a major deployment bottleneck. To make assistive control practical outside rigid laboratories, Synchron collaborated with Apple to integrate BCI inputs into standard Human Interface Device (HID) profiles, allowing implants to emulate native mouse or switch controls. In partnership with Nvidia, Synchron demonstrated a hybrid architecture where external Vision-Language Models (VLMs) and Large Language Models (LLMs) parse spatial environments and suggest context-aware actions, enabling users to execute complex tasks with a single decoded 'brain click.' According to Synchron's Peter Yoo, while third-party multimodal AI currently serves as a context multiplier, the company's long-term objective is to decode situational context directly from upstream neural signals. Yoo delineated clear biological boundaries for this roadmap: Synchron is not pursuing memory consolidation or retrieval, which depend on deeper subcortical structures like the hippocampus, but remains strictly focused on motor restoration. Furthermore, Yoo emphasized that clinical implementations must maintain rigorous artifact rejection, ensuring devices respond strictly to deliberate neural intent rather than ambient biosignals or behavioral noise. While Synchron demonstrates that autoregressive foundation models can decode motor intent from endovascular signals, scaling beyond narrow motor restoration into generalized cognitive primitives depends on assembling vast, time-synchronized neural datasets across diverse environments under stringent clinical trial and privacy constraints.

脑机接口公司Synchron提出，现有人工智能依赖语言等高度压缩的下游产物，而真正的认知基座模型应直接从大脑的生物神经动力学源头构建。该公司借助微创血管内脑机接口采集高保真颅内信号，在恢复瘫痪患者数字自主权的同时，探索通用认知AI的全新表征范式。 从生物源头重构AI：超越下游语言 (0:04) 现有大语言模型依赖人类产出的文本、音频和代码进行训练，但在神经科学视角下，语言属于高度压缩且丢失大量信息的下游产物。人类思维源于自出生起持续运行的递归时空神经放电，感官输入驱动大脑状态演进并生成复杂的内在心语；当这些认知状态经由声带或手指等物理效应器表达时，海量深层信息被过滤抛弃。Synchron主张绕过这一损耗链条，直接接入生物神经这一未压缩的公理化源头。 随着算力设施的商品化、神经科学功能定位的清晰以及自监督深度学习架构的成熟，限制认知模型发展的核心瓶颈转向了规模化的高保真神经数据集。Synchron认为，一旦能够跨更广阔的脑区稳定捕获神经动力学并形成“认知原语”（cognitive primitives），再将其与多模态上下文同步对齐，便能允许系统通过文本或图像等人类常规输入进行调用，从而使脑机接口从针对运动障碍的专用医疗设备拓展为通用认知智能载体。 颅骨滤波壁垒与血管内微创植入方案 (6:31) 获取大规模脑电信号的根本障碍在于人类生理结构。颅骨在物理上充当了严重的低通滤波器，衰减了富含高价值信息的高频神经动态；非侵入性头皮脑电图（EEG）信号存在极大的空间模糊和高频损失，且此类损失无法通过后期算法在数学上逆转。然而，传统开颅手术植入皮层微电极阵列（如BrainGate的穿透式针状电极）风险过高，且人脑包含约860亿神经元，若全面监测单个神经元动作电位需数十万级通道和数万赫兹采样率，会引发热耗、尺寸与临床安全瓶颈。加州大学旧金山分校（UCSF）Edward Chang实验室的皮层脑电（ECoG）研究已证明，无需穿刺脑实质即可实现高精度解码。 为此，Synchron研发了无需开颅的血管内神经植入假体Stentrode。该技术借鉴成熟的心血管介入手术，将微型电极网通过颈静脉微创导管永久锚定于贴近初级运动皮层的上矢状窦内，通过血管壁直接记录局部场电位（直接电生理信号而非血液动力学副产物），并经皮下走线连接至胸部无线发射单元。该设计在避免颅外悬挂线缆的同时完整保留了颅内高频电位，已帮助肌萎缩侧索硬化症（ALS）与脊髓损伤患者通过残存的运动皮层意图生成“脑点击”，重新获得操控电脑与智能手机的数字自主权。 神经信号离散化与自回归预训练解码 (23:28) 为了验证自回归缩放假说在脑电数据中的有效性，Synchron基于临床试验中获取的47小时无污染神经记录（约1700万Token），探索了针对脑电波振荡特性的离散化分词策略。为避免模型利用连续波形的平滑特性产生低效预测，团队通过高频滤波器组提取高伽马等富信息频段，并通过专有能量突发检测算法将连续频谱转换为离散Token序列，随后输入参数量约50万的Transformer架构进行自回归预训练。 预训练采用课程学习策略：从前向预测未来1个Token（10毫秒时间跨度）逐步提升至预测未来10个Token（100毫秒）。训练损失在预测时间跨度扩大时虽有周期性反弹，但整体呈稳定收敛趋势，证明模型习得了神经状态跃迁的底层动态。在下游意图分类中，该预训练嵌入空间无需特定任务微调即可将运动意图与基线自发活动清晰分离。配合一个仅7.5万参数、经5分钟校准即可部署的轻量级解码器头，系统在实际家庭环境中展示了强大的抗噪性：在言语交流、肢体微动及宠物犬吠叫等现实干扰下，解码器仍能准确过滤物理伪迹，精准响应受试者的主观点击意图。 人机闭环平衡与多模态混合AI协同 (18:01) 在真实神经恢复场景中，离线基准测试指标的优化并不等同于临床有效性。脑机交互本质上是人脑与算法的双向适应过程：模型拟合脑电特征，大脑根据实时视觉反馈调整放电模式，两者在闭环交互中寻求动态平衡。因此，系统必须摆脱数十名科研人员服务单例受试者的传统实验室调优模式，在真实日常环境中建立泛化能力。 为了降低神经解码的细粒度负担并拓展控制维度，Synchron与科技生态深度整合。一方面，其与苹果合作将脑机接口配置文件原生接入标准人机接口设备（HID）协议，使植入体无需专用驱动即可直接映射为系统级输入设备；另一方面，与英伟达合作构建空间计算与多模态AI混合架构：利用视觉语言模型（VLM）识别现实场景中的交互目标，结合大语言模型（LLM）生成可能的上下文操作菜单，患者仅需通过基础的神经点击做出确认。这种以环境语境作为意图解码倍增器的混合架构，大幅提升了患者与现实世界的交互效率。 Synchron明确其技术路径集中于单向神经传感与运动意图恢复，而非双向神经电刺激（如DBS）或复杂的记忆整合，旨在降低临床及监管壁垒以实现万级患者规模的部署。发言人Peter Yoo指出，尽管神经缩放定律正在显现潜力，但在受严格隐私和医疗监管约束的临床环境中，采集海量高保真时空对齐脑电数据仍是核心瓶颈；工程团队必须恪守严谨的伦理底线，确保解码核心源于真实的大脑意图，而非依赖外部环境或生理伪迹。

## Background 本视频由 Ai4 于 2025 年 8 月 28 日发布在 YouTube 平台上。内容记录了 Synchron 神经科学与算法高级总监 Peter Yoo 的主题演讲，阐述了 Synchron 如何通过微创脑机接口直接从神经源头构建人类认知基础模型，旨在彻底恢复瘫痪患者的行动自主权并启发下一代通用人工智能系统。 ## TL;DR 演讲指出语言等传统产物仅是大脑思维在物理效应器上的“有损压缩”，要实现人类认知的真正建模，必须直接在神经源头采集未失真的高频时空信号。为解决开颅手术创伤大及头皮脑电低通滤波严重的问题，Synchron 开发了经颈静脉植入运动皮层血管的 Stentrode 支架电极，实现微创全植入式高保真信号采集。团队进一步借鉴大语言模型的自回归预训练方法，将连续脑电离散为神经 Token 进行课程学习，仅用轻量级分类头和极少样本即可精准区分真实意图与假阳性波动。最后，通过融入苹果原生 HID 标准与多模态 AI 智能体，脑机接口正在走出实验室，在真实复杂生活场景中重塑患者的数字自主权。 ## Atmosphere & style 整场演讲呈现出一种兼具深厚人文关怀与硬核工程理性的极客探索氛围。Peter Yoo 的讲解严谨从容、逻辑紧密，从神经电生理底层物理原理平滑过渡到最前沿的生成式 AI 架构，并在播放患者家庭实操录像时展现出令人动容的临床温情。在问答环节中，他幽默风趣且极具科学守门人的清醒边界感，既描绘了认知基础模型的宏大愿景，又时刻坚守以纯净中枢神经信号为核心的工程底线，令整场分享既前瞻宏伟又扎实落地。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=4s) Synchron 的核心愿景是从神经源头直接构建人类认知基础模型，避免传统语言与物理制品等下游“有损压缩”带来的信息损失。 - [01:19](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=79s) 人类认知源于大脑内部的递归神经级联放电，语言只是通过肌肉和声带等物理效应器输出的下游表征，现有大语言模型依赖这些有损数据学习存在天然瓶颈。 - [03:08](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=188s) 脑机接口演进路径从运动控制起步，未来将拓展至解析全脑弥散的“认知基语”，并结合多模态数据演进为通用的认知 AI。 - [05:10](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=310s) 构建认知基础模型的最大障碍不是算法或算力，而是缺乏达到基础模型规模、高信噪比且具备丰富上下文的大脑神经数据集。 - [06:31](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=391s) 颅骨是天然的强低通滤波器导致头皮脑电（scalp EEG）高频信号永久丢失；而传统开颅手术风险高，限制了侵入式脑机接口的大规模普及。 - [09:31](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=571s) 语音解码实践证明无需极致记录全脑单神经元脉冲，皮层脑电等适度分辨率即可满足解码需求，关键在平衡信息保真度与安全性。 - [12:46](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=766s) Synchron 采用血管介入技术，通过颈静脉将 Stentrode 支架电极送入运动皮层邻近静脉，实现无开颅、全植入、无外露线缆的高质量神经信号采集。 - [15:29](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=929s) 通过有监督学习训练轻量解码模型，瘫痪患者能够凭借构想运动产生的神经意图（如脑控点击）操控苹果 iOS 系统，重获数字生活自主权。 - [18:01](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1081s) 脑机接口是人与算法相互适应的实时闭环系统，必须跳出实验室人工测试任务，在真实世界的广泛场景中积累高上下文数据以提升模型泛化能力。 - [21:13](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1273s) 通过与英伟达 XR、视觉语言模型协同以及与苹果建立原生 HID 规范，BCI 正转化为即插即用的通用输入外设，无需患者脑内解码所有动作细节。 - [23:28](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1408s) 研究团队通过高频滤波与能量突发检测将连续脑电离散化为 Token，并采用课程学习进行自回归预训练，证明 Transformer 能泛化学习脑状态演化规律。 - [26:49](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1609s) 仅需 7.5 万参数解码头和 5 分钟标定数据，系统即可在笔记本端实时推理，精准区分有意脑点击与无意生理波动，并在高噪音家庭环境中稳定运行。 - [32:57](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1977s) Synchron 当前坚持单向“传感与解码”而非双向电刺激干预，直接采集血管邻近皮层神经元的局部场电位，以最大限度保障安全性和规模化落地。 - [35:33](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=2133s) 环境上下文是 BCI 性能的巨大倍增器，结合外部 UI 智能体可将单一的神经运动脉冲高效映射为复杂界面操作，但多模态融合必须守住纯大脑信号驱动的底线以避免伪影干扰。 - [37:04](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=2224s) 团队研发聚焦于运动皮层的运动功能恢复而非海马体记忆存储，旨在利用通用 AI 范式直接从神经电活动中提取高维运动与认知意图。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=4s) 从神经源头构建人类认知基础模型 脑机接口公司 Synchron 的核心愿景，是从“神经源头”直接构建人类认知的基础模型（Foundation Models of Human Cognition）。通过让 AI 模型直接从驱动人类思维与运动的大规模、高质量脑电信号中学习，不仅能够彻底恢复瘫痪患者丧失的行动自主权，还将为未来构建更对齐、更具泛化能力的通用人工智能系统提供全新路径。 在传统认知框架中，语言以及人类制造的各种物理构件（Artifacts）在表征人类认知时存在天然局限性，它们充其量只是大脑内部复杂思考过程在下游产生的“有损压缩表示”（lossy compressed downstream representation）。相比之下，神经活动本身才是所有认知能力的真实发源地。通过抽象出大脑这一原始智能载体处理信息的方式，不仅能为脑机接口（BCI）注入强大动力，更将重塑未来 AI 系统学习思考与行动的底层机制。 ### [01:19](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=79s) 从感官刺激到物理效应器：语言作为“有损压缩” 人类大脑在任何时刻都在接收海量的外部信息输入。来自物理世界、环境法则乃至他人思想的刺激信号，通过感觉系统涌入大脑并转化为神经活动，进而汇入自出生以来便持续不断运行的递归神经级联（recursive neural cascade）之中。这种输入不断校准着大脑内部高速运转的时钟状态：当前的脑状态驱动下一时刻的脑状态，神经元在时间和空间维度上的放电动态（spatiotemporal dynamics）构成了所谓的神经信号，并催生出人类的内在独白、思考与行为能力。 ```mermaid flowchart TD accTitle: 大脑信息处理与有损输出流程 subgraph S1["神经信息产生"] A["外部世界刺激<br/>物理规律 / 他人思想"] --> B["感觉系统转化<br/>时空神经动态放电"] B --> C["内部认知表征<br/>递归神经级联 / 思维独白"] end subgraph S2["传统输出瓶颈"] C --> D["物理效应器<br/>手指 / 声带肌肉"] D --> E["有损压缩产物<br/>文本 / 语音 / 人工制品"] E --> F["现有基础模型<br/>LLM 学习的数据源"] end subgraph S3["Synchron 路径"] C -.->|"上游直接捕获<br/>无损神经信号"| G["认知基础模型<br/>Cognitive Foundation Model"] end ``` 当这些神经信号试图转化为现实行动时，必须经由人体的“效应器”（Effectors）——如手指的肌肉或喉部的声带。在大脑内部生成的丰富信息通过这些物理结构输出到外部世界的过程中，原始信息经历了剧烈的过滤与压缩。目前的大语言模型只能依赖这些被高度压缩后的下游产物进行学习；而 Synchron 提出的核心设想是：如果能开辟一条通用通路，直接在神经信号生成语言的上游阶段完整捕获它们，AI 与人类智能的连接方式将被彻底改写。 ### [03:08](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=188s) 脑机接口的演进路径与认知图灵转移 在工程实践中，只要能充分捕获神经活动，便可以建模并解码产生特定认知功能的大脑过程。当前 Synchron 的切入点是运动控制（Motor Control）。BCI 系统通过充分采集原始神经信号并训练解码模型，使用户能够通过输入神经意图直接产生数字行为。公司当前最紧迫的使命，是让所有因肌萎缩侧索硬化症（ALS）或脊髓损伤等原因导致瘫痪的患者，都能安全、规模化地使用这项技术重获生活自理能力。 向未来更进一步，如果能够从全脑更多弥散区域和更广泛的语境中持续捕获原始神经信号，研究人员将有能力建模超越单纯运动控制的高阶复杂认知能力。这些被解析出来的“认知基语”（cognitive primitives）不仅能帮助患者更自由地表达自我，更可在未来与同步记录的多模态数据集相对齐。届时，这些认知基语甚至可以通过人类可理解的文本或图像输入进行调用，标志着“认知 AI”（Cognitive AI）正式跨越 BCI 的单一医疗领域，转型为通用的底层智能技术。 ### [05:10](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=310s) 基础设施已备，唯缺“基础模型级”大脑数据 实现人类认知基础模型的几乎所有先决条件在历史上首次全部就绪。计算与机器学习基础设施已经高度商品化；经典神经科学不断揭示驱动各项认知能力的大脑机制；而现代无监督/自监督 AI 架构展现出了即使在缺乏人工标签的情况下，也能从海量数据中抽象出复杂认知表征的强大能力。 然而，整个拼图中唯一缺失的关键要素，是达到“基础模型规模”（Foundation Model Scales）、具备高信号质量与丰富上下文的宏重大脑数据集。这类数据在过去完全不存在，其核心技术瓶颈并不在于算法，而在于神经信号采集方式的物理与生理限制。 ### [06:31](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=391s) 颅骨的低通滤波效应与开颅手术的物理困境 想要获取高信噪比的脑电信号，传感器就必须尽可能贴近信号源——即大脑皮层表面甚至单个神经元。但这在传统医学中意味着必须进行开颅手术（Open Brain Surgery）。开颅手术伴随着巨大的手术创伤与临床感染风险，且高度依赖顶尖神经外科医生和复杂的医疗基础设施，根本无法实现大规模人群普及。 ```mermaid flowchart LR accTitle: 颅骨滤波与信号采集路径对比 subgraph S1["非侵入式采集"] A["脑皮层神经信号"] -->|"穿透颅骨<br/>高频衰减"| B["头皮脑电 scalp EEG<br/>严重低通滤波 / 动态丢失"] end subgraph S2["传统侵入式采集"] A -->|"开颅切除骨瓣<br/>高风险 / 难规模化"| C["ECoG 皮层电极 /<br/>脑内穿刺微针阵列"] end subgraph S3["血管内微创采集"] A -->|"利用静脉天然通道<br/>无需开颅"| D["Synchron 血管支架电极<br/>高频时空动态完好保留"] end ``` 从物理学与信号处理的角度来看，人类历经亿万年进化用于保护大脑的颅骨，本质上是一个极强的“低通滤波器”（Low-pass Filter）。大脑神经信号的功率谱严格遵循幂律分布（Power Law），这意味着携带丰富信息量的高频时空动态在穿透颅骨到达头皮时被大幅滤除。传统的无创头皮脑电图（scalp EEG）因而丢失了绝大部分关键信息，且这种信息损失在数学和物理上是不可逆的；相比之下，只有穿过颅骨进入颅内（Intracranial），才能在时频谱图（Spectrogram）上清晰观测到患者在静息或尝试书写不同字母时所爆发出的复杂时空动态模式。 - 头皮脑电（scalp EEG）完全无创，但由于颅骨强烈的低通滤波效应，高频信息被完全抹除。 - 颅内信号（Intracranial）能够清晰展现高频神经放电特征，静息状态与尝试运动状态在时频谱上有显著差异。 - 信息一旦在物理介质传播中衰减丢失，下游算法便无法凭空恢复，从而极大限制了解码模型的学习上限。 ### [09:31](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=571s) 采样尺度的工程权衡与前沿语音解码实践 面对信号分辨率的挑战，直觉上的工程思维可能是追求极致——直接采集全脑 860 亿个神经元的单单神经元放电脉冲（Spikes）。然而这种方案在临床转化中面临严峻的现实壁垒： 1. 剥离大面积颅骨并穿刺全脑组织对患者而言极不安全； 2. 即使只记录其中一小部分神经元，也需要数十万个高密度传感器并塞入极小体积的植入物中供电； 3. 单通道脉冲检测需要数十 kHz 的采样率，会瞬间引发数据传输与算力的“维度灾难”（curse of dimensionality）。 真正的工程突破在于找到满足特定信息解码需求的“适度分辨率”（adequate resolution）。加州大学旧金山分校（UCSF）Edward Chang 团队在语音解码领域证明，通过放置在大脑表面的非穿刺皮层脑电阵列（ECoG），解码大脑控制喉头与声带肌肉产生音素（Phonemes）的神经动态，就能在算法辅助下实时重建词级语音。随后，BrainGate 联盟（包括斯坦福大学已故科学家 Krishna Shenoy 及 Jaimie Henderson 实验室）利用微针刺入式阵列（penetrating array，即 Utah 针床电极）虽略微提升了性能，但本质上两者解码的是同一范畴的信息。这表明在不同尺度上均可捕获到关键神经信息，核心在于选择兼顾信息保真度、患者安全与规模化推广潜力的技术路径。 ### [12:46](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=766s) 血管内微创植入：利用脑血管天然通道捕获颅内信号 为了绕开开颅手术这一阻碍 BCI 规模化应用的最大障碍，Synchron 开发了一种无需开颅即可采集高质量颅内脑电信号的血管介入技术。大脑血管网络如同一条天然铺设的信息高速公路，负责运送养分、排出废物并深入到运动皮层等大脑核心功能区域。 ```mermaid flowchart LR accTitle: Synchron 血管内植入与无线遥测架构 A["颈静脉穿刺入路<br/>Catheter Jugular Vein"] --> B["微导管沿静脉窦向上推送<br/>Venous Drainage System"] B --> C["运动皮层邻近静脉定植<br/>血管支架电极网 Stentrode"] C --> D["皮下皮线连接胸部遥测装置<br/>Chest Telemetry Unit"] D --> E["体外无线传输<br/>完全无外露线缆 / 恢复生活常态"] ``` 医生仅需通过微创导管技术，将支架式神经电极（Stentrode）从患者颈部的颈静脉（Jugular Vein）导入，沿着天然的脑静脉引流系统逆行推送到大脑运动皮层上方的血管内定植。电极采集到的颅内神经信号通过皮下连接传输至埋植在胸部的无线遥测装置（Telemetry Unit）并发射到外部设备。整套系统在患者体表没有任何外露线缆，外观与常人无异。这种介入方式完美复刻了现代心脏病学中心脏支架替代开胸手术、以及神经介入中支架取栓治疗中风的技术演进路线，使 BCI 的临床部署门槛与手术普及性实现了量级上的跃升。 - 手术路径：经颈静脉穿刺入路，借助导管穿行至运动皮层附近的脑静脉血管。 - 安全与美观：全植入式设计，胸部无线发射数据，无外露线缆，极大降低感染几率。 - 临床类比：对标心血管支架介入手术（PCI）与缺血性中风支架取栓手术，具有极高的临床可复制性。 ### [15:29](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=929s) 重塑瘫痪患者的数字自主权与有监督解码工作流 在肌萎缩侧索硬化症（ALS）或高位脊髓损伤患者体内，其大脑运动皮层内部构想动作时的神经时空放电机制依然完好，患者仍然保有“尝试运动”的意图，但由于下行神经通路受阻，这些富含信息的信号无法传递至肢体末端效应器。在数字时代，丧失操作智能手机与电脑的能力，实质上意味着被彻底剥夺了现代社会的独立生活与沟通自主权。 Synchron 研发的首款运动脑机接口产品正是为了打破这一困境。通过捕获患者尝试运动时产生的神经信号，系统能够将其转化为数字指令（如“脑控点击” Brain Click）。一名四肢瘫痪且失去语言能力的 ALS 患者，借助该植入系统与苹果 iOS 原生的“切换控制”（Switch Control）功能，即可无障碍浏览应用并完成日常交互任务。在实际落地流程中，团队首先让患者跟随屏幕光标尝试移动手部以收集带有标签的时空神经数据，随后利用经典的监督学习（Supervised Learning）算法训练实时解码模型，通过不断降低损失函数，逐步将患者的神经放电意图稳定、精准地映射为数字世界的操控行为。 ### [18:01](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1081s) 闭环交互与从实验室走向广阔场景 在脑机接口（BCI）领域，单纯在离线基准测试上看到损失函数（loss）下降并没有太大意义。运动恢复类脑机接口必须让真实患者处于控制闭环之中（User-in-the-loop）。正如俗话所说“一个巴掌拍不响”，演讲者风趣地指出“两个人才能跳探戈，两个人才能玩转脑机接口”（"It takes two to tango and it takes two to BCI"）。在实际系统中，模型首先拟合大脑信号，随后大脑再去适应模型，双向反复磨合直至达到平衡，最终实现平滑的实时控制。现场展示的患者 Iron 在经过系统训练后，仅凭在大脑中想象或尝试左右移动手部，就能以相当高的准确率操纵屏幕上的光标。 ```mermaid flowchart LR accTitle: 脑机接口闭环磨合与大模型认知演进 subgraph ClosedLoop["实时人机闭环 (Human-in-the-Loop)"] direction TB A["大脑神经意图"] --> B["算法模型解码"] B --> C["光标/设备实时反馈"] C --> A end subgraph ParadigmShift["应用范式演进"] direction TB D["传统实验室任务\n(狭窄上下文/单任务)"] --> E["真实世界全场景\n(丰富认知/复杂交互)"] end ClosedLoop --> ParadigmShift ``` 然而，要构建真正的人类认知基础模型（Foundation Model of Human Cognition），仅仅拥有血管内介入式 BCI 带来的海量数据还不够，更需要让设备在丰富且广泛的生活场景中被高频使用。早期脑机接口受限于硬件与传感器研发的高难度，大部分研究被局限在实验室环境中。在狭窄的人工测试任务下，患者采集到的数据上下文极其有限。如果目标是理解人类认知的全貌，患者就必须像普通人一样，在日常真实世界中自由使用设备。 - 传统 BCI 研发模式往往是一大群博士与临床工程师服务一名患者，模型训练极度依赖人工干预，缺乏通用性与可扩展性。 - 规模化落地的核心在于提升模型本身的泛化能力，使患者能够自主完成校准与训练，或直接使用跨患者、跨动作泛化的通用模型。 --- ### [21:13](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1273s) 跨界协同：XR、视觉语言模型与通用硬件标准 为了让患者在更广泛的场景中调用复杂的认知能力，Synchron 积极探索与其他前沿技术的协同集成，避免让脑机接口承担所有繁重的计算任务。例如，团队与英伟达（Nvidia）合作，探索 AI 与 XR（扩展现实）如何帮助患者将控制能力从虚拟数字屏幕延伸到物理真实世界。 ```mermaid flowchart LR accTitle: 混合式多模态脑机接口与通用生态集成 subgraph HybridAI["混合多模态辅助系统 (Nvidia XR 实验)"] direction TB H1["物理世界环境"] --> H2["视觉语言模型 (VLM)\n识别可交互物体"] H2 --> H3["语言模型 (LLM)\n预测候选意图动作"] H3 --> H4["BCI 点击确认\n低负荷触发执行"] end subgraph Ecosystem["通用外设生态 (Apple 合作)"] direction TB E1["介入式 BCI 信号"] --> E2["原生 HID 协议规范"] E2 --> E3["即插即用接入 iOS/macOS\n免专有驱动与转接"] end HybridAI -.-> Ecosystem ``` 在演示中，患者 Ronnie 置身于 XR 环境中，视觉语言模型（VLM）实时检测视野中可交互的实体目标，大语言模型（LLM）则根据场景预测并提供可能的动作候选。患者无需在脑海中解码极其精细复杂的每一个动作细节，只需借助这种混合多模态系统注视目标并给出高层确认，即可完成现实环境中的交互。这使患者能够从狭窄的电脑光标控制中解脱出来，参与更高级、更复杂的日常认知与决策。 - **Apple HID 标准集成**：Synchron 与苹果合作，在操作系统底层建立了针对 BCI 的专属人机接口设备（HID）规范。 - **即插即用愿景**：这项标准的建立意味着 BCI 正迈向类似键盘与鼠标的通用输入外设地位。患者接入现代数字生态无需安装特制驱动或复杂转接软件，彻底消除了使用壁垒。 --- ### [23:28](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1408s) 生成式预训练与神经信号 Token 化实验 在 AI4 大会上，虽然整个 AI 行业都在积极拥抱“规模法则（Scaling Laws）”，但大语言模型中的自回归抽象假设（Large Autoregressive Abstraction Hypothesis）是否同样适用于连续、高噪声的脑电信号，在神经科学界仍未有定论。团队提出核心假设：生成式自回归预训练是否比传统的监督学习更能捕捉大脑中的复杂认知表征？ ```mermaid flowchart TD accTitle: 连续神经电信号的 Token 化与自回归预训练流 subgraph Tokenization["神经信号离散 Token 化"] direction TB T1["血管内原始连续脑电\n(高频振荡信号)"] --> T2["高频带通滤波器组\n(提取富信息频段)"] T2 --> T3["能量突发检测 (Burst Detection)\n(专有特征离散化)"] T3 --> T4["离散 Token 序列\n(10ms 时间步长)"] end subgraph Pretraining["课程自回归预训练"] direction TB P1["轻量级 Transformer\n(~50万参数)"] --> P2["预测未来 1 个 Token\n(10ms，难度极低)"] P2 --> P3["逐步递增预测窗口\n(20ms -> 50ms)"] P3 --> P4["目标预测未来 10 个 Token\n(100ms 脑状态演变)"] end Tokenization --> Pretraining ``` 为了验证这一假设，团队从植入患者身上收集了 47 小时的纯净神经信号，经过离散化处理后转化为约 1,700 万个神经 Token。由于原始神经信号具有连续振荡特性，若直接进行简单的下一采样点预测，模型很容易学会利用波形连续性走捷径（"cheap tricks"）来压低损失值。为此，团队构建了一套专有的信号处理与离散化管线： 1. 将原始神经电信号输入多通道高频滤波器组，分离出富含运动与认知信息的高频频段； 2. 利用专有的能量突发检测算法（Burst Detection），捕捉特定频段上的离散能量脉冲； 3. 将连续波动转化为结构化的离散 Token 序列，进而无缝适配成熟的 Transformer 架构。 在模型训练上，团队采用了基于课程学习（Curriculum Learning）的渐进式预训练机制。首先让模型预测未来 10 毫秒（1 个 Token）的脑状态，当模型掌握后再逐步增加难度，外推预测 20 毫秒、30 毫秒直至 100 毫秒（10 个 Token）后的神经表征。虽然在加大预测步长初期损失曲线会阶段性飙升，但随后持续稳步收敛，证明模型切实学到了大脑状态演化的泛化规律。 --- ### [26:49](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1609s) 解码意图：区分真实意图与无意脑点击 在完成生成式自回归预训练后，关键问题在于：这些下降的损失值是否能转化为患者实际获益的临床控制性能？在实际生活中，最大的挑战之一是如何区分患者“有意触发的脑点击”与“无意的自发大脑电活动波动”（即降低误触率 / False Positives）。 ```mermaid flowchart LR accTitle: 意图解码分类器架构与临床验证 subgraph InputData["特征输入源 (极小样本)"] I1["自回归预训练 Embedding\n(~50万参数骨干)"] I2["未来 100ms 预测 Token\n(10 个时间步外推)"] end subgraph DecoderHead["轻量级实时解码头"] D1["75k 参数分类器\n(仅需 5 分钟标定数据)"] end subgraph OutputState["意图判决与设备控制"] O1["抑制无意误触 (Suppress)"] O2["放行双击/单击意图 (Pass)"] end InputData --> DecoderHead DecoderHead --> OutputState ``` 在临床试验中，患者需要反复进行枯燥的“命令控制测试”（Command Control Test）：听到“休息”指令时保持静止、不得产生脑点击；听到“点击”指令时尽快触发动作。利用这类数据，研究人员将预训练模型的表征进行降维分析，发现模型在未经显式监督微调的潜在嵌入空间中，已经自然将有意点击与无意波动清晰地分离开来。 - **极小参数与边缘推理**：构建的意图解码头仅有约 7.5 万参数（预训练骨干网络约 50 万参数），仅需 5 分钟的患者现场校准数据即可完成适配，能够完全在普通消费级笔记本电脑上实现低延迟实时推理。 - **家庭真实环境演示**：现场展示了患者 Mark 在家中使用 iPad 的真实录像。Mark 仅凭脑电双击动作唤出 iOS 快捷菜单，通过单击遍历选项打开股票应用。在整个过程中，临床工程师在旁实时交谈指导，背景中有宠物狗叫声，Mark 本人也边说话边做小动作，但解码器精准过滤了所有谈话与动作带来的假阳性干扰，仅放行真正有意图的指令。 - **意图与运动伪影的科学辨析**：演讲者严谨地指出，假脑点击可能源于生理伪影，但更重要的是意图上下文的差异——伸出手臂去点击屏幕和伸出手臂去摸身边的狗，其底层的运动执行程序可能完全相同，但背后的意图截然不同。生成式预训练能够帮助模型剥离运动本身，精准捕捉这种高层认知意图。 - **神经规模法则展望**：演讲者引用了 NeurIPS 上发表的 BRAN 论文等多项前沿成果，指出随着模型规模、数据量和算力的增长，神经数据解码正在展现与语言模型相似的 Scaling 曲线，这标志着神经科学与生成式 AI 的结合正迎来历史性转折点。 --- ### [32:57](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1977s) 问答环节：传感与刺激的取舍、直接神经电信号采集 在演讲后的问答互动中，现场观众就技术细节提出了两个核心问题：系统是否具备双向交互能力（即不仅感知神经活动，还能向大脑写入信息/施加电刺激）？以及血管内电极采集到的究竟是直接的神经电信号，还是基于血流动力学的间接指标（如 fMRI 中的 BOLD 信号）？ 针对双向交互与神经刺激问题，Peter Yoo 明确解释道，虽然脑机接口技术本身可以做成双向系统，比如目前用于帕金森病等难治性疾病的深部脑刺激（DBS），但 Synchron 当前阶段完全聚焦于单向“传感与解码”（Sensing and Decoding）。主动向大脑施加电刺激属于治疗干预手段，伴随更高的临床与安全风险；而团队的核心使命是通过恢复自主行动能力惠及数百万人，保持单向传感不仅能最大程度保证安全性，更是实现技术大规模商业化与临床普及的关键路径。 针对信号本质的疑问，Peter Yoo 澄清，Synchron 采集的绝非血流动力学等间接生理信号，而是直接的神经电活动。虽然 Stentrode 支架电极置于脑血管腔内，但由于生物电在组织中具有良好的导电性，电极能够直接感应并记录血管壁相邻皮层神经元集群所释放的微弱局部场电位（Local Field Potentials）。随后，观众针对解码空间中的先验概率机制（类似大语言模型中的 Beam Search）展开了进一步探讨。 ### [35:33](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=2133s) 上下文理解作为脑机接口的倍增器与智能体集成 观众就环境上下文的合理性提出讨论：例如当人坐在办公室里时，“抓取一支笔”显然比“抓取一支麦克风”具有更高的情境合理性。Peter Yoo 幽默地回应道：“我完全明白你的意思，我甚至怀疑你是不是偷偷在我们办公室装了窃听器。”他强调，环境上下文（Context）是脑机接口（BCI）性能释放的巨大倍增器。在之前展示的受试者罗德尼（Rodney）的 AI 辅助脑机接口案例中，团队正是将上下文的解析工作剥离并下放给了一个 AI 模型。通过这种方式，即使大脑仅输出一个极为简单的单次运动指令（例如一次“脑力点击”），系统也能结合当前上下文动态将其映射为最合理的实际操作。 团队的长期愿景是直接从大脑信号内部完整解析出这些上下文信息，因此从一开始就极其严格地构建高质量的数据集。Peter 指出，在临床试验和强监管环境下，采集与特定现实事件高度同步的大脑数据集具有极高难度，必须兼顾患者隐私与合规性。此外，他对目前新兴的 AI 智能体（Agentic Models）表现出浓厚兴趣——这类模型能够智能判断在特定计算机界面中哪些元素应该被点击、哪些不该被点击。Synchron 既在推进自己的算法，也乐于将此类外部成熟的智能体方案作为插件接入自身的神经接口系统中。 ```mermaid flowchart TD accTitle: 结合上下文与AI智能体的脑机接口映射架构 subgraph S1["大脑信号采集与解码"] A["大脑运动皮层指令 (如单次脑力点击)"] B["高精度神经数据集 (临床试验同步标注)"] end subgraph S2["上下文感知与AI智能体"] C["环境/界面上下文模型 (Context Model)"] D["Agentic UI 控制智能体 (自动识别可操作元素)"] end subgraph S3["最终执行端"] E["精准上下文映射动作 (如点击特定按键/抓取目标)"] end A --> E B --> C C --> D D --> E ``` - **上下文的乘数效应**：将环境理解交由 AI 处理后，单一维度的神经运动信号可根据当前屏幕状态转化为复杂指令。 - **高壁垒数据构建**：与真实事件完全同步的脑电数据极为稀缺，必须在严格受监管的人体临床试验中逐步积累。 --- ### [37:04](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=2224s) 专注运动功能恢复而非记忆存储与深脑修复 现场观众提及“蓝海战略”并提问 Synchron 如何存储或恢复大脑的记忆。Peter 明确澄清，记忆的存储与恢复并不是 Synchron 当前的研究方向。从神经科学的角度来看，记忆巩固（Memory Consolidation）主要依赖于大脑深部结构（如海马体及周边皮层），而 Synchron 目前植入电极覆盖的区域和研发重点完全聚焦于**运动功能恢复**（Motor Restoration）。 针对观众关于“是否会在外部存储大脑信号”的追问，Peter 阐明了团队对神经数据的核心理解：所有关于运动意图和认知维度的信息都已经内嵌于采集到的神经信号本身，关键在于如何利用算法将其精准提取出来。当下人工智能技术正处于通用化与普惠化的拐点，这种通用模型在各类不同模态的数据上均展现出极强的特征提取能力，Synchron 正在验证这一套底层 AI 范式在大脑信号解码中的有效性。 - **解剖定位差异**：运动功能恢复侧重运动皮层，而记忆巩固涉及以海马体（Hippocampus）为核心的大脑深部网络。 - **信号提取逻辑**：神经信息本身已高度蕴含于微弱的电生理活动中，无需外置构建记忆库，而是依靠通用 AI 模型进行特征解构。 --- ### [38:20](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=2300s) 多模态数据融合的边界与避免“伪影计算机接口” 在回答关于是否会将脑电信号与眼动追踪（Eye Tracking）、体温、穿戴设备等其他模态数据融合的提问时，Peter 确认这些都属于广义的“上下文”范畴。在理想状态下，研究团队希望尽可能多地为大脑信号打上相关的环境与生理标签——甚至连当天的天气都可能潜移默化地影响一个人的思维与神经状态。然而在实际工程与科研中，必须“挑选有价值的战场”（Pick your battles），例如股市的涨跌显然对解析患者当前的运动神经状态没有直接价值。 Peter 强调，尽管将可穿戴设备和生物信号进行多模态对齐是非常有价值的方向，但 Synchron 的核心底线是确保其技术必须由**纯粹的大脑信号**所驱动。研发团队必须扮演好严格的技术守门人角色，确保构建出来的是货真价实的“脑机接口”（Brain-Computer Interface），而不是被肌肉颤动、眼动或环境杂波误导的“伪影计算机接口”（Artifact-Computer Interface）。 - **上下文标注的取舍**：优先融合与认知、意图强相关的生理及环境指标，剔除无关宏观噪音。 - **核心原则防线**：坚决避免外部噪音或生理伪影（Artifacts）主导系统判断，确保输入信号源于真实的中枢神经活动。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=4s) 语言以及人类制造的各种物理构件在表征人类认知时存在天然局限性，它们充其量只是大脑内部复杂思考过程在下游产生的‘有损压缩表示’。 — Peter Yoo > [01:19](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=79s) 如果能开辟一条通用通路，直接在神经信号生成语言的上游阶段完整捕获它们，AI 与人类智能的连接方式将被彻底改写。 — Peter Yoo > [06:31](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=391s) 从物理学与信号处理的角度来看，人类历经亿万年进化用于保护大脑的颅骨，本质上是一个极强的‘低通滤波器’。 — Peter Yoo > [06:31](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=391s) 信息一旦在物理介质传播中衰减丢失，下游算法便无法凭空恢复，从而极大限制了解码模型的学习上限。 — Peter Yoo > [18:01](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1081s) It takes two to tango and it takes two to BCI（两个人才能跳探戈，两个人才能玩转脑机接口）。 — Peter Yoo > [26:49](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1609s) 假脑点击可能源于生理伪影，但更重要的是意图上下文的差异——伸出手臂去点击屏幕和伸出手臂去摸身边的狗，其底层的运动执行程序可能完全相同，但背后的意图截然不同。 — Peter Yoo > [32:57](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=1977s) 保持单向传感不仅能最大程度保证安全性，更是实现技术大规模商业化与临床普及的关键路径。 — Peter Yoo > [35:33](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=2133s) 我完全明白你的意思，我甚至怀疑你是不是偷偷在我们办公室装了窃听器。 — Peter Yoo > [37:04](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=2224s) 所有关于运动意图和认知维度的信息都已经内嵌于采集到的神经信号本身，关键在于如何利用算法将其精准提取出来。 — Peter Yoo > [38:20](https://www.youtube.com/watch?v=CDVyui6M3Ss&t=2300s) 研发团队必须扮演好严格的技术守门人角色，确保构建出来的是货真价实的‘脑机接口’，而不是被肌肉颤动、眼动或环境杂波误导的‘伪影计算机接口’。 — Peter Yoo ## Mentioned Peter Yoo, Synchron, Stentrode, Ai4, Apple, iOS, macOS, Apple HID, Switch Control, Nvidia, UCSF, Edward Chang, BrainGate, Krishna Shenoy, Jaimie Henderson, NeurIPS, BRAN, Iron, Ronnie, Mark, ALS, ECoG, Utah Array, Transformer ## Video description Peter Yoo, Senior Director, Neuroscience and Algorithms, Synchron Abstracts Synchron's vision is to build foundation models of human cognitions directly from the source, to help people communicate their needs that are stuck inside of their bodies due to paralysis. Letting the models learn from brain signals that underly how we move and think, could completely restore the autonomy of our paralyzed patients. Furthermore, we believe it will inform how we can build more aligned and capable AI systems in the future. Join to find out how Synchron's minimally-invasive neural interface technology could enable this vision in our lifetime. [Source](https://www.youtube.com/watch?v=CDVyui6M3Ss)

[0:04] Peter Yoo: 语言以及人类制造的各种物理构件在表征人类认知时存在天然局限性，它们充其量只是大脑内部复杂思考过程在下游产生的‘有损压缩表示’。 [1:19] Peter Yoo: 如果能开辟一条通用通路，直接在神经信号生成语言的上游阶段完整捕获它们，AI 与人类智能的连接方式将被彻底改写。 [6:31] Peter Yoo: 从物理学与信号处理的角度来看，人类历经亿万年进化用于保护大脑的颅骨，本质上是一个极强的‘低通滤波器’。 [6:31] Peter Yoo: 信息一旦在物理介质传播中衰减丢失，下游算法便无法凭空恢复，从而极大限制了解码模型的学习上限。 [18:01] Peter Yoo: It takes two to tango and it takes two to BCI（两个人才能跳探戈，两个人才能玩转脑机接口）。 [26:49] Peter Yoo: 假脑点击可能源于生理伪影，但更重要的是意图上下文的差异——伸出手臂去点击屏幕和伸出手臂去摸身边的狗，其底层的运动执行程序可能完全相同，但背后的意图截然不同。 [32:57] Peter Yoo: 保持单向传感不仅能最大程度保证安全性，更是实现技术大规模商业化与临床普及的关键路径。 [35:33] Peter Yoo: 我完全明白你的意思，我甚至怀疑你是不是偷偷在我们办公室装了窃听器。 [37:04] Peter Yoo: 所有关于运动意图和认知维度的信息都已经内嵌于采集到的神经信号本身，关键在于如何利用算法将其精准提取出来。 [38:20] Peter Yoo: 研发团队必须扮演好严格的技术守门人角色，确保构建出来的是货真价实的‘脑机接口’，而不是被肌肉颤动、眼动或环境杂波误导的‘伪影计算机接口’。

[6:31] Peter Yoo: a no-brainer, pun intended [6:31] Peter Yoo: bed of nails [12:46] Peter Yoo: brain click [18:01] Peter Yoo: it takes two to tango and it takes two to BCI. [35:33] Peter Yoo: I wonder if you've been listening into our conversations in our office... so who are you? [37:04] blue ocean [38:20] Peter Yoo: the state of the stock market probably doesn't really influence what the neural state would be. [38:20] Peter Yoo: brain computer interface, not an artifact computer interface.

[0:04] So, Synchron which is our company uh our [0:07] vision is to build foundation models of [0:10] human cognitions directly from the [0:13] source. [0:15] So, letting the models learn from [0:17] largecale highquality brain signals that [0:19] underly how we move and think uh could [0:22] completely restore the autonomy lost in [0:25] our paralyzed patients who are primarily [0:27] trying to help. [0:29] But furthermore, we believe it'll inform [0:31] how we could build more aligned and [0:33] generalizable AI systems in the future [0:35] as well. [0:37] The notion here is that language and any [0:40] other human artifacts for that matter [0:42] are inherently limited when it to [0:44] representing human cognitions. [0:48] It's acting as a lossy compressed [0:50] downstream representation. [0:53] In contrast, neural activity, or at [0:56] least the eximeatic neuroscientist says, [0:59] is the source of cognitive capabilities [1:01] themselves. [1:03] So we believe abstracting how the [1:05] original source of intelligence, our [1:07] brain, processes information will not [1:10] only supercharge brain computer [1:12] interfaces, but how AI systems can learn [1:14] to think and behave in the future. [1:19] So just to illustrate this, let's you [1:22] know consider the data that existing [1:24] foundation models must learn from. [1:28] At any given moment huge amounts of [1:30] information enter our brain. So through [1:32] our sensory system and these are [1:34] stimuli. [1:38] This is effectively information of the [1:40] world, physics, even ideas from other [1:42] people. These all enter these all [1:45] somehow have to enter our brain and once [1:47] they do they get turned into neural [1:49] activity [1:51] and this influences the cascade of [1:53] recursive neural activity that's been [1:55] going on since effectively when you're [1:58] alive [1:59] and this information that's entered [2:01] through from the external world [2:03] influences our internal clock that's [2:04] churning away. [2:07] So the current activity of the brain [2:09] state causes the next so on and so forth [2:12] they get mended and this dynamics we [2:15] call spatial temporal dynamics of how [2:17] the neurons are firing is effectively [2:20] what we call neural signal [2:23] and these give rise to our cognitive [2:26] capabilities. They don't simply just [2:27] encode but they give rise to behavior [2:30] how we think your internal monologues. [2:35] So some of this neural signals then of [2:37] course you can think but they do tend to [2:39] go down to what's called our aectors. So [2:41] aectors are things that we can affect [2:43] the physical world my fingers my vocal [2:46] cords and somehow this information [2:49] that's come from external world and [2:50] churning inside that you created has to [2:53] go through this out into the real world. [2:56] So through that process a lot of the [2:58] original information is lost. So, but [3:00] they're also compressed. But this [3:03] particular information source is all [3:05] that current models have to learn from. [3:08] But what if there was a general avenue [3:11] to capture the neural signals that [3:13] generated the language themselves and [3:15] everything in between? [3:18] And it turns out that when you can [3:20] capture the neural activity adequately [3:22] enough, you can model and decode some [3:26] narrow range at least some narrow range [3:28] of brain processes that give rise to [3:30] certain cognitive capabilities. Motor [3:33] control and well primarily motor control [3:36] is what we're focusing on and you can [3:38] yield technologies that has the [3:40] potential to drastically improve uh [3:42] people's lives. So brain computer [3:44] interface is one of those technologies. [3:46] So BCIS capture the original signal [3:49] adequately enough so then we can train a [3:52] model to be able to decode that [3:53] information and then users can now input [3:56] their neural signals and get digital [3:58] behaviors out. [4:01] So our immediate goal of synchron is to [4:02] do this so that all motor impaired [4:04] people so people with paralysis for [4:06] whatever reason can have access to this [4:08] technology safely and at scale. [4:13] And just a little bit further forward [4:15] into the future, if you can adequately [4:17] capture more and more of this original [4:19] source, this neural signal from more and [4:21] more diffused areas of the brain in [4:23] wider context, [4:26] then we believe we could model even [4:27] complex human cognitive capabilities [4:30] beyond simple motor control. And [4:32] ultimately, our goal is to make these [4:34] cognitive primitives that we can model [4:36] for our patients so that they can [4:38] express more of themselves and increase [4:40] their autonomy even more. [4:43] But if we do this right and capture the [4:45] adequate accompanying data sets [4:47] synchronized to to the neural data that [4:49] we're recording, then one day we may be [4:51] able to make these cognitive primitives [4:54] be callable through human comprehendable [4:56] inputs, images and texts. [5:01] And when this happens, this is a really [5:02] interesting time. This means that we're [5:04] transferring the cognitive AI that we [5:05] will train from domains of BCI to [5:08] general purpose technology. [5:10] So this all sounds great and actually [5:12] for the first time in history I believe [5:14] nearly every prerequisite to begin this [5:17] work is now in place. As you can see [5:20] computer and machine learning [5:21] infrastructure are now a commodity. Many [5:22] of you are here providing those services [5:24] or be a little bit expensive at [5:26] currently. Neuroscience continuously [5:29] reveal neural mechanisms that give rise [5:31] to cognitive capabilities. So [5:33] neuroscience is a old field and we [5:35] continuously discover how certain [5:38] dynamics of brain activities underly [5:40] certain cognitive capabilities. [5:43] We all know uh with the recent boom [5:45] established AI frameworks to abstract [5:47] information from data sets without [5:50] requiring any labels even complex enough [5:52] ones that yield cognitive like [5:55] capabilities in AI system that you and I [5:57] deal with on a day-to-day basis. For [6:00] whatever reason, this particular [6:01] timeline has it that that method is [6:03] democratized. Well, some of it and many [6:07] people across the world, research labs [6:09] and frontier labs are working on future [6:11] u methods to make this even better. [6:15] But one critical piece has been missing [6:17] which is access to high quality and wide [6:21] contextual data set or brain data set at [6:24] foundation model scales. this data set [6:26] just simply has not existed. [6:31] And the primary reason for that is [6:34] because traditionally if you want high [6:36] quality brain signals, you want to get [6:38] closer to the source, the surface of the [6:40] brain or even down to the neurons. [6:43] And there's no free lunch in this world, [6:44] which means you have to perform an open [6:46] brain surgery. And as you can imagine, [6:50] open brain surgery is very risky. And [6:52] also from a practical perspective, it's [6:54] not a very scalable procedure. There are [6:56] only certain uh neurosurgeons who can [6:58] perform this procedure safely and also [7:01] certain infrastructurees are required to [7:03] be able to perform these open brain [7:04] surgeries. [7:08] And the traditional requirement for open [7:09] brain surgery is to get past the skull. [7:11] Actually, the skull is the main uh enemy [7:14] here, but it's something that evolved [7:16] across eons to protect our brains. So it [7:18] wasn't meant to be you know uh we [7:21] weren't meant to go through it. [7:24] From a physics and signal processing [7:26] perspective skull acts as a lowass [7:28] filter. So filters high frequency [7:30] information and the neural signal that [7:31] I'm talking about happens to follow the [7:33] power law. So the high frequency [7:35] information effectively more dynamics [7:37] that we can capture gets filtered out. [7:39] It's physics. [7:41] So just to illustrate this up the top [7:44] scalp EEG which many of you may be [7:47] already familiar with it's [7:48] electronphilography [7:50] you record from the surface of the [7:52] scalp. So this is completely [7:54] noninvasive. It's got that going through [7:57] and everything below it it's intraranial [8:00] inside of the cranium. So that's gone [8:02] past the skull [8:05] and we don't need to get into the [8:07] details of it. As you can see, even by [8:08] eye, you're starting to see that that [8:10] high frequency information is even [8:12] readily available just by looking at it. [8:15] So, anything underneath it, sorry. And [8:17] as soon as this signal must travel [8:18] through the skull out onto the scalp, a [8:21] lot of this information happens to get [8:23] lost. And once information is lost, it's [8:26] very hard to recover or it's impossible [8:28] to recover it. [8:33] So, just to give you a visual for it, [8:35] this is data from one of our devices. Um [8:37] you can see that on the y-axis we got [8:39] the frequency. Uh many of you might be [8:41] in the audio ML field. So this is this [8:44] is a spectrogram. So you can represent [8:46] brain activity in this spectrogram [8:48] format. So time is going through. I've [8:51] asked the patient to either rest goes a [8:54] little bit quiet. There's some [8:56] spontaneous activity. So it turns out [8:58] even if you're doing nothing your brain [9:00] still ticks away which is a good thing. [9:02] And then when you ask the patient to [9:04] attempt because they're paralyzed to [9:06] write different letters, you're starting [9:08] to see different patterns of activity [9:10] coming out. And this space spatial [9:12] temporal dynamics is the information [9:15] that we're after. [9:17] So as you can see, the more dynamics you [9:20] can capture, the more the model has to [9:22] learn from. But if this gets filtered [9:24] out, then it makes the job of the model [9:26] really difficult. [9:31] So, if such is the case, [9:34] many of you may be thinking that it's a [9:35] no-brainer, pun intended, [9:39] to simply record from simply record that [9:43] neural activity at the finest [9:45] resolution. So, at a single neuron level [9:47] and then derive everything up from [9:49] there. And that's a great engineering [9:51] mindset. That's good first principles [9:52] thinking, but remember, we're building [9:54] this technology for our patients. So [9:56] there's a real person receiving this [9:58] technology. So removing more and more of [10:01] the skull, penetrating more and more of [10:03] the brain surface across your entire [10:06] brain, it's, you know, effectively a [10:08] nogo for our patients. It's not safe. [10:12] And even from a engineering practical [10:14] perspective, it's a huge challenge. [10:17] There's over 86 billion neurons across [10:19] the brain and the cerebellum. Trying to [10:21] record from even a subset of those will [10:23] require hundreds of thousands of [10:24] sensors. You got to pack that into a [10:26] form factor that you can implant. You [10:28] got to power it. Also, if you want to [10:30] detect those spikes, you have to sample [10:33] at tens of kilohz per channel. [10:37] So, cursive dimensionality kicks in. You [10:39] know, you it becomes a very difficult [10:41] challenge. So, the goal is to capture [10:44] this neural signal at adequate enough [10:48] resolutions to capture the information [10:51] that you're after. [10:53] So just to illustrate this, arguably the [10:56] most complex cognitive capability that [10:59] current BCIs can decode in real time. So [11:02] this isn't our work. This is our um our [11:04] colleagues [11:06] is speech production. So don't freak [11:08] out. This isn't semantic thoughts. This [11:11] is your ability your brain's ability to [11:14] coordinate your larynx and your vocal [11:15] cords to produce phone names. And then [11:18] when you decode that information from [11:20] directly from the neural signal and with [11:23] a little help from a model, [11:26] you can begin to decode word level [11:28] outputs directly from brains. And this [11:30] is exactly what the pioneers of this [11:32] speech decoding work showed. So this is [11:34] Eddie Chang's work. You might have heard [11:35] of heard of his work in UCSF. So they [11:38] were able to do this with a surface [11:40] array that I talked about. So they [11:42] implanted a coarser grid on the surface [11:45] of the brain without needing to [11:47] penetrate the tissue itself. [11:50] So you can and then few years later will [11:52] another brilliant BCI lab. So this is [11:55] brain gate consortium um the late schoi [11:57] obviously a great scientist um the [12:00] Henderson lab they're able to up the [12:02] performance by implanting these [12:04] penetrative arrays. So the these are bed [12:06] of nails that you probably would have [12:07] heard of and they've implanted multiple [12:10] of these uh sensors then was able to [12:12] decode categorically the same amount of [12:15] information albeit at a slightly higher [12:18] um performance rate. [12:20] But the point that I'm trying to make [12:21] here is that you can get and sample [12:25] these neural signals that capture the [12:28] information that you're after at various [12:30] scales. So you want to pick the one that [12:33] obviously adequately catches the [12:35] information but that is also good for [12:36] the patient and from our current mission [12:39] goal wanting to build a foundation model [12:41] you want this also to be scalable [12:46] but whether it's an ECoG or a penetrator [12:48] array for now you had to perform an open [12:51] brain surgery so again that's out of the [12:53] question it's not scalable so that [12:55] particular question was something that [12:57] we wanted to address first that's in the [12:58] DNA of our company say synchron [13:01] developed a method to record high [13:03] quality intra cranial signals without [13:07] having to perform open brain surgery. So [13:09] I said there's no free lunch in this [13:10] world. It seems like in this particular [13:12] case you could have your cake needed [13:14] too. [13:15] So the way in which we do that is to go [13:17] through the blood vessels. So blood [13:19] vessels form this lovely natural [13:21] highways into the brain without having [13:24] to directly open the skull. [13:28] So we do this by navigating our device [13:32] through a catheter through the jugular [13:34] vein into the neck and then it travels [13:36] up through the brain with naturally [13:39] occurring drainage system. [13:41] And because blood vessels perform very [13:44] important functions like removing waste, [13:46] delivering energy, reducing pressure, [13:48] all of those good things, they tend to [13:50] go to all the important areas of the [13:53] brain, including the motor cortex, which [13:55] is what we're primarily after. [13:58] So now what you're having is this [13:59] intracranial signal that I talked about. [14:02] So we get past the skull but without [14:04] having to perform an open brain surgery. [14:08] And all of that information gets uh [14:10] sampled by this telemetry in the chest. [14:12] So it gets pumped out uh through [14:14] wireless communication. So from the [14:16] outside perspective um if you looked at [14:18] one of our patients you won't be able to [14:20] see any wires sticking out or anything [14:21] like that. This is is fully um the [14:23] patients were some sense of normaly. [14:28] So this is just to illustrate this. We [14:30] can deliver the device through a [14:33] scalable method using this catheter [14:35] based methods. The analogy that we like [14:38] to draw is from the cardiac space. We [14:40] used to have to perform open brain uh [14:42] open heart surgery to be able to treat [14:45] uh some of the conditions of the heart. [14:48] But now there's a uh transvascular this [14:51] this intravascular minimally invasive [14:53] approach which skyrocketed the [14:55] scalability of cardiac procedures and [14:57] we're doing exactly that in the brain. [15:02] There you go. See? Yeah. And even in the [15:04] neurospace uh most of the stroke [15:07] interventions now treated through this [15:08] method. So some of you probably already [15:10] know a family member who who've probably [15:12] had a stroke and they would have gotten [15:13] one of these procedures the stent B [15:16] based procedures. So that's excess but [15:19] you can see that we can capture [15:21] categorically similar information that [15:23] you would get otherwise from those uh [15:25] directly implantable brain sensors. [15:29] So our initial goal here is to restore [15:33] the lost digital autonomy of paralyzed [15:36] patients to get this technology into as [15:38] many people as possible who need them as [15:40] soon as possible. [15:43] So this particular problem arises from [15:45] various conditions that cause paralysis. [15:47] It could be ALS, spinal cord injury. Um [15:51] several conditions can cause paralysis [15:53] or can lead to paralysis. [15:56] So the problem here is that the neural [15:58] signals that would have usually animated [16:01] the aectors that I talked about before, [16:03] the signal still gets generated in many [16:05] of these conditions. the patients still [16:07] have the capability to think about [16:09] moving or even actually trying try to [16:11] move. That means that input signal that [16:15] spatial temporal dynamical neural [16:16] activity still gets created. It just got [16:19] nowhere to go. [16:22] So what that means is that our patients [16:24] cannot do what you and I probably take [16:26] for granted, access our digital device. [16:28] So for for you and I, we might think [16:30] that you can't get on Tik Tok or [16:32] LinkedIn or whatever it could be, but [16:34] for our patients, it means loss of [16:35] autonomy. we do more and more uh [16:38] societal task that give us independ [16:40] independence and autonomy through our [16:42] digital devices. [16:44] So then we want to restore this. So this [16:47] uh the solution to this problem is our [16:48] first product that we're currently [16:50] developing which is a scalable motor BCI [16:54] allowing our patients to re-engage with [16:55] the digital world. So you were seeing [16:56] one of our uh patients um he has ALS. [17:00] He's he he's non-verbal. He doesn't have [17:03] really any ability to move his limbs [17:05] effectively, but he's able to produce a [17:08] brain click and he can uh utilize [17:10] existing technology like iPhone uh iOS's [17:13] switch control to be able to navigate [17:15] through and perform this simulated task. [17:22] So, how do we actually do this? Uh to [17:24] make this work, we first instruct the [17:27] patients to perform certain movements. [17:29] So, this is our data collection [17:31] exercise. So let's say follow the [17:33] movements of the cursor on the screen. [17:36] As this is one of our patients in [17:37] Australia um we've asked him to move or [17:41] at least attempt to move his hand in the [17:43] direction in which the cursor is moving. [17:45] So this is our label data collection. [17:50] Then we do the classic supervised [17:52] training to try and create a model that [17:54] can decode uh this motor output as best [17:57] as possible. and we can pat ourselves in [17:59] the back when the loss goes down and and [18:01] this is all great but this means nothing [18:03] when it comes to brain computer [18:04] interfaces. So you might see some of [18:07] these benchmarks um these these um being [18:11] proposed online where you can see that [18:13] the loss is going down but brain [18:15] computer interfaces for motor [18:17] restoration you must test it with the [18:19] real patient in the loop or real user in [18:21] the loop. So it takes two to tango and [18:24] it takes two to VCI. The user and the [18:26] motor must reach an equilibrium [18:28] together. So first the model fits to the [18:30] brain and then the brain must fit to the [18:32] model. So on and so forth until you get [18:34] to a point where they can work together [18:36] and then achieve real-time control. So [18:38] this is one of our patients. After that [18:40] exercise, we we trained iron and then [18:42] he's able to move this cursor just by [18:45] thinking or attempting to move his hand [18:47] to the left and right to some accuracy. [18:50] So this is just to give you a taste of [18:52] what a BTI uh is. But coming back to the [18:58] goal at hand of creating a foundation [19:00] model level sorry foundation model of [19:02] human cognition [19:05] requires not only the sheer amount of [19:07] data afforded by a scalable BCR like the [19:10] indescular approach that we're taking [19:13] but also a product that is used uh in [19:16] rich and wide contexts to engage the [19:20] user in a wide array of cognitive [19:22] capabilities because that's what we [19:24] really want record to be able to give [19:26] that back to the patient. So a lot of [19:28] the BCIs earlier on for the right [19:30] reasons was a huge engineering [19:32] challenge. You had to develop the [19:33] sensors all of the methods. None of this [19:35] has been done before. So a lot of this [19:37] work including some of our earlier work [19:39] was confined to a lab which means you [19:42] ask the patients to perform very [19:43] specific tasks which means you collect [19:46] the data with a very narrow amount of [19:48] context. [19:50] But if you want to build a foundation of [19:52] human cognition then you have to engage [19:55] well the user has to be able to engage [19:57] in the world as you and I would normally [19:59] when they use the device. [20:02] So to do so the very first thing to do [20:05] so is to focus on the pure performance [20:08] of the BCR itself. So you have to [20:09] improve the models and that is my day [20:11] job. So I have to do that and that's how [20:13] I keep my job. [20:16] But also we're very cognizant of the way [20:19] in which we yield those models. [20:23] A I asked actually I forgot to put it on [20:25] the slide but uh I asked chat GPT give [20:27] me um you know a photo of a BCI and it [20:30] gives you 10 researchers behind a single [20:33] person and that's really how BCI models [20:35] are trained so far. You have team of [20:37] PhDs, field engineers, a whole team [20:39] behind working with a single patient to [20:41] get this model up. But clearly that's [20:44] not a scalable approach. You have to be [20:46] able to do this so that the patients can [20:48] train the models themselves or we have a [20:50] way of uh model that generalizes to [20:53] movements and patients. [20:56] We're also deeply thinking about how [20:59] BCIs can be utilized with other [21:01] technology. So we don't have to do all [21:03] of the heavy lifting but we can use [21:05] other technologies to enable the BCI to [21:08] increase the context of usage. [21:13] So for example uh we've worked with [21:16] Nvidia earlier this year to envision how [21:18] future BCI systems could enable brain [21:22] computer interfaces through AI. Sorry [21:24] how AI could enable BCI BCIs to extend [21:28] our patients agency beyond a simple [21:30] digital environment into a physical [21:32] environment. [21:34] So as you can see this is Ronnie that [21:36] you just saw before. He's in a XR [21:38] environment. Um, and thanks to the help [21:42] of VLM model, we're able to detect [21:44] objects um, in his environment that are [21:46] interactable and the language model just [21:49] serves up what type of action that he [21:51] would probably would like to perform. [21:54] So, as you can imagine, you don't have [21:56] to decode everything out of the brain. [21:58] Ron can utilize this hybrid system and [22:00] just simply be able to just look at uh [22:04] um the option that like to do. This [22:07] means that he can come out of this [22:09] narrow digital environment of uh [22:12] computer control into his physical world [22:13] as well. And then that gets us that rich [22:16] context. He's solving more difficult [22:18] problems. He's thinking about more [22:20] cognitive things. [22:23] We're also working with world leaders in [22:25] assisted technology. So this is a huge [22:28] field. We um might not be as as um [22:31] widespread, but in our circle assisted [22:33] technology is a big field. and Apple is [22:36] one of the key leaders there. Um, and [22:38] who's done some awesome work recently. [22:40] We've worked with Apple. Uh, but they've [22:42] created a BCI specific profile for HID. [22:46] And for those of you know, this is a [22:48] huge deal because it brings closer the [22:52] vision where BCI's become a ubiquitous [22:55] input device just like a keyboard and a [22:57] mouse. So wouldn't it be a wonderful [23:00] world where if our patients have a BCI, [23:02] it doesn't matter, it doesn't have to be [23:03] our patients, if anyone who has a BCI, [23:05] they can just interact with existing [23:07] infrastructure and digital uh digital um [23:10] economy without having to have some [23:13] special input or a driver. Imagine back [23:16] in the day if you got a new keyboard and [23:17] you have to install a new driver to to [23:20] tune your uh your computer. It would [23:21] have been an absolute nightmare. So we [23:23] don't want that for our patients. [23:28] So we're at a conference called AI4. I'm [23:30] sure many of you have already taken the [23:31] scale pill. But despite the fact we [23:36] continuously want to update our price [23:38] and add to our conviction that large [23:41] auto reggressive abstraction hypothesis [23:43] at least to a certain degree will apply [23:46] to brain data and that is not a given. [23:50] So we asked the question, does [23:52] generative pre-training even yield [23:54] better results than traditional [23:56] supervised training for BCIs? [23:59] In doing so, can it yield better results [24:02] for models? H sorry, can can it uh yield [24:05] better results for modeling things that [24:07] could be considered a complex cognition? [24:12] So it doesn't have to be very fancy, but [24:13] could it could you make a hypothesis [24:15] that being able to decode this is a [24:18] complex cognition. [24:21] So we performed a little experiment. So [24:23] we've been acquiring data of course [24:24] because we have live patients. Um so [24:26] this is data from one of our patients. [24:29] Uh we're able to accumulate clean 47 [24:32] hours of data which amounts to about 17 [24:34] million tokens. And tokenizing this [24:37] neural data is is a whole thing on its [24:39] own. Uh this is a relatively simple [24:41] method of tokenizing but this is a early [24:43] approach that we took. Um so because raw [24:47] neural data is oscilly if you just let [24:50] it predict the next sample it probably [24:52] will learn some cheap tricks to be able [24:53] to push the loss down. So what we do is [24:56] we take the raw data we put them through [24:58] a bunch of high frequency filter banks [25:01] because high frequency encodes for that [25:03] rich information that we're after. Then [25:06] using our proprietary uh method of [25:09] detecting certain bursts, individual [25:11] bursts of energy on the particular [25:14] frequency, now we can begin to tokenize [25:17] this continuous raw signal. [25:20] And if we're able to do that, then we [25:21] can form sequences. Then we can use then [25:24] we can use uh and leverage existing [25:26] methods that have been very powerful. So [25:28] simple transformers. [25:31] One thing interesting that we did was we [25:33] employed a curriculum based um [25:36] retraining. So we will ask the model to [25:38] predict the next token. So that's 10 [25:40] milliseconds into the future. And then [25:43] when it learns a little bit then we make [25:44] it a little bit more difficult. Two [25:46] tokens 20 milliseconds 30 40 up to 100 [25:50] milliseconds into the future. So that's [25:52] why you're seeing that loss curve kind [25:54] of beginning to go up at the start. One [25:57] token obviously very easy. this off the [25:59] chart actually at the bottom and it [26:01] becomes too easy. So we said all right [26:03] now got predict two shoots up but [26:06] interesting um phenomena that we see is [26:09] that up to a certain point it keeps [26:10] going up but then it continues to just [26:12] come down meaning that hopefully it's [26:15] learned something general about [26:16] predicting the next brain state or at [26:19] least in this particular part the next [26:21] token next token space. [26:25] So this is what it looks like at the end [26:26] of training. Up the top actual tokens [26:29] that we curated from the data set itself [26:32] and then bottom is the probability of [26:34] what the next token state should look [26:36] like 100 milliseconds into the future. [26:39] Now we can pat ourselves in the back. [26:40] The loss went down. This looks pretty [26:42] decent but doesn't mean anything. Does [26:44] it result in anything meaningful for the [26:46] patient? [26:49] So to do that, we wanted to see can the [26:52] model classify between an intended brain [26:55] click versus an unintended brain click. [26:59] So our poor patients because they're [27:01] part of a clinical trial, we uh put them [27:03] through a very boring task. So it's [27:05] called a command control test and it's [27:07] just as boring as it sounds. We say rest [27:11] and they rest and they're not supposed [27:13] to produce any brain clicks and then we [27:15] say click and then they'll try and click [27:17] as fast as possible. And I think they've [27:19] done that about thousands of times but [27:22] through that we can with high [27:24] reliability [27:26] what is an intended brain click versus [27:28] not effectively false positives and true [27:30] positives. [27:32] Now when you put that data set through [27:34] this pre-train model you can see that in [27:35] the embedding space is starting to tease [27:37] these two apart despite the fact that it [27:39] wasn't explicitly trained to do so. [27:42] So after getting this uh inkling um we [27:46] wanted to um kind of showcase a real [27:49] life usage of this classification and [27:52] like I said before real time control is [27:54] king when it comes to BCI. So we kept [27:57] the model size relatively small. So you [27:58] saw that the previous pre-training model [28:00] only had about half a million [28:01] parameters. This decoder had only 75,000 [28:05] and we wanted to use small amounts of [28:07] data that you can quickly acquire out in [28:09] the field. It's a simple 5 minutes of [28:10] data. [28:13] So into the decoder head we put in the [28:15] embeddings themselves because I think [28:17] that probably holds the most information [28:19] but also interestingly the 10 token [28:22] predictions into the future as well. [28:25] And what you're seeing here is that that [28:27] training plus post training method [28:29] yields state-of-the-art results compared [28:31] to other baseline. So this is real time [28:33] control performance. [28:36] So charts are great but let's see it in [28:38] action. So we deploy this model to make [28:40] sure that it actually runs on the [28:42] consumer laptop. In this case the brain [28:44] clicks um going to the nipad. Um, and [28:48] the beauty about this approach is that [28:50] from the patient's perspective, [28:52] Mark over here, he doesn't need to think [28:54] about anything explicit to trigger this [28:56] model. He just does what he does. But [28:58] the model will decide if a brain click [29:01] is detected, [29:03] should I suppress it or not? So, if it's [29:06] suppress it, Mark doesn't see the light [29:08] of the day of it. If he decides that [29:11] it's intended, then you just let it [29:13] through. [29:14] So in this particular thing, I'll play [29:16] the video soon, but he's utilizing a [29:18] brain click model that can detect two [29:19] states, relatively simple. So it's [29:21] double click and click. And if you focus [29:23] on his right hand, you might be able to [29:25] see twitches as well. And we've mapped [29:27] the double click to the shortcuts [29:29] function on the iOS. So it can bring the [29:31] menu up and then he uses the single [29:33] click to navigate through the scanner. [29:36] And what I'm going to highlight here is [29:38] the FC, the field clinical engineer, is [29:41] walking the patients through in real [29:42] time. So they're conversing. So maybe at [29:46] the start you might hear her saying do [29:47] the double click and it'll bring up the [29:48] menu and then she'll describe what to do [29:51] next. And that's the blue face period. [29:54] There's bunch of distractions going. [29:58] Okay. No audio. So you'd have to trust [30:00] me. [30:01] So he would do a double click. Yeah. The [30:04] menu comes up. So that's gone through. [30:06] He's nodding in the back. He's a [30:07] software engineer. And then there's dogs [30:10] barking. They're conversing. So he's [30:11] talking as well himself. He's fidgeting. [30:14] He's, you know, doing whatever it needs [30:15] to be. And then uh at one stage they now [30:19] say, "All right, try and click that [30:21] middle top icon, which is the stock set [30:24] for whatever reason." And then because [30:28] when he's told to do that, those brain [30:30] clicks are intended. So then the model [30:32] lets it through. So there you go. [30:37] And then you can see the the and then [30:39] he'll click middle. See that he's able [30:41] to do that. It's underwhelming video, [30:43] but you can see that the model is able [30:45] to decode what's intended versus not. [30:49] Now I'll I'm a I'm a scientist by trade. [30:52] So this also could mean that we are [30:54] picking up on something like an [30:55] artifact. So a false brain click could [30:58] be artifactual in nature. So we could be [31:00] picking up that. But you can see [31:03] in the data set you have to assume that [31:06] of course there will be artifacts but [31:08] there also could be this intention [31:09] behind it. So I could perform the same [31:11] motor program of wanting to reach my arm [31:14] out to touch the iPad but I can do that [31:17] same motor program to go and boop my dog [31:20] but the intention there is totally [31:21] different. So we're hoping that by [31:23] pre-training the model can pick up on [31:25] those intentions. [31:29] So the ultimate question is will the [31:31] scaling or generalize to neural data to [31:32] a certain point [31:35] sca uh scaling so far has been resulting [31:38] in increase in performance generally [31:40] speaking to a certain point we might be [31:42] hitting a ceiling but this hasn't even [31:45] been tried on brain data and we're not [31:47] the only ones who are thinking this way [31:49] this this uh paper called brand um this [31:52] was published in uh sorry presented in [31:54] Europe uh this is showcasing that when [31:56] you scale the model add more data more [31:58] compute [31:59] you're seeing that familiar curve that [32:01] comes down and if you read um this [32:03] author's papers you can see that it also [32:05] reaches state of the performances on the [32:07] clinical task that they do and we're [32:09] also doing uh checking with our own data [32:12] does this kind of hold hoping that when [32:15] we begin to acquire this rich wide [32:19] context scalable data set we're able to [32:22] pick up on these cognit cognitive things [32:24] so that our patients can output more of [32:26] their uh um intentions to regain their [32:30] autonomy even more. So this picture I'm [32:33] actually really excited about and I [32:35] cannot believe that that I'm alive and [32:37] I'm productive at this particular [32:39] juncture in time. So everything seems to [32:41] be lining up. Synchron's introduced a [32:44] scalable way of getting this data. So if [32:46] this mission sounds exciting to you and [32:49] resonates with you, if you want to help [32:51] patients, come and work for us. Thanks. [32:57] I don't know the mechanics of how it [32:58] works. I'm just like uh I didn't you [33:00] didn't go to the details of it. I would [33:01] love to know more about how it works but [33:02] also uh is there a scope for this to [33:05] like you know naturally progress into a [33:07] state where you're able to also not just [33:09] sense the neural activity but also [33:11] influence it? [33:12] >> Yeah. So great question. So BCIs can be [33:14] birectional. We're focusing on sensing. [33:18] So there are other class of BCIs that do [33:20] stimulation. Sorry the question was is [33:22] it birectional? Does it sense and does [33:24] it write information back? Second [33:25] question was are you detecting direct [33:27] neural activity or are you detecting [33:28] blood flow a surrogate of neural [33:30] activity. So first question no we're [33:32] doing sensing only because we're very [33:34] focused on doing this autonomy [33:36] restoration back stimulation has a huge [33:40] thing of thing of its own. You're [33:41] delivering therapy. It's more risk and [33:43] you're influencing the brain as you saw. [33:45] So many of you might know a technology [33:47] called DBS deep brain stimulation. And [33:49] so that is writing information back and [33:51] it it's very effective for the people [33:53] who have them because it's not response [33:56] responsive to any other treatments. But [33:58] we're not doing that yet. Uh but we're [34:01] focusing on uh sensing and decoding to [34:04] begin with because uh we really believe [34:07] in this cognitive AI kind of vision [34:08] that's really going to help millions of [34:10] people and I think scalability is key [34:12] for us. So being able to focus on this [34:14] will help us to scale to many patients. [34:16] And we're not recording surrogate [34:18] surrogate activity. We're recording [34:19] direct brain activity from adjacent [34:22] tissue. So we're sitting inside of the [34:24] blood vessel but [34:27] electricity flows. So we're recording [34:29] electrical activity not uh bold signals. [34:32] >> Great talk Peter. So um you spoke about [34:37] of course the encoding piece which is [34:39] neural activity to a representation that [34:41] makes sense uh to do stuff and that [34:45] doing stuff could be grabbing a pen, it [34:48] could be retrieving glasses, it could be [34:50] doing XYZ and that's the decoding space, [34:52] right? And which is basically your [34:54] mapping [34:56] or or well in this case you're [34:57] articulating or even generating meaning [35:00] out of uh the neural activity. Um in in [35:04] language models decoding also has a [35:07] space of like uh there are techniques [35:10] like beam search and all of that which [35:11] basically make sense of the decoding [35:14] with some probabilistic version of the [35:16] world and I'm I'm curious if you've [35:19] considered that what that means is [35:21] basically you've got an an approximate [35:23] signal for let's say grabbing a pen but [35:26] is there like further intelligence that [35:29] you're using let's say with prior or [35:31] something which means that in this [35:33] plausible scenario in the environment [35:36] because I'm sitting in an office [35:38] grabbing a pen is more plausible than [35:41] grabbing a microphone. You know what I [35:43] mean? [35:43] >> I do exactly know what you mean and I [35:45] wonder if you've been listening into our [35:47] conversations in our office but [35:50] so who are you? But um you got further [35:53] questions. [35:54] >> Uh no no um yeah and that's why I'm [35:56] banging on about the context. The [35:58] context is I think uh going to be a huge [36:00] multiplier. So you saw Rodney's uh that [36:05] AI enabled BCI. So that in part is we've [36:08] offloaded the context piece to an AI [36:11] model to begin with. [36:12] >> Mhm. [36:13] >> So you can have a single motor output, [36:15] for example, a brain click, but you can [36:18] change what that maps to depending on [36:19] the context exactly as you said. Um we [36:23] want to do this fully from the brain. So [36:25] that's why we're trying to curate this [36:27] data set very meaningfully from the [36:28] get-go. um curating brain data set that [36:32] is synchronized to certain events is [36:34] really difficult for obvious reasons [36:36] privacy and it's got to be done through [36:37] a clinical trial highly regulated so you [36:39] want to get this right but exactly [36:41] that's definitely planning on doing that [36:42] and what I'm also very in uh excited [36:45] about is some of these agentic models [36:48] who are effectively mapping the proper [36:50] for example computer control it's kind [36:52] of mapping what should be clicked and [36:54] what shouldn't so there's some [36:58] interesting angle coming there too I [37:00] think but you know someone else can do [37:02] that and can plug into it. [37:04] Hey Peter, thank you. Wonderful [37:06] presentation. So this is blue ocean [37:09] strategy and in on my phone I would like [37:12] to show you after this call because I [37:14] was working on a similar project. Uh [37:17] it's I want you to spend few minutes [37:19] after this meeting. But how you are [37:22] going to store the memory of the brain? [37:26] >> Restore the memory of the brain. [37:28] >> Yeah. [37:28] >> Yeah. That's not an application that [37:29] we're going for. And memory [37:32] consolidation is a huge field in [37:34] neuroscience. [37:36] >> Um those tend to involve deeper areas of [37:38] the brain, the hippocampus, some of the [37:40] surrounding cortex. Um that is not [37:42] currently our application. We're focused [37:44] on motor restoration at the moment. [37:45] >> So only restoration. So what about like [37:47] these signals which you are capturing [37:49] from the brain, you are not storing them [37:51] as such anywhere. Is that correct? So [37:53] the whole idea is that the embedded in [37:57] the signals themselves that we're that [37:59] we're acquiring exist all of the [38:02] information. We just have to extract [38:04] them. And now it seems like there's this [38:07] general purpose technology that for [38:08] whatever reason is democratized that is [38:11] applying really well to all of these [38:12] modalities and we want to see if it [38:14] works well for the brain. Um and yeah [38:20] >> uh this question is um so from BCI I [38:24] believe the major component of the [38:27] inputs uh is from EEGs that you get from [38:31] the brain signals or so is there a [38:34] possibility of incorporating other um [38:37] modalities as well like eye tracking or [38:40] temperature and things and is is is work [38:42] being done in that side? [38:44] >> Yeah, exactly. So that's part of the [38:45] context, right? So you ought to be [38:48] labeling the brain signals with as many [38:51] contextual information that is relevant [38:54] as possible. So if you I mean even [38:57] something like the weather will impact [38:59] what you're thinking, I think. So I [39:00] think but obvious of course you have to [39:02] pick your battles. Um you know the state [39:05] of the stock market probably doesn't [39:06] really influence what what the neural [39:08] state would be. But yeah, like [39:09] wearables, bio signals, if possible, the [39:12] thing to do would be to align [39:14] everything. But because we're making an [39:16] impensible, you want to make sure that [39:18] you're driving the technology from brain [39:20] signals. So we we have to be good [39:22] custodians that you're actually making a [39:24] brain computer interface, not an [39:26] artifact computer interface.