Thales Deploys Offline Edge AI Agents to Bypass Cloud Data Centers

Thales部署离线边缘AI智能体以摆脱云端数据中心 业务级AI的成功90%取决于工程化软件框架而非模型规模，从而使物理隔离的军事平台能够在无需云基础设施的情况下运行自主诊断。 Ajay Chakravarthy提出了在断网且故障关乎生死的高风险边缘环境中部署稳健“简约AI”（Frugal AI）的工程蓝图。他介绍了Thales开源的CSWAP结构化剪枝框架等技术，并强调了“90/10法则”，认为智能体系统90%的成功源自其周边的工程化支撑架构，而非底层模型本身。通过离线海军GraphRAG诊断工具等应用案例，Chakravarthy展示了小语言模型（SLM）如何在严苛的功耗和带宽限制下可靠执行复杂任务。演讲最后的技术问答环节探讨了物理隔离环境下的数据重新索引、边缘硬件限制以及动态多模型路由问题。 与商业应用相比，航空航天、国防和网络安全领域的任务关键型AI对错误的容忍度几乎为零，需要集成到现有的遗留“系统之系统”中，而非依赖超大规模云数据中心。 (0:04) Thales采用从核心到边缘的“简约AI”（Frugal AI）方法，在严苛的散热、功耗和连接限制之间取得平衡，以便在拒止与降级环境（D2E）中可靠运行。 (2:14) cortAIx加速器通过实验室（Labs）、工厂（Factory）和传感器（Sensors）衔接研发与部署，并严格遵循TrUE AI原则（透明、可靠、合乎道德且可解释）。 (2:14) 为了适应潜艇等平台上严格的尺寸、重量和功耗（SWaP）限制，Thales开发了CSWAP，这是一种以可解释性为驱动的训练后剪枝框架，可在不损失精度的情况下削减高达50%的神经网络参数。 (5:16) 自主智能体需要在非结构化决策空间中具备有界自主性，这与遵循固定、刻板执行路径的确定性工作流形成鲜明对比。 (9:22)

Ajay Chakravarthy of Thales argues that mission-critical artificial intelligence must prioritize frugal, edge-native architectures over mega-scale cloud compute. In high-consequence operational domains, resilience depends not on massive frontier models, but on compact models reinforced by rigorous engineering harnesses and local compression. Mission-Critical AI and the Frugal AI Framework (0:04) Chakravarthy contrasts commercial AI risks with the demands of defense, aerospace, cybersecurity, and space. While a retail chatbot error causes minor customer inconvenience, algorithmic failures in combat—such as friend-or-foe misclassification—risk catastrophic loss of life. These environments require AI to function within legacy "systems of systems," such as decades-old software in armored fighting vehicles, rather than relying on hyper-scale data centers or frequent frontier model updates. To address denied and degraded environments (D2E) characterized by zero internet connectivity, tight thermal ceilings, and strict power limits, Thales employs a "Frugal AI" doctrine designed to balance compute, energy, and bandwidth. To bridge research into frontline capabilities, Thales established cortAIx, structured into three divisions: cortAIx Labs for applied research and prototyping, cortAIx Factory for software engineering, and cortAIx Sensors to embed algorithms directly onto operational assets. Systems developed under this framework must comply with the company's "TrUE AI" standards: Transparent data provenance and drift monitoring, Reliable performance across environmental extremes, Ethical alignment with European governance frameworks, and Explainable automated decision-making. Embedded Model Compression via CSWAP (5:16) Deploying neural networks into confined tactical assets, such as submarines, requires meeting severe size, weight, and power (SWaP) constraints that preclude standard server racks. To address this, Thales Canada open-sourced CSWAP (Causal-aware Structured Pruning), an explainability-driven post-training compression tool. CSWAP evaluates trained networks by classifying individual nodes as critical (essential to accuracy), neutral (zero impact), or detrimental (degrading performance), allowing neutral and detrimental nodes to be eliminated. In benchmark tests on architectures like ResNet-18, CSWAP pruned up to 50% of the network without loss of classification accuracy, measured against the Sparsity Area Under the Curve (SAUC) metric. While CSWAP compresses computer vision models from millions of parameters down to low-power edge hardware like a Raspberry Pi or Jetson Nano, Chakravarthy notes that priorities have evolved for large language models. The growing availability of high-quality, pre-optimized open-source small language models (SLMs) has largely superseded the computational need to prune frontier LLMs from scratch. The 90/10 Harness Architecture for Agentic Systems (9:22) Chakravarthy differentiates conversational prompting from true agentic AI, which requires bounded autonomy to execute actions in digital or physical environments. This creates friction with defense customers like the UK Ministry of Defence, whose insistence on deterministic, repeatable paths reflects fixed workflows rather than autonomous agency. Furthermore, unconstrained single-agent systems running prolonged 10-to-12-hour workloads inevitably encounter context compaction and severe hallucinations. To ensure operational stability, Thales structures agentic systems under a 90/10 rule: the underlying small language model—such as Google's Gemma 12B—represents only 10% of the solution, while the engineered operational harness accounts for the remaining 90%. The harness provides system boundaries, safety guardrails (such as restricting root access), isolated sandboxes, observability tracking, and multi-agent coordination where sub-agents execute within isolated context windows before updating shared memory. Chakravarthy cautions against "vibe coding" rapid prototypes, warning that quick, unstructured builds accumulate severe technical debt compared to systematic agentic engineering. Disconnected Deployments and Air-Gapped Maintenance (12:08) In deployed field settings, such as naval maintenance environments, systems operate without external cloud connectivity. Because standard vector Retrieval-Augmented Generation (RAG) proved inadequate for complex diagnostic manuals and legacy fault logs, Thales deployed an edge-native GraphRAG architecture layered with domain ontologies. Running locally on an offline SLM inside an edge harness, the tool guides technicians through multi-turn, interactive troubleshooting. For lifecycle updates in air-gapped environments, the architecture remains model-agnostic to permit SLM swaps. When new documentation arrives, the system carries out local on-device re-vectorization and reindexing entirely offline. Chakravarthy adds that Thales is actively investigating multi-model routing, using the agentic harness to direct tasks between simple deterministic classifiers and SLMs based on task complexity, alongside LLM-as-a-judge configurations to benchmark edge model outputs. While Thales demonstrates that compact SLMs and structured pruning can support mission-critical operations independent of mega-scale cloud infrastructure, dynamic multi-model routing remains an active engineering development. Balancing autonomous agency against defense stakeholders' requirements for absolute determinism continues to dictate the boundary between scripted workflows and deployable edge AI.

泰雷兹（Thales）专家Ajay Chakravarthy指出，在国防与航空航天等生死攸关的任务场景中，单纯依赖超大规模数据中心和前沿模型并不现实；必须推行“节约型AI”（Frugal AI），通过结构化模型剪枝与以工程外壳为主导的系统架构，在受限的物理边缘实现高可靠运行。 关键任务挑战与节约型AI架构治理 (0:04) 泰雷兹在全球68个以上国家拥有逾8.6万名员工，深耕航空航天、国防、网络安全和空间领域。Chakravarthy强调，此类高危任务环境对算法故障的容错率与消费级产品截然不同：零售聊天机器人的失误仅导致客户收到错误的衣物，而在防务平台的自动敌我识别系统中，算法失效将直接引发人员伤亡。这种现实决定了工业应用不能盲目跟风追求算力庞大的数据中心或频繁更迭的通用大模型，必须清醒地认识到AI仅是“系统之系统”（system of systems）的一部分，需要与装甲车辆中已运行数十年的老旧软件无缝集成。 面对北海严苛环境等通信受阻与降级环境（D2E），泰雷兹提出了“节约型AI”范式，要求在断网、严格功耗及散热限制下保障系统可靠性。为落实该路线，泰雷兹组建了全球AI加速器cortAIx，分为三个梯队协同运作：cortAIx Labs负责前沿应用研究与原型验证，cortAIx Factory负责工程化并转化为适配硬件的软件，cortAIx Sensors则直接将算法固化至一线传感器。整个流程严格遵循TrUE原则，即确保数据来源与漂移监控透明（Transparent）、极端环境表现一致且可靠（Reliable）、符合欧洲伦理规范（Ethical），以及决策链路清晰可解释（Explainable）。 战术边缘约束与CSWAP结构化剪枝 (5:16) 战术边缘部署面临严苛的尺寸、重量与功耗（SWaP）物理限制。Chakravarthy批评业内部分厂商宣称提供边缘方案，实则交付“冰箱大小的服务器机架”，这在空间狭窄的潜艇中毫无实用价值。为了压缩传统深度神经网络，加拿大泰雷兹研发并开源了因果感知结构化剪枝框架CSWAP。该技术属于训练后压缩方法，通过可解释性分析将网络神经元划分为三类：关乎精度的关键节点予以保留，对结果无影响的中性节点直接剪除，导致性能下降的有害节点同样予以剥离。 基准测试显示，在ResNet-18等分类架构上，CSWAP在完全不损失精度的情况下可剪除高达50%的参数，并通过稀疏曲线下面积（SAUC）指标验证了剪枝效果。这使传统模型能够顺利部署在Jetson Nano或树莓派（Raspberry Pi）等低功耗单板硬件上。Chakravarthy补充说明，尽管该数学剪枝机制可拓展至大型语言模型，但鉴于当前开源社区已涌现出大量高质量轻量级模型，业界已无需耗费巨额算力从零对前沿大模型执行重度剪枝。 90/10外壳架构与受限自主智能体 (9:22) 在智能体领域，Chakravarthy提醒工程界切勿将交互式聊天提示与真正的自主智能体混为一谈。向模型提问仅属于对话，真正的智能体必须具备在物理或数字世界中自主执行动作的权力。然而，这与英国国防部等客户对确定性结果的偏好存在冲突——固定路径属于工作流，而智能体必须在有边界的空间内拥有自主权。针对长达10至12小时的企业级复杂任务中上下文压缩导致的信息丢失与严重幻觉，泰雷兹提出了核心原则：底层模型仅占系统的10%，而90%的成效应归功于精心构建的工程外壳（Harness）。 这一工程外壳涵盖了边界提示词、跨子任务的智能体编排、隔离且干净的上下文窗口、持久化记忆传递、可观测性监控、沙箱验证以及关键行动中的“人在回路”控制。通过这一架构，团队可使用Google Gemma 12B等小型语言模型（SLM）达到预期能力，完全摆脱对云端万亿参数巨型模型的依赖。Chakravarthy严正警告团队不要沉迷缺乏工程纪律的“氛围编码”（vibe coding），指出三天拼凑出的原型会迅速积累巨大的隐性技术债务，唯有前期严密构建的智能体工程才能保障全生命周期成本的平稳可控。 舰船离线诊断落地与隔离环境数据维护 (12:08) 在实际工程落地中，泰雷兹为海军舰船乘员部署了完全离线运行的边缘原生故障诊断助手。面对繁杂的技术手册、遗留故障日志和维修记录，普通向量检索增强生成（Vector RAG）难以梳理专业本体逻辑，团队为此开发了结合领域本体的GraphRAG架构。整套系统在脱离外部云端API的环境下，完全依托舰载小语言模型与本地外壳执行，引导技术人员按步骤完成硬件故障排查。 针对完全物理隔离（Air-Gapped）环境下的系统维护，该架构采用了模型解耦设计。当数月后迎来新的维护手册时，系统会在本地边缘环境直接进行重新向量化（Local Revectorization）与索引重建，使检索生成持续保持最新，全过程无需外网连接。此外，泰雷兹正探索动态模型路由技术，由外壳根据任务复杂度将指令分发给经典确定性分类器或小语言模型，并结合“大模型充当裁判”（LLM-as-a-Judge）机制对一线边缘模型的表现与安全性进行闭环评测。 泰雷兹展示了关键任务AI脱离中心化超大规模算力的工程路径：依托严苛的物理降维、90%的控制外壳以及离线本地索引，小参数模型足以胜任前沿防务与工业场景。然而，在完全物理隔离的极端边缘，如何在维持低功耗的同时高效处理多模型动态路由与持续评估，仍是边缘智能深化部署中有待持续验证的现实挑战。

## Background 该视频由频道 Ai4 于 2026 年 8 月 19 日发布在 YouTube 平台上。内容由英国泰雷兹（Thales UK）首席 AI 官 Ajay Chakravarthy 主讲，重点探讨在带宽受限、拒止与降级环境以及严苛能耗预算下，如何打破依赖超大规模算力中心的迷思，通过小型专用模型与严密工程外框实现安全、可靠的任务关键型边缘 AI 智能体部署。 ## TL;DR Ajay 区分了容错率为零的任务关键型 AI 与日常消费级 AI，提出应对严苛边缘环境的“极简节约型 AI”（Frugal AI）理念以及用于模型深度压缩的 C-SWAP 因果感知剪枝技术。他直言模型在整体系统成功要素中仅占 10%，而包裹在外的工程外框（Harness）才是决定系统 90% 可靠性与安全性的重心所在，并严厉反思了缺乏工程规范的“氛围感编程”。最后，演讲结合泰雷兹为海军舰艇打造的 GraphRAG 离线诊断助手案例，以及现场关于物理隔离数据刷新、多模型路由与边缘评估的问答，完整展现了边缘智能体的实战工程路径。 ## Atmosphere & style 整场演讲呈现出一种严谨、务实且充满工程清醒感的硬核工业风格。Ajay 谈吐沉稳直接，毫不客气地击碎硅谷追逐万亿参数模型与“氛围感编程”的浮躁炒作，代之以国防与军工级系统的冷酷现实法则。听众能清晰感受到深海潜艇与战备前线的逼仄张力，整体氛围从对宏大算力神话的解构自然过渡到对确定性工程架构的笃定坚守。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=I55hQadX6v4&t=4s) 任务关键型 AI（如国防敌我识别）与消费级 AI 的本质区别在于容错率为零，其系统失效直接关乎生命安全与任务成败，必须遵循完全不同的工程设计哲学。 - [01:26](https://www.youtube.com/watch?v=I55hQadX6v4&t=86s) 盲目追逐万亿参数的前沿云端大模型无益于任务关键型系统，实战场景要求践行“极简节约型 AI”（Frugal AI），能够在断网、受限算力、紧缩能源和极低带宽的拒止降级环境中稳定运行。 - [03:19](https://www.youtube.com/watch?v=I55hQadX6v4&t=199s) 泰雷兹通过 cortAIx 加速器推进技术落地，并严格贯彻涵盖透明性（Transparent）、可靠性（Reliable）、伦理性（Ethical）和可解释性（Explainable）的 TrUE AI 治理准则。 - [05:16](https://www.youtube.com/watch?v=I55hQadX6v4&t=316s) 专有剪枝技术 C-SWAP 利用因果可解释性将训练后神经元划分为关键、中性和有害节点，在 ResNet-18 上即便剪枝 50% 仍能保持准确率不降，有效契合边缘硬件严苛的 SWaP-C 限制。 - [09:22](https://www.youtube.com/watch?v=I55hQadX6v4&t=562s) 智能体的本质是从对话沟通跨越到被赋予自主决策与操作的代理权（Agency）；若业务追求 100% 路径确定性，应直接采用确定性工作流而非智能体。 - [09:22](https://www.youtube.com/watch?v=I55hQadX6v4&t=562s) 智能体工程遵循 10/90 定律：底层模型本身仅贡献 10% 的系统要素，包裹在模型外层的工程外框（Harness）才是决定 90% 效能与可靠性的核心重心。 - [12:02](https://www.youtube.com/watch?v=I55hQadX6v4&t=722s) 企业级智能体 Harness 必须涵盖系统提示词、可观测性、反馈闭环、分层记忆管理、防护栏、人在回路（针对动能打击等高危操作）以及隔离沙箱等核心模块。 - [12:02](https://www.youtube.com/watch?v=I55hQadX6v4&t=722s) 严厉反思并抵制高危领域的“氛围感编程”（Vibe Coding），缺乏系统工程外框的原型隐藏着呈爆炸式递增的技术债务与故障成本，而严谨的智能体工程能保障长期的低维护成本与确定性。 - [15:42](https://www.youtube.com/watch?v=I55hQadX6v4&t=942s) 泰雷兹舰载“维护操作助手”案例表明，在完全离线的深海边缘环境中，结合 GraphRAG 知识图谱与强安全外框的小型语言模型（SLM），能高效且准确地引导复杂故障排查。 - [18:35](https://www.youtube.com/watch?v=I55hQadX6v4&t=1115s) 在物理隔离（Air-Gapped）断网环境中，系统采用模型无关（Model-Agnostic）架构，并通过本地重新向量化（Revectorization）完成知识库与索引的离线更新。 - [19:23](https://www.youtube.com/watch?v=I55hQadX6v4&t=1163s) 开源小型化基础模型的迅速繁荣降低了对大语言模型从头深度剪枝的需求，开发者可直接采用小规格开源模型部署于边缘设备。 - [21:06](https://www.youtube.com/watch?v=I55hQadX6v4&t=1266s) 多模型路由（Multi-Model Routing）技术可根据任务类型在传统分类器、统计模型与小型语言模型间灵活调度，并利用“模型评估模型”构建严密的离线质检闭环。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=I55hQadX6v4&t=4s) 任务关键型 AI 与泰雷兹（Thales）的技术使命 英国泰雷兹（Thales UK）首席 AI 官 Ajay Chakravarthy 首先介绍了泰雷兹的全球规模与核心业务。泰雷兹总部位于法国，是一家业务遍及全球 68 个国家、拥有超过 86,000 名员工的跨国科技巨头，专注于航空航天、国防、网络安全和空间技术等任务关键型（Mission-Critical）领域。 Ajay 明确划定了“任务关键型 AI”与日常消费级 AI 的本质区别。如果为一家零售服装店开发聊天机器人，一旦系统犯错，最坏的结果不过是顾客收到了一件尺码或款式不对的衣服；后果虽有负面影响，但并不致命。然而，如果在国防作战环境中部署 AI，将其用于“敌我识别”（Friend or Foe detector），一旦算法做出错误判断，直接后果就是人员伤亡。因此，在这类容错率为零的环境中，AI 的工程化实现必须遵循完全不同的底层逻辑与设计哲学。 - 泰雷兹在全球 68 个国家开展业务，员工规模超过 86,000 人。 - 业务横跨四大核心硬核领域：航空航天（Aerospace）、国防（Defense）、网络安全（Cyber Security）与空间技术（Space）。 - 核心分歧点在于系统失效的代价：消费级场景只关乎体验与商业损失，而任务关键型场景关乎生命安全与任务成败。 --- ### [01:26](https://www.youtube.com/watch?v=I55hQadX6v4&t=86s) 告别万亿级云端模型：边缘严苛环境与“极简节约型 AI”（Frugal AI） 当前科技界正陷入狂热的算力军备竞赛，争相建造被称为“融化地球的数据中心”（planet-melting data centres），行业目光紧盯着硅谷每隔两周就迭代一次的前沿大模型（Frontier Models）。Ajay 旗帜鲜明地指出，对于任务关键型系统而言，盲目追逐最新的前沿模型毫无意义，因为模型从来都不是 AI 系统的核心主体。在实战环境中，AI 始终运行在一个庞大的“体系之体系”（System of Systems）之内；例如将算法装载进一辆主战坦克时，它必须与车内运转数十年的老旧遗留软件系统（Legacy Systems）严丝合缝地协同，而不是频繁替换最新的庞大模型。 为此，泰雷兹提出了“极简节约型 AI”（Frugal AI）的核心理念。这种节约不仅是指少样本学习（Low-shot learning），更是指系统必须能在“拒止与降级环境”（Denied and Degraded Environment）中稳定工作。以北海（North Sea）的海上作业为例，设备处于极端恶劣的气候条件下，完全切断外网连接，面临严苛的低功耗限制与极其有限的数据输入，同时还要在高风险后果下提供百分之百的可靠性。在英国国防体系中，这种架构被称为“核心到边缘能力”（Core to Edge Capability）——云端数据中心只负责核心智能体的构建与支持，但真正的执行端必须孤悬于完全断网的边缘前线。 ```mermaid flowchart LR accTitle: 核心到边缘的 Frugal AI 部署架构 A["核心层 (Core Data Center)<br>智能体研发与战略编排"] --> B["严苛约束信道<br>(断网/拒止/降级环境)"] B --> C["战术边缘端 (Tactical Edge)<br>北海平台/装甲车辆/潜艇"] subgraph EdgeLimits ["边缘三维约束"] D["受限算力 (Compute)"] E["紧缩能源 (Energy)"] F["极低带宽 (Bandwidth)"] end EdgeLimits -.-> C ``` - 核心与边缘的分工逻辑：数据中心提供中心算力与核心架构，但现场业务必须在彻底脱网、恶劣环境中由边缘自主闭环。 - “极简节约型 AI”的三大核心资源限制评估维度：计算算力（Compute Power）、能源供应（Energy Supply）和通信带宽（Bandwidth）。 --- ### [03:19](https://www.youtube.com/watch?v=I55hQadX6v4&t=199s) cortAIx 全球加速器与 TrUE AI 可信原则 为了将前沿概念转化为真正的作战与工程能力，泰雷兹成立了全球 AI 加速器——cortAIx。Ajay 强调，团队的目标绝不是成为一个只坐而论道的“好点子俱乐部”（Club of good ideas），而是要把技术推向实际的作业场景。cortAIx 下设三大分支，构成严密的落地管道： 1. **cortAIx Labs（实验室）**：负责前沿研究与概念验证，评估各种技术路线的可能性； 2. **cortAIx Factory（工程工厂）**：将实验室的验证成果工程化，改造成能够部署在网络安全系统、实体硬件或太空载荷等严苛环境下的可用方案； 3. **cortAIx Sensors（传感器端）**：将工厂制造的工程产品最终烧录并无缝集成到雷达、声呐等前端硬件产品中。 在整个推进过程中，泰雷兹严格贯彻名为 **TrUE AI** 的治理准则。在欧洲日益严格的 AI 伦理法规和监管法案背景下，合规与可信构成了技术部署的硬性门槛： ```mermaid flowchart TD accTitle: cortAIx 组织架构与落地流程 A["cortAIx Labs<br>(技术孵化与 PoC 原型)"] --> B["cortAIx Factory<br>(工程化/软硬件集成)"] B --> C["cortAIx Sensors<br>(最终载荷与传感器产品化)"] subgraph TrUE ["贯穿全流程的 TrUE AI 准则"] T["T: 透明性 (Transparent) - 数据溯源与模型漂移监控"] R["R: 可靠性 (Reliable) - 极端多变环境下的确定性输出"] E1["E: 伦理性 (Ethical) - 欧洲伦理宪章合规与责任约束"] E2["E: 可解释性 (Explainable) - 决策逻辑的全链路白盒化"] end TrUE -.-> A TrUE -.-> B TrUE -.-> C ``` - **Transparent（透明性）**：明确数据血统与训练路径，并建立长期性能漂移（Drift）监测体系。模型即使部署在北海孤岛上，也必须有机制评估其随时间推移的性能衰退。 - **Reliable（可靠性）**：在极其复杂多变的工况下，必须保证系统始终稳定、重复地输出预期结果。 - **Ethical（伦理性）**：严格遵循欧洲及国际 AI 伦理宪章，使一切部署置于制度与道德框架内。 - **Explainable（可解释性）**：给出明确的因果判定依据，确保操作员能够获知 AI 结论的来龙去脉。 --- ### [05:16](https://www.youtube.com/watch?v=I55hQadX6v4&t=316s) 传统 AI 的极致压缩：C-SWAP 因果感知结构化剪枝技术 在战术边缘部署时，物理硬件的 SWaP-C（尺寸 Size、重量 Weight、功耗 Power 和总部署成本 Cost）是决定性的瓶颈。Ajay 指出，当前许多厂商宣称的边缘服务器动辄需要一个“像冰箱一样大”的机架，这种庞然大物在空间逼仄的潜艇或战车内部根本无法安装。 为了打破这一僵局，泰雷兹加拿大团队研发并开源了一项专有模型压缩技术——**C-SWAP（基于可解释性的因果感知结构化剪枝，Causal-Aware Structured Pruning）**。与在训练周期内进行的常规压缩不同，C-SWAP 作用于训练完成后的深度神经网络（Post-trained Network），利用因果可解释性评估每个神经元对整体分类性能的具体影响，将其分为三类： 1. **关键节点（Critical Node）**：移除该神经元会导致整体性能严重下滑，必须予以保留； 2. **中性节点（Neutral Node）**：移除后对输出结果几乎没有任何影响，直接剪除以降低模型体量； 3. **有害节点（Detrimental Node）**：该神经元的存在实际上会干扰或降低最终精度，移除它反而能够提高分类准确率，属于必须清除的对象。 ```mermaid flowchart TD accTitle: C-SWAP 剪枝评估分类逻辑 A["训练完毕的深度神经网络<br>(Post-trained Network)"] --> B["C-SWAP 因果评估机制"] B --> C{"神经元因果属性判定"} C -->|"移除则性能下降"| D["关键节点 (Critical)<br>保留节点"] C -->|"移除后毫无影响"| E["中性节点 (Neutral)<br>剪除以缩减体积"] C -->|"移除反而提升准确率"| F["有害节点 (Detrimental)<br>坚决剪除并优化性能"] ``` 在标准基准测试（如 ResNet-18）中，实验数据显示，即便将网络整体剪枝高达 50%，模型的准确率依然没有发生任何下降，只有在剪枝率超过该拐点后性能才会急剧下跌。论文提出的评估指标 **SAUC（稀疏度曲线下面积，Sparsity Area Under the Curve）** 证实，C-SWAP 在性能保持与神经元精简的权衡表现上全面超越了现有的 SOTA 技术。 针对传统边缘 AI 的工程落地，Ajay 总结了一套清晰的方法论，并严肃警告行业内普遍存在的“拿着锤子找钉子”的方案主义（Solutioneering）倾向： - **先定义核心问题**：不要盲目认定“AI 就是万灵药”。首先弄清待解决的业务本质，AI 可能适用，也可能根本不是最佳解法。 - **构建语义模型**：用机器可理解的方式明确定义训练数据的特征画像、期望模型学习的目标以及预测变量。 - **实施深度压缩**：利用类似 C-SWAP 的工具压缩网络，契合边缘物理载体的尺寸与功耗极限。 - **注入 TrUE 原则并交付**：在受限边缘硬件中完成验证与最终上架。 --- ### [09:22](https://www.youtube.com/watch?v=I55hQadX6v4&t=562s) 智能体工程的本质：工作流与 Agency 的抉择及 10/90 定律 当话题转向当前大热的智能体 AI（Agentic AI）时，Ajay 直言不讳地指出认知误区：单纯调用前沿大模型写演讲稿或策划点子，根本谈不上是智能体，那仅仅是与大模型展开对话。真正的智能体意味着从“对话沟通”跨越到“赋予代理权以执行操作”（giving agency to do stuff）。 然而，一旦赋予系统代理权，如何控制它便成为军工和高危领域的最大难题。以英国国防部（Ministry of Defence）为例，军方客户本能地要求对系统动作进行绝对严格的控制、确定性的输出并执行分毫不差的固定路径。Ajay 指出：“那根本不是智能体，那只是一套确定性工作流（Workflow）。”如果业务场景要求 100% 的路径确定性，团队就应该直接去写确定性规则和工作流代码，根本不需要引入智能体；智能体的存在价值正是基于其在模糊空间中自主探索并决策的“代理能力”（Agency）。 在长周期执行任务中，智能体面临严重的上下文崩溃问题。若让单个智能体连续执行长达 10 到 12 小时的庞大工程任务（如重构整个底层代码库），随着上下文窗口被过度压缩，模型会不可避免地遗忘关键信息并陷入幻觉。因此，依托单智能体维持超长任务是极其拙劣的架构设计。 泰雷兹推崇的**智能体 10/90 定律**指出： - **模型仅占 10%**：无论是万亿参数前沿大模型还是边缘小模型，模型本身在整体系统成功要素中仅贡献 10%。 - **外框设计占据 90%**：系统 90% 的效能与可靠性取决于包裹在模型外层的**工程外框（Harness）**。 - 在边缘断网环境中，完全无需依赖昂贵的超大参数模型，像 Google 开源的 Gemma 12B 这类小型语言模型（SLM），在精准外框的配合下足以高效胜任绝大多数专业任务。 --- ### [12:02](https://www.youtube.com/watch?v=I55hQadX6v4&t=722s) 智能体外框（Harness）架构与反思“氛围感编程”（Vibe Coding） 为了让 10% 的模型发挥出确定、可靠的作战效力，必须倾力打造 90% 的工程外框（Harness）。Ajay 详细拆解了企业级智能体 Harness 必须涵盖的核心模块： ```mermaid flowchart TD accTitle: 智能体工程外框 (Agentic Harness) 核心模块 subgraph Harness ["智能体外框工程 (90% 系统重心)"] direction TB SP["系统提示词 (System Prompts)<br>明确行为边界与通用底线"] OB["全维可观测性 (Observability)<br>算力/能耗/运行状态监控"] FL["反馈与迭代循环 (Feedback Loops)<br>自校验与递进优化"] MM["分层记忆管理 (Memory Management)<br>子智能体上下文隔离与持久化交付"] GH["防护栏与钩子 (Guardrails & Hooks)<br>拦截高危行为 (如禁止 Root 提权)"] OR["编排与人在回路 (Orchestration & HITL)<br>致死/动能打击动作强制人工决策"] SB["隔离沙箱 (Sandboxes)<br>上线前的边缘极限用例安全测试"] end SLM["10% 核心驱动引擎<br>(本地轻量模型 如 Gemma 12B)"] Harness --> SLM ``` - **系统提示词（System Prompts）**：为智能体注入根本性的行为准则，界定行为边界与绝对禁区。 - **可观测性（Observability）**：在边缘断网环境中虽然无需监控公有云的 API Token 费用，但必须精确监控算力资源消耗与软硬件运行状态，杜绝由于不可见导致的任务挂死或失控。 - **反馈与迭代循环（Feedback Loops）**：严禁单向线性的指令执行；必须建立自评估和多轮反思循环，直至任务指标达标。 - **分层记忆管理（Memory Management）**：打破“单智能体单一大上下文”的陈旧设计，改用多子智能体（Sub-agents）架构。每个子智能体持有独立的纯净上下文，处理完专项任务后提炼关键成果并持久化存储，再无缝交接给下一个子智能体。 - **防护栏与安全钩子（Guardrails & Hooks）**：硬编码系统级红线逻辑，例如严格禁止智能体尝试获取系统 Root 最高权限。 - **编排与人在回路（Orchestration & Human-in-the-Loop）**：关键核心环节。在可能引发动能毁伤（Kinetic Effect）或决定性实弹打击的操作链路上，严禁 AI 擅自裁决，必须将最终决策权收束至人类指挥员手中。 - **隔离沙箱（Sandboxes）**：部署上线前，智能体必须在完备的虚拟仿真沙箱中经受极端边缘测试（Edge-case testing）。 Ajay 借此严厉批判了当下流行的“氛围感编程”（Vibe Coding）风气。如果有人在国防、航空航天等高危自动化领域吹嘘“我花三天用氛围感编程手搓了一个原型，能不能直接部署进系统？”，CTO 应当当场回绝并亮起最高级别红灯。因为这种原型虽然启动成本极低，但缺乏系统工程外框，其隐藏的技术债务（Hidden Debt）和故障成本会随着时间呈爆炸式复合递增；相反，严格的智能体工程（Proper Agentic Engineering）虽然前期投入成本较高，但能够迅速收敛并维持极低、可预测的长期维护成本曲线。 --- ### [15:42](https://www.youtube.com/watch?v=I55hQadX6v4&t=942s) 软件研发范式重构与舰船边缘维护案例 Ajay 指出，智能体 AI 正在重塑传统软件开发生命周期（SDLC）。以往从需求分析、系统设计、代码编写到测试验证和部署审查的漫长线性流程已被打破。借助配置严密 Harness 的智能体集群，大型代码库不仅可以实现全自动编写与日常维护，更能在发布前完成高强度的自主测试验证。 在具体实战案例方面，泰雷兹近期为海军舰艇落地了一款名为**“维护操作助手”（Maintenance Operator）**的智能诊断边缘系统。战舰在远离本土的深海航行时，随时可能出现设备故障。舰上虽然堆积了海量的故障日志（Fault Logs）、极其冗长的技术手册（User Manuals）以及过往的维修记录，但当舰桥或动力舱的警报红灯疯狂闪烁时，水兵根本没有时间去翻阅成千上万页的文档。 泰雷兹在研发初期尝试将所有技术资料直接灌入通用的 RAG（检索增强生成）智能体，但很快发现标准 RAG 在处理专业深度的复杂因果排错时表现极差。为了满足任务关键型的高精度需求，团队改用架构更加严谨的 **GraphRAG**，在底层文档之上构建深度的设备本体知识图谱（Ontology）。 由于战舰处于完全战备无线电静默或深海脱网环境，系统严禁调用任何云端商业模型 API，研发团队将一套完全离线的小型语言模型（SLM）部署于舰载边缘硬件中，依靠精心定制的外部安全外框驱动运作。现在，当舰艇某个控制系统的红灯报警闪烁时，水兵只需口头或在终端询问：“指示灯亮红灯报警，我该怎么做？”，智能体便会立刻交叉检索历史故障日志与多本维修手册，以严谨且可解释的排查流程引导水兵执行检修：“请先尝试排查 A 接口电压，或采用第 X 种维护规程进行复位。”这生动展现了极简、脱网、带强外框的智能体如何在任务关键型场景下转化为不可或缺的实战战斗力。 ### [17:54](https://www.youtube.com/watch?v=I55hQadX6v4&t=1074s) 边缘落地的实际案例与资源共享 在对话式系统维护等实际场景中，边缘智能体展现出了出色的上下文保持能力。运维人员可以非常自然地与系统进行多轮交互，例如直接说“顺便说一下，我已经重启过设备了，接下来该怎么做？”，系统能够紧跟上下文并持续提供精准指导。类似这样的边缘部署已在多个领域成功落地并投入使用。 演讲者在结束分享前兑现了此前提到的承诺，向参会者指引了相关的学术文献。对于希望深入了解剪枝技术细节的开发者和研究人员，相关论文已公开发表，核心算法的开源代码也已托管在 GitHub 上供大家下载使用。 ```mermaid flowchart LR accTitle: 边缘多轮交互流程 A["现场运维人员"] -->|"报告操作（例如：已重启）"| B["边缘对话系统"] B -->|"维护上下文状态"| C["下一阶段排错指引"] C -->|"反馈建议"| A ``` - **学术资源与开源代码**：相关技术论文已正式发布，剪枝方案源码已在 GitHub 开源。 - **现场落地表现**：在受限边缘场景中，系统支持自然流畅的多轮排错对话，兼顾实时性与准确性。 --- ### [18:35](https://www.youtube.com/watch?v=I55hQadX6v4&t=1115s) 物理隔离（Air-Gapped）环境下的数据刷新与模型重构 在问答环节，现场观众针对物理隔离（Air-gapped）环境下的系统维护提出了疑问：当将小型语言模型（SLM）连同数据集复制到完全断网的边缘端部署运行数月后，如果产生了新数据需要更新，系统该如何管理这部分数据的刷新与再索引？ 演讲者解释称，整体架构在设计之初就贯彻了“模型无关”（Model-Agnostic）的原则，如果需要，底层的 SLM 可以随时被替换或升级为其他模型。针对系统维护类应用中新数据的接入问题，数据更新的机制是在边缘环境内部直接进行重新向量化（Revectorization），进而完成本地知识库的重新索引，整个流程无需依赖外部云端连接。 ```mermaid flowchart TD accTitle: 边缘本地知识库刷新架构 A["新数据输入（边缘离线导入）"] --> B["本地重新向量化引擎"] B --> C["边缘向量索引库"] D["模型无关架构（SLM 可随时插拔替换）"] --> E["边缘推理系统"] C --> E ``` --- ### [19:23](https://www.youtube.com/watch?v=I55hQadX6v4&t=1163s) 模型剪枝的适用范围与硬件部署考量 针对观众提出的“模型剪枝从多少参数量开始、剪枝后保留多少参数、是否需要客户采购专门的 GPU 机架或在 7B/8B 规格的通用服务器上运行”等多维度问题，演讲者详细阐述了剪枝技术的演进及其与当前开源生态的关系。 演讲中讨论的核心剪枝机制最初主要面向传统机器学习（ML）与计算机视觉场景。例如，针对用于图像分类的深度神经网络（DNN），初始模型可能包含数百万参数，经过剪枝优化后，能够直接压缩并高效部署在 Jetson Nano 或树莓派（Raspberry Pi）这类极低功耗的微型硬件上。 尽管该剪枝方法在架构上具有良好的可扩展性，完全可以套用到大语言模型（LLM）上，但演讲者指出，随着目前开源社区的迅猛发展，从小规模到大规模的各类前沿开源模型层出不穷。对于 Frontier 模型和 LLM 而言，开发者可以直接选用现成的小型开源模型，不再绝对依赖针对超大模型从头进行剪枝。 - **传统剪枝场景**：百万级参数的深度神经网络经过剪枝后，可运行在 Jetson Nano 或树莓派等边缘硬件设备上。 - **LLM 生态现状**：开源小型化基础模型的爆发式发布，在很大程度上降低了对前沿模型进行深度剪枝的刚性需求。 --- ### [21:06](https://www.youtube.com/watch?v=I55hQadX6v4&t=1266s) 多模型路由与边缘评估架构 现场观众进一步探讨了将轻量级确定性模型（如简单的决策分类器或传统统计模型）与更具情境理解能力的语言模型协同工作的可行性，即通过超轻量路由引擎采集信号并触发相应处理。 演讲者对这一思路给予了高度肯定，并确认团队正在深入探索多模型路由（Multi-Model Routing）技术。很多具体任务并不需要调用体积庞大、计算昂贵的语言模型，系统完全应该为特定场景匹配最适宜的模型类型。通过构建合理的底层测试与调度框架（Harness），系统可以像调用外部工具（Tool Calling）一样，在最恰当的时间自动选择并路由到正确的模型。 此外，多模型架构还在系统评估（Eval）中扮演关键角色。团队采用“模型评估模型”的方案，利用不同的大语言模型对边缘端部署的小型语言模型的输出结果和整体性能进行打分与多维度验证，从而在保证边缘高效运行的同时，建立严密的质量控制闭环。 ```mermaid flowchart TD accTitle: 多模型路由与评估机制 A["输入请求 / 边缘信号"] --> B["调度 Harness"] B -->|"轻量/确定性任务"| C["统计分类器 / 传统 ML"] B -->|"复杂推理/语言任务"| D["小型语言模型 (SLM)"] D --> E["生成任务结果"] E --> F["评估模型 (Eval Model)"] F -->|"质量与性能分析"| G["系统优化闭环"] ``` ## Notable quotes > [00:04](https://www.youtube.com/watch?v=I55hQadX6v4&t=4s) 如果为一家零售服装店开发聊天机器人，一旦系统犯错，最坏的结果不过是顾客收到了一件尺码或款式不对的衣服……然而，如果在国防作战环境中部署 AI，将其用于‘敌我识别’，一旦算法做出错误判断，直接后果就是人员伤亡。 — Ajay Chakravarthy > [01:26](https://www.youtube.com/watch?v=I55hQadX6v4&t=86s) 盲目追逐最新的前沿模型毫无意义，因为模型从来都不是 AI 系统的核心主体。在实战环境中，AI 始终运行在一个庞大的‘体系之体系’之内。 — Ajay Chakravarthy > [03:19](https://www.youtube.com/watch?v=I55hQadX6v4&t=199s) 团队的目标绝不是成为一个只坐而论道的‘好点子俱乐部’，而是要把技术推向实际的作业场景。 — Ajay Chakravarthy > [05:16](https://www.youtube.com/watch?v=I55hQadX6v4&t=316s) 当前许多厂商宣称的边缘服务器动辄需要一个‘像冰箱一样大’的机架，这种庞然大物在空间逼仄的潜艇或战车内部根本无法安装。 — Ajay Chakravarthy > [05:16](https://www.youtube.com/watch?v=I55hQadX6v4&t=316s) 不要盲目认定‘AI 就是万灵药’。首先弄清待解决的业务本质，AI 可能适用，也可能根本不是最佳解法。 — Ajay Chakravarthy > [09:22](https://www.youtube.com/watch?v=I55hQadX6v4&t=562s) 单纯调用前沿大模型写演讲稿或策划点子，根本谈不上是智能体，那仅仅是与大模型展开对话。真正的智能体意味着从‘对话沟通’跨越到‘赋予代理权以执行操作’。 — Ajay Chakravarthy > [09:22](https://www.youtube.com/watch?v=I55hQadX6v4&t=562s) 那根本不是智能体，那只是一套确定性工作流。如果业务场景要求 100% 的路径确定性，团队就应该直接去写确定性规则和工作流代码，根本不需要引入智能体。 — Ajay Chakravarthy > [09:22](https://www.youtube.com/watch?v=I55hQadX6v4&t=562s) 模型仅占 10%……系统 90% 的效能与可靠性取决于包裹在模型外层的工程外框（Harness）。 — Ajay Chakravarthy > [12:02](https://www.youtube.com/watch?v=I55hQadX6v4&t=722s) 如果有人在国防、航空航天等高危自动化领域吹嘘‘我花三天用氛围感编程手搓了一个原型，能不能直接部署进系统？’，CTO 应当当场回绝并亮起最高级别红灯。 — Ajay Chakravarthy > [12:02](https://www.youtube.com/watch?v=I55hQadX6v4&t=722s) 在关键核心环节，在可能引发动能毁伤或决定性实弹打击的操作链路上，严禁 AI 擅自裁决，必须将最终决策权收束至人类指挥员手中。 — Ajay Chakravarthy > [21:06](https://www.youtube.com/watch?v=I55hQadX6v4&t=1266s) 很多具体任务并不需要调用体积庞大、计算昂贵的语言模型，系统完全应该为特定场景匹配最适宜的模型类型。 — Ajay Chakravarthy ## Mentioned Ajay Chakravarthy, 泰雷兹 (Thales), Thales UK, cortAIx, cortAIx Labs, cortAIx Factory, cortAIx Sensors, TrUE AI, C-SWAP, ResNet-18, SAUC, Gemma 12B, GraphRAG, Google, 英国国防部 (Ministry of Defence), Jetson Nano, 树莓派 (Raspberry Pi), GitHub ## Video description Ajay Chakravarthy - Chief AI Officer, Thales UK I will challenge the assumption that mission-critical AI demands planet-melting data centres and show how intelligent agents can operate securely and reliably at the edge. In a world of constrained bandwidth, contested environments, and tight power budgets, pushing AI closer to where decisions are made is not just efficient; it is often essential. I will walk through example implementations developed within cortAIx - Thales' global accelerator for AI, showing how we deploy AI agents on edge platforms across defence, transportation, and critical infrastructure, while preserving robustness, security, and human-on-the-loop control. Attendees will learn practical approaches for combining small, specialised models with system-level autonomy, as well as strategies for validation, monitoring, and fail-safe behaviour in the field. The talk will equip you with a blueprint to move from cloud-only prototypes to trustworthy edge deployments that respect operational constraints and environmental impact, and will show how to make AI both mission-ready and responsible. [Source](https://www.youtube.com/watch?v=I55hQadX6v4)

[0:04] Ajay Chakravarthy: 如果为一家零售服装店开发聊天机器人，一旦系统犯错，最坏的结果不过是顾客收到了一件尺码或款式不对的衣服……然而，如果在国防作战环境中部署 AI，将其用于‘敌我识别’，一旦算法做出错误判断，直接后果就是人员伤亡。 [1:26] Ajay Chakravarthy: 盲目追逐最新的前沿模型毫无意义，因为模型从来都不是 AI 系统的核心主体。在实战环境中，AI 始终运行在一个庞大的‘体系之体系’之内。 [3:19] Ajay Chakravarthy: 团队的目标绝不是成为一个只坐而论道的‘好点子俱乐部’，而是要把技术推向实际的作业场景。 [5:16] Ajay Chakravarthy: 当前许多厂商宣称的边缘服务器动辄需要一个‘像冰箱一样大’的机架，这种庞然大物在空间逼仄的潜艇或战车内部根本无法安装。 [5:16] Ajay Chakravarthy: 不要盲目认定‘AI 就是万灵药’。首先弄清待解决的业务本质，AI 可能适用，也可能根本不是最佳解法。 [9:22] Ajay Chakravarthy: 单纯调用前沿大模型写演讲稿或策划点子，根本谈不上是智能体，那仅仅是与大模型展开对话。真正的智能体意味着从‘对话沟通’跨越到‘赋予代理权以执行操作’。 [9:22] Ajay Chakravarthy: 那根本不是智能体，那只是一套确定性工作流。如果业务场景要求 100% 的路径确定性，团队就应该直接去写确定性规则和工作流代码，根本不需要引入智能体。 [9:22] Ajay Chakravarthy: 模型仅占 10%……系统 90% 的效能与可靠性取决于包裹在模型外层的工程外框（Harness）。 [12:02] Ajay Chakravarthy: 如果有人在国防、航空航天等高危自动化领域吹嘘‘我花三天用氛围感编程手搓了一个原型，能不能直接部署进系统？’，CTO 应当当场回绝并亮起最高级别红灯。 [12:02] Ajay Chakravarthy: 在关键核心环节，在可能引发动能毁伤或决定性实弹打击的操作链路上，严禁 AI 擅自裁决，必须将最终决策权收束至人类指挥员手中。 [21:06] Ajay Chakravarthy: 很多具体任务并不需要调用体积庞大、计算昂贵的语言模型，系统完全应该为特定场景匹配最适宜的模型类型。

Ajay Chakravarthy: I will challenge the assumption that mission-critical AI demands planet-melting data centres and show how intelligent agents can operate securely and reliably at the edge. Ajay Chakravarthy: In a world of constrained bandwidth, contested environments, and tight power budgets, pushing AI closer to where decisions are made is not just efficient; it is often essential. [0:04] Ajay Chakravarthy: planet-melting data centers [0:04] Ajay Chakravarthy: system of systems [2:14] Ajay Chakravarthy: club of good ideas [5:16] Ajay Chakravarthy: the size of a refrigerator [9:22] Ajay Chakravarthy: They want deterministic results and they want it to follow exactly the same path. That’s a workflow. So that’s not an agent. [12:08] Ajay Chakravarthy: vibe coding [17:54] oh, by the way, I've switched it on and off. What do I do now? Ajay Chakravarthy: move from cloud-only prototypes to trustworthy edge deployments that respect operational constraints and environmental impact

[0:04] So, hi, I'm AJ. I'm the chief officer for Talis  in the UK. If you don't know what Thalus is or you [0:10] never heard of it before, uh, it's got its head  headquarters in France and, uh, but it's a global [0:16] company. So, we operate in over 68 countries and  we have a workforce of over 86,000 employees. [0:23] And what thus does is we are technology leader in  mission critical applications. So we provide um [0:30] the we provide solutions to um aerospace, defense,  cyber security and space. So it's it's pretty big. [0:38] Um so today's talk I'm going to focus on um  what it means to deploy AI applications in in [0:46] mission critical environments. So what do I mean  by that? So if you're designing a chatbot to work [0:51] in a retail environment, so if you want to if you  want to design a chatbot for your fashion store [0:56] and your AI bot gets it wrong, you know, the the  customer has received the wrong the wrong set [1:01] of clothes. So okay, you know, the consequences  are are bad, but then they're not they're not uh [1:06] they're not severe. If you deploy an AI into a war  fighter where you have a friend or four detector [1:14] and you get it wrong there, that could lead to  loss of life. So how do you engineer AI solutions [1:20] to work in such mission critical environments?  So that's going to be the focus of my talk. So [1:26] um if we move to the practical realities of  deploying AI um at the edge, you've got all [1:32] the planet melting data centers which are quite  popular now and people are constantly running [1:36] after the next frontier model after the other. Uh  whilst that is great um on one hand for mission [1:43] critical deployment you don't actually need to  run after the next frontier model and always keep [1:48] you know constant um surveillance on what's being  released in sil silicon valley in in the past two [1:54] weeks because the model is not model is not the  main part of your AI applications. So with within [2:00] mission critical environments you always deploy  within a system of systems. So if you're deploying [2:04] something into a tank, it needs to it needs to  work with the legacy software system which is in [2:08] the tank already. So it's not about um chasing  after the next frontier model all the time. [2:14] And the other thing is we work with something  called frugal AI. So this is a concept which [2:20] tales has come up with frugal AI is not just  about lowshot learning. So frugal means you're [2:26] working in a denied degraded environment. So if  you want to deploy AI somewhere in the North Sea, [2:32] you have extremely harsh um environmental  conditions. You're not connected to you're [2:37] not connected to the internet. Um you need  to operate in a completely disconnected way. [2:42] You have low power requirements and you have  limited amount of data and all that needs to [2:48] work with the level of reliability um in a high  consequence environment. So how do you design AI [2:53] to to fit into these kind of scenarios? So that's  what that's what frugal AI is all about. So you [2:57] look at your compute power, you look at your  energy supply and you look at your bandwidth. [3:02] So um within defense at least in the UK we call  it core to edge capability. So core capabilities [3:07] where you deploy your agentic setup or whatever  it is you want to deploy um at the data center [3:13] but your AI needs to work at the edge in in  a completely denied and degraded environment. [3:19] So for this purpose uh we've uh we've got cortex  in thalis. So Cortex is uh the global accelerator [3:27] for AI inales. So we don't want to become um um  a club of good ideas. So good ideas are great. [3:34] Everyone has them. But how do you take those good  ideas and actually deploy it into operational [3:38] theater? So that's that's the thing we are we are  mostly focused on. Um so Cortex is divided into [3:44] three three groups. Cortex labs which does the  research. Um so it takes all the good ideas and [3:49] create prototypes to see what is possible. So out  of the possible uh proof of concept demonstrators, [3:55] cortex factory is where we take the outputs  from the lab and engineer it into the deployable [4:01] environment. So the deployment environment could  be a cyber security system, it could be physical [4:06] hardware, it could be something in space and  then you've got cortex sensors. So sensors [4:12] takes the product which has been engineered in the  factory and deploys it into the product itself. [4:18] So if you look at that we are we always operate  on this concept of true AI. So true stands for [4:23] transparent which is you need some level of  transparency on where your data prominence what [4:27] your data provenencies how your model has been  trained. How are you monitoring for drift from [4:32] the model just because you've deployed a model and  you deployed it somewhere in the North Sea doesn't [4:36] mean it's going to perform uh over time. So how do  you how do you how do you measure the performance [4:40] drift? It's reliable which is you are able to  produce the results um under varying conditions [4:47] and you are able to do that reliably. It's ethical  uh especially in Europe the ethical charter which [4:54] has been enforced now where we you you must  follow um certain guidelines and rules and [5:00] principles when you deploy AI applications. So we  need to be quite stringent with those things and [5:04] finally the explanability of it which is you know  how do you how do you explain what the result is. [5:09] So that that forms this narrative of true AI.  And then if you go into the um next one. So for [5:16] tactical for tactical edge um you acquire certain  principles which is you always need to be um [5:22] aware of the cost which is you know it's not just  token cost but what's the total cost of deployment [5:29] uh the size and proportionality. So if you have  a massive server, so when people talk about edge [5:34] deployments, they still come up with a rack which  is about uh the size of a refrigerator and that's [5:39] still not deployable if you want to deploy that  into the submarine for example. So it needs to be [5:43] it needs to have some size constraints for it.  Uh what's the weight? So with size comes weight [5:48] as well. And then what are the power constraints  for it? So I'm going to talk about two examples [5:54] um in in this um um particular set of principles.  The first one is what I call is traditional AI. [6:02] So Talis in Canada has designed something  called C swap which is a causal uh causal aware [6:10] structured pruning methodology uh which is which  is using explanability and what that does is [6:16] basically you take you take a neural network say  a classification network a deep neural network [6:20] and then you start to remove neurons from it  which are not relevant for that particular [6:24] classification and then you prune the size of  the network to a point where you can deploy it [6:29] in an edge environment without losing  the performance. or accuracy of of that. [6:34] So in the last slide I think I' I've got some  references on on C swap. So you can go and read [6:39] the paper yourself. It's quite a detailed paper  and we made it open source as well. So you can [6:44] go to GitHub, download it and try and use it  for yourself. So essentially how it works is [6:50] it goes through a post train network. So you're  not doing this uh as you're training your neural [6:55] network. you you trained your neural network  already and then you pass it through C swap [7:00] where you categorize each neuron into a critical  node which is you know removing that particular [7:05] neuron is going to have an effect on the overall  performance of your of your result. So you keep it [7:10] a neutral node even if you remove that neuron it  makes no difference to the end result. So in which [7:15] case you remove it and then a detrimental node  which is by removing that you actually improve the [7:20] performance of your classification for example  in that case you you definitely remove it. So [7:26] once you do this and then you evaluate the  performance of the neural network as a whole for [7:31] every neuron then you're able to progressively  reduce the size of your neural network [7:36] and um you'll see it in the paper uh  we've conducted some results against [7:40] standard data sets evaluation data sets and  for this particular ResNet 18 data set um [7:47] even when you uh prune the network  by up to 50% it doesn't result in [7:53] um reduction in performance And after that it  it falls um quite sharply as you can see there. [7:58] The other thing which the paper discusses is  something called source metric which is uh [8:02] sparsity area under the curve which is showing you  the trade-off between the performance versus the [8:07] amount of neurons you've pruned in the network and  that shows the highest amount highest performance [8:13] amongst all similar um techniques which are  which are in state-of-the-art at the moment. [8:18] So again I suggest that you uh read that paper u  take a screenshot of the slide right at the end. [8:24] So if you're looking at edji then uh that's the  kind of recommendation which we follow in talis [8:32] at least and that seems to work which is define  your problem first. People are quite easily [8:38] uh prone to solutionering which is you know AI  is the solution now let's try and find problems [8:43] to fit around it. So that's that's absolutely  the wrong way. Um at least in our view define [8:48] what the problem is do what solve first. AI may  be the solution, AI may not be the solution and [8:53] then build a semantic model around it so that you  define um in machine understandable ways what what [8:59] exactly is the profile of the data you're training  it on what exactly you want it to learn and what [9:03] you want it to predict and then use something  as C swap to compress in your neural network. [9:10] um apply the true AI principles which  I discussed earlier and then deploy it [9:14] uh within the within the constraint environment.  So that's the thing for um for traditional AI [9:22] and then the hot topic of the day uh all the talks  I'm seeing at least in this conference they seem [9:25] to be focused around agentic AI um and we are  looking at that as well. So when you look at [9:32] um agentic AI, if you're look if you're  looking at the latest frontier model and [9:37] asking it to write your speech or um you  know design your um design your next idea, [9:43] that's not really agentic AI. It's just you're  having a conversation with the frontier model. [9:47] Agentic AI is when you move from having simple  conversations to letting the agent do stuff. So [9:54] and once you start you know once once you start to  give agency to your agents then how do you control [10:00] it and that's the other problem especially in the  ministry of defense we come up with which is they [10:04] want completely stringent controls on what the  agent is doing. They want deterministic results [10:11] and they want it to follow exactly the same  path. That's a workflow. So that's not an agent. [10:16] So again you need to decide whether you want a  workflow or is it is it an agent you want because [10:22] using agencies when you give it some agency  you give it some you give it some room to make [10:27] decisions and uh without that maybe you don't  need um um an agentic network network to solve [10:33] that particular problem if you want complete  determinism in what you want to achieve. [10:36] So once you move into um agentic engineering  then I think the last speaker spoke about it [10:42] as well the first thing is your is your context.  If you're running a 12-hour task to um redefine [10:49] your entire code base or it could be some other  task which takes a very long time then uh frontier [10:55] models often start compacting your context  and that's where they start forgetting things [10:59] and that's where hallucination comes in as well.  So if you're running a 10 12 hour task and you're [11:05] expecting it to run using a single agent then  that's that's again bad design I would say. So [11:13] we are following this principle um where an agent  is your model. So your frontier model or whatever [11:19] model it is you want to use is only 10% of  the overall goal um um of your architecture. [11:27] 90% of it is achieved through the  harness you designed around your model. [11:31] So in the case of uh mission critical environments  you often need to deploy models which are on prem. [11:36] So that does not mean you take trillion par  trillion parameter models and you deploy them [11:41] into into the edge environment. Often you can  get away with using a small language model. So [11:46] Google's released a number of models like Gemma  for example. Um the 12 billion parameter model [11:51] might be more than enough to do the task you  want to do. So the first thing to do is have [11:57] have some discovery on what kind of model  do you need to perform your particular task [12:02] and then design a harness around it. So what is  the harness? So these are the key ingredients for [12:08] any kind of agentic harness. So let's start from  the beginning. System prompts which is what what [12:13] are the what are the generic rules you want your  agents to follow. So what what what to do and what [12:19] not to do observability. So there's lots of talk  about observability already which is how do you [12:25] monitor your agents u that they're doing what they  are supposed to do. How do you monitor the cost? [12:30] uh you've seen horror stories in in in the news in  recently where you know the costs have gone up to [12:35] millions of pounds because people people didn't  have enough observability on their token costs [12:39] in edge environments because your model is on  prem you may not need to monitor the to cost as [12:44] such but you still need to monitor the overall  resources of uh how you're spending your u the [12:48] compute and then you've got your feedback loops  which is it's not enough to just ask the agent to [12:54] do stuff in a sequential manner. You often need to  iterate again and again until the level of quality [13:00] for your particular task is met. So how do you  define your iterative loops? Memory management. [13:05] Are you going to maintain your entire context in  memory for the entire duration of the task or are [13:09] you going to persist it? Particularly if you're  using multiple agents. So often for complex task [13:15] the best architectures use many sub aents to do  that task. What that means is each one of those [13:21] sub aents has its own context and it does the task  with a fresh context. learns from it, persists the [13:27] memory and then passes it on to the next agent. So  how do you do the handoff uh between these agents [13:33] guardrails and hooks. So uh you could have some  you know some stringent measures there you know [13:39] should never take root access could be one  measure there could be many others for your [13:42] particular system. So you define all those in  there. What's your orchestration logic? How do [13:48] the agents work with each other and how do you how  do you bring human human into the loop as well. So [13:54] uh for mission critical scenarios in particular,  you often need human in the loop systems [13:58] which is if a decisive action which is leading to  kinetic effect is is going to occur then you often [14:04] need a human user to make that decision rather  than an agent making the decision on its own. [14:09] So again that's something to consider. And the  last thing is sandboxes which is before going into [14:14] deployment itself you want you want to be able  to test that particular thing within a sandbox [14:19] environment um and then do all the edge case  testing etc before going into into deployment. So [14:26] that that we would say is is a harness um around  your model. So as I said in the previous slide [14:34] uh the model is only 10% of of your solution. 90%  of it is how effectively you design your harness [14:41] and that helps with your um your costs as well.  So if you're going for proper agentic engineering, [14:50] you start with a higher cost initially, but then  the blue line there you see indicates you you [14:56] taper down the cost and then you maintain the cost  over time because you're managing your context. [15:01] You've defined your harness properly. So it's  sustainable in the long term. If you start with [15:07] vibe coding again this is a popular phrase. So  if someone comes to you especially in defense [15:13] space automation and says hey I've w coded this  prototype shall we deploy it because it's only [15:19] going to take 3 days. I think as a CTO the first  thing you should say is no. So that should raise [15:24] um red flags already because although it starts  out cheap eventually because it's not properly [15:30] engineered the cost of it and the hidden debt uh  raises all the way around and then you end up with [15:35] more problems than you started. So investing  in proper agentic engineering is is quite key. [15:42] Right? So if we go to the next one um so that's  where uh there's a really interesting paper from [15:51] uh from Google on u how agentic AI is changing  the software development life cycle. So again [15:58] that's in the references you'll see it in  a bit where you know traditional SDLC you [16:03] started with requirements you came up with the  design implementation you do some testing and [16:08] then review and deploy I think that's that's  that's completely changed now in the sense that [16:13] you you are able to use agentic setups for  every phase um in this particular cycle [16:19] particularly uh when you get to implementation  and testing uh if you're managing very large [16:24] code bases with that then you're able to not  not only write the code, maintain the code, [16:29] but also test and verify the code before  you deploy all using an agentic setup. [16:34] But having a proper having a proper harness  around your agentic setup is quite key. [16:39] So these are some of the lessons learned  for us in thalis as we've deployed um [16:44] um AI into mission critical environments. So um  we've done quite a few deployments. So one of the [16:50] one of the things we've done recently is something  called a maintenance operator. So this is for uh [16:56] people who want to diagnose problems in  ships. So they've got lots of fault logs, [17:02] they've got lots of user manuals, they've got lots  of repair logs on how how things are repaired. [17:08] And then we fed all of that into typically a rag  agent and soon we found that a simple rag agent [17:13] is not is not enough for that purpose. You needed  to use something called graph rag where you have [17:16] an ontology on top of it in order to give you  the most accurate results. But then the problem [17:21] of mission critical deployment came into being  because we couldn't use an API to link to your [17:26] open AI model or whatever it is. So we had to  deploy it at the edge to work in a completely [17:32] disconnected way using a small language model.  So that's where you had to engineer engineer [17:37] the harness to make sure that the results it  was producing what it does is diagnosis. So [17:41] if you're in shape something goes red the the  red light starts blinking and then you say hey [17:46] light's blinking red what do I do? And  then it goes through your user manuals, [17:50] your previous fault logs. And then it says, why  don't you try this method? And then it maintains [17:54] the context. So you can say, oh, by the way, I've  switched it on and off. What do I do now? And then [17:57] you can continue the conversation. So that that  was an example of successful deployment. We've [18:02] deployed in many other places as well. Um, which  we won't have time to go into all of that now. Um [18:09] and then the last thing is uh what I was  promising before that's the literature [18:14] where you can read the paper on C swap as  well and you can download some of the um [18:19] you can download the code which is on which  is on GitHub for C swap. I've still got [18:25] 2 minutes left so happy to answer any questions  you may have. Um if not thank you. Yeah. [18:35] So how do you maintain? So it's a airgapped. So  let's say you you take you take a data set copy [18:41] to the edge along with this SLM you you deploy  it and it works and after say couple of months [18:48] new data comes in the refresh. How do you manage  the refresh? So reference with respect to data. [18:58] So uh we design our architecture so it's model  agnostic. you can switch the model out and you [19:02] know you can slot in a different different SLM  for that matter but the way we are doing the [19:09] reindexing of the data is you revector so for  this application I was talking about where it [19:14] does system maintenance you you vectorize your  data again so within the edge environment yeah [19:23] um okay so uh so you were talking about  pruning okay so this is a multiart I apologize. [19:33] Um, so pruning, you're typically prompting  it. You're giving it questions and seeing [19:39] what which weights light up and then you're  extracting out the the ones that don't. Um, so [19:46] what model, what size are you starting with? How  many parameters are you starting with? And then [19:50] how many parameters are you uh ending up with that  you are installing on their system? And are you [19:58] are are you forcing them to purchase like a GPU  rack or are you doing this directly on their [20:05] server like a small enough like a seven or 8b  model that can carry out that's big enough that [20:11] it can carry out tasks but small enough that  you know five people at once can use the same. [20:18] So yeah uh really good question the the  pruning mechanism I was talking about was [20:22] more for traditional ML applications. So we take  a deep neural network for an image classification [20:27] task for example and then we start pruning that.  So that could be you know millions of parameters [20:32] but then you you prune it prune it down to a  stage where you can deploy it on a JSON nano [20:37] or a Raspberry Pi. So that those are the orders  of scale we are talking at but what we found was [20:43] uh that method is quite scalable. So you can  use it for large language models as well. Um but [20:49] having said that nowadays the the amount of models  which are being released particularly in the open [20:54] source both at the small and the large scales we  don't know if we really require something like C [21:00] swap for frontier models and large language models  because they're being released anyway. Yeah. Yeah. [21:06] So what what are the steps you're taking or  are we even thinking about like routing models [21:13] which might be even more lightweight but more  deterministics like we have seen timelessly like [21:19] simple decision classifiers or like  traditional statistical models turn [21:23] out really super lightweight accurate enough  for these kind of systems and applications. [21:28] So have we considered something in tandem  with something like C swap where it could be a [21:33] more contextual driver and it could take  signals from such smaller routing engines. [21:40] Yeah, again really good question and the answer is  is yes because we're looking at multiple models. [21:45] Um you're absolutely right in the sense that you  don't you may not require a language model for [21:51] particular tasks where if you you know you need  to find the right model for your particular use [21:56] case and then route it to to that particular  model. So that's where again the the harness [22:00] comes in because if you got if you got your own  harness set up in the right way then it can call [22:04] just like calling tools it can choose the right  model at the right time. So that's something we [22:09] are we are looking at we haven't done it fully.  I can't claim to have uh full you know to have to [22:14] have done it but we are looking at multimodel  routting. The other thing we're looking at [22:19] um using uh different models is for eval as well.  So if you want to evaluate the performance of [22:25] a small language model or a language model we're  using we're using a different different language [22:30] model to evaluate the performance of the results  etc. So we we we're using both those approaches.