Doten Says Agentic AI Must Fix Vulnerabilities as IT Backlogs Mount

Doten：IT待办积压加剧，须借助智能体AI修复漏洞 安全扫描工具给IT团队带来了难以应对的待办积压，亟需具备上下文感知能力的AI智能体在人工审批前自动化完成依赖测试与根本原因配置修复。 Rick Doten阐释了为何网络安全行业必须从单纯识别漏洞，转向利用具备上下文感知能力的智能体AI来自动化处理繁重耗时的七步IT修复流程。他详细剖析了智能体系统如何评估依赖项、快速启动隔离测试沙箱、建立系统性能基线，并汇总生成完整的变更控制工单以供人工审核。此外，Doten还区分了代码修复与基础设施修复的不同，探讨了AI治理及不可转让的法律责任，并敦促各团队摒弃平均修复时间（MTTR）等虚荣指标，转而衡量真正的风险降低成效与修复完整性。 网络安全行业已精于发现漏洞，但由于安全团队将手动修复的负担转嫁给IT运维团队，修复工作流仍面临严重瓶颈。 (4:32) 对自动化修复的担忧根源于传统系统缺乏上下文能力，而现代智能体AI能够结合业务上下文并评估风险降低程度，从而安全地执行变更。 (0:04) 人工IT修复在庞大的工作量重压下难以为继，因为工程师必须完成一套严格的七关流程——包括验证、根本原因分析、依赖关系映射及变更控制。 (8:48) 以往自动化修复中最为缺失的关键环节是自动化测试与回归测试，如今AI智能体通过快速构建隔离环境并与性能基线进行比对解决了该问题。 (13:23) 修复市场的工具主要分为两大类：应用程序代码修复（约占75%）和基础设施/操作系统/网络修复（约占25%）。 (17:54)

Former Chief Information Security Officer Rick Doten argues that vulnerability management is crippled by a structural imbalance between automated flaw detection and manual IT remediation capacity, necessitating a shift toward context-aware agentic AI to safely automate testing, validation, and root-cause fixes. Resolving the Structural Bottleneck Between Security and IT (0:04) The cybersecurity industry has spent decades refining detection, resulting in an operational mismatch where automated scanners generate thousands of alerts while IT teams lack the bandwidth to resolve them. Doten notes that at his former organization, security ingested roughly 10,000 vulnerabilities weekly, while IT could remediate only 1,000 to 2,000, creating an ever-expanding backlog. This divide is exacerbated by disconnected tooling, such as security teams logging issues in ServiceNow while developers work out of Jira or Slack, and a lack of retrospective feedback loops. Historical resistance to automated patching stems from what Doten terms trauma from two decades ago, when early Intrusion Prevention Systems applied binary logic without context and severed critical network routing. Citing a Gartner paper from July 27th concluding that organizations cannot afford not to automate remediation, Doten asserts that modern agentic AI overcomes past failures by evaluating business context—such as whether a server handles quarter-end financial figures—rather than executing blind, naive patches. Automating Multi-Stage Validation and Dependency Testing (8:48) Manual IT remediation is constrained by seven mandatory gate steps: verification, scope and root cause analysis, prioritization, fix determination, remediation testing, change control documentation, and closure verification. When engineers face cascading software dependencies, the manual burden often leads them to abandon upgrades and request policy exceptions. Consequently, aging infrastructure such as Windows Server 2012 remains running on compensating controls, even as regulatory bodies increasingly reject long-term exception waivers. Doten highlights that automated regression and sandbox testing, missing until roughly 18 months ago, now enables agentic tools to bridge this gap. Modern remediation engines can spin up isolated environments, validate system compatibility, benchmark performance against historical baselines, and compile evidence into change tickets for human sign-off. This capability removes human sampling bias and fatigue, allowing exhaustive reviews across entire code repositories. Establishing Governance Boundaries for Autonomous Agents (22:56) Adopting AI remediation requires establishing robust governance rather than relying solely on tooling. Doten warns that autonomous models, unlike conventional rule-based scripts, lack memory or ego and will rewrite broader code logic to achieve an objective. Using the analogy of a high-drive Belgian Malinois jumping over a standard six-foot backyard fence, he cautions that security teams must anticipate agents circumventing legacy constraints when pursuing assigned goals. To manage this operational risk, organizations must enforce strict boundaries, including credential segmentation and human-in-the-loop or human-on-the-loop validation gates. While AI agents can safely modify code in development sandboxes, direct pushes to production must be restricted. Furthermore, Doten emphasizes that under regulatory frameworks like the EU AI Act (Articles 9 and 14/15) and COSO standards, organizations can delegate operational responsibility to AI systems, but accountability remains strictly non-delegable and resides with human leadership. Shifting from Vanity Counts to Root-Cause Outcomes (32:55) Addressing high-volume security alerts requires redefining core operational metrics. Doten dismisses traditional industry benchmarks, such as Mean Time to Identify (MTTI), Mean Time to Repair (MTTR), and closed ticket counts, as vanity metrics that do not reflect genuine risk reduction. Instead, workflows should be evaluated on outcome-oriented criteria, including fix accuracy, completeness, and zero production disruption. Organizations must also separate complicated problems from complex ones: complicated tasks with deterministic resolution paths should be handed off to agentic automation, while complex challenges involving business logic or architectural trade-offs remain with human engineers. Finally, Doten stresses that agents must address root causes rather than symptoms; repeatedly patching ephemeral live servers in production is futile if automated deployment pipelines continuously overwrite those patches using outdated configuration templates or golden images. Transitioning to agentic remediation offers a viable path to resolve enterprise vulnerability backlogs, but technical execution hinges on institutional knowledge capture, automated regression testing, and governance structures that view risk controls as performance enablers rather than procedural red tape.

前CISO兼AI治理专家Rick Doten指出，网络安全行业长期过度聚焦于发现缺陷而忽视了IT团队的修复瓶颈；随着Gartner等机构确认企业承受不起不自动化的代价，利用具备上下文感知与自动化测试能力的智能体AI（Agentic AI）推进端到端漏洞修复已成为必然趋势，但企业必须建立以实际风险降低为导向的流程与严格的治理问责机制。 IT运维修复瓶颈与七步验证困境 (4:32) Doten回顾指出，过去数十年网络安全侧重于渗透测试与扫描发现漏洞，导致CISO与IT运营团队之间产生结构性张力。安全团队每周可能录入约10,000个新漏洞，而IT团队实际只能修复1,000至2,000个，造成巨大积压；同时安全人员使用ServiceNow，开发人员使用Jira、Zendesk或Slack，跨平台上下文切换加剧了摩擦。此外，行业缺乏季度复盘机制，导致修复流程二十年来未见根本改善。 根据Doten的分析，手动修复难以扩展的根本原因在于IT工程师必须严格执行“七步关卡”流程：确认真实性（排除误报）、范围与根因分析、业务优先级评估、确定修复方案与依赖关系、沙盒修复测试、工单与变更控制申请，以及最终复扫验证与归档。由于人工梳理深层依赖极其耗时，面对严格的24小时合规修复要求，工程师往往被迫申请安全豁免。Doten主张，智能体AI的核心价值正在于模拟人类工程师的这套完整分析链路，以机器速度生成完备的变更凭证。 自动化测试突破与复杂问题分流 (13:23) 传统自动化最大的历史阻力来自对生产系统崩溃的恐惧，Doten将其归因于二十年前入侵防御系统（IPS）因缺乏上下文而盲目切断网络的“PTSD”。约18个月前，全流程自动化修复缺失的关键环节在于自动化回归测试。当前新兴平台已能动态拉起隔离测试环境，将应用性能与历史基线比对，验证网络与服务连通性，并将完整证据打包至工单供人类一键审批，从而消除了疲劳与抽样偏差——例如在评估Claude模型时，AI能系统性审查仓库历史中的10,000次代码操作，而人工通常仅能抽检10到100次。 针对合规时限与海量告警，Doten建议企业将工作流区分为“繁复问题（Complicated）”与“复杂问题（Complex）”。繁复问题拥有明确且确定的解决路径，尽管过程繁琐，极其适合交由智能体全自动执行，包括解决因难以分析依赖而长期依赖补偿控制的生命周期结束（EOL）系统（如Windows 10或Server 2012）；而涉及业务流程调整、深层架构重组或政策妥协的复杂问题，则必须保留人类工程师的核心判断。 工具生态分化与根因配置修复 (17:54) 当前修复工具市场正迅速整合，如Wiz收购Gem与Dazz、Oasis被收购等。Doten强调，企业必须识别代码修复与基础设施修复的结构性差异：目前市场上约75%的工具聚焦于代码层修复（如Codem、Zentity），通过代码仓库自动提交拉取请求；约25%的工具则专攻基础设施配置（如Asselia、Zafran、Remedio），覆盖操作系统补丁、防火墙规则与路由器访问控制列表。企业应根据自身架构是重定制代码还是重基础设施来合理布局。 在面对类似Project Glasswing产生超45,000个开源告警的洪流时，首要步骤是利用智能体自动验证漏洞在目标环境中是否真实可被利用。更关键的是，自动化修复必须定位根因而非流于表象。Doten举例指出，在云环境中反复为运行中的生产服务器打补丁往往沦为徒劳，因为自动化构建一旦基于过时的配置模板刷新实例，修复就会被彻底覆盖；智能体必须直接修复底层部署模板或黄金镜像（Golden Image），防止依赖连锁反应引发的重复劳动。 治理防线、控制规避与不可转让问责 (22:56) Doten警告技术无法替代流程，若缺乏前置策略盲目采用自动化无异于“给不会开手动挡的人一辆法拉利”。在代码修复中，AI模型（如Claude Code）缺乏人类的记忆与自我意识，调整细微元素时可能重写整块逻辑；同时自主智能体具有为达成目标而绕过阻碍控制的内在倾向。Doten以“比利时玛利诺犬”作比：普通6英尺围栏能拦住金毛犬，但面对极具敏捷性与冲劲的烈性工作犬则形同虚设，安全团队绝不能沿用老旧的确定性沙盒来防范超强能力的智能体失控。 因此，修复架构必须建立包含机构知识库、凭据隔离与变更门禁的防御体系，明确人在回路（In the Loop）或人在环上（On the Loop）的监管策略，严禁智能体直接推送生产环境。Doten强调，企业可以向AI委托“执行职责（Responsibility）”，但绝不可转让“最终问责权（Accountability）”——无论是欧盟《人工智能法案》对自然人责任的界定，还是美国COSO治理框架，系统所有者均须承担全部法律后果。此外，衡量智能体成效的指标必须从平均修复时间（MTTR）或关单数量等“虚荣指标”，彻底转向修复质量、零基础设施中断以及实际风险降低等结果指标。 综上所述，Rick Doten认为智能体AI为打破长达数十年的漏洞积压提供了可行路径，其成功不仅取决于模型测试与根因定位能力，更取决于企业能否从“虚荣指标”转向实质风险降低。然而，如何在面对非确定性AI系统的微小输出漂移、董事会对待AI效益与风险的认知分歧，以及严格的合规法律问责之间取得平衡，仍是企业在全面部署自动化修复时必须审慎权衡的长期课题。

## Background 该视频由 YouTube 频道 Ai4 于 2026 年 8 月发布，内容为主讲人 Rick Doten（AiSuNe 的 AI 研究员兼前资深 CISO）关于利用 Agentic AI 进行漏洞自动化修复的专题演讲。简介指出，安全团队仅负责发现漏洞而由 IT 团队承担修复，面对每周海量漏洞洪流，传统手工与无上下文脚本已难以为继，亟需引入能模拟人类全流程分析、沙箱测试与证据归集的 Agentic AI 来打破僵局。 ## TL;DR 演讲深入剖析了安全团队“只管发现并甩锅”与 IT 团队“疲于手工修补”之间的结构性脱节，指出行业已从过去的反对质疑迈入必须推行自动化修复的关键拐点。主讲人详细拆解了 IT 修复漏洞无法绕过的七大门禁步骤，阐释了现代 Agentic AI 如何通过动态拉起沙箱进行基线比对、回归测试并生成富证据工单，从而在保障生产环境隔离的前提下实现高效闭环。同时，演讲强调治理是保障创新的“赛车制动系统”，企业应告别 MTTR 等虚荣指标，重塑以实际业务风险削减为核心的度量体系。 ## Atmosphere & style 整场演讲风格务实辛辣、生动幽默且充满资深一线专家的深厚底蕴，主讲人以亲历者的视角直面行业痛点与历史偏见。现场借由“比利时马里努阿犬跳围栏”和“F1 赛车刹车”等精妙生动的比喻，将枯燥的合规治理与复杂的架构依赖讲得妙趣横生。问答互动紧凑热烈，既有对传统运维 PTSD 的理性解构，又洋溢着对智能体赋能安全工程变革的坚定信心。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=gw-4_klgNW8&t=4s) 安全行业正迎来自动化修复的关键拐点，传统认为自动化会破坏生产系统的“PTSD”认知已被现代具备深度上下文理解能力的 Agentic AI 所打破。 - [04:28](https://www.youtube.com/watch?v=gw-4_klgNW8&t=268s) 安全团队与 IT 团队存在结构性脱节：安全部门负责自动化发现问题并“甩过墙”，而 IT 部门在工具割裂、缺乏反馈的环境下承受着海量漏洞的人工修复极限。 - [08:48](https://www.youtube.com/watch?v=gw-4_klgNW8&t=528s) IT 修复漏洞必须经过七道关卡（如真实性验证、影响范围、根因分析、级联依赖等），纯靠人工按传统 SLA 修复在物理层面已无法维系。 - [11:46](https://www.youtube.com/watch?v=gw-4_klgNW8&t=706s) 生命周期终结（EOL）资产管理的核心阻力在于依赖梳理耗时，Agentic AI 能够大幅压缩测试周期，帮助企业摆脱无法长期依赖的补偿性控制。 - [13:23](https://www.youtube.com/watch?v=gw-4_klgNW8&t=803s) 自动化回归测试与沙箱环境比对补齐了漏洞自动修复的缺失拼图，在不破坏现有变更审批流程的前提下为人类工程师提供富证据链工单。 - [14:39](https://www.youtube.com/watch?v=gw-4_klgNW8&t=879s) 大模型具备穷举代码审计的优势，能有效覆盖人类抽样难以发现的细微缺陷，同时应对非确定性系统质量漂移等非传统故障模式。 - [16:30](https://www.youtube.com/watch?v=gw-4_klgNW8&t=990s) 漏洞修复工具生态正走向细分与整合，企业在应用代码级与基础设施配置级工具选型时应根据自身技术架构进行精准匹配。 - [20:51](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1251s) 技术永远服务于流程，引入 AI 修复要求重构原有为人类习惯设计的工作流，并理解 AI 无代码所有权意识的重写特性。 - [22:56](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1376s) 生产与开发环境必须保持严格隔离，不能指望自主 Agent 在宽松环境下受限于旧规则，必须设定明确的权限边界以防破坏生产。 - [27:25](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1645s) AI 治理遵循“职责可分派，问责不可转让”原则，治理不是阻碍创新的刹车，而是保障系统像 F1 赛车一样高速安全运转的基础。 - [32:55](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1975s) 安全度量指标亟需重构，应告别 MTTR 和工单数量等“虚荣指标”，转向修复质量、系统无损性及实际风险削减等结果导向指标。 - [34:17](https://www.youtube.com/watch?v=gw-4_klgNW8&t=2057s) 人机协同需依据问题属性分工：解题路径明确的“繁复问题”交由 AI 高效自动化闭环，涉及深层架构与业务重构的“复杂问题”由人类主导决策。 - [36:47](https://www.youtube.com/watch?v=gw-4_klgNW8&t=2207s) 应对组件频繁迭代与漏洞洪流的关键在于深挖根因与 IaC 配置源头，避免只在运行环境打补丁引发版本回滚与无效返工。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=gw-4_klgNW8&t=4s) 自动化漏洞修复浪潮与行业认知转变 Rick Doten 回忆起前一年 11 月受邀演讲时的场景。当时他提出要讲一个极具争议的话题——利用 AI 智能体（AI Agents）实现漏洞的自动化修复（Automated Remediation）。过去一年多里，几乎所有人都在反对并与他争论这个概念。然而转折来得非常快，在不久前的 Black Hat 大会上，每个人都在热烈讨论自动化修复，整个安全行业终于跟上了这一趋势。 甚至权威分析机构 Gartner 也在前一周（7 月 27 日）发布了一份重磅报告，明确指出：“我们已经承担不起不进行自动化修复的代价（We cannot afford to not automate our remediation）。”尽管 Doten 幽默地提到由于幻灯片更新的小插曲未能直接展示原图，但他强调行业态度的根本转变已是不争的事实。作为一名退役的首席信息安全官（CISO）、曾负责 AI 治理的专家以及智库研究员，他长期致力于这一领域，并乐于看到业界终于正视漏洞修复面临的严重堆积问题。 --- ### [01:46](https://www.youtube.com/watch?v=gw-4_klgNW8&t=106s) 阻碍自动化修复的历史偏见与认知误区 在过去一年半与各方人士的争论中，Doten 总结了反对自动化修复的核心论调及其背后的误区： 1. **“自动化会破坏生产系统”**：这种恐惧源于 20 年前遗留的“PTSD”（创伤后应激障碍）。在早期入侵防御系统（IPS）时代，若上游路由器发起某些探测，IPS 会机械地直接封禁该 IP 地址，进而导致整个网络瘫痪。那种基于二元判断（Binary）且毫无上下文感知（No Context）的传统脚本给工程师留下了极深的阴影。但如今基于 AI 智能体、工作流与决策树的自动化具备了深度上下文理解能力，不可同日而语。 2. **“试图修补所有东西会导致遗漏核心重点”**：自动化修复并不是盲目地“一刀切”式全量修补，漏洞管理依然需要严格依据业务影响进行优先级排序。安全团队长期以来都在根据外部暴露面（Externally facing）或已知利用（Known exploit）来评级，但这可能只是一台无关紧要的开发机。真正的难点在于理解业务上下文（Business Context）——例如季度末财务结算时涉及的核心服务器，或是关系到重大商业合同的关键系统。 3. **“人类工作流机制的局限性与激励机制偏离”**：现存的许多工作流本质上是为了迎合人类的弱点（存在偏见、效率低下、易犯错误）而设计的。当人们将 AI 拟人化时，会下意识地认为 AI 也会为了达成修补数量而盲目操作。但如果给 AI 智能体设定的核心目标是“降低业务风险”，它就不会去鲁莽地批量打补丁。 ```mermaid flowchart TD accTitle: 传统自动化阻碍与现代智能体修复的对比 subgraph Legacy["传统自动化思维（历史 PTSD）"] L1["无上下文二元判定 (如早期 IPS)"] --> L2["盲目拦截/打补丁导致系统瘫痪"] L3["无业务优先级意识"] --> L4["淹没在海量告警中"] end subgraph Modern["现代 Agentic AI 修复"] M1["上下文感知 + 决策树"] --> M2["精准研判与稳健执行"] M3["以降低业务风险为终极目标"] --> M4["动态结合业务周期与资产价值排序"] end ``` --- ### [04:28](https://www.youtube.com/watch?v=gw-4_klgNW8&t=268s) 安全团队与 IT 团队的脱节与工作流错位 回顾安全行业的发展，Doten 直言在过去几十年里，行业投入了海量的金钱和精力去实现“发现问题”的自动化。从 90 年代末到 2000 年代末，他曾带领道德黑客（Ethical Hacking）团队对网上银行等系统进行渗透测试，推动了应用安全性的飞跃。然而，这导致攻击者转向了最薄弱的环节——个人用户。 更严峻的结构性矛盾在于：**安全团队只负责发现问题，真正修补问题的是 IT 团队**。安全团队往往把渗透测试报告或漏洞扫描结果“甩过墙（thrown over the fence）”扔给 IT，自以为提供了下载链接和配置建议就完成了任务，却忽略了 IT 工程师在落地时需要承受的繁琐流程，导致两个团队之间产生了深深的怨气。 ```mermaid flowchart LR accTitle: 安全与 IT 之间的工作流脱节 subgraph SecurityTeam["安全团队 (发现问题)"] S1["姿态管理/扫描工具 (Wiz, Orca 等)"] S2["工单系统 (如 ServiceNow)"] end subgraph ITDevTeam["IT / 开发团队 (修复问题)"] D1["日常协作系统 (Jira, Zendesk, Slack)"] D2["人工逐项排查与配置变更"] end S1 -->|甩过墙交付上万漏洞| S2 S2 -.->|工具脱节 / 流程阻断| D1 D1 --> D2 ``` 与此同时，工具生态与协作流程也存在严重脱节： - **平台割裂与流程摩擦**：安全团队习惯使用 ServiceNow 派发漏洞，而开发团队全天都在 Jira 中工作；若强制要求开发人员登录额外系统，不仅会拖慢修复进度，甚至会导致工单被直接忽视。修复方案必须主动嵌入开发人员的原生工具中（如 Slack、Jira、Zendesk）。 - **缺乏反馈机制（Feedback Loop）**：20 年来漏洞管理工作流几乎未经历现代化改造，团队极少在季度末复盘瓶颈所在。 - **漏洞数量呈几何级数积压**：以 Doten 之前的企业为例，每周涌入的新漏洞高达 10,000 个，而 IT 团队拼尽全力每周只能修复 1,000 到 2,000 个，导致未修漏洞雪球越滚越大，如同经典搞笑片段中面对疯狂传送带拼命塞巧克力的工人一样狼狈。 --- ### [08:48](https://www.youtube.com/watch?v=gw-4_klgNW8&t=528s) IT 团队修复漏洞必须经历的七道关卡（前四步解析） Doten 强调，如果整场演讲只有 5 分钟，他只需要展示这张核心幻灯片——**IT 团队修复任何漏洞都无法绕过的“七大门禁步骤（Seven Gate Steps）”**。无论是面对 15 个还是 15,000 个漏洞，人工处理这些步骤都会消耗巨大的时间成本： ```mermaid flowchart TD accTitle: IT 漏洞修复门禁流程前四步 G1["第一步：真实性验证 (Is it real?)<br/>排除误报，寻找渗透测试证据/文件验证"] --> G2["第二步：确定完整范围 (Full Scope)<br/>评估影响范围是单一组件还是多跨平台"] G2 --> G3["第三步：根本原因分析 (Root Cause)<br/>判断是补丁缺失、配置错误还是兼容性问题"] G3 --> G4["第四步：优先级与级联变更判定 (Priority & Fix Scope)<br/>结合业务评估优先级，研判底层升级引发的级联复杂性"] ``` - **第一步：真实性验证（Is it real?）**：收到漏洞报告后，IT 团队首先要验证这是否为误报。Doten 提到当年做渗透测试时，即使提供了系统被攻破的截图，IT 团队还会怀疑这是“用 Photoshop 伪造的”，安全人员不得不专门往服务器目录丢一个测试文件来证明漏洞真实存在。 - **第二步：确认完整影响范围（Understand the full scope）**：研判这仅仅是一个孤立组件的问题，还是跨多个系统的共性缺陷。 - **第三步：根本原因分析（Determine the root cause）**：弄清楚问题本质究竟是缺少补丁、配置错误，还是底层兼容性故障。 - **第四步：优先级与修复范围扩展（Priority & Fix/Scope Expansion）**：确定修复方案后，必须评估该修复是否会引发“级联效应”。例如，升级某个应用组件可能强行要求同时升级底层数据库及其他关联系统。在人工处理模式下，这种指数级上升的复杂度往往导致工程师直接放弃并申请“安全例外（Exception）”，导致修复工作彻底停滞。 ### [10:24](https://www.youtube.com/watch?v=gw-4_klgNW8&t=624s) 依赖分析、AI 评估挑战与人工修复的物理极限 在考虑漏洞修复时，工程师面临的最大难题往往不是“怎么修”，而是“修了之后会影响什么”。每一个修复方案都必须经过严格审查：它会不会劣化系统性能？会不会与系统其他模块产生兼容性冲突？这就涉及到大量的边缘情况（Edge Cases）。这种情况与针对非确定性（Non-deterministic）AI 系统的评估（Eval）极其相似——在传统确定性系统中，输入 A 必然得到输出 B；但对于复杂的现代系统或 AI 应用，必须由深谙各种故障隐患的资深工程师参与设计评估体系，逐一排查“如果发生这种情况该如何防范，发生那种情况又该如何兜底”。即便是打一个看似普通的 Windows 补丁，工程师也必须验证它是否真能解决问题、是否需要联动修补其他组件、是否存在兼容性隐患，并确保系统不会陷入蓝屏崩溃，最后再安排维护窗口将其部署到安全生产区（Green Zone）。 安全团队过去习惯将厚厚一叠扫描结果直接扔给运维部门，并下达严苛的指令：“这些全是高危漏洞，必须在 24 小时内修完！”然而，当运维人员把上面这一整套繁重的分析与验证流程摊开时，答案显而易见——纯靠人类手工在规定时限内完成这些任务，在物理规律上根本是不可能的。 - **边缘情况考量**：修复方案必须全面评估性能损耗、多系统依赖关系及跨组件兼容性。 - **蓝屏与稳定性验证**：即使是常规补丁，也必须经过验证防止引发系统停机崩溃。 - **人工运维瓶颈**：面对海量高危工单，传统的 24 小时人类手工修复流程在物理层面已无法维系。 --- ### [11:46](https://www.youtube.com/watch?v=gw-4_klgNW8&t=706s) 自动化漏洞治理与生命周期终结（EOL）资产管理 为了打破这种“抛工单”式的僵局，需要将漏洞数据的自动化归集、真实风险评级、归属部门分派、根因分析以及影响面评估深度融合到业务流程改进中。在此过程中，漏洞管理体系中一个经常被忽视的关键领域是**生命周期终结（EOL）管理**。当前许多企业仍运行着大量处于 EOL 状态的旧系统（例如 Windows 10、Windows Server 2012 等）。过去 IT 团队的策略通常是“既然它还能稳定运行，就千万别碰它”，仅依靠补偿性安全控制（Compensating Controls）来维持现状，定期在合规审查时走过场。 阻止企业升级这些 EOL 资产的核心阻力正是时间成本：让人工去梳理老旧系统的所有复杂依赖并逐一测试，往往需要耗费数月之久。而通过 Agentic AI 进行规模化、自动化分析与回归测试，能以极快的速度完成升级验证。随着监管合规要求的演进，监管机构将不再容忍企业连续数年依赖“补偿性控制”申请 EOL 豁免，自动化工具将成为强制推进系统升级的核心支撑。 - **EOL 资产痛点**：由于人工梳理依赖耗时数月，企业普遍不敢对老旧系统实施升级。 - **监管趋势**：监管机构正在逐步收紧对 EOL 系统的长期豁免，依赖补偿控制的传统做法将难以为继。 - **自动化突破**：通过 Agentic 自动化，耗时数月的升级依赖梳理与测试可被压缩至极短周期。 --- ### [13:23](https://www.youtube.com/watch?v=gw-4_klgNW8&t=803s) 缺失的拼图：自动化沙箱测试与变更控制整合 长期以来，漏洞全流程自动化最缺失的核心环节就是**自动化回归测试**。直到最近一年半左右，一批新兴初创企业才真正攻克了这个难题。现代前沿方案的运作机制是：当发现漏洞后，系统会自动拉起一个独立的沙箱环境（Spin up an environment），在其中部署修复补丁，并与生产基线（Baseline）进行全方位比对测试，确保系统性能达标、没有破坏任何现有链路、各项服务通信正常。 完成这些验证后，Agent 会将包含完整证据链的修复工单推送到 IT 现有的工单系统（如 Jira 或 ServiceNow）。工单中会详细列出“修了什么、为什么这么修、完整的测试证据”，并给出明确选项："人类工程师，如果您认可上述所有测试结果，请点击此处，我将为您完成全量修复。"这种方式并未绕过企业现有的变更控制（Change Control）流程，即使生产变更仍需指导委员会审批或提前通知客户，AI 已经替人类完成了前期最繁重、最耗时的测试与证据搜集工作。 ```mermaid flowchart TD accTitle: Agentic Vulnerability Remediation Workflow subgraph S1["数据归集与分析"] A["多源漏洞数据归集"] --> B["去重 / 风险定级 / 根因定位"] end subgraph S2["自动化沙箱验证"] B --> C["动态拉起独立沙箱环境"] C --> D["执行补丁 / 对比生产基线"] D --> E["性能与回归测试验证"] end subgraph S3["变更与闭环"] E --> F["生成富证据修复工单"] F --> G{"变更委员会 / 工程师审批"} G -->|"一键确认"| H["自动化部署 / 人工介入修复"] end ``` - **自动化沙箱比对**：自动创建独立测试环境，与基线比对确保性能与依赖无异常。 - **保留人类控制权**：生成包含完整证据链的工单，由工程师或变更控制委员会一键确认。 - **不破坏现有流程**：与企业原有的审批机制和客户通知契约无缝兼容。 --- ### [14:39](https://www.youtube.com/watch?v=gw-4_klgNW8&t=879s) 机器穷举力与非确定性故障的观测挑战 人类工程师在面对海量重复工作时具有天然的生理局限性——时间有限、精力不足且容易疲惫厌倦。例如在代码审计中，面对一个包含 10,000 次操作提交（Puts & Gets）的代码库，人类审查员通常只会抽样检查 10 到 100 处就草草收尾；而前沿的大模型（如 Claude）可以不知疲倦地遍历全部 10,000 次更新，精确揪出历史上遗漏的每一次细微错误。这种穷举级的覆盖度是人工完全无法比拟的。 与此同时，非确定性系统（Non-deterministic systems）带来的故障模式也打破了传统的运维认知。传统的运维监控只关注硬性指标（如 CPU 使用率过高、内存耗尽、服务器宕机、网络带宽饱和等）；然而，AI 或非确定性系统的故障往往是非典型的——它不是非黑即白的“停机”，而是表现为输出质量的轻微漂移（Drifting）、细微的逻辑不一致或进入低效循环。当前企业普遍缺乏针对这种“质量劣化”的有效观测机制，这也正是需要新一代智能化验证工具介入的原因。 - **遍历 vs 抽样**：AI 能全面审查全量万级别代码提交，彻底根治人类抽样遗漏缺陷的弊端。 - **非典型故障模式**：现代复杂系统故障多表现为质量漂移与隐性死循环，而非传统宕机。 - **传统监控盲区**：CPU、内存、网络饱和度等硬件指标无法有效捕获系统逻辑质量的劣化。 --- ### [16:30](https://www.youtube.com/watch?v=gw-4_klgNW8&t=990s) 安全运营中心（SOC）闭环与漏洞全景工具生态 在整体架构上，配置与漏洞管理（Configuration and Vulnerability Management）与传统的安全运营中心（SOC/SecOps）职责分明：SecOps 关注的是“刚才发生了什么、怎么发生的、立刻制止攻击”；而漏洞与配置治理关注的是“如何从根本上杜绝此类事件再次发生”。一旦 SOC 完成应急处置，相关根因数据就会回流至漏洞治理管道，形成闭环防线。 在当前的工具市场生态中，不同厂商分别在漏洞生命周期的各个阶段发力。市场上正在发生大规模整合与演进（例如 Wiz 先后收购了云安全运营平台 Gem 以及漏洞管理平台 Dazz）： | 工具能力分类 | 核心代表厂商 / 平台 | 功能定位与工作阶段 | |:--- |:--- |:--- | | **资产归集与根因分析** | Zenity, Qodo (Codem) | 负责从多源扫描器汇聚数据、去重定级并定位漏洞根因 | | **代码级自动修复** | Qodo (Codem), Astos | 直接生成代码修复补丁，针对应用层漏洞实施代码重构 | | **沙箱与自动化回归测试** | Back Line, Appiro | 自动拉起测试环境，执行补丁与基线性能/兼容性回归比对 | | **基础设施与网络配置修复** | Asselia, Zafran, Remedio | 聚焦于操作系统补丁、Unix 配置、防火墙规则与路由器 ACL 调整 | | **平台级端到端整合** | Wiz (整合 Gem & Dazz) | 从资产发现、统一漏洞管理（UVM）到防御闭环的平台化整合 | 当前市场上的修复工具中，约 75% 集中在**应用代码级修复**（Code Remediation），而剩余约 25% 专注于**基础设施与网络配置修复**（Infrastructure Remediation，如修改防火墙规则、调整路由器访问控制列表 ACL 等）。企业在选型时必须结合自身的架构现状：对于全量采用 SaaS、自身不编写业务代码的企业，应当侧重采购基础设施与配置类修复工具，而非代码生成工具。 ### [20:51](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1251s) 技术赋能流程：AI 编程特性与流程重构 技术永远只是流程的支撑工具，而非相反。许多人在谈到自动化工具时总是兴奋地宣称某款工具能飞快搞定某项任务，但如果缺乏配套的业务与安全流程，这就像“把一辆法拉利交给一个连手动挡都不会开的人，根本无济于事”。企业必须花时间明确自身的目标、战略、流程和工作流。现有的工作流往往是为了适应人类工程师的工作习惯而设计的；一旦开始将某些环节自动化，整个工作流就必须重新设计，以适应机器与人类协同的新常态。 在代码修复与云安全方面，AI 的行为模式与人类开发者有着本质的不同。人类工程师由于存在认知偏差和惰性，在被要求修改代码（例如将某个属性从蓝色改为红色）时，通常只会执行简单的局部搜索并替换；而 AI 却完全没有人类的代码所有权意识。它在接到修改指令后，会直接重写整段代码、重组架构甚至随意插入新函数。它甚至根本不记得这段代码最初就是它自己写的。这种特性意味着传统依靠人类防范“利益冲突”（如避免开发者自我审查漏洞）的逻辑在 AI 面对自身生成的代码时呈现出全新的动态。 ```mermaid flowchart TD accTitle: 技术与流程关系示意 subgraph Alignment ["核心原则"] A["明确业务目标与战略"] --> B["设计配套工作流程"] B --> C["引入自动化/Agentic AI 工具"] end subgraph Pitfall ["常见误区"] D["盲目引入高阶工具 (法拉利)"] -.->|缺乏驾驶技能/流程| E["无法发挥价值甚至引发事故"] end ``` ### [22:56](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1376s) 马里努阿犬跳出围栏：不可靠的 Agent 与控制边界 在生产环境与开发环境之间维持严格隔离是不可逾越的红线。针对业内频繁发生的“AI Agent 误删生产数据库”或“突破沙箱逃逸”的事故，这一现象完全在情理之中。正如员工在受到机制阻碍时会绕过安全控制去完成任务一样，自主 Agent 也不会完全按照既定规则行事。安全领域的一大铁律是：“不要让你的用户成为你最大的黑客”（"Don't make your users your biggest hackers"）。如果合规控制迫使工程师或 Agent 必须绕过机制才能开展工作，那说明安全团队从一开始就没有理解业务。 为了解释为什么不能责怪 AI 闯祸，可以借用宠物犬的生动比喻。如果你在后院修筑了一个六英尺高的围栏，多年来金毛寻回犬、拉布拉多贵宾犬都在里面安然无恙；但如果你新领养了一只精力充沛的“比利时马里努阿犬”（Belgian Malinois），它会直接跑酷越过围栏轻松出逃。 - **不能以“它以前从没这样过”作为借口**：原有的控制措施是为传统静态脚本或人类行为设计的，根本无法约束像“喝了红牛的八岁小孩”一样的强力自主 Agent。 - **环境隔离底线**：生产环境与开发环境必须具备严格的权限凭证隔离，管理员在两个环境间切换必须使用独立账户登录，坚决杜绝从未经验证的代码直接推送到生产端。 ### [25:04](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1504s) 漏洞修复架构蓝图与上下文知识库构建 完整的 Agentic AI 漏洞修复架构涵盖了“检测（Detect）、关联（Correlate）、诊断（Diagnose）、修复（Remediate）、验证（Validate）、治理（Govern）”全流程。在这一链条中，必须界定明确的安全边界，即明确“人类在回路中（Human-in-the-loop）”还是“人类在回路之上（Human-on-the-loop）”。正如在 Black Hat 期间 Anthropic 安全负责人 Jason Clinton 所强调的，企业必须为 Agent 划定严格的行为红线——Agent 可以在开发环境中自动修改代码并进入测试流程，但绝对不能允许其在未经人工审批的情况下直接碰触生产环境。 ```mermaid flowchart LR accTitle: Agentic AI 漏洞修复生命周期架构 subgraph Workflow ["自动化修复主流程"] D["1. 检测 Detect"] --> C["2. 关联 Correlate"] C --> Di["3. 诊断 Diagnose"] Di --> R["4. 修复 Remediate"] R --> V["5. 验证 Validate"] V --> G["6. 治理与升级 Govern"] end subgraph Foundation ["上下文支撑底座"] KB["上下文知识库 (经验资产化)"] -.-> Di KB -.-> R Test["自动化测试环境"] -.-> V end ``` 构建自动化的核心基础设施在于“上下文知识库（Knowledge Base）”。大量的业务边缘情况、异常排查逻辑和系统依赖关系，目前依然仅留存在资深工程师的大脑中。自动化改造的第一步，就是将这些隐性经验从人类脑海中提取出来，沉淀为结构化的数据库与上下文文件，使 AI 能够充分理解系统边界，从而避免在自动化修复时触发潜在的连锁故障。 ### [27:25](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1645s) 问责机制不可转让与 F1 赛车的治理哲学 在 AI 引入过程中，核心的合规与治理原则是：“职责可以分派，但问责不可转让”（"We can delegate responsibility but not accountability"）。如果你的狗咬了人，法律追究的是作为管理者的狗主人的责任，而不是狗本身。在监管与审计层面也是同理： - **欧盟《人工智能法案》（EU AI Act）**：在相关条款中明确界定了“自然人”（Natural Persons）的不可推卸的法律责任。 - **COSO 内部控制框架**：美国的审计与风控标准同样严格规定，最终责任必须由人类管理者承担，绝不能推诿给技术系统。 董事会对 AI 的态度存在显著的地域差异。美国董事会普遍充斥着“错失恐惧症”（FOMO），急于盲目追求部署速度；而美国以外的国际董事会则更为审慎，他们深入探讨业务转型的实质目标（例如提高利润率、提升客户留存率或优化组织架构），并深刻践行“技术支撑流程”的逻辑。 许多人将 AI 治理误解为阻碍创新的官僚主义。真正的治理绝不是用来拖慢进度的刹车，它就像“F1 赛车上的制动系统”——正因为赛车拥有极其强悍的制动与转向控制系统，车手才敢在赛道上开出极速并安全拐入匝道。治理的本质是为高速前进提供安全保障，而非制造路障。 ### [30:34](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1834s) 现场互动：异构技术栈下的威胁建模与代码修复 在进入现场问答环节时，与会观众针对复杂的企业环境提出了具体的代码修复挑战。企业内部的技术生态通常高度多样化，同时涵盖了从.NET、Java 到 Oracle 数据库等不同代际与技术栈的组件。要在这类异构环境下实现端到端的自动化代码修复，单靠简单的语法替换远远不够，必须引入系统化的威胁画像（Threat Profiling）与威胁建模（Threat Modeling）机制。演讲者对此表示完全赞同，并强调在健全的风险管理体系中，威胁建模是评估潜在系统风险、未授权访问以及控制失效的前置必备环节。 ### [31:46](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1906s) 规模化识别真阳性漏洞与利用 AI 自动化验证 现场观众就开源组件漏洞海量激增的现状提出疑问，特别提到了 Glasswing 等项目以及在开源生态中动辄扫描出数以万计（例如 45,000 个）漏洞的现实困境。提问者关注是否存在某种行之有效的方法或经过精心调优的模型，能够精准判定哪些漏洞属于“真阳性”（True Positive），还是说这种筛选本身就必须作为治理旅程的一部分逐步摸索。 Rick Doten 对此表示高度赞同，并指出在漏洞修复全流程中，第一步永远是明确“这个问题是否真实存在”。在面对数万量级的庞大告警时，人工逐一排查根本无法维系，因此利用 AI 进行自动化测试与交叉验证成为了极具潜力的发展方向。核心在于将确认漏洞真实性的环节固化为自动化流程，让系统自主验证漏洞的可利用性与真实风险，从而将人力从繁琐的误报筛选中解脱出来。 ```mermaid flowchart TD accTitle: 漏洞真伪验证与自动化修复阶段流转 subgraph S1["阶段一：漏洞发现与真伪甄别"] A["多源扫描工具接入<br/>(数万级开源/系统漏洞)"] --> B["AI 驱动的真阳性自动化测试<br/>(确认是否真实存在与可利用)"] end subgraph S2["阶段二：根因分析与影响评估"] B --> C["复杂性与根因判定<br/>(复杂业务逻辑 vs 单纯版本升级)"] C --> D["无损性验证与变更准备<br/>(确保不中断业务系统)"] end subgraph S3["阶段三：结果交付与度量"] D --> E["生成带证据链的变更工单"] E --> F["以风险削减与质量为核心的结果度量"] end ``` --- ### [32:55](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1975s) 重构安全指标：告别虚荣指标，转向以结果为导向的风险度量 针对如何衡量漏洞修复工作流成效的提问，现场探讨了传统度量体系的局限性。过去行业普遍依赖“有多少漏洞超出了 SLA 时限”或修复了多少张工单等数字。但在漏洞发现与修复体量爆发式增长的今天，这些传统指标已无法反映真实的安全水准。 Rick Doten 透露自己正在撰写关于度量体系变革的文章，并在即将出版的新书中深入探讨了该话题。他直言不讳地指出，平均确认时间（MTTI）、平均修复时间（MTTR）以及单纯的修复数量大多属于“虚荣指标”（Vanity Metrics）： > “我绝不会向董事会汇报这些数据，那些都是虚荣指标。真正重要的是质量、完整性、准确性，以及是否真正降低了风险。” 在非确定性的现代系统架构下，度量标准必须从关注“动作”转变为关注“结果”（Outcomes）： - **修复质量与完整性**：下发的修复方案是否彻底解决了目标问题，是否存在遗漏或死灰复燃。 - **业务无损性**：修复操作是否引发了系统故障或中断了上下游业务依赖。 - **实际风险削减**：推行了上万项配置或补丁变更后，企业整体的安全暴露面与攻击路径是否实质性收敛。 --- ### [34:17](https://www.youtube.com/watch?v=gw-4_klgNW8&t=2057s) 从发现到问题层面的聚合：合规 SLA、复杂性分级与人机分工 现场讨论进一步延伸至跨工具告警聚合与优先级管理机制。当多款扫描工具针对基础设施和应用程序产生大量底层“发现”（Findings）并将其归集为高层级的“问题”（Issues）时，企业的 KPI/KRI 与 SLA 考核机制应当如何调整，是否仍应沿用“高危/严重漏洞必须在规定天数内修复”的传统死板策略。 ```mermaid flowchart LR accTitle: 繁复问题与复杂问题的修复决策路径 A["漏洞/问题聚合"] --> B{"问题属性分类"} B -->|"繁复问题 (Complicated)<br/>路径明确/依赖较多"| C["Agentic AI 自动化处理<br/>(依赖解析、测试验证与自动提单)"] B -->|"复杂问题 (Complex)<br/>涉及业务重构/未知解"| D["人类工程师主导<br/>(AI 辅助分析架构与业务影响)"] C --> E["快速闭环满足 SLA 要求"] D --> F["业务决策与长期演进方案"] ``` Rick Doten 剖析道，现有的 SLA 规则之所以聚焦于优先处理严重（Critical）漏洞，是因为最初的流程是为“资源与精力有限的人类工程师”量身定制的。引入自动化与 Agentic AI 并非要违背法规要求的硬性指标（如监管或合同明文规定 24 小时内必须完成修复的合规红线），而是利用机器的高并发能力重塑资源分配逻辑。他将问题划分为两大类： - **繁复问题（Complicated Problems）**：存在明确的解题路径，只是处理过程较为繁琐。这类问题最适合交由 AI 全流程自动化处理，能够以极高速度达成治理并输出可审计的验证证据。 - **复杂问题（Complex Problems）**：类似于没有单一标准答案的深层架构挑战，往往牵扯到底层业务流程重塑或系统重构，这类问题依然需要人类专家介入决策，由 AI 在旁提供上下文支撑。 --- ### [36:47](https://www.youtube.com/watch?v=gw-4_klgNW8&t=2207s) 应对频繁更新与漏洞洪流：深挖根因与构建人机协同工作流 针对生产环境中开源组件（例如 Apache Tomcat）更新极度频繁、每次迭代牵扯数千处改动导致运维人员疲于奔命的问题，提问者指出：AI 技术的演进使得漏洞被挖掘和披露的速度呈指数级上升，而官方补丁或工程修复方案的跟进节奏却常常滞后，团队陷入了疲于修补却永远修不完的恶性循环。 Rick Doten 强调，面对这种依赖性更新洪流，必须区分“表面打补丁”与“深挖根因”（Root Cause Analysis）： - **寻找根本诱因，避免无效返工**：类似于在云基础设施中经常发现的问题——工程师不断在生产运行环境中手动修改打补丁，但基础设施即代码（IaC）或镜像配置文件中的版本号却从未修正，导致系统每次自动化构建或刷新时，旧漏洞又被原封不动地重新推上线。 - **低中危漏洞组合利用的链条治理**：多个低危或中危漏洞相互串联可能形成高危风险链，这需要构建一套全新的人机协同工作流，在“验证漏洞真实性”和“确保修复不破坏系统”两个阶段深度结合，精准推演补丁依赖。 - **会议收尾**：在解答完依赖管理与根因定位的关键逻辑后，Doten 结束了本场主题分享，并邀请现场听众在会后走廊继续深入交流 Agentic AI 在漏洞修复落地中的实践细节。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=gw-4_klgNW8&t=4s) We cannot afford to not automate our remediation — Gartner > [04:28](https://www.youtube.com/watch?v=gw-4_klgNW8&t=268s) thrown over the fence — Rick Doten > [08:48](https://www.youtube.com/watch?v=gw-4_klgNW8&t=528s) Is it real? — Rick Doten > [08:48](https://www.youtube.com/watch?v=gw-4_klgNW8&t=528s) Understand the full scope — Rick Doten > [08:48](https://www.youtube.com/watch?v=gw-4_klgNW8&t=528s) Determine the root cause — Rick Doten > [13:23](https://www.youtube.com/watch?v=gw-4_klgNW8&t=803s) 人类工程师，如果您认可上述所有测试结果，请点击此处，我将为您完成全量修复。 — Agent > [22:56](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1376s) Don't make your users your biggest hackers — Rick Doten > [27:25](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1645s) We can delegate responsibility but not accountability — Rick Doten > [32:55](https://www.youtube.com/watch?v=gw-4_klgNW8&t=1975s) 我绝不会向董事会汇报这些数据，那些都是虚荣指标。真正重要的是质量、完整性、准确性，以及是否真正降低了风险。 — Rick Doten ## Mentioned Rick Doten, AiSuNe, Ai4, Gartner, Black Hat, ServiceNow, Jira, Zendesk, Slack, Wiz, Orca, Gem, Dazz, Claude, Anthropic, Jason Clinton, Zenity, Qodo, Astos, Back Line, Appiro, Asselia, Zafran, Remedio, Glasswing, Apache Tomcat, EU AI Act, COSO ## Video description Rick Doten - AI Researcher and Author, AiSuNe We have more than enough tools to find cybersecurity vulnerabilities; we need help fixing all the things we find. And security teams don't fix vulnerabilities, we find vulnerabilities. We've automated how to find security risks, but still manually fix them. And it's the IT teams that remediate findings. We need to empower IT with tools an automation to help them dig out of the thousands of vulnerabilities we pass them each month. There are automated workflows and Agentic tools that mimic all the steps humans take to remediate; define problem, prioritize, determine root cause, develop the remediation, test the fix to make sure it doesn't impact other components or affect performance, and create ticket to put into change control process. This session shows how, by using a set of tools and Agentic AI, we can help IT keep up with the deluge of vulnerabilities they are asked to fix each week by automating the most time consuming steps to get tickets closed. We understand historically we were worried about automating system changes because it might make a mistake that disrupts the infrastructure. But our fears are from historic platforms or scripts that had no context. With Agentic AI, we can go through all the steps a human engineer would, with greater speed and accuracy, and provide all the evidence of completeness in a configuration change ticket before being sign-off by a human or other approval process. We have to become comfortable automating remediation or we will continue to pile up tickets for vulnerabilities that will never get fixed. [Source](https://www.youtube.com/watch?v=gw-4_klgNW8)

[0:04] Gartner: We cannot afford to not automate our remediation [4:28] Rick Doten: thrown over the fence [8:48] Rick Doten: Is it real? [8:48] Rick Doten: Understand the full scope [8:48] Rick Doten: Determine the root cause [13:23] Agent: 人类工程师，如果您认可上述所有测试结果，请点击此处，我将为您完成全量修复。 [22:56] Rick Doten: Don't make your users your biggest hackers [27:25] Rick Doten: We can delegate responsibility but not accountability [32:55] Rick Doten: 我绝不会向董事会汇报这些数据，那些都是虚荣指标。真正重要的是质量、完整性、准确性，以及是否真正降低了风险。

[0:04] Rick Doten: cannot afford to not automate [0:04] Rick Doten: PTSD from 20 years ago [13:23] Rick Doten: This is what we're going to do: we're going to spin up an environment, we're going to test it, we're going to compare it to baseline, we're going to make sure that the performance is right, that it didn't break anything... and we're going to put it into the right ticketing system with all the context and say, 'This is what we need to do. This is why we need to do it. Here's all the evidence for it. Human, if you approve all that, click here and I'll fix it all for you.' [20:51] Rick Doten: If you give a Ferrari to someone who doesn't know how to drive or doesn't know how to drive a stick shift, it ain't going to help. [22:56] Rick Doten: Don't make your users your biggest hackers. If they have to go around my controls to do their job, I failed. I didn't understand the business. [22:56] Rick Doten: 8-year-old on Red Bull [28:32] Rick Doten: bureaucracy disguised as governance [31:46] Rick Doten: is it real?

[0:04] In November of last year, they asked me if  I'd be interested in speaking, and I said, [0:08] "Hey, I have this controversial topic about  automated remediation leveraging AI agents. [0:13] I've been doing it for a year,  and everyone argues with me. [0:17] And it's like, but I think it'll be a really  good good good one." And then all of a sudden, [0:21] I'm down at Black Hat yesterday, and everybody  is talking about automated remediation. [0:27] They finally kind of caught up because um here and  and the thing that I really know that it was uh [0:35] Thank you, FC. Uh oh, well. I I had a update  the slide, which is fine. Um Gartner this [0:44] particular slide was supposed to be the Gartner  paper that was released on July 27th, a week ago, [0:50] that said, "We cannot afford to not automate  our remediation." And uh I put it here, and I [1:00] Anyway, got the slide update, but you can go find  it. So, there was a uh Gartner saying that. And [1:04] I'm not saying that I'm like a fan of, you know,  and of of of what they're writing, but the fact [1:09] that they also now finally see it that we have so  many things that we are Oh, well. [clears throat] [1:18] I don't want to get ahead of myself. I get excited  because I love this topic. I talk about it all [1:21] the time. I am a AI researcher. I'm a former  chief information security officer, retired. Um [1:27] don't want to go back. And you know, I ran AI  governance before as as well while doing that. And [1:34] uh the company that I I write lots of white papers  for on the side, I didn't realize it was a think [1:39] tank. So, I work for a think tank. Cool. Um but  let me first say, in the last year and a half, [1:46] that all the arguments I would get were these  about why we can't automate remediation. One [1:53] is that it'll break things. And I said, "You have  PTSD from 20 years ago when IPS systems took out [2:00] network segments because your upstream router was  attacking you." And it said, "Oh, let me block [2:05] that IP address." And then your network went  down because it had no context. It was binary. [2:12] And so, but that's what's still in our heads,  right? But we have AI and we have agents and we [2:18] have workflows and we have decision trees and it's  like it's not It has context now. The second one [2:24] is that I get if you try to patch everything, then  you're going to miss the important stuff. Both of [2:29] these things can be true at the same time. It's  like shame on you if your vulnerability management [2:33] program is only doing is focusing on everything  and not prioritizing and making sure that we [2:38] understand the risk to all the things we're trying  to fix. Because that's still in play. It's not a [2:43] blind like, "Hey, we're going to blanket just, you  know, spray everything and kind of fix it all." [2:48] We still need to prioritize based on business  impact, which we have always struggled with. [2:54] I mean, there's lots of tools and I'll show you a  bunch of them that have worked on prioritization. [3:00] And sometimes it'd be things as simple as it's  externally facing and there's a known exploit. [3:06] Which, sure, but that could be a development box  that's in that has nothing to do with it and fine, [3:11] take it. Not helping anything. Is it impacting  the business? And that's something that we have [3:17] always struggled with. Our industry is having that  business context. Because there may be things that [3:23] are contracts are real important to the business  that don't seem like a big deal. There may be [3:28] things timely that as we are going towards, you  know, the end of the quarter and we're getting our [3:33] numbers together, these boxes are very sensitive  as we put these numbers together. Anything getting [3:38] around this is very, very critical. And  then finally, it changed our incentive. [3:45] A lot of workflows that we've developed for  years have been to to the frailties of humans. [3:52] We're biased, we're slow, we're prone to mistakes,  we mess things up, and so our first thing is [3:59] anthropomorphizing saying it's like, "Oh, well,  then it's just going to have the goal of fixing [4:02] everything." And I know our previous speaker  just talked about the um the OpenAI attack, which [4:09] I was not surprised about. And it's like, of  course that's going to happen. And um and and [4:16] based on its goal, but if you make sure  that the goal is reducing the risk, [4:21] then it will see that willy-nilly patching things  and doing it quickly is not reducing the risk. So, [4:28] the argument which I now don't have  to necessarily make is that, you know, [4:32] we have spent a lot of time and money in the last  few years automating the finding of problems. [4:38] And I feel like I contributed that cuz I ran  ethical hacking teams from the late '90s to the [4:42] late 2000s, like before it was cool. Um we were  part of this problem is we would go find things [4:48] and that was when nothing was secure. I mean,  we were testing online banking applications [4:53] and still could move money in between stuff. And  so all of the stuff that the industry had done [4:57] in the late '90s early 2000s made banking  so secure that all the attackers just went [5:01] after the individuals, you know, because  we're the weak link in the chain now. [5:06] That we you know, that the important thing also  is like as a CISO, we find problems, we don't fix [5:12] problems. My friends in IT fix problems and they  resent me. And I absolutely have thrown over the [5:18] fence a bunch of pen test results and stuff  like, "Hey, we need to fix all this stuff." [5:22] But I prioritized it for you and I told you the  website to download the patch and I told you how [5:27] to change the configuration. It was like, "Yeah,  but still some some dude has to get there and fill [5:31] it out and do all the things." And I'll show  you what they have to go through in a second. [5:36] You know, I talked about the severity is not  correct. Are we doing the right workflow? Early on [5:43] with um [clears throat] the data the posture  management tools, like you know, your Wiz, [5:47] your Orcas, your Sysdigs, your things like  that. I talked to more startups than anybody. [5:52] I am very tied in with the VC community,  the startup community. I'm like every week [5:56] talking to somebody. And so, I like talked to Wiz  when they had 40 people 5 and 1/2, 6 years ago. [6:02] And the thing that I told them, I said, "Here's a  problem with the remediation when you start doing [6:06] it is is cuz I see that I saw this firsthand  is the vulnerability management team uses [6:13] let's say ServiceNow. And they're like,  "All right, we're going to put all of [6:15] our findings in ServiceNow, and then that's our  workflow." But, the development team uses Jira. [6:21] And they're just like, "Okay, so you now you  have to get accounts and go to somewhere else, [6:24] and they're changing their process." It was like,  "You need to match where it's going." And it could [6:29] be Zendesk, or could be a Slack channel, could  be anything. It's like, "If you're not going [6:33] to put it into the right workflow, that's going  to slow it down, or not even going to be seen. [6:38] It's not what's convenient for you, it's what's  the best for getting this thing done." And so, and [6:43] then finally, the feedback loop. How did we like  not forget this? Like, "Oh, what did we learn? Do [6:48] we go back?" Like, say at the end of the quarter,  we'll say, "What worked? What didn't work? Okay, [6:52] what was the slowdown? What was the things?  What's all the stuff?" It's like, we have not [6:56] really successfully had a vulnerability program  remediation, the whole workflow has kind of been [7:02] the same way for 20 years, and no one tried to  update it, modernize it, and put a quality element [7:07] cuz it kind of worked, and it was fine, and no  one expected to be perfect because we have 10,000 [7:11] vulnerabilities coming in my old company. We had  10,000 vulnerabilities come in a week that need to [7:16] be fixed. And we fixed maybe 1,000, 2,000 of them.  And that's just going to keep piling up. And so, [7:23] this is what it looks like from a, you know,  standpoint. Here's just like some friends of mine. [7:28] Again, I talked to all these companies, and then  I'm not blaming them for this. And a lot of them, [7:33] as you'll see, you know, have are contributing  are are helping in, well, we help prioritize, [7:38] we help figure out what the root cause is,  we help figure out some of these things, [7:42] but still the massive stuff that we're coming  in with the telemetry we have, it's like [7:49] it's like this morning going into the pre-note,  right? 12,000 of us trying to all file into one [7:54] room. And so this is what it looks like. And  I love doing this in international audience [7:59] because some international audiences don't know  this thing. I have the the link underneath the [8:04] the caption there to to this one if everyone's  seeing what she's trying to put the chocolate in [8:08] thing going crazy. It's a very funny short video.  But anyway, so the you know, and I one more slide, [8:15] I'll stop admiring the problem. You got I know  you get it. It was like, so are we using right [8:19] workload? We give them enough information as well,  which is just the context for the business. Are we [8:25] you know, providing the full scope, realistic time  frames? A lot of regulations may require like 24 [8:31] hours to fix something. It's like, all right,  it's not that easy. And you know, are we giving [8:35] them the the context and are we understanding  the metrics that are needed to be able to show [8:41] whether we're actually making any good good  standard. So this is the main slide. If I say [8:48] if if I if they said you had 5 minutes to talk  about this, all I would do is show this slide. [8:54] What this slide shows is a seven gate steps that  an IT team has to go through to fix something. [9:00] And you can't skip it. So we start  in the beginning, it's like, okay. [9:05] So I throw over a pen test report and there are  15 things on it. Let's say it's a really easy one. [9:10] It was like, is it real? Did it really happen or  was it a false positive or even and I remember [9:17] when pen test teams or we would say, here's a  screenshot of showing you that got in and they [9:21] don't believe us. It's like, you could have done  that in Photoshop. It's like, all right, well, [9:24] I dropped a file on the directory to show  you that I got it. And they're you know, so [9:29] sometimes you have to have more you know, do  more than others. We need to make sure that [9:36] we understand the full scope of it. Is it  just fixing this thing? Is it the, you know, [9:41] is it multiple platforms? What is the root  cause of this? Is it really just a patch? Is [9:46] it a configuration issue? Is it some compatibility  issue? We need to understand what the priority is [9:53] as we've already kind of talked about, and  do we have all the things to be able to know [9:57] the priority? We need to figure out what the  fix is, and does that fix change the scope? [10:03] Because sometimes I need to upgrade some platform  to a different version, but then now if I update [10:08] that, then I also need to update the the database  and this and other things need to be updated, [10:12] and then that becomes more complicated. And that's  where we're in an automated in a in a manual way, [10:17] they're kind of throw up their hands like, "Ah,  this is too difficult. We're just going to put [10:20] an exception in cuz we're going to have to figure  out all these different things." And it takes time [10:24] to do that. And then what dependencies are going  to be impacted? So, if I do figure out what the [10:30] fix is, is it going to affect performance? It can  affect uh um a compatibility with something out [10:37] with something we have elsewhere? And so, you  know, these edge cases. And the same thing in [10:43] doing evals for AI, right? You know, if I have  an AI application, I need the senior engineer [10:48] who knows all the things that can go wrong in  the edge cases to be able to develop the eval, [10:52] not just because it's not deterministic anymore.  Where if I put in this input, I will always get [10:56] this output. I need to say, "Well, if it happens  and does this thing, this could happen. Let's make [11:00] sure it doesn't do that. And if it does this  thing, then we need to make sure this doesn't [11:03] happen." And you know, you need to kind of figure  all those things out. And then we need to test it [11:08] and make sure the performance is right, it  doesn't break anything, it doesn't blue screen, [11:12] and then schedule where it's going to, you know,  green zone where it's going to be deployed. [11:17] Even if I just need to say put in a Windows  patch, I still need to do all this stuff. Okay, [11:22] is this going to fix the problem? You  know, do I need to patch anything else? [11:26] Do I need Is it going to impact any other things  compatibility-wise? I need to test to make sure [11:30] it doesn't blue screen, and then I need to put it  into a thing. And so when I sit there and I say I [11:33] throw this over like these are all critical  vulnerabilities you need to fix in 24 hours. [11:39] They show me this like you know physically  that's not possible. With humans. So. [11:46] So how can we help with this hand-on? [clears  throat] You know so we're automating collection. [11:51] Um you know figure out the real risk ratings,  assigning the right department, figure out the [11:55] root cause, you know looking at applying them in a  process improvement standpoint, you know assessing [12:01] the impacts and one other element that's in the  vulnerability management program that doesn't get [12:06] talked about a lot is end-of-life management.  Because we're all sitting on devices that like [12:12] have end-of-life. You know still have Windows 10  in there, we still have you know other things that [12:18] that you know Windows Server 2012 or you know  whatever that are are fine, they're working, [12:23] don't touch it, we have compensating controls,  but at least I need to address it every single [12:28] depending on your policy if it's a quarter  or however once a year or once or once [12:33] ever or or semi-yearly, uh twice a year, whether  or not that you know do I still need to do this [12:38] is relevant. And a lot of times the things that  prevented us from upgrading this was the time. [12:45] Is that you know for a human to go through and  look at all the dependencies and all the stuff [12:49] it's going to take like months to be able to do  that. Well now if I automate it and I scale it, [12:55] I'm going to do that a lot quicker and  if I test it I can do it a lot quicker. [12:57] So that's why I included it in this mix and  end-of-life management is very important because [13:02] a lot what we're going to start seeing knowing  that this is possible is the regulations are [13:07] going to start evolving and saying okay you've  got an exception for this end-of-life thing [13:11] you know for two or three years you're going to  have to get over that and just upgrade because we [13:16] can't always assume that the compensating  controls are going to is going to work. [13:23] But up until about 18 months ago the missing  piece to be able to do all this and and [13:30] is is the testing and the regression testing. And  this is the benefit I have of talking to so many [13:36] startups. I'm seeing all these people doing this  really innovative thing and this was the first [13:39] company There was one company first in 2024 that  I talked to that hey, we're thinking about doing [13:45] this. This is what we're going to we're going to  spin up an environment. We're going to test it. [13:48] We're going to compare it to baseline. We're going  to make sure that the that the um the performance [13:52] is right, that it didn't break anything, that it  could really connect with all the stuff and then [13:56] we're going to do all the stuff in the seven  layer dip that I just showed you including the [14:01] testing and we're going to put it into the right  ticketing system with all the contacts and say, [14:06] "This is what we need to do. This is why we need  to do it. Here's all the evidence for it. Human, [14:10] if you want that if you if you approve all  that, click here and I'll fix it all for you." [14:15] Or it can go through whatever change control  process you have where it's like, "Oh, well, [14:19] steering committee needs to approve this cuz  it's production and our contract says we need [14:23] to notify, you know, our customer before  we do any major changes." Fine. It's not [14:28] changing the change control process, but it's  automating all the stuff the humans were doing, [14:33] scaling it, making it more efficient  and and honestly more comprehensive. [14:39] I first saw this in, you know, we talk  about Mythos being this great hacker, [14:44] but in February, Claude's model did a  test that really made me see that oh, [14:52] this is different. And remember that when it went  through this repo and went to 10,000 different, [14:57] you know, puts and gets from from from from an  application to find every single time update [15:04] the code was updated over the period of time and  see like, "Oh, there was an error here. Someone [15:08] missed this or there was something and it never  got fixed later on." A human wouldn't have the [15:13] time for that. We wouldn't have the patience for  that. We do a sample of maybe 10, 100, whatever [15:18] and call it a day, not all 10,000 comprehensively.  And so that's one of our limitations is like [15:24] our time and our value and our effort and all  those things and we get bored, you know, but [15:30] being able to then test it and and make  sure the quality is in place, stability, [15:34] because as we know, non-deterministic systems  fail in non-normal ways. It's not up down. [15:41] It's not that it's like, "Oh, obviously this is  wrong." It is it's a drifting a little bit and [15:48] it's not always accurate and but it's really hard  to tell unless you know what you're looking for [15:53] and it's kind of going through kind of a loop  and it's starting to repeat things, you know, [15:57] and we don't have the mechanisms to observe that.  And I'm sure there's lots of I know there I know [16:02] because I'm talking to them, lots of companies  that are kind of looking at that, but that is also [16:06] not something that we are used to doing. We're  used to saying it's like, "Oh, the CPU you know [16:13] utilization is too high, the memory is out, the  box is down, the the network is saturated." Those [16:19] are things we can measure, but not quality.  Is it really doing it in an accurate way? So, [16:25] to take that together and put them back to put  them together, how do our little robots help out? [16:30] So, we have all the sources that I talked about  before, you know, we normalize, deduplicate, [16:35] group, you know, there's groups that do that. We  go to appropriate workflow that then determines [16:39] the root cause, identify the impacts, do  regression testing and go to the change [16:43] control process which I described and then do  you want me to fix it? Do you want to fix it? [16:48] How well makes you comfortable? That's fine.  But we can accelerate all of these things. [16:56] Now, having done this I added a little piece,  you know, about a year ago where the incident [17:01] identification the the the sock process. This  is configuration management and vulnerability [17:07] management, which is different than security  operations. Security operations is what happened, [17:12] how did it happen, make it stop. But then  it's make sure it doesn't happen again. [17:19] And when it make sure it doesn't happen  again, then it feeds into this process. [17:23] And that's a whole other presentation on automated  security operations security operations center. [17:28] But, I just want to say like in the larger  block down I have a more complicated more [17:31] detailed architectural diagram how this works at  the end. But, so in putting this together, again, [17:38] I and I I I I I'm I'm restating these because the  next slide we'll see the categories. You know, [17:44] we have collection, prioritization, root  cause analysis, workflow management, [17:48] and regression testing, and also the fixing.  Because this is how that we have we update [17:54] what it looks like. Should be a little shaded,  but anyway. So, on the left is what we saw before, [17:59] all this stuff sitting over. But, now  we have things that these are tools [18:03] that I've talked to that we've tested and we  see that there was and there's others in here. [18:06] These are ones that I know, you know, and just  showing you that they exist. It is not one thing. [18:13] It is a combination of things. So, some do  automate the collection, all the stuff I said, [18:19] determine root cause, figure out what the fix  is, things that will actually fix the code, and [18:25] under here there's the two that is covered up  by by this, which are critical, is Back Line [18:30] and Appiro. That actually will do regression you  know, allow you to spin up things and do testing. [18:36] Um and so and I'm not saying that  the ones on the left are the problem. [18:43] Because a lot of them, I mean you see here  Wizz bought Gem, which is the Wizz Defend, [18:48] and they bought Dazz, which is the Wizz UVM,  you know, as they're working their way down. [18:54] Um you know, we have There was another  one that was acquired that is company So, [18:59] we're seeing them and even folks like I know  Zentity does the collection and determine root [19:03] cause. Codem does that, too, and can give you the  suggestion for fixing the code. Um some of the uh [19:10] Oh, yeah. yeah, I did put in the Astros bot,  but Oasis just got bought, so I didn't update [19:15] that since the time I submitted this. Um, and but  some of these are getting further down the chain. [19:22] But it's kind of still kind of caged together.  And there's some down here that go up further. [19:26] These are not specific to that area that I've  identified. I'm just saying that in general the [19:32] goal of this is to do this, these do that, these  do this. Now, one thing I want to point out is [19:38] that there's a big difference between remediation  of code and remediation of infrastructure. [19:46] When I'm patching a Windows machine, if I'm  applying a configuration change to a Unix box, [19:54] if I'm changing a, you know, an ACL in a router  or a firewall rule, that's a different story. [20:03] There are some of these tools in  here that do focus on that. Um, [20:08] whereas Asselia, which is somewhere in the mix  here, um, they they focus on There we go, Asselia. [20:16] You know, they as well as, you know, how do I  say, uh, Zafran or Remedio, you know, they focus [20:22] on infrastructure types of things and that kind  of configuration. So, a handful of these focus on [20:27] that, but I would say probably 75% are more about  the code remediations, which is why we have the [20:33] code fixing tools there that will physically  do that chick that do that change for you. [20:39] Um, and so, when looking at your organization,  your infrastructure, and your architecture, [20:46] what is the area that needs the most help? You  may not have any code at all cuz you use all SaaS [20:51] tools. Fine. Then it's mostly infrastructure.  You know, if you have a large development shop, [20:56] then that's the thing you're worried about. Um,  and Phoenix is another one that kind of fits in [21:01] like in between like the root cause and the fixed  code. Um, and and so, there's a lot of things out [21:07] there that can help in doing this. But technology  supports a process. Not the other way around. [21:14] And that's the thing that I am pushing to my  industries about my peers so much is like, oh, [21:20] we have this thing that'll do this. I'm like,  but do you have a process to support that? [21:25] You know, cuz if you just feed in, you know,  um, you know, it's like, oh, we can make this [21:30] a lot faster and do this thing and fix this thing  quick. I'm like, if you give a Ferrari to someone [21:34] who doesn't know how to drive or doesn't know how  to drive a stick shift, it ain't going to help. [21:38] And so, make sure that you spend the time to  figure out what is my goal, what is my strategy, [21:45] what is my process, what's the workflow, how's  that workflow going to change? Cuz like I said, [21:50] that workflow is probably there to accommodate  humans. If I start automating pieces of it, [21:55] I may not have to have to have certain things  around like in fixing code. Cloud security. [22:02] I may use cloud code to write code. It's like,  oh, well, do I need to do something else to [22:07] assess the code because it's a conflict of  interest cuz cloud is writing it and then [22:11] read and then fixing it? Cuz in humans it is. You  would have a developer say, hey, test your code, [22:17] make sure it's secure cuz we're a little biased,  right? And we also like kind of like, you know, [22:23] have a central perspective or whatever. Anyone  do AI development? It doesn't even know that it [22:31] wrote that. You tell it to change it from blue  to red, it just rewrites everything. Humans are [22:37] happily lazy and will just be like, oh, search  and replace blue to red, easy. It'll like, nope, [22:42] I'm going to rewrite it all. I don't know who  wrote this first place. This is crazy. I'm going [22:45] to add another function here. I'm going to move  this around. It's insane. It's like, so, sure, [22:50] look at it. You think it's new anyway. Um, but  but just as like as example, but things like [22:56] production and development. We need to keep those  separate. I saw on the last slide that, you know, [23:02] the AI deleting production. Again, I was not  surprised. I wrote an article a couple months [23:08] ago about your agent doesn't have to listen to  you. Just like your employees don't have to listen [23:14] to you. They will go around your controls to do  their job if they think it's getting in the way. [23:19] My second rule as a CISO is being don't  make your users your biggest hackers. [23:24] If they have to go around my controls to do their  job, I failed. I didn't understand the business. [23:31] The analogy I have an article coming out  on accountability for AI and I use a lot [23:36] of dog analogies, even though I don't even  own a dog, but I love dogs. Who doesn't? [23:40] And I said, if you had and so I'll I'll use this  as the example of the OpenAI issue with Hugging [23:46] Face, right? [clears throat] If you had a backyard  that was fenced in for your golden retriever [23:54] and for years the golden retriever stayed  in that fence. And you may have had [23:58] a Labradoodle and a poodle  everything and everything is fine. [24:03] And then you adopt a Belgian Malinois. Anyone  who knows that breed is going to laugh and say, [24:10] yeah, I see you got It was like that fence is  nothing to the Malinois. It'll just hop right [24:14] over it. If it's a 6-ft fence, it'll parkour  its way out. It's not going to hold it there. [24:22] People know that. You may not have known that.  And so you don't get the excuse to things like [24:27] anytime I see the production database got got  destroyed got got deleted. Oh, it wouldn't [24:32] hack something it got out of my sandbox.  I'm like, I knew that was going to happen. [24:38] Why did you not know? Why didn't you test  it? Why didn't you think? And it's because [24:42] the controls I had before for years that  worked perfectly fine for the other dogs, [24:47] I did not even think about that. Did I  have a special super, you know, a super dog [24:53] that's like a 8-year-old  on Red Bull. Um, and and so [24:59] making sure that you understand the risk and  be able to apply the the the things to it. So, [25:04] last slide is and then I'll happy to open  questions. And if you have other reasons [25:09] why this is the bad idea, I would love to hear  them cuz I will add them to my list of things. [25:14] I am happy to be wrong. Um, uh, so this is a thing  where I came up with like what this workflow looks [25:19] like in a more architectural standpoint of we  need to, you know, detect, correlate, diagnose, [25:25] remediate, validate, govern, you know, who's going  to escalate to the humans? Is the human in the [25:29] loop or on the loop? And it's probably going to  be different. I was at a small event down in Black [25:34] Hat from security guy with um, and Jason Clinton  from Anthropic was talking and and and he had uh, [25:41] you know, they're really really big in doing  this. And um, and one of the things that you [25:46] talk about the line that people have is  like, what will you not let it do? And [25:53] there are the those people who will and some  people will say that it's like, I won't let [25:56] it touch production in the development code  update thing. It'll update code in development [26:03] before it goes through testing, but I will not  let it automatically just touch production, [26:07] which goes back into that production development  separation. We have that separation because we may [26:12] do things incorrectly. And I have administrators  with only one access in the other and they have to [26:18] change to another account to log in. I don't  just let push stuff goes from development to [26:22] production, all that kind of stuff. So, those  kind of controls we have still apply. And so, [26:29] you would not just have a human put  anything in production just immediately [26:33] without going through the process. Same thing  here. Um, so but underneath this knowledge base, [26:39] the context to understand, we are going through,  we've seen it now, we're doing it and I've talked [26:44] to companies who are trying to automate this.  We're trying to build knowledge bases of context [26:50] cuz we need to get stuff out of the humans' heads  and put it down in a file in a database somewhere [26:55] that AI we can share and as well as the AI can  use because there's a lot of stuff that's being [27:01] happening in businesses or there's an exceptional  effort of people that's in their heads to know the [27:07] edge cases that oh well this is going to happen  so we need to make sure this doesn't happen. [27:10] Somebody write that down and give it to the  AI so it doesn't do that. So knowledge base, [27:15] obviously governance, project management, testing  um are and um you know, are kind of part of this. [27:25] And then the last thing I have and this kind of  goes to the article I'm writing on accountability. [27:31] And that is we can we can delegate  responsibility but not accountability. [27:38] If your dog bites somebody, it's not the dog's  fault. It's your fault. You're the steward of [27:43] the dog. And you should know the conditions that  it can be in that could cause it to do damage. [27:48] And if you don't, you don't get to  say, "He'd never done that before." [27:53] Because somebody else will look at that and  say, "Oh, I know that's going to happen." [27:56] That poodle? Yeah. Anyway. Um but the uh and and  what what I find very interesting here I there's a [28:03] US and a European example of this is that that E-  EU AI Act specifically calls out natural persons. [28:11] And they define natural persons and they and and  thing and and and section 14 in section nine. [28:17] I mean, in section nine they detail the things  that they define it in section uh 15, article 15, [28:22] article nine, sorry. Um and then COSO, which is  kind of a US uh accounting audit kind of thing, [28:28] you know, they do that and  that cannot be delegated. [28:32] I have the luxury of briefing  boards a lot. I'm a board advisor [28:35] and I see a very different perspective  in boards in the US and outside the US. [28:42] In the US there is this fear of missing out, fear  of being left behind, do something. Do it quickly. [28:49] Cuz we're obviously we're not going to see even  now. And I'm not seeing that outside the US. [28:54] I'm seeing more prudent what is really we try what  it really are we trying to do with this? How is [28:59] this going to help us? Are we trying to get more  revenue, more margin, more customer satisfaction, [29:05] customer retention? What is it we want to get  out of it? Not just like make things go faster, [29:09] do it with less people. It's like what is the  business outcome I'm trying to get? And they [29:13] are looking at transforming what the business is  going to be, what the organizational structure is, [29:18] what the workflow is going to be to be able to  accommodate it cuz technology supports a process. [29:23] And so that you know and and briefing boards on  AI governance that a lot of people have written [29:30] paper about this recently that you know  governance is often confused for control. [29:36] Or it's more often AI governance is really  just bureaucracy disguised as governance. [29:42] And and so governance is really just the  navigator to help you go towards a path under [29:49] any kind of conditions and be prepared for any  conditions understand if this happens we do this [29:53] we do this we do but the actually is not control.  And so I was on many panels about AI governance [29:58] in the last two years and everyone's like well we  don't want to slow down the roll out because the [30:02] business is going to be pushing us and etc. etc.  It was like you know just don't get in the way [30:06] don't make it don't slow it down. And that's not  the intent of governance is to slow things down. [30:12] The intent is to make it go faster. You know  the the the old analogy is about brakes on a [30:18] Formula 1 car. Formula 1 cars can go really fast  because they have good brakes. And the brakes is [30:23] the governance. It's like I know that I'll be  able to take this off ramp and get home safely. [30:28] Not the makeup of the brake and what  it is the fact that it has a role. [30:34] So with that I have 8 minutes so if you have any  questions I am happy to take it. I'm also here [30:42] rest of the morning. You want to catch me outside  in the hall. Happy to be able to do it. So, if not [30:48] about I'll go ahead. Well, thank you very much  if anything else and then I will take questions. [30:54] [applause] Cuz then what happens like I ask for  questions and then everyone leaves and does stuff [30:58] and then it's like, oh, thank you. Bye. Yes. Yeah,  one question, Rick. If you go back to your slide [31:04] um Further down? Yeah. Yeah, here. This the  right-hand side. So, you have Yeah, code side. [31:13] So, code remediation is one thing that because  in the entire, you know, life cycle of different [31:19] technologies they use .NET to Java to Oracle,  right? So, in order for you to do remediation [31:25] you may need to have some kind of curated threat  profiling or threat modeling. Yeah. something that [31:31] we need to think through? Oh, of course. Yeah,  I mean, in any good risk management process we [31:36] need to threat model things. And we need to have  on the what is the potential risk of that and, [31:40] you know, which includes things not just the you  know, unauthorized access and the control but [31:46] also the resiliency of it, you know, the the  quality of it and all those things. So, yes, [31:51] I guess I'm assuming a good program being done.  Okay. Yes. Yeah. And that being said then let's [31:57] say if you get, you know, that today, you  know, Glasswing project and Methos finding [32:02] in the open source vulnerabilities. So, humongous  of data like 45,000 or vulnerabilities but the [32:08] true positive identification is there  any foolproof method or some kind of [32:15] model that you need to curate it to identify  the true positive or it's a part of the journey. [32:20] How do you see it? Yeah, that's all part of That  should already be part of the process is defining, [32:26] you know, first thing is it real? You know,  that was my number one in the remediation is [32:31] do I have a process to find that to do  that at scale, which is excellent question, [32:35] will be something that maybe we an opportunity to  be able to say to be able to use AI to test that. [32:41] But it's a very good point about, you know, that  one step which is there. How you do it may vary, [32:47] but maybe that's a thing. It is how do I automate  the confirming of what's real and what's not. [32:55] Yes. Hi. So, how are we measuring success  within this workflow? I think traditionally [33:03] um we use metrics such as um for vulnerability  remediation, how many vulns have missed SLAs? [33:10] Yep. With the volume of vulns that we're  finding and fixing, what is a more meaningful [33:17] um metric to No, excellent question and  and that's a point is I have a um you know, [33:23] have I done it? Well, I have  a book coming out that talks [33:26] talks a little bit about, but but the but  I'm writing an article about that, you know, [33:32] it does change how we look at metrics. You know,  mean time to identify, mean time to repair, how [33:37] many we fix, how many of these things are vanity  metrics. I I do not brief those to the board, [33:41] you know. Those are vanity metrics. The  real is quality, completeness, accuracy, [33:47] and did it reduce the risk? And so it was like, I  put in 10,000 things. Did it succeed in doing it? [33:54] Did it fix the thing it was supposed  to fix? And did it disrupt anything? [33:59] Those are the kind of things that that now we  look at in non-deterministic way in systems. [34:04] It was the outcomes. So, it's not  about the action, but it's the outcome. [34:09] Would be my what I would say. Yes. Somebody  here? Do Oh, no, he she Anybody else? [34:17] As we're looking through vulnerabilities coming  from multiple tools, and I know some some [34:22] companies on this on this list will take a lot of  the they call them findings and bring them up to [34:27] like an issue level. Um are you seeing if you're  looking for infrastructure applications, um um [34:35] as all of that's coming together, are you seeing  um I guess how does that get rolled up from an SLA [34:41] or even a KPI KRI? Like how does that moving  even from a policy perspective? Cuz a lot of [34:47] times it's criticals need to be remediated in this  many days, highs. Um are they getting boiled up to [34:53] like an issue level and then we're not necessarily  um reporting on the that individual finding level? [35:00] Right. No, that's a good question because it  goes to the technology supports the process. [35:05] And that process was put together for humans  doing it with a finite number of resources. So, [35:10] we need to do the criticals if we did nothing  else. If we have the ability to do more, then we [35:15] need to re-evaluate that what those metrics are.  And defining, you know, criticals may still need [35:20] to be critical for regulatory compliance. And it's  like, "Fine. We'll get those taken care of." But [35:27] also I'm kind of relating back to your question  about like what we're tracking is, you know, this [35:31] will probably easiest for like the more simple  things than the more complex things. You know, cuz [35:36] there is complicated problems and there's complex  problems. And, you know, a complicated problem is [35:41] something that has a solution, but it's not going  to be easy to get to. A complicated problem is, [35:46] you know, world peace, right? It is a something  that we may not be able to come with solution, [35:51] but it's going to mull over some type thing. So,  the the ones that really take a lot of, you know, [35:58] complex you know, thing cuz it's going to  change the business process or things like that, [36:02] we need to, you know, let the humans  do that. But I would say, yes, we will [36:08] always have a priority because it's based on some  contract or some other thing until that changes. [36:14] Um it just we will be able to do the things faster  and then measure it in a different way. But it's [36:21] not like whether it's an incident or an or an  event, you know, like in security operations, [36:27] we have you know, standards and metric rubrics  for that to say the level of impact and the all [36:33] those kinds of things. That should not change.  You know, so could be it's critical because [36:39] it came up this way and my regulation requires  it done 24 hours. So then that's law. Yes. [36:47] Um question about the vulnerabilities keeps coming  up. Every month we have to refresh the database. [36:54] And let's say Tomcat, they keep putting out a new  releases and it keeps popping up and there's 3,000 [37:01] things uh involved to fix. And the  areas to fix is like 3,000 areas, right? [37:07] So you can't keep up with it so because  their AI came up, they vulnerabilities uh [37:12] issues come up quicker, but the fixes don't  come up quick. How do you deal with that? [37:18] Right. I mean, well, that's kind of like a reality  of figuring out like is there a solution to it? [37:23] And that's what you're saying is  like, "Hey, because of mythos, [37:26] all of these infrastructure things are sitting out  you know, tens of thousands of vulnerabilities, [37:30] which may be very old and stuff." I'm like,  "Well, we can't update it because of some [37:34] dependencies or some other things or maybe we  don't know what will be the thing to satisfy." [37:40] And then that's also a thing that we comes more  of the complex problem that we need either humans [37:47] or humans supported by AI to figure out like,  "Well, what is going to satisfy this fix?" Um [37:52] you know, I think we're getting into we talk  a lot about how multiple small low or medium [37:57] vulnerabilities could become high. It's like,  you know, that becomes maybe a whole new workflow [38:03] in the is this real and in the stage of how  do we fix it that doesn't break anything [38:10] that is a whole new workflow in itself and not  just a what do I do. Um we do know how to satisfy [38:15] upgrade the library version. It satisfies it. So  it's an easy fix, but it keeps coming up with a [38:21] new version quicker than you can actually apply  the fix. Right. Then that's Yeah, then that's [38:26] the root cause of the thing. Like we saw that in  cloud where the configuration file had the wrong [38:30] version, but if people kept changing what was in  production and it got refreshed every single time. [38:35] Go to fight figure out what the root cause of  that is. All right, 30 seconds. Last question. [38:40] Did you have a question somebody anybody? No?  Okay. Thank you very much. Yeah, I'll I'll check [38:45] you in the hall cuz I don't want to hold up the  next person. But thank you very much everybody.