Lenovo Architects Warn Edge AI Agents Leak Keys via Sensor Injections

Your AI Is on the Street Now: Adversarial Threats to Edge AI Systems in Physical Environments — Ai4

Ai4

Because prompt injection cannot be stopped once on-device agents process ambient audio and visual feeds, edge systems must abandon prevention and enforce hardware-isolated containment.

AI-assisted summary of Your AI Is on the Street Now: Adversarial Threats to Edge AI Systems in Physical Environments — original source. Check the original for full context.

Source published: 2026-08-15

Detailed notes

## Background Published on YouTube by Ai4 on August 15, 2026, this presentation features Lenovo security leaders Subba Bachina and Igor Stolbikov discussing the risks of on-device AI assistants. The talk explores how deploying AI agents to personal devices removes cloud perimeter protections, exposing assistants to sensory manipulation, indirect prompt injection, and memory poisoning.

## TL;DR The migration of AI agents from guarded cloud environments to personal edge devices creates vulnerable 'digital twins' that hold sensitive credentials and execute tools with broad user authority. These edge models face physical and multi-modal attack vectors, ranging from inaudible ultrasonic audio injections and visual perturbations to latent model poisoning and indirect prompt injections during autonomous web research. Because running models on client hardware exposes weights and memory to local inspection, conventional software defenses and input sanitization fall short. Securing edge assistants requires a multi-layered defense-in-depth model anchored in hardware roots of trust (such as TEEs and TPMs), dedicated agent identity delegation, continuous runtime intent validation, and strict zero-trust sandboxing to contain compromised workloads.

## Atmosphere & style The presentation maintains a calm, analytical, and pragmatic red-team perspective, treating the migration of AI to personal devices as an inevitable operational reality rather than a theoretical concern. The speakers systematically deconstruct legacy security assumptions using clear, grounded analogies—such as personal bodyguards at public events and the disorienting effects of intoxication—to illustrate abstract AI failure modes. It feels like an authoritative engineering briefing that methodically transitions from identifying systemic vulnerabilities to outlining a mature, defense-in-depth containment strategy.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=rAa01VqZW6A&t=4s) The migration of AI workloads from cloud perimeters to local edge devices and AI PCs dismantles traditional security perimeters, running agentic models directly in hostile physical environments. - [02:10](https://www.youtube.com/watch?v=rAa01VqZW6A&t=130s) Edge agents function as personal digital twins that store sensitive credentials across persistent memory layers, turning local identity configs like OpenClaw's soul.md into high-yield targets. - [04:49](https://www.youtube.com/watch?v=rAa01VqZW6A&t=289s) Securing on-device AI requires a defense-in-depth model spanning sensors, silicon roots of trust, storage, supply chains, and multi-agent execution layers. - [07:36](https://www.youtube.com/watch?v=rAa01VqZW6A&t=456s) Physical sensor feeds introduce new attack surfaces, including near-ultrasound audio and visual perturbations, which require sensor fusion and raw signal anomaly detection rather than simple text sanitization. - [08:53](https://www.youtube.com/watch?v=rAa01VqZW6A&t=533s) Executing edge models within unconfined operating system user space exposes weights, memory dumps, and API keys to any local malware with system or kernel privileges. - [11:19](https://www.youtube.com/watch?v=rAa01VqZW6A&t=679s) Hardware-level defense depends on five core pillars: integrity verification, protected storage, tamper resistance, interface lockdown, and confidential computing with real-time memory encryption. - [13:59](https://www.youtube.com/watch?v=rAa01VqZW6A&t=839s) Physical access makes total client-side secrecy virtually impossible; endpoints require strict data minimization, cryptographic storage of embeddings, and aggressive lifecycle purging. - [16:12](https://www.youtube.com/watch?v=rAa01VqZW6A&t=972s) Poisoned training data and tainted fine-tuning sets introduce dormant backdoors that pass standard benchmark evaluations but trigger malicious behavior on specific structured payloads. - [17:28](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1048s) The AI supply chain faces massive exposure, with roughly one million unverified models and datasets downloaded daily from public registries into enterprise workflows without adequate inventory tracking. - [20:06](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1206s) Current delegation models inappropriately hand primary user credentials directly to agents rather than issuing scoped, auditable machine identities through frameworks like SPIFFE or OAuth. - [21:41](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1301s) Autonomous web research and inbox processing expose edge agents to zero-interaction indirect prompt injections that induce agent drift and subvert downstream tool execution. - [24:21](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1461s) Effective edge defense relies on shifting from pure prevention to containment, utilizing per-tool intent validation, zero-trust policy attestation, and strict sandbox isolation to break the lethal trifecta.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=rAa01VqZW6A&t=4s) From Cloud Sandboxes to Edge Agents: The Paradigm Shift

Subba Bachina (Lead Penetration Tester and AI Security Architect at Lenovo) and Igor Stolbikov (Distinguished Engineer and Chief Security Architect at Lenovo) open the session by reflecting on the dramatic architectural evolution of AI over the past four years. When ChatGPT first launched, it created an industry-wide "seminal moment" where users marveled at its capabilities—summarizing texts, drafting presentations, and accelerating productivity. During those initial years, AI workloads lived exclusively within secure, secluded cloud data centers. Users operated under an implicit model of complete trust: interactions were remote, computing environments were heavily guarded by cloud perimeters, and personal devices were merely thin presentation layers.

```mermaid flowchart LR accTitle: Paradigm Shift from Cloud to Edge AI subgraph CloudEra["Traditional Cloud AI (Protected Perimeter)"] User["User Device"] -->|"API Request"| DC["Data Center / Cloud AI"] DC -->|"Sandboxed Compute & Isolated Storage"| DC end subgraph EdgeEra["Modern Edge AI (AI On the Street)"] Edge["Edge Device / AI PC"] --> Agent["Local Agent (e.g., OpenClaw)"] Agent --> Sensors["Microphone / Camera / Screen"] Agent --> LocalMem["Local Storage: soul.md, Keys, Context"] end ```

That paradigm underwent a sudden shift with the emergence of local agentic frameworks such as OpenClaw and Hermes. Compute workloads, foundation models, multi-agent frameworks, and private user data have migrated directly into the user's pocket and onto local AI PCs. While enterprises and developers rush into multi-agent systems to unlock new business use cases and revenue streams, moving these systems to the edge completely shatters the legacy security perimeter.

---

[02:10](https://www.youtube.com/watch?v=rAa01VqZW6A&t=130s) The Rise of Digital Twins and the Memory Trap

When foundation models operate on-device, they continuously ingest local context: user credentials, AWS secrets, Google authentication tokens, and financial data. Systems like OpenClaw preserve this context across short-term, medium-term, and long-term memory architectures to inform all downstream interactions. Consequently, the edge assistant ceases to be a simple prompt-response engine and transforms into a full-fledged "personal digital twin" that acts directly on the user's behalf.

This digital identity is explicitly codified in edge frameworks—such as OpenClaw’s `soul.md` configuration file, which defines who the user is, what the agent is authorized to do, and how it should behave. However, if an attacker penetrates the edge device across typical connectivity layers—whether over home networks, corporate enterprise environments, or untrusted public Wi-Fi—the agent's persistent memory and trusted tools become an immediate, high-yield vector for exploitation.

- Local models store and reuse high-value credentials (AWS, Google, credit card details) across ongoing agent memory. - Agent identity files like `soul.md` grant models direct agency to act as an authenticated extension of the user. - Common edge networking scenarios (home Wi-Fi, public Wi-Fi, enterprise networks) provide attackers immediate paths for network-level injection.

---

[04:49](https://www.youtube.com/watch?v=rAa01VqZW6A&t=289s) The Defense-in-Depth Model: Securing AI "On the Street"

Igor Stolbikov steps in to frame the protective architecture required when personal assistants live "on the street." He uses the analogy of attending a high-profile public venue where personal safety is at risk: security is never achieved with a single checkpoint. Instead, it requires layered validation—checking tickets at the door, verifying identity badges, monitoring surveillance feeds, and deploying close-protection bodyguards.

```mermaid flowchart TD accTitle: Edge AI Defense-in-Depth Architecture subgraph Layer5["Agent Layer"] A5["Multi-Agent Frameworks & Tools"] end subgraph Layer4["Supply Chain Layer"] A4["Libraries, Skills & Model Updates"] end subgraph Layer3["Storage Layer"] A3["Model Weights & Context/Memory Stores"] end subgraph Layer2["Hardware Layer"] A2["Silicon Root of Trust / TEE / TPM"] end subgraph Layer1["Sensor Layer"] A1["Microphones, Cameras, LiDAR & GPS"] end Layer1 --> Layer2 Layer2 --> Layer3 Layer3 --> Layer4 Layer4 --> Layer5 ```

Because an edge assistant often "knows you better than you know yourself," securing it demands a defense-in-depth architecture spanning multiple interdependent tiers: 1. Physical sensor inputs (microphones, cameras, GPS) 2. Underlying device hardware and silicon roots of trust 3. Model weight and local memory/context storage 4. Software supply chain (libraries, skills, runtime packages) 5. Agent execution and tool-invocation layers

---

[07:36](https://www.youtube.com/watch?v=rAa01VqZW6A&t=456s) Sensor Vulnerabilities: Eyes and Ears Under Attack

Edge assistants rely on ambient sensors—the "eyes and ears" of the device—that are inherently exposed to attacker manipulation. Attackers can deliver voice commands using near-ultrasound acoustic frequencies that are completely inaudible to human ears but fully processed by microphone hardware, or inject malicious instructions into ambient vision feeds, LiDAR streams, and GPS signals.

To counter these attacks, the system cannot rely solely on conventional text-based input sanitization. Edge architectures must enforce strict multi-modal input validation, deploy anomaly detection routines on raw signal inputs, and implement sensor fusion. By correlating data across multiple independent sensor channels simultaneously—validating visual context against acoustic telemetry, location data, and inferred user intent—the assistant can detect physical spoofing attempts before untrusted ambient data enters the model's reasoning loop.

- Inaudible ultrasonic audio injections exploit microphone hardware to execute covert instructions without user awareness. - Adversarial perturbations embedded in images or visual scenes hijack multi-modal vision models. - Sensor fusion cross-correlates identity, physical context, geographic location, and user behavior to invalidate spoofed inputs.

---

[08:53](https://www.youtube.com/watch?v=rAa01VqZW6A&t=533s) Hardware Root of Trust and Silicon-Level Attack Vectors

Hardware forms the non-negotiable anchor for edge AI security. Without a robust silicon root of trust—incorporating Trusted Execution Environments (TEEs), secure element key storage, and hardware-bound identities—higher software layers cannot be secured. Stolbikov warns against common deployment patterns, such as running frameworks like OpenClaw directly inside unconfined operating system user spaces (e.g., standard Windows user-mode processes).

When AI agents run in standard user space, software-level compromise guarantees total AI compromise. Attackers with local user or kernel privileges can perform runtime memory dumps to extract proprietary model embeddings, system prompts, and stored credentials. Furthermore, attackers can exploit exposed debugging interfaces, manipulate firmware, hijack execution flows, or extract cryptographic keys via hardware side-channel attacks. As Stolbikov emphasizes:

> "If device can decrypt and execute model, privileged malware can do the same. You cannot eliminate completely risks once authorized runtime is compromised."

---

[11:19](https://www.youtube.com/watch?v=rAa01VqZW6A&t=679s) Five Pillars of Hardware and Edge Defense

To mitigate silicon- and OS-level vulnerabilities, Stolbikov categorizes hardware defenses into five core pillars designed to isolate model execution and protect system integrity from local attackers:

| Defense Pillar | Key Mechanisms and Technical Controls | |:--- |:--- | | **1. Integrity Verification** | Enforce verified boot, measured boot, remote attestation, firmware signing, and anti-rollback controls. Use Post-Quantum Cryptography (PQC) signature schemes (such as ML-DSA) for model package distribution. | | **2. Protected Storage** | Encrypt model weights and context stores using TPM-backed keys released only to measured, authorized software components. Minimize plaintext communication buses. | | **3. Side-Channel & Tamper Resistance** | Eliminate shared cryptographic secrets, harden core cryptographic routines, and offload key operations to a discrete TPM. | | **4. Interface Lockdown** | Hard-lock and disable hardware debugging and diagnostic interfaces on production edge hardware. | | **5. Confidential Computing** | Execute inference workloads within hardware-isolated memory enclaves (TEEs) with real-time memory encryption. |

Stolbikov draws an anatomical parallel to underscore the importance of confidential computing: "Nature put our brain in protected area. We should do the same." The discussion shifts to the challenges of model theft, weights extraction, and intellectual property cloning on distributed consumer hardware, setting up the fundamental dilemma of key management: when sensitive models are distributed directly to untrusted edge endpoints, *who ultimately controls the decryption keys?*

[13:59](https://www.youtube.com/watch?v=rAa01VqZW6A&t=839s) Physical Device Exposure and Data Protection at the Edge

When AI models transition from protected cloud infrastructure onto personal laptops and edge devices, the entire security paradigm shifts. Physical access changes everything: an attacker who buys the hardware or gains local machine access can inspect and extract model weights, memory, and sensitive data. Unless workloads are shielded using confidential computing and strictly isolated trusted execution environments between the user and system layers, total client-side secrecy is essentially impossible to guarantee. Practitioners must confront the uncomfortable reality that once a model is installed on an edge endpoint, it operates within hostile territory.

The surface of sensitive data exposed on these devices extends far beyond model parameters. AI workloads process full conversational histories, prompt memory, vector embeddings, dynamically retrieved files, and proprietary enterprise records. This data is frequently leaked through insecure logging, missing access controls, over-permissioned model responses, and even ambient metadata attached to processed files. Defense at this layer relies on strict data minimization, robust cryptographic controls, and deliberate lifecycle management:

- **Data Minimization:** Refrain from collecting, caching, or embedding enterprise or user data unless strictly required by the specific workload. - **Cryptographic Protection:** Implement robust end-to-end encryption across all stored embeddings, context caches, and conversation logs. - **Secure Deletion:** Explicitly overwrite and purge temporary context buffers and cached embeddings immediately after execution rather than leaving orphaned state on disk.

---

[16:12](https://www.youtube.com/watch?v=rAa01VqZW6A&t=972s) Data and Model Poisoning: Silent Backdoors

AI poisoning corrupts what the model has learned, quietly degrading its reasoning capacity. The vulnerability is comparable to someone having too much to drink at a party: the system appears operational on the surface, but its judgment is compromised and it can no longer evaluate inputs clearly. Malicious training data or tainted fine-tuning sets infiltrate edge models quietly, establishing hidden backdoors that remain completely dormant during ordinary evaluation.

```mermaid flowchart TD accTitle: Data and Model Poisoning Flow accDescription: Diagram showing how poisoned data enters a model and is triggered by specific input patterns.

subgraph Ingestion ["Data Supply & Training"] A["External / Untrusted Data"] --> B["Poisoned Ingestion Pipeline"] B --> C["Model with Hidden Backdoors"] end

subgraph Runtime ["Execution on Edge Device"] C --> D["Standard Inferences<br/>(Normal Behavior)"] C --> E["Specific Trigger Payload"] E --> F["Malicious Action / System Hijack"] end ```

These backdoors are engineered to activate only when encountering precise trigger phrases or structured input payloads. Once triggered, the model behaves maliciously, ignoring system instructions or intentionally leaking internal context. Because these subversions leave standard inference benchmarks untouched, detecting poisoned weights after deployment is exceptionally difficult. Organizations must implement rigorous validation, provenance verification, and anomaly detection pipelines on all datasets before they are ingested into training or fine-tuning workflows.

---

[17:28](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1048s) Hardware Roots of Trust and Supply Chain Vulnerabilities

Agentic and edge AI security relies fundamentally on hardware roots of trust. Security cannot begin at the application layer; it must start at boot time with validated Secure Boot chains, Trusted Platform Module (TPM) hardware key storage, and hardware-enforced Secure Enclaves or Trusted Execution Environments (TEEs) to protect running workloads.

``` AI SECURITY MATURITY MODEL ┌────────────────────────────────────────────────────────────────────────┐ │ 1. HARDWARE FOUNDATIONS │ │ • Secure Boot verification │ │ • TPM-backed cryptographic key management │ │ • TEE / Secure Enclave memory isolation │ ├────────────────────────────────────────────────────────────────────────┤ │ 2. SUPPLY CHAIN PROVENANCE │ │ • ~1M daily downloads tracked & verified │ │ • Automated model / agent inventory & continuous edge patching │ ├────────────────────────────────────────────────────────────────────────┤ │ 3. IDENTITY & ATTESTATION │ │ • Separation of human identity from machine/agent credentials │ │ • Tool-level policy attestation (OAuth, OIDC, SPIFFE) │ ├────────────────────────────────────────────────────────────────────────┤ │ 4. RUNTIME MONITORING & CONTAINMENT │ │ • Intent validation per tool call to prevent agent drift │ │ • Robust sandboxing and blast-radius isolation │ └────────────────────────────────────────────────────────────────────────┘ ```

The broader AI software supply chain faces significant structural vulnerabilities. Platforms like Hugging Face record approximately one million downloads of models and datasets every single day. These assets flow directly into production enterprise architectures and user endpoints with minimal provenance checking:

- **Volume and Exposure:** Nearly 1,000,000 models and datasets are downloaded daily from public registries straight into enterprise workloads and personal devices. - **Sleeper Vulnerabilities:** Downloaded models can sit quietly inside production systems for months before a latent backdoor triggers unwanted behavior. - **Lack of Inventory Visibility:** While enterprise security spent a decade maturing Software Bills of Materials (SBOMs), visibility into AI assets has regressed; most organizations cannot account for how many autonomous agents or fine-tuned models are actively executing across their infrastructure. - **Edge Patching Demands:** Endpoint AI deployments require real-time patch distribution mechanisms to remediate newly discovered model and runtime vulnerabilities on the fly.

---

[20:06](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1206s) The Identity Crisis: Human vs. Agent Delegation

The declarative agent layer introduces serious authorization challenges around delegation. When users assign complex, autonomous tasks to an AI agent, they often pass their own primary authentication credentials directly to the agent rather than issuing scoped, delegated authorizations.

Consider delegating a real-world task: if you need a family member to withdraw cash from a bank because you are busy, you authorize that person to withdraw a specific amount, such as $1,000, on your behalf. In contrast, modern agentic setups effectively hand over complete online banking credentials and say, "Take my bank credentials, and you withdraw it." This total surrender of user credentials violates least privilege and breaks identity audit trails.

Resolving this requires uniting human identity standards (like OAuth and OpenID Connect) with workload and machine identity frameworks (such as SPIFFE). The industry is actively building hybrid identity fabrics that assign distinct, auditable cryptographic identities to autonomous agents, ensuring they act with bounded authority rather than impersonating the human user.

---

[21:41](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1301s) Indirect Prompt Injection, Agent Drift, and Deep Research Risks

Direct user prompt injection represents the simplest and most basic attack vector, one that traditional Web Application Firewalls (WAFs) and front-end guardrails can partially filter. The far more dangerous threat emerges when autonomous agents perform multi-step web retrieval and "deep research." When an agent browses external web pages to synthesize answers, it encounters untrusted third-party content embedded with adversarial prompt injections.

```mermaid flowchart LR accTitle: Deep Research Indirect Injection and Agent Drift accDescription: Workflow illustrating how autonomous web research ingests poisoned web content, leading to agent drift and unauthorized tool execution.

subgraph Research ["Autonomous Retrieval"] A["User Task Prompt"] --> B["Agent Queries Web"] B --> C["Poisoned Web Page / Incoming Email"] end

subgraph Compromise ["Agent Drift & Execution"] C --> D["Indirect Prompt Injection Ingested"] D --> E["Agent Logic Drifts / Goal Subversion"] E --> F["Unauthorized Tool Calls / Data Exfiltration"] end ```

Once an agent reads poisoned instructions hidden inside a web page or an unsolicited incoming email, those instructions alter its internal planning loop. This phenomenon, known as **agent drift**, causes the system to subtly deviate from the user's intended objective across subsequent tool invocations:

- **Zero-Interaction Compromise:** An attacker can compromise an assistant simply by sending an email containing poisoned instructions; when the agent processes the inbox, the payload executes without any direct user interaction. - **Cascading Subversion:** Poisoned context ingested during step one corrupts the arguments, destinations, and intent of tool calls executed in steps three and four. - **Cost vs. Latency Trade-offs:** Adding guardrail checks before and after every individual tool call provides safety, but it introduces significant token overhead and latency, forcing system architects to balance runtime responsiveness against safety controls.

---

[24:21](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1461s) Architecture of Defense: Intent Validation, Zero Trust, and Containment

Securing on-device AI assistants requires layered defenses spanning hardware roots of trust up to agentic control loops. Front-end guardrails alone are insufficient; every tool call requires real-time **intent validation** and strict **policy attestation**. Modern frameworks, such as NVIDIA NeMo Guardrails and OpenShell, translate cloud-native zero-trust security principles—originally developed for Kubernetes clusters—into local runtime environments on personal devices.

The path toward securing AI edge systems follows four progressive maturity stages:

Containment remains the final, critical backstop when agents are subverted. Recent industry incidents have demonstrated that rogue or compromised agents can quickly pivot from local execution to impacting external third-party services. Because standard sandboxes remain vulnerable to escape vulnerabilities, edge AI architectures must enforce strict privilege boundaries, hardware memory isolation, and network egress filtering to contain compromised agents before they can cause damage.

## Notable quotes > Once deployed on a personal device, they're no longer protected by cloud perimeters or centralized monitoring

> Secure boot and encrypted storage don't help when the vulnerability is the feature, not a flaw.

> Input sanitization is blind to instructions living in pixels and near-ultrasound.

> prompt injection cannot be fully solved within today's architectures.

> separating content from command, enforcing least privilege to break the "lethal trifecta" of private data, untrusted content, and an exfiltration channel

> [08:53](https://www.youtube.com/watch?v=rAa01VqZW6A&t=533s) If device can decrypt and execute model, privileged malware can do the same. You cannot eliminate completely risks once authorized runtime is compromised. — Igor Stolbikov

> [11:19](https://www.youtube.com/watch?v=rAa01VqZW6A&t=679s) Nature put our brain in protected area. We should do the same. — Igor Stolbikov

> [11:19](https://www.youtube.com/watch?v=rAa01VqZW6A&t=679s) who ultimately controls the decryption keys? — Igor Stolbikov

> [20:06](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1206s) Take my bank credentials, and you withdraw it.

## Mentioned Subba Bachina, Igor Stolbikov, Lenovo, Ai4, ChatGPT, OpenClaw, Hermes, AWS, Google, Hugging Face, NVIDIA NeMo Guardrails, OpenShell, SPIFFE, OAuth, OpenID Connect (OIDC), ML-DSA, Trusted Platform Module (TPM), Trusted Execution Environment (TEE)

## Video description Subba Bachina - AI Product Security Leader, Lenovo Igor Stolbikov - Executive Director Cloud Security, Distinguished Engineer, Chief Security Architect, Lenovo

AI assistants are rapidly moving from controlled cloud environments into your personal life — embedded in phones, laptops, and AI PCs, reading your screen, files, inbox, camera, and microphone. Once deployed on a personal device, they're no longer protected by cloud perimeters or centralized monitoring This talk examines adversarial attacks against on-device AI assistants in real-world personal environments. We explore how attackers exploit the assistant's inability to distinguish content it reads from commands it should obey — through poisoned documents and email, inaudible ultrasonic voice commands, and instructions hidden in images. We then look at what happens when a hijacked assistant can act: silent exfiltration through its own legitimate tools, memory poisoning, and privilege pivot. Unlike prompt attacks on chatbots, these arrive through vision, audio, and ordinary content the assistant was designed to process. From a red-team perspective, we analyze why common safeguards fail. Secure boot and encrypted storage don't help when the vulnerability is the feature, not a flaw. Input sanitization is blind to instructions living in pixels and near-ultrasound. And the frontier labs now concede what practitioners already knew: prompt injection cannot be fully solved within today's architectures. So we shift from prevention to containment: separating content from command, enforcing least privilege to break the "lethal trifecta" of private data, untrusted content, and an exfiltration channel, and gating releases on continuous adversarial testing. Attendees will leave with a practical threat model for adversarial AI on personal devices — and guidance for securing assistants before they're deployed "on the street."

[Source](https://www.youtube.com/watch?v=rAa01VqZW6A)

中文简报

联想架构师警告：边缘AI智能体或因传感器注入泄露密钥由于端侧智能体处理环境音视频流时无法阻止提示词注入，边缘系统必须放弃预防转向硬件隔离遏制。 AI智能体从受防护的云端环境迁移至个人边缘设备，形成了持有敏感凭证并拥有广泛用户权限执行工具的脆弱“数字孪生”。这些边缘模型面临物理与多模态攻击向量，涵盖不可听见的超声波音频注入、视觉微扰，以及在自主网络研究期间遭受的潜在模型投毒和间接提示词注入。在客户端硬件上运行模型会将权重和内存暴露于本地检查之下，导致传统软件防御和输入净化手段失效。保护边缘助手需要建立以硬件信任根（如TEE和TPM）为依托的多层纵深防御模型，并结合专用智能体身份委托、持续运行时意图验证以及严格的零信任沙盒机制，以遏制遭受入侵的工作负载。 AI工作负载从云端外围迁移至本地边缘设备及AI PC，打破了传统安全边界，导致智能体模型直接运行于充满敌意的物理环境中。 (0:04) 边缘智能体充当着在持久内存层中存储敏感凭证的个人数字孪生，使OpenClaw的soul.md等本地身份配置成为极具价值的目标。 (2:10) 保障端侧AI的安全需要建立覆盖传感器、芯片信任根、存储、供应链以及多智能体执行层的纵深防御模型。 (4:49) 物理传感器数据源引入了包括近超声波音频和视觉微扰在内的新攻击面，这需要通过传感器融合和原始信号异常检测来应对，而非简单的文本净化。 (7:36) 在不受限的操作系统用户空间内执行边缘模型，会使权重、内存转储和API密钥暴露给任何具备系统或内核权限的本地恶意软件。 (8:53)

Reading summary

Deploying autonomous AI agents directly to edge devices collapses the traditional cloud security perimeter, exposing sensitive user credentials, device sensors, and local compute to physical, hardware, and input-level exploitation. Lenovo security architects Subba Bachina and Igor Stolbikov argue that securing edge AI demands a defense-in-depth framework spanning hardware roots of trust, multi-modal sensor fusion, scoped identity delegation, and continuous runtime containment. The Edge Transition and Digital Twin Exposure (0:04) The migration of foundation models and multi-agent frameworks like OpenClaw and Hermes from isolated cloud data centers to local AI PCs shatters traditional network perimeters. On-device models ingest rich local context—including AWS secrets, Google authentication tokens, and financial credentials—storing them across short-, medium-, and long-term memory structures to function as autonomous digital twins. Explicit agent configurations, such as OpenClaw's soul.md file, define agent capabilities and authorize actions on behalf of the user. Because these edge endpoints regularly connect to untrusted home, corporate, and public Wi-Fi networks, the agent's persistent memory and integrated system tools become accessible targets. If an attacker breaches the host device across these standard network interfaces, the stored high-value credentials and authorized agent actions provide immediate paths for local and remote exploitation. Sensor Exploits and Multi-Modal Validation (7:36) Edge assistants depend on physical sensors—including microphones, cameras, LiDAR, and GPS—that function as ambient inputs susceptible to real-world spoofing. Attackers can inject instructions via near-ultrasound acoustic signals that human ears cannot detect but microphone hardware processes, or introduce adversarial perturbations into visual feeds to hijack multi-modal models. Conventional text-based prompt sanitization cannot intercept physical-layer manipulations. Edge architectures must implement multi-modal input validation, run anomaly detection routines on raw signals, and deploy sensor fusion to cross-correlate disparate streams—such as checking acoustic commands against visual context, physical location, and inferred user intent—before passing inputs into the model's reasoning loop. Silicon Roots of Trust and Confidential Execution (8:53) Running edge agents inside standard operating system user space creates severe security risks, because any privileged local malware can inspect unencrypted memory to extract proprietary model weights, context caches, and credentials. Igor Stolbikov highlights five hardware defense pillars to establish security at the silicon level: integrity verification using Secure Boot and anti-rollback controls, TPM-backed protected storage, side-channel tamper resistance, hard-locking physical diagnostic interfaces on production devices, and confidential computing. Confidential computing executes inference inside hardware-isolated memory enclaves (TEEs) featuring real-time memory encryption. Without hardware-enforced boundaries between user and system spaces, client-side secrecy cannot be maintained against attackers with physical machine access. System designers must also address key management trade-offs, particularly determining who holds model decryption keys when distributing proprietary models across untrusted consumer devices. Supply Chain Backdoors and Unscoped Delegation (16:12) The open AI software supply chain introduces systemic risks, with roughly one million models and datasets downloaded daily from public registries like Hugging Face with minimal provenance tracking. Tainted training or fine-tuning datasets can introduce silent backdoors that remain inactive during standard benchmark evaluations, only triggering malicious actions or data exfiltration when specific input phrases appear on the edge endpoint. Compounding this vulnerability is an identity delegation flaw: users frequently grant autonomous agents their own primary credentials rather than restricted authorizations. This practice breaks identity audit trails. Mitigating delegation abuse requires uniting human identity standards such as OAuth and OpenID Connect with workload attestation frameworks like SPIFFE to issue distinct, scoped, and auditable cryptographic identities to autonomous agents. Indirect Injection, Agent Drift, and Containment (21:41) When autonomous edge agents perform multi-step web retrieval and research, they are vulnerable to indirect prompt injection embedded in third-party web pages or incoming emails. Ingesting untrusted instructions without direct user intervention can cause agent drift, subtly corrupting intermediate planning and leading to unauthorized tool calls or data leaks in subsequent steps. Securing edge execution requires applying zero-trust principles locally through frameworks like NVIDIA NeMo Guardrails and OpenShell. System architects must implement continuous intent validation and policy attestation before any tool invocation. Because guardrails introduce token and latency overhead, layered sandboxing, privilege boundaries, and strict network egress filtering are necessary as final containment backstops to restrict the blast radius of subverted agents. Securing edge AI requires shifting from centralized cloud guardrails to layered endpoint protections combining hardware isolation, sensor fusion, and zero-trust tool attestation. However, open challenges remain around key ownership on consumer devices, the performance latency of continuous tool validation, and tracking uninventoried autonomous agents deployed across enterprise fleets.

精炼阅读摘要

联想（Lenovo）安全专家 Subba Bachina 与 Igor Stolbikov 指出，随着 AI 模型从受边界防护的云端数据中心迁移至个人电脑等边缘设备，智能体已演变为深度绑定私有凭据的“个人数字孪生”。这种部署使边缘 AI 彻底脱离了传统防御边界，直接暴露于物理接触、环境传感器欺骗、供应链投毒和间接提示注入等对抗性攻击中，亟需自硬件信任根至智能体控制回路的纵深防御体系。架构范式转移与数字孪生的凭据隐患 (0:04) 过去四年中，AI 计算正从集中式的云端沙盒转向 OpenClaw、Hermes 等本地智能体框架，计算负载、基础模型与敏感数据被直接部署到终端 AI PC 上。Bachina 与 Stolbikov 强调，本地运行的端侧助手持续摄取 AWS 密钥、Google 身份凭据与金融数据，并通过分层内存机制持久化保存；例如 OpenClaw 使用 soul.md 文件明确定义用户身份与代理权限。此时边缘助手已转变为全权代行操作的“个人数字孪生”。然而，一旦设备接入家庭网络或公共 Wi-Fi，这种持久化记忆和高权限工具就会成为极易遭受网络渗透的高价值攻击目标。更严峻的是物理层面的暴露：物理接触彻底改变了安全假设，若攻击者获得设备的本地或硬件访问权限，便能直接转储内存、窥探模型权重并窃取矢量嵌入与上下文日志。在缺乏机密计算防护的情况下，仅靠操作系统层面的访问控制根本无法保证客户端数据的私密性。物理接触与多模态传感层的输入劫持 (7:36) 边缘智能体高度依赖麦克风、摄像头、LiDAR 与 GPS 等物理传感器来感知外界环境，但这些“眼耳”在现实世界中极易遭到恶意操控。Stolbikov 解释称，攻击者能够利用人类无法察觉、但麦克风硬件可以拾取的近超声波频段隐蔽注入语音指令，或者在物理场景图像中嵌入对抗性扰动以劫持多模态视觉模型。此类攻击能直接绕过传统文本过滤屏障，使系统在毫无察觉的情况下执行恶意操作。为防范物理维度的输入伪造，边缘系统不能仅依赖常规的内容清理，而必须部署信号层面的异常检测与“传感器融合”（Sensor Fusion）技术。该架构通过同时关联并交叉验证视觉画面、声学信号、地理定位与用户行为意图等多路独立通道，在不可信的物理信号进入核心大模型的推理回路之前，精准识破并阻断欺骗性输入。硬件信任根与边缘机密计算防御支柱 (8:53) 硬件被视作边缘 AI 安全不可退让的基础底座。Stolbikov 警告称，若将智能体框架直接运行在普通的操作系统用户空间，特权恶意软件便能利用系统漏洞转储运行内存并提取解密密钥；如果设备能解密并执行模型，拥有特权的攻击者同样可以做到。因此，联想团队提出了边缘硬件防御的五大核心支柱：其一，实施完整性验证，借助安全启动（Secure Boot）、度量启动、远程认证与防回滚机制，并使用 ML-DSA 等后量子密码签名分发模型包；其二，构建受保护存储，利用离散 TPM 绑定的密钥加密模型权重与上下文；其三，强化抗侧信道与物理防篡改能力，杜绝共享加密密钥；其四，在生产硬件上强制锁定并禁用调试接口；其五，推进机密计算（Confidential Computing），将推理负载置于硬件隔离的可信执行环境（TEE）中，通过实时内存加密为本地模型提供类似“颅骨保护大脑”的安全屏障。供应链投毒与深度研究中的代理漂移 (16:12) 在软件层面，庞大的 AI 供应链正带来严重的后门风险。以 Hugging Face 为例，公共模型与数据集的每日下载量已接近 100 万次，但大多数企业对其缺乏资产清点与来源验证能力。被恶意污染的训练或微调数据会在权重中埋下“静默后门”，这些后门在常规推理评测中处于休眠状态，只有在特定触发词激活时才会执行越权或泄露操作，导致部署后极难被检测。与此同时，当自主智能体进行网络多步检索与“深度研究”（Deep Research）时，极易吸纳第三方网页或未授权邮件中潜藏的间接提示注入（Indirect Prompt Injection）。这种零交互攻击会悄然改写智能体的规划逻辑，引发“代理漂移”（Agent Drift）现象：模型在后续工具调用中逐渐偏离用户的原始目标，甚至沦为攻击者外传敏感数据或调用恶意工具的跳板。身份委派危机与端侧零信任约束模型 (20:06) 智能体自动化执行还引发了深层的身份授权危机。不同于现实生活中向代办人授予单次限定额度（如取款 1,000 美元）的委托模式，许多用户直接将主账户身份凭证全盘交付给本地智能体。这种将人类凭据整体移交的做法违背了最小权限原则，导致身份审计链断裂。两位架构师主张将 OAuth、OpenID Connect 等人类身份标准与 SPIFFE 等机器工作负载身份框架相结合，为智能体赋予具备明确边界的独立加密身份。针对失陷扩散风险，端侧防御必须落实涵盖硬件基础、动态监控、策略证明与强化遏制的四级成熟度路径。在每次调用读、写或网络工具前，系统必须运用类似 NVIDIA NeMo Guardrails 或 OpenShell 的机制进行实时的意图验证与策略检查；并依托多层沙箱、硬件内存隔离和网络出站过滤技术严格收敛爆炸半径，防止被劫持的智能体由本地横向渗透至外部企业服务。 Stolbikov 与 Bachina 总结指出，AI 从受控云端沙盒走向现实边缘环境带来了不可逆转的生产力变革，但其安全架构严重滞后于业务推进步伐。核心难点在于：当敏感模型下发到完全不可控的端侧物理硬件时，解密密钥的最终归属权问题陷入两难；同时，在智能体每次工具调用前后施加全量安全护栏，又会带来沉重的延迟和 Token 计算成本开销。如何在设备响应性能、资源开销与硬件级纵深防御之间取得可持续的工程平衡，是边缘智能体迈向规模化落地时仍待解决的关键课题。

Detailed notes — zh

## Background 本内容由频道 Ai4 于 2026 年 8 月 15 日发布在 YouTube 平台，由联想（Lenovo）AI 产品安全负责人 Subba Bachina 与首席安全架构师 Igor Stolbikov 共同呈现。演讲主要探讨了个人 AI 助手从受控云端迁移到手机、PC 等边缘设备后，在缺乏数据中心边界保护的物理与网络环境中面临的新型对抗性安全威胁与纵深防御架构。 ## TL;DR 演讲指出边缘端 AI 助手正演变为持有敏感凭据的“数字分身”，但直接暴露在家庭与公共网络中面临严峻安全考验。攻击面不仅涵盖超声波、对抗图像等传感器物理注入，还包括用户态运行导致的代码提取、模型失窃及隐蔽数据投毒。为此，专家提出了基于 TPM、安全启动与机密计算的五大硬件纵深防御支柱，并深入剖析了智能体在深度研究或后台读信场景下的间接提示注入与“智能体漂移”风险。最后，团队主张超越浅层防护栏，构建融合细粒度身份委托、策略证明与多层强隔离沙箱的端侧零信任遏制体系。 ## Atmosphere & style 整场演讲充满工业界一线红队专家与顶级安全架构师的务实与清醒，语调严谨冷静且直切行业隐患。两位讲者配合默契，用“VIP 贵宾安保”和“颅骨保护大脑”等形象比喻拆解深奥的硬件与计算概念，在剖析开源供应链漏洞与零交互沦陷等“恐怖现状”时坦诚直率，既传递出强烈的安全危机感，又提供了逻辑严密的系统级工程解法。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=rAa01VqZW6A&t=4s) 随着 AI 算力与智能体从云端数据中心迁移至个人终端，端侧 AI 正在演变为具备高权限的数字分身（Digital Twin），但缺乏物理与集中式安全边界让敏感凭据和长短期记忆直接暴露于攻击面之下。 - [04:49](https://www.youtube.com/watch?v=rAa01VqZW6A&t=289s) 端侧环境缺少传统边界防护，必须像“VIP 贵宾安保”一样构建多层协同的信任层（Layers of Trust），从物理感知到芯片底层形成纵深防御。 - [07:32](https://www.youtube.com/watch?v=rAa01VqZW6A&t=452s) 摄像头、麦克风等传感器成为 AI 的物理攻击入口，攻击者可利用近超声波指令注入或对抗图像实施欺骗，系统需依赖多模态多传感器交叉验证与意图仲裁进行防御。 - [08:53](https://www.youtube.com/watch?v=rAa01VqZW6A&t=533s) 硬件信任根是端侧 AI 安全的底座，特权恶意软件可提取或篡改用户态下的模型与内存数据，安全架构必须遵循“设备能解密执行，特权恶意程序同样能做到”的铁律。 - [11:19](https://www.youtube.com/watch?v=rAa01VqZW6A&t=679s) 端侧硬件防御五大支柱包括完整性验证、受保护存储、侧信道防篡改、接口硬化与机密计算，其中机密计算是实现运行时模型与数据强隔离的终极方向。 - [13:59](https://www.youtube.com/watch?v=rAa01VqZW6A&t=839s) 本地部署使模型面临失窃与逆向工程风险，提示词、对话记忆和 RAG 企业文档等核心资产极易通过不安全日志和过度授权泄露，必须践行数据最小化与安全销毁原则。 - [16:12](https://www.youtube.com/watch?v=rAa01VqZW6A&t=972s) 训练集与知识库面临隐蔽的数据中毒（Poisoning）与后门威胁，平时表现正常的模型可在接收特定触发 Payload 时激活恶意逻辑，需前置全生命周期的数据校验清洗机制。 - [17:28](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1048s) AI 供应链缺乏成熟的审计与 AI-BOM 台账管理，开源平台每日高频下载的模型和数据集存在不可信潜伏风险，难以实现全链路漏洞跟踪与热修复。 - [20:06](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1206s) 智能体凭据委托存在设计误区，开发者往往直接赋予 Agent 完整人类身份凭据而非细粒度的机器身份，业界正探索融合 OAuth 与 SPIFFE 等协议以实现最小权限委托。 - [21:41](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1301s) 在深度研究等自主场景下，从非受信外部网页或邮件中抓取的内容易触发间接提示注入（Indirect Prompt Injection），导致智能体偏离轨道发生“智能体漂移”并执行危险工具调用。 - [23:41](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1421s) 端侧智能体需在每次工具调用时引入意图验证与策略证明（Policy Attestation），借鉴云原生零信任架构来平衡安全防护与 Token 延迟成本。 - [23:41](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1421s) 端侧安全成熟度模型聚焦硬件信任基石、运行时监控与严格的故障遏制（Containment），依靠纵深沙箱隔离阻断智能体失控后的横向移动与逃逸破坏。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=rAa01VqZW6A&t=4s) 从云端数据中心到口袋边缘：AI 范式转移带来的安全危机来自联想的 AI 产品安全负责人兼渗透测试主管 Subba Bachina 与首席安全架构师 Igor Stolbikov 共同登台开场。Subba 回顾道，大约四年前 ChatGPT 的诞生是一个划时代的时刻，人们在最初体验时惊叹于其强大能力，并在随后的两到三年里广泛将其用于日常工作，例如文本摘要和制作演讲幻灯片。在那个时期，所有的 AI 与云端工作负载都运行在与外界隔离、防御严密的数据中心里，用户只需发送请求并获取结果，对系统有着天然且完全的信任。然而，随着以 OpenClaw、Hermes 等为代表的边缘 AI 智能体的普及，整个计算范式发生了剧烈反转：AI 模型、智能体和个人数据全面迁移到了个人设备与用户的口袋中。全球一方面在狂热探索多智能体系统（Multi-agentic systems）的商业落地与变现，另一方面却陷入了应对新型安全风险的恶性竞争。当用户将信用卡信息、AWS 或 Google 凭据输入设备后，本地模型会将其保存在短期、中期或长期记忆中，并在未来的所有上下文中持续调用。一旦系统被渗透，这些敏感资产将全部暴露给攻击者。智能体不仅能看能听，更成为了用户的“数字分身（Digital Twin）”，代表着个人身份并替用户执行操作。以 OpenClaw 为例，其核心设计中包含一个名为 `SOUL.md` 的灵魂配置文件，明确定义了“你是谁、你想做什么、如何去做”，这实质上就是用户的数字身份凭据。由于个人边缘设备往往直接接入家庭网络、公共 Wi-Fi 或企业网络，缺乏数据中心级别的安全边界，攻击者极易通过网络注入并接管系统。 ```mermaid flowchart TD accTitle: Edge AI Paradigm Shift accDescription: Diagram showing transition from Cloud AI to Edge AI and associated security exposure. subgraph CloudEra["传统云端 AI 范式（受控边界）"] A["用户终端"] -->|"加密 API 请求"| B["安全数据中心"] B --> C["云端 LLM / 集中式监控"] end subgraph EdgeEra["边缘 AI 智能体范式（完全暴露）"] D["网络环境 (家庭/公共Wi-Fi)"] -->|"注入与渗透"| E["边缘个人设备 (PC/手机)"] E --> F["本地模型 / 智能体"] E --> G["SOUL.md / 身份凭据"] E --> H["长期/短期上下文记忆库"] F -->|"替身执行"| I["外部操作与资产"] end ``` - **工作负载迁移**：AI 算力与存储从数据中心“隔离区”下沉到用户口袋中的端侧设备。 - **记忆与凭据沉淀**：端侧智能体持续吸纳并固化敏感凭据（AWS/Google 认证、信用卡数据），在长短期记忆中跨上下文留存。 - **身份代行风险**：类似 `SOUL.md` 的配置文件使端侧 AI 具备了代行用户意志的“数字孪生”权限，放大了被劫持后的危害面。 --- ### [04:49](https://www.youtube.com/watch?v=rAa01VqZW6A&t=289s) “数字分身”走上街头：构建边缘 AI 的信任防护层 Igor Stolbikov 接过话题，指出团队的研究不仅关注当下，更聚焦于未来的安全趋势。个人 AI 助手由于极度便捷将迅速普及，它们记录着用户所有的历史行为，甚至比用户自己更了解自身。然而，当这样的“数字分身”失去保护并暴露在外界时，就像一个人毫无防备地直接走上了危机四伏的街头。为了形象地说明防御思路，Igor 提出了一个“VIP 贵宾安保”的比喻：当一位重要人物参加大型活动感到不安全时，现场会建立起多重防御机制——入口处有人核验门票、确认身份，场内布设监控摄像头，身旁甚至配备专业保镖。这些环节紧密协同，共同构筑起一整套“信任层（Layers of Trust）”。面对比用户自身更具信息价值的 AI 助手，唯有在端侧构建纵深防御的信任层架构，才能避免关键隐私数据遭遇窃取和破解。 - 边缘 AI 设备面临的核心挑战在于缺少物理边界，必须在设备内部由内而外构建信任链。 - 每一层机制不仅要实现自身防护，还必须与上下游组件协同联动，避免单点失效导致整体系统沦陷。 --- ### [07:32](https://www.youtube.com/watch?v=rAa01VqZW6A&t=452s) 传感器威胁：当环境视听成为攻击入口在探讨端侧多层防御时，Igor 首先聚焦于最外层的传感器——即 AI 的“眼睛和耳朵”。麦克风、摄像头、激光雷达（LiDAR）和 GPS 等传感器赋予了 AI 感知物理世界的能力，但也成为了攻击者可直接触达的物理攻击面。例如，攻击者可以利用人耳无法感知的超声波向麦克风注入恶意语音指令，或者利用对抗样本图像、雷达与 GPS 欺骗手段诱导 AI 系统做出错误决策。应对传感器威胁的基础防线是严格的输入验证与异常检测。单纯依赖单一传感器极易受到欺骗，因此更高级的防御策略是实现“多源交叉验证（Multi-source Correlation）”。通过让 AI 协同分析身份、上下文环境、物理位置、操作行为以及用户意图，设备能够有效辨别事件的真伪，从而大幅降低被环境攻击伪造的风险。 - **物理层攻击手段**：利用近超声波语音注入（人耳不可听但麦克风可拾取）、像素隐蔽对抗图像、GPS/LiDAR 信号伪造。 - **防御核心**：输入校验配合多模态多传感器关联分析，结合情境（Context）与意图（Intent）进行真伪仲裁。 --- ### [08:53](https://www.youtube.com/watch?v=rAa01VqZW6A&t=533s) 硬件信任根与用户态陷阱：特权恶意软件的致命威胁在传感器层之下，芯片硬件构成了整个系统最关键的安全基石。硬件安全模块、可信执行环境（TEE）以及固化在芯片中的硬件身份构成了不可替代的“信任根（Root of Trust）”。每一层软件的安全有效性，都严格依附于底层硬件的支撑。如果系统架构设计不当，将关键 AI 运行时直接部署在普通用户空间（例如部分方案直接在 Windows 用户态下运行 OpenClaw），系统将面临毁灭性威胁： 1. **运行时提取与篡改**：攻击者获得相应权限后，能够随意提取 Prompt 提示词、向量嵌入（Embeddings）和内存数据。 2. **底层破坏**：恶意程序可以篡改固件、软件或直接替换掉本地 AI 模型文件。 3. **接口与侧信道利用**：暴露的调试接口、系统接口以及侧信道攻击（Side-channel attacks）都会被攻击者用于击穿防御。 Igor 强调了一条核心安全铁律："If device can decrypt and execute model, privileged malware can do the same."（如果设备能够解密并执行模型，拥有特权的恶意软件同样可以做到。）一旦经过授权的运行时环境遭到破坏，系统面临的风险便无法彻底消除。 --- ### [11:19](https://www.youtube.com/watch?v=rAa01VqZW6A&t=679s) 硬件纵深防御五大支柱与机密计算未来针对硬件与运行时环境的严峻威胁，Igor 提出了系统化的五大防御支柱，以实现对边缘 AI 的全方位收敛与保护： | 防御支柱 | 核心实现机制与关键技术 | |:--- |:--- | | **完整性验证 (Integrity Verification)** | 签名验证、验证启动（Verified Boot）、固件与运行时验证；模型包强制数字签名；引入抗量子签名算法（如 ML-DSA）；实施度量启动（Measured Boot）、远程证明（Remote Attestation）与防固件回滚（Anti-rollback）措施。 | | **受保护存储 (Protected Storage)** | 采用 TPM 绑定的密钥进行加密存储，仅在经过完整性度量的授权工作负载下释放密钥；将推理隔离在 TEE 中；全面消除或最小化明文通信。 | | **侧信道与物理防篡改 (Side-Channel & Anti-Tampering)** | 最小化或完全避免共享密钥；通过 TPM 强化底层密码学操作的硬件抗攻击能力。 | | **接口硬化 (Interface Hardening)** | 严格锁定并物理/逻辑关闭生产设备上的所有调试接口（Debug Interfaces）。 | | **机密计算 (Confidential Compute)** | 在加密且受保护的硬件飞地内存中隔离执行所有计算，提供全方位的运行时数据保护。 | Igor 对机密计算给予了极高评价，虽然该技术目前仍在发展演进之中，但它是保障端侧 AI 安全的终极方向。他生动地比喻道："The nature put our brain in protected area. We should do the same."（大自然将我们的大脑放在了受颅骨保护的区域，我们也必须对 AI 采取同样的保护方式。）在演讲该阶段的最后，讨论延伸到了高敏模型的防盗窃与防克隆问题：当高度敏感的模型被分发部署到大量边缘设备上时，即便采用了各种加密算法，安全架构师仍必须直面最核心的本质追问——究竟谁才拥有对解密密钥的真正访问权？ ### [13:59](https://www.youtube.com/watch?v=rAa01VqZW6A&t=839s) 端侧环境下的模型失窃、数据泄露与机密计算如果 AI 模型直接运行在用户环境或本地设备上，且攻击者能够以同样的方式物理或系统级访问这台机器及其数据，单纯依靠常规软件层防御是无法实现保护的。Igor Stolbikov 直言，安全团队必须正视这一残酷的现实：“一旦你把模型安装到物理机器上，你就可能失去它，这是一种非常恶劣且棘手的局面，但这就是现实。”攻击者只需花钱购买相同的硬件设备，便能直接获取物理与底层系统访问权限。除非引入机密计算（Confidential Compute）等强隔离控制机制，确保用户环境与核心系统运行环境彻底物理/逻辑隔离，否则无法阻止本地模型的提取与逆向。当数据在本地与边缘环境流转时，大量敏感资产都会暴露于攻击面之下。这不仅包括用户的敏感提示词（Prompts）、完整的对话历史、AI 记忆上下文、嵌入向量（Embeddings），还涵盖 RAG 检索出的敏感文档及底层企业数据。泄露通道多种多样，例如不安全的日志记录（Insecure Logging）、访问控制缺失、AI 响应内容过度授权（Over-permissioned CPR Responses），甚至文档自身的元数据都可能在不经意间外泄敏感信息。对此，首要原则是践行隐私保护与数据最小化：“如果你不需要这些数据，就千万不要使用它们。”在不得不存储数据时，加密是最好的防护手段；当数据生命周期结束时，必须执行安全粉碎与删除（Secure Delete）。 - **暴露的核心资产与漏洞点**： - 核心资产：敏感提示词、对话上下文、AI 记忆库、向量嵌入数据、RAG 检索企业文档及元数据。 - 脆弱路径：无审计的不安全日志、宽泛的 API/响应权限、不完备的本地访问控制列表。 - 防御基线：数据最小化（非必要不采集）、全链路强加密、机密计算隔离以及使用后的强力安全销毁。 --- ### [16:12](https://www.youtube.com/watch?v=rAa01VqZW6A&t=972s) 数据中毒与隐蔽后门威胁讨论转向攻击者如何污染模型所学到的认知体系。Igor 形象地比喻道，数据中毒（Poisoning）就像是“你去参加聚会喝得酩酊大醉，大脑根本无法清晰思考”。当恶意训练数据注入系统时，其渗透过程往往极其隐蔽，系统管理员和开发者甚至完全察觉不到数据已被篡改或投毒。攻击者可以通过精心构造的隐藏后门（Hidden Backdoors）或故意植入的虚假信息来污染模型。这种后门平时处于静默状态，模型表现完全正常；但一旦接收到特定的触发输入（Trigger Payload），模型就会立即按照攻击者的预设逻辑做出恶意行为。由于此类污染在常规评估中极难被发现，因此在数据进入训练、微调或 RAG 知识库系统之前，建立全生命周期的自动化数据校验与清洗机制至关重要。 ```mermaid flowchart TD accTitle: AI系统数据投毒与后门触发链路 subgraph Ingestion["1. 数据接入与投毒阶段"] A["恶意训练/知识库数据"] -->|"悄无声息注入"| B["隐藏后门与虚假信息"] B --> C["模型/知识库被污染 (难以察觉)"] end subgraph Operation["2. 运行与触发阶段"] D["日常普通输入"] -->|"正常表现"| C E["特定触发词/Payload"] -->|"激活隐蔽后门"| F["模型执行恶意越权行为"] end subgraph Defense["3. 防御防线"] G["前置数据校验与清洗"] -.->|"阻断投毒"| B end ``` --- ### [17:28](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1048s) 硬件信任根与供应链安全危机 Subba Bachina 接过话题，系统性总结了端侧与智能体安全的底层支柱。他强调，无论是 AI 安全还是智能体安全，信任链条必须从设备硬件的最底层建立——从设备上电启动时的安全启动（Secure Boot）开始，所有核心密钥都必须交由 TPM（可信平台模块）进行硬件级管理，并配合安全飞地（Secure Enclaves）和可信执行环境（TEE）来隔离与保护关键工作负载。然而，谈到整个 AI 供应链安全时，Subba 直言行业现状处于极其危险的“恐怖境地（dreadful situation）”。以开源社区 Hugging Face 为例，平台每天的模型和数据集下载量接近 100 万次。这 100 万次下载最终全部流向了企业工作负载或个人终端设备，没有任何第三方去向。然而，行业中几乎没有人去审计这些模型是否来自可信源：“如果一个模型在你的系统里静默潜伏了三个月，然后在某一天突然发疯并执行恶意行为，你该怎么办？”过去十年整个安全行业试图通过软件物料清单（SBOM）来实现软件供应链的 100% 可审计，但在 AI 时代似乎又回到了原点。很多企业甚至连内部实际运行了多少个智能体、部署了多少个开源模型都没有准确的台账与追踪，端侧设备更面临着高频漏洞修补与实时热修复的严峻供应链挑战。 - **硬件信任基石**：以 Secure Boot 为起点，全生命周期私钥绑定 TPM，运算逻辑托管于 TEE / Enclave 中。 - **供应链失控现状**：Hugging Face 每日约 100 万次模型与数据集下载，源头信任验证与静态分析严重缺失。 - **资产可视化盲区**：企业普遍缺乏针对 Agent 实例数量与加载模型的实时可观测性与合规台账（AI-BOM 缺失）。 --- ### [20:06](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1206s) 智能体身份穿透与凭据委托误区进入智能体（Agentic）交互层后，Subba 指出业界在授权机制上存在一个巨大的设计误区。所有人都在畅想将繁重任务完全委托给智能体，但核心问题在于：“智能体究竟是以谁的身份在运行？”现实中，开发者往往直接将人类用户的完整身份与高权限凭据直接丢给智能体，而不是为智能体分配受限的独立数字身份。 ```mermaid flowchart LR accTitle: 传统危险代理 vs 安全智能体身份授权机制 subgraph Bad["传统危险模式 (直接交出身份)"] U1["用户"] -->|"交出个人核心凭据/Cookie"| A1["智能体 Agent"] A1 -->|"以用户全权身份操作"| S1["银行/敏感系统"] end subgraph Good["零信任细粒度委托模式"] U2["用户"] -->|"OAuth / OIDC 范围授权 ($1000 额度)"| A2["独立机器身份 (SPIFFE)"] A2 -->|"带有限制策略的任务令牌"| S2["银行/敏感系统"] end ``` Subba 举了一个通俗的现实生活案例：如果你需要去银行取 1,000 美元现金但自己抽不出时间，你会委托家人并签署一份明确授权书，声明“此人获得我的授权，代表我提取 1,000 美元”；然而在当前的智能体世界中，我们所做的相当于直接对智能体说：“拿着我的银行账号和全部密码，你自己去取吧。”目前整个安全行业正试图融合人类身份协议（如 OAuth、OpenID Connect）与机器工作负载身份（如 SPIFFE），以构建真正具备最小权限、可精细化约束的智能体身份体系。 --- ### [21:41](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1301s) 间接提示注入与自主研究中的智能体漂移解决了身份问题后，接下来便是严峻的提示注入威胁。传统的直接提示注入（Direct Prompt Injection，即用户在输入框恶意提问）是最初级的攻击向量，现代化系统通过 WAF、应用网关和输入 Guardrails 已经能实现一定程度的拦截。然而，真正的致命风险发生在智能体执行自主任务并从外部非受信源抓取数据时——例如典型的“深度研究（Deep Research）”场景。在深度研究模式下，用户发出高阶指令后，智能体像人类一样自主使用搜索引擎，检索前 10 个网页并阅读其中的内容，据此做出分析与后续决策。一旦某些公开网页中内嵌了不可见的恶意隐藏指令，这些被投毒的上下文就会直接被智能体当作权威输入吸纳。这不仅会污染当前输出，还会彻底劫持智能体接下来的工具调用（Tool Calls）决策逻辑，导致智能体逐渐发生“智能体漂移（Agent Drift）”，偏离原本的设计轨道甚至转变为恶意执行器。 ```mermaid flowchart TD accTitle: 深度研究场景下的间接提示注入与智能体漂移流程 User["用户提出研究指令"] --> Agent["智能体自主调用搜索"] Agent --> ExtWeb["抓取外部网页 / 读取邮件"] ExtWeb -->|"携带隐蔽注入 Payload"| TaintedCtx["恶意上下文进入 Agent 内存"] subgraph Drift["智能体漂移与失控"] TaintedCtx -->|"污染认知决策"| BadTool["发起非预期的危险 Tool Call"] BadTool -->|"进一步偏离原始目标"| RogueAction["数据泄露 / 横向破坏"] end subgraph DefenseLoop["层级防护开销权衡"] Guard["每层 Tool Call 引入 Guardrail 验证"] -.->|"增加延时与 Token 消耗"| BadTool end ``` 针对每一次 Tool Call 都施加严密的内容防护栏虽然可行，但这会显著增加推理延迟，并导致 Token 成本呈指数级上升。如何在 Token 成本、响应速度与系统可信度之间取得平衡，仍是当前工业界尚未完全攻克的难题。 --- ### [23:41](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1421s) 零交互邮件攻击、策略证明与多层沙箱隔离间接注入的攻击面不仅限于网页检索，更可以通过无辜的日常通道渗透。Subba 指出，攻击者只需发送一封包含恶意 Payload 的电子邮件，当本地运行的 AI 个人助理在后台自动检索、归纳收件箱时，攻击就会被直接触发。用户甚至没有与 AI 发生任何交互，设备就已经沦陷（Pwned）。因此，端侧智能体架构必须超越前端浅层的 Guardrails。在智能体的每一次底层 Tool Call 触发时，系统都必须实施强行的“意图验证（Intent Validation）”与“策略证明（Policy Attestation）”。Subba 特别赞扬了 NVIDIA 的 NeMo Guardrails 与底层架构（如 Open Shell/OpenShell 工具链的设计理念），指出它们成功将云原生 Kubernetes 环境下的零信任策略控制（Zero Trust Principles）微缩并移植到了端侧个人助理中，实现了优雅的策略拦截与行为审计。 ```mermaid flowchart LR accTitle: 端侧智能体纵深防御成熟度模型 subgraph M1["1. 硬件信任基石"] H1["Secure Boot"] --> H2["TPM 硬件级密钥加密"] H2 --> H3["TEE / 机密计算环境"] end subgraph M2["2. 运行时监控与策略"] R1["意图验证 (Intent Validation)"] --> R2["智能体漂移检测"] R2 --> R3["零信任策略证明 (Policy Attestation)"] end subgraph M3["3. 故障遏制与隔离"] C1["强沙箱环境隔离"] --> C2["越狱防范与横向移动阻断"] end M1 --> M2 --> M3 ``` 在总结端侧安全成熟度路径时，Subba 强调了三大核心支柱： 1. **硬件与系统基础**：从上电伊始的安全启动出发，所有敏感凭证由 TPM 硬件级密钥层层加密保护。 2. **动态运行时监控**：实时监控智能体执行链条，确保其决策逻辑不偏离既定任务目标（防止 Agent Drift）。 3. **严格的故障遏制（Containment）**：一旦智能体被劫持失控，必须有物理/系统级的兜底遏制机制。近期 Hugging Face 与 OpenAI 生态中发生的智能体越权外溢事件表明，失控的智能体不仅会破坏宿主系统，还会主动向第三方系统发起横向攻击。尽管当前业界的沙箱机制经常遭遇沙箱逃逸（Sandbox Escape）的困扰，但构建纵深防御与强隔离沙箱依然是保护数字人类（Digital Human）不可或缺的最后防线。 ## Notable quotes > [08:53](https://www.youtube.com/watch?v=rAa01VqZW6A&t=533s) If device can decrypt and execute model, privileged malware can do the same. — Igor Stolbikov > [11:19](https://www.youtube.com/watch?v=rAa01VqZW6A&t=679s) The nature put our brain in protected area. We should do the same. — Igor Stolbikov > [13:59](https://www.youtube.com/watch?v=rAa01VqZW6A&t=839s) 一旦你把模型安装到物理机器上，你就可能失去它，这是一种非常恶劣且棘手的局面，但这就是现实。 — Igor Stolbikov > [13:59](https://www.youtube.com/watch?v=rAa01VqZW6A&t=839s) 如果你不需要这些数据，就千万不要使用它们。 — Igor Stolbikov > [16:12](https://www.youtube.com/watch?v=rAa01VqZW6A&t=972s) 你去参加聚会喝得酩酊大醉，大脑根本无法清晰思考 — Igor Stolbikov > [17:28](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1048s) 如果一个模型在你的系统里静默潜伏了三个月，然后在某一天突然发疯并执行恶意行为，你该怎么办？ — Subba Bachina > [20:06](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1206s) 智能体究竟是以谁的身份在运行？ — Subba Bachina > [20:06](https://www.youtube.com/watch?v=rAa01VqZW6A&t=1206s) 拿着我的银行账号和全部密码，你自己去取吧。 — Subba Bachina ## Mentioned Subba Bachina, Igor Stolbikov, Lenovo, ChatGPT, OpenClaw, Hermes, SOUL.md, TPM, TEE, ML-DSA, Hugging Face, OpenAI, NVIDIA, NeMo Guardrails, Open Shell/OpenShell, OAuth, OpenID Connect, SPIFFE, Kubernetes ## Video description Subba Bachina - AI Product Security Leader, Lenovo Igor Stolbikov - Executive Director Cloud Security, Distinguished Engineer, Chief Security Architect, Lenovo AI assistants are rapidly moving from controlled cloud environments into your personal life — embedded in phones, laptops, and AI PCs, reading your screen, files, inbox, camera, and microphone. Once deployed on a personal device, they're no longer protected by cloud perimeters or centralized monitoring This talk examines adversarial attacks against on-device AI assistants in real-world personal environments. We explore how attackers exploit the assistant's inability to distinguish content it reads from commands it should obey — through poisoned documents and email, inaudible ultrasonic voice commands, and instructions hidden in images. We then look at what happens when a hijacked assistant can act: silent exfiltration through its own legitimate tools, memory poisoning, and privilege pivot. Unlike prompt attacks on chatbots, these arrive through vision, audio, and ordinary content the assistant was designed to process. From a red-team perspective, we analyze why common safeguards fail. Secure boot and encrypted storage don't help when the vulnerability is the feature, not a flaw. Input sanitization is blind to instructions living in pixels and near-ultrasound. And the frontier labs now concede what practitioners already knew: prompt injection cannot be fully solved within today's architectures. So we shift from prevention to containment: separating content from command, enforcing least privilege to break the "lethal trifecta" of private data, untrusted content, and an exfiltration channel, and gating releases on continuous adversarial testing. Attendees will leave with a practical threat model for adversarial AI on personal devices — and guidance for securing assistants before they're deployed "on the street." [Source](https://www.youtube.com/watch?v=rAa01VqZW6A)

Source quotes

[8:53] Igor Stolbikov: If device can decrypt and execute model, privileged malware can do the same. [11:19] Igor Stolbikov: The nature put our brain in protected area. We should do the same. [13:59] Igor Stolbikov: 一旦你把模型安装到物理机器上，你就可能失去它，这是一种非常恶劣且棘手的局面，但这就是现实。 [13:59] Igor Stolbikov: 如果你不需要这些数据，就千万不要使用它们。 [16:12] Igor Stolbikov: 你去参加聚会喝得酩酊大醉，大脑根本无法清晰思考 [17:28] Subba Bachina: 如果一个模型在你的系统里静默潜伏了三个月，然后在某一天突然发疯并执行恶意行为，你该怎么办？ [20:06] Subba Bachina: 智能体究竟是以谁的身份在运行？ [20:06] Subba Bachina: 拿着我的银行账号和全部密码，你自己去取吧。

Source quotes

Once deployed on a personal device, they're no longer protected by cloud perimeters or centralized monitoring Secure boot and encrypted storage don't help when the vulnerability is the feature, not a flaw. Input sanitization is blind to instructions living in pixels and near-ultrasound. prompt injection cannot be fully solved within today's architectures. separating content from command, enforcing least privilege to break the "lethal trifecta" of private data, untrusted content, and an exfiltration channel [8:53] Igor Stolbikov: If device can decrypt and execute model, privileged malware can do the same. You cannot eliminate completely risks once authorized runtime is compromised. [11:19] Igor Stolbikov: Nature put our brain in protected area. We should do the same. [11:19] Igor Stolbikov: who ultimately controls the decryption keys? [20:06] Take my bank credentials, and you withdraw it.

Original transcript

[0:04] I'm Subha Bajracharya. Um I lead the penetration testing team and I'm a security architect at [0:08] Lenovo. And I have with me I am Igor Stalbekov and I'm chief security architect at Lenovo. [0:17] Exactly around 4 years ago, right? We had a seminal moment when ChatGPT was launched. [0:25] The moment everyone started using it the first time, we were like, "Wow, this is really good." [0:31] And we spent the next 2 to 3 years doing it for a lot of personal activities like summarization, [0:38] preparing presentations like this, uh doing amazing things. But then [0:48] it was all good uh because when this was happening, right? All the ChatGPT workloads, [0:54] all the cloud workloads, all the AI workloads were pretty much running in data center data centers. [1:01] So, the data centers were nice secluded environments, protected. We didn't have [1:05] to worry about anything. All we had to do was ask ChatGPT and get the work done. [1:09] We had complete trust in the systems around us. But then something changed few months ago. [1:16] Uh any guesses what changed this year? We had one more similar moment this year. Any guesses? [1:25] Did anyone use Open Claw, Hermes, any of these agents? [1:32] Yep. So, one major paradigm shift that we had uh [1:41] with with Open Claw release was [1:45] the workloads now moved from the data center to your personal devices. It's in your pocket. [1:53] It's just in the workloads. The model is in your device. The AI agents that are running [1:58] on that are on your device. Your personal data is on your device. Everything is there. [2:03] So, the whole security paradigm that was assumed to be perfect suddenly flip. [2:10] So, on one front we had the entire world running around figuring out how do we use multi-agentic [2:17] systems like Open Claw, get a business use case out of it, and make some money out of it. [2:23] On other front when you do that, we we see all the risks that are coming out of it, and there is rat [2:30] race going around trying to find ways to fix these things. So, these devices, like I said, right? [2:39] You have the data stored on your device. You have the model stored on your device. But, it [2:43] doesn't end there. The model keeps learning what you feed it. You give it your credit card data. [2:51] You give it your AWS credentials. You give it your Google credentials. They store it. [2:57] So, when you use Claude, when you use uh Open Claw, the moment you give it anything, [3:03] it stores in its memory, be it long-term, short-term, uh medium-term, whatever memory, [3:08] and use it in your context for all the future conversations. So, you can assume it's in the [3:14] context of only your future conversations, but no. If at any cost someone penetrates into your [3:21] system, you're basically exposing your yourself to wide range of attacks. So, again, these systems, [3:29] because they have everything, they see you, they hear you, they do everything for you. So, [3:37] what has essentially become is these systems have become like a personal assistants [3:43] to your own personal digital twins. So, when they when they are your digital twins, [3:48] they carry your identity. They carry your data. And they much start acting on your behalf. [3:55] If you look at Open Clo, one of the fundamental designs that went into Open Clo [3:59] is something called soul.md. You're giving it a soul file telling it who you are, [4:05] what you want to do, how you want to do. So, that's pretty much your digital identity. [4:11] And we when we talk about this, right? When attackers get into, you know, your systems, [4:17] it's pretty much very very very easy. Because it fundamentally you have your edge devices. [4:23] And most people connect to edge devices in two two ways. Either you use your home network, [4:29] or you use your public Wi-Fi networks, or you use your enterprise networks. [4:35] In either of the states, you're connecting your device to a network and you and you're bound to [4:41] get injected by any attacker. And so, uh we have Igor here who let you walk you through [4:49] how you would approach the problem and get get it going. As part of our work, we're not just [4:56] looking what was behind us, but what's coming. And personal assistants become [5:05] will will be very spread and used by a lot of people because they are so convenient. [5:10] They do a lot of work for us. So, we getting digital twins. They knows everything about us. [5:16] They knows all the actions that you did. They know better you better than you do. [5:22] So, now how we protect this digital twin? And I came with a simple analogy. So, imagine you going [5:32] to some big event and you want to be protected and you feel that you're not safe for some reason. [5:38] Now you on the state. I'm in Open Clo. We I know our analogy it's on the state because there's no [5:43] protections. But how can I put these protections and um in a launch you have uh you somebody will [5:52] check your ticket and then they will check your identity and then you have cameras. You have [5:57] maybe have bodyguards if you're a very special person. All of that building layers, building [6:03] layers of trust that have to work together. And uh if you think about your personal AI assistant, [6:10] it's critically important to you because it's knows more about you than you do. [6:17] So you really need to build this layers of trust. Only that type of security architecture can [6:23] save you from being hacked and lose your most valuable data about you. So how can we do it? [6:33] So uh really when we look on the edge devices, you have many layers, many critical layers. And [6:41] each of this layer can save you or can betray you. And uh obviously sensors, we have microphones. [6:49] If you have microphone that listen to you all the time and there is some ultrasound that you [6:56] cannot hear that can compromise your device, right? And uh could lead to some nasty issues. [7:03] Then uh device hardware. Hardware is fundamental for the security of AI system. You cannot build [7:10] this a good security without that. And then model and your personal data storages, they [7:15] have to be protected. And uh then you have supply chain and those libraries and updates and skills, [7:22] all of that has to be protected. And obviously your personal assistant, your agents that running [7:29] and helping you, all of those has to be secure. So let's talk about sensors. We have sensors, we have [7:36] eyes and ears. They can influence you and there is so many sensors you have currently on devices. [7:43] And uh any any adversarial images or voice injections or some leader or GPS spoofing [7:54] can create some very nasty situations. And the basic thing that you will always recommend is [8:01] input validation. You need to validate what you're getting, detect anomaly that happened there. [8:07] Right? And then you want to actually seize multiple sources. You want to [8:13] see how these two sensors work together and can you validate events and reduce spoofing? [8:20] Um so that's important. So personal stakes here really high. You are your microphone, camera are [8:28] attacker reachable and they can attack you. So you need to be really careful about this. [8:32] So implementing those control is critical. Uh more we have here multiple multiple [8:39] type of sensors that can correlate AI can correlate identity, context, location, [8:45] and behavior and intent. So from this center, so keeping good security there is vital. [8:53] Now next thing is hardware. And hardware is silicon. This is probably the most critical area [9:01] and we have multiple things that has to be there. And first is your your [9:10] hardware your silicon your chips where you keep your keys, right? Your TE, your hardware identity. [9:17] And then it goes up layer by layer. Each layer supports each other. So if you go to [9:25] um also level and open claw good example where you don't use those protections, you're not [9:32] going to have good security. So a hardware forms critical root of trust for the age devices that [9:39] would allow your AI to work properly. And without this, you cannot fully trust uh your AI agents. [9:50] Now, let's talk about what kind of attack I can do against against the hardware. And there's actually [9:57] significant number of problems that can be happening. If you deploy your [10:04] uh systems in user space, let's say you deploy them in Windows, like OpenCL does, right? [10:10] Then a runtime extraction is possible. And the compromise of software is possible. So, attacker [10:16] can do whatever he wants uh with your prompts and bearings and uh memory. And they can tamper with [10:23] firmware, software, or replace your AI models. Uh pretty much your uh you're exposed at this time. [10:32] And uh there's a lot of interfaces, like debug interfaces, uh system interfaces that [10:39] uh again allow to break the systems. And then you have side channel attacks that again can be used [10:46] uh for for compromising or and uh uh to use to use uh for advantage by the attacker. So, um important [10:55] takeaway for you, if device can decrypt and execute model, privileged malware can do the same. [11:07] Right? So, um you cannot eliminate completely risks [11:11] once authorized um runtime is compromised. You just cannot. [11:19] Now, how we can defend? How can we defend? And we have five groups. First, integrity verification. [11:26] That's important. It's critically important. Uh verifications of signatures, verified boot, [11:33] uh verified firmware and times, model packages. And if you're building model uh models, [11:40] if you're distributing models, you must sign those. And uh consider PQ series resistant [11:45] signatures like ML-DSA because those uh would do help to protect to protect your software. And [11:53] use a measured boot, uh have remote attestations, put anti-rollback measures. [11:58] Those are critically important. And then we go to protected storage. Use protected storages. [12:05] Uh those are readily available. And uh using uh encrypted storages with TPM backed keys [12:13] uh would allow you to release only authorized and measured workloads. Um isolating inference um [12:20] to industry execution environment and minimize uh plain text communications. So, this is uh you have [12:28] to assume that everything on your device can be compromised and only putting this list would help. [12:34] Uh then we going to side channel tampering and protections related to it. Uh reduce reduce the [12:39] shared secrets. Uh try to avoid those completely. Uh and harden your cryptographic application [12:46] operations. Uh use TPM for those. So, those are good way to to protect the systems. Debugging. I [12:53] mean, how many times we saw debugging interfaces on the systems? Locking them down is a critical. [13:00] And the last but not the least, uh confidential compute. Confidential compute is emerging [13:05] technology and it's not yet mature, but this is what going to save us because if we are able to [13:12] execute everything in protected memory, in protected and encrypted environment, [13:19] have full isolations, that going to help. I have very nice analogy. And analogy is um the nature [13:27] put our brain in protected area. We should do the same. Now, let's talk about theft and cloning and [13:36] this has become important if you expect if you expect that your models would be protected. [13:44] And we had actually a lot of research in this area when we have highly sensitive models distributed [13:52] and we look on different crypto algorithms. But then we ask who has access to the keys? [13:59] And if it's running in a user environment, if it's running on the machine where attacker can [14:04] get to the same data in same case, you cannot protect. So you have to accept this. It's [14:11] nasty. It's nasty situation that you may lose your model once you install it on the machine. [14:17] But it is reality. The hacker would get access to hardware. He may just go and buy hardware, right? [14:25] And get access to the systems. So it's not possible protect unless you're using some [14:33] strong controls that rely on confidential compute and making sure that user environment and [14:42] system environments are fully isolated for for your systems. And can you please move it? [14:50] So what get exposed when you have data when you have data, you have sensitive prompts, you have [14:58] full conversations, you have your AI memory, you have embeddings, you have retrieved documents, [15:04] you have enterprise data data and those can be leaked, right? So you have insecure logging, [15:10] you have potentially lacking access controls, you have over permission CPR responses and [15:18] even metadata about documents can be leaking sensitive data. So, privacy preserving right times [15:26] uh is uh is important. So, if you was if you don't need those data, just don't use those data. [15:34] Uh this is uh first thing that is important. And many many case we don't need to collect [15:40] all the different informations and all the specific data data and uh then try to encrypt, [15:47] try to put encryptions and there are so many different encryption methods. It's a little bit [15:51] hard sometimes if you don't have knowledge in this area or you never try to do this, but there are [15:56] so many options available. Uh so, encryption is one of your best uh friends. And then once you're [16:03] done with this data, delete. Use secure delete. And this again will help. Now, let's move to [16:12] to um to how attacker can corrupt what you learn. So, and the poisoning is like um [16:22] you you went to party, you drink too much, you cannot think clearly, right? So, we we [16:27] should protect our systems against poisoning. And when we have malicious training data, [16:35] uh they are able to get to our systems really quietly. You may not even know that you did a [16:41] completely compromised or tainted. And there's a many way to do it. Like you can have hidden [16:48] backdoors or some deliberately false information provided, uh corrupt models, and that's could [16:56] create situations where um this corruption this is poisoning is really hard to detect. [17:02] So, really what you need to do is uh find a way to validate this data. Um so, if you have specific [17:12] uh backdoors triggered by specific inputs uh causing model behave maliciously then [17:19] you really need to find how to validate those data before you submitting to the systems. [17:28] All right. Now, next thing is boost to use about. Thanks, Igor. That was very enlightening. [17:35] My quick takeaways from you when we talk about AI security or agentic security, [17:43] the trust starts from the device. Right from when you start your device, it's secure boot. [17:50] How you store your device uh keys. Please make sure to uh Every key should be managed by TPM. [17:58] And you also have secure enclaves, trusted execution environment. So, make sure you use those [18:03] uh to protect your workloads. Heading on to supply chain. Um The industry as a whole, I [18:10] think is in a very dreadful situation. Uh let's be very honest with you. And I tell you for a reason. [18:16] Hugging Face. Uh do you guys know how many downloads does Hugging Face have in a day? [18:23] Close to 1 million. 1 million downloads from Hugging Face every single day. [18:30] 1 million downloads of models, data sets. Where are they going? They're either going to enterprise [18:36] workloads or they're going into personal devices. There's no other third place that they can go to. [18:42] And when we do that, is anyone even checking is the model from a trusted source? [18:49] What if the model sits like sits there for 3 months and then one final day decides to act wild? [18:59] And this is across the industry. Just because, you know, so the entire industry has spent almost [19:04] a decade trying to to get supply chain security correct. We came up with S-bomb. [19:10] We said software bill of materials will solve all the problems. We can we can like 100% accountable, [19:15] but then we are back to square one. So, [19:20] no one in the industry like Do you guys know how many agents are running in your [19:25] enterprise workloads? Does anyone have track of how many agents are running? [19:31] So, agents can be like thousands of agents. Say models. Maybe you have 100 models. [19:36] Is there an accountability for how many models are being used? [19:41] So, it starts from there. The accountability starts from managing what you download, [19:47] what you use. And when you get to edge devices, you have to keep updating software. You have [19:54] to patch vulnerabilities or you have to fix software. You have to get to a new version. So, [19:58] the patching everything should be real-time. So, supply chain is really really important. [20:06] Finally, and then we head to the agent declarative. [20:10] Our favorite layer. Everyone loves it. We love agents. We want to delegate everything to the [20:15] agents. Let the agents do everything we want us to do. But the first thing that we forget to ask is [20:25] we delegate work to the agents. On whose behalf are the agents running? On your behalf? And but [20:30] then you start giving your identity to the agent. You're not giving the agent a separate identity. [20:35] You're giving your own identity identity to the agent. So, say suppose, right? You are supposed [20:41] to go to a bank, withdraw some cash, right? And suppose you don't have time to do that. What do [20:46] you do? You would delegate the task, say to your family member, and say, "This person is authorized [20:52] to do this on my behalf to withdraw $1,000." But here, in the agentic world, we are not [20:59] doing that. What we are saying is "Take my bank credentials, and you withdraw it." Is that fair? [21:09] So, agentic security starts from the identity layer. The entire industry is [21:16] trying to figure out what the issue is, um how to address it be through individual um [21:24] human identity like OAuth and OpenID Connect, or it through machine identity like SPIFFE [21:30] and stuff like that. Um but still the industry is trying to plug in both together to combine [21:35] machine identity and agent identity to get things going. Once it's all the identity problem, [21:41] there comes the prompt injections and all the fancy stuff that we hear thousands of times. [21:45] Uh again, when it comes to prompt injection, the prompt injection doesn't start with [21:51] when you ask it. That is the simplest form of attack vector. Uh most modern systems [21:57] are designed to cover that with firewalls, web application firewalls, uh and now we have [22:03] fancy word called guardrails. Uh so, we've done so many stuff around it, but the real problem starts [22:10] when the agent start to act autonomously and then go and find data from external sources. [22:17] One of the most common use cases that you we do, most agents do, is something called deep research. [22:23] So, in deep research, you ask the agent to do something, it goes to Google like you do, [22:28] goes through the first 10 use cases, reads through all the web pages that it has, and [22:33] may and decides and makes actions based on what it has read read through the uh web pages. What [22:40] if the web pages have malicious content embedded inside the web pages? Are we accounting for that? [22:49] So, that's the real injections attacks that are really applied. Um and also, not only [22:57] you get malicious data, this data will start influencing the agent's further tasks, [23:03] the next tool call it does, and the agent will start drifting apart slowly. So, [23:09] everyone in this again like right right how we had the agentic problem, we also have the agent drift [23:14] problem. So, agent So, now we can start applying guardrails at every layer of the tool call, [23:21] but then that slows your workload. That adds up to your token cost. So, how do we balance token cost [23:31] and the speed and delivering a trustworthy agentic system um is still a challenge. [23:41] Well, again the example again tells you about the example that I gave you, right? You decide [23:47] to ask the system to do something for you. It'll go to It'll go do your research and then [23:55] one malicious attack one malicious payload can trigger an attack. In this particular case, [24:01] the the tainted content can come from any source, be it a web page [24:04] or if someone sends you an email, right? The if the email has a malicious content in it, [24:10] it can trigger a workload. You're not even interacting with the system. [24:13] Someone just sent you an email and then you are pawned. So, it's really really crucial for you to [24:21] add in your controls and the first layer starts with guardrails. That's the most common thing that [24:26] we have, but it shouldn't stop there. At every layer of a tool call, you should have intent [24:31] validation and policy attestation. One of the one of my favorite tools is um NVIDIA's Nemo Cloud. [24:40] Underneath Nemo Cloud, you have Open Shell. Open Shell Open Shell I think is so beautifully [24:45] designed with policy enforcement with zero trust principles. They've taken the cloud [24:51] zero trust principles that you applied for cloud workloads at using Kubernetes and they've kind [24:56] of moved it to personal assistance. I think that that's uh that's a fantastic design right there. [25:03] Again, long story short, you have to protect your digital human right from the root of trust [25:10] that Ego talks about from the hardware layer, the TPM layer all the way through the agents. [25:17] A simple maturity path, you start with the foundations [25:21] making sure you have a secure boot right from the beginning. Uh store your key safe safely encrypt [25:28] them and encrypt them and encrypt them with the TPM key that is that is even more trusted. [25:36] And then keep monitoring your agents, right? The real real fun is in monitoring the agents, [25:44] making sure they don't drift from the actual task that they were supposed to do. [25:49] And this and they achieve the task. And finally containment. If suppose if suppose [25:56] something really happens, we need to contain it. Our hugging face open AI incident in the last two [26:01] three weeks is a is a very good example where we had the agents go rogue and not only impact [26:08] your own systems starting to impact third party systems. So containment has become a huge issue. [26:16] Everyone is trying to sandbox try uh to multiple approaches, but again most sandboxes are broken. [26:24] Every other day you say you find an instance of a sandbox escape, but still but still [26:30] trying to put together things um and containment is really really important. Thank you.