Former OpenAI Ethicist Says AI Safety Requires Roadmap Metrics

前 OpenAI 伦理学家称 AI 安全需纳入路线图指标 Chloé Bakalar 认为，除非科技公司将明确的伦理护栏与权衡取舍融入生产路线图，否则企业的 AI 安全原则就依然只是毫无意义的表面功夫。 Chloé Bakalar 提出了一个实用框架，旨在将 AI 伦理从高层次的理论原则转化为具体的生产工作流程。她概述了七项核心信条——重点强调技术本质上并非中立，而不作为同样具有道德分量——随后详细阐述了“嵌入式伦理的三大支柱”：原则清晰度、务实工具化和参与式问责制。通过关于 AI 个性化与权衡评估的案例研究，她论证了伦理治理需要在模型、系统和用户层面进行持续衡量，从而超越单纯的上线审批，转向以人类福祉为导向的持续维护态势。 前 OpenAI AI 伦理负责人 Chloé Bakalar 指出，脱离实际发布软件与生产代码的伦理原则只是“表面文章”，AI 伦理必须直接嵌入工程构建中。 (0:04) 技术本身并非价值中立，在规模化应用中微小的设计抉择都会产生巨大的社会连锁反应，而机构的真实道德取向往往直接体现在其资源分配与产品路线图上。 (2:39) “嵌入式伦理”（Embedded Ethics）框架由原则清晰性、务实工具化和参与式问责三大支柱构成，要求将道德考量贯穿于产品生命周期的每一环，而非仅作为上线时的终审签署。 (11:41) 评估 AI 风险需涵盖模型级、系统级与用户级遥测，单凭孤立测试无法防范由产品包装和用户交互引发的“社会技术（sociotechnical）”系统性危害。 (15:14) 负责任的 AI 是一种长期的“维护姿态”而非一次性的上线审查；面对不可避免的不确定性，团队必须将伦理约束明确落实为具体的指标、门禁和应急手册。 (25:54)

Former AI ethics lead Chloé Bakalar argues that technology is inherently non-neutral and that high-level ethical principles remain mere theater unless operationalized directly into product roadmaps, metrics, and engineering artifacts. To navigate competing business pressures and inevitable trade-offs, organizations must replace vague intentions with an embedded, continuous governance discipline across the entire product lifecycle. Core Tenets and Limits of Abstract Principles (2:39) Bakalar asserts that every technical and design choice embeds moral values, whether creators intend it or not. At scale, seemingly minor user experience decisions—such as the selection of a reaction icon—can produce outsized societal effects, while organizational non-action merely defaults to reinforcing the status quo. Because human values frequently clash (such as individual liberty versus collective safety), AI systems cannot satisfy all stakeholders, requiring explicit decisions about which trade-offs to accept. Furthermore, Bakalar contends that the proliferation of corporate AI principles expressing support for "fairness, freedom, and safety" accomplishes little on its own. Unless ethical standards are embedded into shipped software, she characterizes these statements as "simply theater." True organizational values are ultimately reflected not in published declarations, but in resource allocation, hiring priorities, and product roadmaps. The Three Pillars of Embedded Ethics (11:41) To bridge the gap between academic theory and high-pressure software delivery—often constrained by rapid development cycles, incomplete information, and competing cross-functional interests—Bakalar introduces a framework termed "Embedded Ethics." This model translates philosophical ideals into tangible organizational artifacts across three pillars: Principled Clarity, Pragmatic Instrumentalization, and Participatory Accountability. Principled Clarity requires teams to articulate early non-negotiables, explicit optimization targets, and documented launch-blocking guardrails. Pragmatic Instrumentalization converts abstract goals like fairness into actionable metrics, evaluations, review gates, and incident response playbooks. Finally, Participatory Accountability incorporates external domain specialists and affected communities into governance loops to counter internal blind spots, acting as an epistemic and moral safeguard before harms appear in telemetry. Multi-Tiered Measurement and Sociotechnical Risk (15:14) Operating within tech engineering environments requires empirical quantification, following the industry norm that unmeasured priorities rarely reach product roadmaps. Bakalar outlines a three-tier measurement framework consisting of model-level evaluations (testing base capability, robustness, and safety), system-level evaluations (assessing context windows and agentic tool integrations), and user-level telemetry (tracking real-world misuse, model drift, and harm). Bakalar warns against the "sociotechnical fallacy"—the false assumption that passing isolated model-level benchmarks guarantees safety. Even a model that performs flawlessly in isolated testing can produce severe harms once integrated into specific product wrappers, user interfaces, and complex social contexts where incentives and human behaviors interact unpredictably. Managing Trade-Offs in AI Personalization (19:24) Applying embedded ethics to product design, Bakalar examines the development of an AI personal assistant. Rather than treating personalization as a binary feature, she maps it along an operational spectrum ranging from zero persistence and session-scoped memory to opt-in account memory, adaptive learning, and full behavioral tracking by default. Each step toward greater utility and automated retention increases risks of context leakage, filter bubbles, and behavioral manipulation. To make a defensible product decision, teams must synthesize business objectives, regulatory risks, and empirical data from user studies and safety evaluations. In Bakalar's illustrative scenario, the balanced resolution avoids opaque automated inference in favor of an opt-in memory model defaulted to off, reinforced by clear data boundaries and staged rollouts to preserve user agency without sacrificing product value. Bakalar concludes that responsible AI cannot be treated as a one-time launch approval, but must function as a continuous maintenance posture of iterative learning. While developer teams operate under uncertainty and cannot foresee every second-order consequence, she insists that the inevitability of risk does not excuse disengagement; rather, it demands that organizations enforce principled constraints and measure what they build.

伦敦大学学院高级研究员、前 OpenAI 伦理主管 Chloé Bakalar 指出，人工智能技术绝非价值中立，任何未落实在发布软件、工程路线图与度量指标中的伦理准则都只是“行业戏剧”。她主张建立涵盖明确原则、务实工具化与参与式问责的“嵌入式伦理”框架，将抽象道德准则转化为应对速度、不确定性与商业博弈压力的日常决策纪律。 技术非中立与负责任AI的现实法则 (2:39) Bakalar 认为，负责任 AI 的基础前提是承认技术内嵌特定道德价值，不存在所谓价值中立。规模化系统会急剧放大看似微小的设计选择，而不做决定本身也是一种偏向现状的道德抉择。由于多元文化和全球用户群体的诉求并不统一，伦理问题无法依靠单一标准答案解决。当个人自由与集体安全等核心价值发生冲突时，团队必须在互斥选项间做出明确权衡，无法满足所有利益方。 她进一步指出，伦理并非某个首席伦理官的专属任务，而是全员的共同责任。检验一家技术机构真实价值观的标尺不是高调的原则宣言，而是其资源流向——即研发预算、团队招聘、开发路线图与项目优先级如何分配。 研发重压与嵌入式伦理的三大支柱 (11:41) 在真实的产品研发流程中，价值判断贯穿于确定研究方向、设定成功指标、配置审查门禁与监测上线表现等各个环节。然而，这一过程面临三重现实压力的严重挤压：模型与产品迭代极快的“速度”、缺乏完整信息的“不确定性”，以及法律、商业、技术与合规部门相互交织的“利益竞争”。如果治理介入过晚，伦理审查极易沦为拖慢发布的摩擦阻力。 为化解这一困境，Bakalar 提出了“嵌入式伦理”（Embedded Ethics）架构，由三大支柱构成：其一是“原则明确”（Principled Clarity），要求预先清晰界定系统优化目标、不可妥协的底线与已知盲区，严格区分阻断发布的红线与引导优化的目标；其二是“务实工具化”（Pragmatic Instrumentalization），将安全与公平等理念转化为具体的评估基准、审核卡点与应急预案；其三是“参与式问责”（Participatory Accountability），建立吸纳外部利益相关者的常规治理回路。 三层级度量与社会技术风险防范 (15:14) 在工程实践中，无法度量的内容往往无法进入开发路线图。Bakalar 将评估体系划分为三个层级：评估能力、安全与稳健性的“模型级测试”；评估上下文窗口、工具集成与智能体链路的“系统级验证”；以及追踪真实环境滥用、模型漂移与用户反馈的“用户级遥测”。她警示团队警惕“社会技术谬误”（Sociotechnical Fallacy），即孤立测试中表现安全的模型，在放入具体产品界面包装与社会激励情境后，仍可能引发严重的下游连锁危害。 为弥补内部视角的结构性盲区，研发团队必须引入领域专家、区域代表与受影响群体。Bakalar 说明，外部参与并非赋予各方绝对否决权，而是作为认知与道德层面的双重防线，在系统性危害显现于遥测数据之前提供预警，促使团队以透明方式声明所保护的价值与接受的风险。 实战权衡：个性化AI的光谱与阶梯决策 (19:24) 以 AI 个人助手为例，Bakalar 说明伦理决策不应被简化为“要或不要”的二元对立，而应放在操作光谱上进行精细评估。个性化方案涵盖从“零持久性”（会话结束即清除，高隐私但低效用）、“会话级记忆”、“用户自选加入的账户记忆”（Opt-In Memory）、“自适应学习”到“默认全行为追踪”（效用最大但隐私与操纵风险最高）的完整梯度。 在评估此类功能时，团队必须权衡实用性与隐私、自适应与可预测性、跨情境便利与偏见泄露等多重矛盾。根据试点研究与安全审查数据，自动化推断偏好可能会固化刻板印象并锁死用户行为模式。因此，更为合理的工程方案是采用“默认关闭的自选记忆”模式，设定严格的数据边界并实施分阶段灰度上线，通过结构化护栏建立兼顾效用与责任的技术实践。 Bakalar 总结强调，AI 领域的不确定性与潜在滥用绝非放弃责任的借口，组织无法企求绝对完美，而应追求兼顾结果与稳健流程的“足够好”治理标准。负责任 AI 本质上不是产品发布时的单次过关姿态，而是持续迭代与反馈学习的“维护姿态”。决定技术未来的根本问题不在于 AI 具备何种能力，而在于开发者愿意确立何种约束并将其真正落实进工程实践。

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=53846JjFolY&t=4s) 开场与免责声明：在行业前沿落地应用 AI 伦理 Chloé Bakalar（前 OpenAI AI 伦理主管、前 Meta 首席伦理学家，现任伦敦大学学院高级研究员）向听众问好，开宗明义地指出本次演讲的核心议题：在大型科技公司的工业级规模下，究竟该如何实践 AI 伦理，特别是如何将伦理转化为研发与产品周期中切实可落地的操作机制。 在正式展开前，她给出了一个简短的法律免责声明：今天的发言完全代表她个人的学术观点与个人经验，并不特指她在 OpenAI、Meta 或其他前任职机构的具体工作细节。她风趣地调侃道，如果现场有人是冲着探听所谓“不为人知的深层伦理黑幕与商业机密”来的，那今天恐怕要失望了，“或者我们留到稍后的 Happy Hour 再聊”。玩笑过后，她强调将依托自己在学术界与工业界十余年来定义和执行应用 AI 伦理的扎实经验，探讨这一处于技术前沿的紧迫课题。 ### [01:20](https://www.youtube.com/watch?v=53846JjFolY&t=80s) 核心锐评：无法随版本一同交付的 AI 伦理纯属演戏 Chloé Bakalar 抛出了一个被她称为“Hot Take”的尖锐观点。过去十年里，整个行业充斥着各种宏大的“AI 原则与价值观”清单，罗列着公平（Fairness）、自由（Freedom）、安全（Safety）等辞藻。她吐槽道：“当然，谁会反对这些呢？每个人都喜欢泛化的公平。”在清单里发表这类宽泛、定义不清的宣言轻而易举，真正的难点在于如何确保这些价值观能在最终构建和交付的技术产品中被真实、忠实地体现出来。 由此，她直截了当地给出了结论：“**AI that doesn't ship with the build is simply theater.**”（无法随构建版本一同交付上线的 AI 伦理，纯粹是一场演戏）。人们固然可以在意研发和部署机构的主观意愿，但最终必须落脚于 AI 系统与产品在现实世界中的实际行为，以及它们到底向社会反馈并投射了何种价值观。 ### [02:39](https://www.youtube.com/watch?v=53846JjFolY&t=159s) 负责任 AI 的六大基石原则与“顺藤摸瓜”的第七原则 Chloé 梳理了她在实践中总结出的六大负责任 AI（Responsible AI）核心原则，作为指导整个伦理评估的底层脚手架： 1. **技术从来不是价值中立的（Technology is not value-neutral）**：无论开发者是有意还是无意，构建和交付的每一个系统底层都内嵌了道德取向与价值判断。 2. **没有唯一正确的答案（There is no one right answer）**：伦理实践没有放之四海皆准的标准答案（尽管绝对存在显而易见的错误答案）。面对全球化产品服务的多元社群，必须保持开放心态。 3. **不存在无足轻重的“小选择”（No such thing as a small choice）**：在 AI 公司的超大规模量化效应下，即使是选择“勾选框、点赞拇指还是爱心图标”这样看似微小的交互细节，一旦放大到数亿用户层面，都会产生巨大的连锁反应，且规模化扩散的方式往往出人意料。 4. **不作为本身就是一种作为（Non-action is still action）**：避开决策、选择不做任何事，绝不代表免除了决策责任。选择不作选择依然是一种道德抉择，而且往往伴随着维持既有现状的保守偏见。“你无法选择退出伦理”。 5. **不可能兼得一切（We can't have it all）**：理性的人们在伦理抉择上必然存在分歧，许多价值观本身是互斥的。例如，绝对的个人自由与绝对的系统安全无法同时达成，没有任何一套人类价值体系能够满足所有人。 6. **伦理是所有人的责任（Ethics is everyone's responsibility）**：伦理工作不能仅依赖某一个伦理负责人或将某个人当成全公司的“道德中心”。伦理专家提供工具、指导与视角，但所有团队成员都必须对自己的选择及其实际影响负责。 在此之外，她还补充了至关重要的第七条隐藏原则——**顺着资金与资源的流向看（Follow the money）**。任何机构表达自身真实价值观的最佳方式，就是看它把资源投向何处。因此，运营决策、路线图规划（Roadmapping）、优先级排序以及招聘决策，本质上全都是伦理决策。 ```mermaid flowchart TD accTitle: 六大 AI 伦理基石原则与延伸原则 subgraph CorePrinciples["负责任 AI 核心原则 (Core Tenets)"] T1["1. 技术非价值中立<br/>(内嵌道德判断)"] T2["2. 无唯一正确答案<br/>(包容多元社群)"] T3["3. 无微小琐碎决策<br/>(规模效应放大影响)"] T4["4. 不作为亦是作为<br/>(无法逃避伦理选择)"] T5["5. 无法兼得一切<br/>(价值观存在天然冲突)"] T6["6. 伦理是全员责任<br/>(拒绝单一道德中心)"] end T7["7. 资源投向即是价值观<br/>(Follow the Money: 预算/招聘/路线图)"] -.-> CorePrinciples ``` ### [06:56](https://www.youtube.com/watch?v=53846JjFolY&t=416s) 产品研发全生命周期的嵌入式伦理与现实挑战 伦理贯穿于产品生命周期的每一个环节：从最初的研究立项（“我们在做什么？我们是否应该使用 AI 来做这件事？”），到成功指标的设定、评估（Eval）维度的测量与阈值定义，再到决定哪些缺陷属于阻断上线（Launch-blocking）的标准，乃至产品部署上线后的持续跟踪与迭代。整个闭环中没有任何一个决策是纯粹的技术问题，处处都是关于优化目标、风险容忍度和利益相关方权衡的价值判断。 而在现实中，这种产品生命周期绝非静态流程，决策总是在高压、信息不完备、模型和产品上下文不断变动的极度动态（Thrash）环境中做出。她指出当前实践面临的三大核心张力： - **速度（Speed）**：模型能力演进和产品迭代极快，而治理往往滞后。若伦理直到最后上线前夕才被动介入把关，就会倒逼团队重新审视前期架构，从而拖慢整体交付节奏。 - **不确定性（Uncertainty）**：如同“在飞行中边造飞机边开飞机”，充斥着已知与未知的未知数，必须在缺乏完整证据时做出判断。 - **利益冲突（Competing Interests）**：伦理固然重要，但在决策室里极少是唯一的考量。技术可行性、法律合规、商业利益、公关传播（Comms）以及运营诉求交织在一起，彼此目标并不总是协同一致。 因此，伦理的目标不可能是建立在真空里的“完美先见之明”，而必须是一套**扎实的决策纪律（Decision-making discipline）**：它必须承受住压力、具备明确指导性、经得起现实碰撞，并能随着这门仅有约十年历史的新兴交叉学科不断迭代演进。 ### [11:41](https://www.youtube.com/watch?v=53846JjFolY&t=701s) 嵌入式伦理的三大支柱（Embedded Ethics Framework） 为将伦理深度融入日常工作机制，Chloé 提出了“嵌入式伦理”（Embedded Ethics）框架，由三大支柱及对应的具体工程管理交付物组成： ```mermaid flowchart LR accTitle: 嵌入式伦理三大支柱及其具体产出 subgraph P1["支柱一：原则清晰度<br/>(Principled Clarity)"] direction TB A1["明确优化目标与底线"] A2["权衡取舍显性化"] A3["交付物: 决策日志/准入标准"] end subgraph P2["支柱二：务实工具化<br/>(Pragmatic Instrumentalization)"] direction TB B1["将价值观转化为指标/阈值"] B2["多层级评估与遥测体系"] B3["交付物: Eval集/看板/Playbook"] end subgraph P3["支柱三：参与式问责<br/>(Participatory Accountability)"] direction TB C1["引入外部与领域专家"] C2["消除中心化认知盲区"] C3["交付物: 咨询机制/透明度报告"] end P1 --> P2 --> P3 ``` - **原则清晰度（Principled Clarity）**：明确界定团队到底追求什么、抵制什么，定义成功的标准与不可接受的后果。在这一阶段必须使价值冲突显性化。其最佳实践包括：制定具有可操作性（Decisionable）的判定标准、明确“护栏（Guardrails，用于阻断）”与“目标（Targets，用于引导）”的区别，并公开记录已知的未知数（Known unknowns），绝不回避模糊性。 - **务实工具化（Pragmatic Instrumentalization）**：将抽象的价值观转化为产品工程体系中的测量指标、准入门禁（Review Gates）和操作手册。正如团队工程师常提醒她的：“无法测量的东西，就无法进入产品路线图（Roadmap）；而进不了路线图，就等于不存在。” - **参与式问责（Participatory Accountability）**：建立包含多元利益相关方与领域专家的真实反馈闭环。这并不意味着给予所有人一票否决权，而是为了打破核心团队的认知盲区。地域文化专家能发现常规审查遗漏的语境缺陷，受影响社群能在风险量化为指标前感知到下游危害，这既是道德保障，更是认知（Epistemic）上的纠偏机制。 ### [15:34](https://www.youtube.com/watch?v=53846JjFolY&t=934s) 测量维度与社会技术系统（Sociotechnical）视角 在谈到务实工具化时，Chloé 强调测量必须分层进行： 1. **模型层评估（Model-level Evals）**：评估基础能力、安全性、公平性与鲁棒性； 2. **系统层测试（System-level Tests）**：涵盖上下文窗口（Context Windows）、外部工具调用及系统集成； 3. **用户层遥测（User-level Telemetry）**：监控潜在危害、滥用行为、模型漂移（Drift）以及用户满意度。 但她着重发出警告：**并非所有重大风险都停留在模型层**。除了模型自身固有的风险，更多风险是在产品外包装（Wrapper）、部署场景、用户激励机制以及长期的交互行为中涌现出来的。AI 系统本质上是“社会技术系统（Sociotechnical System）”，如果测量策略只孤立地审视模型本身，必将遗漏大量关于风险和价值的关键维度。 ### [18:04](https://www.youtube.com/watch?v=53846JjFolY&t=1084s) 实践落地案例分析：AI 助手的个性化记忆权衡 为了展示伦理框架在实际场景中的应用，Chloé 引入了一个高度典型的案例：构建一款具有**个性化（Personalization）**功能的 AI 助手。 产品团队希望助手能够记住用户偏好、适应特定工作流，使交互不再机械重复。这能带来显而易见的用户价值，提升留存率与满意度。但伦理隐患也随之爆发： - 究竟哪些信息应该被记住？谁来做决定？ - 个性化是否会带来新的隐私泄露风险？ - 它是否会放大固有偏见，巩固不良行为模式？ - 它是否会导致算法操纵或非自愿诱导变得更加容易？ | 个性化谱系等级 | 机制特点 | 收益 (Pros) | 风险与代价 (Cons) | |:--- |:--- |:--- |:--- | | **完全无记忆 (Do Nothing)** | 无状态持久化，每次交互完全从头开始 | 隐私纯净安全 | 丧失大量用户价值与体验便利 | | **会话级记忆 (Session-scoped)** | 仅在单次会话内保留上下文记忆 | 风险可控，交互流畅 | 跨会话依然缺乏连续性 | | **选择性加入 (Opt-in Account Memory)** | 用户显式授权账户记忆 | 用户具备控制权与透明度 | 需权衡授权流程摩擦与数据暴露 | | **自适应个性化 (Adaptive Personalization)** | 动态根据交互调整响应风格 | 体验深度定制 | 透明度降低，潜在偏见累积 | | **默认全行为学习 (Full Behavioral by Default)** | 默认全面捕获并学习用户行为痕迹 | 体验高度自动化与无缝 | 隐私暴露极大，易在用户无感知下操纵决策 | Chloé 指出，个性化伦理绝不是非黑即白的“Yes or No”判断，而是一个连续的光谱（Spectrum）。沿光谱每向前迈一步，在获得更大效用的同时，必然伴随着系统不透明度、隐私暴露面以及在用户无感知情况下利用其行为模式的潜在风险。做出负责任决策的前提，是在试图解决权衡之前，先让所有的得失代价清晰可见（Legible）。 ## Video description Chloé Bakalar - Senior Research Associate, University College London; Former AI Ethics Lead, OpenAI & Former Chief Ethici Every AI system reflects decisions about what to optimize, how to measure success, which tradeoffs to accept and how a product should behave. At scale, even small choices can shape what becomes easier, harder or possible for millions of people. In this keynote, Chloé Bakalar draws on a decade of applied AI ethics experience in industry and academia to examine how companies can make the values behind those choices explicit, consistent and defensible, and carry them through measurement, evals, model behavior, product design and deployment. [Source](https://www.youtube.com/watch?v=53846JjFolY)

[25:54] Chloé Bakalar: If ethics doesn't show up in a roadmap, a metric, a playbook, it's not going to show up in the world. [2:39] Chloé Bakalar: Technology is inherently non-neutral; every algorithm and product choice embeds a set of underlying moral values, whether the creators intend it or not. [2:39] Chloé Bakalar: Ethics in AI cannot be solved with a single dogma, as diverse global communities require open-mindedness and an acknowledgment that there is 'no one right answer,' even if clear wrong answers exist. [2:39] Chloé Bakalar: At scale, there is no such thing as a small choice. [2:39] Chloé Bakalar: Inaction carries its own moral weight; opting not to decide defaults to a conservative bias that preserves the status quo. [2:39] Chloé Bakalar: Resource allocation, hiring, prioritization, and roadmaps are the ultimate reflection of an organization's true values. [15:14] Chloé Bakalar: If you cannot measure it, it's not going on the roadmap. [24:24] Chloé Bakalar: Good intentions are not a sustainable governance model. [24:24] Chloé Bakalar: Responsible AI is not merely a 'launch posture' but an ongoing 'maintenance posture' of iterative learning and tighter feedback loops.

[0:04] Hi, good afternoon everybody. I'm Chloe Valar  and I'm going to talk a little bit about what [0:09] it means to do AI ethics, especially to do it at  scale uh at an industry level. More specifically, [0:18] today is going to be about making ethics  operational in research and product cycles. [0:24] But before I start, just one  quick uh little legal disclaimer. [0:30] I'm speaking as myself in an academic  capacity here um based on my own experience, [0:36] my own words, my own views. My presentation  is not specific to OpenAI to Meta or any of [0:43] the other institutions that I've worked at. And so  obviously, you know, if you're looking for secret [0:48] proprietary information about ethics, all of  those deep dark ethics secrets, that's uh that's [0:54] not going to happen today or maybe later at the  happy hour. Um, thank you. A couple of laughs. Uh, [1:02] but instead of talking about sort of the deep  dark secrets, right, I'm going to be drawing on [1:07] my decade of industry and academic experience  defining and executing applied AI ethics [1:14] uh, within a large technology company that's  working at the cutting edge of AI. And so with [1:20] that disclaimer out of the way, I'm going to start  us off with what I consider kind of a hot take. [1:27] So, since AI ethics first came came to me about a  decade ago, um we've probably all gotten sick of [1:34] seeing these lists of AI principles uh and values  out there. Lists that say things like we care [1:41] about fairness and freedom and safety. And it's  like, yeah, of course you do. Who doesn't? Um, [1:50] but it's pretty easy to make those kinds of  broad claims in a listical about underspecified [1:55] values. I like fairness, you know, broadly  construed. What's hard is to ensure that those [2:02] values are actually represented faithfully in the  technologies that we make, that we build, that we [2:08] ship. And so, here's the hot take. Uh, AI that  doesn't ship with the build is simply theater. [2:17] Now, we might care deeply about the intentions  of the people and the organizations that develop [2:23] and deploy AI, but we also need to care about  what AI powered systems and products actually do [2:32] and which values they end up representing  and reflecting back into the world. [2:39] So, here I've listed out my six tenants  of responsible AI or AI ethics tenants. [2:45] These are the underlying scaffolding for how  I approach these spaces and how I encourage [2:50] others to approach them too. I'm going to run  through them fairly quickly. So first, the notion [2:57] that technology is not value neutral. You may  recognize that from the title of this keynote. Um [3:04] the idea here is that everything that we make  and ship is embedded with values. Whether we [3:10] do that on purpose or not, the things that  we build have a morality underneath them. [3:18] Second, uh there is no one right answer. There are  definitely some wrong ones and I'm like very happy [3:26] to talk about some of the wrong answers. Uh  but there's no one right way to do ethics. [3:33] Ethics needs to be approached with a  fairly open mind, especially in light [3:37] of the diverse and broad communities that  are served by a global technology like AI. [3:45] Third, there's no such thing as a small choice or  an insignificant choice. Given the scale at which [3:51] AI companies are operating, even seemingly  really minor decisions, right? Should we do [3:58] a a check? Should we do a thumbs up?  Should we do a heart? those kinds of things [4:03] can actually have enormous impact  when implemented at scale. Uh [4:08] and things don't always scale the  way that you think they're going to. [4:11] So it's important to also have some humility and  uh you know keep this in mind from the beginning. [4:19] The fourth tenant is that non-action is still  action. This one is especially important. [4:26] It's really easy and enticing for all of us  to think to tell ourselves that when we're [4:33] not doing something or we're avoiding a  decision, that means that we aren't acting. [4:39] And more importantly, that means we  aren't responsible for what comes next. [4:44] But choosing not to choose is definitely  still a choice. Uh, and it's a choice of moral [4:50] implications. Those moral implications tend to  come with a conservative bias towards whatever was [4:57] already happening. So you can't opt out of ethics.  Anything you do or don't do is an ethics choice. [5:05] Fifth, we can't have it all. So reasonable people  are going to disagree about what's right. That's [5:12] normal. That's healthy. That's good. Uh some  values are going to be mutually exclusive. [5:18] You can't have perfect individual freedom uh and  perfect safety at the same time. There are going [5:25] to be conflicts. Uh as I said, people disagree,  which means that there's no one set of human [5:32] values that's going to apply equally to everyone.  And that means that you cannot please everyone. [5:38] And in those cases, we're going  to have to make hard choices. [5:43] And again, opting out of choosing is is really not  an option. Uh, six, ethics is actually everyone's [5:51] responsibility. And I I say that as someone  whose job it is to to lead this work. Uh, [5:57] you shouldn't just count on people like me, right?  There should never just be one person who serves [6:03] as the moral center of an AI developer, an  AI company, and an AI provider. Ethics can [6:10] provide tools, guidance, perspectives, all sorts  of really useful stuff. uh but each of us needs [6:16] to be responsible for the choices that we make  and for their impacts on people and communities. [6:24] And then there's a seventh that I don't put up  here, but I I still think is important to mention, [6:29] and that's something like follow the money.  There's no better way for any organization [6:35] to show you what it values, what its values are,  than where it puts its resources. That means that [6:42] operations decisions, road mapping decisions,  prioritization decisions, hiring decisions, [6:49] these are also ethics decisions, even though  we don't necessarily think of them that way. [6:56] And so if it sounds like ethics is  basically everywhere, that's because it is. [7:03] Um, every component of the product development  life cycle includes or ought to include uh moments [7:10] for deliberate reflection and decision  making about ethical considerations. [7:16] So, please enjoy this like highly stylized version  of a a product development uh life cycle. Um, [7:25] so the the different points at which we  think ethics plugs in or ought to plug in, [7:31] right? It it's true when we decide what we're  going to research and how. So that first question, [7:38] what are we doing and should we be using AI to do  this? Um it's true when we decide what we're going [7:44] to goal against, what does success look like?  what we measure in eval what are our thresholds [7:51] how we decide to measure when we decide what  something counts as blocking or not blocking and [7:59] when we decide to do and how much we decide we're  going to care after a system has already launched [8:06] ethics doesn't end with the launch right we  are constantly iterating constantly learning [8:12] and nothing in that cycle  is a purely technical call, [8:18] right? There are value judgments about what we  optimize for, what risks we're going to tolerate, [8:24] and what interests are visible in the process.  And values get embedded at every stage. [8:32] Now, the additional challenge here is not that  ethics is everywhere, but that in real life, this [8:37] image is not at all static. I mean, first, this  is quite simplified, but also it's not static. [8:43] uh decisions are always happening in motion.  They're happening under significant time pressure [8:50] with incomplete information while the product  and the model and often the context is still [8:54] changing. So we're having to make these  calls in movement with a lot of thrash. [9:03] But if we have this like you know sort of  clean mapping of where ethics fits and plugs in [9:10] uh we might ask ourselves why is it so  hard to do this work right uh I'd say that [9:17] part of what makes it so hard or at least  three major pressures that are always exerting [9:22] themselves around the practice of AI ethics. The  first one is speed, right? Capabilities move fast. [9:30] Uh product cycles move really fast. Uh governance  can move quite slow. Ethics is actually capable of [9:38] moving really fast. Uh but often it only  comes in at the end at which point you [9:43] have to sort of start to revisit questions  which slows the slows the model down. Uh, [9:50] a second condition is uncertainty. And I apologize  for the now sort of sickeningly overused phrase [9:58] that we're we're constantly building the the plane  while we're flying it, but it's it's still true. [10:04] You know, we have plenty of unknown unknowns,  known unknowns, and a lot of judgment calls [10:10] that we have to make before the evidence is  complete for us to make ethics decisions. [10:17] And third, there are competing interests. While  it's nice to think that ethics might be the most [10:23] important view in a room, that's rarely the case,  right? Uh ethics matters, but there are a lot of [10:32] other things that matter too when you're building  and deploying an AI model or product. So for [10:39] example, not just technical interests but legal,  business, policy, comms and operational concerns [10:46] are all part of the same decision environment  in with in which ethics has to operate. [10:52] Sometimes these interests are going to align quite  nicely. Sometimes perhap often times they don't. [11:00] And that means that our goal can't be perfect  foresight, right? It has to be something closer to [11:06] decision making discipline that is sturdy enough  to operate under pressure, clear enough to guide [11:13] our choices, practical enough to survive contact  with reality, right? It can't be too idealized [11:21] and iterative iterative enough to to improve  over time. Remember, AI ethics is still a very [11:26] new field. It's about a decade old. Ethics is  one of the oldest fields of study at, you know, [11:33] several thousands of years old. So, we're still  learning what this is and how it how it applies. [11:41] Now, one way to create good habits that  enable us to manage these challenges [11:46] is to build them right into the way that we  work. Again, ethics not coming in at the end, [11:51] but integrated into our own work processes.  And this is what I call embedded ethics. [11:58] These three pillars uh are kind of a loose  framework for how to embed ethics in the [12:04] things that we build and the things that we ship.  So I'm going to very quickly outline them here and [12:09] then go through them each in a little bit more  depth. The first principle principle clarity uh [12:16] is about deciding what it is that we optimize for,  what it is we care about, what success looks like, [12:23] what it doesn't look like, right? What outcomes  we want to avoid. It's at this stage we want to [12:29] name and adjudicate tradeoffs. The second  pillar is pragmatic instrumentalization. [12:36] Here is where we translate these broad values like  freedom and safety into measurements measurements, [12:43] gates and playbooks. This is where we develop  evals that help us understand how the models are [12:49] doing. This is where we learn and we iterate. And  the third pillar is participatory accountability. [12:58] So here is where we develop and implement  governance models to ensure transparency [13:04] and openness. It's also where we build  feedback loops with our relevant stakeholders. [13:10] Now each of these three pillars  maps to concrete artifacts, [13:15] decision criteria and logs, metrics, eval incident  rbooks, consultation cadences and transparency [13:24] artifacts and all of these artifacts help  keep us honest through the decision-m process. [13:33] So let's start with principled  clarity pillar number one. [13:38] This again is about knowing what it is that you  stand for and which values you want to uphold. [13:43] That sounds really simple and basic, but it's  actually much harder than you think, especially [13:48] again when we're working with these global  technologies. And as I said, while while it's all [13:54] good and well to say that that, you know, we care  about broad values like safety and fairness, um [14:03] that doesn't do much good until you can actually  crisply define what it is that you mean by those [14:09] terms, right? Then figure out how are we going  to measure them, what benchmarks will we set, [14:17] and how do we make hard calls about these kinds  of values? like so is this going to be a launch [14:23] blocking concern? Would I trade off regressions  here against progress elsewhere? These are these [14:30] are tough conversations and tough decisions. But  there are a few best practices that can help. [14:37] For example, criteria needs to be decisionable. Uh  someone needs to be able to say yes or no today. [14:44] Uh you can think about guard rails  versus targets. So guardrails block [14:50] targets guide which one's appropriate given  a particular value in a particular context. [14:55] You can document known unknowns. Don't hide  from ambiguity. Make ambiguity very clear. [15:04] The point is that this kind of principle uh is  how we make ethics usable in an actual decision. [15:14] Now pillar two pragmatic in instrumentalization.  uh this is the point where ethics becomes [15:20] operational. It has to enter the machinery  of product development. So eval thresholds, [15:27] launch criteria, dashboards, review  gates and measurement is central to that. [15:34] If you can't measure something, it's  extremely difficult to govern it consistently [15:40] or as my colleagues always remind me, if you  cannot measure it, it's not going in the road map. [15:46] And if it doesn't go in the road map, does it  even matter? I don't know that the tree falls in [15:51] a forest, right? Uh this kind of measurement has  to happen at several levels. So model level evals [16:00] for capability, safety, fairness, robustness,  system level tests for context windows, uh [16:07] tools and integration, and user level telemetry  for harms, misuse, drift, and satisfaction. [16:15] But there's a really important caveat  or caution here. Not every meaningful [16:20] risk is a model level risk. Some risks  are going to live in the model for sure. [16:26] And others emerge from the product wrapper,  the deployment context, the user incentives, [16:32] or the way that the people interact with  the system over time. So in other words, [16:38] many of the things that we care about aren't just  at the model level, they're sociotechnical, right? [16:46] So if our measurement strategy only looks at the  model in isolation, we're going to miss a lot of [16:51] the things that we that we want to to focus  on both from a risk and a value standpoint. [17:00] The third pillar is uh participatory  accountability. So building real feedback [17:05] loops with the people who have the knowledge,  the experience, the understanding that the core [17:09] product team may lack. Participation doesn't mean  that every stakeholder gets a veto or gets to, [17:17] you know, impact every decision, but it does  mean recognizing that concentrated decision [17:22] making creates blind spots. Domain experts  may see risks that the product team misses. [17:30] Regional or cultural experts may spot context  failures that don't show up in a generic review. [17:36] Affected communities may understand downstream  harms long before they actually show up as [17:41] legible in metrics. So participation is  both an epistemic safeguard and a moral one. [17:49] Right? It helps us know more, helps us know  better, and helps us ensure that the people [17:55] most likely to bear the consequences of an  AI system are not invisible in the process. [18:04] But even if we get all of that perfect, right, we  still have to grapple with an inherent tension in [18:09] applied AI ethics. Again, you cannot have it all  in ethics. You certainly cannot have it all at [18:16] once. Some values are mutually exclusive. There's  no univers universal consensus on what is the good [18:24] and not all values translate directly into  different cultures or legal systems. These are [18:30] not problems that can be solved, right? But they  are conditions that can and need to be managed. [18:40] So, we're not going to find some kind of perfect  template to resolve these sorts of tensions, [18:46] but hard trade-offs get easier when you reason  about them and make them visible and specific. So, [18:53] instead of just vaguely talking about  ethics, we dig in like what are the [18:58] values that we're trying to protect? What  risk comes with that particular choice? [19:03] What mitigations might change the decision? And  what would we need to know in order to say yes [19:10] or no to something and act with integrity in doing  so? That's the kind of habit, the kind of decision [19:16] structure I want to encourage, right? Make the  trade-off legible before you try to resolve it. [19:24] Um, and to make all this real, I'm going to walk  or honestly like run you through a very stylized [19:33] hypothetical case so you can see what this kind  of AI ethics looks like in action. In this case, [19:38] we're going to look at personalization. So,  imagine that you have a team building an AI [19:43] assistant. Um, they want it to be more useful over  time. So to remember user preferences, adapt uh [19:50] pick up workflows that make the product  feel less repetitive, that kind of thing. [19:54] And this is a real good that brings real  value to users. Users often want tools [20:00] that feel less forgetful and less generic.  And personalization improves retention. It [20:06] improves satisfaction. It improves usefulness.  But the ethics questions pop up right away. [20:14] Uh what exactly gets remembered? Who decides?  Does personalization create new privacy risk? [20:21] Does it amplify bias or reinforce patterns  that we'd rather not have entrenched? [20:27] Could it make certain kinds of influence  or manipulation easier or more likely? [20:32] And so, I like this example because the upside is  definitely real, the demand is definitely real, [20:39] and the risks are real, too. And once we start  to tease out the problem, it becomes clear that [20:46] it's not really a yes and no decision when it  comes to the ethics of personalization, right? We [20:53] have to think in terms of an opportunity space or  a spectrum of options. At one end of the spectrum, [20:59] there's doing nothing, right? No persistence,  no adaptation. Every interaction starts fresh. [21:06] This is clean from a privacy perspective. Uh  but it leaves a lot of user value on the table. [21:12] And then as you move towards more personalization,  you can see session scoped uh an opt-in account [21:18] memory adaptive personalization all the  way to full behavioral learning by default. [21:25] And the important thing really to keep in mind  along the spectrum is that every step you move [21:32] buys you something and costs you something, right?  So more personalization gets you more usefulness. [21:39] It can also mean more opacity, more  privacy exposure, and more potential [21:44] for the system to act on patterns that it that  the users don't realize they're creating. So, [21:50] how do you make a decision an an ethicsinformed  decision about this kind of scenario? Uh, well, [21:58] you have to hold several different kinds of inputs  together. The first is is the mixed perspectives, [22:04] right? So, consider business interest.  Does this materially improve the product? [22:08] legal and regulatory concerns, um policy issues  around the kind of precedent that we're setting. [22:16] Uh and then second, we have to look at the actual  trade-offs, right? Helpfulness versus privacy, [22:22] adaptivity versus predictability, crosscontext  convenience versus context leakage uh and bias. [22:30] And third, and maybe most importantly, we have  to look at the data, right? What do the pilots [22:35] show us? What are users actually asking for?  What does the safety review tell us? A a real [22:42] defensible decision that's ethics informed has  to consider all of these kinds of inputs at once. [22:49] Uh again, I think the data here is particularly  important. Uh if we're making datadriven, [22:56] you know, defensible decisions, uh we need to look  at what it actually says. So in this hypothetical [23:03] pilot, let's say the the results tell us that  personalization is of great value to users. [23:11] Uh the safety review tells us something like  more automated preference inference isn't benign. [23:17] It can in fact reinforce stereotypes or  lock people into patterns that the system [23:22] shouldn't be confidently inferring. And  let's say the user research tells us that [23:27] people really want personalization but  they also want some control over it. [23:32] So the data doesn't give us the decision but it  gives us really important inputs for making it. [23:39] So where do we land uh given the data  we have given the interest that we have [23:46] uh the decision is not just personalization  yes or no. What kind of personalization is [23:51] a better characterization of the question? Uh,  and given this hypothetical case, as I said, we [23:56] might land on something more like opt-in account  memory, default off with meaningful user control [24:02] and clear data boundaries, maybe rolled out in a  staged roll out with with limited access to start. [24:10] Uh, now these guardrails are not merely  window dressing. They're what makes this [24:14] decision morally different or a more extractive  or opaque version of the same product idea. [24:24] All right. So, stepping  back from the hypothetical, [24:27] uh, as you can see, it's not easy  or simple to embed ethics into AI. [24:32] Even when you can agree about which values  you care about, implementing them is never [24:36] easy and never straightforward. Uh, but if  we aren't going to be perfect, and we're not, [24:42] can we at least get ourselves to something good  enough? And what would good enough look like? I [24:47] think about success here in terms of outcomes and  processes. So both doing good and doing good well. [24:54] So first talking about outcomes, right? Ethics  should help us define what success actually looks [24:59] like. A system can uh be impressive and still  fail morally if it undermines or fails to promote [25:08] human flourishing. The second is non-harm.  Right? There's no zerorisk path with AI. Um, [25:16] but there's a difference between acknowledging  risks and normalizing avoidable harm. [25:21] And the third is iterative. Responsible  work isn't just a launch posture. It's a [25:26] maintenance posture. But you revisit,  you learn, you tighten the loop. [25:31] Processes are also part of the equation. It's not  enough to just do good. You have to do it well. [25:38] Uh, good intentions are not a  sustainable governance model. Um, [25:46] we need to develop processes that are robust and  strong, transparent, dis defensible and adaptive. [25:54] All right. So, for my final hot take, I'm going to  take a take a note from Spider-Man who just made [26:00] it back into movie theaters. Right? With with  great power comes great responsibility. Right? [26:06] It sounds trit at this point, but it's still true.  We're not going to get every call right. We're not [26:12] going to be able to foresee every misuse, every  failure mode or second order effect. Uh the [26:19] failure to be perfect and uncertainty  is not permission to disengage. [26:24] It's the reason to build decision systems that  are principled uh instrumented and accountable. [26:31] So we'll never get it perfect. We'll never  get it exactly right, but we have to try. Um, [26:37] and if if ethics doesn't show up in a road map, a  metric, a playbook, it's not going to show up in [26:43] the world, right? The question isn't only what AI  can do. It can do a lot. It's what we're willing [26:50] to build, what we're willing to con constrain, and  whether we're serious enough to make those choices [26:57] real in practice. And those are the ideas I'd  like to leave you with. Thank you so much.