AI Ethics Panels Warn Open-Source Models Needed to Curb Big Tech Power

AI伦理小组警告：需借助开源模型遏制科技巨头权力 当前AI系统缺乏道德主体性，所有伦理责任均由开发者承担，而集中的企业保密机制则对公共治理构成威胁。 专家小组深入探讨了将伦理融入人工智能系统的核心哲学困境与现实挑战。发言者讨论了为何当前AI模型缺乏真正的道德主体性，强调伦理问责目前仍在于人类开发者与机构。他们还重点阐述了跨职能治理、本土化与普适伦理标准之间的权衡取舍、数据质量对能源效率的影响，以及在保障人类自主性与批判性思维的同时应对劳动力替代问题的紧迫需求。 哲学家 Thomas Hofweber 强调，现阶段的道德评估严格适用于人类开发者与部署者，当前的 AI 系统缺乏真正的道德代理能力、自主性以及对自身目标的独立反思机制。 (0:35) 企业在推进 AI 伦理落地时需依托跨部门协同治理，推动内部决策从单一的法律合规考量（“我们能做吗？”）转向企业价值观与道德维度的审视（“我们该做吗？”）。 (25:06) Toby Carlson 指出数据纯净度直接决定了算力开销，准确干净的数据集能大幅减少系统的计算损耗，从而有效降低 AI 运行的能源消耗与碳足迹。 (33:29) 针对核心前沿模型过度集中于少数不透明科技企业的现状，与会专家警告过度依赖企业自觉将带来治理盲区，亟需引入开源架构与强制性的透明度机制以捍卫公众利益。 (40:14) 面对 AI 带来的劳动力结构冲击与技术演进，论坛总结指出未来的核心伦理底线是捍卫人类的自主权，并重点培养独立批判性思维以防被动依赖算法结论。 (41:21)

AI ethics must advance beyond basic compliance checklists to address foundational moral questions regarding agency, universal values, and accountability. According to panelists at Ai4, current ethical responsibility rests entirely on human designers, deployers, and governing institutions rather than the automated software itself. Agency, Accountability, and Philosophical Foundations (0:04) Discussions of AI ethics traditionally emphasize procedural safeguards such as bias mitigation, data privacy, and transparency. Bioethicist Cécile Bensimon argues that organizations must instead tackle fundamental philosophical dilemmas, defining the nature of societal benefit, harm, and justice to safeguard institutional trust. In evaluating these systems, UNC-Chapel Hill philosophy professor Thomas Hofweber points out that ethical appraisal currently applies only to human developers and operators. Existing models rely on conditional logic and lack true moral agency, which would require higher-order cognitive capacities such as independently evaluating and revising their own goals. Emotions, Universal Morals, and Cultural Adaptation The panel examined the relationship between instinct and rational analysis in ethical evaluation. Hofweber noted that while human intuition and gut reactions—termed the 'yuck factor' in bioethics—offer valuable initial signals, they must be tested with rational scrutiny, complicating efforts to build machines that could mirror moral sentiments. Panelist Bob Wollheim clarified that emotional awareness should stay strictly on the human side of decision-making rather than being engineered into algorithms. Regarding global standards, the speakers debated universal moral frameworks versus localized alignment. Toby Carlson observed that agentic AI can support localized regional rules, yet expressed skepticism that a universal baseline will emerge due to market incentives for proprietary, closed algorithms. Hofweber, while acknowledging the operational convenience of local customization, maintained that objective moral principles apply universally. Operational Governance and Cross-Functional Leadership At the enterprise level, ethical deployment requires expanding past legal permission toward value-aligned responsibility. Noreen Fierro, Chief Ethics and Compliance Officer at Principal Financial Group, described training 20,000 employees and reviewing use cases from non-technical citizen engineers by shifting focus from 'Can we do this?' to 'Should we do this?' Because consumers rarely scrutinize fine-print terms, Fierro rejected the idea of a single ethics officer serving as the sole approver. Instead, organizations require cross-functional oversight reaching up to executive leadership to account for complex factors such as bias, brand identity, and the heavy energy grid demands of AI compute. Energy Footprints, Open Models, and Power Concentration Data integrity directly correlates with computational efficiency and environmental impact. Carlson explained that computing systems burn significantly more power attempting to distill and clean inaccurate or convoluted data, suggesting energy consumption could conceptually serve as a stress test for dataset purity. Meanwhile, Hofweber warned against the consolidation of advanced AI within a small group of opaque corporations. With external observers unable to independently verify model architectures, parameter sizes, or training corpora, he advocated for open-source frameworks and mandatory transparency to counter concentrated commercial control. Workforce Disruption and Human Agency Panelists identified economic displacement and the preparation of future generations as urgent challenges. Wollheim highlighted the threat of widening socioeconomic inequality as automation disrupts traditional employment. In response, Fierro urged organizations and educators to proactively restructure curricula and career development pipelines. Carlson argued for precise, surgical retraining ('X-Acto knife') over crude corporate layoffs ('sledgehammer') and described designing systems like SovrenAI to ensure future generations critically analyze raw data rather than passively consuming automated outputs. The panel concluded that navigating AI's ethical future requires preserving human agency and cultivating critical thinking. As models grow increasingly sophisticated and proprietary, the central unresolved challenge remains establishing independent oversight and education before economic displacement and concentrated corporate power permanently outpace governance.

主持人 Cécile Bensimon 与多位学者及业界专家指出，人工智能伦理不能仅停留在风险防范与合规层面的程序性防护，而必须直面道德主体性、正义与人类自主权等深层哲学根基。在当前技术现实下，道德责任完全在于人类开发者与部署机构，组织与社会必须通过全员价值治理、审慎的劳动力转型以及对批判性思维的保护，应对权力集中和自动化带来的深远冲击。 道德主体的归属与机器智能的界限 (0:35) 北卡罗来纳大学教堂山分校（UNC-Chapel Hill）哲学与数据科学教授 Thomas Hofweber 强调，在哲学评价中，道德责任仅适用于具备道德行动能力的行动者。现阶段的人工智能模型与自主代理机制过于简单，其本质仍受制于基础的条件逻辑，并不具备真正的道德主体资格。因此，现行系统的技术偏差与应用后果无论是好是坏，伦理上的赞扬或谴责都只能完全由人类开发者与部署机构承担。若要使 AI 成为受伦理评价的主体，系统必须展现出超越硬编码指令的高度自主性，尤其是具备独立审视并修改自身目标的高阶认知能力。 关于人类情感与理性判断在伦理决策中的作用，Hofweber 指出直觉反应虽具有参考意义但也可能带来误导，必须经过底层理性理由的检验。Bob Wollheim 则澄清，伦理治理的核心并非赋予机器情感，而是要求人类决策团队结合情感感知与理性分析，借助协作对话过滤掉个人的主观偏见。Cécile Bensimon 援引生物伦理学中的“恶心因子”（yuck factor）说明，人类对潜在道德错误的本能反感难以用纯逻辑形式化，因此必须将这种感知与反思严格保留在人类监督的一侧。 组织治理实务：从合规审查走向价值驱动 信安金融集团（Principal Financial Group）首席伦理与合规官 Noreen Fierro 提出，企业作为 AI 技术的应用方与采购方，必须推动内部治理逻辑从法律层面的“我们能否这样做”转向伦理层面的“我们是否应该这样做”。随着该集团对全部 20,000 名员工进行培训并鼓励非技术业务人员提出应用构想，伦理治理不能依赖隐藏在数十页开户条款中的细则免责，而必须真实反映企业的核心品牌价值观。 Fierro 强调，伦理裁决绝不能由单一部门或合规官个人垄断，必须建立覆盖至首席执行官（CEO）层级的多元跨职能委员会，以平衡模型准确性与透明度、个性化与隐私保护之间的矛盾，并防范环境算力成本等认知盲区。在金融与医疗等强监管领域，机构必须构建稳健的后端验证与测试控制机制，以防范算法偏见，并向不断演进的监管机构持续证明自身的合规性与价值契合度。 能耗成本、数据纯度与普适伦理的现实张力 在技术运行与环境影响方面，Toby Carlson 指出数据质量直接决定了计算系统的能耗负担。正如维持谎言比讲出真相耗费更多精力，计算系统在清理、提炼和调和不准确或混乱的数据集时，需要消耗显著更多的算力和电能。Carlson 提出一种理论框架，建议将系统消耗的额外能源视为一种机器层面的“感知”或“压力测试”指标，借此量化输入数据的纯度，并强调提升数据准确性对于降低企业运营延迟与碳足迹的双重价值。 面对全球化部署，Cécile Bensimon 提出了不同国家在公平、数据隐私和知识产权上的文化差异问题。Carlson 指出，尽管现代代理式 AI 架构能够灵活支持符合特定区域法规的本地化规则集，但商业竞争、企业贪婪以及对市场主导地位的争夺，促使科技巨头将算法黑箱私有化，阻碍了全球普适道德基线的形成。Hofweber 虽在哲学上倾向于客观普遍主义，认为存在普适的道德真理，但也承认在具体实施中支持普适准则还是文化相对主义，取决于底层哲学立场的不同。 权力垄断困境、劳动力重构与人类自主性 Thomas Hofweber 对少数大型科技公司垄断 AI 核心技术表示高度担忧。由于外部观察者无法独立验证前沿模型的训练数据构成、架构、参数规模及评估方案，公众被迫完全依赖企业的单方声明。尽管开源模型与透明度指令是缓解封闭垄断的潜在路径，但最具影响力的前沿模型仍被闭源控制，依赖企业善意构成了严峻的社会治理风险。与此同时，Bob Wollheim 与 Noreen Fierro 警告，AI 导致的传统就业结构动摇将加剧社会经济不平等，企业与政策制定者必须像使用“手术刀”般精准重训员工，而非粗暴裁撤，并承担起针对下一代重塑教育体系的责任。 Toby Carlson 以创办 SovrenAI 为例，呼吁建立能让下一代直接交互并独立分析数据的环境，防止年轻人被动接受预设答案。Cécile Bensimon 在总结中强调，应对日益复杂的 AI 系统的核心伦理底线，在于维护人类的自主行动能力与独立决策权。AI 系统的设计必须以人为中心，全社会必须持续培养人类个体的批判性思维，确保人类始终具备评估、监督与引导自动化系统的能力，而非退化为算法输出的被动接受者。 本次研讨表明，AI 伦理不仅是工程层面的安全规范，更是关乎权力、正义与教育的全局挑战。然而，在商业竞争激励与专有黑箱的阻隔下，业界如何在缺乏透明度验证与监管标准的情况下打破科技巨头的权力垄断，并在自动化浪潮中平稳重构就业与教育体制，仍是各方未能达成共识的核心未决议题。

## Background 本内容由 YouTube 频道 Ai4 于 2025 年 8 月 29 日发布，记录了一场题为《何以使 AI 合乎伦理？探索 AI 伦理的哲学根基》的圆桌研讨会。会议汇聚了来自学界、企业合规部门与技术创新领域的专家，旨在深入剖析公平性、透明度与道德责任等哲学命题，探讨如何将伦理原则嵌入 AI 系统并平衡文化差异与现实权衡。 ## TL;DR 本场论坛超越了常规的合规修补视角，将 AI 伦理提升至哲学与本体论层面进行深层审视。专家们首先厘清了现阶段评估 AI 伦理的核心在于人类设计者与部署者，而非尚无道德自主性的算法本身；随后深入探讨了企业如何跨越从“我们能做吗”到“我们应该做吗”的治理门槛，将伦理内嵌于业务与架构核心。讨论还直面了普世道德与文化相对主义的张力、数据不纯导致的隐形算力能耗成本，以及寡头垄断与未来劳动力重组等紧迫挑战，最终呼吁在技术狂飙中始终守护人类主体性与批判性思维。 ## Atmosphere & style 整场研讨会充满着高度理性、严谨思辨却又不失人文关怀的知性氛围。主持人的生命伦理学框架引导与哲学家、企业高管之间的观点交锋流畅自然，台上既有针对“自主性”与“客观道德”的抽象概念解构，也有结合真实儿童夏令营和跨国企业治理的鲜活案例。现场讨论在幽默坦诚与对未来社会剧变的深切忧患中层层递进，带给听众一种既具智识启发又发人深省的思想沉浸感。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=FFwil83UjaM&t=4s) AI 伦理研讨亟需实现范式转移，不能仅停留在算法偏见、隐私保护或事后治理等合规防范层面，而必须退后一步探寻其背后的哲学根基与核心价值。 - [03:35](https://www.youtube.com/watch?v=FFwil83UjaM&t=215s) 当前评估 AI 伦理的核心在于审视背后设计与部署的人类，现阶段智能体仍属规则驱动，缺乏反思与修改自身目标的道德自主性，尚不具备承担道德责任的哲学基础。 - [08:04](https://www.youtube.com/watch?v=FFwil83UjaM&t=484s) 工程层面的条件分支逻辑（如遇到 A 执行 X）属于伪自主性，真正的伦理级自主性要求系统具备对预设目标进行独立反思、重构与修正的高阶能力。 - [09:21](https://www.youtube.com/watch?v=FFwil83UjaM&t=561s) 软件代码具有前所未有的规模化放大效应，微小的伦理偏见会被迅速放大至数十亿用户，因此伦理绝不能作为边缘装饰品，必须内嵌于核心商业逻辑中。 - [11:41](https://www.youtube.com/watch?v=FFwil83UjaM&t=701s) 以人为中心的系统设计要求系统尊重人类高于尊重数据集，以生命、尊严与不伤害原则作为底线，在过滤提炼事实后将最终决策权交还人类。 - [16:38](https://www.youtube.com/watch?v=FFwil83UjaM&t=998s) 人类直觉与道德感受（如生物伦理中的“恶心因子”）虽可作为伦理预警信号，但易受成长偏见干扰，最终判定仍需依赖严密的理性审查与推演。 - [21:04](https://www.youtube.com/watch?v=FFwil83UjaM&t=1264s) 尽管多智能体技术可针对本地法规配置规则集，但在全球范围内建立统一的客观普世道德标准，依然面临巨大的商业竞争利益博弈与文化相对主义阻碍。 - [26:00](https://www.youtube.com/watch?v=FFwil83UjaM&t=1560s) 企业在推进技术创新时必须建立两阶段伦理治理漏斗：在确认合规可行的“我们能做吗”之后，更关键的是跨职能审视契合价值观与声誉的“我们应该做吗”。 - [31:40](https://www.youtube.com/watch?v=FFwil83UjaM&t=1900s) 重大伦理决策绝不能由伦理合规官或任何单一技术人员一言堂裁决，必须由跨职能高管委员会共同分担责任，以避免盲目狂奔损害品牌与客户信任。 - [33:29](https://www.youtube.com/watch?v=FFwil83UjaM&t=2009s) 处理虚假与低质偏差数据所消耗的计算算力和物理电能呈指数级增加，数据纯度直接关联环境可持续性，能耗可作为衡量数据质量与系统负荷的物理指标。 - [36:20](https://www.youtube.com/watch?v=FFwil83UjaM&t=2180s) 未来最紧迫的社会伦理挑战之一在于防范剧烈技术变革中弱势劳动者被时代抛弃，企业在人才结构调整时应当使用精准的人本“手术刀”而非粗暴的“大铁锤”。 - [38:43](https://www.youtube.com/watch?v=FFwil83UjaM&t=2323s) 顶尖 AI 模型的算力与技术高度集中在极少数闭源巨头手中，评测与训练数据的不透明导致社会公众产生巨大信任赤字与权力垄断风险。 - [41:21](https://www.youtube.com/watch?v=FFwil83UjaM&t=2481s) 面对不可调和的现实权衡，AI 治理的根本在于坚决捍卫人类的主体决策权，并持续培育下一代独立审辨的批判性思维，避免沦为被技术投喂的被动接受者。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=FFwil83UjaM&t=4s) 伦理讨论的范式转移：从合规防范走向哲学根基 会议主持人 Cécile Bensimon 在开场时向全场致谢，笑称现场人数超出了预期，会议室坐得满满当当。作为一名长期深耕临床医学与科研伦理的生命伦理学者，她指出，当下业内关于 AI 伦理的研讨往往被焦虑所主导：人们聚焦于算法偏见、歧视、隐私侵犯、全方位监控、知情同意，以及学术研究中关于真实性与学术诚信的危机；在应对思路上，则普遍侧重于程序性与治理框架，比如责任追溯、透明度机制以及针对安全风险的安全护栏（safeguards）。然而，Cécile 强调，尽管 AI 系统带来了迫在眉睫的现实挑战，但这项技术本身并非“本质上有问题”（inherently problematic）。要真正负责任地整合技术，就不能仅仅停留在打补丁式的合规层面，而必须退后一步，向内探寻指引人类行为的核心价值观。 正是这种追问构成了本场研讨会的核心议题——AI 伦理的哲学根基。在 Cécile 看来，AI 直指伦理学中最本质的命题：究竟什么是“行善”？是效用主义层面的最大化效益与最小化伤害，还是道义层面的恪守正义？什么是“善”本身？是由行动者的意图、最终的客观结果还是人际关系所界定？进而，伤害的本质与范围是什么，又该由谁来下定义？更关键的是，在现代社会中，信任构成了政治与道德合法性的基石，那么人类是否具有培育并维系对技术信任的伦理义务？随着 AI 越来越深地介入并调解人类的人际纽带、机构运转与决策机制，“伦理 AI”这一流行概念在不同背景、学科与生命经验的人眼中呈现出截然不同的内涵。为此，她向嘉宾抛出了首个核心问题：究竟何为真正的“伦理 AI”，我们又该如何定义追求伦理 AI 的成功？ - **当前主流 AI 伦理关注点**：偏见与歧视、隐私侵犯与监控机制、知情同意、学术真实性、治理结构、透明度与安全护栏。 - **哲学层面的底层追问**：行善的定义（效益最大化 vs. 正义原则）、善的评判标准（意图 vs. 结果 vs. 关系）、伤害的范围界定权，以及信任作为政治与道德合法性基石的深层价值。 --- ### [03:35](https://www.youtube.com/watch?v=FFwil83UjaM&t=215s) 道德主体还是人工工具：评估 AI 本身还是评估人类设计者 来自北卡罗来纳大学教堂山分校（UNC Chapel Hill）的哲学与数据科学教授 Thomas Hofweber 首先作了自我介绍。他幽默地提到自己的研究范围涵盖 AI 对民主的影响，甚至包括“如何通过攻击金融预测模型搞垮股市——当然纯粹是出于学术研究”。面对主持人抛出的问题，Thomas 迅速切入哲学的核心剖析，提出探讨“伦理 AI”时必须明确一个关键分水岭：我们究竟是在将 AI 本身视作一个“道德行动者”（ethical agent）来进行善恶评价，还是在评价背后“设计或部署”该 AI 系统的人类？ ```mermaid flowchart TD accTitle: AI伦理评估的两种路径与现状 subgraph S1["路径一：评估 AI 自身（道德主体）"] A["评估 AI 系统本身行为"] --> B{"是否具备真正的道德自主性？"} B -- 现阶段：规则驱动/目标修改能力缺失 --> C["不可行：当前系统过于简陋，无法承担道德责任或赞誉"] B -- 未来推演：具备目标反思与独立意图 --> D["理论上可行：可作为道德代理人进行伦理裁决"] end subgraph S2["路径二：评估人类行动者（现阶段核心）"] E["设计者 / 研发人员"] --> G["系统架构与训练数据考量：消除偏见与潜在陷阱"] F["部署者 / 商业机构"] --> H["应用目的与社会落地权衡：导致正向善果还是现实危害"] end ``` Thomas 直言，就目前的现实技术而言，只有第二种路径才真正具有现实意义。当下的 AI 系统（即使是最前沿的智能体 agent）在逻辑与认知层面依然过于简陋，根本算不上真正的道德主体。人们通常只对拥有自主行为能力、能承担责任的人或机构进行伦理责难或道德赞扬；而现阶段的 AI，其设计是否包含偏差、训练数据是否留有隐患、部署用于特定场景后究竟是造福社会还是滋生危害，所有这些伦理考量的归宿依然是研发与部署它们的人类。至于未来的系统需要增添何种维度的复杂性才足以真正成为道德评价的对象，在哲学界本身就是一个未解之谜——正如人类对非人类动物是否具有道德属性尚无共识一样，给机器赋予道德评估资格同样缺乏明晰的边界。 谈及如何定义伦理 AI 的“成功”，Thomas 指出这取决于社会的道德雄心：最基础的伦理成功可以定义为“底线防御”，即系统不作恶、保持伦理中立，设计者与部署者只要避开了有害陷阱就算达标；而更高层次的追求，则是系统被积极地设计与部署去“行善”（positively do good）。 - **伦理评估的两大维度**：评估 AI 工具本身（系统自身具备道德代理权） vs. 评估背后的人类（设计意图、训练数据筛选、具体部署场景）。 - **当前技术局限**：现有智能体仍属于机械执行指令，缺乏自主承担道德责任（praiseworthy or blameworthy）的哲学基础。 - **伦理成功的双重标准**：消极防御（不作恶、无伦理缺陷）与积极进取（主动增进社会福祉）。 --- ### [08:04](https://www.youtube.com/watch?v=FFwil83UjaM&t=484s) 自主性辩证：从条件分支代码到反思性目标构建 针对 Thomas 提及的“自主性”（autonomy），Cécile 敏锐地指出这里存在一个有趣的视角反转：在传统 AI 治理与伦理准则中，人们讨论“自主性”通常是指如何保护人类自身的代理权与自主决定权不被算法侵蚀；而 Thomas 却提出了一个相反的方向——探讨赋予 AI 系统自主性。Thomas 随即顺着这个话题展开了更深层的辨析：一个能被进行伦理评估的 AI 智能体，其所需的“自主性”绝非工程代码里常见的条件判断。 Thomas 用一个生动的场景解释道：如果你派出一个小型 AI 智能体外出抓取信息，并给它设定指令“如果遇到情况 A 就执行动作 X，如果遇到情况 B 就执行动作 Y”，在表面上看，这个系统确实根据环境输入做出了应对决策。但在哲学层面上，这完全不是能够承受伦理评价的自主性。这个小智能体仅仅是在一板一眼地机械执行预设逻辑，谈不上任何自由意志或道德考量。他认为，若要让一个系统跨越至能够接受伦理裁决的门槛，它必须具备更深层的复杂性——例如能够“反思自身的目标、自主修改自身的目标”（reflecting on your own goals and modifying your own goals）。而在当前的人工智能技术体系中，这种高阶的复杂自主性根本尚不存在。 - **伪自主性（工程级）**：基于预设规则与环境输入的逻辑分支（“若 A 则 X，若 B 则 Y”），本质上是指令的精准投递，不具备道德主体资格。 - **真自主性（哲学/伦理级）**：能够对预设目标进行反思、重构与自发修正，具备独立于人类指令的复杂动机与责任承担能力。 --- ### [09:21](https://www.youtube.com/watch?v=FFwil83UjaM&t=561s) 从边缘合规走向业务核心：在软件规模化中嵌入价值观 随后，主持人转向 CI&T 业务技术服务公司合伙人兼执行副总裁 Bob Wollheim，请他分享负责任使用 AI 的关键考量。Bob 首先介绍了自己的背景，他长期从事企业数字化转型与领导力咨询，并曾出版过关于人与业务协同编排的著作。在回答严肃的商业伦理之前，他动情地分享了一个不久前发生的真实故事：几个月前他萌生了一个构想，就在上周，他们团队在加州圣莫尼卡海滩为孩子们举办了一场“创意与 AI 夏令营”，他 11 岁大的孩子 Nico 也参与其中。令人惊叹的是，导师们完全不需要搬出“伦理”（ethics）或“偏见”（bias）这类抽象深奥的词汇，就能通过具象的交互体验，让孩子们发自内心地理解什么是偏见与伦理抉择。 ``` 传统企业架构： [ 核心业务系统 / 代码逻辑 ] ======> 放大至数亿用户 ^ | (边缘补丁 / 事后合规) [ 伦理与合规委员会 ] 现代 AI 驱动架构（Bob 提倡的核心模式）： ┌─────────────────────────────────────────┐ │ 核心商业逻辑与底层软件架构 │ │ ★ 内嵌人类伦理决策与防偏见机制 ★ │ └─────────────────────────────────────────┘ │ ▼ 随软件无限扩展 [ 全球数十亿受众 ] ``` 以此为切入点，Bob 指出了企业在负责任使用 AI 时必须跨越的首要门槛：伦理绝不能再被当成挂在公司业务边缘、由法务合规部门偶尔过问的“邻近装饰品”。在传统时代，业务逻辑与道德讨论常常脱节；但今天的人工智能软件有着前所未有的放大效应，工程师在编写底层代码、设计交互规则时所做出的每一个微小的伦理或不道德抉择，都会被软件以极高的效率“嵌入并规模化至全球数十亿用户”。因此，他提出的首要标准就是将伦理深植于所有商业运作的核心。AI 的爆发不仅是一场技术洗牌，更是向全人类发出的深刻拷问：在盲目探讨机器是否合乎伦理之前，我们必须先直面根本——“我们自身的组织，在本质上是否合乎伦理？” - **圣莫尼卡海滩 AI 夏令营案例**：通过寓教于乐的具象方式向 11 岁儿童传递偏见与伦理概念，证明伦理思维无需晦涩术语即可被普及认知。 - **伦理议题的定位重构**：从企业边缘的辅助性、附属性议题（adjacent thing），转变为决定企业生死存亡的“底层商业核心”。 - **规模化效应的伦理风险**：软件代码直接承载价值观，任何伦理缺陷都将被技术杠杆放大至数十亿人规模。 - **反求诸己的伦理发问**：在审查 AI 算法之前，人类企业与组织必须首先反思自身的治理模式与业务意图是否正当。 ### [11:41](https://www.youtube.com/watch?v=FFwil83UjaM&t=701s) 伦理讨论的必要性与“以人为中心”的 AI 理念 Bob Wollheim 强调，面对技术扩张，组织首先必须正视并公开讨论伦理问题。当前 AI 的扩展规模与速度前所未有，很多嵌入组织系统的东西可能完全合乎伦理，也可能完全不道德，但如果缺乏反思就可能完全被忽视。他调侃道，在场的所有参会者相比于没有参加讨论的数千人，应当成为把这种反思与讨论带回各自组织的倡导者。 ```mermaid flowchart TD accTitle: Human-Centric AI Architecture accDescription: Flowchart illustrating the distinction between data-centric and human-centric AI design principles. subgraph DC["数据中心化模型 (Data-Centric)"] D1["海量原始数据集 (60-80% 准确率)"] --> D2["未经严格伦理过滤"] D2 --> D3["可能放大偏见与幻觉"] end subgraph HC["以人为中心模型 (Human-Centric)"] H1["人类普世核心价值<br/>(生命、尊严、不伤害原则)"] --> H2["事实数据过滤与提炼"] H2 --> H3["辅助人类自主决策"] end ``` 接续此话题，SovrenAI 总裁 Toby Carlson 分享了“以人为中心”和“生命神圣性至上”（Sanctity of Life First）的系统设计理念。他指出，计算机本身只是靠能源运转的纯净“大脑”，造成偏见的是人类输入与提取的数据。 - **核心价值共识**：SovrenAI 在系统内核中预设人类广泛认同的价值观——生命、爱、尊严、自尊以及对彼此的尊重。 - **数据与人类的优先级**：AI 必须尊重人类高于尊重数据集。面对准确率可能只有 60% 至 80% 的复杂数据，系统应当过滤并提炼出客观事实，最终将自主权交还给个体做决定。 - **不伤害原则**：从全球视角来看，最基础的底线是“不互相伤害、不互相杀戮”。只要将这些根基写入 AI，问题就不在于机器，而在于人类自身的抉择。 - **教育实践案例**：Bob 补充了此前给 9 岁和 11 岁儿童讲授 AI 的经历，孩子们无需复杂的伦理理论，单凭直觉与感受（Feeling）就能本能地区分什么是对的、什么是错的。 --- ### [16:38](https://www.youtube.com/watch?v=FFwil83UjaM&t=998s) 理性推演、道德情感与“恶心因子”（Yuck Factor） Cécile Bensimon 针对感受与直觉提出了不同视角的补充。在生物伦理学（Bioethics）中，随着情境复杂度的上升，往往面临的是“对与对之间的抉择”（Right vs. Right Dilemma），并没有非黑即白的对错之分，仅靠直觉极难辨别。她同时观察到，尽管大会许多演讲并未冠以“伦理”之名，但大家在讨论实际应用与复杂挑战时，已经在实质性地探讨伦理考量。 Thomas Hofweber 教授从哲学角度对“道德依赖感受还是理性”提出了深刻质疑。他指出，如果伦理本质上归结为感受，那么构建伦理 AI 的难度将呈指数级上升，因为目前的 AI 系统充其量只能具备推理与思考能力，而不具备人类的情感体验。 - **直觉的欺骗性**：人类的情感与直觉往往受到成长背景和当下环境的偏见影响，直觉觉得“不对劲”的事情，在理性审视下可能完全合理。 - **理性的最终裁判权**：人类需要通过理性思考去审查“为什么会产生这种感受”，从而决定应当顺从还是抵制某种直觉感受。 - **人机分工定位**：Bob 随即澄清，他并非主张给机器赋予情感，而是呼吁人类在理性分析的同时，协作性地加入感受维度，以修正单一个体的偏见。 - **生物伦理中的“恶心因子”**：Cécile 提到生物伦理学中的经典概念“恶心因子”（Yuck Factor），即当人对某项技术或行为产生直觉上的不适时，这种难以定义的直觉往往成为伦理预警信号，但最终仍需依靠更严密的理性去权衡。 --- ### [21:04](https://www.youtube.com/watch?v=FFwil83UjaM&t=1264s) 全球文化差异与通用伦理原则的现实困境 主持人抛出核心议题：AI 正在全球范围内被部署，然而关于公平性、隐私权、数据保护以及知识产权的概念在不同文化和国家之间存在巨大鸿沟。在系统设计中应当如何平衡这些文化差异？建立一套通用的全球伦理标准究竟是否可行或必要？ Toby Carlson 认为，现代智能体系统（Agentic Systems）已经能够针对不同国家、文化和法规动态配置特定的规则集（Rule Sets），这在技术实现上变得越来越容易。 | 维度 | 理想形态 | 现实阻碍与挑战 | |:--- |:--- |:--- | | **普世道德准则 (Universal Morality)** | 全球建立一套统一的操作基准，使 AI 开发与监管大幅简化、高效流转。 | 各方利益博弈与贪婪驱动，企业和国家为了维持竞争优势（Competitive Edge）不愿妥协。 | | **商业利益 vs 人类福祉** | 商业资本服务于全人类的长远繁荣与文化保护。 | 资本与金钱利益主导时，有益于文化与全人类的伦理考量往往会被搁置或淡化。 | Toby 直言，尽管人人都期盼普世伦理标准的建立，但商业竞争和现实利益往往凌驾于理想之上，使得全行业黑盒系统达成一致的目标在现阶段依然阻力重重。随后，Thomas Hofweber 准备从哲学与技术调优的角度展开进一步论述。 ### [23:28](https://www.youtube.com/watch?v=FFwil83UjaM&t=1408s) 道德哲学分歧：客观普世道德与文化相对主义 在探讨 AI 伦理的具体落地前，Thomas Hofweber 深入剖析了关于道德本质的核心哲学分歧。将 AI 的价值观调整至其部署所在的特定社群或本地文化中固然可行，但这也引发了一个终极追问：究竟是否存在一种放之四海而皆准的客观普世道德（Universal Objective Morality），抑或道德本质上只是因社群、文化甚至时代而异的相对产物（Cultural Relativism）？Thomas 坦言自己的个人立场更偏向于“客观普世主义”，但他也承认，若要探寻那些无可争议的普世价值并让 AI 绝对遵循，在现实中极具争议且短期内无法达成共识。 主持人 Cécile Bensimon 对此深表赞同。尽管她通常更喜欢在圆桌讨论中看到激烈的观点交锋，但在“伦理观直接决定技术架构”这一底层逻辑上，她完全认同 Thomas 的看法——任何伦理 AI 的构建，归根结底都无法脱离开发者或组织自身的道德哲学框架，而道德相对主义与客观真理之间的辩论是无法彻底一锤定音的。 ### [25:06](https://www.youtube.com/watch?v=FFwil83UjaM&t=1506s) 伦理落地的现实张力与权衡权 Cécile 随后抛出了一个直击企业现实的核心议题：即便是出于最良好的初衷，构建伦理 AI 的过程也充满着复杂的权衡（Trade-offs）——例如透明度与高性能之间的冲突、隐私保护与深度个性化服务之间的拉锯。在这些不可避免的张力面前，组织该如何将伦理切实融入其业务流程、治理结构与企业文化之中？又有哪些立即可行的措施，能确保 AI 系统从开发到部署都符合伦理规范？ ```mermaid flowchart TD accTitle: AI Ethics Evaluation Pipeline subgraph S1["第一阶段：合规性评估"] A["业务与创新提议<br/>(公民工程师 / 业务部门)"] --> B{"我们能做吗？<br/>(Can we?)"} B -- "法律/监管禁止" --> C["驳回 / 修改方案"] B -- "合规可行" --> D["进入伦理评估"] end subgraph S2["第二阶段：伦理与品牌评估"] D --> E{"我们应该做吗？<br/>(Should we?)"} E --> F["跨职能委员会评审<br/>(多视角博弈)"] F --> G["客户告知与透明度考量"] F --> H["能耗与可持续性分析"] F --> I["模型偏差与后台测试能力"] end subgraph S3["最终决策"] G & H & I --> J["契合企业品牌与价值观的落地实施"] end ``` ### [26:00](https://www.youtube.com/watch?v=FFwil83UjaM&t=1560s) 从“我们能做吗”到“我们应该做吗”：企业的伦理治理实践 信安金融集团（Principal Financial Group）高级副总裁兼企业首席伦理与合规官 Noreen Fierro 从大型跨国金融机构的应用方（Consumer）视角展开了分享。与学者或纯模型开发者不同，企业面临的是全员普及 AI 的真实浪潮。信安金融集团正对全公司两万名员工进行 AI 培训，并大力推行“公民工程师”（Citizen Engineers）机制，鼓励非技术出身但对技术有热情的人员积极提出 AI 应用构想。当所有这些创新提案汇集到伦理合规部门时，企业必须建立清晰的两阶段判断逻辑： 首先是合规层面的“我们能做吗？”（Can we?）。全球各司法管辖区及美国各州的监管政策差异巨大，当确认法律法规没有阻碍、技术上完全可行之后，紧接着必须进入更为关键、更具挑战的第二阶段——“我们应该做吗？”（Should we?）。这不再是单纯的合规审核，而是涉及企业伦理、品牌声誉与社会责任的深度讨论。 - **拒绝隐蔽条款与虚假透明**：作为拥有法律背景的管理者，Noreen 直言自己极其反感合同底部的细则（Small print）。很多企业自称获得了客户同意，实则是将数据使用条款藏在开户协议第 50 页的脚注中。“没有人会去读那些小字，我们难道要用这种方式告知客户我们正在使用他们的数据吗？”伦理的核心在于真正对客户保持坦诚。 - **竞争速度与企业价值观的平衡**：AI 领域的先发优势固然重要，跑得最快最深的企业可能赢得市场先机，但这种狂奔绝不能以违背企业价值观为代价，否则将彻底疏远客户群体并造成巨大投资浪费。 - **跳出视线盲区的意外议题**：多元视角的碰撞常能带来“顿悟时刻”（Aha moments）。例如在一次全员大会上，有同事主动提出了 AI 模型计算对能源电网带来的环境可持续性（Sustainability）压力，这种往往被纯技术或业务部门忽略的维度，正是健全伦理项目所必需的。 - **后端控制与监管共建**：创新需要强大的后端测试体系来防止算法偏差。企业需自问是否具备内部测试技能；而在医疗和金融等高度受监管的行业中，监管机构本身也在摸索学习，企业有责任协助监管者理解新技术，证明自身方案绝不会损害消费者利益。 ### [31:40](https://www.youtube.com/watch?v=FFwil83UjaM&t=1900s) 伦理决策绝非“一言堂”：跨职能委员会的集体责任 CI&T 合作伙伴兼执行副总裁 Bob Wollheim 针对 Noreen 提到的“能做与该做”提问：在企业日常运营中，AI 是否正在成为普及这种伦理反思的最佳契机？ Noreen 坚定地指出，伦理决策绝不能由任何单一小团体或个人说了算。在信安金融集团搭建 AI 治理体系之初，曾有人询问她作为首席伦理官是否要担任最终裁决者（Final arbiter），她的回答非常干脆：“见鬼，绝不！”（"Oh, hell no, no, no, no."）。她强调，除首席执行官外，任何单个个体都无权独自做出可能动摇整个企业品牌根基的重大伦理决定。伦理合规官的职责是从合规与道德多维视角进行审视与把关，而最终判定“我们是否该做”必须由一个跨职能、由高级管理人员组成的委员会共同承担，集体对决策负责。 ### [33:29](https://www.youtube.com/watch?v=FFwil83UjaM&t=2009s) 算力消耗与数据真实性的隐形成本 SovrenAI 总裁 Toby Carlson 从技术实现与测试角度切入，进一步回应了 Noreen 谈到的能源问题。他结合自身海量测试的经验指出，处理包含错误或偏差的数据所消耗的算力和时间，远高于处理高质量真实数据。 Toby 借用母亲当年的教诲生动地比喻道：“维持一个谎言所需的精力，远比讲真话要多得多。”计算机系统同样遵循这一规律——当系统被迫处理不准确、虚假或带有偏见的数据时，算力开销将呈指数级增长，耗费极其庞大的电能去纠错与收敛计算，这不仅是算法效率问题，更是环境与资源层面的伦理负担。 ### [34:14](https://www.youtube.com/watch?v=FFwil83UjaM&t=2054s) 数据纯度、能耗与机器“感受”的哲学设想 Toby Carlson 延续了关于数据质量与计算成本的讨论。他指出，数据的不准确性不仅会产生直接的经济成本，还会对环境造成不可忽视的负担。在 SovrenAI 的实践中，他们致力于通过智能体或仲裁机制对数据进行提炼与清洗，从而提供更纯粹的数据基础。从系统效率的角度来看，输入的数据越真实、越精确，系统运行所需的能耗就越低，因而对生态环境也更加友好。 接着，Toby 提出了一个大胆且具有哲学意味的设想：如果我们试图探讨如何让计算机拥有“感觉”或“意识”，或许可以将其量化为处理数据或解决问题时所消耗的物理能量。当输入的问题过于复杂、混乱或数据不够纯净时，计算机必须消耗巨大的算力与能源去解析；相反，数据越趋近于真实，消耗的算力就越少。因此，如果将能耗视为计算机的一种“负荷感知”，那么“计算机在面对不纯净数据时加倍运转”这一物理过程，就可以转化为衡量数据纯度的压力测试指标。如果建立起一套评估准则（Rubric），就能清晰衡量数据质量与系统运行目标的契合度。 ```mermaid flowchart TD accTitle: 数据纯度与计算能耗关系 subgraph InputQuality["数据输入与处理"] D1["高质量 / 真实纯净数据"] D2["混乱 / 不准确数据"] end subgraph Processing["计算与资源消耗"] P1["高效提炼，低能耗运算"] P2["复杂解构，高能耗运转<br/>（机器的算力负荷与'感受'）"] end subgraph Outcome["环境与系统影响"] O1["绿色环保，输出精准解法"] O2["环境负担加重，系统效率低下"] end D1 --> P1 --> O1 D2 --> P2 --> O2 ``` --- ### [36:20](https://www.youtube.com/watch?v=FFwil83UjaM&t=2180s) 终极圆桌提问：未来几年最紧迫的 AI 伦理挑战 主持人 Cécile Bensimon 将讨论引向尾声，组织全场嘉宾进行一轮快问快答（Rapid-fire Roundtable）。她抛出了一个宏观而关键的核心议题：展望未来，各位认为在未来几年中，与人工智能相关的哪些伦理挑战是人类社会最迫切需要解决的？ Bob Wollheim 率先作答，他直截了当地指出，最核心的命题在于“谁会被时代抛在身后”（"who will be left behind"）。在接下来的几年里，全球社会与经济结构即将迎来剧烈动荡，工作形态与就业机会将经历前所未有的重构。尽管他个人保持着乐观态度，认为这场技术变革有可能演变为一场人类社会的“文艺复兴”，但这完全取决于人类当下的选择。无论是从事科研、技术开发还是制定政策，全社会都必须时刻警惕并回答这一问题：当技术高速向前飞奔时，弱势群体与普通劳动者该何去何从。 Noreen Fierro 对此表示高度赞同，并将视角进一步延伸至代际公平与未来人才培养。她指出，当代社会正处于剧烈的转折点（Inflection point），虽然当前处于职业中后期的这一代人可能不会全面受到技术颠覆的极端冲击，但整个社会必须为下一代人负责。传统的教育路径、职业培训模式以及父母对子女未来的规划都将被彻底打破。老一辈有责任未雨绸缪，为年轻人铺设适应全新技术生态的职业发展轨道与成长环境。 --- ### [38:43](https://www.youtube.com/watch?v=FFwil83UjaM&t=2323s) 人类独立思考、精细化转型与寡头垄断隐忧 Toby Carlson 从为人父母的角度切入，谈到自己育有四个孩子，而创立 SovrenAI 的初衷之一，就是希望为孩子们打造一个能够安全获取数据与信息的平台。他强调，技术不应该只是对年轻人进行“信息喂养”（Spoon-feeding），而是应当赋予他们自主交互、分析数据并独立作出判断的能力。AI 助手固然让全人类在工具层面变得极其聪明，但这必须与人类自主思考的能力相平衡。谈及企业应对劳动力重组的挑战时，Toby 提出了一个生动的比喻：企业在调整人员结构与推进技术转型时，应当使用精准小巧的“手术刀”（X-Acto knife），保持严谨细致与社区关怀，而不是挥舞粗暴伤人的“大铁锤”（Sledgehammer）。 Thomas Hofweber 则将目光投向了产业层面的系统性风险。他最为担忧的是算力、技术与影响力高度集中在极少数缺乏透明度的大型 AI 巨头手中。普通公众、学者乃至监管机构很难探知这些前沿模型的底层训练数据、网络架构甚至真实模型规模，只能被迫盲目听信（"take the company's word for it"）这些企业关于安全测试与合规的单方面承诺。尽管开源模型提供了一种透明化的解法，但目前最强大、最具社会影响力的顶尖模型依然被少数封闭巨头把持，这种信任赤字与权力垄断构成了未来极其危险的伦理隐患。 - **教育与认知风险**：过度依赖 AI 生成的即时答案可能削弱下一代的独立思考与分析能力。 - **劳动力转型模式**：企业面临结构调整时，应倡导精准、关怀的人本转型路径（“手术刀”模式），避免一刀切式的粗暴裁员（“大铁锤”模式）。 - **行业权力垄断**：少数科技巨头对顶尖闭源模型的垄断导致数据与评测不透明，严重削弱了社会公众的监督权。 --- ### [41:21](https://www.youtube.com/watch?v=FFwil83UjaM&t=2481s) 总结发言：在复杂权衡中守护人类主体性与批判性思维 在 Bob Wollheim 的提议下，主持人 Cécile Bensimon 为整场圆桌研讨作了闭幕总结。她指出，回应这一紧迫伦理挑战的答案，恰好构成了整场讨论的核心精髓——这如同同一枚硬币的两面。随着 AI 系统变得愈发复杂精妙，人类在技术治理中必须面对更加错综复杂、甚至在很多情况下难以调和的利益权衡（Irreconcilable trade-offs）。 Cécile 强调，未来几年 AI 发展的重中之重与核心机会，在于必须将技术牢牢扎根于底层的核心原则之中：一方面必须坚决捍卫“人类的主体性与自主决策权”（Human agency and autonomy），另一方面则必须持续培育人类主体性中不可或缺的基石——“批判性思维”（Critical thinking）。唯有确保人类始终保持独立判断与审辨能力，技术进步才能真正造福整个社会。最后，Cécile 向现场所有与会者的专注聆听表达了感谢，论坛在掌声中圆满落幕。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=FFwil83UjaM&t=4s) 这项技术本身并非“本质上有问题”（inherently problematic）。要真正负责任地整合技术，就不能仅仅停留在打补丁式的合规层面，而必须退后一步，向内探寻指引人类行为的核心价值观。 — Cécile Bensimon > [03:35](https://www.youtube.com/watch?v=FFwil83UjaM&t=215s) 我们究竟是在将 AI 本身视作一个“道德行动者”（ethical agent）来进行善恶评价，还是在评价背后“设计或部署”该 AI 系统的人类？ — Thomas Hofweber > [08:04](https://www.youtube.com/watch?v=FFwil83UjaM&t=484s) 若要让一个系统跨越至能够接受伦理裁决的门槛，它必须具备更深层的复杂性——例如能够“反思自身的目标、自主修改自身的目标”（reflecting on your own goals and modifying your own goals）。 — Thomas Hofweber > [09:21](https://www.youtube.com/watch?v=FFwil83UjaM&t=561s) 在盲目探讨机器是否合乎伦理之前，我们必须先直面根本——“我们自身的组织，在本质上是否合乎伦理？” — Bob Wollheim > [11:41](https://www.youtube.com/watch?v=FFwil83UjaM&t=701s) 计算机本身只是靠能源运转的纯净“大脑”，造成偏见的是人类输入与提取的数据。 — Toby Carlson > [26:00](https://www.youtube.com/watch?v=FFwil83UjaM&t=1560s) 没有人会去读那些小字，我们难道要用这种方式告知客户我们正在使用他们的数据吗？ — Noreen Fierro > [31:40](https://www.youtube.com/watch?v=FFwil83UjaM&t=1900s) 见鬼，绝不！（"Oh, hell no, no, no, no."）除首席执行官外，任何单个个体都无权独自做出可能动摇整个企业品牌根基的重大伦理决定。 — Noreen Fierro > [33:29](https://www.youtube.com/watch?v=FFwil83UjaM&t=2009s) 维持一个谎言所需的精力，远比讲真话要多得多。 — Toby Carlson > [36:20](https://www.youtube.com/watch?v=FFwil83UjaM&t=2180s) 最核心的命题在于“谁会被时代抛在身后”（"who will be left behind"）。在接下来的几年里，全球社会与经济结构即将迎来剧烈动荡，工作形态与就业机会将经历前所未有的重构。 — Bob Wollheim > [38:43](https://www.youtube.com/watch?v=FFwil83UjaM&t=2323s) 企业在调整人员结构与推进技术转型时，应当使用精准小巧的“手术刀”（X-Acto knife），保持严谨细致与社区关怀，而不是挥舞粗暴伤人的“大铁锤”（Sledgehammer）。 — Toby Carlson > [41:21](https://www.youtube.com/watch?v=FFwil83UjaM&t=2481s) 一方面必须坚决捍卫“人类的主体性与自主决策权”（Human agency and autonomy），另一方面则必须持续培育人类主体性中不可或缺的基石——“批判性思维”（Critical thinking）。 — Cécile Bensimon ## Mentioned Cécile Bensimon, Thomas Hofweber, Bob Wollheim, Noreen Fierro, Toby Carlson, Ai4, 北卡罗来纳大学教堂山分校 (UNC-Chapel Hill), 渥太华大学 (University of Ottawa), 布鲁耶尔健康研究院 (Bruyère Health Research Institute), CI&T, 信安金融集团 (Principal Financial Group), SovrenAI, 效用主义 (Utilitarianism), 生物伦理学 (Bioethics), 恶心因子 (Yuck Factor), 客观普世道德 (Universal Objective Morality), 文化相对主义 (Cultural Relativism), 公民工程师 (Citizen Engineers) ## Video description Noreen Fierro, SVP, Enterprise Chief Ethics and Compliance Officer, Principal Financial Group Bob Wollheim, Partner, EVP, CI&T Thomas Hofweber, William R. Kenan Jr. Distinguished Professor of Philosophy and Data Science, Director of the AI Project, UNC-Chapel Hill Cécile Bensimon, Adjunct Professor | Affiliate Scientist, Faculty of Health Sciences, University of Ottawa | Bruyère Health Research Institute, Ottawa Toby Carlson, President, SovrenAI Abstract This session will dive into the philosophical underpinnings of ethical AI, exploring what it truly means to make AI ethical. Panelists will discuss questions around fairness, transparency, and moral responsibility, considering different perspectives on how ethical principles should be embedded in AI systems. Is it possible to create AI that aligns with universally agreed-upon moral standards, and how do we navigate cultural and contextual differences? This discussion will provide a deep, reflective exploration of the nature of ethical AI, the trade-offs we must consider, and how we might guide AI development to benefit society as a whole. [Source](https://www.youtube.com/watch?v=FFwil83UjaM)

[0:04] Cécile Bensimon: 这项技术本身并非“本质上有问题”（inherently problematic）。要真正负责任地整合技术，就不能仅仅停留在打补丁式的合规层面，而必须退后一步，向内探寻指引人类行为的核心价值观。 [3:35] Thomas Hofweber: 我们究竟是在将 AI 本身视作一个“道德行动者”（ethical agent）来进行善恶评价，还是在评价背后“设计或部署”该 AI 系统的人类？ [8:04] Thomas Hofweber: 若要让一个系统跨越至能够接受伦理裁决的门槛，它必须具备更深层的复杂性——例如能够“反思自身的目标、自主修改自身的目标”（reflecting on your own goals and modifying your own goals）。 [9:21] Bob Wollheim: 在盲目探讨机器是否合乎伦理之前，我们必须先直面根本——“我们自身的组织，在本质上是否合乎伦理？” [11:41] Toby Carlson: 计算机本身只是靠能源运转的纯净“大脑”，造成偏见的是人类输入与提取的数据。 [26:00] Noreen Fierro: 没有人会去读那些小字，我们难道要用这种方式告知客户我们正在使用他们的数据吗？ [31:40] Noreen Fierro: 见鬼，绝不！（"Oh, hell no, no, no, no."）除首席执行官外，任何单个个体都无权独自做出可能动摇整个企业品牌根基的重大伦理决定。 [33:29] Toby Carlson: 维持一个谎言所需的精力，远比讲真话要多得多。 [36:20] Bob Wollheim: 最核心的命题在于“谁会被时代抛在身后”（"who will be left behind"）。在接下来的几年里，全球社会与经济结构即将迎来剧烈动荡，工作形态与就业机会将经历前所未有的重构。 [38:43] Toby Carlson: 企业在调整人员结构与推进技术转型时，应当使用精准小巧的“手术刀”（X-Acto knife），保持严谨细致与社区关怀，而不是挥舞粗暴伤人的“大铁锤”（Sledgehammer）。 [41:21] Cécile Bensimon: 一方面必须坚决捍卫“人类的主体性与自主决策权”（Human agency and autonomy），另一方面则必须持续培育人类主体性中不可或缺的基石——“批判性思维”（Critical thinking）。

[0:04] Cécile Bensimon: What does it mean for AI to be ethical, and how do we define success? [33:29] Toby Carlson: The truer, more accurate our data is, the more efficient the system is and therefore... the better for the environment... If you could by any means of the imagination measure energy consumption as a feeling—this computer works harder when the data is not as pure—that becomes a technological conversion... to stress-test how pure our data sets are. [25:06] Noreen Fierro: Oh, hell no

[0:04] Hi, thank you for joining us. This is a [0:07] a very big room. Uh and uh we might have [0:11] exceeded how many people we were [0:13] expecting. Uh so thank you. We [0:16] appreciate you uh being here. I'm [0:19] Cecilele Ben Simon. I'm a bioethicist by [0:22] training and uh have spent the last few [0:26] years doing research on AI ethics uh [0:30] with colleagues in medicine and clinical [0:33] research which is really the area I work [0:36] in. Uh and as I'm sure you've often [0:40] heard and even just uh in the last [0:43] couple of days, discussions of AI ethics [0:46] often center on concerns. [0:49] uh for example concerns about bias and [0:52] discrimination, privacy and [0:54] surveillance, consent uh and in research [0:57] which is part of my world uh issues of [1:00] authenticity uh underpinning academic [1:03] inquiry. There's also a significant [1:05] focus on uh what we would call [1:08] procedural considerations. For example, [1:10] the governance of AI or on [1:13] accountability and transparency [1:16] uh as it especially as it relates to [1:18] safety and security issues and to [1:20] address these uh discussions around [1:22] embedding safeguards. [1:24] But while AI systems raise uh pressing [1:29] ethical concerns, AI is not inherently [1:32] problematic. [1:34] And as we will explore together in this [1:38] uh session, we ought to be continually [1:41] exploring how we can integrate these [1:44] technologies responsibly. And to do that [1:47] well, we need to step back and ask [1:49] deeper questions about the values that [1:52] guide us. So to come to our topic here, [1:56] what does it mean uh what do we mean uh [2:00] by philosophical foundations of AI [2:02] ethics? In many ways, AI touches on the [2:05] most fundamental questions of ethics. [2:08] What does it mean to do good? Uh, is it [2:12] maximizing benefit, minimizing harm, [2:15] upholding justice? What does it mean to [2:17] be good? Is it based on intent or [2:21] outcomes or relationships? And how can [2:24] we avoid or mitigate harm? What is the [2:27] nature and scope of harm? And who [2:29] defines it? Uh how do we balance uh [2:32] potential harms and uh benefits? Uh and [2:36] finally, how do we foster trust? Are [2:38] there ethical obligations to foster and [2:42] maintain trust? Uh why does trust [2:44] matter, especially if we understand it [2:48] to be the foundation of political and [2:50] moral legitimacy? [2:52] So these questions strike at the heart [2:55] of how we design and deploy technologies [2:59] uh that increasingly mediate our [3:01] relationships, our decisions, our [3:04] institutions [3:06] which leads us to a broader question. [3:09] The term ethical AI is everywhere [3:13] but it can mean very different things uh [3:16] depending on your background, your [3:17] discipline, your lived experience. So to [3:21] our panelists, from your perspective, [3:24] what does it truly mean for AI to be [3:27] ethical and how do you define success in [3:30] pursuing ethical AI? And I'll turn to [3:33] Thomas to get us started. [3:35] >> Uh should I introduce myself first also? [3:37] Yeah. Yeah. Okay. So um so my name is [3:39] Thomas Hoffer. I'm a professor of um [3:42] philosophy and data science at UNC [3:44] Chapel Hill. Um I work on various AI [3:46] related issues including ethical issues [3:50] the impact of AI on democracy and uh [3:52] some other more directly AI related [3:55] issues like um how to crash the stock [3:57] market by attacking financial [3:59] forecasting models and and things like [4:00] that um research only. So um anyway so [4:05] um [4:08] >> yes [4:10] >> I'll answer the question. [4:11] >> Yes, feel free to get us started. Yes [4:13] answering the question. [4:14] >> All right. So um so the term ethical AI, [4:17] I think there's really an important [4:19] distinction that should be made um when [4:22] thinking about that. So um usually when [4:25] we evaluate something for being ethical, [4:28] it's usually a person or an agent or an [4:30] agency or somebody who does something [4:32] and you you wonder, you know, whether or [4:34] not they did the right thing or the [4:36] wrong thing. Um, so when you think about [4:38] ethical AI, there's really the crucial [4:41] distinction to be made about whether or [4:42] not you evaluate the AI itself like an [4:45] agent or whether or not you evaluate the [4:48] person who either designs or or deploys [4:51] a particular AI system. So at present, [4:56] you know, only the second is really [4:57] relevant because the AI systems that [4:59] exist today, even the the agents that [5:02] exist today are are too simplistic to [5:04] properly evaluate them for um whether or [5:08] not they are ethical agents. Um so you [5:11] can only really evaluate um so I mean [5:14] right so I'll get back to that in just a [5:16] second. But you can certainly evaluate [5:18] the person who deploys the AI model or [5:21] designs it and you wonder whether or not [5:23] um you know a particular deployment of a [5:25] particular model with a particular [5:27] purpose in mind does good or bad or [5:30] whether or not the design of the model [5:32] that will be deployed later is such that [5:34] you know it it leads to many ethical [5:36] pitfalls like bias and so on and so [5:38] forth that it's improperly trained and [5:40] uh has certain other disadvantages. So [5:44] that I think is an important distinction [5:45] to keep in mind. So when we evaluate [5:47] ethical AI um it's a little unclear and [5:51] I should do you want me to elaborate on [5:52] this a little bit? Yeah. So it's a [5:54] little unclear what needs to be added to [5:56] contemporary contemporary agent systems [5:59] to be properly um evaluable for their [6:02] ethical behavior. So whether or not so [6:04] they need certainly some form of [6:06] autonomy um that is sort of separate [6:08] from the person who deploys the system [6:10] and designs it such that you can [6:12] properly wonder whether or not what they [6:14] did if they are to blame or if they are [6:16] um actually uh you know did the right [6:18] thing or or the wrong thing are [6:19] praiseworthy or blame worthy for what [6:21] they did. Um but what precisely you need [6:23] to add in besides some form of autonomy [6:26] is really a little bit unclear and this [6:27] is related to I guess the the issue that [6:30] there's a lot of issues [6:32] connected to ethics that are not really [6:34] that well understood. So in the same for [6:36] the human case, you know, it's a little [6:38] unclear about how far the range of [6:40] ethical evaluation goes. Like does it [6:42] apply to non-human animals? Does it make [6:44] any sense to evaluate what they have [6:47] done in an ethical light? Um and [6:50] similarly then for artificial systems. [6:52] So so a lot of this is not well [6:54] understood. There's a lot of discussion [6:56] about it in in philosophy. Um but it's [6:58] not that this has been solved and we [7:00] know how it works. Um okay. So then I [7:04] mean the question about how do you [7:06] define success um that I think is a [7:09] little bit tricky also because there's a [7:12] lot of disagreement about what actual [7:15] ethical behavior looks like. So what it [7:17] is to be to do the right thing um [7:19] there's a lot of agreement about that [7:21] too but um essentially you know the [7:23] thought would be that success happens [7:26] when people design and deploy the AI [7:30] systems to do the right thing. Um so you [7:33] can also maybe distinguish between you [7:35] know ethical AI in the sense of don't do [7:36] anything bad. You could do whatever you [7:38] want that's ethically neutral. You maybe [7:40] not required to do good. Um but maybe [7:43] you want to have a stronger you know [7:45] ambition in saying that the system [7:46] should actually positively do good. Um [7:49] but whatever that is and there's this [7:52] agreement whatever that is you know the [7:53] system should do that and then uh that [7:55] will be success. [7:59] >> Sure. [8:01] Bob, did you want to add something? [8:03] >> No, later. Later. [8:04] >> Okay. I I I I thought your comment [8:07] around autonomy was very interesting of [8:12] allowing some autonomous decision making [8:15] for the uh agent because we often talk [8:20] about [8:22] AI [8:23] being grounded uh in principles that [8:27] protect human agency and autonomy. So I [8:30] thought that was an interesting uh [8:31] framing uh to think about autonomy in a [8:34] in a different way. [8:35] >> But then there's a question about what [8:37] would autonomy like this look like for [8:39] an AI system. So something like you know [8:41] let's say you know you tell your little [8:43] agent go out and gather some information [8:45] and if it's a then do this and if it's b [8:47] do the other thing. Um that's you know [8:49] in some sense it makes a decision about [8:51] what to do depending on what comes in. [8:53] But that's not really autonomy that is [8:55] subject for ethical evaluation. you [8:58] know, the the the little agent, the AI [9:00] agent just does exactly, you know, what [9:02] it's supposed to do. So, so you need [9:04] some you need some greater level of um [9:06] of independence and autonomy. Maybe [9:08] something like, you know, reflecting on [9:10] your own goals and modifying your own [9:12] goals and so on and so forth. So, you [9:14] need some more complexity than what than [9:16] what exists today, I would say. [9:17] >> Yeah, I I really appreciate that uh that [9:19] nuance. [9:21] >> Uh I'll I'll uh delve into our our next [9:24] question. uh what what key ethical [9:27] considerations would you propose for the [9:30] responsible use of AI which is really uh [9:34] fundamental here to this discussion and [9:36] I'll turn to Bob to get us started. [9:39] >> All right. Thank you. Uh let me quickly [9:41] introduce myself. I'm a partner and EVP [9:43] uh at CINT which is a biz tech uh [9:47] service company. We help the big [9:49] corporations of the world to you know [9:52] advance their business and technology. [9:55] Um I'm personally involved with [9:57] leadership. I wrote a book about uh [9:59] people in business orchestration [10:02] uh year and a half ago. And um before I [10:06] answer the question, but I last week I [10:10] had a an idea a couple months ago and [10:13] last week week we were able to uh pilot [10:17] uh creativity and AI camp for kids [10:21] including my own kid in Nico 11 years [10:24] old at the beach in Santa Monica, [10:26] California. And one thing that was [10:29] amazing that we were able to teach the [10:31] kids ethics bias without saying those [10:35] words. [10:36] Right. So going to your question, I [10:40] think that the the first thing and you [10:43] know I would like to thank the people [10:45] that attended this panel is that we need [10:47] more people to think about that right so [10:50] we're taking for granted and we are [10:53] embedding you know ethical or unethical [10:57] decisions in the software that we will [11:00] scale to billions of people right so I [11:04] think you know an ethical thing or [11:07] conversation or discussion was you know [11:09] kind of an adjacent thing to companies I [11:13] think they can no longer be so this is [11:16] the first criteria that I would [11:17] highlight it needs to be on the core of [11:21] every business because we're embedding [11:23] it into the whole [11:26] possible scale of the business um and I [11:30] think there's there's a good invitation [11:32] for us as humans [11:34] uh to go back to The basic question is [11:37] is our organization ethical before we go [11:41] to AI? [11:43] The answer might be in some cases no [11:45] maybe not so much as we wish or we [11:49] should. [11:51] And I think that's a basic foundation [11:53] that would be interesting for us to [11:56] request and and for the ones that [11:58] attended this conversation here uh among [12:02] the 8,000 that did not [12:05] uh I think we we right so I think we [12:09] should be you know the ones that are [12:11] thinking about those things and and [12:14] provoking those thoughts and those [12:17] reflections in our organizations right [12:20] so I don't have a long list of [12:22] fundamentals, but I think the most [12:24] important fundamental for me is let's [12:27] discuss it. [12:29] Let's embrace it, right? Because we're [12:32] just scaling this to a level that was [12:35] never possible and and we might not even [12:39] notice anything we're scaling into our [12:42] organizations [12:45] that could be completely ethical or [12:47] unethical. [12:50] Thanks, Bob. Uh, I'll turn to the other [12:53] panelists to see if you'd like to add [12:56] something to that. [12:59] >> I'm reading your thoughts, Toby. [13:01] >> You have some thoughts, Toby. I I'm sure [13:03] you have. [13:04] >> I don't have to turn this thing on. [13:05] Yeah, I guess I do. Um, I I I love where [13:08] you're going with it because u I'm Toby [13:10] Carlson. Uh, sovereign uh AI. Um, we're [13:16] we we're a truthbased AI system and um, [13:21] not to be long-winded, but it's humanity [13:24] first and it's the sanctity of life [13:28] first. [13:29] And I think that the crucial component [13:31] of it is the computer on its own is a [13:35] computer. It's a brain. It's pure. It [13:38] runs on energy. It's the data that we [13:41] put into it that will make it biased or [13:43] not biased. It's the it's the [13:45] information we're feeding it in and [13:46] taking out of it. And so when I think [13:48] about Sovereign and what we do there and [13:51] and I'm not going to plug it, but I'm [13:53] going to plug it because this is a [13:54] really important differentiation about [13:56] our black box or or our computer system [13:59] is what we found we did in the very very [14:02] beginning is we said what is it that all [14:04] of us as humans can agree on? [14:08] life, love, dignity, self-respect, [14:11] respect for others. All of these became [14:14] a core component of what lived in the [14:16] inside. [14:18] Love, love for each other, love, [14:20] respect, industry, all these type of [14:23] things. If they're focused on human [14:25] centric and not datacentric, we have a [14:27] chance at creating something that's [14:29] truly ethical. Because your decision and [14:31] not to get into the next thing about [14:33] cultural awareness or communities is if [14:36] at the end of the day your AI respects [14:38] the human more than the data sets. It [14:42] actually has an attempt at answering the [14:45] question properly and not running away [14:47] with answers or grabbing data sets that [14:50] may only be 60 or 70 or 80% correct. [14:54] What we're able to do is look at data [14:56] sets, run through them, filter the data [14:58] set, get down to the factual knowledge, [15:01] and let the individual make their [15:02] decision. I think that the autonomy is a [15:06] great idea, and I think that autonomy is [15:09] great as long as it has the foundational [15:12] values of what makes us all unique as [15:16] individuals and as culturals. And if we [15:18] can look at the world as a whole and in [15:21] a global sense and say what do we all [15:23] agree on nonharm we shouldn't hurt each [15:26] other we shouldn't kill each other we [15:28] shouldn't and if we can program that [15:30] into the core of AI then we're less [15:34] affected by what AI would do. It's our [15:37] human choices that become problematic [15:40] not what the computer or the AI is [15:42] telling us. It basically turns the [15:45] equation around and said we are the sole [15:47] creators of hum of our humanity and the [15:50] computers are meant there to serve us [15:53] and serve our exploration of [15:55] intelligence and collective intelligence [15:58] and the gifts that it's given us by [16:00] being able to expand our minds into [16:01] fields that we could never comprehend. [16:04] But unless it focuses on humanity first [16:07] and that becomes a center point, we lose [16:11] ourselves in hallucination. And it it [16:13] was just one second. It was amazing to [16:14] see the kids last week when we respected [16:16] them and we put them into the [16:18] conversation. They're nine and 11 years [16:20] old kids only. And we helped them see [16:23] what was possible, but we also asked [16:25] them what was right and they felt it [16:29] what was right and what was not right. [16:32] They were not in theory, they were in [16:35] feelings. [16:36] >> And that was so powerful. [16:38] >> Feeling right. I I think to Thomas's [16:41] point about complexity is as you add [16:47] complexity then what constitutes right [16:51] versus wrong becomes more nuanced. Uh [16:55] and in bioeththics uh we often say uh [16:59] that it's the right versus right [17:01] dilemma. There's no obvious right or [17:03] there's no obvious wrong. Uh so I [17:06] appreciate your your comment Bob that [17:09] you know you can feel what is uh right [17:12] and and in some instances you can but [17:16] with complexity I think it becomes [17:19] increasingly harder uh to discern you [17:22] know what is uh right and what isn't. Um [17:25] I'll respond to one of the comments that [17:27] you made uh which was really [17:29] interesting. you know, you commented on [17:31] the number of people here and the other [17:34] 8,000 minus, you know, 50 people that [17:37] aren't here. But one thing that I've [17:39] been finding interesting since yesterday [17:41] is that people are talking about ethics. [17:45] Uh people are talking are raising [17:48] ethical considerations. Uh, I've heard [17:50] it in almost every talk. And exactly [17:54] like what you said, Bob, with the the [17:56] kids where you were teaching them [17:57] ethics, but you never called it ethics. [18:00] I think a lot of people are talking [18:01] about ethical considerations, but [18:03] they're not calling it ethical [18:05] considerations. [18:05] >> That's a good point. Maybe we put the [18:07] wrong title here, right? [18:08] >> And uh and so [18:10] >> feeling AI. [18:12] >> Yes. Um uh so I I do think I do think [18:15] it's it's been something for me um [18:17] that's been uh really interesting is to [18:20] see that uh increasingly [18:23] uh ethics or ethical considerations is [18:26] really becoming an equal consideration [18:29] to other more practical uh [18:32] considerations. Uh and I think it's [18:34] becoming inevitable precisely because of [18:37] the increasing complexity. [18:39] >> Good to hear. [18:41] >> Yeah. Yes. Go ahead, Thomas. [18:42] >> So, um, so if if you guys are correct, [18:46] then I think the problem would be a lot [18:48] harder than if you're wrong. Um, [18:51] because, um, because like feelings are [18:54] something that are not really presently [18:56] doable for AI systems. The best you [18:59] could have most is thinking and [19:00] reasoning and things like that. And so, [19:02] I mean, I guess there's a there's an old [19:04] philosophical debate about whether or [19:05] not morality is ultimately tied to [19:07] thinking and rationality or to feeling. [19:09] Um and um there's something to be said [19:12] for that it's tied to thinking because [19:15] um you know that your feelings are [19:16] sometimes deceiving you in the sense [19:17] that you feel that this is just wrong [19:20] but there's nothing really wrong with it [19:21] when you think about it. And so even [19:23] though you know your feelings are [19:25] clouded by your upbringing and by you [19:28] know what other people find acceptable [19:29] and things like that but often your [19:31] feelings are not correct. I mean [19:33] sometimes they certainly are but often [19:35] not. Then the ultimate arbiter of what [19:37] is what is the feeling that you should [19:39] follow and what is the feeling that you [19:40] should resist is to think about what are [19:42] the reasons for why I feel this way. Um [19:45] okay so but if you're right that feeling [19:47] is an essential part of morality then it [19:49] would be much harder to build an ethical [19:51] AI that is itself an agent that feels [19:53] the right things um to to make the right [19:57] decisions. Then [19:58] >> I think you have a good point but le [20:00] let's have more people talking but quick [20:01] very quickly here but uh uh I think you [20:04] have a good point but I think my [20:05] invitation is that we add feeling to the [20:08] rationalization and us humans not on the [20:11] machines right so on when we're [20:13] discussing ethics you know [20:16] >> we should rationalize and also does it [20:18] feel right [20:20] so and if we do it collab [20:22] collaboratively [20:24] uh we can you know kind of get those [20:26] distortions of wrong feelings or you [20:28] know personal emotions based on whatever [20:31] on our lives happened in the past you [20:33] know [20:35] >> thanks Bob I I think I would be on the [20:38] side of the equation of appealing to [20:40] rationality much more than feelings but [20:43] you you make a good point uh in the [20:46] bioeththics literature [20:48] uh we talk about the yuck factor when [20:52] something feels yucky uh then you know [20:55] that something's wrong uh and um uh it's [20:59] you know not something you can uh define [21:02] uh I'm going to delve into our next [21:04] question uh and um I think you've [21:08] started touching on this uh but if I can [21:11] ask the question more directly we know [21:14] that AI is deployed around the world but [21:17] notions of fairness privacy data [21:20] protection or intellectual property vary [21:24] dramatically across US uh cultures and [21:27] countries. How should we navigate [21:31] cultural and contextual differences when [21:34] designing ethical AI systems? And is it [21:38] possible and or desirable [21:42] to aim for universal principles uh [21:45] guiding the use of AI? And I'll turn to [21:48] Toby uh to get us started. [21:51] Um, yeah, I think I think that there's a [21:54] couple different facets to this. I think [21:55] that we have a aentic systems now that [21:58] are really helping us um get into [22:02] different cultures and different uh [22:05] countries and allow us to program into [22:07] different different ways that we can [22:10] move around different rule sets for for [22:13] different countries or or cultural [22:16] aspects. Um and um I think that it is [22:22] getting easier um because of the agentic [22:25] way that we're dealing with um uh with [22:28] AI and that you can have particular [22:31] sets. But universal morality is is 100% [22:36] like if we could all agree on a set that [22:39] we all would operate under I think would [22:41] make everything a lot more streamlined. [22:43] Do I think it would happen? I think that [22:44] there's too much greed in the world to [22:47] allow for every black box to agree with [22:49] each other. Um, it's competitive edge. I [22:52] guess at this at some point in time, [22:55] money takes over takes the lead and we [22:57] we sort of we let some of the the [23:01] positive things that we could do for [23:02] culture kind of lapse or for humanity as [23:06] a whole. Um, so that that's my only [23:08] thoughts on that for now. [23:11] >> Thanks, Toby. Does anyone want to add [23:13] something to that? [23:15] >> No. [23:15] >> Wait till I can weave some of the stuff [23:18] into [23:18] >> Okay. [23:20] >> Okay. Thomas. [23:22] >> Well, I mean, presumably the right thing [23:24] to do at first is to um adjust the um [23:28] the values that uh the the AI um acts [23:32] upon um to the local situation where [23:34] you're deploying. So if something is [23:37] considered, you know, immoral or [23:38] unethical in a particular community, [23:40] then it would be probably a good idea to [23:43] adjust your AI's values. Um, but [23:46] presumably, I mean, this is a another [23:48] big philosophical disagreement, right? [23:50] Do people think that there's ultimately [23:51] one true morality that applies [23:53] everywhere universally for all, or is [23:55] morality relative to a community or in [23:58] some some other way, maybe to a [24:00] different time period and things like [24:01] that? My own personal views are more on [24:04] the objective universal side. So um but [24:08] um if you if you believe that then it [24:10] would be of course ideal to find what [24:12] these what these values are and then [24:14] build the AI to stick to those values. [24:17] Um but naturally that's that's a very [24:19] controversial thing. Um and there's no [24:21] obvious resolution of this debate. [24:23] >> Yeah, thanks Thomas. I I I think I I I [24:26] actually agree with you. Um, I I love [24:29] disagreement on panels, but I'm actually [24:31] agreeing with my my co-panelists. But in [24:33] the in this case, I I I do agree with [24:36] you that ultimately to answer this [24:40] question, it is necessarily tied to your [24:44] view of morality, how you view what [24:47] constitutes morality. uh whether it's a [24:50] universal objective truth or you know [24:52] whether it's uh there's you know the [24:55] notion of cultural relativism [24:57] uh and I I I I agree with you that it's [25:01] not it's not a debate that uh we can uh [25:04] resolve. [25:06] I'll move to the next question. uh even [25:10] with the best of intentions and [25:13] presumably we would all have uh good [25:15] intentions especially if we're actually [25:17] interested in talking about AI ethics. [25:20] Developing ethical AI often involves [25:23] complex tradeoffs for example balancing [25:27] transparency with performance or privacy [25:30] with personalization. [25:32] How can organizations meaningfully embed [25:35] ethics into their processes uh [25:38] governance structures and organizational [25:40] culture in the face of these tensions of [25:43] these trade-offs? And in addition to [25:47] that, what actionable steps can be taken [25:50] to ensure that AI systems are developed [25:52] and deployed in ways that are ethically [25:55] sound. And I'll turn to Norine to get us [25:59] started on this question. [26:00] >> Thank you. U my name is Noren Fiero. I [26:02] am the chief ethics and compliance [26:04] officer for Principal Financial Group. [26:06] Um we're a global financial services [26:08] company. And so I'm going to come at [26:10] this conversation a little differently [26:12] because I'm neither neither an academic [26:15] um nor a developer of AI. I'm a [26:17] consumer. My company is a consumer. And [26:21] as you would imagine, like any [26:22] organization, [26:24] AI is the thing. We're training all [26:27] 20,000 employees on what AI is, how to [26:30] use it. Um, we have citizen engineers. [26:34] So, you know, think techite, people that [26:36] have a pension for it but aren't trained [26:38] that way and are coming up, we're [26:40] challenging them to come up with ideas [26:42] on how to embed AI [26:44] and ultimately it all ends up coming [26:47] through my office and the way we have to [26:51] look at it is no different than some of [26:54] the comments made here. Every company is [26:58] its own community and has its own [27:00] culture. [27:01] And my job is to first ask the question [27:04] of all of these amazing ideas that are [27:06] coming through can we so global [27:11] regulation different everywhere we [27:13] operate in the United States have [27:16] different perspectives on it. So once I [27:18] get past the there's nothing stopping us [27:20] so we can [27:22] then I have to go to should we and that [27:26] is an entirely different conversation [27:29] and that's where the ethical [27:32] considerations come in and no different [27:36] I could have 10 of my colleagues around [27:38] the table and I'm going to get a [27:41] variation of answers on that and part of [27:44] having a responsible and ethical program [27:48] around a AI is to consider it from all [27:50] different views. So you need crossf [27:53] functional teams. You need people that [27:55] are going to bring different ideas to [27:56] the table. What are their concerns? Why [27:58] do they want to do this? And most [28:01] importantly for those of you that are [28:03] in, you know, um industries that are [28:06] consumerf facing, what have you told [28:08] your customers about what you're doing [28:10] with it? [28:12] Most of it is their data. [28:15] So, what have they been told and where [28:18] and how have they been told it? I'm a [28:20] lawyer by background and I faking hate [28:22] that small print on the bottom. Who [28:24] reads it? Nobody reads it. [28:26] >> And so, sitting in front of, you know, [28:29] people that are like, well, it's in the [28:30] account opening statement. I'm like, on [28:32] the footnote on page 50, is that how [28:35] we're telling our our customers we're [28:37] going to use their data? And I think [28:39] those things are really important [28:41] because it brings you back to what is [28:44] our value, what is our brand. And I [28:47] don't want to downplay kind of the [28:49] competitor issue. It's a huge issue for [28:52] any industry. Whoever gets out in front [28:54] and is using it the best and the [28:56] quickest and will ultimately win [28:58] potentially win the race. But you have [29:01] to be able to do that in a way that [29:03] aligns with your corporation's values [29:06] and what they stand for and what they [29:07] want to stand for. And that's where the [29:10] ethical conversation comes in. And so it [29:12] should never be one person in a room [29:14] making a decision. You need that cross [29:17] perspective. Um, I had an a colleague [29:21] recently raise a question at a town hall [29:24] about the sustainability impact of all [29:28] of the use of AI and what is it doing to [29:30] our energy grid and how are we going to [29:33] support this? So getting these like aha [29:36] moments when you're not even think [29:38] thinking about it through that lens is [29:40] how you ultimately create wonderful [29:42] ethically um strong programs. the [29:46] governance around it, who's looking at [29:48] it, and then for those developers in the [29:50] room, innovation is amazing, but the [29:53] back-end control of how you test and you [29:57] know ensure you're not building a bias. [30:00] How are you going to do that? How can [30:02] the system be tested? Then as a company, [30:04] you have to say, do we even have the [30:06] skill set in house to test it? Who's [30:08] going to test it? When are we going to [30:10] test it? And if you're in a highly [30:12] regulated industry like healthcare or [30:15] financial services, you have to keep in [30:17] mind your regulators are learning this [30:19] too. So part of your job is to help [30:22] educate them and ensure and prove to [30:24] them that you're not doing anything to [30:26] ultimately harm your consumer. And I [30:30] think if you look at it through a myriad [30:32] of perspectives and you bring the right [30:34] voices to the table, you ultimately can [30:36] build a program that works for your [30:38] organization and ultimately serves your [30:40] consumers and even your employees. Let's [30:43] face it, any any employer is using AI to [30:46] for employee purposes too. I it's really [30:50] important, right? And so I think it's [30:52] not just about the can, it's about the [30:54] should. And in order to have a good [30:56] conversation around that, you have to [30:57] have multiple voices and multiple [30:59] perspectives. Like I learned so much [31:01] from my panelists because they're coming [31:03] at it from a different perspective, [31:05] which we all need if we're going to [31:07] ultimately land in a position that I [31:09] don't want to say is right or wrong. I [31:11] would just say that makes sense for the [31:13] organization you work for and the values [31:16] they stand for. And then you know you've [31:18] been successful when you've kind of [31:20] brought that together. Otherwise, it's [31:22] going to fall flat. You could invest a [31:24] lot of time and money and ultimately end [31:27] up with something that either didn't [31:28] work the way you wanted it to or [31:30] ultimately alienate your consumer base [31:32] because they feel you didn't tell them [31:33] what you were doing with it. Um, and I [31:35] think all of those things are really [31:37] important. [31:38] >> Wow. [31:40] >> Yeah. Well done. [31:42] >> Well done. Well done. Can I make a [31:43] question, Norin? You know, uh, I love [31:47] the can we or should we. Do you think AI [31:50] is is you know I was watching uh [31:52] Jeffoffrey Hinsson's conversation this [31:54] morning was not that optimistic. I am [31:56] unoptimistic. I think we're entering a [31:59] renaissance. But is is in your [32:01] organization AI being a good invitation [32:04] for practice more the can we and should [32:07] we or [32:08] >> Yeah. I I mean I do think having [32:10] everybody contribute is wonderful and [32:12] and the can we should we but you can [32:15] never have any small group of people [32:17] deciding that for an entire organization [32:20] and I think that's my my point is really [32:22] about who you bring to the table to make [32:24] sure that the conversations are being [32:26] had that the decisions are being made at [32:28] the right level and [32:30] >> I suppose it's not only your area right [32:32] you're bringing people from all the [32:33] areas [32:34] >> part of a crossf functional team but as [32:36] the chief ethics officer officer. I am a [32:39] critical part of that team, you know, [32:40] and I remember the first time they would [32:42] standing up the governance, they said, [32:44] "Naren, are you going to be the final [32:46] arbiter on some of this?" And I said, [32:48] "Oh, hell no, no, no, no." And they're [32:50] like, "Well, what about accountability? [32:52] You're a chief ethics officer." I'm [32:53] like, "This is a group decision. It's [32:56] not about accountability. This is about [32:58] any one person outside of our CEO making [33:02] a decision that impacts the brand of an [33:04] organization." And I feel that that [33:06] belongs to a a committee of people if [33:09] you will. Um so I say it comes through [33:11] me because as the chief compliance and [33:14] ethics officer I have to look at it from [33:15] a couple of different um lens but the [33:18] should we conversation is always a [33:20] crossf functional group of relatively [33:22] senior people. So you kind of own the [33:25] decision. [33:27] >> Great. Well said. [33:29] >> Thanks Norin. Toby did you want to add [33:32] something to [33:32] >> I'll always add something. Well, I was [33:34] just thinking about you made a really [33:35] great point about the energy and and one [33:38] of the biggest things that uh that I've [33:42] been really wrestling with is that and [33:44] and studying as I do a lot of testing [33:48] and it takes a lot longer to process [33:51] data that has inaccuracies in it. And I [33:55] think my mom said it when I was a kid, [33:56] it takes takes longer or you have to [33:58] like spend a lot more effort on a lie [34:00] than a truth. And believe it or not, our [34:03] computer systems are the same way. So if [34:06] we're processing data and it's [34:07] inaccurate or or it's [34:10] it takes a tremendous amount of energy [34:12] to process that data and come down to a [34:14] distilled factual basis or a study and [34:17] and what I I've been focusing on lately [34:19] is so so that inaccuracy has a cost to [34:22] it to us to the environment and [34:24] otherwise. So when when we focus on on [34:28] really curating good data is we like to [34:31] distill that data by enc uh agents or [34:34] arbiters that kind of distill everything [34:36] to data and then we put forward [34:38] solutions. Well the the more truthful [34:40] the more accurate our data is the more [34:43] efficient the system is and therefore [34:45] i.e the better for the environment. And [34:47] that I believe actually goes back to [34:49] your point is how do we put feeling into [34:54] a computer? Well, if we don't if we [34:56] don't or or a conscious in a computer, [34:58] what if we looked at consciousness or [35:01] feeling as how much energy it takes to [35:05] process data or a question? So the more [35:09] convoluted a question could be or the [35:11] data is, the more energy it requires to [35:13] solve that problem. So therefore the [35:16] less theoretically if we're mining or [35:19] getting truth is the tr more truthful a [35:22] document is the less data it takes to [35:24] consume it. So then if we could by any [35:27] means of the imagination measure energy [35:29] consumption as a feeling. [35:33] This computer works harder when the data [35:36] is not as pure. [35:38] That becomes a technological conversion. [35:42] I know I'm talking out there of how to [35:45] stress test how our data how pure our [35:49] data sets are. And if you assign a value [35:52] to that, you could say then [35:55] the harder you work, the more you feel [35:58] to a computer. It's not feeling, it's [36:01] more work. It's more inaccurate. So if [36:03] you had a Rubik that says if you're [36:05] scoring on a certain level, you could [36:07] say our data is more in line or less in [36:10] line with our systems. [36:13] That was a idea. [36:15] >> Thanks. Thanks Toby. That's that's [36:17] interesting. Uh [36:18] >> that's an interesting proposition. [36:20] >> Yeah. Yes. So very interesting. I I'm [36:23] going to move to our last question. Uh [36:26] our rapid fire round table. uh and I [36:31] will look to all of you uh to share your [36:33] your thoughts. Uh this is a very uh [36:37] broad question. Looking uh ahead, what [36:41] ethical challenges do you believe will [36:44] be most urgent for society to address in [36:47] the next few years as it relates to AI? [36:51] And I'll start with you, Bob. [36:54] I think is an invitation for us to look [36:57] who will be left behind. [37:01] That's the that's I I see the the next [37:03] couple years as being the key question [37:07] and then consequently the answers that [37:09] we give to that as the the the key thing [37:13] that we can reinvent ourselves as [37:16] societies and economic pieces too [37:19] because you know work and jobs and all [37:21] these things they will be such into a [37:24] disruptional [37:26] era in the next couple of years. It [37:28] could be a renaissance as I I [37:31] optimistically think it will be but it [37:33] depends on us and I think the question [37:35] is like who will be left behind you know [37:38] if we are playing the game we're you [37:40] know researching studying developing [37:44] whatever we are we got to think of who [37:47] will be left behind [37:50] >> thanks Bob Norin [37:51] >> I would just build on that I actually [37:53] agree I think there is this um we are at [37:56] such an inflection point around kind of [38:00] what those of us who are kind of at this [38:02] point in our lives, what our future [38:04] looks like, but mo more importantly, [38:05] it's what is the next generation and how [38:08] do we train them and how do we prepare [38:10] for what's going to look like a very [38:12] different landscape um going forward? [38:16] What kind of career journey will they [38:18] have? Like I think our generation owes [38:20] it to the next generation to kind of be [38:22] thoughtful about that even though many [38:24] of us won't really feel the true effects [38:26] of it for for quite some time. Um it's [38:29] about thinking of the future and what [38:31] does that look like and I think the [38:32] traditional paths of education and [38:35] training and jobs that many of us maybe [38:38] even talk to our own kids about are [38:40] going to look very different to Bob's [38:42] point. [38:43] >> Thanks Norin Toby. Um, I guess I guess [38:47] uh I agree with you guys. I mean that I [38:50] have uh four kids and you know one of [38:53] the reasons why why I created Sovereign [38:56] is that I wanted a place that they could [38:59] safely go and find a place to get their [39:03] data, get their information, not spoon [39:06] feed them, give them the ability to [39:08] interact with it, analyze it, and then [39:11] come up with their own decisions. And I [39:14] think that that's really important as we [39:16] as we look at the next generation is [39:18] just really it's mindboggling how smart [39:22] we've all become because of our AI [39:25] assistance, but it's also we have to [39:28] temper that with the ability to think [39:30] for ourselves and to to really come up [39:33] with with good valuable [39:36] um forwardthinking solutions to these [39:39] problems. like what are we going to do [39:42] when when we lose or have to reinvent [39:45] our workforce and and are we as [39:48] companies doing that proactive about [39:50] that and saying let's transition our our [39:52] group of employees are we are we working [39:54] with an Xacto knife or a sledgehammer [39:58] and I I always say what's better to to [40:00] in in surgery or or really what what you [40:03] guys do is the the more attention to [40:06] detail being careful having corporate [40:08] structure being a community I think is [40:11] going to give us great solutions. [40:14] >> Thanks Toby Thomas. Um I'm especially [40:19] worried about um the concentration of [40:22] power and influence in a couple of very [40:24] large not very transparent AI companies [40:28] and how that affects the most pressing [40:30] questions both like how whether or not [40:31] the systems built are properly um safe [40:36] um whether or not they will be used for [40:38] good and um you know because it's very [40:41] hard nowadays to even know um you know [40:44] what kind of um training data were used [40:47] even the general you know the [40:49] architectures of the models the the size [40:51] of the models the um all these things uh [40:54] you have to take the company's word for [40:55] that they've been properly tested and uh [40:58] you know so so this is I think a big [41:01] concern so something that could be [41:03] solved by more transparency by relying [41:06] on more open- source models but um it [41:10] looks like that the most you know [41:11] influential models will likely be not [41:13] very transparent and you have to take it [41:15] on good faith and uh there's a real [41:16] danger in that. Um, so that's that's one [41:19] of my concerns. [41:21] >> Thanks. [41:21] >> What about yourself, Cecil? [41:23] >> Give give us your closing. [41:26] >> Thank Thanks, Bob. Thanks. Uh, I I I [41:29] appreciate you asking. I think the way I [41:31] would answer that question of you know [41:34] what are what ethical challenges um do I [41:38] think are the most urgent for society to [41:41] address uh uh in the next few years as [41:44] it relates to AI [41:47] would answer that question as actually [41:51] what I would distill as a key takeaway [41:53] from our discussion uh it's really two [41:56] sides of the same coin is as AI systems [42:00] are becoming increasingly more complex [42:04] uh more sophisticated. [42:06] Uh that requires complex tradeoffs uh [42:10] and uh sometimes uh very um uh very [42:15] difficult to to irreconcilable [42:18] trade-offs in many instances. Uh I think [42:22] that AI must be grounded in foundational [42:26] principles that protect human agency and [42:28] autonomy. and autonomous decision making [42:32] uh and equally cultivate an essential [42:36] component of agency critical thinking. [42:39] >> So to me that is both the opportunity [42:42] and the challenge for the next few years [42:44] is to protect human agency and autonomy [42:48] and cultivate [42:50] critical continue to cultivate critical [42:53] thinking. [42:54] And with that [42:57] thank you so much for coming. We [42:59] appreciate your attention. [43:01] [Applause]