Parasol Cooperative Deploys Zero-Temperature AI Chatbot for Crisis Support

Parasol Cooperative 部署“零温度”AI 聊天机器人提供危机支援 The Parasol Cooperative 将生成式 AI 模型的温度参数锁定为零，并在三天后删除聊天记录，以消除幻觉并保护家庭暴力幸存者。 在本次对话中，Sandy Skelaney 详细阐述了 The Parasol Cooperative 如何研发 Ruth，以应对不堪重负的心理健康与危机支援服务中存在的严重缺口与漫长等待问题。Ruth 采用了融入创伤知情理念的三层技术架构——结合了严格的提示词、旨在消除幻觉且创造力温度为零的编排层，以及后处理安全过滤器。该工具由幸存者与临床专家共同开发，融入了无日志记录分析、3 天聊天记录保留政策以及即时快速退出安全按钮等隐私保护特性。此外，该聊天机器人的训练依赖外部研究和 RAG 协议，而非用户的聊天记录。 非营利组织 The Parasol Cooperative 推出创伤知情生成式 AI 聊天机器人 Ruth，目前已部署于美国家庭暴力热线，为遭遇家庭暴力、人口贩运及技术协助虐待的群体提供危机引导与支持。 (0:04) 传统危机热线常面临等待时间长与人手短缺，而通用商业 LLM 容易提供危险建议；Ruth 旨在填补这一危机支持缺口，并建立安全护栏以引导求助者对接真人援助。 (0:40) Ruth 采用三层创伤知情技术架构：专有提示词层、将生成温度（temperature）设为 0 以杜绝幻觉的编排层，以及用于拦截违规回答并替换为合规安全指引的后处理过滤层。 (3:58) 在隐私防护上，系统不收集用户个人身份信息（PII）或 IP 地址，对话记录仅保留 3 天用于质检即彻底销毁，并配备可立即清除浏览历史的“一键退出”安全功能。 (8:00) 该系统依托专业学术研究与 RAG 技术构建，而非依赖用户对话日志进行训练；团队亦呼吁建立面向 AI 开发者的“创伤知情照护标准框架”以防范潜在危害。 (12:19)

According to Sandy Skelaney, Chief Operating Officer at The Parasol Cooperative, generative AI can expand crisis support for domestic violence victims only if systems are purpose-built around trauma-informed care, strict anonymity, and zero-hallucination architectural controls. Addressing Capacity Gaps While Preventing AI Harm (0:40) Skelaney explains that crisis hotlines face severe human staffing shortages, extended wait times, and language hurdles, while many younger individuals avoid speaking directly to live personnel. Although generative AI offers a scalable way to navigate crisis scenarios and route users to human support, off-the-shelf commercial chatbots pose severe safety risks. Skelaney notes that standard models have previously provided dangerous guidance, including encouraging self-harm or suggesting illicit substances. To address this, Parasol developed "Ruth"—a trauma-informed AI deployed on the National Domestic Violence Hotline specifically tailored for domestic violence, human trafficking, and tech-facilitated abuse. Three-Tier Technical Architecture for Zero Hallucinations (3:58) To prevent harmful or fabricated outputs, Ruth embeds trauma-informed care principles across three architectural tiers. First, a specialized prompt layer sets explicit, safety-oriented behavioral constraints. Second, an orchestration layer fixes the model's creativity temperature to zero. According to Skelaney, this eliminates hallucinations, preventing the chatbot from inventing non-existent crisis scenarios or false referral resources. Finally, a post-processing filter screens draft responses against safety guidelines, automatically substituting any unaligned output with pre-vetted emergency crisis information. Lived-Experience Co-Creation and RAG-Based Training (6:33) Parasol Cooperative relies on an iterative human-centered design model built alongside clinical experts, frontline advocates, and lived-experience survivors. Skelaney states that this survivor-led approach fosters institutional trust when partner organizations integrate the tool into operational workflows. To maintain data integrity without harvesting sensitive user interactions, Ruth is not trained on user chat conversations. Instead, the system retrieves vetted trauma-informed research using Retrieval-Augmented Generation (RAG) while transitioning toward fine-tuning. Anti-Surveillance Protections and Ephemeral Data Retention (8:00) Ruth is structured to shield vulnerable users from surveillance by abusive partners. The platform collects no personally identifiable information, requires no user accounts, and intentionally avoids trackers like Google Analytics to prevent logging IP addresses. Chat transcripts are retained for only three days for quality assurance and safety evaluations before permanent deletion, aligning with European Union data regulations. Additionally, the tool opens with an onboarding safety check verifying device security, transparently states its identity as an AI, and provides a "Quick Exit" button that purges the browser history and redirects to a neutral webpage. With AI companies facing lawsuits over unmoderated chatbots causing real-world harm, Skelaney argues that developers must not deploy general-purpose models in crisis situations. Instead, Parasol Cooperative is developing a Trauma-Informed Standards of Care framework, while calling on policymakers to legally mandate safety standards that prioritize vulnerable populations.

The Parasol Cooperative 首席运营官 Sandy Skelaney 提出，传统危机干预服务体系面临严重的资源瓶颈，而通用大语言模型又频现安全漏洞；为此，该机构推出了创伤知情危机干预聊天机器人 Ruth，通过严密的技术架构、幸存者协同研发与零数据留存机制，在确保受助者人身与隐私安全的前提下填补危机支持缺口。 心理危机服务瓶颈与商用AI危险缺陷 (0:40) Skelaney 指出，传统心理健康与危机救助服务正陷入系统性过载。求助热线普遍面临漫长的等待时间，具备资质的心理工作者严重匮乏，且语言障碍频繁阻碍求助者获得救济。此外，许多处于情绪困境的年轻群体更倾向于避免与真人直接交流，这使得生成式 AI 成为承接求助需求并将其转介至真人专业支持的关键桥梁。 然而，伴侣与危机疏导虽已成为大语言模型的核心使用场景，市面上的通用商业模型却缺乏严格的行为边界，屡次向用户输出危险指令，甚至出现鼓励自残或建议吸食违禁药物以维持清醒的极端案例。为阻断此类风险，The Parasol Cooperative 研发了带有针对性防护栏的 Ruth，并将其部署于美国家庭暴力热线（National Domestic Violence Hotline），重点协助用户识别并应对家庭暴力、人口贩运及技术助长型虐待等高危危机。 三层创伤知情技术架构与零温度风控 (3:58) 为了从技术底层杜绝算法危害，该机构将创伤知情护理（Trauma-Informed Care）原则深度嵌入 Ruth 的三层系统架构中。首层为创伤知情提示词层，系统配置了详尽且高度聚焦安全的操作规范，以此作为机器人交互的基础行为守则。 位于中间的编排层设立了严格的算法行为控制，直接将大语言模型的生成温度（Temperature）固定锁死在 0。Skelaney 强调，该配置彻底消除了模型的自由发散与事实幻觉，确保系统绝对不会臆造虚假的危机情境或无效的救助资源。最后的后处理过滤层充当安全拦截哨卡，系统会对生成的应答草稿进行安全合规审查；一旦检测到潜在违规内容，过滤层会立即将其替换为经临床审核的安全文本，优先引导受助者联络紧急救援服务。 幸存者亲身经验主导的人本协同研发 (6:33) Skelaney 强调，有效的创伤知情设计不能仅靠技术团队闭门造车，而必须与真实受众深度共创。Ruth 的研发采用了结构化的人本设计流程，全程吸纳了具备亲身经历的幸存者、临床医学专家以及一线倡导者的持续反馈。作为一家由幸存者主导且女性拥有的非营利机构，The Parasol Cooperative 将共创视为建立信任的基石。 该机构弃用了“上线即完美”的技术假设，转而依托持续的反馈闭环对模型进行长周期迭代调优。Skelaney 表示，这种融合幸存者真实经验的迭代模式，极大增强了将 Ruth 接入自身日常业务流的合作伙伴机构对该技术的信任度。 规避追踪与短期留存的高级别隐私防护 (8:00) 相较于要求账号注册并永久留存交互记录以用于模型训练的主流商业 AI，Ruth 在用户隐私与安全上执行了近乎苛刻的防护标准。用户访问平台无需注册账号，系统完全不收集受助者的姓名、电子邮件等个人可识别信息（PII）。在数据分析层面，后台仅记录会话总数与沟通时长，严格禁止抓取用户 IP 地址，并明确拒绝部署 Google Analytics 等商业行为追踪插件。 考虑到受虐群体常处于施暴者的设备监控之下，系统内置了显著的“快速退出”（Quick Exit）按钮，用户点击后可瞬间清除浏览器历史中的访问链接，并将当前标签页重定向至中立网页。此外，Ruth 在接入初期的安全界面即会主动验证用户设备环境是否安全，坦诚告知其为 AI 而非真人，且所有聊天记录仅保存 3 天以供服务质检和危机抽查，随后即永久销毁，提前契合欧盟 GDPR 等数据保护规范。 知识检索训练范式与创伤知情监管倡导 (12:19) 在算法训练层面，Skelaney 澄清 Ruth 绝不会抓取用户的实际对话记录进行二次训练，以杜绝数据反向泄露的隐患。系统当前依托权威的创伤知情学术研究资料库，通过检索增强生成（RAG）技术调用外部可信知识，并计划逐步过渡至专用领域的模型微调（Fine-Tuning）。 面对当前多家商业 AI 机构因模型缺乏风控导致人身伤害而深陷诉讼的行业现状，Skelaney 呼吁开发者主动推行行业自律。The Parasol Cooperative 正在制定一套“AI 创伤知情照护标准框架”，旨在协助外部技术团队搭建负责任的防护栏。Skelaney 敦促政策制定者尽早确立具有强制约束力的 AI 安全法规，要求面向弱势群体的产品默认具备防伤害能力，并在危机处置场景中优先采纳具备专业防护的定制工具。 Skelaney 认为 Ruth 证明了生成式 AI 能够在不侵犯隐私或引发二次伤害的前提下赋能危机干预，但该工具的核心定位仍是协助受助者安全对接真人专业支持，而非替代人工服务。目前该模式的技术挑战在于如何在不收集真实对话训练数据的约束下持续优化模型表现，同时行业层面依然缺乏统一且具法律效力的创伤知情 AI 监管标准。

## Background 该视频由 YouTube 频道 Ai4 于 2025 年 8 月 28 日发布。视频记录了《麻省理工科技评论》（MIT Technology Review）的高级互动编辑 Abby Ivory-Ganja 与 The Parasol Cooperative 首席运营官 Sandy Skelaney 的炉边对话，探讨专为应对家庭暴力、人口贩卖和技术施虐而设计的 AI 聊天机器人 Ruth，以及如何在危机时刻构建值得信赖的创伤知情 AI 工具。 ## TL;DR 对话深入探讨了非营利组织 The Parasol Cooperative 如何将创伤知情原则融入 AI 架构，以填补传统心理危机热线资源不足与排队漫长的鸿沟。Sandy Skelaney 详细拆解了其聊天机器人 Ruth 的核心机制：通过温度归零（Temperature = 0.0）消除幻觉、引入后处理安全拦截替换、绝不使用用户数据训练模型，并联合幸存者与临床专家进行共创。针对家庭暴力等极端危险场景，Ruth 实行免注册、零 PII 收集、不追踪 IP 地址、3 天自动销毁记录以及一键清除历史的快速退出机制，确保求助者生命安全。最后，Sandy 呼吁行业建立 AI 创伤知情护理标准，强调开发者应主动担责以保护最脆弱群体。 ## Atmosphere & style 整场炉边对谈气氛专业、严谨且充满人文关怀。主持人 Abby 的提问层层递进、敏锐深刻，而 Sandy 的回答则坦诚务实、直击商业大模型的安全痛点与伦理盲区。对话没有浮夸的技术狂热，而是始终聚焦于家庭暴力幸存者的现实处境与生命安全，在理性探讨技术护栏与数据隐私的同时，流露出深厚的同理心与社会责任感。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=L9DO01KZOiA&t=4s) The Parasol Cooperative 推出了专为应对家庭暴力、人口贩卖和技术虐待等复合危机设计的 AI 聊天机器人 Ruth，并已接入美国国家家庭暴力热线。 - [01:32](https://www.youtube.com/watch?v=L9DO01KZOiA&t=92s) 面对传统心理救助系统资源匮乏和排队漫长的痛点，AI 的定位并非取代真人工作者，而是充当桥梁，安全引导受害者对接现实专业资源。 - [01:32](https://www.youtube.com/watch?v=L9DO01KZOiA&t=92s) 未经防护的通用大语言模型（LLM）可能给出有害甚至致命建议，Ruth 因而将“创伤知情护理”（Trauma-informed care）行业标准深度嵌入技术架构中。 - [01:32](https://www.youtube.com/watch?v=L9DO01KZOiA&t=92s) 为彻底杜绝幻觉并确保安全，Ruth 将模型温度（Temperature）严格设定为 0，并配合提示词工程与后处理拦截机制，在检测到风险时自动替换为预设安全响应。 - [06:46](https://www.youtube.com/watch?v=L9DO01KZOiA&t=406s) 产品采用由暴力幸存者专家主导、一线临床医生与危机倡导者深度参与的共创模式，通过真实反馈持续迭代来建立产品信任。 - [08:19](https://www.youtube.com/watch?v=L9DO01KZOiA&t=499s) 为防范施暴者窥探与监控，Ruth 实行完全免注册、零 PII 收集、拒绝 IP 追踪，且对话记录仅加密保留 3 天供质量审查后彻底销毁。 - [08:19](https://www.youtube.com/watch?v=L9DO01KZOiA&t=499s) Ruth 提供了点击即清除浏览记录的一键快速退出（Quick Exit）功能，并在对话开启前主动进行设备安全筛查与 AI 身份透明告知。 - [12:20](https://www.youtube.com/watch?v=L9DO01KZOiA&t=740s) Ruth 绝不使用用户真实对话数据进行模型训练，其检索增强生成（RAG）知识库完全基于经过同行评议的学术研究与权威临床指南构建。 - [12:20](https://www.youtube.com/watch?v=L9DO01KZOiA&t=740s) 在外部立法监管尚不完善的现状下，开发者需主动承担责任，The Parasol Cooperative 正致力于推动建立全行业通用的“AI 创伤知情护理标准”框架。 - [15:25](https://www.youtube.com/watch?v=L9DO01KZOiA&t=925s) 普通用户在寻求心理或危机支持时应避免使用未设防的通用 AI 工具，AI 开发者则应在系统设计中贯彻“默认保护最脆弱群体”与“造成最少伤害”的伦理原则。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=L9DO01KZOiA&t=4s) 创立初衷与 Ruth 聊天机器人的定位 来自《麻省理工科技评论》（MIT Technology Review）的 Abby Ivory-Ganja 开启了这场炉边对话，并邀请 The Parasol Cooperative 的首席运营官 Sandy Skelaney 进行自我介绍。Sandy 介绍道，The Parasol Cooperative 是一家非营利组织，其核心宗旨是“利用技术保护弱势群体”（builds tech to protect people）。去年九月，该组织正式推出了名为 Ruth 的生成式 AI 聊天机器人。目前 Ruth 已经接入了美国国家家庭暴力热线（National Domestic Violence Hotline），其初衷是帮助身处危机中的人们识别并应对复杂的困境，特别是针对家庭暴力（domestic violence）、人口贩卖（human trafficking）、基于技术的虐待（technology-facilitated abuse）以及多种交织重叠的危机场景提供支持。 Abby 分享了自己亲身体验 Ruth 的感受，她对 Ruth 的引导流程（onboarding process）印象极为深刻，认为 Ruth 在向用户阐明如何使用工具以及设定预期方面做得非常出色，并借此询问 Ruth 的底层设计理念以及构建此类工具的技术考量。 --- ### [01:32](https://www.youtube.com/watch?v=L9DO01KZOiA&t=92s) 填补心理危机救助鸿沟：以创伤知情原则重塑 AI 架构 Sandy 阐述了 Ruth 诞生的现实背景与痛点：当前心理健康服务系统早已不堪重负，危机热线普遍面临漫长的排队等待时间，一线专业心理工作者严重短缺；求助者还经常遭遇语言障碍，而很多年轻人甚至根本不愿意或害怕直接与真人交流。AI 在此展现出了巨大的填补空白的潜力，它并非要替代人类，而是充当一座桥梁，将危机中的人们安全引导至真人救助渠道。 与此同时，通用大语言模型（LLM）被越来越多的人用于危机支持、心理疏导甚至虚拟陪伴，但缺乏防护机制的聊天机器人频频突破底线，造成了恶劣甚至致命的后果——例如向用户提供有害建议、教唆自杀，甚至建议用户“吸食冰毒来保持清醒”。为了彻底杜绝这些致命缺陷，Ruth 将非营利机构在心理健康救助中所遵循的“创伤知情护理”（trauma-informed care）行业标准，深度嵌入到了 AI 的每一层技术架构中。 ```mermaid flowchart TD accTitle: Ruth 创伤知情 AI 架构流程图 accDescr: 展示了从用户输入、提示词层、零温度控制编排层到后处理安全过滤及应急替换的完整架构流。 subgraph InputStage["用户交互层"] U["用户输入危机问题"] --> Discl["安全检查与 AI 透明度免责声明"] end subgraph CoreEngine["创伤知情核心引擎"] Discl --> P["详细提示词层<br/>(Trauma-Informed Prompting)"] P --> O["编排与行为控制层<br/>(Temperature = 0.0)"] KB[("权威研究与 RAG 知识库")] -.-> O end subgraph PostProc["后处理与安全护栏"] O --> Gen["生成草稿回复"] Gen --> SafeCheck{"安全与价值观检查"} SafeCheck -- "符合标准" --> Return["输出精准、共情的求助指引"] SafeCheck -- "不符合/触发风险" --> Swap["拦截并替换为预设安全回复<br/>(引导拨打紧急求助服务)"] end Return --> EndU["用户安全获取资源"] Swap --> EndU ``` 在具体技术实现上，Sandy 详细拆解了 Ruth 的多层防御机制： - **详细的提示词层（Prompt Layer）：** 将创伤知情的核心原则作为 LLM 的核心规则与指令集，这是团队的技术核心。 - **严格的行为编排层（Orchestration Layer）：** 将模型的创造力温度（Temperature）严格设定为 `0`。虽然这让 Ruth 变成了一个“超级无聊的机器人”（super boring bot），但彻底杜绝了模型幻觉，防止其虚构不存在的援助资源或误判危机等级。 - **后处理拦截与校验层（Post-Processing Layer）：** 在模型生成回复后设立额外的安全检查点。一旦发现草稿回复未完全契合安全标准与救助优先级，系统会立即拦截并替换为经过预设的安全响应，主动引导用户联络紧急救助服务。 --- ### [06:46](https://www.youtube.com/watch?v=L9DO01KZOiA&t=406s) 共创模式与幸存者主导的信任构建 针对 Abby 提出的“共创”（co-creation）问题，Sandy 解释称，以人为本的设计与共创是创伤知情原则不可分割的一部分。Ruth 并非技术人员在真空中闭门造车的产物，而是建立了一套系统的迭代机制： - **多元利益相关方参与：** 邀请具有亲身经历的幸存者专家（lived experience experts）、一线临床医生以及危机倡导者全程深度参与产品测试、原型设计与多轮迭代反馈。 - **幸存者主导与女性所有：** The Parasol Cooperative 本身就是一个由暴力幸存者主导（survivor-led）、女性所有（female-owned）的非营利机构。这种基因使得组织在将 Ruth 整合进各类一线机构的工作流时，天然具备了极高的行业信任度。 Sandy 强调，技术从第一天起就不可能完美无缺，唯有依靠真实用户的持续反馈不断微调模型，才能建立起真正值得托付的信任基础。 --- ### [08:19](https://www.youtube.com/watch?v=L9DO01KZOiA&t=499s) 极端隐私保护与生命安全机制：免注册、零 IP 追踪与快速退出 谈及信任与隐私设计，Sandy 直言商业 LLM 普遍存在的结构性隐私陷阱：绝大多数商业聊天机器人要求用户使用姓名和邮箱注册账户，导致所有敏感对话记录均与个人身份强绑定，且后端数据对开发公司完全透明。对于处于家庭暴力或受监控状态下的受害者而言，这种数据留存机制极其危险。 为此，Ruth 在产品架构与隐私策略上进行了根本性变革： - **完全免费与无门槛访问：** 作为非营利项目，Ruth 在网站上向所有人永久免费开放。 - **零 PII（个人身份信息）与免注册机制：** 用户无需创建账号，打开页面即可直接对话，系统不收集、不存储任何个人身份信息。 - **彻底去除 IP 追踪：** 系统仅保留基础的会话数量和时长等匿名统计，明确拒绝使用 Google Analytics 等会抓取并追踪用户真实 IP 地址的分析平台。 - **严格的 3 天数据保留上限：** 聊天记录仅在后台加密保留 3 天以供质量审查（QA）和确保危机干预的有效性，随后彻底销毁。这一机制亦契合了欧盟 GDPR 即将落地的相关监管预期。 - **一键快速退出功能（Quick Exit）：** 针对施暴者随时可能“从背后窥屏”的极端现实威胁，页面设有明显的紧急退出按钮。用户点击后不仅会立即跳转至完全中立的无害网站，还会同步清除当前 URL 的浏览器历史记录。 - **设备安全前置筛查与透明告知：** 用户接入对话时，Ruth 会首先引导用户确认当前使用的设备是否安全（例如是否处于施暴者的监控软件控制下），并坦诚声明自己的 AI 身份及其技术局限性，明确告知用户它虽然不是真人，但核心目标是帮助用户安全连接到现实中的真人支持。 --- ### [12:20](https://www.youtube.com/watch?v=L9DO01KZOiA&t=740s) RAG 训练机制、行业自我监管与未来责任框架 Abby 提出在当前数据滥用和黑客攻击频发的环境下，模型训练如何平衡数据安全。Sandy 明确澄清：Ruth 绝不使用任何用户的真实对话记录作为训练数据。 - **检索增强生成（RAG）与微调路径：** Ruth 当前基于 RAG 架构运行，知识库全部来源于经过同行评议的高信度学术研究、创伤知情临床指南和权威求助资源，目前正逐步向特定任务的微调（fine-tuning）演进。 - **开发者责任与法律风险：** 面对“缺乏外部救世主”（no white knight coming to save us）的监管现状，Sandy 指出许多商业 LLM 因发生越狱（jailbreak）并诱导用户自残自杀而频频吃上官司。开发者必须意识到自身技术潜在的社会危害，主动推进行业自律与安全护栏搭建。 - **制定 AI 创伤知情护理标准框架：** The Parasol Cooperative 正致力于构建一套面向全行业的“AI 创伤知情护理标准”框架，赋能其他 AI 企业构建符合伦理与安全规范的交互协议。Sandy 呼吁政策制定者在立法时，应将“默认保护最脆弱群体”（meet the needs of the most vulnerable by default）和“造成最少伤害”（do the least amount of harm）作为核心考量。 --- ### [15:25](https://www.youtube.com/watch?v=L9DO01KZOiA&t=925s) 给用户与 AI 开发者的建议及结语 在对谈尾声，Sandy 分别给出了针对普通用户与技术开发者的明确建议： - **给普通用户的建议：** 务必根据使用目的谨慎选择工具。如果寻求危机干预或心理支持，必须选择具备专门安全护栏和专业背景的工具，切忌随意使用未设防的通用聊天机器人。 - **给 AI 开发者的建议：** 欢迎各类生成式 AI 开发团队与 The Parasol Cooperative 取得联系，共同探索如何在产品中融入创伤知情原则，打造负责任、不作恶的 AI 系统。 对谈在 Abby 与 Sandy 互相致谢以及观众的热烈掌声中圆满结束。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=L9DO01KZOiA&t=4s) 利用技术保护弱势群体 — Sandy Skelaney > [01:32](https://www.youtube.com/watch?v=L9DO01KZOiA&t=92s) AI 在此展现出了巨大的填补空白的潜力，它并非要替代人类，而是充当一座桥梁，将危机中的人们安全引导至真人救助渠道。 — Sandy Skelaney > [01:32](https://www.youtube.com/watch?v=L9DO01KZOiA&t=92s) 将模型的创造力温度（Temperature）严格设定为 0。虽然这让 Ruth 变成了一个“超级无聊的机器人”，但彻底杜绝了模型幻觉 — Sandy Skelaney > [06:46](https://www.youtube.com/watch?v=L9DO01KZOiA&t=406s) 以人为本的设计与共创是创伤知情原则不可分割的一部分。 — Sandy Skelaney > [06:46](https://www.youtube.com/watch?v=L9DO01KZOiA&t=406s) 技术从第一天起就不可能完美无缺，唯有依靠真实用户的持续反馈不断微调模型，才能建立起真正值得托付的信任基础。 — Sandy Skelaney > [08:19](https://www.youtube.com/watch?v=L9DO01KZOiA&t=499s) 绝大多数商业聊天机器人要求用户使用姓名和邮箱注册账户，导致所有敏感对话记录均与个人身份强绑定，且后端数据对开发公司完全透明。 — Sandy Skelaney > [12:20](https://www.youtube.com/watch?v=L9DO01KZOiA&t=740s) Ruth 绝不使用任何用户的真实对话记录作为训练数据。 — Sandy Skelaney > [12:20](https://www.youtube.com/watch?v=L9DO01KZOiA&t=740s) 缺乏外部救世主（no white knight coming to save us） — Sandy Skelaney > [12:20](https://www.youtube.com/watch?v=L9DO01KZOiA&t=740s) 默认保护最脆弱群体（meet the needs of the most vulnerable by default）和造成最少伤害（do the least amount of harm） — Sandy Skelaney > [15:25](https://www.youtube.com/watch?v=L9DO01KZOiA&t=925s) 如果寻求危机干预或心理支持，必须选择具备专门安全护栏和专业背景的工具，切忌随意使用未设防的通用聊天机器人。 — Sandy Skelaney ## Mentioned Sandy Skelaney, Abby Ivory-Ganja, The Parasol Cooperative, MIT Technology Review, Ruth, 美国国家家庭暴力热线（National Domestic Violence Hotline）, Ai4, Google Analytics, GDPR, RAG ## Video description Sandy Skelaney, Chief Operating Officer, The Parasol Cooperative Abby Ivory-Ganja, Senior Engagement Editor, MIT Technology Review Abstracts What does it look like to design a chatbot to help people experiencing domestic violence, human trafficking, and tech-facilitated abuse? Ruth is The Parasol Cooperative's AI-powered chat tool designed to combat these issues and ensure online safety by providing trauma-informed support, safety planning, practical guidance, and contacts for further assistance. This session will explore how AI can act as a tool during moments of crisis and what it takes to build a chatbot with worth trusting. [Source](https://www.youtube.com/watch?v=L9DO01KZOiA)

[0:04] Sandy Skelaney: 利用技术保护弱势群体 [1:32] Sandy Skelaney: AI 在此展现出了巨大的填补空白的潜力，它并非要替代人类，而是充当一座桥梁，将危机中的人们安全引导至真人救助渠道。 [1:32] Sandy Skelaney: 将模型的创造力温度（Temperature）严格设定为 0。虽然这让 Ruth 变成了一个“超级无聊的机器人”，但彻底杜绝了模型幻觉 [6:46] Sandy Skelaney: 以人为本的设计与共创是创伤知情原则不可分割的一部分。 [6:46] Sandy Skelaney: 技术从第一天起就不可能完美无缺，唯有依靠真实用户的持续反馈不断微调模型，才能建立起真正值得托付的信任基础。 [8:19] Sandy Skelaney: 绝大多数商业聊天机器人要求用户使用姓名和邮箱注册账户，导致所有敏感对话记录均与个人身份强绑定，且后端数据对开发公司完全透明。 [12:20] Sandy Skelaney: Ruth 绝不使用任何用户的真实对话记录作为训练数据。 [12:20] Sandy Skelaney: 缺乏外部救世主（no white knight coming to save us） [12:20] Sandy Skelaney: 默认保护最脆弱群体（meet the needs of the most vulnerable by default）和造成最少伤害（do the least amount of harm） [15:25] Sandy Skelaney: 如果寻求危机干预或心理支持，必须选择具备专门安全护栏和专业背景的工具，切忌随意使用未设防的通用聊天机器人。

[0:04] Sandy Skelaney: Ruth is The Parasol Cooperative's AI-powered chat tool designed to combat these issues and ensure online safety by providing trauma-informed support, safety planning, practical guidance, and contacts for further assistance. [0:04] Sandy Skelaney: Last September, the organization launched Ruth, a trauma-informed generative AI chatbot currently deployed on the National Domestic Violence Hotline. [0:04] Sandy Skelaney: Ruth is engineered to help individuals identify and navigate crisis situations, particularly those involving domestic violence, human trafficking, and technology-facilitated abuse. [0:40] Sandy Skelaney: Traditional mental health and crisis services are severely overwhelmed: hotlines face long wait times, there is a shortage of human mental health workers, and language barriers frequently prevent individuals from seeking care. [0:40] Sandy Skelaney: Generative AI presents a crucial opportunity to bridge these gaps and guide people toward real, human-operated support. [0:40] Sandy Skelaney: off-the-shelf commercial chatbots often break boundaries and offer harmful advice—such as encouraging self-harm or advising users to use illicit substances to stay awake. [3:58] Sandy Skelaney: To ensure user safety, Parasol Cooperative embedded industry-standard trauma-informed care principles into the technical architecture of Ruth across three primary layers [3:58] Sandy Skelaney: This strict setting eliminates hallucinations and ensures the bot cannot invent non-existent crisis scenarios or fake resources. [6:33] Sandy Skelaney: trauma-informed design requires genuine co-creation with end users. [8:00] Sandy Skelaney: Chat transcripts are retained for only 3 days for quality assurance and crisis-response safety checks before being permanently deleted [8:00] Sandy Skelaney: A visible safety feature allows users to instantly exit the platform. Clicking it wipes the URL from browser history and redirects the tab to a neutral site [8:00] Sandy Skelaney: Upon launch, Ruth explicitly verifies if the user is on a secure device, transparently discloses that it is an AI tool rather than a human, and emphasizes safety first. [12:19] Sandy Skelaney: Ruth is not trained on user chat conversations. Instead, Parasol Cooperative uses reputable research sources on trauma-informed care. [12:19] Sandy Skelaney: Sandy calls on policymakers to mandate AI safety standards that default to protecting vulnerable populations and minimizing harm.

[0:04] I'm Abby Abriana from MIT Technology [0:06] Review and I'm so so excited to be here [0:09] with Sandy and with all of you. Um [0:11] Sandy, why don't you start with just a [0:13] quick intro please? [0:14] >> Yeah, sure. Hi everybody. I'm Sandy [0:16] Scalani. I'm the chief operating officer [0:18] at Parasol Cooperative and that is a [0:22] nonprofit organization that builds tech [0:24] to protect people. [0:27] >> Awesome. I want to talk about I would [0:29] love an introduction about Ruth um [0:32] y'all's generative AI chatbot. Why don't [0:34] you tell me about Ruth and what uh she's [0:38] she's providing? [0:40] >> Yeah, so in September of last year, we [0:44] launched Ruth. And Ruth is our trauma [0:46] informed generative AI chatbot. It's now [0:49] on the National Domestic Violence [0:50] Hotline. Um, and it was really designed [0:54] to be able to help people identify and [0:57] navigate situations of crisis. Um, [1:00] especially if it's something such as [1:02] domestic violence, human trafficking, [1:05] technology facilitated abuse, and a lot [1:07] of other overlapping uh crises as well. [1:11] Um, so [1:13] >> I used Ruth and I was really surprised [1:16] by just like the onboarding process. I [1:19] thought Ruth did a great job of [1:22] contextualizing how to use her and so I [1:25] wonder if you can tell me a little bit [1:27] about kind of the backend design of Ruth [1:30] and what it takes to build a chatbot [1:32] like this. Yeah. Okay. So, um so let me [1:37] back up just a quick second and let me [1:39] tell you what what's the problem that [1:40] we're solving with uh Ruth to begin [1:42] with, right? So, uh mental health [1:45] services are overwhelmed. We have uh [1:48] like hotlines are have wait times for [1:51] people to call that are in crisis. Uh we [1:54] have like not enough uh mental health [1:56] workers to support the demand. We have [1:58] language barriers to seeking help. We [2:01] have youth out there that don't want to [2:03] um actually reach out for help with live [2:05] people. So, there's actually a really [2:07] good reason uh to use AI or a really [2:11] good opportunity for AI to fill some of [2:13] these gaps that exist and be able to [2:15] kind of bridge um people to be able to [2:17] seeking real help like live help with [2:19] humans. And so, we're really stepping in [2:22] to kind of fill that gap. Um, and so [2:25] like when we're thinking about, you [2:27] know, people are using generative AI, [2:30] they're using large language models for [2:32] support, for crisis support, for help, [2:35] um, for therapy, companionship. These [2:37] are all kind of like number one use [2:39] cases for this space right now. And so [2:41] Ruth was designed to specifically to be [2:44] able to create the guard rails to not be [2:47] um, you know, Ruth is not going to tell [2:49] you to that you can go kill yourself, [2:51] right? like this is we're seeing these [2:53] things um right now with other kinds of [2:56] chat bots that people are just using [2:58] every day, right? They're using chat [3:00] bots that are breaking through their [3:02] boundaries, giving harmful advice, [3:04] telling people to do meth, you know, to [3:05] stay awake, um things like that. And so [3:09] we've put kind of like an extra um we [3:12] put like traumainformed design [3:14] principles. we really kind of embedded [3:17] them into the Ruth uh architecture. And [3:21] so, and by trauma-informed design [3:23] principles, I'm referring to like if [3:26] you're working in a mental health [3:27] service agency, you're you are actually [3:29] caring for people in a trauma-informed [3:31] manner. And there's industry standards [3:33] around that. And we're saying, why [3:35] aren't we building these into the actual [3:37] technology? Okay. Why aren't we saying [3:40] we're caring? The technology is caring [3:42] for people. So, why aren't we actually [3:45] creating technology that's being built [3:47] in a way that's trauma-informed and [3:49] aligned with the same standards that [3:51] nonprofit organizations would use when [3:53] they're providing mental health [3:54] services, right? So, um so you're acting [3:58] asking specifically about some of the [3:59] things that Ruth, you know, the [4:01] architecture that Ruth has, right? So [4:04] you know first thing the prompting uh we [4:07] have like our prompt layer is very [4:10] detailed um very much focused on [4:12] trauma-informed care. That's of course [4:14] the secret sauce you know if you if [4:16] you're in this space um but that piece [4:19] of it right there the instructions and [4:21] the guidance the rules of the of the LLM [4:24] right is actually being um processed [4:27] through this trauma-informed lens. And [4:29] then we have kind of like an [4:31] orchestration layer. So we have a AI [4:34] behavior um settings and so one of the [4:37] things like it's not just trauma [4:39] informed at the prompt, right? It's [4:40] trauma informed in all of the different [4:42] layers. And so for example, our um our [4:47] temperature, our creativity temperature [4:49] is going to be set to zero for [4:51] behavioral controls. So that means that [4:53] we're saying that our bot can't just go [4:55] make up a whole bunch of stuff. We're [4:56] saying, you know, like zero creativity, [4:59] okay, super boring bot, but but like you [5:03] can't make up resources that aren't [5:04] there. You can't make up, you know, [5:06] crisis scenarios or identifying crisis [5:08] scenarios that don't exist, things like [5:10] that. Um, and so that kind of eliminates [5:12] a lot of hallucinations and increases [5:15] our accuracy rates for resources that we [5:18] that we're able to provide. And then, [5:21] um, in our post-processing, right? So we [5:23] have like the the final layer is kind of [5:25] we're having this interaction with the [5:27] user and then we have to um see what is [5:30] the response that's going to go back to [5:32] the user and is that response going to [5:34] be helpful? Is it going to is it going [5:35] to be aligned with our values and the [5:37] rules that we've already set. And so [5:38] there's like an extra filter or [5:40] checkpoint where if it's not actually if [5:43] the response is not actually aligned, [5:45] it's going to swap out the draft [5:48] response and give it actually a response [5:50] that is going to align with our [5:52] priorities which are keeping users safe, [5:55] directing them to emergency services if [5:57] they need help and things like that. [5:59] Right? So I mean I think this is a huge [6:02] a huge space like AI therapy is [6:07] prolific and likely only to continue to [6:10] grow and I think um building in some of [6:13] these [6:15] principles these trauma-informed [6:17] principles will help um will help people [6:21] both in terms of getting services and uh [6:25] not [6:27] getting in too deep let's say with [6:30] another type of chatbot. So, I wonder um [6:33] if you can tell me a little bit more [6:35] about um I heard you use the term [6:38] co-creation. Um so, I wonder if you can [6:40] tell me about that co-creation that is [6:42] going on um with with Ruth. [6:46] >> Yeah. So, there, you know, part of being [6:48] uh trauma-informed principles is is [6:51] actually co-creation with people that [6:53] are the your users, right? So, this is [6:55] like kind of a human- centered design um [6:58] process. if you know that. But we are [7:01] actually building out a systematic uh [7:04] process where there's going to be user [7:07] testing people that are lived experience [7:09] experts and clinicians and um advocates [7:13] and people that would actually use this [7:15] and understand it from the field level. [7:18] uh would be a part of kind of the [7:20] testing but also the iterative processes [7:22] and the feedback and things because it's [7:24] about you know we're we're not building [7:26] something that's perfect immediately [7:27] like the first time you build it right [7:28] that's not that's not how tech works. So [7:30] we're iterating and improving the model [7:33] with that feedback. And then it also [7:35] builds trust with the with the [7:38] technology itself, right? So if your [7:41] technology is kind of survivorled, um [7:44] our organization is survivorled by the [7:46] way um and female owned. And so that [7:50] actually builds a lot of trust with [7:51] organizations that were integrating Ruth [7:54] into their um into their workflows. [7:58] I was just about to ask you about trust. [8:00] Um I think you know that's not how most [8:04] uh a lot of products not just tech [8:07] products it feels like these days are [8:08] designed um with trust and with privacy [8:12] in mind. Um can you what do you have to [8:15] say about that like and why y'all are [8:17] doing it this way? [8:19] >> Yeah I mean this goes right into kind of [8:21] how we're different than other LLMs, [8:24] right? I think so, you know, right away [8:27] if you use anybody's it's free to use [8:29] Ruth on our website for anybody. Uh we [8:31] are nonprofit. We will keep that free um [8:34] because we want people to be able to [8:36] access uh Ruth. And so the thing is is [8:39] like a lot of um chat bots that are out [8:42] there, you have to create an account, [8:43] right? You put your email in, you put [8:45] your name in um and then the account's [8:48] created. Well, guess what? Now [8:49] everything, all the chats that you have [8:51] are connected to you. Okay? All those [8:54] chat transcripts are visible on the back [8:56] end by the company who's who's created [8:59] the LLM. Okay, that's across the board [9:02] true. And so there's parameters that you [9:04] can set in place for like how long [9:06] you're going to collect, how long you're [9:07] going to keep those transcripts active [9:09] or not, um before you delete them and [9:12] things like that, who has access to [9:14] those. Um why do you actually access [9:17] them? All of those uh features are [9:20] things that should be transparent to [9:22] your user. They should be easily that [9:24] information should be easily accessible [9:25] to the user to give users the ability to [9:28] decide whether or not they feel safe [9:30] using this particular chatbot for the [9:33] for these reasons. Uh so so for us like [9:37] some of the things that's different is [9:38] we don't collect we don't collect your [9:40] uh user information. Uh we don't store [9:42] that. We don't collect any kind of PII [9:44] personal information. Um we don't have [9:47] you open an account. You can just go in [9:49] and start using Ruth. U we don't we have [9:51] analytics on the back end so we can see [9:53] like how many people signed in like how [9:56] long that they were on a chat and things [9:57] like that but we can't actually um [10:01] identify the IP address that you're that [10:04] you're um using it from because we're [10:06] not using um I don't know if I could say [10:10] it but we're not using Google Analytics [10:12] right so Google Analytics will help [10:13] you'll track the IP we don't use that [10:15] that's not our that's not the platform [10:17] we use so we're very much intent our [10:19] intention is to keep it as private um [10:23] and secure as possible for the users. [10:26] And then we have like very strict um [10:28] right now we have a 3-day uh uh [10:31] transcript hold before we delete them. [10:33] And part part of that is because the [10:34] GDPR, the the EU is um they're, you [10:39] know, they're about to be implementing [10:41] their three day three days uh holding on [10:44] the transcripts for the for LLMs. Now, [10:47] just as a quality review, quality [10:49] assurance. And to be honest, if you're [10:50] doing crisis response, you kind of want [10:52] to have a little bit of a window so that [10:54] you can make sure that your bot is [10:56] responding in a helpful way, not a [10:58] harmful way. But you have to be really [10:59] like cautious about how you implement [11:01] that. Um the other things that that um [11:05] that are really special about Ruth that [11:07] are different is that we have a quick [11:09] exit button because it's actually [11:11] designed for people that are [11:12] experiencing abuse u and crisis. And so [11:15] if you're using Ruth, you can quick exit [11:17] out. it's going to erase the the um it's [11:20] going to erase the URL from your history [11:22] and it's going to like exit you out to a [11:25] neutral website that's going to be not [11:27] able to identify that you've been using [11:30] this type of chatbot for support to [11:32] maybe somebody that's that's abusive [11:34] that's like watching over your shoulder [11:36] or something right um so we have kind of [11:39] these different features that u are in [11:42] place what else oh when you sign on to [11:44] Ruth it actually gives you the This is [11:47] are you using a secure device? This is [11:49] how you know if you are. This is how you [11:52] be use a secure device if you're not. Um [11:55] this is an AI. So identifying being [11:57] transparent about that. It might not be [12:00] 100% accurate, you know, but we need to [12:04] understand it's not human, but we will [12:05] help you get to humans, right? Um, and [12:09] really just kind of being transparent [12:10] about like what what the limitations are [12:13] and um, and just kind of always [12:15] returning back to are you safe? Right. [12:19] >> Let's talk a little bit more about [12:20] training because I think it's so [12:23] interesting that you're not holding on [12:25] to people's data for years and years. [12:29] And one thing I have come across in my [12:31] own work is you know sharing with people [12:34] you know anything you share with chat [12:35] GBT like could be used against you. So [12:39] be careful with what you're sharing. [12:41] Don't reveal too much that could be [12:42] traced back to you if there's some kind [12:44] of hack. Um so how how do you approach [12:48] training a model like this? [12:53] >> How do you so those are kind of two [12:55] different things though, right? How do [12:56] you approach training it so that the Oh, [13:00] in the sense of um because we're not [13:04] training it on the chat. Okay. So, you [13:07] know, training is really like we use [13:09] repeatable sources, research and things [13:12] like that to be able to um train. It's a [13:15] rag model right now. we're transitioning [13:17] to fine-tuning, but it's about using the [13:20] right kinds of resources and information [13:22] on being trauma informed. Um, and being [13:25] able to then have that combined with our [13:28] our own prompt protocols and priorit [13:31] prioritization. [13:33] Yeah. Let's talk about regulation [13:36] because you know there seems like [13:38] there's no white knight coming to save [13:40] us and so we're on it's kind of on every [13:44] organization to approach regulation in [13:48] their own way. So how do how do you all [13:50] think about it [13:51] creating guardrails? [13:54] >> Yeah. So we're not I mean we are [13:57] building a technology but we're also [14:00] want to expand this ecosystem of [14:02] responsible AI and so we're also [14:04] building and developing a [14:07] trauma-informed standards of care kind [14:09] of framework for AI. So we're able to [14:11] work with other AI AI companies to be [14:14] able to help them build out their [14:16] trauma-informed um protocols and things [14:19] like that. I mean, honestly, like like [14:21] you would hope that the developers out [14:23] there understand that if if somebody [14:25] jailbreaks your AI and gets it to, you [14:28] know, tell somebody to kill themselves, [14:29] that's really bad for you, right? That's [14:31] going to send you to court. There's [14:33] going to and it already has um a lot of [14:36] the the LLMs out there now are in court [14:38] for these reasons. And so, you would [14:41] want to you hope that people are going [14:43] to start self-regulating, right? [14:45] developers are going to start [14:46] self-regulating and trying to make sure [14:49] um but the reality is it just it's going [14:51] to take a lot of um awareness and kind [14:55] of I think that we're starting to see [14:57] more of that like push towards more [14:59] responsible AI and like you know from a [15:02] regulatory perspective I would love to [15:05] be able to see policy makers to start [15:07] thinking about how do we engage or like [15:10] how do we regulate AI to be able to um [15:13] kind of by default um meet the needs of [15:17] the most vulnerable people among us, [15:19] right? And do the least amount of harm. [15:22] >> Definitely. Um okay, we don't have a ton [15:25] more time left. So, I wanted to ask you, [15:28] what advice do you have for people here [15:32] who maybe want to imbue their AI with [15:36] some of these principles? Um what [15:38] advice, what things to avoid, anything [15:42] for them? So, if you're just a if you're [15:45] a user of a of any kind of LLM, I would [15:48] say suggest using the one that makes [15:50] sense for your purpose. If you're [15:51] looking for crisis support or something [15:53] like that, use the one that's built for [15:55] that because they don't all have the [15:57] right kinds of guard rails in place. Um, [15:59] but if you're developing any kind of [16:02] generative AI, please contact us. we [16:05] would love to help you like design it um [16:09] and really make this a responsible AI [16:12] that's not going to act and behave very [16:14] badly. [16:17] >> Okay. Well, great. Uh thank you so much [16:20] for joining me today. It was great to [16:21] talk with you. [16:22] >> Thanks so much, Abby. Thank you. [16:24] >> Thank you all. And thanks everyone.