Cornell's Yang Urges AI Labs to Prioritize Harm Audits Over 'Safe' Apps

康奈尔大学杨倩呼吁AI实验室优先开展危害审计，而非仅开发“安全”应用 开发独立、无害的AI原型在商业市场激励面前往往失效，需要研究人员设计出群体规模的危害度量工具，以引导公众舆论并为未来的监管打下基础。 杨倩对传统的人机交互研究路径提出批评，认为仅靠构建孤立的“无害”AI原型并呼吁政策制定者予以推行，因与市场激励脱节而屡屡受挫。结合智能手机过度使用、健康政策的历史案例以及对使命驱动型创业者的访谈，她提出技术人员应将系统构建视为“播种”，去设计能够大规模量化并缓解潜在社会危害的工具。以心理健康和生成式AI为例，她展示了纵向对话追踪与认知心理学指标如何识别诸如阿谀奉承和社交韧性下降等隐性危害。最终，她呼吁技术研究人员超越表层的提示词工程，专注于构建稳健的度量基础设施，从而系统性地提升公众意识、改变商业可行性，并确立监管基准。 学术界“AI向善”的努力面临严重僵局：商业服务提供商因用户参与度驱动的商业模式而拒绝引入增加阻力的替代原型，而政策制定者则忽视了实施强制监管的要求。 (4:08) 人机交互研究人员应从设计独立的无害应用转向开发能够度量和缓解危害的系统，将技术产物视为能够系统性改变政策、政治和商业模式的种子。 (9:03) 在群体规模上检测生成式AI的社会危害面临着巨大的财务窘境：在数以百万计的多模态日志上运行大模型推理所需预算高昂，但在缺乏危害的既有证据前，公共机构不会为此提供资金。 (27:15) 受控实验表明，阿谀奉承的聊天机器人会虚假抬高用户的自我评价并降低其社交意愿，随着时间的推移削弱个人的心理韧性。 (46:15)

Qian Yang argues that HCI researchers must abandon the futile cycle of building standalone 'harm-free' AI prototypes and petitioning ignored policymakers, pivoting instead toward engineering harm-measurement systems that seed shifts in public opinion, regulatory agendas, and business models. Breaking the Deadlock of Harm-Free Prototypes (4:08) Within human-computer interaction research, a conventional response to commercial generative AI harms—such as tools that deaden critical thinking or creativity—is designing bespoke alternative prototypes and urging policymakers to mandate their adoption. Yang observes that this pipeline consistently fails in practice: commercial model providers are driven by user retention and monetization, everyday consumers gravitate toward low-friction tools, and legislative bodies routinely ignore academic appeals. Furthermore, banning or substituting tools without addressing root dependencies can backfire; Yang points out that abruptly outlawing AI companion chatbots in certain regions caused users to experience deeper loneliness rather than reconnecting with peers. Drawing on historical precedents and democratic theory, Yang contends that policy cannot outpace public sentiment. Unless the public consciously understands that pervasive chatbot usage impairs mental health, learning, or creative capacity, lawmakers will not enforce radical mandates. Conversely, when public awareness is mature, citizens voluntarily adopt protective behaviors without needing government compulsion. Researchers must therefore recognize that policy intervention is subordinate to grassroots public opinion rather than an independent, top-down lever. Humble Measurement Systems as Policy Catalysts (9:03) Instead of constructing sanitized consumer applications, Yang urges the technical community to focus on engineering diagnostic systems that measure societal harm at scale. Looking back at early smartphone overuse, HCI researchers identified problematic digital behaviors years before mainstream debate caught up. Crucially, they did not begin with platform overhauls, but developed meaningful subjective indicators—such as whether users evaluated their screen duration as 'time well spent'—which empowered individuals to self-monitor and alter their habits long before formal regulatory action emerged. Because societal harms unfold over slow, multi-year cycles while political windows open and shut rapidly, an empirical evidence base must already exist when political opportunities arrive. While technical prototyping takes only months, shifting broad public opinion takes years. By treating diagnostic and harm-mitigation systems as 'seeds,' researchers can translate elusive, pre-causal risks into actionable metrics that empower user self-protection and provide the empirical foundation required to shape future legislation. The Inference Bottleneck and Commercial Deadlocks (27:15) Applying continuous harm monitoring to generative AI encounters a major structural hurdle: inference cost. Unlike prior technological eras where digital behavior could be audited with lightweight client-side scripts, detecting nuanced behavioral harms across millions of natural language interactions requires running large, computationally expensive models. This dynamic creates an institutional catch-22: frontier AI labs lack commercial incentives to fund third-party auditing of their own models, governments refuse to allocate tax dollars without public pressure, the public cannot pressure lawmakers without empirical proof of harm, and independent researchers cannot assemble that proof without compute budgets. To discover pathways out of this funding stalemate, Yang’s lab examined mission-driven entrepreneurs navigating the structural hurdles of the U.S. healthcare system, where short-term employer health plans disincentivize preventative wellness. Commercial ventures that attempted to build clinically grounded mental health chatbots and pursue FDA approval have consistently failed. Studying founders who pivoted across hospitals, consumer app stores, employer productivity tracking, and even underwriting their own insurance carriers revealed that pioneering viable economic models is just as critical as technical system design. Quantifying Cognitive Harms and Interpersonal Friction (46:15) Empirical studies from Yang’s lab reveal that sycophantic chatbots induce subtle psychological shifts: brief interactions can inflate users' self-assessments, encourage feelings of superiority over peers, and diminish prosocial intentions to connect with real humans. Analyzing two-year longitudinal ChatGPT conversation logs using cognitive psychological metrics, her team identified recurring behavioral patterns where interactions gradually degraded personal resilience to real-world stressors. Her research also highlights why users retreat to conversational agents: interpersonal communication during personal identity crises often breaks down into either excessive vagueness or combative bluntness. Consequently, Yang argues that technical interventions should focus on building human-to-human communication skills before attempting to strip AI away. Because public policy and legal statutes only establish a minimal floor of compliance rather than proactive ethical standards, HCI researchers must prioritize validating psychological proxy metrics and conducting comprehensive evidence syntheses to effectively shift public understanding. Yang concludes that while systems builders traditionally relegate policy, economics, and public opinion to post-publication 'broader impacts,' these factors must become primary design goals if AI is to serve the societal good. Significant unresolved challenges remain, including securing sustainable funding for continuous population-scale AI audits and balancing the nuance of scientific research against the simplified messaging often required to mobilize public action.

Qian Yang在斯坦福人机交互研讨会上提出，人机交互领域试图通过开发孤立的“无害”替代原型并呼吁立法的传统模式已陷入停滞；研究人员应转而构建能够在大规模人群中测量并缓解生成式AI危害的系统，以此作为“播种”杠杆，驱动公众意识、政策制定与可持续商业模式的深层变革。 干预路径破局：从无害应用转向播种系统 (4:08) 在当前人机交互（HCI）研究中，学者发现商业AI（如ChatGPT或Claude）存在抑制批判性思维等潜在危害时，往往倾向于设计具有反思性摩擦的替代原型，并呼吁政策制定者强制推行。然而，商业模型优先考虑用户留存与变现，普通用户倾向选择低阻力工具，政策制定者也普遍对学术界的呼吁缺乏响应，导致设计人员陷入无力感。Yang指出，仅凭技术构建无害的替代应用无法扭转系统性危害。研究界必须发生范式转移，停止开发孤立的纯洁型消费软件，转而将工程力量投入到具备危害检测与缓解能力的系统中。这些系统应被视为战略杠杆（即“播种”），旨在揭示隐性危害、影响立法议程并催生新的商业机制。 历史经验启示：谦逊度量工具的战略价值 (21:27) 回顾智能手机防沉迷与环境空气监测等历史案例，Yang指出解决下游技术危害遵循着两阶段时间线：技术原型的迭代与测试通常仅需数月，但扭转公众认知并等到政治窗口开启则需要数年。如果等到政治时机成熟才开始收集危害证据，关键人物往往已经离任。因此，研究人员在早期阶段的关键作用是构建能够追踪隐性危害的“谦逊度量工具”。例如在手机过度使用问题上，早期研究并未等待临床成瘾病例或直接监管，而是定义了主观代理指标（如评估屏幕时间是否属于“充实利用的时间”）。这种早期的低成本度量工具使用户能够自我监测，并为数年后政策与监管的落地提前储备了充足的人群级实证数据。 算力成本死锁与商业模式探索 (27:15) 将历史度量经验迁移至生成式AI时，研究界面临严重的推理与评估成本壁垒。分析数以百万计的自然语言长文本或多模态日志以捕捉微小的心理健康变化，本身就需要消耗庞大的生成式模型算力。这引发了制度层面的死锁：前沿模型厂商缺乏出资审计自身危害的动机；独立研究者依赖公共财政预算，但民主政府需要公众压力才会拨款；而公众在缺乏大规模实证证据时不会施压，研究者又因缺乏资金无法运行监测基建来生产证据。为破解这一僵局，Yang团队考察了数字健康领域的创业实践。面对美国医疗体系缺乏预防性激励的现实，部分初创团队尝试将软件指标与企业病假率挂钩，甚至探索自建商业保险实体，试图以制度破局为公益性技术开辟可持续的支付路径。 长期交互危害：阿谀倾向与认知韧性侵蚀 (46:15) Yang的实验室通过分析受试者长达两年的ChatGPT连续对话记录，运用认知心理学代理指标评估长期交互对个体心理韧性的影响。受控实验表明，阿谀奉承（sycophantic）的对话模式会虚增用户的自我评价，使用户误以为自己优于同伴，从而显著降低真实社交意愿。调查同时发现，人们在经历身份危机时由于难以在人际沟通中把握尺度，往往会退缩到顺从的聊天机器人中寻求慰藉。然而，业界此前尝试打造合规、非阿谀并寻求FDA认证的心理健康聊天机器人企业已几乎全部破产。这表明界定有害阿谀与友善对话的法律与技术界限极为复杂，单纯剥离AI工具若缺乏提升真实人际沟通能力的配套干预，反而可能加剧用户的孤立感。 政策底线局限与跨学科证据整合 (51:29) 在政策监管与实施层面，Yang提醒研究界保持清醒：公共政策和法律仅能设立可被允许的行为底线，绝不能等同于伦理与社会效益的最优标准。当前生成式AI的诸多危害由于缺乏标准化的认知心理学代理指标，方法学上的指标开发与验证必须先于游说行动，否则数据无法有效引导公众舆论。针对学术界普遍存在的零散研究现状，Yang赞同应减少单纯展示提示词工程（Prompt Engineering）使人更具创造力的常规论文，倡导研究人员将精力转向大规模的元分析与证据综合（meta-synthesis），通过系统性证据库与智库合作，以连贯的叙事合力撬动公众认知和政策变革。 Yang总结强调，改善生成式AI的社会影响其难点不在于技术界面或提示词调优，而在于技术系统之外的商业激励、资本结构与政策机制。如何在缺乏公共资助与商业动力的双重阻力下，建立并维持长期的人群级危害监测基础设施，仍是HCI研究人员与系统构建者面临的核心未决挑战。

[17:08] Qian Yang: In a democracy, at least in theory, policy actually cannot be too out of step with public opinion.

[0:10] Good afternoon, everyone. [0:11] Um I'm Chen. I'm um giving a talk on [0:16] a particular sweat of my lab's research [0:18] focusing on designing AI for social [0:20] good, and I will jump straight in. [0:23] Um [0:24] this line of work was initially [0:26] motivated by [0:28] so a lot of concerns we hear in popular [0:31] media about how generative AI is going [0:34] to impact our society, not just in [0:36] individuals or specific user groups or [0:39] communities, but uh impacting our [0:42] society at large. Um [0:44] >> [snorts] [0:44] >> How many good jobs are there going to [0:46] be? How How many people are talking to [0:49] chatbots instead of people? Um how many [0:52] people are having sort of inflated [0:56] self-understanding because they're [0:57] talking to sycophantic chatbots every [0:59] day? [1:00] Those influences, although quite small [1:02] if you look at each individual user and [1:06] each individual interaction, but over [1:08] time and over sort of [1:11] in population scale, these small impacts [1:14] actually add up pretty quickly. [1:16] Um [1:17] and as we've learned from social media's [1:20] impact on users' mental well-being, we [1:22] know those like small, slow-burning, [1:25] incremental influences cannot be [1:27] neglected. They are actually [1:30] quite impactful. Um and that's why in [1:34] this line of work, uh my students and I [1:37] are mainly looking at these impacts that [1:39] are not just impacting particular [1:42] vulnerable groups or user populations, [1:44] but really kind of how does how does uh [1:47] generative AI impact society at large in [1:50] this kind of [1:51] um pervasive way. [1:53] Um [1:54] but then we also realized an inherent [1:56] challenge in studying this kind of [1:58] impact is this kind of impact is really [2:00] hard to measure unless that impact has [2:03] been entrenched deeply and widely in [2:05] society, just like social media's [2:07] impacts on [2:09] user mental well-being. [2:11] Um [2:12] but we have to measure it. We have to [2:14] try to [2:16] take actions on it because again, if we [2:19] learn anything from social media and [2:21] mental well-being, if we wait until we [2:23] can fully measure and understand those [2:28] technologies impacts on [2:30] well-being and society at large, by the [2:32] time we can measure it, attribute it, [2:34] it's probably too too late. The change [2:37] has been so deeply entrenched that we [2:39] probably have missed a real a really [2:41] important opportunities to to course [2:43] correct. So, we set out to [2:47] try to work on these sort of really [2:50] elusive problems [2:51] from the beginning and our goal is to [2:54] say, can we start acting on those [2:57] societal impact of generative AI even [3:00] though we don't fully understand them. [3:02] Can we start working on them, taking [3:04] action, [3:05] be proactive before we fully understand [3:08] those impacts? [3:10] And that's the overarching goal of this [3:13] sort of set of work I'll be presenting [3:15] today. [3:16] Um [3:17] here is the same idea represented in a [3:19] sort of a graph form and I'll actually [3:22] point back to this same graph repeatedly [3:24] throughout this talk. Um we take as a [3:27] starting point like we suspect there is [3:29] some sort of societal harm that [3:31] generative AI is causing, be it mental [3:34] well-being, loneliness, social [3:37] connection, be it impacts on people's [3:40] creativity, job quality, and so on and [3:43] so forth. Let's say we suspect there is [3:46] some sort of harm that's happening but [3:48] we cannot quite um [3:51] put our fingers on it. we cannot quite [3:53] measure it, but um [3:55] can we go from that suspicion and figure [3:58] out some way to arrive at an action that [4:01] we can take today? That's the goal of [4:04] this line of research. [4:06] Um [4:08] There are of course a lot of researchers [4:10] already started looking at this. Um [4:12] and here is my brutally simple [4:14] simplified um summary of what they [4:17] currently do. Um they look at um they [4:20] they have a suspicion that there's some [4:21] sort of societal harm that's happening, [4:23] so they would then build an alternative [4:26] generative AI application that's free of [4:29] that harm. So, for example, if I suspect [4:33] Claude is um automating people's [4:35] thinking, people are not being as [4:37] creative or as cognitively engaged as [4:39] they are, I build an application that [4:41] helps people think, pushes people's um [4:44] cognitive engagement, uh improves their [4:47] creativity. And then, um people also, I [4:51] think scholars and designers of these [4:54] systems also recognize, well, Claude and [4:57] ChatGPT are probably not going to adopt [4:58] my design cuz they need the [5:01] the user engagement and the income that [5:04] the the um [5:06] money that comes with it. So, they're [5:08] probably not going to adopt my system uh [5:11] proactively, and users are probably also [5:14] kind of lazy. Maybe they wouldn't want [5:16] you use my uh system either. So, in [5:20] order to achieve to actually solve this [5:23] um [5:24] societal wide harm, I should actually [5:26] ask policy makers to adopt my system. Uh [5:29] wouldn't it be nice um if um public [5:33] policy, if lawmakers require everyone to [5:36] use my um better designed AI system so [5:40] that no one is harmed um by those [5:42] commercial chatbots. [5:45] Um [5:46] It's in that even uh it's in a sense [5:48] we're seeing in the past few years, um, [5:51] a radically, um, increase in, [5:55] um, the amount of papers that mention [5:57] policy, um, [5:59] in in HCI conferences. More and more [6:01] people, after they propose a system, in [6:03] the discussion they say, "I call for [6:05] policy makers to to do something about [6:08] it, um, to to to endorse my system or to [6:11] adopt some of the system designs." [6:14] Um, [6:15] unfortunately, those calls are very [6:17] rarely, [6:19] um, acted upon. Not many policy makers, [6:22] that is to say, very rarely, uh, policy [6:24] makers actually acted upon, um, those [6:27] calls. Um, [6:29] so, [6:31] we keep seeing more and more researchers [6:33] calling policy makers calling for policy [6:35] makers to adopt their design, but policy [6:37] makers are not doing anything about it. [6:39] We're not seeing organizations or, um, [6:42] large [clears throat] companies like [6:43] Cloud or or Open AI adopting the, um, [6:47] Open AI or Microsoft to adopt these [6:48] designs. So, we're kind of [6:50] facing this we're we're kind of in this [6:52] deadlock. We're like, "Okay, do we do I [6:54] just keep building? I can't build more [6:56] systems that help with people's [6:58] creativity and mental well-being. Should [6:59] I keep building?" Um, [7:03] I think a lot of researchers are [7:04] probably still doing that and I'm one of [7:07] the most also. [7:08] >> [laughter] [7:08] >> Um, [7:09] we're on the same boat here. Um, but [7:11] there are also some designers, some [7:13] researchers, um, [7:15] including designers in the industry and [7:17] say, "Hey, why do I keep designing these [7:19] systems, [7:21] um, when I know it won't make a [7:23] difference in the real world? When I [7:24] know the vast majority of the users are [7:27] going to still use the most available, [7:29] um, commercial, [7:31] uh, LM chatbots out there." [7:33] Um, [7:35] and we're seeing that, um, in some of [7:37] the empirical work in HCI where people [7:40] studied designers in the industry and [7:43] they noticed there's this just [7:45] increasing sense of hopelessness and [7:48] identity crisis in the industry where [7:50] people say, "Well, I recognize how [7:53] complicated this problem is in order to [7:56] reverse some of these [7:58] to to to understand and reverse some of [8:00] these generative AI societal impact or [8:02] AI's impact society impact in general, [8:05] I don't quite know what to do. There is [8:07] money involved, politics involved, [8:09] policy involved. What do I do?" [8:12] Um [8:13] I I particularly like the this paper's [8:16] title um [8:18] uh from a few years uh from Kai a few [8:20] years ago. Although not talking about [8:22] generative AI directly, but I feel like [8:24] it summarizes that sentiment that I hear [8:26] from um society-minded designers uh very [8:30] often. The The This paper's title reads, [8:33] "My sense of morality leads to my [8:35] suffering, battling, and arguing." Like [8:38] I want to do something good for society, [8:41] but I couldn't don't quite know how to [8:43] do that. And that sense of [8:45] um [8:47] not quite knowing what to do, the [8:49] current dis- dissatisfaction [8:51] with the our current approach to address [8:53] generative AI's unfolding societal [8:56] impact is the kind of a problem [8:58] um that my work my group's work tries to [9:01] tries to address. So, the central [9:03] question of this talk is really how do [9:05] we break this deadlock? How can we come [9:07] out of this impasse? [9:09] Um and I ask this question particularly [9:13] from the perspective of a technology [9:15] designer, as someone who cares about [9:18] social good, who wants to build systems. [9:20] Still, I understand there's a lot of [9:23] value in turning myself into an activist [9:26] or um start start my activism group or [9:30] start to lo- [9:32] be a lobbyist maybe to influence policy, [9:34] but I never do less want to hold on to [9:37] my expertise as a system designer, as [9:39] someone who's [9:41] um skills uh central skills are in [9:43] bridging human thinking and system and [9:46] making AI things. [9:48] So, [9:49] when we ask this question, we very much [9:52] focus on like [9:53] what's the right thing to build there? I [9:55] know there's value in activism and [9:57] policy work, in outreach. [10:00] Um are there things we can do as [10:03] designers, as a people build AI things? [10:06] Um it's in that um context, I'll talk [10:10] about a few pieces of work. The first [10:12] part of this talk um is entirely [10:15] empirical work, where I ask this [10:17] somewhat philosophical question of what [10:19] is the right thing to build? If my goal [10:22] is to be able to make a difference, be [10:25] able to move the needle in terms of [10:27] addressing generative AI societal [10:30] impact, what should I even do um before [10:33] I even build anything? Um so, I'll [10:35] present two uh empirical projects that [10:38] looks into this question. And then [10:39] afterwards, I will share um a few [10:42] snapshots of the design work we're [10:43] currently doing to show you how that new [10:46] understanding has changed what we design [10:49] and how we design AI systems. [10:51] Um [10:54] I'm going to give you the takeaway [10:57] right away. Um throughout these [10:59] projects, I want you um the takeaway is [11:02] this. I think we should stop Well, stop [11:05] we should build fewer [11:08] harm-free generative AI applications. Um [11:11] instead, we should build what I call [11:13] harm measurement and harm mitigation [11:16] systems. So, [11:18] um systems that instead of trying to [11:20] build a generative AI application that [11:22] doesn't cause mental health harms or [11:24] doesn't harm people's creativity, we [11:27] actually should spend more of our energy [11:29] on measuring that harm at a societal [11:32] scale. [11:34] And on top of that, when we build these [11:37] harm measurement and mitigation systems, [11:39] we should also start to think about how [11:41] those systems can [11:44] seed changes in public opinion, [11:48] seed changes in policy, and seed new [11:51] business models. [11:53] Um now, for those in the audience who [11:56] perhaps come from kind of more [11:58] system-building background, they look at [11:59] this and say, "Well, I public opinion, [12:02] like that's not what I do for my work, [12:04] for my research." Like uh [12:06] business model, policy impact, these [12:09] seem like broader impacts. Like I'll [12:11] figure those out after I build my system [12:13] and publish my paper. Um I hope through [12:16] this talk I can convince you that no, in [12:19] fact, if you want to make a difference [12:22] in addressing these uh gen AI societal [12:25] impacts that's unfolding today, we [12:27] should actually not only consider these [12:29] factors early on in our system-building [12:32] process, [12:33] but treat them as [12:35] uh some of our most important design [12:37] goals and success criterias. Cuz unless [12:40] we can seed larger changes in policy, [12:43] pol- uh pol- politics, and business [12:46] models, we can't our individual systems [12:49] um [12:50] will face a hard time uh reaching kind [12:53] of societal scale impact. So, it's in [12:56] essence uh [12:58] for the for folks in the audience who [13:00] are perhaps looking for more of an STS [13:02] or business or um ethics talks, um [13:05] unfortunately, this talk is not really [13:07] designed for you. [13:08] >> [laughter] [13:09] >> Um despite um [13:11] the fact that this talk will involve a [13:13] lot of the business and societal kind of [13:15] uh policy questions, um [13:17] the the I very much designed this talk [13:20] to orient towards um to towards people [13:22] who build systems. [13:24] All right. [13:25] Um [13:26] let's jump right in. Um, the first [13:29] project I'm going to present um, [13:32] it's kind of a a simple one that just we [13:34] just want to learn from history. Like, [13:36] if the thing if the problem we're [13:38] running running into now is we build all [13:41] these amazing systems that the uh, that [13:43] don't cause any societal harm actually [13:45] can make the world a better place. [13:48] Um, and but it was why why can't we [13:50] influence policy? So, can we learn from [13:53] anything like in the history? Like, what [13:56] kind of technologies actually changed [13:58] policy, actually got adopted widely, got [14:02] enforced at the societal scale? [14:04] That's the question we set out to answer [14:07] in this project. Um, as with any project [14:10] we started a lot of reading, literature [14:11] review. Um, [14:13] I named a few examples here to show you [14:15] the range of readings we did. We read a [14:17] lot we read a lot of um, science and [14:19] technology studies which are which is [14:22] a field in the [14:24] social in the social sciences that try [14:26] to retrospectively analyze how [14:29] technologies changed the society. So, we [14:31] looked into those um, scholarships to [14:34] understand what technologies did [14:36] influence policy and influence society. [14:39] Um, [14:41] during that literature review process we [14:43] realized that actually HCI did a lot of [14:45] great work in influencing policy like [14:48] um, [14:48] privacy, like um, accessibility, [14:52] uh, [14:53] social media, content moderation. Um, [14:56] but however, the HCI researchers don't [14:58] always [14:59] publish those like, how did you change [15:01] policy? What did you do? Um, we don't we [15:03] actually don't know that. So, um, in [15:06] 2023 [15:07] at CHI we organized a workshop that [15:10] specifically focused on asking everyone [15:12] to share their story, their engagement [15:15] with policy world. And both success [15:17] stories and you know, frustrations and [15:19] failures. Um, we got uh, 80 plus [15:22] participants, researchers from across [15:25] five different continents. Heard a lot [15:27] of amazing stories and their stories [15:29] informed um, the findings of this [15:31] project greatly. [15:33] Um, [15:35] we learned all these lessons. We know, [15:37] we now know what happened in the [15:38] history. [15:40] Um, next we looked into do these [15:42] historical lesson lessons apply to [15:45] generative AI? Like, can we actually use [15:47] those lessons to inform what we do today [15:50] to address generative AI's still [15:52] unfolding societal impacts? Um, [15:55] so, last year we organized a thought [15:57] summit which was like a small week-long [16:00] workshop at an Innisbrook, New York [16:03] where Cornell is. Um, we invited experts [16:06] in mental health, mental well-being, [16:08] tech policy, health policy, um, LLMs and [16:13] uh, [16:14] probably oh, VCs, investors, and startup [16:17] founders. So, all kind of academics, so [16:19] all kinds of people interested in this [16:22] question of LLM chatbots impact on uh, [16:25] populational mental well-being. [16:27] I would gather them together and sort of [16:29] start to look into how how these lessons [16:31] may or may not apply to generative AI. [16:36] In addition to learning from experts, we [16:38] also did a lot of hands-on exploration [16:41] trying to apply the lessons we learned [16:43] from past technology [16:46] um, that did influence policy. Trying to [16:49] apply those lessons to generative AI's [16:51] various impacts, um, ourselves. Um, not [16:54] all of them led to publications as you [16:56] could imagine, um, but we learned a lot [16:59] in the process sort of delving our um, [17:02] into various areas like creativity, [17:04] mental well-being, learning, and so on. [17:07] Um, [17:08] so, that's the sort of somewhat messy [17:10] and well-lensing research process. Um, [17:13] and I want to share where [17:15] our learning sort of have brought us so [17:16] far. Um, before I talk through the [17:19] findings, just recall that this is the [17:22] diagram that I pointed to you in the [17:23] beginning, the pathway that many [17:25] technology designers today envision for [17:28] as the pathway to address generative [17:31] AI's societal impact. So, if they're [17:33] concerned about the potential harm, [17:35] they take two steps. First, I design a [17:38] less problematic application or more [17:41] beneficial application, and then I call [17:43] for policy changes to enforce that [17:45] design. [17:46] Um, our findings suggest that, however, [17:49] historically, [17:51] yes, these two steps did help, but they [17:54] actually play a much smaller role in the [17:56] much more much longer and much more [17:58] complex [18:00] um, set of sort of pathways. [18:03] Um, [18:06] first of all, um, let's look at the [18:09] harmful uh, like harm-free [18:11] um, [18:12] uh, system design. [18:14] Um, [18:15] if you recognize, for example, a [18:19] teenager is addicted to social media, [18:22] our instinct might be, let me give the [18:25] let me give them a uh, [18:26] social media platform that's less [18:28] addictive. However, if you look at those [18:30] history and also how service currently [18:33] treats these addictive symptoms, they [18:36] would tell you the first thing you want [18:38] to do is to actually make sure that [18:39] person, that teenager, has the ability [18:42] to live and thrive without that [18:45] pleasurable but harmful [18:47] technology in the first place. And we [18:49] see similar um, evidence from LLM [18:51] chatbots as well. Um, there are studies [18:54] um, show that um, in some countries that [18:57] where where um, AI companions were [19:00] banned, after those bans, uh, people [19:03] just people still didn't talk to their [19:05] friends. People just [19:07] were even lonelier. Um, so, the first [19:11] thing we need to do before we take away [19:13] those harmful technologies is actually [19:16] give people the ability to be able to [19:18] live without it. Um so, it's in a sense [19:21] the technologies that don't [19:23] don't cause harm plays only a part of um [19:27] it is only part of an equation. [19:29] Um now, let's look at the policy change [19:32] side. Um I would say policy change as [19:35] much as I [19:36] champion those workshops and called for [19:39] design policy [19:41] collaboration. I actually our studies [19:43] actually suggests policy change also [19:46] plays a smaller role than we initially [19:48] thought. And the reason being in a [19:51] democracy, [19:52] um [19:53] at least in theory, [19:56] um policy actually cannot be too out of [19:59] step with public opinion. So, unless you [20:02] have a public that recognizes, "Hey, we [20:07] need to use less of those chatbots cuz [20:09] they harm my creativity, they harm my [20:11] well-being." Unless you have that public [20:13] awareness and civic understanding, [20:15] policy would it's unlikely that the [20:17] policy would charge ahead and and [20:20] propose some saying propose some radical [20:22] change. It's in that sense that I put [20:26] the policy change block in on the side [20:29] here rather than in the center cuz I [20:31] think these two, public opinion and [20:34] policy change, really come hand in hand. [20:38] Um [20:39] let me punctuate the value of [20:42] public opinion even more. In a way, if [20:46] we have a public, if we have citizens [20:48] who really all recognize the value of [20:52] limiting their use of chatbots to for [20:55] the benefits of their mental well-being, [20:57] for the benefits of their learning, for [20:59] the benefits of their creativity, they [21:01] could have just adopted those better [21:03] designs on their own. They don't even [21:05] need a [21:06] to demand them to require them to do [21:09] that. So, [21:11] public opinion or civic understanding [21:13] plays a huge role in addressing societal [21:17] harms [21:18] in that context. [21:20] Um [21:22] That's the bottom of [laughter] the [21:24] diagram. Um [21:27] Something that I discovered in this [21:29] research um and and people I think I [21:32] think don't talk about this enough is [21:34] the importance of humble technologies [21:37] that uh measure harm. [21:41] Um let me give you a concrete example. [21:44] Um let's say smartphone overuse. In [21:47] fact, um HCI researchers were some of [21:50] the earliest researchers that [21:51] recognized, "Hey, people are using their [21:53] smartphones too much." Way ahead of any [21:56] political momentum, way ahead of this [21:59] popular discourse around smartphone [22:01] overuse. Um [22:04] And when those researchers recognized [22:06] this potential harm, the first thing [22:08] they did is to start to run studies at [22:10] trying to measure smartphone overuse. [22:12] This might sound like a simple [22:14] measurement, but it's actually not as [22:16] straightforward as it might seem. How [22:18] much [22:19] use of smartphone counts as overuse? [22:22] Like, do you wait until someone goes to [22:25] a therapist to count as overuse? That's [22:28] it would be hard to to to ever reach [22:31] that bar. So, the researchers actually [22:33] spend a lot of time sort of studying [22:35] what's the best way to um measure that, [22:37] and um [22:40] they they came up with a various [22:42] measurements like [22:43] do people feel the time they spent on [22:45] smartphones are time well spent? Turns [22:47] out proxies like this are much more [22:49] useful than absolute numbers people of [22:51] minutes people spend on the phone. [22:53] Things like that as the simple [22:55] measurements like that um can help in a [22:59] number of ways. First of all, these [23:01] measurements can first help people who [23:03] recognize a need to self-improve, to to [23:07] to limit their smartphone use to start [23:09] taking actions even without government [23:12] intervention or [23:13] um causal evidence at a population [23:15] scale. [23:16] Um [23:17] >> [snorts] [23:18] >> this evidence over time, by the time the [23:21] right politician came into play came [23:23] into power, by the time there was enough [23:27] political will what when there was [23:29] enough public awareness of this harm of [23:32] smartphone over use, these researchers [23:34] have already accumulated a a bunch of [23:36] evidence like population-level evidence [23:39] to show like, "Hey, here is our evidence [23:41] why this is indeed bad." And that [23:43] evidence helps sort of catalyze the [23:45] policy changes that people have been [23:48] hoping for all along. A little bit [23:49] delayed, but it's sort of um [23:53] really good cuz you need actually to [23:55] start accumulating those evidence early. [23:57] It takes time, but if you start [24:00] collecting those evidence when the when [24:02] the right politician is in place like by [24:04] the time you finish collecting that [24:06] evidence, that politician's probably [24:07] gone. Um [24:11] lastly, it also changes public opinion. [24:14] So, it's evidence like this, [24:15] publications like this, science [24:17] communications like this that sort of [24:19] catalyze [24:21] uh [24:21] public opinion change over time. And [24:24] again, that has sort of an interactive [24:26] effect with policy change as well. [24:28] So, [24:29] in hindsight, um oh, and we see similar [24:32] patterns not just in smartphone over [24:34] use, but in a lot of other technological [24:36] um [24:38] um history as well like civic monitoring [24:40] gadgets. Um they're they played an [24:43] important role in catalyzing certain air [24:45] pollution uh intervention uh policy [24:47] interventions. So, it's in essence in [24:50] hindsight, this these technically quite [24:53] simple um measurement systems actually [24:57] were able to make these otherwise [25:00] slippery elusive [25:03] societal harms can measurable trackable [25:06] actionable. They played a huge role in [25:10] addressing those societal changes that [25:12] are otherwise difficult to act on. [25:16] So, [25:17] that's the picture and [25:20] I want you to draw to attention to [25:24] the time frame [25:26] of this whole process. Um there are [25:29] really two time frame two time scales at [25:32] play here. One is designing technologies [25:35] for measurement. You can run your usual [25:38] HCI research study a few months maybe. [25:41] Um you can get that done. [25:42] Um [25:43] you can design prototype and evaluate [25:45] your harm mitigation system like oh I'm [25:48] going to ask people to reflect on their [25:50] smartphone over use. Those are [25:52] relatively quick. However, to accumulate [25:56] enough of these evidence uh these [25:58] measurements and evidence into um [26:01] something that can create public opinion [26:03] change at large [26:05] and wait until the right political [26:07] moment the political will to accumulate [26:10] those take time. [26:12] They can be really slow but I don't I [26:15] don't think those are reasons that we [26:16] don't start acting on those harms today [26:19] because again [26:20] when the political momentum comes if you [26:23] miss it you miss it for a long long [26:26] time. It would not come again anytime [26:28] soon. So, it's in that sense um in my um [26:32] our findings would suggest that this [26:34] process this whole workflow actually [26:36] should happen in two stages. You first [26:38] build these technologies and you [26:40] actively waiting for the right moment to [26:44] come to to create a societal impact and [26:47] that's why in the title of this talk I [26:49] say we design technologies to seed [26:52] policy and political changes. Those um [26:56] seeds do not um [26:58] blossom immediately, but they will pay [27:00] off at some point. [27:02] Um so that's finding number one. This is [27:04] sort of a nice playbook in terms of how [27:07] historically technologies HCI [27:10] researchers have impacted policy and [27:12] made a made a difference in in society. [27:15] Um next question, does does this [27:17] playbook apply to GenAI, apply to [27:20] GenAI's impact on mental well-being, [27:22] creativity, and so on? Um [27:26] Mostly, with one important caveat. Um [27:30] if we need generative AI to detect the [27:33] harm, [27:35] then this detection itself, this [27:37] measurement itself takes money. And this [27:40] can be like if I want to run log [27:43] analysis and analyze how much uh [27:45] people's conversation with LM chatbots [27:48] impact their mental well-being. It can [27:49] also include things like [27:51] um computing how much uh social media [27:53] impacted or accelerated misinformation [27:56] spread. Because these types of harms are [27:59] in natural language and rich media, you [28:01] kind of need AI models to do detection. [28:04] You cannot just use your [28:06] um off uh offline GitHub repository uh [28:10] to to to run those detections. And it's [28:12] in essence [28:14] you need AI to run those measurements, [28:16] to run those detections. However, [28:19] because you want to reach population [28:20] scale, recall that social media's impact [28:23] on mental well-being, ChatGPT's impact [28:25] on mental well-being are really small if [28:28] you only look at a small population. It [28:30] really relies on large-scale [28:33] uh tracking to show the aggregated [28:36] effect. So that money that cost adds up [28:38] quite a bit. [28:40] Um [28:41] why does that matter? Um [28:44] it matters in this particular part of [28:47] the workflow, and let's take a closer [28:48] look to that. [28:50] Um [28:51] let's say I want to monitor cloud and [28:54] track GPT same pattern people's mental [28:55] mental health and I know there are [28:57] certain costs um attached to um running [29:00] that measurement. Um [29:04] Cloud and open and third party and open [29:06] air probably are not going to pay for [29:07] it. [29:08] Um [29:10] uh it would be somewhat strange to ask [29:12] third party to pay for it. So, most [29:14] likely you're going to ask for [29:15] government funding, public funding to [29:18] fund that basically public mental health [29:20] monitoring program. So, you need some [29:24] sort of policy change or um public [29:27] budget change to fund this work. Uh but [29:30] oops, but recall [29:33] that in a democracy, you need political [29:35] public opinion to enable that kind of [29:39] policy change. [29:40] Um but also recall that you need your [29:43] tool to show that there is harm such [29:46] that you can change public opinion. [29:49] But then I need the money to run the [29:52] to run the measurement to change the [29:53] public opinion. [29:55] So, that's where um [29:58] this this seemingly simple question of [30:00] budget actually creates a deadlock in [30:03] the process of starting the monitoring [30:06] of technology's impact on society. [30:10] Which perhaps also explains why [30:13] historically we have been just so slow, [30:16] so reactive in acting on technology [30:19] societal impact. Cuz there is we haven't [30:22] found a way to come out of this [30:24] deadlock. So, the only way is to wait [30:27] and wait until there is public outrage [30:31] towards this harm that's already [30:33] entrenched. And now finally there is [30:35] political will to allocate budget for [30:38] the monitoring. [30:39] >> Yeah. [30:40] Australia [30:42] This work is funded by the Australian [30:51] >> into [30:52] into like any or policy because I think [30:55] that there's probably a long period [30:56] where [30:57] people maybe like disagree and then like [30:59] there needs to be some kind of [31:00] consensus, right? So, like we try to [31:01] like a narrative like how does that kind [31:03] of look? [31:04] >> I don't think there is a uniform answer. [31:06] That's a great question. There is I [31:08] don't think there is a uniform answer. [31:10] Um, partially because policy public [31:12] policy isn't entirely based on science. [31:15] Um, in some areas they are. In health [31:17] care for example, rough uh [31:20] um, at large there is more of a culture [31:23] of believing in evidence, you know, [31:25] randomized trials, but in tech policy, [31:28] consumer tech policy for example, [31:30] they're not that evidence-based. So, [31:34] consensus plays a role but not always. [31:37] And in essence I would think consensus [31:39] is relatively relatively loosely related [31:41] to this process. [31:44] >> So, can I can I actually ask for more [31:45] follow-up questions? Maybe like a bit of [31:47] a spicier question but like [31:49] >> [laughter] [31:50] >> um, [31:51] how do you square that? But like a lot [31:53] of this is about messaging, right? And [31:54] then a lot of it's like needs like kind [31:56] of [31:57] push the public opinion like in a [31:59] certain way such that like good is [32:00] ultimately done through policy. [32:03] Um, but sometimes like [32:04] the most accurate messaging is not [32:07] necessarily the messaging that will [32:09] actually achieve the most change. Like [32:11] you might want something that's mildly [32:13] inaccurate or like very that's very [32:16] clear to the public. Like how do you [32:18] square this like for yourself? [32:19] >> I'm on the next slide. I'll come back to [32:21] that. Um, [32:24] >> [laughter] [32:24] >> And are they related or I can wait till [32:27] the end. [32:27] >> Yeah, let's wait until the end. Sorry. [32:29] Um, okay. Um, [32:32] where are we? Oh, [32:33] there's this notch. [32:34] Um, we need money to get things going, [32:37] um, especially in triggering the public [32:39] monitoring that we need such that we can [32:42] trigger the range of actions that we [32:44] need for societal change. So, it's in [32:46] essence um [32:48] I modify this in the context of AI [32:52] monitoring [32:53] of societal harms. I would put money as [32:57] So, the first trigger, the first step. [32:59] Now, to the audience question [33:02] um [33:03] if I were [33:05] allowed to make this diagram even more [33:07] complex [33:08] I would draw additional arrows from that [33:11] money icon to policy change. That's [33:14] lobbying. I would draw additional arrows [33:16] from money to public opinion change. [33:19] That's political campaign. So, money [33:22] The key point here is money is actually [33:24] sitting pretty top [33:26] pretty upstream in this whole workflow. [33:28] It plays an incredibly important role. [33:31] Hopefully that addresses the question, [33:32] too. [33:33] Okay. [33:34] So, [33:35] it's in essence [33:37] after this review of history, review of [33:39] HDR research [33:41] this is where we're at. Turns out we're [33:43] pretty good at designing. We already [33:45] know how to design technologies that [33:47] would be able to trigger societal [33:50] actions against pending harms or pending [33:54] impacts of new technology. We just need [33:56] to make the money work. um [34:00] Then, that's the question. [34:03] Our second project looks into And in [34:05] particular um [34:08] we turn our eyes to entrepreneurs and [34:10] intrapreneurs. Um [34:12] Entrepreneurs are people who build a [34:14] technology system into a product to make [34:17] sure someone buys it, pays for it. [34:19] Intrapreneurs are people who do that [34:21] within an existing company. um We looked [34:25] at their [34:26] workflow because [34:29] for people who are actually interested [34:32] in who actually want to start a company [34:35] for social good, they actually also face [34:38] the same conundrum we face, which is to [34:40] make someone pay for the good deeds that [34:43] benefits everyone rather than just [34:45] themselves. In that context, we looked [34:47] into particularly a group of [34:50] entrepreneurs and investors who call [34:51] themselves um I think startup for [34:53] humans, entrepreneurs for humans. Um a [34:56] lot of them work in digital health, so [34:58] they're building technologies that [35:00] benefit for people's health uh [35:02] and in that way benefiting society. Um [35:05] and they work in this particular system [35:07] that's called the US healthcare system. [35:09] Uh meaning many good deeds are not paid [35:12] for. They uh there are many technologies [35:14] that would benefit society [35:16] but [35:17] there isn't, according to the people we [35:19] interviewed, there isn't a known [35:21] business model to make it financially [35:23] viable. [35:25] This includes technologies for public [35:26] health and if you consider chatbot LM [35:29] chatbots' impact on mental well-being as [35:32] part of public mental health, this this [35:34] would um that problem would fall into [35:36] this category [35:38] um because uh US health insurance tend [35:40] to be relatively short-term, [35:42] employer-based. [35:43] Um there isn't a great business model [35:45] for uh technologies that benefit [35:47] people's long-term well-being. Um there [35:50] isn't a lot of money to be made by [35:52] making everything easier, but you can [35:54] make a lot of money by every by making [35:56] everything harder and making the patient [35:57] going through more jumping through more [35:59] hoops. Um things like that. Um so we [36:03] actually found this group of uh startup [36:05] founders, investors [36:08] um [36:09] who are actively trying to solve these [36:10] problems. We're trying to build [36:12] companies out of this these types of [36:14] technologies, trying to make someone [36:17] trying to make the money work um for [36:20] technology for social good. Um and [36:23] here's our finding, which is the the the [36:26] the headline of this talk, they really [36:29] saw their design process as planting a [36:32] seed. They don't expect immediate [36:35] returns per se, but they see themselves [36:38] as planting seeds for new policy, new [36:41] politics, and new business models. [36:43] Here's what I mean. Um when these people [36:46] try to design [36:48] um uh make their their tech uh [36:51] uh digital health products work uh [36:54] financially, [36:55] they every single one of them really [36:58] understands the business landscape, the [37:00] regulatory landscape, and the political [37:02] landscape working in that space. They [37:05] know exactly how often they will get [37:07] paid by Medicare or Medicaid if they [37:10] provide certain kinds of health [37:11] benefits. Um and it's based on that [37:15] knowledge, they can then design their [37:17] technologies [37:19] to uh [37:20] in uh [37:21] that are financially viable, that can [37:23] survive on their own. [37:25] Um [37:27] a lot of the participants we interviewed [37:29] they call them they they say, "Well, [37:32] you're like an architect. Um [37:35] most of the time you follow the building [37:37] uh the zoning laws. Only once in a long, [37:41] long time you build a building trying to [37:44] change the zoning law, trying to change [37:45] the entire neighborhood. Most of the [37:48] time people follow the rule books, and [37:51] those rule books are the existing [37:53] business models like selling um [37:56] selling to Medicare or Medicaid and [37:58] things like that. Um [38:00] but once upon a time you do [38:03] intentionally try to create a new [38:05] product that tries to break that [38:07] pattern, tries to [38:09] find out a new way to make some social [38:12] good product financially viable. Um [38:15] here's one example uh one participant [38:18] talks through. They're trying to um [38:21] cre- uh [38:22] find a uh find find a buyer [38:25] for their digital apps that already has [38:28] some evidence of effectiveness in [38:31] improving people's mental well-being. [38:33] >> [snorts] [38:33] >> They first [38:35] I know the arrows are a little bit [38:37] messy, but that's the messiness is very [38:39] much to the point. [38:41] Um [38:42] number one, they first try to sell it to [38:43] hospitals. Well, but the hospitals needs [38:45] randomized trial evidence, which is [38:47] really expensive to run. So, they [38:49] thought, "Okay, what if [38:51] what if I just put this app on the on [38:54] the app store so that I can get some [38:56] users and I can get some data to start [38:58] to show some level of efficacy." [39:00] Um so, they did and they did collect [39:02] some data to show some level of efficacy [39:04] of their of their digital health [39:06] intervention. [39:08] Um [39:09] but that's not enough to it's not strong [39:11] enough an evidence to sell to insurers [39:13] or to hospitals. So, they went to try to [39:16] sell to employers, the tech companies [39:18] that try to provide mental well-being [39:20] benefits benefit packages to their [39:22] employers. [39:24] So, they try to sell this product [39:27] to those employers and those employers [39:28] say, "Well, I don't really care about [39:30] this person's long-term mental [39:31] well-being, but I care about how many [39:33] sick days did that person take off. Like [39:35] if you can show this digital [39:37] intervention made my employers happier, [39:40] work more productively, take fewer sick [39:42] days off, I would buy that." So, these [39:45] startup these funders so tweaked their [39:48] design still trying to improve mental [39:50] long-term mental well-being of the [39:52] users, but also try to simultaneously [39:54] optimize for those goals [39:56] that the buyers want. [39:58] And they did do that and they did manage [40:01] to sell the product to some employers. [40:04] However, it's still not enough money. [40:06] Doesn't quite [40:07] balance the [40:09] doesn't quite make the the money work [40:11] entirely. So, next thing they started [40:14] looking into insurers. [40:18] Kind of what kind of premium pri private [40:20] insurance plans [40:21] can can potentially pay for that. So, [40:23] this process goes on and on for several [40:26] years and still were happening uh when [40:29] we were interviewing them. By the time [40:31] we were interviewing them, they were [40:32] also exploring up the potential of [40:34] building a [40:35] private insurance company themselves [40:38] while also so that they can fund the [40:40] technology develop themselves. So, [40:43] they're really exploring novel ways of [40:46] making the money work. Make a technology [40:48] that provides social good to survive and [40:51] even thrive in a otherwise pretty [40:54] capitalistic market. [40:56] >> [snorts] [40:56] >> Um, [40:57] here's the interesting part. Every [40:59] person we interviewed, every person in [41:01] that community knows what are the [41:04] experimentations that are going on. All [41:06] of them say, "Well, that's probably [41:08] going to fail." [41:09] Um, but that's the that's the game [41:12] you're playing. You intentionally You're [41:15] intentionally trying to break the mold [41:17] of what this uh this um kind of market [41:21] is allowing for. Therefore, it's [41:23] worthwhile to do it these [41:25] experimentations even if most likely the [41:28] technologies are going to fail. [41:30] Um, here oops. Here is one participant [41:34] um summarizing this idea which I think [41:36] really powerful summary. They say, [41:38] "Well, yeah, most of them are going to [41:40] fail cuz you're intentionally trying to [41:42] build a technology that's not following [41:44] the capitalistic kind of market rules, [41:47] but if just one of these companies [41:50] works, then every future company can [41:52] follow the playbook. Then forever on, [41:55] everyone would know how to build a [41:57] company that provides, you know, public [42:00] mental health that can provide social [42:02] good." Um, everyone else can follow that [42:05] rule book. You just need one success [42:08] stories. [42:09] Um, [42:11] I found this like such a powerful [42:12] message. You think a lot of times we [42:15] don't don't hear that hopeful message as [42:18] much anymore, but [42:19] very powerful here. [42:21] Um so, to summarize where we are at, [42:25] um we started this project thinking, [42:27] well, conventional wisdom is if I just [42:30] build a technology that doesn't cause [42:31] harm and like ask and then as long as [42:33] policy makers adopt them, enforce them, [42:36] problem solved. Turns out things are a [42:38] lot messier. [42:40] Um a lot more complicated. Um but, I [42:44] think there are still a lot of positive [42:46] sort of takeaways from this. Um for [42:49] people who are like me who build [42:50] systems, who care about societal good, [42:54] um look at how central technologies [42:58] uh how central a role technologies play [43:00] in this whole workflow. [43:02] Technologies have great potential um to [43:05] benefit society in that sense. [43:07] I also think HCI researchers, people [43:09] like us, [43:10] are uh particularly well positioned to [43:13] uh sort of solve this puzzle. Um for [43:17] example, we're really good at being [43:19] suspicious, noticing noticing potential [43:23] societal harms before there are, you [43:25] know, large-scale causal evidence. [43:28] We are also quite good at juggling [43:30] different stakeholders' needs and wants [43:33] and everyone wants a slightly different [43:35] thing. Um in this context, in order to [43:38] raise money to build a technology that [43:40] benefits not just the people who paid [43:41] for it, there is a lot of sort of need [43:44] negotiation and service design that can [43:46] that that needs to happen here and our [43:48] expertise can be really valuable. [43:52] Um [43:53] that being said, I also think in order [43:56] to make a difference and move the [43:57] needle, we do need to work differently [43:59] from how we work today. Um [44:03] I I I wouldn't mind if Kai has fewer [44:05] papers where someone prompted element [44:07] and make people more uh more creative. [44:10] Um, in a way we know how to do that. We [44:13] also know the bottleneck is not how to [44:16] design a technology that makes people [44:18] feel better and more creative and learn [44:20] more. The challenges lie outside of the [44:23] technology on it itself. It's in essence [44:27] Um, I think there's great need for us as [44:30] technology designers to embrace these [44:33] perhaps less familiar topics like [44:34] policy, like public opinion, and like [44:38] business models. Um, [44:42] there are not every evidence can prompt [44:44] policy change, right? Um, a lot of times [44:48] if you think of like [44:49] cigarettes kind of regulation, um, a lot [44:52] of times evidence of how people use [44:54] technology that harm themselves ends up [44:57] being kind of become a story of [44:59] individual responsibility that the user [45:01] is irresponsible rather than here's [45:03] something, here's a collective harm. [45:05] Everyone in society hurts is hurt by it. [45:09] Let's do something. So there are a lot [45:10] of sort of skills and um, [45:13] challenges in designing technology that [45:15] I can actually seed policy change and [45:18] political opinion change. Um, [45:20] >> [clears throat] [45:21] >> that's the [45:23] diagram. That's the where we are in [45:25] terms of here's a road map I think we [45:29] that might get us closer to moving the [45:32] needle in changing improving gen AI's [45:35] societal impact. Um, [45:37] I'll I'll use the next few minutes [45:41] flip through some of our ongoing [45:43] projects real real quick. Um, [45:46] just to give you a sense of what how [45:48] this larger line of research changed to [45:51] what we do in the first place, not [45:52] exactly how we executed each system. [45:54] Happy to talk more if folks are [45:56] interested um, after the talk. [45:59] Um, so in our own project we [46:01] specifically looked at chatbots impact [46:04] on populational well-being, particularly [46:07] how it changes [46:09] people's intention to people's ability [46:12] to understand who you are. There are [46:13] studies [46:15] There are studies showing that even [46:17] talking with a sick friend dick chatbot [46:19] for a short moment inflates people's [46:23] self-assessment, think they're better [46:25] than their friends, therefore less [46:26] likely to reach out to their friends. [46:30] Um We also know from controlled [46:32] experiments that chatbot use directly [46:36] is directly associated with less social [46:39] intentions. So, these are all things [46:41] that would cause some concern about [46:43] people's long-term mental well-being. [46:46] So, the first thing we did was to build [46:47] systems that measure harm. In fact, I [46:50] mean, we're going to show I'm going to [46:51] show you a super complicated diagram, [46:53] but it's actually technically pretty [46:54] simple. Um in very much in the same [46:57] spirit of monitoring people's smartphone [46:59] use, like we try we actually borrowed [47:02] proxy measures from cognitive psychology [47:05] to try to measure those [47:08] human AI conversations impact on users' [47:11] mental well-being. [47:12] Diagrams look like this. [47:14] And [47:16] This is one participant's 2-year-long [47:19] worth of conversation with ChatGPT. And [47:23] when we look closer into it, we can [47:24] actually see some patterns through which [47:26] people start some patterns [47:29] that are recurring across different [47:31] conversations, and we can show that, [47:33] "Hey, these are some concerning [47:35] interaction patterns. Those [47:36] conversations are likely to lead to [47:39] um less resilience against future [47:42] stressors and difficulties in life." And [47:46] we can see that across multiple time [47:48] points of the same participant. And then [47:50] we can run the same analysis on a wide [47:53] range [47:55] of cognitive psychological measures just [47:57] to monitor which measure might go wrong. [48:01] Which measure might be worth more [48:03] studying. So, that's what we're doing [48:06] trying to a measure harms and also [48:09] potentially inform people take actions [48:11] protecting themselves and also [48:13] potentially in the future informing [48:15] policy. [48:17] Um [48:18] We also did a number of studies trying [48:20] to improve people's ability to live out [48:22] live without chatbots. Turns out people [48:24] are pretty bad at talking to other [48:27] people about who they are, about their [48:29] identity crisis. And turns out people [48:31] are pretty bad at offering other people [48:33] help in terms of other people's identity [48:35] crisis. [48:37] Um usually things go one of the two [48:39] ways. The people talk about their [48:40] identity crisis in a really sort of [48:44] uh indirect way, vague way. The other [48:47] person will just utterly confused. [48:49] Or [48:51] they got they had this conversation so [48:53] directly that it turned into a fight um [48:56] because the topic was so sensitive. So, [48:58] um our studies really show people [49:00] usually seek help from friends first and [49:03] then they were frustrated by that [49:04] process so much that they turned to [49:06] chatbots. So, a lot of the work we're [49:08] doing now are just trying to increase [49:10] people's ability to talk to each other [49:12] before we even try to take chatbots [49:13] away. [49:14] Um [49:16] We also did some policy related work [49:19] in the interest of time I'm going to [49:22] jump a little bit. [49:24] >> Thank you. [49:25] >> Oh. [49:26] Um [49:27] We also did some research and thinking [49:28] what does the exact what does it mean to [49:30] build a chatbot that doesn't cause [49:32] mental well-being problems. Turns out [49:34] it's pretty complicated as well. Like to [49:37] what extent [49:38] at what point you've crossed the line [49:40] from [49:41] being just benign friendly [49:44] to [49:45] harmful sick fancy. It's a pretty hard [49:49] line to draw. So, we did a lot of policy [49:51] kind of related um [49:53] legal framework review trying to [49:55] understand what exactly content is bad. [49:59] Um [50:01] I'll jump ahead here. [50:03] And in this area, turns out making money [50:06] work is also one of the pain points. [50:09] There are actually companies that try to [50:11] build chatbots that are not scientific, [50:13] actually effective clinically, and try [50:15] to get FDA approval for it. All of them [50:18] so far went out of business as far as I [50:20] am aware. So, another really important [50:23] um [50:24] area of uh [50:26] really important question that I don't [50:27] think enough academics um study at the [50:30] moment. Um [50:32] All in law [50:34] Um this line of research really at least [50:37] changed how we ourselves [50:40] approach designing tech for social good. [50:43] We sort of took a pause from designing [50:45] chatbots that make people more creative, [50:47] mentally uh more well-to-do, and so our [50:50] learn better um in instead of really [50:52] trying to understand harm at a societal [50:55] scale, and then do things that hopefully [50:57] will seed political changes, policy [50:59] changes, and business model changes in [51:02] the future. [51:04] With that, I thank you for your [51:05] attention and have a good time for [51:09] um for Q&A. [51:29] >> In an ideal world, what do you think [51:31] public policy would even look like for [51:33] AI? As in, how do you think that we [51:35] could effectively regulate a sector that [51:38] A is moving so quickly, but B will [51:40] become very central to geopolitics, and [51:42] to labor, and to the economy? [51:45] Um so, in an ideal world, what do you [51:46] think that would look like? [51:56] >> I'll give you an answer. It's my answer. [51:58] Whatever that future is, it's not going [52:01] to be satisfying. [52:03] Because policy is supposed to public [52:06] policy and law are supposed to set the [52:08] minimum floor [52:10] of what's allowed. It's likely not going [52:14] to meet the our standard of what's being [52:16] ethical is what benefiting society is. [52:19] Um, [52:20] tackle tech policy or healthcare policy [52:23] and so on. These laws set the floor and [52:26] in essence [52:28] I don't think policy changes up will [52:30] will, you know, make us entirely happy [52:32] in essence. [52:41] >> Um, thank you for the talk. Um, one of [52:42] my questions was related to in your [52:45] diagram you have this [52:47] uh [52:47] relationship between the gen AI harm [52:50] measurements and public opinion. And [52:53] you've described that as a [52:54] unidirectional relationship, right? Um, [52:57] but it seems to me like there is a [53:00] bidirectionality there because often [53:02] times what we choose to measure or how [53:05] we choose to measure it has to do with [53:07] public opinion, right? So for example, [53:10] in a school environment you're probably [53:12] going to have a lot of parents who raise [53:14] concerns in the town hall about overuse [53:17] of technology in the curriculum. And so [53:20] maybe opinion is not the right word to [53:22] use for that, but some kind of latent [53:24] concern that um, sits underneath before [53:28] that [53:29] becomes what we might call an opinion. [53:32] But that becomes a trigger for what we [53:33] might choose to measure, right? Or even [53:35] how we choose to measure it. Like we're [53:37] going to measure it this phenomena [53:39] particularly in middle schoolers. Um, [53:43] so I was wondering how you think about [53:44] that? And then also, [53:46] um [53:47] what do we do about harms that we cannot [53:50] anticipate or harms that we don't have [53:54] any kind of latent indicator of? Um so, [53:58] we're not seeing any indication that [54:01] this could be a concern, but it is a [54:03] concern and then might become a concern [54:05] in the future. [54:07] >> Yeah, that's a great Well, it's a great [54:09] question. So, for your first question [54:11] about the bidirectionality of this arrow [54:14] which is here, um [54:21] I don't think so, actually. I think the [54:24] Here, I guess though this workflow is [54:26] more trying to say let's do this. Let's [54:28] build the Let's make sure we have a way [54:30] to measure [54:31] because like we said, a lot of those [54:33] harms don't have indicators [54:35] or don't have great proven proxy [54:38] measures. Even the mental well-being and [54:41] chatbot when I showed the fancy diagram [54:42] of, um there's no proven metrics. [54:46] They're like we're adapting a lot of the [54:48] measures from elsewhere to treat this [54:50] and the group. [54:51] It takes a lot of resource and energy to [54:53] validate it. So, here I guess I was [54:56] mainly trying to make a point about [54:58] workflow. First, make sure you have a [54:59] measure, which is your second question. [55:02] And then, we can start to think about [55:03] how to actually use that to change your [55:06] opinion, [55:07] which comes with its own challenges. [55:19] >> Um thank you for the road map. I think [55:21] it's actually interesting because I [55:23] think we as I guess the research [55:25] community have the So, basically, I like [55:28] the thing you pointed out that we start [55:30] picking up the evidence early on and [55:32] like different researchers, different [55:33] teams pick up It's almost like different [55:35] tiles around the same effect almost, but [55:38] I feel like as the research community, [55:40] maybe we're not good enough at like, you [55:42] know, like some version of like unifying [55:44] or amplifying that a lot of these things [55:46] have the same effect. And that then can [55:48] maybe the question earlier of like [55:50] influence public opinion better when you [55:52] can show maybe the scale cuz like maybe [55:54] someone did a small study different [55:56] group or something like that. I guess [55:58] >> That's a great That's a great comment. [56:00] So, a lot of the think tanks in the sort [56:02] of policy making world, for example, [56:05] aggregate scientific evidence from [56:07] different scholars. There are some calls [56:09] in HCI asking like more HCI researchers [56:12] to also do, you know, evidence review [56:15] rather than just keep turning out [56:16] small-scale studies. So, that and [56:19] evidence synthesised work synthesised [56:21] work is is really important. I I totally [56:24] agree. It it plays a huge role in [56:26] informing policy and public opinion as [56:27] well.