Panel 2 - 2: Non-Language Is Language: Designing Communicative AI for Populations Whose Language

Panel 2 - 2: Non-Language Is Language: Designing Communicative AI for Populations Whose Language — Ai4

Ai4

AI-assisted summary of Panel 2 - 2: Non-Language Is Language: Designing Communicative AI for Populations Whose Language — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by Ai4 on August 19, 2026, this presentation features Juliana Marchand, Founder and CEO of Marchand AI LLC. The session introduces a paralinguistic governance framework designed to enable conversational AI to interpret non-verbal signals—such as silence, tone, grunts, and memory invocation—for populations living with dementia, aphasia, and other cognitive conditions.

## TL;DR Juliana Marchand argues that modern conversational AI structurally excludes millions of individuals with speech impairments, dementia, and cognitive conditions by treating non-verbal vocalizations and pauses as meaningless noise. Drawing from her bedside caregiving experiences and legal transcription background, she introduces a five-category taxonomy of non-verbal signals alongside a five-parameter paralinguistic governance framework that prioritizes prosody, enforces response delays, and eliminates predictive hallucination. She showcases Soyla Care, a proactive multimodal AI companion built on these safety parameters, and issues targeted calls to action for investors, regulators, and engineers to establish benchmarks and safeguards for vulnerable populations.

## Atmosphere & style The presentation strikes an authoritative yet deeply intimate and empathetic tone, seamlessly connecting bedside grief with technical machine learning architecture. Marchand speaks with steady, passionate conviction, grounding complex governance frameworks in visceral human anecdotes from family life and courtroom transcription. Listening to the talk feels both like a sobering indictment of current AI design oversights and an inspiring, urgent moral call to re-evaluate what true listening requires.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=4s) Conversational AI suffers from a structural failure by assuming users have intact linguistic abilities, excluding 55 million people with dementia as well as individuals with autism, aphasia, and developmental conditions. - [04:35](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=275s) Automated speech recognition routinely misinterprets non-verbal pauses and processing latency as evasive hesitation or admissions of guilt in legal depositions and interrogations. - [02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) Non-verbal vocalizations like grunts, moans, and humming correlate directly with specific clinical states such as anxiety, pain, and self-soothing rather than being meaningless acoustic noise. - [06:49](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=409s) A formal non-verbal taxonomy establishes five core categories of signal: vocalic non-language, latency and silence patterns, prosodic residue, perseveration, and memory invocation. - [09:38](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=578s) The paralinguistic governance model implements five interdependent parameters, including a minimum 5-second wait paired with visual tracking and a strict tone-over-words hierarchy. - [12:45](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=765s) Predictive LLM gap-filling and sentence completion constitute dangerous hallucinations in cognitive care, as individuals with dementia cannot fact-check or contest erroneous system assumptions. - [12:45](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=765s) Soyla Care applies paralinguistic governance using proactive multimodal engagement across smart displays, trained across nine languages to respect cultural variations in prosody. - [16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) Healthcare and government leaders are urged to regulate conversational AI by barring systems that cannot interpret distress silence from care environments. - [17:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=1030s) When conventional vocabulary deteriorates, communication does not stop but transforms, requiring systems designed around the principle that the human being precedes the technology.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=4s) The Unheard Population: Who Are We Building AI For?

Juliana Marchand, Founder and CEO of Marchand AI, introduces the central dilemma that emerged while engineering an AI companion for seniors living with dementia: "What happens to the people that AI was never designed to hear?" While conversational artificial intelligence was promised as a universal tool to listen, act, unlock research, and democratize access to information, that design fundamentally assumes a user whose language functions normally. For millions of individuals across the globe whose linguistic abilities are compromised or non-traditional, current conversational AI represents a structural exclusion rather than an empowering tool.

Marchand stresses that these individuals are far from marginal anomalies; they represent vast, vulnerable demographics across families and communities. The sheer scale of the population left behind exposes the blind spots of modern system architecture:

- **55 million people worldwide** live with Alzheimer’s disease and related dementias. - **1 in 36 children in the United States** is diagnosed with autism spectrum disorder. - **180,000 new cases of aphasia** occur annually in the United States alone, abruptly stripping stroke survivors of functional speech. - **End-of-life patients, individuals with non-verbal developmental conditions, and children with speech impediments** navigate daily life without standard fluency. - **Deposition witnesses and defendants** in courtrooms frequently have pauses, nervous delays, or slower processing rhythms systematically misinterpreted by legal automated transcription engines, permanently altering legal outcomes.

---

[02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) Bedside Origins: A Grunt is Data, Not Noise

The genesis of Marchand's work originated not in an academic computer science laboratory, but at the bedside of her uncle as he succumbed to Alzheimer’s disease. As his illness advanced, his communication deteriorated in stages: first forgetting proper names, then losing coherent sentence structures, and finally losing words entirely. Stripped of formal vocabulary, he resorted to vocalizations—grunts, sighs, and moans. While outside observers and conventional tools dismissed these sounds as unintelligible friction or failed attempts at speech, Marchand recognized an urgent subtext: "I am here. I am trying. I am frustrated. I am in pain. I need you to slow down and try to listen."

This realization is deeply corroborated by clinical literature. Empirical medical research demonstrates that non-verbal vocalizations correlate directly with distinct physiological and emotional states. Rather than random acoustic noise, a grunt is a structured, classifiable data signal that machine learning models have simply never been trained to parse:

- **Sustained grunting** correlates clinically with elevated anxiety, frustration, and agitation. - **Moaning** exhibits a statistically significant association with acute physical pain and somatic discomfort. - **Humming** functions as a primary acoustic marker of self-soothing, emotional stabilization, and calm.

---

[04:35](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=275s) The High-Stakes Justice Problem of Ignored Non-Language

Drawing on her background as a legal transcriptionist in courtrooms, depositions, and investigative interrogations, Marchand highlights how the inability of automated speech recognition (ASR) to parse latency creates systemic injustice. When an individual with a speech impediment, heightened anxiety, or slower cognitive processing pauses for six to eight seconds, or emits a brief hum while gathering their thoughts under pressure, automated transcription software routinely logs the interval as a blank "no response." In legal proceedings, this automated omission is frequently weaponized on the record as evasive hesitation or an implicit admission of guilt.

Marchand emphasizes that failing to interpret non-verbal latency in high-stakes environments is not a mere user experience flaw, but an institutional justice failure with life-altering repercussions. The foundational thesis of her framework directly confronts this paradigm:

> "Non-language is language. Silence is data. Tone is primary. A grunt carries grief or pain or lack of acknowledgement or relief. And a system that cannot tell the difference is not a companion. It's a well-intentioned failure."

---

[06:49](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=409s) Taxonomy: Five Categories of Non-Verbal Signal

To engineer systems capable of interpreting communication beyond standard grammar, Marchand introduces a formal taxonomy defining five distinct categories of non-verbal signals:

```mermaid flowchart TD accTitle: Five Categories of Non-Verbal Signal Taxonomy subgraph Taxonomy["Taxonomy of Non-Verbal Signals"] V["1. Vocalic Non-Language<br/>(Grunts, sighs, moans, humming)"] L["2. Latency & Silence Patterns<br/>(Processing, lost thread, distress)"] P["3. Prosodic Residue<br/>(Pitch, rhythm, tempo, emotional core)"] PR["4. Perseveration<br/>(Repeated phrases as unresolved thoughts)"] M["5. Memory Invocation<br/>(Divergent recollections as current emotion)"] end

V --> Sys["Multimodal AI Signal Interpreter"] L --> Sys P --> Sys PR --> Sys M --> Sys ```

1. **Vocalic Non-Language**: Paralinguistic utterances—including grunts, moans, sighs, and humming—that carry quantifiable physiological and affective information rather than semantic void. 2. **Latency and Silence Patterns**: Non-uniform pauses classified into three operational categories by evaluating temporal duration alongside facial expression: - *Processing Silence*: The user is actively assembling a cognitive response; the system must grant additional temporal margin. - *Lost Thread Silence*: The user has lost their train of thought; the system must provide a gentle rephrasing cue. - *Distress Silence*: Acute cognitive or emotional block; requires immediate supportive intervention. 3. **Prosodic Residue**: The baseline musicality, cadence, tempo, and pitch variation that endure after lexical capacity has vanished. Marchand illustrates this with a personal anecdote about her Peruvian background and her Cuban stepdad, where entire familial dialogues are conducted purely through varied intonations of "ha" and "aha." 4. **Perseveration**: The continuous repetition of a specific word or phrase. Conventional conversational interfaces view this as a broken loop to interrupt, whereas the taxonomy treats it as an unresolved underlying thought that the system must patiently track to its emotional resolution. 5. **Memory Invocation**: Sudden mid-conversation shifts by individuals with dementia toward distant, historically inaccurate, or contextually disconnected memories. Rather than correcting or reality-testing the user, the system treats the invoked memory as an accurate metaphor for the user’s real-time emotional state.

---

[09:38](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=578s) The Paralinguistic Governance Framework: Five Interdependent Parameters

To translate this taxonomy into safe machine behavior, Marchand outlines a paralinguistic governance model composed of five interdependent, non-sequential parameters that continuously condition one another:

```mermaid flowchart TD accTitle: Five Interdependent Governance Parameters subgraph Governance["Paralinguistic Governance Model"] P1["Parameter 1: Conditional Response Threshold<br/>(Wait >= 5s; evaluate face + time)"] P2["Parameter 2: Rephrasing Escalation Protocol<br/>(Simpler/binary queries; max 3 attempts)"] P3["Parameter 3: Tone-Primary Signal Hierarchy<br/>(Prosody overrides words; verify conflicts)"] P4["Parameter 4: Interpretable Thought Threshold<br/>(Zero hallucinated gap-filling)"] P5["Parameter 5: Distress Escalation Protocol<br/>(Halt immediately; notify human caregiver)"] end

P1 <--> P3 P2 <--> P4 P3 <--> P5 P1 <--> P2 ```

- **Parameter 1: Conditional Response Threshold**: Imposes an intentional minimum wait threshold of 5 seconds before the AI responds. During this window, the model actively pairs the temporal clock with computer vision analysis of facial expressions, mannerisms, and physical tension to differentiate cognitive formulation from disengagement. - **Parameter 2: Rephrasing Escalation Protocol**: When a user cannot respond, the engine simplifies the prompt into shorter, binary (yes/no) choices. This escalation is capped at three attempts; if unfulfilled, the system exits the exchange gracefully to prevent conversational pressure from triggering user agitation. - **Parameter 3: Tone-Primary Signal Hierarchy**: Prosody is prioritized over lexical content. If a user’s spoken words contradict their acoustic tone, the system refuses to assume intent based solely on words and instead prompts a clarifying binary confirmation. - **Parameter 4: Interpretable Thought Threshold**: The system is strictly prohibited from inferring beyond direct signal evidence or completing unfinished sentences. In dementia care, predictive LLM gap-filling constitutes dangerous hallucination rather than helpful assistance. - **Parameter 5: Distress Escalation Protocol**: Upon sensing acute pain, severe agitation, or self-directed harm, the AI immediately suspends dialogue and dispatches an alert to a designated human caregiver, operating strictly within the boundaries of its clinical competence.

---

[12:45](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=765s) AI Safety in Vulnerable Care and the Soyla Care Implementation

Marchand draws a sharp distinction between standard AI hallucinations and hallucinations generated in cognitive care environments. For an individual with intact cognitive faculties, an AI hallucination is an easily corrected inconvenience. For an individual with dementia who lacks the capacity to challenge the error, an AI's hallucinated assumption becomes an unchallengeable reality, inflicting compounding psychological harm. True system confidence must be strictly earned through verifiable signal parsing rather than fabricated via predictive language generation.

These governance rules serve as the architectural foundation of **Soyla Care**, a proactive conversational AI companion engineered by Marchand AI for seniors living with cognitive decline:

- **Proactive Initiation**: Unlike standard reactive assistants that require an explicit wake word or verbal prompt, Soyla Care autonomously initiates engagement, accommodating users incapable of verbal triggering. - **Multimodal Video Hardware**: Deployed on smart display form factors (such as Google Nest or Amazon Echo Show devices) so that temporal pauses are continuously contextualized with visual facial tracking. - **Nine-Language Architecture**: Trained across nine languages to account for how cultural and linguistic norms alter emotional acoustic features, preventing systematic misinterpretation of non-English prosody. - **Industry Acceleration**: Integrated into NVIDIA Inception, AWS Startups, and Google for Startups to scale infrastructure. - **Universal Applicability**: Beyond dementia, the governance model is designed to support autistic children, stroke survivors with aphasia, hospice patients, and students whose communication profiles are documented in Individualized Education Programs (IEPs) but have historically been ignored by generative AI.

[15:41](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=941s) High-Stakes Paralinguistics: Legal, Healthcare, and Market Imperatives

Misreading non-verbal signals carries severe, life-altering consequences far beyond clinical dementia care. The same vulnerability plays out in critical legal environments—depositions, police interrogations, and courtroom hearings—where an uninterpreted or mischaracterized silence can pivot a verdict and irreversibly alter someone's life. Whether evaluating a patient in healthcare or a constituent navigating government services, conventional systems systematically misinterpret non-standard communication. For the investment and technology sectors, this blind spot represents a massive, overlooked market opportunity that virtually every major AI company on the planet has walked past without recognizing.

```mermaid flowchart TD accTitle: Multi-Sector Impact of Paralinguistic AI subgraph S1["High-Stakes Environments"] A["Legal Settings\n(Depositions, Interrogations)"] B["Healthcare Facilities\n(Patients with Non-Verbal Signals)"] C["Public Sector\n(Constituents & Civic Services)"] end subgraph S2["Structural AI Blind Spot"] D["Misread Pauses & Silences"] E["Ignored Paralinguistic Cues"] end subgraph S3["Real-World Impact"] F["Altered Legal Outcomes"] G["Inadequate Care & Governance"] H["Untapped Market at Scale"] end A & B & C --> D & E D & E --> F & G & H ```

[16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) Calls to Action: Investors, Policymakers, and AI Builders

Addressing these structural failures requires coordinated action across three core stakeholder groups: investors, regulatory leaders, and AI engineers. Investors are urged to recognize that serving populations with degraded or atypical language is not a niche pursuit, but an immense unmet need at scale where the paralinguistic governance framework serves as a defensible moat. For leaders across healthcare and government, the mandate is regulatory accountability: establishing strict standards to guarantee that conversational systems incapable of interpreting distress silence are categorically barred from deployment in care settings. Finally, builders and researchers must collaborate on the fundamental data infrastructure by sharing de-identified interaction datasets to benchmark, validate, and measure paralinguistic models.

- **To Investors:** Recognize paralinguistic governance as a foundational moat for an enormous, historically neglected market. - **To Healthcare & Government Officials:** Exercise institutional authority to enforce safety standards: "A conversational AI that cannot read distress silence should not be deployed in a care setting, full stop." - **To AI Builders & Researchers:** Contribute de-identified interaction data from vulnerable populations to establish rigorous benchmarks, because "we cannot build what we cannot measure."

[17:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=1030s) The Human Origin: Bedside Roots and the Priority of the Person

The entire paralinguistic framework was not conceived inside an academic laboratory, but at the bedside of a loved one whose language was deteriorating. That experience proved that when conventional words fail, communication does not vanish—it fundamentally transforms. An individual unable to speak according to standard linguistic expectations is not communicating less, but communicating differently. As the technology sector races to build AI capable of listening and acting, that listening must be expanded to encompass every human being. The sequence of priorities remains absolute: the framework precedes the technology, caregiving preceded the framework, and the human being always precedes everything.

> "He taught me that communication does not stop when language does. It transforms. And the people who cannot speak the way we expect is not communicating less, is communicating differently."

## Notable quotes > What if the biggest failure in conversational AI isn't what it says, but what it can't hear? — Juliana Marchand

> Conversational AI has been designed, almost without exception, for users whose language works. — Juliana Marchand

> These populations don't stop communicating when language degrades. They communicate through silence, tone, grunts, repeated phrases, and invoked memories. Current AI systems aren't designed to receive any of it. — Juliana Marchand

> [00:04](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=4s) What happens to the people that AI was never designed to hear? — Juliana Marchand

> [02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) I am here. I am trying. I am frustrated. I am in pain. I need you to slow down and try to listen. — Juliana Marchand

> [04:35](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=275s) Non-language is language. Silence is data. Tone is primary. A grunt carries grief or pain or lack of acknowledgement or relief. And a system that cannot tell the difference is not a companion. It's a well-intentioned failure. — Juliana Marchand

> [16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) A conversational AI that cannot read distress silence should not be deployed in a care setting, full stop. — Juliana Marchand

> [16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) we cannot build what we cannot measure. — Juliana Marchand

> [17:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=1030s) He taught me that communication does not stop when language does. It transforms. And the people who cannot speak the way we expect is not communicating less, is communicating differently. — Juliana Marchand

## Mentioned Juliana Marchand, Marchand AI LLC, Ai4, Soyla Care, Google Nest, Amazon Echo Show, NVIDIA Inception, AWS Startups, Google for Startups, Individualized Education Programs

## Video description Juliana Marchand, Founder, Marchand AI LLC

What if the biggest failure in conversational AI isn't what it says, but what it can't hear?

Conversational AI has been designed, almost without exception, for users whose language works. For the 55 million people living with dementia, and for individuals with autism, aphasia, and non-verbal developmental conditions, that assumption is a structural failure. These populations don't stop communicating when language degrades. They communicate through silence, tone, grunts, repeated phrases, and invoked memories. Current AI systems aren't designed to receive any of it. I will present a paralinguistic governance framework for communicative AI in dementia care: five formally defined, interdependent parameters governing how a system interprets and responds to the full signal space of human communication, verbal and non-verbal alike. Attendees will leave with a concrete taxonomy of non-verbal signal, a replicable governance model, and an AI safety argument for why hallucinating meaning for a person with dementia isn't an edge case , it's the central design problem.

If you build AI that interacts with people, this talk will change how you think about what listening actually requires.

[Source](https://www.youtube.com/watch?v=uVy59r_e7Bg)

中文简报

专题讨论 2 - 2：非语言亦是语言：为语言受限群体设计交流型AI Juliana Marchand 指出，现代对话式AI将非语言发声和停顿视为无意义的噪音，在结构上排除了数百万患有言语障碍、失智症以及认知障碍的人群。结合其床边护理经验与法律转录背景，她提出了一套包含五大类别的非语言信号分类体系，以及一个优先考量韵律、强制执行响应延迟并消除预测性幻觉的五参数副语言治理框架。她还展示了基于这些安全参数构建的主动型多模态AI伴侣 Soyla Care，并向投资者、监管机构和工程师发出针对性呼吁，以期为弱势群体建立评测基准与安全保障。 对话式 AI 存在预设用户语言能力健全的结构性缺陷，直接排除了全球 5500 万失智症患者以及自闭症、失语症等群体。 (0:04) 自动语音识别常将非言语停顿和认知延迟误判为回避或认罪，而咕哝、呻吟、哼唱等发声实则对应着焦虑、身体疼痛与自我安抚等明确临床状态。 (2:47) 非言语信号分类法确立了副语言发声、停顿与沉默模式、韵律残留、言语重复和记忆唤起五大类别，作为多模态解析的基础。 (6:49) 副语言治理框架包含至少 5 秒等待结合面部追踪、语调优先于词义以及严禁大模型通过预测性补全产生危险幻觉等五个相互依存的参数。 (9:38) 基于该框架落地的 Soyla Care 采用支持 9 种语言的主动多模态交互；同时呼吁监管机构明令禁止无法识别困境沉默的对话式 AI 进入护理环境。 (12:45)

Reading summary

Juliana Marchand, Founder and CEO of Marchand AI, argues that conventional conversational AI structurally excludes vulnerable populations by misinterpreting non-verbal vocalizations, latency, and prosody as meaningless noise rather than structured communicative data. To prevent severe harms in healthcare and legal environments, she advocates for a formal paralinguistic taxonomy and governance framework that subordinates lexical content to acoustic tone and context. Systemic Exclusion of Atypical Communicators Across Sectors (0:04) Conversational AI models routinely presume users possess standard linguistic fluency, marginalizing vast populations who communicate outside typical lexical structures. Globally, 55 million people live with dementia, 1 in 36 children in the United States is diagnosed with autism, and 180,000 Americans develop aphasia annually following strokes. Beyond clinical settings, this design bias carries high-stakes institutional consequences: in legal depositions and courtroom proceedings, automated speech recognition software routinely logs six- to eight-second processing pauses as a blank "no response," which is frequently weaponized on the record as evasiveness or an admission of guilt. Marchand maintains that treating non-standard timing and vocalizations as empty friction converts vulnerability into institutional liability. A system incapable of recognizing the emotional and communicative intent behind a pause or paralinguistic sound fails basic utility and risks inflicting measurable real-world harm. Clinical Evidence and the Non-Verbal Signal Taxonomy (6:49) Drawing from bedside observations of advancing Alzheimer’s disease and supported by medical literature, Marchand stresses that non-verbal vocalizations are structured physiological indicators rather than acoustic noise. Clinical research correlates sustained grunting with frustration and anxiety, moaning with acute physical pain, and humming with self-soothing emotional regulation. To operationalize these inputs, Marchand introduces a taxonomy defining five categories of non-verbal signals: vocalic non-language, latency and silence patterns, prosodic residue, perseveration, and memory invocation. Under this taxonomy, pauses are evaluated alongside visual tracking to distinguish cognitive processing from a lost train of thought or acute distress. Similarly, prosodic residue—the underlying pitch, rhythm, and intonation that persist after vocabulary is lost—retains core emotional meaning, while repeated phrases (perseveration) and disconnected recollections (memory invocation) are treated as unresolved thoughts and real-time emotional metaphors rather than errors to interrupt. Paralinguistic Governance and Mitigating Care Hallucinations (9:38) To govern automated behavior safely, Marchand outlines five interdependent parameters: enforcing a minimum five-second wait threshold paired with computer vision to evaluate physical tension; capping rephrasing escalation at three binary choices before gracefully exiting; prioritizing vocal tone over contradictory spoken words; strictly barring predictive sentence completion; and immediately suspending operation to alert human caregivers upon detecting acute distress. Marchand emphasizes that while large language model hallucinations are merely inconvenient to cognitively intact users, they inflict severe psychological trauma on dementia patients who lack the cognitive capacity to challenge automated fabrications. These governance parameters form the core of Soyla Care, a proactive, nine-language conversational companion deployed on smart display hardware that initiates interaction without requiring verbal wake words. Regulatory Accountability, Benchmarking, and Market Imperatives (16:10) Translating paralinguistic AI into standard practice requires coordinated intervention across government, research, and capital markets. Marchand calls on regulatory and healthcare authorities to mandate strict safety thresholds, asserting that conversational AI tools unable to interpret distress silence must be categorically barred from care settings. Simultaneously, she urges researchers and builders to share de-identified interaction datasets from vulnerable populations, noting that paralinguistic capabilities cannot mature without empirical benchmarks. For investors and software developers, Marchand frames this architecture as an overlooked market moat. Addressing atypical communication spans not only dementia and palliative care, but also special education programs, stroke rehabilitation, and public sector services where conventional AI interfaces systematically break down. Marchand concludes that when standard language diminishes, communication does not disappear but fundamentally transforms into paralinguistic signals. The framework's broader adoption remains dependent on cross-sector data sharing to establish rigorous benchmarks, ensuring AI systems are evaluated on their ability to interpret silence, tone, and distress before deployment in high-stakes environments.

精炼阅读摘要

Marchand AI 创始人兼首席执行官 Juliana Marchand 指出，主流对话式人工智能默认用户具备标准语言能力，从而在系统架构上排除了数以千万计语言受损或非典型的弱势群体；她提出非语言信号亦是关键数据，必须建立副语言治理模型以重塑人机交互。 技术盲区与弱势群体的系统性排除 (0:04) 对话式人工智能一直被许诺为普惠的信息与服务工具，但其底层假设却将语言能力受损者彻底边缘化。Marchand 指出，这类人群绝非少数特例：全球有 5500 万阿尔茨海默病及相关认知障碍患者，美国每 36 名儿童中就有 1 人确诊孤独症谱系障碍，仅美国每年就新增 18 万例因中风导致的失语症患者；此外还包括临终关怀患者、非言语发育障碍群体以及口吃儿童等。在法庭问讯与作证中，自动语音识别技术（ASR）经常将证人因紧张或认知延迟导致的 6 到 8 秒停顿误录为“无应答”，甚至被曲解为回避问题或默认认罪，直接损害司法公正。 非语言发声的生理关联与数据价值 (2:47) Marchand 从照护患阿尔茨海默病亲属的经历中发现，随着词汇与句式完全瓦解，长辈所发出的呼噜声、叹息与呻吟并非杂音，而是传递挫折、痛苦或需要关注的有效信号。临床医学研究证实，非语言发声与生理及情绪状态直接挂钩：持续的咕哝声（grunts）在临床上与焦虑、挫败及烦躁密切相关；呻吟声（moaning）与急性躯体疼痛呈现统计学显著相关；而哼唱声（humming）则是自我安抚、情绪平复与冷静的主要声学标志。现有机器学习系统尚未被训练去解析这些结构化的声学数据。 非语言信号分类法及其实践意涵 (6:49) 为了让系统理解非标准语法，Marchand 建立了包含五类非语言信号的分类框架：一是具有量化生理与情绪信息的发声副语言（如哼鸣、呻吟）；二是延迟与沉默模式，通过结合面部微表情将停顿划分为正在组织思维的“处理型沉默”、丢失思路的“遗失线索型沉默”以及急性情绪受阻的“痛苦型沉默”；三是残存韵律，即在词汇丧失后依然存续的语调、节奏与基频变化；四是持续言语（Perseveration），将重复出现的字词视为尚未解决的核心思绪，而非单纯需打断的死循环；五是记忆调用，将痴呆症患者突发脱离现实的回忆视作实时情绪隐喻，而非需要纠偏的逻辑错误。 副语言治理框架与照护安全模型 (9:38) 针对认知障碍人群中大模型“幻觉”会演变为无法反驳的心智现实并造成二次伤害的风险，Marchand 提出了相互制约的五大副语言治理参数：设置至少 5 秒的条件反应阈值，结合视觉检测判断用户是否处于思考中；通过最多三次尝试逐步将问题简化为是非二元选择，若无果则体面退出以避免施压；确立语调优先层级，在词义与声学基调冲突时以语调为准；设定可解释思维阈值，严厉禁止大模型利用预测算法随意脑补未完成的句子；一旦检测到急性疼痛或自我伤害倾向，立即中止对话并触发警报转接人工照护者。该框架已应用于多模态陪伴产品 Soyla Care，支持主动发起对话并内置了九种语言的声学模型。 高风险场景延伸与多方协作呼吁 (15:41) 副语言识别缺陷所带来的后果正蔓延至司法、医疗以及公共事务管理等多个高风险领域。Marchand 呼吁各界采取协同行动：敦促投资者将副语言治理视为服务未满足需求并具备竞争壁垒的巨大市场；敦促医疗与政府监管部门确立准入红线，明令禁止无法识别“痛苦型沉默”的对话式系统部署至照护场所；呼吁研究人员与开发者开放共享弱势群体的去标识化交互数据集以建立基准。她强调，当人类语言退化时，沟通并未停止而是发生了形态转变，技术演进必须恪守以人为本的伦理底线。 该报告主张，忽视停顿、发声副语言与情绪隐喻的对话式人工智能是不合格的辅助工具；如何建立严密的副语言评估标准，并在司法审讯与临床照护中杜绝算法盲区与有害脑补，是保障边缘群体权益亟待落实的核心议题。

Detailed notes — zh

## Background 该内容由 Ai4 频道于 2026 年 8 月 19 日发布在 YouTube 平台上。视频记录了 Marchand AI 创始人 Juliana Marchand 的演讲，探讨传统对话式 AI 在面对失智症、自闭症及失语症等语言障碍群体时的结构性缺失，并提出了专为解读非语言信号而设计的副语言治理框架。 ## TL;DR 主流对话式 AI 普遍基于“用户具备健全语言能力”的假设，导致全球数以千万计的认知与语言障碍群体在医疗和司法等场景中遭到技术漠视与误判。Juliana Marchand 提出了涵盖发声、延迟沉默、韵律残留、言语重复和记忆唤起的五大非语言信号分类法，并建立了以视觉结合、重述降级、音调优先、严控幻觉与痛苦升级为核心的副语言治理框架。这一框架已在其主动式伴侣产品 Soyla Care 中落地实践，证明停顿与发声皆是关键数据而非背景噪音。她呼吁投资人、监管者与开发者共同建立准入规范与基准数据，重申技术必须服务于每一个真实存在的个体。 ## Atmosphere & style 整场演讲兼具深刻的人文关怀与严密的工程逻辑，语调沉稳、坚定且极富感染力。演讲者从病榻旁照护至亲的切肤体验与法庭转录中的制度性误判切入，将复杂的算法治理参数与真实的人类情感紧密交织。整场分享既是一次对主流 AI 研发盲区的有力警示，又充满温暖而坚定的号召力，让观众深刻体会到倾听与理解对于弱势群体的非凡重量。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=4s) 传统对话式 AI 普遍建立在用户语言功能完好的假设上，在结构上排斥了失智症、自闭症及失语症等数以千万计的特需群体。 - [02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) 呼噜声、叹息与呻吟等非语言发声绝非背景噪音，而是蕴含焦虑、疼痛或自我抚慰等明确状态的高价值分类数据。 - [02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) 在司法录供与审讯场景中，AI 将言语障碍者的停顿沉吟粗暴转录为“无应答”并误判为心虚，造成严重的制度性司法不公。 - [06:49](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=409s) 系统化建立了非语言信号五大分类学：发声类非语言、延迟与沉默模式、韵律残留、言语重复以及记忆唤起。 - [09:38](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=578s) 副语言治理框架确立了条件响应阈值、重述升级协议、音调优先层级、可解释思维阈值与痛苦升级协议五大协同控制参数。 - [09:38](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=578s) 由于认知障碍患者缺乏纠错与反驳能力，AI 幻觉在照护场景下是致命安全隐患，系统绝不可过度推断或擅自替患者补全想法。 - [13:55](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=835s) 主动式伴侣 Soyla Care 必须依托视频视觉界面，因为单靠音频无法区分用户的长时间停顿究竟是深度思考还是陷入了痛苦。 - [13:55](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=835s) 副语言治理框架具有高度普适性，能广泛赋能自闭症儿童、中风失语症患者、临终关怀人群及发育障碍群体。 - [15:41](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=941s) 特需群体的沟通障碍绝非边缘利基市场，而是被当今主流 AI 产业全面忽略、规模极其庞大的刚需蓝海。 - [16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) 呼吁监管与医疗决策者行使准入权力：无法识别与解读“痛苦沉默”的对话式 AI 严禁部署于临床一线照护场景。 - [16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) 倡导工程研发界打破数据壁垒并共享去标识化交互数据集，合力攻克副语言基准衡量与基础设施建设难题。 - [17:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=1030s) 语言能力的衰退不等于沟通的终止，“框架先于系统，照护先于框架，人永远先于这世上的一切技术与系统”。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=4s) AI 究竟为谁而建？被传统对话系统遗忘的庞大群体 Marchand AI 创始人兼首席执行官 Juliana Marchand 在为患有阿尔茨海默病及相关认知障碍的长者开发 AI 伴侣时，反复思考着一个核心问题："当人们无法使用正常语言时，那些从未被设计用来倾听他们的 AI 该如何面对他们？" 现代对话式 AI 的底层承诺是"倾听并采取行动"，这项技术确实开启了前所未有的信息获取、研究与沟通大门。然而，当技术建立在"用户的语言功能完好无损"这一假设之上时，绝大多数现有系统实际上在结构上放弃了数以千万计无法清晰表达的人群。 这些被主流技术忽视的群体并非无足轻重的边缘个案（edge cases），而是真实生活在我们身边的父母、孩子与邻居。仅在认知与语言障碍领域，全球就有 5500 万阿尔茨海默病及相关失智症患者；在美国，每 36 名儿童中就有 1 人被确诊患有自闭症；每年新增因中风导致失语症（aphasia）的患者高达 18 万人。此外，还包括临终关怀患者、患有非语言发育障碍的群体以及存在言语障碍的儿童。 令人震惊的是，语言与理解的断裂不仅发生在医疗护理场景中，甚至蔓延到了司法系统。Juliana 指出，在法庭、听证会和供词记录中，由于当事人的迟疑与停顿无法被现有的语音识别 AI 正确理解，AI 转录往往直接将其判定为"无应答"或心虚犹豫，进而改变个人的命运。当下市场上的主流对话系统，从根本上就缺乏感知这部分人群沟通意图的设计。 - **全球失智症患者规模**：5500 万人患有阿尔茨海默病及相关认知障碍。 - **自闭症谱系发生率**：美国儿童中确诊比例达到 1/36。 - **中风后失语症新增病例**：美国每年新增约 180,000 例。 - **波及场景与人群**：言语障碍儿童、临终关怀患者、发育障碍群体，以及在法庭与供词场景中因语速缓慢而遭到 AI 误判的当事人。 --- ### [02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) 床边启示与法庭误读：非语言即语言，沉默亦是数据 这项研究框架的起点并非来自冷冰冰的实验室，而是源于 Juliana 在患有阿尔茨海默病的叔叔病床前的亲身体验。随着病情恶化，叔叔逐渐失去了说出人名、组织完整句子乃至吐出单个单词的能力，最终只能发出呼噜声（grunts）、叹息（sighs）和呻吟（moans）。旁人往往只将这些声音视作毫无意义的杂音或徒劳的尝试，但作为至亲，Juliana 从中听出了明确的情感呼救："我在这里。我在努力。我很沮丧。我很痛苦。我需要你慢下来，试着听听我。" 临床文献完全印证了这种直觉：持续的哼哧与呼噜声在临床上与焦虑和躁动显著相关；呻吟声高度关联着身体疼痛与不适；而哼唱声（humming）则常常伴随着平静与自我抚慰。"呼噜声不是噪音，而是数据（A grunt is not noise, it's data）。"它是一种完全可分类的信号，只是现有的 AI 从未接受过识别这类信号的训练。 与此同时，Juliana 曾在法律系统中担任法庭与审讯转录员的经历，让她看到了非语言信号被误读所带来的制度性危害。当被讯问者因为言语障碍、思维处理缓慢或过度紧张而产生 6 到 8 秒的停顿、沉吟或思维游离时，AI 自动转录系统会粗暴地将其标注为"无应答（no response）"。在后续的司法认定中，这种停顿常被直接推断为迟疑甚至默认有罪。这 6 到 8 秒并非空白的沉默，而是当事人在巨大心理压力下艰难组织语言的过程。Juliana 严正指出："在供词记录中忽略非语言信号，绝不是一个单纯的 UI/UX 问题，而是一个严肃的司法公正问题。" ```mermaid flowchart LR accTitle: Non-Language Signal Interpretation Comparison accDescription: Comparison between traditional AI misunderstanding and communicative AI correct understanding of non-verbal signals. subgraph Traditional["传统 AI 的粗暴处理"] A1["非语言声音 / 呻吟 / 呼噜"] --> B1["判定为无效背景噪音 (Noise)"] A2["长达 6-8 秒的停顿与沉吟"] --> B2["转录为'无应答'并误判为迟疑或认罪"] end subgraph Proposed["副语言治理框架的正确解析"] C1["非语言发声 (Vocalics)"] --> D1["解析为焦虑、疼痛或自我抚慰等结构化数据"] C2["延迟与沉默模式 (Latency)"] --> D2["结合面部观察，识别为信息处理、迷失线索或求助信号"] end ``` --- ### [06:49](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=409s) 非语言信号的五大分类学体系 为了构建能够识别非语言沟通的系统，首先必须为这些信号建立严谨的命名与分类标准，因为"你无法治理你无法命名的东西"。Juliana 提出了 AI 必须学会解读的五类非语言信号分类学（Taxonomy of Non-Verbal Signals）： | 信号分类 | 核心表现形式 | 系统临床解读与交互要求 | |:--- |:--- |:--- | | **发声类非语言 (Vocalic Non-Language)** | 呼噜声、叹息声、呻吟声、哼唱声等 | 每一类声音都具备特定的情感与生理含义（如疼痛、焦虑或舒缓），属于高价值分类数据。 | | **延迟与沉默模式 (Latency & Silence Patterns)** | 依据诱因划分为：处理型沉默、断线型沉默、困窘型沉默 | 沉默并非千篇一律。系统必须结合面部视觉线索与时间跨度，分别采取给予时间、温和重述或即时干预措施。 | | **韵律残留 (Prosodic Residue)** | 语言词汇丧失后依然存留的音高、语速、节奏 | 音调承载着语言无法表达的真实意图。即使像拉美家庭中简单的"ha"与"aha"互动，语调本身就是完整的语言。 | | **言语重复 (Perseveration)** | 持续不断地重复相同的词汇或短语 | 传统系统将其视为死循环，而本框架将其定义为"未解的思绪"，系统应顺应并跟进线索直至其释怀。 | | **记忆唤起 (Memory Invocation)** | 在对话中突兀跳转至过去某段记忆（无论真伪） | 绝非对话脱轨。该记忆必然映射着当事人当下的真实情绪，系统严禁纠错，必须顺应记忆探寻其当下心境。 | 在谈及韵律残留时，Juliana 分享了一个生动的家庭趣事：作为秘鲁人，她与来自古巴的继父有时仅靠不同的笑声和感叹声（"ha"、"aha"）就能完成完整的交流，这充分证明了音调与节奏在词汇之外传递意义的强大力量。 --- ### [09:38](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=578s) 五大副语言治理参数与 AI 幻觉的安全边界 识别信号之后，系统必须依据一套相互制约、协同工作的副语言治理框架（Paralinguistic Governance Framework）做出响应。这五个参数并非线性先后发生，而是互为条件： 1. **条件响应阈值（Conditional Response Threshold）**：系统在响应前至少等待 5 秒。在这 5 秒内，系统并非单纯空转，而是通过摄像头同步捕捉用户的面部表情、神态与肢体语言。"时间结合视觉观察，绝非单看时间"，以此综合评估用户状态。 2. **重述升级协议（Rephrasing Escalation Protocol）**：若用户未作应答，系统会自动将语句重述得更加简短易懂（尽量收敛为"是/否"问题）。该重述最多尝试 3 次；超过 3 次后系统优雅退出，避免持续施压引发患者的烦躁与痛苦。 3. **音调优先信号层级（Tone-Primary Signal Hierarchy）**：韵律与语调的处理优先级高于词汇本身。当用户的语调与其所说的词汇产生矛盾时，系统绝不擅自假设，而是发起简单的"是/否"问题进行确认。 4. **可解释思维阈值（Interpretable Thought Threshold）**：系统绝不可做出超出当前信号支撑范围的推断，不得替患者补全未表达的想法，严禁产生 AI 幻觉（Hallucination）。 5. **痛苦升级协议（Distress Escalation Protocol）**：一旦检测到急性痛苦、剧烈疼痛、严重躁动或自伤倾向，系统立即终止对话并向人工护理人员报警，绝不擅自尝试独自处理危机。 Juliana 强调，在失智症护理场景中，AI 幻觉绝非普通软件中的轻微瑕疵，而是致命的安全威胁。认知正常的健全人遇到 AI 幻觉时，可以纠正、重新提问或指责系统；但失智症患者根本丧失了纠错和反驳的能力。一旦系统将自以为是的臆测强加给患者，系统的错误就会直接被固化为现实，造成复合型伤害（compounding harm）。"AI 系统的置信度必须由输入信号来赋予，而不是由系统强行投射给信号。" ```mermaid flowchart TD accTitle: Paralinguistic Governance Framework Architecture accDescription: The 5 interdependent governance parameters controlling communicative AI behavior. subgraph Sensing["感知与响应控制"] P1["参数一: 条件响应阈值<br/>(至少等待5秒 + 结合视觉面部分析)"] P3["参数三: 音调优先层级<br/>(韵律优先于文本，矛盾时简要确认)"] end subgraph Reasoning["交互与推理边界"] P2["参数二: 重述升级协议<br/>(简化/是非题，上限3次，优雅退出)"] P4["参数四: 可解释思维阈值<br/>(杜绝过度推断，严禁填补空缺产生幻觉)"] end subgraph Safety["安全兜底机制"] P5["参数五: 痛苦升级协议<br/>(检测到急性痛苦/自伤时立即中断，呼叫人工看护)"] end P1 -.-> P2 P3 -.-> P4 P4 -.-> P5 ``` --- ### [13:55](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=835s) Soyla Care 的落地实践与全人群赋能愿景 上述治理框架目前已全面落地于 Marchand AI 旗下的核心产品 —— **Soyla Care**。这是一款专为患有失智症的长者打造的主动式（proactive）对话伴侣。由于目标用户往往无法主动发起唤醒词，Soyla Care 具备主动发起交流的能力，并部署在 Google Nest、Amazon Echo Show 等带有屏幕与摄像头的硬件载体上。必须依赖视频界面的原因在于：仅凭音频根本无法区分用户的停顿究竟是正在思考还是陷入了痛苦，唯有结合面部微表情才能做出准确判断。 此外，Soyla Care 当前正支持 9 种语言的研发。情感在声学层面的表征因语言和文化而异，仅基于英语校准的模型在面对西班牙语人群时会出现系统性的误读，这在护理场景下是不可接受的安全隐患。目前，该项目已成功入选 Nvidia Inception、AWS Startups 以及 Google for Startups 等孵化计划，正在搭建规模化部署的基础设施。 Juliana 总结道，这套副语言治理框架的价值远远超越了失智症护理本身。它适用于所有言语功能不可靠、不完整或完全缺失的群体： - **自闭症谱系儿童**：弥合其非典型沟通方式与外部世界的鸿沟。 - **中风失语症患者**：在失去词汇组织能力时协助其表达情感与需求。 - **临终关怀阶段患者**：捕捉微弱的发声与神态意图。 - **非语言发育障碍儿童**：其个别化教育计划（IEP）中详细记录了丰富的沟通特征档案，但此前从未有技术团队尝试将其转化为 AI 交互系统的输入依据。 ### [15:41](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=941s) 跨越司法与医疗的信号漏读代价与庞大市场机遇 非语言信号的误读绝不仅限于技术误差，在许多高风险现实场景中，它往往直接关乎一个人的命运。在法律场景下——无论是证词录供（depositions）、审讯（interrogations）还是法庭听证（hearings），当事人的短暂停顿若被遗漏或被错误解读，很可能直接改变一生的轨迹。这种对沉默和副语言信号的忽视绝非孤立现象：对于医疗卫生领域而言，这些人就是亟待照护的患者；对于政府部门而言，他们是必须被倾听与服务的选民；而对于投资人来说，这代表着一个被当今地球上几乎所有人工智能公司完全略过、规模庞大却长期严重匮乏服务的空白市场。 - **司法场景中的高昂代价**：在录供与审讯等法律环节，系统若漏读或误判受访者的停顿与沉默，其后果将直接左右案件判决，彻底改变当事人的生活轨迹。 - **跨领域的共性群体**：对语言功能受损人群的理解，直接关联到医疗系统的病患照护质量与政府对选区选民（constituents）的基本权益保障。 - **被忽视的商业蓝海**：全球几乎所有主流 AI 研发机构都聚焦于语言功能健全的用户，从而错过了服务数以千万计特需人群的庞大市场需求。 ```mermaid flowchart TD accTitle: 副语言理解缺失的多领域连锁反应 subgraph HighStakes["高风险现实场景"] Legal["司法系统<br/>（录供/审讯/听证）"] Health["医疗体系<br/>（病患护理）"] Gov["政府公共服务<br/>（选民权益）"] end subgraph Impact["信号遗漏与误读的后果"] M1["停顿与沉默被误读<br/>直接改变当事人命运"] M2["无法感知病患痛苦<br/>造成护理与安全事故"] M3["特需人群被技术排斥<br/>公共服务无法触达"] end Legal --> M1 Health --> M2 Gov --> M3 ``` --- ### [16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) 对投资人、监管决策者与构建者的行动倡议 面向现场不同背景的听众，演讲者发出了明确而坚定的行动呼吁。对于投资界而言，这套副语言治理框架所服务的群体绝非无足轻重的“小众利基市场（niche）”，而是一个规模极其宏大、长期未被满足的刚性需求；旗下产品 Soyla Care 正是切入该赛道的载体，而底层的治理框架则是坚实的技术护城河。对于医疗卫生系统管理者与政府决策者，演讲者强烈呼吁必须以行政和行业权力设立硬性门槛与准入标准：“一款无法解读‘痛苦沉默（distress silence）’的对话式 AI，压根就不应该被部署到照护场景中，绝无妥协余地（full stop）。”决策者拥有确立标准的权力，现在正是行使权力的时刻。 面对 AI 系统的实际构建者与工程师，演讲者则呼吁大家加入到联合科研中来。行业当下极度缺乏来自这些特需人群的去标识化（de-identified）交互数据集，而没有数据就无法进行严谨的衡量，也就无法构建出真正可靠的系统基础设施。唯有打破数据孤岛、共享去标识化的交互样本，才能对副语言治理框架进行全面的基准验证与扩展，共同搭建起包容性 AI 的底层架构。 - **面向投资人**：特需群体的沟通障碍是规模巨大的未满足刚需，Soyla Care 作为承载产品、副语言框架作为核心护城河，蕴藏着巨大的商业与社会双重价值。 - **面向医疗与政府决策者**：行使监管与采购权力，制定强制准入规范；无法识别痛苦停顿与沉默的对话式 AI 严禁进入照护一线。 - **面向技术构建者与研究者**：倡导共享去标识化的真实互动数据，“我们无法构建我们无法衡量的事物”，携手攻关基础设施层面的副语言测量难题。 --- ### [17:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=1030s) 始于病榻的初心：沟通未曾停滞，人永远先于一切 在演讲的结尾，分享者深情地道出了支撑整个研究与创业历程的精神原点。这套复杂的副语言治理框架并非诞生于冰冷的高校实验室，而是始于她挚爱之人的病榻旁——当时对方正竭尽全力试图与她沟通，但他赖以表达的语言能力却在不断退化和失效。正是这位至亲教会了她一个朴素而深刻的真理：当语言走向衰退或终止时，沟通本身并没有停止，它只是转变了形态。那些无法按照常人期望方式说话的人，并不是在“减少”沟通，而是在“以不同的方式”沟通。 人们构建人工智能的初衷是让它学会聆听并采取行动，那么现在的 AI 就必须学会去倾听每一个人。在技术演进的层级结构中，必须认清何者为本：治理框架先于具体系统而存在，长期的照护实践先于框架而存在，而人——活生生的人类个体——永远、永远先于这世上的一切技术与系统。 > “框架先于系统，照护先于框架。而人，活生生的人类，永远、永远先于一切。”（The framework precedes the system. The caregiving preceded the framework. And the person, the human, always, always precedes everything.） ## Notable quotes > [00:04](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=4s) 当人们无法使用正常语言时，那些从未被设计用来倾听他们的 AI 该如何面对他们？ — Juliana Marchand > [02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) 我在这里。我在努力。我很沮丧。我很痛苦。我需要你慢下来，试着听听我。 — Juliana Marchand > [02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) 呼噜声不是噪音，而是数据（A grunt is not noise, it's data）。 — Juliana Marchand > [02:47](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=167s) 在供词记录中忽略非语言信号，绝不是一个单纯的 UI/UX 问题，而是一个严肃的司法公正问题。 — Juliana Marchand > [06:49](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=409s) 你无法治理你无法命名的东西 — Juliana Marchand > [09:38](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=578s) 时间结合视觉观察，绝非单看时间 — Juliana Marchand > [09:38](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=578s) AI 系统的置信度必须由输入信号来赋予，而不是由系统强行投射给信号。 — Juliana Marchand > [16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) 一款无法解读‘痛苦沉默（distress silence）’的对话式 AI，压根就不应该被部署到照护场景中，绝无妥协余地（full stop）。 — Juliana Marchand > [16:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=970s) 我们无法构建我们无法衡量的事物 — Juliana Marchand > [17:10](https://www.youtube.com/watch?v=uVy59r_e7Bg&t=1030s) 框架先于系统，照护先于框架。而人，活生生的人类，永远、永远先于一切。（The framework precedes the system. The caregiving preceded the framework. And the person, the human, always, always precedes everything.） — Juliana Marchand ## Mentioned Juliana Marchand, Marchand AI LLC, Soyla Care, Google Nest, Amazon Echo Show, Nvidia Inception, AWS Startups, Google for Startups, Ai4 ## Video description Juliana Marchand, Founder, Marchand AI LLC What if the biggest failure in conversational AI isn't what it says, but what it can't hear? Conversational AI has been designed, almost without exception, for users whose language works. For the 55 million people living with dementia, and for individuals with autism, aphasia, and non-verbal developmental conditions, that assumption is a structural failure. These populations don't stop communicating when language degrades. They communicate through silence, tone, grunts, repeated phrases, and invoked memories. Current AI systems aren't designed to receive any of it. I will present a paralinguistic governance framework for communicative AI in dementia care: five formally defined, interdependent parameters governing how a system interprets and responds to the full signal space of human communication, verbal and non-verbal alike. Attendees will leave with a concrete taxonomy of non-verbal signal, a replicable governance model, and an AI safety argument for why hallucinating meaning for a person with dementia isn't an edge case , it's the central design problem. If you build AI that interacts with people, this talk will change how you think about what listening actually requires. [Source](https://www.youtube.com/watch?v=uVy59r_e7Bg)

Source quotes

[0:04] Juliana Marchand: 当人们无法使用正常语言时，那些从未被设计用来倾听他们的 AI 该如何面对他们？ [2:47] Juliana Marchand: 我在这里。我在努力。我很沮丧。我很痛苦。我需要你慢下来，试着听听我。 [2:47] Juliana Marchand: 呼噜声不是噪音，而是数据（A grunt is not noise, it's data）。 [2:47] Juliana Marchand: 在供词记录中忽略非语言信号，绝不是一个单纯的 UI/UX 问题，而是一个严肃的司法公正问题。 [6:49] Juliana Marchand: 你无法治理你无法命名的东西 [9:38] Juliana Marchand: 时间结合视觉观察，绝非单看时间 [9:38] Juliana Marchand: AI 系统的置信度必须由输入信号来赋予，而不是由系统强行投射给信号。 [16:10] Juliana Marchand: 一款无法解读‘痛苦沉默（distress silence）’的对话式 AI，压根就不应该被部署到照护场景中，绝无妥协余地（full stop）。 [16:10] Juliana Marchand: 我们无法构建我们无法衡量的事物 [17:10] Juliana Marchand: 框架先于系统，照护先于框架。而人，活生生的人类，永远、永远先于一切。（The framework precedes the system. The caregiving preceded the framework. And the person, the human, always, always precedes everything.）

Source quotes

Juliana Marchand: What if the biggest failure in conversational AI isn't what it says, but what it can't hear? Juliana Marchand: Conversational AI has been designed, almost without exception, for users whose language works. Juliana Marchand: These populations don't stop communicating when language degrades. They communicate through silence, tone, grunts, repeated phrases, and invoked memories. Current AI systems aren't designed to receive any of it. [0:04] Juliana Marchand: What happens to the people that AI was never designed to hear? [2:47] Juliana Marchand: I am here. I am trying. I am frustrated. I am in pain. I need you to slow down and try to listen. [4:35] Juliana Marchand: Non-language is language. Silence is data. Tone is primary. A grunt carries grief or pain or lack of acknowledgement or relief. And a system that cannot tell the difference is not a companion. It's a well-intentioned failure. [16:10] Juliana Marchand: A conversational AI that cannot read distress silence should not be deployed in a care setting, full stop. [16:10] Juliana Marchand: we cannot build what we cannot measure. [17:10] Juliana Marchand: He taught me that communication does not stop when language does. It transforms. And the people who cannot speak the way we expect is not communicating less, is communicating differently.

Original transcript

[0:04] My name is Juliana Marchand. I am  the founder and CEO of Marchand AI. [0:09] And I am here with you today because of a question  that kept coming to me while I was building [0:15] um a AI companion for seniors with  dementia. The question that kept coming was [0:22] what happens to the people that AI was never  designed to hear? That's an important question. [0:33] We built AI to listen and act. And that was  the promise. And that is a very good promise. [0:41] AI has opened doors that were never opened before.  We can do better research. We can communicate [0:48] better. We can access information. We can be  heard like never before. But what about the people [0:57] that AI cannot hear? Not everybody is  heard. Should we just leave them behind? [1:07] Who are we building AI for? And I  want you to sit with that question. [1:13] Because everything that I am about  to tell you is my answer to it. [1:21] Let me tell you about who we're  talking about. 55 million people [1:27] in the world suffer with an Alzheimer  disease and related dementias. [1:33] 1 in 36 children in the US  alone are diagnosed with autism. [1:40] 180,000 people are new cases of aphasia every year  in the US. That means people who had language but [1:49] lost it because of a stroke. And those are just  numbers that I could put on a slide. We also have [1:57] as stroke survivors, people in end-of-life care,  people with non-verbal developmental conditions, [2:04] children with a speech impediments, and this  one may surprise you because it surprised me. [2:11] People sitting in courtrooms and deposition rooms  and hearings whose pauses and hesitations are [2:17] being misread by AI transcription every  single time. And life's that changed. [2:26] These are not just edge cases. These are our  parents, our children, our friends, our neighbors. [2:35] And right now every major conversational AI  system in the market was not designed for them. [2:47] I want to tell you about my uncle because  he's the one who started me on this path. [2:54] He had Alzheimer's disease. And the disease  took his ability to communicate as we perceive [3:02] communication. First, he started forgetting  names. Then the sentences started to go. [3:11] And then the words. So but he would still  communicate and he resorted to grunts, [3:18] sighs, moans. But most people just heard a noise. [3:23] Most people just heard him trying but actually  not saying anything. I saw something different. [3:32] He would um I I heard I am here. I am  trying. I am frustrated. I am in pain. [3:42] I need you to slow down and try  to listen. That broke my heart. [3:49] And this was just not my intuition as someone  who loved him. The research backs it up. [3:56] Clinical literature literature tells us that  sustained granting correlates with anxiety [4:03] and agitation. That moaning is significantly  associated with pain and physical discomfort. [4:11] That humming frequently accompanies calm and  self-soothing. A grunt is not noise, is data. [4:21] It is classifiable signal. We just  never trained AI to classify it. [4:27] And that is where this framework started,  not in a laboratory by but by his bedside. [4:35] Now I want to take you somewhere different. A  deposition room, a hearing room, an interrogation. [4:42] I have worked as a transcriptionist in a  legal system for um courts, um interrogations. [4:52] And there is a pattern that I kept seeing  that worried me. Someone is being questioned [4:58] and they have any speech impediment or they  are processing slowly or they're nervous [5:04] or they're simply someone whose language does not  move at the pace that the legal system expects. [5:11] There is a pause. 6 seconds, 8 seconds,  and all a hum, a wandering thought. [5:21] And the AI transcript marks it as a no response.  And that is later assumed as a hesitation [5:29] or admitting guilt. And a life is changed. That  6-second pause or 8-second pause was not silence. [5:39] It was a person trying to formulate an  answer under pressure against the clock [5:44] that was never designed for them. The AI could  not read it. So, the legal record got it wrong. [5:52] Non-language ignored in a deposition is not a UX  problem. It's a justice problem. And the people [6:01] in this room, in healthcare, in government,  and investment have the power to change that. [6:08] So, here is the argument that I am making. And I'm  going to say it as plainly as possible. As you can [6:14] see it there, huge in the screen. Non-language  is language. Silence is data. Tone is primary. [6:28] A grunt carries grief or pain or  lack of knowledge meant or relief. [6:34] And a system that cannot tell the difference is  not a companion. It's a well-intentioned failure. [6:41] That is the thesis. And everything that  I'm about to tell you is how we fix it. [6:49] The first thing we need to do is  define what non-language actually is. [6:55] Because you cannot govern what you cannot name.  So, this framework starts with a taxonomy. [7:02] I love taxonomies, by the way. So, ask  me anything about taxonomy. I'm here. [7:06] Five categories of non-verbal signals  that AI must be designed to read. [7:13] The first is vocalic non-language.  And as you can see, there is grunts, [7:18] sighs, moans, humming. Each one carries  content. A grunt is not noise. It's data. [7:29] The second one is latency and silence  patterns. And this one is crucial. [7:36] Silence is not uniform. We have several types of  silences, but we have classified this in three. [7:44] The process in silence. The person is still  formulating the thought. Give them time. [7:53] Then we have the lost thread silence.  They have lost the thread of thought. [8:00] Offer a gentle rephrase. Then we have distress  silence. Something is wrong and the system needs [8:07] to act. Three different silences, three different  responses, reading the face alongside the clock. [8:16] Then we have the prosodic residue. This  is the emotional intent that survives [8:23] in pitch, tempo, and rhythm even after the words  are gone. Tone carries truth when language cannot. [8:33] Um like I am Peruvian and my stepdad is from  Cuba and sometimes it's funny because we just [8:40] communicate with ha's. I could say ha and he  says ha. Then I said aha. So there is language. [8:49] You can see language being displayed there. The  fourth one is perseveration. When someone keeps [8:56] returning to the same word or phrase, most systems  treat that as a loop that needs to be broken. [9:04] This framework says that is an unresolved  thought. Follow the thread until it resolves. [9:11] And the fifth one is memory invocation. When  somebody with dementia pivots mid-conversation [9:18] to a memory, whether it's accurate  or not or follows the thread or not, [9:23] that is not derailment. That memory still carries  emotional relevance to the present moment. [9:29] Never correct it. Follow it. It will  tell you how they are feeling right now. [9:38] So now that we know what the signals are,  we need to know how a system and AI system [9:44] should respond to them. And this is  what this governance framework does. [9:50] Five parameters, and I want to be clear, they are  not individuals. They They work together. They [9:57] don't precede one another. They are not in order,  first one, then second, then third. They are [10:04] um They condition each other because that's how  real communication works. Parameter number one. [10:13] The conditional response threshold. [10:17] The system waits a minimum of 5 seconds  before responding. And during those 5 seconds, [10:24] it is not just waiting. It is reading the  face, expressions, mannerisms, body language. [10:33] Time plus visual observation, never time alone,  is processing, is working. Parameter number two. [10:43] The rephrasing escalation protocol. If there is no  response, the system rephrases. Simpler, shorter, [10:53] yes or no if possible. It does this a maximum of  three times. After three, it does not persist. [11:01] It exits gracefully because persistence becomes  pressure, and pressure becomes distress. [11:09] Parameter three. The tone  primary signal hierarchy. [11:15] Prosody is preceded is processed first, and  words are secondary. And when they conflict, [11:23] when the tone says one thing and the words  say another, then the system does not assume [11:28] that is dangerous. It confirms with a  simple yes or no question before it acts. [11:35] Parameter four, the interpretable thought  threshold. The system may not infer beyond [11:44] what the signal actually supports. It does not  complete thoughts that have not been offered. [11:51] It does not hallucinate, and we here an  AI understand the word hallucination. [11:59] And I choose that word on purpose, hallucination.  Because for a person with dementia, [12:05] an AI that fills the gaps with its own  assumptions is not helpful. It is dangerous. [12:12] Parameter five, the distress escalation protocol. [12:17] When the system detects acute distress, pain,  agitation, self-directed behavior, it is stops [12:24] the conversation immediately and notifies a  human caregiver. It does not try to manage [12:30] the crisis alone. It needs to know an AI needs  to know the boundaries of its own competence. [12:37] And honestly, I think that's the most  important safety principles in AI that I know. [12:45] I want you to stay on that safety point for  just a moment. Because I think it touches on [12:51] something important. When a cognitively  intact person receives a hallucination, [13:00] it's at most a minor inconvenience. It can  rephrase, it can correct, it can say that's [13:06] not what I mean. And it can train the AI again.  But for a person with dementia, that same misread [13:14] is a compounding harm. Because they cannot  correct the misinterpretation. They cannot say, [13:21] "No, that is not what I meant." So, the system  error becomes the reality, and that is wrong. [13:29] The confidence of an AI system must be earned  by the signal, not be projected onto it. [13:36] So, this is not a conversation about  how to make chatbots more polite. [13:41] This is a conversation about what it means to the  deploy AI safety in high-stake care environments. [13:49] And that conversation needs  to happen in rooms like this. [13:55] So, the framework that I just walked you through  is the same work that we are using in Soyla Care, [14:01] a proactive conversational AI companion for  seniors with dementia that I am building at [14:06] Marsha AI. I say proactive because Soyla  initiates, it doesn't wait for a prompt. [14:13] Because the people who needs her the most are  not able to utter um a prompt. Soyla runs on a [14:22] video interface, like Google Nest or Alexa video.  Because reading silence requires seeing a face. [14:30] Audio alone cannot tell you whether that pause  is a processing pause or a distress pause. [14:38] Soyla Care is being built  across nine languages right now. [14:42] Because the acoustic correlates of emotion vary by  language and culture. And a system calibrated only [14:49] on English will systematically misread Spanish.  And for that population, that is a safety failure. [14:57] Uh Soyla Care has been accepted into the Invidia  Inception, AWS Startups, and Google for Startups. [15:04] And the infrastructure for  scale is being built right now. [15:09] Soyla Care is only the starting point. This  framework does not only belong to dementia. [15:16] It belongs to every population for whom verbal  language is unreliable, incomplete, or absent. [15:23] Children with autism, stroke survivors with  aphasia, people at the end of life, children [15:29] with nonverbal developmental conditions whose  communicative profiles are documented on IEPs, [15:36] but no one has ever thought them to apply to AI. [15:41] And as I showed you earlier, people in legal  settings, depositions, interrogations, hearings, [15:47] anywhere that a pause gets missed,  read, and a life gets altered. [15:52] If you are in healthcare, this is your patient. If  you are in government, this is your constituent. [16:00] If you are an investor, this is an  enormous underserved market that [16:04] every AI company on the planet has walked past. [16:10] So, here is what I'm asking in for this  room today. If you are an investor, [16:15] the populations this framework serves  underserved by every AI system on the market. [16:22] That is not a niche. That is an unmet need at  enormous scale. Soy la care is the vehicle, [16:29] and the framework is the moat. Contact me.  If you are in healthcare or government, [16:35] I am asking you to set the standard,  please. A conversational AI that cannot [16:40] read distress silence should not be  deployed in a care setting, full stop. [16:46] You have the authority to require that. Use it. If  you are a builder, join the research. We need to [16:54] share the identified interaction data from these  populations to validate and extend the framework. [17:01] We cannot build what we cannot measure. Reach  out. Let's build the infrastructure together. [17:10] I want you I want to leave you with the thought  that has driven everything that I have shared [17:14] today. This framework did not start in a  laboratory. It started by the bedside of [17:21] someone I loved who was trying to reach  me through language that was failing him. [17:27] He taught me that communication does not stop  when language does. It transforms. And the [17:34] people who cannot speak the way we expect is not  communicating less, is communicating differently. [17:42] We build AI to listen and act. It is time  that we build it to listen to everyone. [17:50] The framework precedes the system. The caregiving  preceded the framework. And the person, the human, [17:57] always, always precedes  everything. Thank you. [applause]