Manulife Executive Says Voice AI Cuts Call Center Costs 80%

宏利高管称语音AI可削减呼叫中心80%的成本 将日常客户支持从人工坐席转移至具备上下文感知能力的语音模型，可将单次通话成本从9.00美元锐减至0.40美元，同时将延迟降至185毫秒。 Venkata Harikishan Koppuravuri概述了联络中心从传统IVR分支结构向具备上下文感知的智能体语音AI的转型，该技术将单次通话的运营成本从高达9.00美元降至约0.40美元。他重点介绍了包括低延迟语音到语音模型和声音生物识别在内的关键技术里程碑，并探讨了如何通过15秒的语音生物信号检测帕金森病等病症。最后，会议探讨了关键的企业治理挑战，重点关注深度伪造风险、EU AI Act等监管框架，以及将AI视为组成构件而非独立系统的核心原则。 与视觉识别相比，个体对人类声音的自我感知通常极为困难，尽管其中蕴含着丰富且尚未挖掘的信号。 (0:04) 通过AI语音智能体取代常规人工交互来推动联络中心现代化，可将单次通话成本从7.00–9.00美元降至约0.40美元（降幅达80%以上）。 (2:22) 人类声音可划分为三个分析层级：对话界面、气管生物信号和独特的生物特征身份。 (2:22) 联络中心已从2017年时代僵化的IVR分支和基础NLU，演进为能够保留通话者上下文的上下文感知型智能体语音系统。 (5:51) 声音生物特征认证仅凭声音特征即可自动认证呼叫者身份，免去了人工识别环节并降低了平均处理时间（AHT）。 (5:51)

Venkata Harikishan Koppuravuri of Manulife John Hancock argues that voice AI is rapidly evolving from basic call routing into agentic systems capable of biometric authentication and physiological screening, though scaling these tools requires embedding them into broader enterprise architectures governed by strict compliance controls. Contact Center Economics and Low-Latency Voice Architectures (2:22) Modernizing contact centers by replacing traditional Interactive Voice Response (IVR) command trees with voice AI agents can reduce operating costs from $7.00–$9.00 per traditional human agent call down to approximately $0.40 per call, cutting expenses by over 80%. A key operational efficiency comes from voice biometrics, which authenticates callers through their vocal signatures rather than manual account verification, achieving roughly a 37% customer consent rate in Koppuravuri's enterprise environments. Additionally, contact centers are shifting key performance indicators away from simple call containment toward resolved intent. Technological progress in latency has made real-time agentic interactions viable. Traditional pipelines that chained Automatic Speech Recognition, large language models, and text-to-speech produced latencies exceeding 800 milliseconds, causing over 80% of users to hang up. Frontier speech-to-speech models have reduced this latency to 185 milliseconds. Backed by $1.23 billion in venture funding in January alone, Koppuravuri notes the broader voice AI market is projected to expand from $2.4 billion to $47.5 billion by 2034. Passive Health Screening Through Vocal Bio-Signals (13:38) Beyond conversational interfaces, vocal bio-signals offer non-invasive diagnostic capabilities. Because voice production involves physical coordination across the trachea, vocal cords, and internal organs, speech carries acoustic markers that reflect underlying physiological conditions. Next-generation speech-to-speech tokens decode these acoustic properties natively, moving past older methods that relied on manual Speech Synthesis Markup Language (SSML) tags to approximate vocal tone. Analyzing just 15 seconds of speech for micro-tremors, pitch variations, intensity shifts, and breathing patterns can yield actionable health data. Koppuravuri highlights that voice AI algorithms can identify early indicators of Parkinson's disease with over 80% accuracy across men and women, while showing strong acoustic correlations with respiratory illnesses and depression, enabling passive screening during routine customer calls. Mitigating Deepfake Fraud and Regulatory Enforcement (16:48) The rapid advancement of voice synthesis introduces severe security risks, including identity theft, authentication bypass, and unauthorized voice cloning. Bad actors harvest short audio snippets to impersonate authority figures or deceased family members, while commercial attempts at "digital resurrection" create unresolved legal questions regarding posthumous consent and vocal likeness rights. To counter synthetic fraud, global regulatory frameworks implemented major compliance deadlines around August 2nd. The EU AI Act, alongside emerging regulations in China and the United States, imposes strict requirements on enterprise voice platforms. Organizations are now legally mandated to disclose to users when they are conversing with an AI agent and must embed imperceptible vocal watermarks into synthetic audio to ensure traceability and avoid steep financial liabilities. Integrating Voice AI as an Enterprise Component (20:00) Drawing from implementations across healthcare, fintech, and insurance, Koppuravuri cautions organizations against treating AI models or voice bots as standalone business solutions. While startups and enterprise teams frequently position a bot as a complete product, voice AI lacks operational value without deep integration into underlying enterprise workflows, identity management, and governance frameworks. Sustainable modernization requires viewing conversational AI strictly as a functional component within a wider, interconnected architecture. While agentic voice AI offers significant efficiency gains and diagnostic capabilities, unresolved questions persist around vocal consent, legal protections against posthumous cloning, and caller willingness to adopt biometric verification. Long-term enterprise success depends on treating voice models as modular components supported by rigorous governance and mandatory watermarking.

Manulife John Hancock 云工程总监 Venkata Harikishan Koppuravuri 提出，基于语音大模型与生物特征识别的代理型语音 AI 正在重塑企业联络中心，不仅能大幅削减运营成本，还能挖掘健康生物标志物，但企业必须将其视为整体架构的组件并严格应对治理与欺诈风险。 联络中心成本骤降与技术架构演进 (2:22) 根据 Koppuravuri 的从业经验，企业通过在人工客服与 AI 语音代理之间取得平衡，将日常咨询迁移至自动化语音机器人，可使联络中心运营成本降低 80% 以上。传统人工坐席单次通话成本约为 7.00 至 9.00 美元，而现代化 AI 语音代理的单次通话成本降至约 0.40 美元。联络中心的评估核心也从单纯阻止转接人工的“拦截率”（containment），转向衡量客户问题是否得到彻底解决的“意图解决率”（resolved intent）。在技术路线上，客服系统从 2017 年依赖按键的 IVR 树状菜单和早期自然语言理解（NLU），演进至端到端语音到语音（S2S）前沿模型。以往由自动语音识别（ASR）、大语言模型（LLM）与文本转语音（TTS）拼接的多阶段流水线，因延迟超过 800 毫秒导致逾 80% 用户挂断；新一代 S2S 模型将延迟压缩至 185 毫秒，实现了即时自然的交互。同时，语音 AI 市场正迅速扩张，行业单月融资曾达 12.3 亿美元，整体市场规模预计将从 24 亿美元增长至 2034 年的 475 亿美元（增长 20 倍）。 声纹身份认证与健康生物标志物 (13:38) Koppuravuri 将人类语音划分为接口交互、生物信号和身份特征三个层面。在身份验证方面，声纹生物识别能够在获得客户明确同意的情况下自动核验身份，免除询问社保号码、出生日期或账户 ID 的繁琐流程，从而缩短平均处理时长（AHT）。在其负责的受监管企业业务中，客户声纹录入的同意率平均约为 37%。在生物信号层面，语音形成需气流经气管并协调内脏器官，因此包含丰富的生理健康数据。模型仅需分析 15 秒的语音，即可检测音高变化、微震颤、强度波动及呼吸停顿等声学特征。研究显示，该技术检测帕金森病的准确率超过 80%（男女皆适用），并与抑郁症及呼吸道病变高度相关。以往工程师依赖 SSML 标签模拟人类情绪，而原生语音到语音技术能够直接保留和解析这些声学特征，使得在常规呼叫中被动进行健康评估成为可能。 深度伪造风险与全球合规治理 (16:48) 语音技术的普及带来了身份盗用、认证绕过以及未经授权“数字复活”逝者等严重风险。不法分子通过截取音频片段克隆声音，冒充亲属或权威机构进行网络欺诈；在娱乐领域，使用已故名人的声音重现亦引发了严重的法律与知情同意争议。针对日益严峻的深伪威胁，全球监管机构于 8 月 2 日前后推进了合规执法要求。例如，欧盟《人工智能法案》（EU AI Act）强制要求企业必须明确告知用户当前正与 AI 语音代理通话，并在合成音频中嵌入不可磨灭的声学水印；中国出台了针对合成语音的标注与登记要求；美国联邦及各州也在推动严格责任与隐私保护防护机制。若未能建立透明度及声学水印防线，企业将面临重大的法律处罚和财务追责风险。 架构集成：AI 是系统组件而非整体方案 (20:00) Koppuravuri 结合受监管行业的工程落地经验指出，企业在扩展新技术时常面临严峻的运营挑战，因此切勿将人工智能视为一揽子端到端解决方案。AI 和语音机器人必须严格作为大型互联企业架构内部的功能组件来定位。针对部分初创团队将“打造了一个语音机器人”作为全部价值定位的现象，他明确反对这种单一思维，强调独立的语音模型由于欠缺更广泛的系统上下文、系统间集成以及治理机制，无法在大型企业环境中有效运转。企业实施联络中心现代化升级时，需要依靠系统性的架构蓝图与整合模式，以确保语音 AI 在合规与业务流程中协同生效。 语音 AI 虽然为联络中心带来巨大的降本潜力和多维感知能力，但其落地依赖于深度的企业系统集成与严密的安全治理。未经授权的克隆黑客手段与滞后的知情同意法律框架仍是悬而未决的挑战，合规与水印机制将成为企业不可逾越的底线。

## Background 该视频由 Ai4 于 2026 年 8 月 21 日发布在 YouTube 平台。内容为宏利金融/恒康保险（Manulife John Hancock）云工程总监兼《Thinking Voice》作者 Venkata Harikishan Koppuravuri 的主题演讲，主要探讨如何利用 Agentic AI 语音智能体实现企业联络中心现代化改造、提升意图解决率、降低运营成本，并构建强监管环境下的 AI 治理与安全框架。 ## TL;DR 演讲者结合在大型受监管金融保险机构的落地经验，分享了语音 AI 如何通过人机协同将呼叫中心单通处理成本从 7-9 美元直降至 40 美分，实现超 80% 的成本削减。技术层面，系统正由高延迟的传统 ASR 串行架构升级为响应仅 185ms 的端到端 Speech-to-Speech 大模型，推动核心考核指标从单纯的“拦截率”转向“意图解决率”，并引入了声纹无感认证与 15 秒语音生物标记物筛查疾病的前沿应用。最后，演讲强调了防范“数字复活”与声音克隆欺诈的伦理与监管要求，重申语音 AI 只是系统中的一个功能组件，必须与企业全局风控与合规治理深度集成。 ## Atmosphere & style 演讲开篇以幽默的“摔角登场”自嘲和百元悬赏互动轻松破冰，瞬间抓住了全场注意力，随后迅速切入扎实硬核的工程架构与 ROI 实战数据。演讲者兼具一线架构师的技术洞察与高管的战略视野，语调从展现技术降本增效时的从容笃定，自然过渡到探讨声音克隆犯罪与伦理边界时的严肃与警醒。整场分享节奏紧凑、案例生动，既展现了端到端语音前沿突破带来的兴奋感，又传递出对强监管下 AI 治理与安全的敬畏之心。 ## Key takeaways - [03:48](https://www.youtube.com/watch?v=jjOcScpHpqU&t=228s) 语音兼具接口层、生物信号层与身份层三重属性，蕴含着远超文字本身的生理健康特征与声纹身份信息。 - [02:15](https://www.youtube.com/watch?v=jjOcScpHpqU&t=135s) 在受监管企业中，语音 AI 与人工坐席的高效协同可降低超过 80% 的联络中心运营成本，将单通处理成本从 7-9 美元降至约 40 美分。 - [05:51](https://www.youtube.com/watch?v=jjOcScpHpqU&t=351s) 现代化联络中心通过声纹生物特征实现无感静默身份认证，消除了背诵 SSN 等繁琐验证环节，显著缩短了平均处理时长（AHT）。 - [05:51](https://www.youtube.com/watch?v=jjOcScpHpqU&t=351s) 语音 AI 市场正在爆发式增长，全球市场规模预计将从当前的约 24 亿美元急剧扩张至 2034 年的 475 亿美元。 - [09:22](https://www.youtube.com/watch?v=jjOcScpHpqU&t=562s) 传统的 ASR-LLM-TTS 串联流水线存在端到端延迟高（800ms-1s）和单点脆弱的硬伤，难以满足严苛的生产环境要求。 - [11:59](https://www.youtube.com/watch?v=jjOcScpHpqU&t=719s) 端到端语音大模型（Speech-to-Speech）将交互响应延迟压缩至 185 毫秒，大幅提升了实时对话的自然度与精准度。 - [11:59](https://www.youtube.com/watch?v=jjOcScpHpqU&t=719s) 联络中心的核心评估指标正从单纯追求通话不转人工的“拦截率”（Containment Rate）转向真正解决客户问题的“意图解决率”（Resolved Intent）。 - [13:38](https://www.youtube.com/watch?v=jjOcScpHpqU&t=818s) 基于前沿端到端语音 Token 解码，系统仅需分析约 15 秒声音即可对帕金森病、抑郁症等疾病进行早期预测，准确率超 80%。 - [16:48](https://www.youtube.com/watch?v=jjOcScpHpqU&t=1008s) 语音克隆与“数字复活”技术带来了严重的身份伪造与伦理欺诈风险，监管机构已要求企业必须明确披露 AI 身份并强制嵌入数字水印。 - [16:48](https://www.youtube.com/watch?v=jjOcScpHpqU&t=1008s) 语音机器人或 AI 并不是独立的完整系统，而只是企业整体业务与治理架构中的一个功能组件，必须与风控和合规深度集成。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=jjOcScpHpqU&t=4s) 语音与 AI：一个被低估的双重命题与有奖互动 演讲者 Venkata Harikishan Koppuravuri（大家都叫他 Hari）以幽默的开场拉开序幕，打趣主持人热情洋溢的介绍仿佛让他置身于 "WWE 职业摔角登场现场"。随后他明确了整场演讲的核心主线：平衡探讨两个极具分量的主题——**语音（Voice）**与**人工智能（AI）**。 在正式展开前，Hari 向全场观众抛出了一个悬赏 100 美元的互动挑战：“大家可以闭上眼睛，随意借助 GPT、各类大语言模型或任何工具，只要能以**倒序**的方式复述出你刚刚说过的最后三句话，就能拿走这 100 美元。” 现场无人能够应答。Hari 由此引出核心观点：语音对人类而言既熟悉又极其神秘。例如，在庞大人群中看一张照片，你能瞬间认出自己的脸；但如果仅仅播放一段录音，普通人甚至很难 100% 确信那是自己的声音。语音中潜藏着极其丰富、过去未被充分挖掘的物理与生理信号。 --- ### [02:15](https://www.youtube.com/watch?v=jjOcScpHpqU&t=135s) 生产级部署实战：成本降低 80% 与《Thinking Voice》 Hari 介绍了他兼具大型受监管企业高管与独立研究者的双重背景。他任职于一家拥有 170 年历史的金融保险机构（宏利金融 / 恒康保险），并曾在医疗健康与金融科技两大强监管行业深耕多年，负责管理横跨 8 个业务线（LOB）的联络中心现代化改造。针对市面上大量在缺乏规范或极少监管下盲目上线的语音机器人现状，他撰写并出版了《Thinking Voice》一书，系统性总结了受监管企业在语音 AI 治理、落地模式及工程架构方面的实战经验。 在呼叫中心智能化改造的商业回报（ROI）方面，Hari 分享了极具震撼力的真实数据：通过在语音机器人与人工坐席之间建立高效的协同平衡，团队成功降低了超过 **80%** 的联络中心运营成本。 - **人工接听单通成本**：原本完全由人工坐席处理的来电，单通成本约为 **7 至 9 美元**。 - **AI 全流程接听单通成本**：在最佳场景下，语音 AI 完整接听并解决问题的单通成本直降至约 **40 美分（$0.40）**。 --- ### [03:48](https://www.youtube.com/watch?v=jjOcScpHpqU&t=228s) 语音的三层解构与演讲的三阶段框架 Hari 指出，人类的语音信号绝非凭空产生的简单声波，而是从气管出发、与体内多个脏器精密协调后发出的复合载体。他将语音在技术与分析维度划分为三个核心层级： 1. **接口层（Interface）**：自然的交流与对话交互流动，即人们日常所讲的文字与语意内容。 2. **生物信号层（Bio Signal）**：蕴含人体健康标记（Health Markers）的生理特征。由于发声机制涉及复杂的身体器官运动，语音能够携带远超人们想象的身体机能与健康数据。 3. **身份层（Identity）**：用于唯一表征说话人身份的声纹与生物特征。 基于此，Hari 设定了整场演讲的三个核心阶段： - **第一部分（业务与体验）**：联络中心语音 AI 的现代化改造路径，以及如何实现客户体验（CX）的跨越式提升。 - **第二部分（治理与合规）**：在强监管企业中落地治理框架的实战机制与踩坑经验。 - **第三部分（警示与风险）**：揭示技术突破背后令人不寒而栗的安全与隐私隐患，为全场敲响警钟。 --- ### [05:51](https://www.youtube.com/watch?v=jjOcScpHpqU&t=351s) 联络中心演进史：从按键 IVR 到声纹自动认证 回顾从 2017 年至今的演进历程，联络中心的客户体验发生了颠覆性变革。Hari 举了一个极具共鸣的痛点场景：当用户拨打电信运营商（如 AT&T 或 T-Mobile）或银行客服处理疑难问题时，如果沟通长达一小时后电话意外中断，再次拨入往往需要向新坐席从头复述所有背景上下文，极其消耗时间，体验堪称灾难。 早期的交互完全依赖指令式 IVR（"中文请按 1，英文请按 2"），随后引入自然语言理解（NLU），如今已全面迈入 Agentic AI 与语音生物特征（声纹）融合的时代。在现代化语音 AI 系统中，只要客户事先授权同意，拨入电话时系统会通过声纹生物特征进行**完全无感的静默自动认证**，用户无需再背诵社保号后四位（SSN Last 4）、出生日期或客户编号，直接将繁琐的身份验证耗时归零，大幅缩短了平均处理时长（AHT, Average Handle Time）。 ```mermaid flowchart LR accTitle: 联络中心身份认证演进流程 subgraph Traditional["传统身份认证流程"] A["拨入电话"] --> B["按键/NLU菜单"] B --> C["口述验证信息\n(SSN/生日/ID)"] C --> D["耗费大量 AHT\n接通人工坐席"] end subgraph Modern["现代化 Agentic 声纹认证"] E["拨入电话"] --> F["声纹无感比对\n(获取授权后)"] F --> G["即时完成认证\n(验证耗时降为 0)"] end ``` - **声纹授权采纳率**：在其实际负责的 8 个强监管业务线中，声纹生物标记的客户授权同意率约为 **37%**（且多数客户并未完全意识到授权背后的深层数据含义）。 - **语音 AI 市场爆发**：仅在当年 1 月份，语音 AI 领域获批的融资金额就达到约 **12.3 亿美元**；当前整体市场规模约为 **24 亿美元**，预计到 2034 年将急剧扩张至 **475 亿美元**（迎来超 20 倍的市场膨胀）。 --- ### [09:22](https://www.youtube.com/watch?v=jjOcScpHpqU&t=562s) 监管法案潮与传统 ASR 串联架构的延迟硬伤 在谈及合规环境时，Hari 指出北美此前在 Agentic AI 领域长期缺乏统一的全局监管法规（相较于欧盟和中国），导致各州法律碎片化（例如得克萨斯州有专门的法规考量）。然而监管风暴已然来临，特别是在 8 月 2 日这一天，欧盟 AI 法案（EU AI Act）、中国以及北美均密集推出了里程碑式的监管更新与合规细则。如果企业未能跟上合规条款的要求，将面临巨大的诉讼风险。 在系统架构演进方面，Hari 深入剖析了早期语音 AI 生产级部署的技术瓶颈。最初团队采用的是经典的串行流水线架构（ASR 驱动模式）： ```mermaid flowchart LR accTitle: 经典 ASR 串联语音处理流水线 A["用户语音输入"] --> B["ASR 语音转文本"] B --> C["LLM 思考与生成文本"] C --> D["TTS 文本转语音"] D --> E["合成语音输出给用户"] ``` 这种类似单体应用（Monolithic）的 10 步串联架构存在两大致命缺陷： 1. **链路脆弱性**：整个管道对各组件的完美运行高度敏感，任何一个环节异常或出错，全链路立即崩溃。 2. **端到端延迟灾难**：串行处理累积了巨大的响应延迟。在语音交互场景中，若响应延迟达到 **800 毫秒至 1 秒**，就完全不可用于生产环境——"想象一下你问了一个问题，语音机器人停顿整整 1 秒才回答，客户早就直接挂机了。" ### [11:59](https://www.youtube.com/watch?v=jjOcScpHpqU&t=719s) 从语音识别到端到端语音大模型：服务指标向“问题解决”演进 在客服互动场景中，研究表明超过 80% 的客户极度缺乏耐心——大家拨打电话唯一的目的就是快速获取答案。过去基于传统自动语音识别（ASR）的系统表现相当糟糕，当系统在后台进行分析时，任何微小的停顿都会让客户产生怀疑：“这个机器人到底知不知道答案？”而前沿技术向端到端语音（Speech-to-Speech，S2S）前沿模型的升级彻底改变了这一局面。如今系统响应延迟已大幅优化至惊人的 185 毫秒，不仅反应极快，而且回答极为精准。 与此同时，呼叫中心的核心 KPI 也发生了根本性转变。过去业界过度关注“拦截率”（Containment Rate，即让客户在机器人环节结束通话而不转接人工），但团队意识到，客户直接挂机并不代表问题得到了解决，单纯追求拦截率反而会损害体验。因此，关键指标已正式转变为“意图解决率”（Resolved Intent）。语音 AI 正在演变为贯穿多个行业的底层操作层（Operating Layer）。 - **响应延迟**：前沿 Speech-to-Speech 模型将处理与响应延迟压缩至 185ms。 - **KPI 重塑**：从单纯不转人工的 Containment 转向真正解决客户诉求的 Resolved Intent。 ```mermaid flowchart LR accTitle: 语音交互架构与指标演进 subgraph S1["传统架构 (ASR + 文本处理)"] A["用户语音输入"] --> B["ASR 文本转换"] B --> C["高延迟思考停顿"] C --> D["传统考核: 拦截率 (Containment)"] end subgraph S2["前沿架构 (Speech-to-Speech)"] E["用户语音输入"] --> F["S2S 端到端模型 (185ms 延迟)"] F --> G["核心考核: 意图解决率 (Resolved Intent)"] end ``` --- ### [13:38](https://www.youtube.com/watch?v=jjOcScpHpqU&t=818s) 语音生物标记物：通过 15 秒声音前瞻性预测疾病 脱离 AI 概念本身，声音本身就蕴含着巨大的科研价值。演讲者即将发表的研究论文表明，仅凭人类的声音特征，系统就能够以超过 80% 的准确率识别男性与女性的多项生理及心理疾病，包括帕金森病（Parkinson's Disease）、抑郁症乃至 COVID-19。如果在呼叫中心接入这种分析能力，当客户致电办理业务时，语音 AI 可以在客户自身毫无察觉的情况下，捕捉到抑郁或帕金森病的早期潜在征兆并主动发出健康预警，从而实现相当于“免费初步问诊”的预防性关怀。 在通用场景下，系统通常仅需提取客户大约 15 秒的声音片段，即可通过音高（Pitch）、呼吸停顿、能量、强度及声音质感等声学特征进行综合判断。过去工程师利用语音合成标记语言（SSML）标签为人造声音注入情感与语调；如今在端到端语音模型中，系统直接在解码管道中利用语音标记（Voice Tokens）提取生物标记物（Voice Biomarkers）。需要强调的是，这并非最终的医疗确诊，而是语音 AI 对异常情况发生概率的高精度预测。 - **识别准确率**：帕金森病、抑郁症及其他多类疾病的检测准确率普遍达到 80% 以上（基于全球公开数据集与实证研究）。 - **极短采样时间**：常规情况下仅需约 15 秒客户语音即可提取关键特征。 - **技术原理**：从 SSML 情感标签演进为基于 Speech-to-Speech 语音 Token 的声学标记物直接解码。 --- ### [16:48](https://www.youtube.com/watch?v=jjOcScpHpqU&t=1008s) 语音克隆风险、数字复活伦理与企业级 AI 治理 随着语音生成技术的飞跃，安全治理与伦理合规成为了必须严控的底线。演讲者举了一个令人不寒而栗的真实威胁场景：如果一位多年前去世的至亲突然给你打来电话，声音、口吻完全一致，甚至知晓你们之间的所有私密背景，进而索要密码或敏感凭证，绝大多数人都难以防范。这种被称为“数字复活”（Digital Resurrection）的技术正在引发严重的网络诈骗与身份冒用危机；同时，好莱坞也出现了未经已故演员生前明确许可便克隆其声音用于商业宣传的法律争议。 由于口头同意难以取证且逝者无法现场维权，语音克隆绕过身份验证导致的欺诈损失正逐年攀升。为此，全球监管正在迅速收紧，例如欧盟出台的相关法案（包括 8 月 2 日生效的相关条款）明确规定：企业在实施语音 AI 应用时，必须明确告知客户当前正在与机器人交互，且必须为 AI 生成的语音嵌入数字水印，否则将面临极其严厉的巨额罚款。 演讲者在总结时特别向广大企业和初创团队强调一个核心架构认知： > "Voice bot or the AI is not a system. It's just a component in the system."（语音机器人或 AI 并不是一个完整的系统，它仅仅是系统中的一个组件。） 企业绝不能把单点的语音模型误当成业务系统的全部，必须将其置于严格的合规审计、身份校验与全局企业架构蓝图之下进行规范化部署。 - **核心伦理与法律风险**：数字复活（Digital Resurrection）、未授权声音克隆、声纹身份验证绕过。 - **监管合规要求**：必须向用户明确告知 AI 身份，语音输出必须嵌入合成水印。 - **企业架构定调**：AI 只是业务体系中的功能组件，必须与外围合规、风控及业务系统深度集成。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=jjOcScpHpqU&t=4s) 大家可以闭上眼睛，随意借助 GPT、各类大语言模型或任何工具，只要能以倒序的方式复述出你刚刚说过的最后三句话，就能拿走这 100 美元。 — Venkata Harikishan Koppuravuri > [00:04](https://www.youtube.com/watch?v=jjOcScpHpqU&t=4s) 在庞大人群中看一张照片，你能瞬间认出自己的脸；但如果仅仅播放一段录音，普通人甚至很难 100% 确信那是自己的声音。 — Venkata Harikishan Koppuravuri > [02:15](https://www.youtube.com/watch?v=jjOcScpHpqU&t=135s) 通过在语音机器人与人工坐席之间建立高效的协同平衡，团队成功降低了超过 80% 的联络中心运营成本。 — Venkata Harikishan Koppuravuri > [03:48](https://www.youtube.com/watch?v=jjOcScpHpqU&t=228s) 人类的语音信号绝非凭空产生的简单声波，而是从气管出发、与体内多个脏器精密协调后发出的复合载体。 — Venkata Harikishan Koppuravuri > [05:51](https://www.youtube.com/watch?v=jjOcScpHpqU&t=351s) 只要客户事先授权同意，拨入电话时系统会通过声纹生物特征进行完全无感的静默自动认证，用户无需再背诵社保号后四位、出生日期或客户编号，直接将繁琐的身份验证耗时归零。 — Venkata Harikishan Koppuravuri > [09:22](https://www.youtube.com/watch?v=jjOcScpHpqU&t=562s) 想象一下你问了一个问题，语音机器人停顿整整 1 秒才回答，客户早就直接挂机了。 — Venkata Harikishan Koppuravuri > [11:59](https://www.youtube.com/watch?v=jjOcScpHpqU&t=719s) 客户直接挂机并不代表问题得到了解决，单纯追求拦截率反而会损害体验。因此，关键指标已正式转变为‘意图解决率’。 — Venkata Harikishan Koppuravuri > [13:38](https://www.youtube.com/watch?v=jjOcScpHpqU&t=818s) 仅凭人类的声音特征，系统就能够以超过 80% 的准确率识别男性与女性的多项生理及心理疾病，包括帕金森病、抑郁症乃至 COVID-19。 — Venkata Harikishan Koppuravuri > [16:48](https://www.youtube.com/watch?v=jjOcScpHpqU&t=1008s) Voice bot or the AI is not a system. It's just a component in the system. — Venkata Harikishan Koppuravuri ## Mentioned Venkata Harikishan Koppuravuri, Manulife, John Hancock, 《Thinking Voice》, AT&T, T-Mobile, EU AI Act, GPT, ASR, TTS, Speech-to-Speech (S2S), SSML ## Video description Venkata Harikishan Koppuravuri - Author, Director of Cloud Engineering, Manulife Johnhancock Customer service is entering a new era where Voice agents are moving towards becoming context aware systems. This session explores how Agentic AI, can personalize conversations, resolve issues faster, improve customer loyalty. We will discuss on the strategies to modernize contact centers, improve containment, reduce AHT, measure CX impact. We will also cover governance, implementation patterns, real world lessons for building trusted AI powered customer experience at enterprise scale. [Source](https://www.youtube.com/watch?v=jjOcScpHpqU)

[0:04] Venkata Harikishan Koppuravuri: 大家可以闭上眼睛，随意借助 GPT、各类大语言模型或任何工具，只要能以倒序的方式复述出你刚刚说过的最后三句话，就能拿走这 100 美元。 [0:04] Venkata Harikishan Koppuravuri: 在庞大人群中看一张照片，你能瞬间认出自己的脸；但如果仅仅播放一段录音，普通人甚至很难 100% 确信那是自己的声音。 [2:15] Venkata Harikishan Koppuravuri: 通过在语音机器人与人工坐席之间建立高效的协同平衡，团队成功降低了超过 80% 的联络中心运营成本。 [3:48] Venkata Harikishan Koppuravuri: 人类的语音信号绝非凭空产生的简单声波，而是从气管出发、与体内多个脏器精密协调后发出的复合载体。 [5:51] Venkata Harikishan Koppuravuri: 只要客户事先授权同意，拨入电话时系统会通过声纹生物特征进行完全无感的静默自动认证，用户无需再背诵社保号后四位、出生日期或客户编号，直接将繁琐的身份验证耗时归零。 [9:22] Venkata Harikishan Koppuravuri: 想象一下你问了一个问题，语音机器人停顿整整 1 秒才回答，客户早就直接挂机了。 [11:59] Venkata Harikishan Koppuravuri: 客户直接挂机并不代表问题得到了解决，单纯追求拦截率反而会损害体验。因此，关键指标已正式转变为‘意图解决率’。 [13:38] Venkata Harikishan Koppuravuri: 仅凭人类的声音特征，系统就能够以超过 80% 的准确率识别男性与女性的多项生理及心理疾病，包括帕金森病、抑郁症乃至 COVID-19。 [16:48] Venkata Harikishan Koppuravuri: Voice bot or the AI is not a system. It's just a component in the system.

[0:04] Venkata Harikishan Koppuravuri: without looking back, can anyone recite the last three sentences they just spoke? [5:51] Venkata Harikishan Koppuravuri: press 1 for English, press 2 for Spanish [20:00] Venkata Harikishan Koppuravuri: a voice bot

[0:04] Hello everyone. Uh I'm Hari and yeah, my usual  name is very long, so I go by Harikishan. [0:10] And thank you so much for that introduction. I  felt as it is a WWE introduction for a second, but [0:14] yeah, I appreciate you. So, before we start the  session, I'd like to uh set the agenda, right? So, [0:22] this session is uh going to balance two powerful  keywords. One is voice, two is AI. I don't know [0:30] which one is to be emphasized more than which, but  I'll let you all decide at the end of the session. [0:35] But uh before even we start the session, I'd have  a quick question, and you could win, let's say, [0:41] a $100 bill from me if you can answer that.  You could use GPTs, LLMs, do whatever you wish, [0:47] right? So, the question goes like this. Uh  maybe before I ask, let's close the eyes, [0:53] uh and then think through the question. What is  I mean, if you were to tell in a reverse way, [0:59] the last three sentences you just spoke. Right?  That was the question. Can anybody answer? [1:09] Weird, right? So, it's not a tough question.  It's not an easy question, either, right? Because [1:15] voice is super unknown, while it is also  also already known to us, right? For example, [1:22] can you see yourself in a large  crowd and recognize your face? [1:28] The answer is yes. But, can you  really recognize your voice? [1:35] If if some if you hear back a voice audio  signal, sorry, audio recording, let's say. [1:43] Are How sure are you that you can recognize  your own voice? The answer is maybe, right? [1:49] It's not 100% confident, because voice has a  lot of signals that are not discovered yet, [1:56] because they are not relevant till today, but with  with a lot of other things going on, which is not [2:03] relevant to AI again, voice is tremendously  progressing and will unveil all those facts [2:10] in this session. And the other thing is about  the AI. With the introduction of this AI [2:15] and the modularity of this voice, we are making  amazing progress in multiple segments. So, again, [2:22] my name is Harry and I I'm an executive in a  170-year-old organization, but also I'm an active [2:28] researcher. I'm an independent researcher. I  authored a book called Thinking Voice because [2:33] there are a lot of voice bots deployed in the  production with with zero to minimal regulations. [2:38] So, I thought I might as well release a book that  has the design patterns on the voice AI governance [2:45] and the enterprise situation that works,  right? What works or doesn't work because [2:49] across my career in two major regulated firms  and enterprises, one is on healthcare and the [2:55] other one is on fintech and insurance platform.  We have deployed, modernized the contact centers [3:00] where it's being managed and balanced between  the AI bots and the human interactions, right? [3:07] We have done a lot of cool implementations that  literally reduced 80 plus percentage of the [3:13] cost that we are going to invest on the  contact centers today. Okay, so for example, [3:19] a call that was supposed to charge around 7 to  9 dollars per call if a human takes completely, [3:25] it was now going for about 40 cents in the best in  the best case scenario, the entire call. That's a [3:30] huge huge achievement that we have progressed in  this domain so far, but then at the same time, [3:37] again, this is where you connect with me if at  all you need to talk about or learn more about [3:41] the enterprise regulated AI voice AI deployments,  but coming back to our topic, our voice. There are [3:48] a lot of hidden factors or patterns in our voice  and this can be divided into three signals. Okay, [3:55] again, just a dis- a disclaimer, I got a lot of  slides. I'll just run across for a few of them. [4:01] If you have any questions, we can absolutely  discuss about those right after the session. [4:06] But, I really did not plan for a 20- 20-minute  session, so I hope you'll bear with me. So, [4:11] the three layers of this entire voice  is interface, bio signal, and identity, [4:16] right? So, interface is your natural way of  flow. Like, whatever you speak, right? But, [4:21] then at the same time, there's something called  as bio signal, which has the health markers. [4:28] Uh your voice will not just appear out of the blue  just like that. It's a force from your trachea [4:35] that coordinates with a lot of um touch points  that touches multiple organs in your body [4:42] before it gets released out, right? So,  um these markers give us gives the data [4:50] a lot of data than that we can imagine. We  couldn't imagine that we could do these things [4:55] with those markers. I'll I'll show you about  what has been achieved so far, right? Um and then [5:03] um what can be possible in future as well, but  that sounds a little scary, too. So, my vision to [5:09] today is three parts. One, I'll talk to you about  how we made a progress in modernizing the voice AI [5:15] and contact centers and what how we achieved  the CX experience, the customer experience [5:19] has been elevated in tremendous way. That is first  part. Second part is I'd like to introduce you the [5:25] governance features, the functionalities, where we  have implemented the governance, and how did we do [5:31] that, and what are our learnings. Cool stuff.  But, the third part, I really try to scare you [5:37] as much as I can about whatever we have achieved  with the first part. Okay? So, the the third part, [5:45] if I'm successful, you'll definitely be scared at  the end of the session, but let's give it a try. [5:51] So, this is a road map on how we have improvised,  upgraded, enhanced our customer experience. [5:59] For all the people who are taking screenshots, I  appreciate it, but you can absolutely ping me on [6:03] LinkedIn. I can give you the PPT, right? So, just  just making sure that you get the exact slides. [6:09] So, from 2017 till today, there's a huge huge  upgrades that we did. So, imagine you're calling [6:18] your your customer center, a bank, or a telecom  provider, right? So, there are there are people [6:25] that are waiting on an open issue that is being  going on and on and on for about more than a year, [6:31] maybe related to a telecom provider. That's the  best example, right? If you were to call back your [6:37] AT&T or T-Mobile, then to talk to them about the  same very same issue you're facing, you're talking [6:43] to them, explaining them every call. Let's say you  spoke them with them in detail for about an hour, [6:49] the call gets disconnected, you'll have to call  back, you have to make sure they catch up to the [6:54] situation, the context. You give  everything that has happened. That [6:58] eats up a lot of time. It's a terrible customer  experience. People are facing it till today, [7:03] right? In in few of the enterprises. Right? So,  during those days, the IVR system is operated by [7:11] the command system, right? Press one for  English, press two for Spanish. From that, [7:16] we've added NLUs. We've emphasized the agent AI  interfaces where the voice bots will actually do [7:24] automatic authentication, which means you today  you call a contact center. If it is aligned with [7:31] voice AI integration, it doesn't ask your  authentication. The moment it hears your voice, [7:38] obviously it is consent-driven. Once you give  the consent to be authenticated using your voice [7:43] biometrics. Uh you will not need to provide any  authentication factors like the SSN last four, [7:51] your date of birth, your customer ID. Nothing  is needed. It's a automatic authentication. [7:57] And those voice biomarkers are also having pros  and cons. We'll discuss about that later, but [8:03] uh this really is about the road map on how  we are evolving, right? Uh uh when when we [8:08] deployed our voice biomarker markers in my  uh enterprise, right? I I lead around um [8:13] eight uh different LOBs. Uh completely regulated  again. Uh we've we've got only around 37% um [8:21] consent rates for the voice biomarkers, which  means 37% of the customers uh were okay to consent [8:28] except for these voice biomarkers. And they do not  know what they're consenting towards. Okay, I'll [8:32] I'll talk to you about that later. Uh but then uh  when they do that, it intentionally reduces your [8:40] average handle time. The authentication, which  is a big deal of a difference when you're making [8:43] a call, is now zero, right? So, but then again,  the industry right now it's moving. It's 20x the [8:51] market expansion. In January alone, uh we've got  about 1.23 billion dollars funding approved for [8:58] just the voice AI implementations. And uh these  are uh furthermore metrics. Uh we've got about 2.4 [9:04] billion dollar uh market right now if you see all  the implementations related to the voice AI, but [9:08] then uh by 2034, it is expected to span over 47.5  billion dollars, right? It's a huge industry. So, [9:17] if you're trying to invest, if you're starting  for your next gig, it could be this one, right? [9:22] But then, there are a lot of caveats, right? One  one flexibility for enterprises all these days is [9:29] especially in North America, there is no specific  global regulated uh legal and compliance act [9:34] that was released on the agentic AI. EU has  one, right? China has one, right? But then [9:41] America because of the diversification maybe,  right? Texas has a certain law because I am [9:46] from Texas, I know a lot about what's going  on. People are really getting cautious and [9:52] even they're trying to get a global recognized  entity for legal act on AI, especially on voice [10:00] AI too. We got a huge update on August 2nd about  China has an update on August 2nd. North America [10:07] released something on August 2nd. EU act is one of  the biggest movers in August 2nd that is released. [10:13] I don't know why they chose the date, but  everybody went on that date, but we got cool stuff [10:18] that was released on this regulatory compliances.  So if any enterprise, which I'm sure there are [10:24] definitely a lot of enterprises that may  not be catching up to this particular [10:28] governance principles or the sub clauses that  that might lead to another sub clause that [10:34] they could be missing out. You could  have absolutely find those, sue them, [10:39] make money out of it, right? So I also covered  those in in the next coming slides. But yeah, [10:44] so this is what I'm talking about the the ROI  we are getting out of the modernized AI contact [10:49] centers, right? But again, I'll have to stress on  this in a different angle. When we first started [10:55] implementing this voice AI in the production  level, the models were ASR driven models. It's [11:00] automatic speech to text recognition, okay?  So you have a pipeline where your voice is [11:07] converted to text and then fed to LLM, get  the response, convert back to the voice, [11:14] and then boom, right? The problem with that  is everything works good, but it has a lot of [11:18] components involved and they need to function  as expected. In an ideal scenario they do, [11:23] right? So but then let's say out of this 10-step  pipeline, one step breaks, one step may misbehave, [11:30] one step ahead, right? Everything is gone.  It's like a monolithic architecture where [11:35] you expect the ideal scenario and perfection  in the entire pipeline components. But then, [11:40] uh due to this entire pipeline, the response  time we used to get is terrible. In Voice AI, [11:46] if you are getting 800 milliseconds, that's no  to deploy that Voice AI component in production. [11:53] Imagine you ask a question, your voice bot  takes 1 second to answer, the customer hangs up. [11:59] That's an analysis of a study uh that proves  that at least more than 80% customers do that, [12:04] right? We don't have patience. We are here  for answers, and we are here only for answers, [12:08] right? So, if you're taking pauses, we'll  first uh get a doubt, suspicion, right? Whether [12:16] the bot knows it or not, but uh at the back  end, the bot is actually doing the its analysis. [12:21] Right? But then, ASR models were terrible,  honestly. But then, there's another other [12:26] upgrade we got, speech-to-speech models, uh that  is into frontier, right? So, those frontier models [12:32] are doing amazing job. The latency right  now we are um focusing or progressing is [12:36] uh amazing. It is 185 milliseconds, and then you  get the response, and it is accurate, right? So, [12:43] uh I'll just Giving the time in mind, I'll  just move across the other parts, and um [12:49] containment has been tremendously increased. Okay,  containment is a situation where you don't pass [12:55] the call to the actual human. You pass the call  you kill the call while you're interacting with [13:00] the bot. The entire life cycle will be maintained  inside the bot. But that was not the primary goal [13:04] is what we understood. Uh resolving is  the major goal compared to containment, [13:10] right? So, containment can The customer can hang  up without his issue being resolved, right? So, [13:15] that that metric has been now changed uh to The  KPI has been now changed to resolved intent, [13:20] and uh Voice AI now is becoming an operating layer  for a lot of things. So, uh the next uh beautiful [13:26] thing that I'm going to stress on is related to  the health care. Um, but yeah, you have voice AI [13:32] spread over in all these domains, right? So, now  the second part, the cool stuff about voice AI. [13:38] What it can do about the research. Uh,  for 30 seconds, if I forget about AI, uh, [13:44] voice in itself has a research. Uh, I also have  one of the paper getting published very soon. Uh, [13:50] let you know what I mean. In fact, you can,  uh, look it up in my LinkedIn, but with voice, [13:55] just the voice, not the AI, with just the voice,  we have the ability to detect a lot of diseases. [14:01] Parkinson's disease or mental depressions, right?  So, uh, it has you can you have the ability to [14:08] uh, identify at least 80 plus percent  accuracy in both males and females, right? [14:14] You have the ability to analyze all these parts  that you're seeing in the slide, right? In fact, [14:20] including COVID-19, right? But then, if you feed  this analysis to the voice AI, right? Imagine your [14:29] your customer is calling a contact center and then  he doesn't know about the depression that he has [14:35] or the Parkinson's disease he has, right? But then  the bot recognizes, notifies, warns, and then he [14:43] has a checkup done. It's being proactive and it's  it's a very beautiful thing that it doesn't even [14:50] need a consultation to identify, right? It's a  free consultation and he's already being diagnosed [14:56] with some certain percent of accuracy, right?  So, that is something uh, the act the most [15:01] advanced research that we are doing right now.  And uh, the voice AI health markers are having [15:08] the patterns to recognize just by the by detecting  around 15 seconds of voice from the customer [15:14] in generic cases, depending on the disease.  Okay, so, this is all the studies. Uh, [15:19] again, I made sure I evaluated all the numbers  and metrics. Uh these are not the ones that are [15:25] conducted by my research alone. Uh uh the  numbers that I that that I'm displaying is [15:30] uh the product of the publicly available data  data sets as well. So, it's globalized. So, [15:36] these are the accuracy uh points for the detection  of all these uh diseases or disorders. Right? So, [15:44] uh again, how are we doing this? The voice markers  will have pitch and proceeding breaths, energy, [15:50] intensity intensity, the voice quality, right?  But then uh we have our own learnings too, [15:55] right? In fact, there's something called as SSML.  We started with that with our ASR models when we [16:01] convert the voice to text. We will add the SSML  tags, speech synthesis markup language tags, [16:07] to add the emotions, expressions to make it  more sound human, right? So, likewise, the [16:12] the very same tags can be extracted nowadays using  the voice uh tokens because it's speech to speech, [16:17] we deal with the tokens, right? So, those will  have the decoded uh pipeline that will give [16:23] away these voice biomarkers with which uh  the voice AI agents are able to discover, [16:29] diagnose, and uh predict these the presence of  these abnormalities. Okay, again, this is not a [16:36] diagnosis, this is a probability of an occurrence  of certain thing that your voice AI predicts, [16:41] right? So, uh that that's one thing that we have  uh achieved uh so far. So, again, I'll have to [16:48] stress upon the governance part. Governance part  is very crucial because imagine a situation, [16:55] you're very close to your grandmother, and let's  say she passed away a few years ago. And one fine [17:03] day, you receive a call from your grandmother. She  speaks the exact same way, she knows the context, [17:09] she knows everything about you, and she asks  for credentials or whatever, right? That's a [17:16] major problem that's going on, and there are a lot  of uh cyber losses that are being registered. Uh [17:22] so that's called as digital resurrection. You can  resurrect your people. Actually, that is true, but [17:29] that's not legal. Again, the human does not come  up to the life, right? His voice will live. Even [17:36] uh people are trying to get um permissions and  approvals for um Hollywood actors and actresses, [17:43] right? To use their voices what did that the the  people that are no more with us, they wanted to [17:50] use their voices in the upcoming movies just for  their PR promotions. That's absolutely possible, [17:55] but uh the legality is not um allowing  it yet. Uh but again, uh imagine [18:02] if you are using some dead person's  voice, are you allowed to use it or not? [18:10] Are you allowed? [18:14] Who says that? The dead person didn't ever  give the consent, right? When he was alive, [18:21] if he had given the consent, yes, true.  You're true. But he did not give the consent. [18:26] So it's a it's a uh imbalance situation. Nobody  can catch you for today. I mean, before at least [18:33] the act has been finalized, nobody can catch  you, okay? So if you you're doing wrong, yes, [18:38] for sure, but let's say the dead person gave  you the consent verbally. There's no proof, [18:43] right? So by by means of those twisted  tails, uh people are actually um [18:48] getting scared and getting scammed as well, okay?  So this voice cloning and authentication bypass, [18:54] identity theft has a terrible um things going on  and this is this is the cost of each scam that [19:02] was highlighted every year, right? So and then  the digital resurrection that I spoke about, [19:08] and uh this has pushed us a lot to improvise and  strengthen and tighten the governance layer Uh [19:14] across multiple areas of the world. All right. So,  these are the acts that we see today stringently [19:21] and in fact the EU act on August 2nd it actually  intensifies a lot of things but the major thing is [19:28] if you are doing the voice AI implementation you  have to let the customer know. You'll have to add [19:33] the watermarks for the voice AI and you have to  let the customer know that this is generated by [19:37] the voice bot otherwise you'll be penalized very  heavily. Right. So, we are getting there but then [19:45] I also have like I said there are a lot of slides.  So, these are all the identified published legal [19:53] compliance rules right now we have in  place to set the controls by. Okay. So, [20:00] but then I also have added my own experiences at  the enterprise level what we have what we face and [20:06] how the AI component is only just a component  in the system. It is not treated as a system [20:11] because I've seen people because I advise a  lot of enterprises. I've seen some startups [20:15] approaching me saying that hey I have a voice  bot. This is my company. No. Voice bot or the [20:21] AI is not a system. It's just a component in the  system. I totally emphasize on that and I have [20:26] the blueprints that I've written in my book.  Feel free to scan it over, get in touch with [20:31] me for anything you wish. But yeah, this is what  I would like to cover today. Thank you so much.