## Background
Published by Ai4 on YouTube on August 27, 2025, this presentation features Arijit Sengupta, Founder and CEO of Aible, and Deepu Kumar, Vice President of Engineering at CVS Health. The session explores how CVS Health empowers non-technical business teams to rapidly build, test, and securely scale AI agents from concept to production in enterprise healthcare environments.
## TL;DR
The speakers present a pragmatic framework for enterprise AI adoption centered on rapid prototyping, stakeholder empowerment, and explainable trust rather than waiting for clean data or perfect specifications. CVS Health enabled over 450 non-technical builders to experiment across 750 sandbox projects analyzing billions of rows of data before filtering concepts through rigorous production gates. To eliminate hallucinations, the platform pairs generative language models with deterministic, auditable code execution under a 'Blue is True' interface. Throughout the implementation, AI is treated like an intern requiring granular qualitative feedback, running secure in-database compute on systems like Snowflake and Google BigQuery to augment frontline workers rather than replace them.
## Atmosphere & style
The presentation is candid, fast-paced, and refreshingly pragmatic, cutting through AI hype with tangible metrics and real-world implementation stories. Arijit brings high-energy entrepreneurial candor and lighthearted humor, while Deepu offers a grounded, reassuring enterprise engineering perspective. The dynamic feels like an open masterclass between close collaborative partners, creating an empowering and demystifying vibe focused on organizational humility, psychological safety, and human-centric innovation.
## Key takeaways
- [00:00](https://www.youtube.com/watch?v=luyznCueMM8&t=0s) Enterprise AI initiatives frequently fail due to the persistent disconnect between data scientists and business users, which can be resolved through rapid prototyping, stakeholder empowerment, and transparent trust.
- [01:36](https://www.youtube.com/watch?v=luyznCueMM8&t=96s) Waiting for perfect enterprise datasets stalls progress; development must prioritize rapid prototyping and iterative data reduction to surface actionable business value within days.
- [01:36](https://www.youtube.com/watch?v=luyznCueMM8&t=96s) Scaling AI responsibly across a large enterprise requires standardized architectural foundations and reusable governance frameworks so teams avoid reinventing core components.
- [04:26](https://www.youtube.com/watch?v=luyznCueMM8&t=266s) Organizations should avoid waiting for complete agent specifications upfront, instead granting teams 'many shots on target' in safe sandboxes because business users rarely know the ideal agent design in advance.
- [04:26](https://www.youtube.com/watch?v=luyznCueMM8&t=266s) CVS scaled internal adoption by enabling over 450 non-technical builders to launch 750 projects, exploring 300 million variable combinations across 100 billion rows of data in just two months.
- [04:26](https://www.youtube.com/watch?v=luyznCueMM8&t=266s) A fail-fast sandbox culture powered by realistic mock data must be balanced by a rigorous, zero-risk production validation gate involving IT, business leaders, and executive sponsors.
- [08:01](https://www.youtube.com/watch?v=luyznCueMM8&t=481s) Building employee trust in autonomous systems requires total algorithmic transparency, clear audit trails, and detailed qualitative feedback loops rather than superficial thumbs-up or thumbs-down ratings.
- [08:01](https://www.youtube.com/watch?v=luyznCueMM8&t=481s) Generative AI hallucinations can be overcome by coupling LLMs with deterministic secondary verification engines that link outputs directly to underlying enterprise source data and verified code.
- [08:01](https://www.youtube.com/watch?v=luyznCueMM8&t=481s) AI agents must be trained like human interns rather than pets, using specific business rules, metric definitions, and actionable corrections to continuously refine their performance.
- [12:03](https://www.youtube.com/watch?v=luyznCueMM8&t=723s) Pushing agent compute directly down into enterprise data stores like Snowflake and BigQuery enables millions of automated queries securely without transferring raw data outside client perimeters.
- [12:03](https://www.youtube.com/watch?v=luyznCueMM8&t=723s) Positioning AI strictly as human augmentation creates the psychological safety needed for frontline domain experts to actively champion and design high-value automation use cases.
- [16:15](https://www.youtube.com/watch?v=luyznCueMM8&t=975s) Prototyping environments should be accessible enterprise-wide via Single Sign-On to unite business operators and technical engineers, reserving strict compliance gates exclusively for production releases.
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=luyznCueMM8&t=0s) The Reality of AI Failures and the Three Pillars of Success
Arijit Sengupta, Founder and CEO of Aible, opens the session by reflecting on his background founding BeyondCore (which became Salesforce Einstein Discovery) and authoring the book *AI is a Waste of Money*. Drawing from over a thousand AI projects, he emphasizes that the fundamental reason most enterprise AI initiatives fail remains the chronic disconnect between data scientists and business users. To ground the session in reality rather than "vaporware," he invites Deepu Kumar, Vice President of Pharmacy Engineering at CVS Health, to co-present real-world execution strategies. Arijit jokes about Deepu's extensive background at Disney and Baptist Health, playfully pushing him to elaborate beyond his humble one-line introduction before outlining the core framework of the talk.
To successfully scale enterprise AI, organizations must anchor their strategy on three foundational pillars:
- **Rapid Prototyping & Iteration**: Moving fast without waiting for perfect data.
- **Stakeholder Empowerment**: Enabling business teams across the enterprise to build and experiment.
- **Building Trust**: Establishing transparent reasoning and auditability between human users and AI.
```mermaid
flowchart TD
accTitle: Enterprise AI Scaling Methodology
A["Iterative Prototyping"] --> D["Production-Ready AI Agents"]
B["Stakeholder Empowerment"] --> D
C["Explainable Trust & Human Feedback"] --> D
```
---
[01:36](https://www.youtube.com/watch?v=luyznCueMM8&t=96s) Rapid Prototyping Over Data Perfection
Arijit shares an anonymized case study illustrating the velocity required in modern enterprise AI development. In the first two days of a project, the team executed 20 rapid iterations. They started by analyzing 4.8 billion rows of data evaluated across 10 million rows, then adjusted down to 1.8 billion rows evaluated at 550,000 rows. Arijit points out that reducing dataset size is often necessary because waiting for a "perfect" dataset in a massive enterprise like CVS is impossible. By adopting this aggressive prototyping cadence, the initiative reached direct business user review within days and was reviewed by the Chief Digital Officer, Tony Ambrosie, by day eight.
Deepu emphasizes that scaling AI responsibly at enterprise caliber requires establishing robust architectural foundations and reusable governance frameworks so developers do not have to reinvent the wheel. He underlines the necessity of a "prototype, pilot, production" mindset, explaining that hands-on prototyping frequently uncovers unanticipated insights that deliver incremental business value. Arijit recalls an engineer named Yogesh who adjusted data variables in real time as new patterns emerged. This willingness to adapt requires organizational humility—entering projects ready to learn from the data rather than assuming all answers are known upfront.
---
[04:26](https://www.youtube.com/watch?v=luyznCueMM8&t=266s) Permission to Fail: Scaling From Hundreds of Prototypes to Production
Addressing common industry pitfalls, Arijit criticizes the tendency of organizations to wait for a "perfectly defined agent spec" from business stakeholders before writing code. Since business users rarely know what an ideal agent looks like in advance, the goal must be giving teams "many shots on target." In the first two months of CVS's initiative, 450 non-technical users built (rather than merely consumed) agents across 750 projects, exploring over 300 million variable combinations across 100 billion rows of data.
```mermaid
flowchart LR
accTitle: Prototyping Sandbox vs Production Pipeline
subgraph Sandbox ["High-Velocity Sandbox Innovation"]
A["450+ Business Builders"] --> B["750+ Pilot Projects"]
B --> C["Rapid Failure & Iteration"]
end
subgraph Rigor ["Rigorous Enterprise Gate"]
C --> D{"IT, Business & CDO Validation"}
end
subgraph Deploy ["Production Deployment"]
D --> E["Audited Production Agents"]
end
```
Deepu explains that while production solutions require tight cross-functional collaboration between Subject Matter Experts (SMEs) and technical engineering teams, the upstream experimentation must happen freely in controlled sandbox environments using realistic mock data. Senior leadership buy-in is vital; Chief Digital Officer Tony Ambrosie champions this by encouraging teams to try ideas and demonstrate value rather than gatekeeping early exploration.
- **Sandbox Experimentation Metrics**: Over 450 active builders, 750 exploratory projects, 300M+ variable combinations, and 100B rows analyzed.
- **Fail-Fast Philosophy**: Failing fast in sandboxes generates the organizational learning necessary to filter down to high-impact production candidates.
- **Zero-Risk Production Gate**: Strict, multi-stakeholder validation (IT, Business, and Executive leadership) ensures high reliability before deployment to patient- or consumer-facing environments.
---
[08:01](https://www.youtube.com/watch?v=luyznCueMM8&t=481s) Transparent AI and Training Agents Like Interns
Deepu breaks down CVS's framework for fostering trust between human workers and autonomous AI systems, emphasizing that AI must serve employee convenience rather than being mandated from above. He details three operational rules:
1. **Algorithmic Transparency & Audit Trails**: Every decision must expose the underlying rationale, data inputs, and algorithms used.
2. **Granular Feedback Loops**: Users provide rich qualitative feedback rather than simplistic binary thumbs-up/down ratings.
3. **Augmentation Over Replacement**: AI assists human judgment, leaving final sign-off and accountability strictly with employees.
Arijit details Aible's interface implementation called "Blue is True" to address generative AI hallucinations. While generative models inherently produce hallucinations, Aible pairs them with a deterministic secondary verification engine. When a user hovers over blue highlighted text generated by the LLM, an interactive deterministic chart generated by verified Python code appears on the right panel, linked directly to the enterprise source data.
```mermaid
flowchart LR
accTitle: Blue is True Verification Architecture
A["Generative Model (LLM Output)"] --> B["Blue Highlighted Text"]
B --> C["Secondary Verification Engine"]
C --> D["Deterministic Python Code"]
D --> E["Audited Source Data & Interactive Visualizations"]
```
Arijit compares this workflow to "training an AI like an intern, not a pet." Using a non-healthcare lost-baggage vision model as an example, he illustrates that when an agent misidentifies lost luggage because people are in the background, telling the model *"the people are too far away"* or defining specific business metrics (such as gross margin calculations) trains the system effectively, whereas a simple "thumbs down" provides no actionable corrective signal.
---
[12:03](https://www.youtube.com/watch?v=luyznCueMM8&t=723s) In-Database Processing and Giving Workers Superpowers
After highlighting public thought leadership resources from CVS leaders Tony Ambrosie and Tilak Mandadi (including collaborations with NVIDIA), Arijit outlines the architectural underpinnings of Aible's platform. Aible runs specialized agents directly on top of enterprise data stores such as Snowflake and Google BigQuery without moving raw data outside the client's secure boundary. The system leverages in-database compute pushed down to the source systems, enabling agents to evaluate millions of automated variable queries in the time it takes a human analyst to run seven manual queries—incurring only the cloud compute duration costs of a standard query session.
During the concluding Q&A, an audience member asks whether keeping AI in an augmentative, non-replacement role is a temporary stepping stone or a permanent future stance. Deepu and Arijit reiterate that CVS and Aible are firmly committed to augmentation. Arijit argues that treating AI as a human replacement is a flawed mental model; the real economic leverage comes from granting frontline workers "superpowers" to solve complex operational challenges. By guaranteeing job security and establishing psychological safety, organizations empower frontline employees—who possess the deepest domain expertise—to propose high-value use cases that technical teams would never anticipate. Deepu concludes that AI's greatest mission is stripping away mundane, repetitive tasks so human teams can focus on high-value, meaningful work.
[16:15](https://www.youtube.com/watch?v=luyznCueMM8&t=975s) Cross-Functional Collaboration and Ungated Innovation
The ultimate goal of deploying these agentic systems in healthcare is human-centered: freeing clinicians from getting bogged down in software interfaces and troubleshooting technical issues so they can spend dedicated time caring for patients. In a final audience question regarding user demographics—specifically whether the platform is intended exclusively for non-technical business users—the speakers clarify that organizational models vary across enterprises. While some deployments, such as the State of Nebraska where the CIO empowered teams of interns, rely primarily on business users, CVS Health deliberately embraces a collaborative, all-hands approach where "everybody at the party" is invited to participate.
At CVS Health, development operates as a deeply cross-functional effort between business and engineering teams. Technical team members who have spent years maintaining core systems often possess an intimate understanding of business workflows that matches or exceeds that of day-to-day operations staff. By bringing tech and business teams together on the platform, both sides work in continuous alignment toward practical outcomes.
To foster this environment, access to the ideation and rapid prototyping environment is intentionally left open to all internal employees through Single Sign-On (SSO) rather than segregated by job title or technical background. The operational philosophy hinges on keeping early-stage experimentation completely open and applying strict guardrails only when taking a concept to the finish line:
- **Open Innovation Layer:** Anyone with enterprise SSO credentials can enter the platform and immediately begin building and testing prototypes without pre-approval gates restricting early ideas.
- **Production Guardrails:** Rigorous enterprise gating, validation, and security compliance are applied downstream during the transition from prototype to full production deployment rather than during initial ideation.
## Video description
Arijit Sengupta, Founder and CEO, Aible
Deepu Kumar, Vice President of Engineering, Pharmacy and Consumer Wellness, CVS Health
Abstracts
Join this session with Chief Digital & Technology Officer of CVS Health on how the retail leader empowers non-technical business teams to build and deploy AI agents within days to solve real-world business challenges.
Learn how CVS is transforming enterprise AI development through rapid prototyping and human-centered design. Discover their proven methodology for moving from concept to production in just 30 days, why they treat AI agents like interns who need training and feedback, and how reasoning models are creating more reliable and trustworthy AI systems. They also explore the future of autonomous agents, the importance of responsible AI in healthcare, and why the next wave of AI will focus on empowering humans rather than replacing them.
AI Agent 落地实践：CVS Health 如何在业务团队安全规模化扩展智能体
演讲者提出了一个务实的企业 AI 落地框架，其核心在于快速原型设计、赋能利益相关者与建立可解释的信任，而非等待清洗好的数据或完美的规范说明。CVS Health 支持了 450 多名非技术构建者在 750 个沙盒项目中进行实验并分析数十亿行数据，随后再通过严格的投产门限筛选这些概念构想。为消除幻觉，该平台在“Blue is True”界面下将生成式语言模型与确定性、可审计的代码执行相结合。在整个实施过程中，AI 被视作需要精细定性反馈的实习生，通过在 Snowflake 和 Google BigQuery 等系统上运行安全的数据库内计算来增强一线员工的能力，而非取代他们。
CVS Health 摒弃对“完美数据集”的前期等待，在短短两个月内赋能超过 450 名非技术业务人员创建了 750 个 Agent 项目，并在 1000 亿行数据上探索了超 3 亿种变量组合。 (4:26)
为消除大模型幻觉，平台通过“Blue is True”架构将 LLM 输出与确定性二次验证引擎配对，用户悬停高亮内容即可查看由已验证 Python 代码生成的交互图表与底层源数据。 (8:01)
Aible 架构将 Agent 计算直接下推至 Snowflake 和 Google BigQuery 等企业数据仓库内执行，在保障数据安全不出域的同时，以辅助一线员工为核心而非取代人力。 (12:03)
企业推行面向全体员工开放的 SSO 原型创新沙盒，通过宽容前期的快速失败来沉淀经验，并在最终面向生产环境部署前设立严密的跨部门零风险验证关卡。 (16:15)
Reading summary
Aible CEO Arijit Sengupta and CVS Health Vice President Deepu Kumar argue that enterprise AI succeeds by prioritizing rapid prototyping and employee empowerment in open sandboxes, underpinned by in-database compute and deterministic auditability rather than the pursuit of perfect datasets or worker replacement.
Velocity Over Data Perfection (1:36)
Sengupta observes that most enterprise AI initiatives fail due to a chronic disconnect between data scientists and business users, exacerbated by prolonged efforts to curate flawless datasets. Instead of waiting for data perfection, CVS Health and Aible advocate rapid iteration across representative data subsets. In one referenced project, the team completed 20 iterations in two days, scaling their evaluation down from 4.8 billion rows (sampled at 10 million) to 1.8 billion rows (sampled at 550,000). This pace allowed frontline business users to evaluate prototypes within days and enabled a review by Chief Digital Officer Tony Ambrosie by day eight.
Kumar emphasizes that practical prototyping consistently surfaces unexpected operational insights. To support this approach, organizations must adopt a 'prototype, pilot, production' methodology supported by reusable governance architectures, encouraging technical teams to adjust variables dynamically based on emerging data rather than presuming fixed specifications in advance.
Decentralized Innovation and Open Sandboxes (4:26)
Rather than demanding rigid, upfront agent specifications from business stakeholders who may not know what an ideal agent requires, CVS Health created an open experimentation environment. Through enterprise Single Sign-On (SSO), non-technical employees across departments were granted direct access to build and test prototypes using realistic mock data without early administrative gatekeeping. Within the first two months, 450 non-technical builders launched 750 exploratory projects, testing over 300 million variable combinations across 100 billion rows of data.
This workflow establishes a deliberate split between exploratory innovation and deployment. While early experimentation is intentionally ungated to encourage rapid failure and organizational learning, rigorous enterprise gating—involving cross-functional reviews by IT, business leaders, and executive leadership—is strictly applied before any agent transitions into production.
Deterministic Verification and Algorithmic Trust (8:01)
To build workforce confidence in automated tools, Kumar outlines three core operational criteria: algorithmic transparency, granular qualitative feedback, and final human sign-off. The speakers argue that binary thumbs-up or thumbs-down feedback fails to train models effectively, advocating instead for qualitative, contextual coaching similar to instructing an intern on domain-specific rules.
To counter generative AI hallucinations, Aible developed the 'Blue is True' interface architecture. When an LLM generates blue-highlighted text, users can hover over it to reveal an interactive chart powered by secondary, deterministic Python code connected directly to audited enterprise source data. This dual-layer approach pairs generative capabilities with verifiable mathematical proof.
In-Database Execution and Worker Augmentation (12:03)
To maintain security and manage costs, Aible executes agent operations directly within enterprise data repositories such as Snowflake and Google BigQuery. By pushing compute down to the source database, raw enterprise data never moves outside the organization's perimeter, allowing agents to evaluate millions of automated variable queries within standard query execution windows and standard compute budgets.
Addressing workforce concerns, both Sengupta and Kumar assert that AI must permanently function as an augmentative tool rather than a labor replacement. Sengupta contends that guaranteeing job security creates the psychological safety frontline workers require to identify high-impact operational use cases, while Kumar notes that automating repetitive administrative software interactions ultimately redirects staff and clinical focus toward patient care.
CVS Health and Aible demonstrate that broad enterprise AI adoption relies on democratizing prototyping access while enforcing strict deterministic verification and in-database governance; however, the model's ongoing success hinges on maintaining zero-risk production gates and clear human-in-the-loop accountability.
## Background
该视频由 Ai4 频道于 2025 年 8 月 27 日发布在 YouTube 平台。内容聚焦于 Aible 创始人兼 CEO Arijit Sengupta 与 CVS Health 药房和消费者健康工程副总裁 Deepu Kumar 的炉边对谈，重点探讨 CVS Health 如何赋能非技术业务人员快速安全地构建和部署企业级 AI Agent，并在医疗零售场景中实现规模化落地。
## TL;DR
对谈围绕 CVS Health 成功规模化落地 AI 的三大支柱——快速原型构建、全员赋能以及人机信任机制展开。双方强调了在沙箱环境中进行高频试错与变量探查的重要性，展示了 Aible 存算分离与原位查询下推的技术架构，并提出了“像培训实习生一样训练 AI”的双向反馈与可解释推理理念。核心观点明确指出企业智能化转型的关键在于“赋予现有业务骨干超能力”而非裁员替代，通过前端零门槛探索与后端严密生产准入的结合，释放一线业务的真实创新潜力。
## Atmosphere & style
整场对话务实、严谨且充满落地锐气，彻底摒弃了空洞的行业炒作，直击企业 AI 落地中技术与业务脱节的核心痛点。两位讲者配合默契，既有技术架构层面的冷静拆解，又有管理心智与组织文化上的深刻洞察。现场氛围兼具启发性与信服力，让听众能真切感受到前沿 AI 技术与复杂医疗零售巨头碰撞出的敏捷创新活力。
## Key takeaways
- [00:00](https://www.youtube.com/watch?v=luyznCueMM8&t=0s) AI 规模化落地的三大核心支柱是快速原型设计与高频迭代、全面赋能业务人员直接参与构建，以及建立透明可信的人机信任机制。
- [01:36](https://www.youtube.com/watch?v=luyznCueMM8&t=96s) 企业推进 AI 时无需等待“完美数据集”，应践行敏捷思维，通过极短周期内的高频模型迭代快速验证并呈报高层审查。
- [04:26](https://www.youtube.com/watch?v=luyznCueMM8&t=266s) 不存在所谓的“完美 Agent 规格说明书”，企业应通过“增加射门次数”的沙箱实验策略，赋能数百名非技术人员尝试海量原型探索。
- [04:26](https://www.youtube.com/watch?v=luyznCueMM8&t=266s) 高层应建立“允许快速试错”的容错文化，形成“沙箱狂热实验、中间价值验证、生产严苛准入”的创新漏斗机制。
- [07:53](https://www.youtube.com/watch?v=luyznCueMM8&t=473s) 建立人机信任需恪守三大原则：算法与推导逻辑具备可解释性、提供结构化深度反馈机制，以及坚守“增强而非替代人类”的定位。
- [07:53](https://www.youtube.com/watch?v=luyznCueMM8&t=473s) 大模型幻觉无法彻底根除，必须在架构层引入独立的自动化二次验证与底层源数据回溯机制，筑牢可信防线。
- [10:10](https://www.youtube.com/watch?v=luyznCueMM8&t=610s) 应像培训实习生一样训练 AI，在 AI 出错时提供具体口径与推导步骤的结构化指引，而非仅仅给出模糊的单向差评。
- [10:10](https://www.youtube.com/watch?v=luyznCueMM8&t=610s) 采用确定性计算与生成式分析并行的界面分工架构，让自然语言洞察与可审计的实际代码图表形成互证。
- [12:45](https://www.youtube.com/watch?v=luyznCueMM8&t=765s) 坚持“数据绝不搬家”的架构原则，将百万级并发探查直接下推至企业现有数仓（如 Snowflake/BigQuery）执行，以极低成本撬动海量变量归因分析。
- [14:27](https://www.youtube.com/watch?v=luyznCueMM8&t=867s) 管理心智应从“用 AI 裁减员工”彻底转变为“赋予业务骨干超能力”，释放机械杂务，将精力聚焦于高价值业务与医患关怀。
- [14:27](https://www.youtube.com/watch?v=luyznCueMM8&t=867s) 为一线员工建立心理安全感是激发隐性知识的关键，只有消除被替代的恐慌，基层才会主动挖掘外部技术专家无法察觉的高价值痛点用例。
- [16:27](https://www.youtube.com/watch?v=luyznCueMM8&t=987s) 推行技术工程师与一线业务人员紧密结对的跨职能协同模式，兼顾底层系统逻辑理解与实际业务场景诉求。
- [16:27](https://www.youtube.com/watch?v=luyznCueMM8&t=987s) 实行创新与生产的门禁分离，前端通过企业 SSO 零门槛开放沙箱实验以释放全员创意，后端在正式上线生产时实施严密治理。
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=luyznCueMM8&t=0s) 破除“AI 浪费钱”的魔咒与成功的三大支柱
Arijit Sengupta 开场回顾了自己创办 BeyondCore（后来发展为 Salesforce Einstein Discovery）的经历，并提到自己基于过往主持的 1000 多个 AI 项目撰写了《AI is a waste of money》（AI 是在浪费钱）一书。他强调自己至今依然为这本书感到自豪，因为其中的核心观点在当下依然成立：绝大多数 AI 项目之所以折戟沉沙，根本原因在于数据科学家与业务用户之间的严重脱节。为了直面这一痛点，本场演讲摒弃空洞的概念炒作与“虚假宣传（vaporware）”，坚持“先讲做了什么，再讨论为何这么做”的务实原则。
随后，Arijit 邀请了拥有丰富履历的老朋友——现任 CVS Health 药房与消费者健康工程副总裁 Deepu Kumar（曾任职于迪士尼与浸信会健康中心）同台。Arijit 概述了企业成功规模化落地 AI 的三大核心支柱：
1. **快速原型设计与高频迭代（Rapid Prototypes & Quick Iteration）**：以极快的速度验证想法；
2. **全面赋能业务利益相关方（Empowering Stakeholders）**：让业务人员亲自参与构建；
3. **建立用户与 AI 之间的信任机制（Building Trust）**：确保 AI 的透明、可控与辅助定位。
---
### [01:36](https://www.youtube.com/watch?v=luyznCueMM8&t=96s) 快速原型构建：不必等待“完美数据”，践行“原型-试点-生产”思维
在探讨快速原型构建时，Arijit 分享了一个脱敏项目的具体运作机制：团队在项目开展的前两天内就完成了整整 20 次迭代。初始阶段涉及的数据集规模高达 48 亿行，并在 1000 万行数据上进行评估；到了第二阶段，数据集调整为 18 亿行，评估范围精简至 55 万行。Arijit 指出，数据量变小并非倒退，而是因为在像 CVS 这样庞大的真实企业环境中，根本不可能一开始就拿到“完美无瑕的数据集”。关键在于能否在数天内让业务用户完成审查，并在第 8 天就直接呈报给首席数字化官（Chief Digital Officer, CDO）进行高管评审。
Deepu Kumar 针对 CVS 如何在大规模组织中推广 AI 进行了深入阐述。他指出，负责任的 AI 应用必须建立在坚实的基础设施之上：首先需要抽象出治理框架与跨领域底层能力，避免业务人员“重复造轮子”；其次是业务专家与技术团队从第一天起就紧密跨界协作；最后也是最关键的，是必须具备“测试、迭代与改进”的敏捷思维，绝不能等待所有条件完全成熟或数据彻底完美才启动。
Arijit 补充了 CVS 团队成员 Yogesh 的生动案例：在实时迭代中，Yogesh 观察到某个数据模式后立刻提出“我知道为什么会这样了，让我们加进这个变量试一下”，随后整个团队当场重新切分数据并运行新版本。这种不预设自己全知全能、保持谦逊并随时根据反馈快速调整的心态，正是原型设计成功的精髓所在。
- **前 2 天迭代强度**：完成 20 次快速模型与数据迭代。
- **数据演进路径**：从 48 亿行（评估 1000 万行）快速收敛至 18 亿行（评估 55 万行）。
- **高管接入速度**：第 8 天即直接进入首席数字化官的决策审查视野。
---
### [04:26](https://www.youtube.com/watch?v=luyznCueMM8&t=266s) 广撒网式实验：在沙箱中快速试错与高层“允许失败”的文化
针对当前市场上许多企业试图寻找“完美 Agent 规格说明书（Agent Spec）”的误区，Arijit 指出这完全是不切实际的幻想，因为最终用户在亲眼看到和使用之前，根本不知道完美的 Agent 应该长什么样。因此，正确的策略是“增加射门次数（many shots at target）”。在 CVS 项目启动的前两个月里，平台赋能了 450 名业务用户亲自构建（而非仅仅被动消费）Agent，累计催生了 750 个项目，探索了超过 3 亿种变量组合，处理了 1000 亿行数据。
```mermaid
flowchart LR
accTitle: Sandbox Prototyping to Production Pipeline
accDescription: Flowchart showing how vast sandbox experimentation is filtered through rigorous IT and business validation before reaching production.
subgraph Sandbox["沙箱实验环境（宽进 / 允许快速失败）"]
A["450+ 业务构建者"] --> B["750+ 实验性 Agent 原型"]
B --> C["探索 3 亿+ 变量组合 / 1000 亿行数据"]
end
subgraph Validation["严密验证关卡"]
C --> D{"业务价值与 IT 安全审查<br/>(首席数字化官把关)"}
end
subgraph Production["生产部署（严出 / 零容错）"]
D -->|高价值验证通过| E["生产级受控 Agent 应用"]
D -->|价值不足 / 风险未决| F["安全淘汰 / 经验沉淀"]
end
```
Deepu 强调，虽然生产环境必须严谨对待（尤其涉及患者健康与业务运营时绝不允许灾难性失败），但在前端必须为团队提供配备模拟真实场景数据的沙箱环境，让业务专家（SME）与技术团队自由碰撞想法。这种敏捷创新的核心在于高层领导的宽容支持。Arijit 赞赏了 CVS 首席数字化官 Tony Ambrosie 的领导风格：面对团队成员提出的各种 Agent 构想，Tony 从不说“不”，而是鼓励“去试吧，只要你能证明它创造了足够的业务价值，我就给资金支持”。这种“在沙箱中狂热创新、在进入生产时极度严苛”的漏斗机制，消除了员工对失败的恐惧，使 30 分钟到 48 小时内快速产出原型成为常态。
- **2 个月规模化指标**：450 名实际构建者，750 个 Agent 项目，3 亿+ 变量组合，1000 亿行处理数据。
- **创新漏斗法则**：左端沙箱大胆试错，中间严密验证，右端生产极度严格。
---
### [07:53](https://www.youtube.com/watch?v=luyznCueMM8&t=473s) 建立人机信任：透明度、深度反馈与“赋能而非替代”
在谈到人机信任时，Deepu 明确表示 CVS 绝不会采取强推工具、命令员工“你必须用”的做法，AI 的引入必须是为了方便员工工作，而建立信任的前提在于决策过程的清晰透明。Deepu 总结了 CVS 遵循的三大核心原则：
1. **算法透明度与可解释性（Transparency & Explainability）**：AI 必须能够清晰解释其决策逻辑，包括使用了哪些数据、应用了何种算法模型、背后的推导依据是什么，并为员工提供完整的审计追踪记录（Audit Trail）。
2. **深度双向反馈机制（Sophisticated Feedback Loops）**：员工不仅能给出简单的“是/否”或点赞点踩，还能提供结构化、深度的专业反馈，从而持续训练并优化 Agent 下一次的表现。
3. **增强而非替代（Augmentation vs. Replacement）**：AI 的核心定位是增强人类员工的能力，绝非取而代之；最终的业务决策、审查与专业判断权始终牢牢掌握在员工手中。
Arijit 对此深表赞同，并呼应了 Aible 公司的口号“I am able”——其本质就是赋能人类。针对业界普遍面临的生成式 AI 幻觉问题，Arijit 直言“如果你认为自己能彻底消除生成式 AI 的幻觉，说明你还没真正理解这项技术”。Aible 在架构上设计了名为“Blue is True”的独立验证机制：由第二个独立系统对大模型的输出内容进行全自动二次校验，并独立回溯至底层源数据进行核实，以此从机制上筑牢安全与信任的防线。
### [10:10](https://www.youtube.com/watch?v=luyznCueMM8&t=610s) 把 AI 当作实习生培养：可解释的推理步骤与双向信任
在企业级 AI 系统的落地过程中，透明度与可审计性是建立用户信任的核心基石。系统的操作痕迹会完整保存在 CVS 的安全审计日志中，供任何人独立审查。在产品界面设计上，团队采用了严密的“左脑与右脑”分工架构：用户将鼠标悬停在任何蓝色高亮文本上时，系统都会即时调出对应的图表。界面右侧展示的所有图表均属于“确定性”（Deterministic）计算，底层完全由实际运行的 Python 代码支撑，用户可直接追溯数据的确切来源；而界面左侧则呈现大模型的“生成式”（Generative）内容。这种架构直观地兑现了系统对每一步推理逻辑的严谨拆解。
```mermaid
flowchart TD
accTitle: AI推理与人类反馈的双向信任循环
subgraph System["透明化系统架构"]
A["生成式分析<br/>（界面左侧呈现）"]
B["确定性代码与图表<br/>（界面右侧 Python 支撑）"]
end
subgraph FeedbackLoop["实习生式反馈闭环"]
C["AI 暴露具体推理步骤"] --> D{"人类专家介入校验"}
D -- "具体业务指引<br/>（如修正毛利定义/距离判定）" --> E["AI 调整逻辑并沉淀经验"]
D -. "简单的差评 (Thumbs Down)<br/>（无法指明根因）".-> F["低效/无效迭代"]
end
System --> C
E --> G["双向信任桥梁建立"]
```
针对 AI 的纠错与调优，团队提出了一个极具启发性的核心理念：“把 AI 当作实习生来培训，而不是当作宠物”（"training an AI like an intern not a pet"）。当 AI 在某些复杂场景中判断失误时，人类专家不应只是粗暴地给一个差评图标（Thumbs down），而必须针对其具体的推理步骤提供结构化反馈。现场分享了一个中立的“行李丢失识别”案例：AI 看到画面背景中有人，便草率判断行李并未丢失；此时用户给出的反馈并非简单否决，而是明确指出“背景中的人距离行李太远，不具备从属关系”。同样在商业分析中，若 AI 算错了指标，人类专家应指导它“这是我们公司内部对毛利率（Gross Margin）的统计口径”，而非仅仅给一个差评，因为简单的差评可能代表很多含义——可能是废话太多、口径错误，也可能是调错了数据集。通过这种推理步骤的互动，系统在人机之间架起了一座“双向信任的桥梁”：人类因 AI 能自证逻辑而产生信赖，而 AI 也获得了人类明晰的指导（"Hey, you got this wrong. This is why you got it wrong. Don't do it again."），从而实现持续进化。
- **确定性与生成式的双屏协同**：右侧 Python 代码支撑的可视化图表负责严谨的数据归因，左侧生成式内容负责自然语言解释与洞察提炼。
- **结构化反馈胜过单向差评**：向 AI 明确指出业务术语口径或数据源误区，避免传统 thumbs down 带来的模糊歧义。
- **行业标杆学习资源**：现场鼓励参会者扫描屏幕二维码，参考 CVS 与英伟达（NVIDIA）联合开展的 AI 探索，以及技术高管 Tony 和 Tilak 在播客与专栏中系统阐述的 CVS AI 落地方法论。
---
### [12:45](https://www.youtube.com/watch?v=luyznCueMM8&t=765s) Aible 架构揭秘：数据不动、下推查询与百万级并发提问
Aible 平台的定位在于依托企业自身的数据资产、业务流程与特定职能，敏捷构建专业化的 AI Agent。由于各业务部门对 Agent 复杂度的诉求千差万别（涵盖从极简的工作流辅助到极复杂的多步骤决策），平台在底层采用了统一的共享服务架构：一旦涉及结构化数据分析，底层会自动激活对应的结构化计算服务；若涉及非结构化文本，则无缝切换并唤醒相关组件。
最关键的架构原则在于“数据绝不搬家”——无论数据原本存储在 Snowflake 还是 Google BigQuery 中，数据始终留在原地。平台在处理包含数十亿行数据、数百万种变量组合的极大规模分析时，会将所有复杂的计算查询直接“下推”（Pushdown）到企业现有的底层数据仓库中执行。这种机制创造了惊人的效率红利：得益于数仓的弹性并发与按执行时长计费的模式，AI 能够在人类提出约 7 个问题的时间窗口内，高并发向底层数仓发起多达 100 万次变量探查与计算，而企业最终支付的云端算力账单仅相当于人类提出 7 个问题的系统运行成本。
```mermaid
flowchart LR
accTitle: Aible 存算分离与查询下推架构
subgraph EnterpriseData["企业自有数据层（数据不离开原地）"]
SF["Snowflake"]
BQ["Google BigQuery"]
end
subgraph AibleEngine["Aible 统一智能引擎"]
Agent["专业化 AI Agent<br/>（极简到极复杂）"]
Pushdown["高并发查询下推机制<br/>（百万次并行探查）"]
end
subgraph BusinessInsights["业务洞察输出"]
Change["'发生了什么变化' 归因分析"]
HumanQA["人类可理解的业务问答"]
end
Agent --> Pushdown
Pushdown <--> EnterpriseData
Pushdown --> Change
Change --> HumanQA
```
这种毫秒级、大规模变量扫描能力被直接应用于“发生了什么变化”（What's changed）的归因分析中。在零售快消、供应链物流等外部环境瞬息万变的业务场景下，系统能够自动穿透海量变量交叉组合，迅速锁定数据异动的深层原因与潜在模式，并将底层的复杂波动转化为业务人员一目了然的解释与建议。
- **零数据迁移安全性**：完全基于 Snowflake / BigQuery 原生环境进行计算下推，保障敏感数据合规与治理边界。
- **算力杠杆与成本套利**：借助数仓按运行时长计费的特性，以 7 次人类提问的时间与费用，撬动 100 万次并发假设验证。
- **归因分析赋能**：在复杂变量交叉中秒级捕捉零售与供应链的异动根因，并翻译成可落地的行动建议。
---
### [14:27](https://www.youtube.com/watch?v=luyznCueMM8&t=867s) 赋能而非替代：为 50 名业务骨干赋予“超能力”
在现场问答环节，针对观众提出的核心疑虑——“AI 不会替代人类是当前的技术妥协还是对未来的长远判断”，两位讲者给出了坚定且一致的回应：AI 的核心使命绝非减员裁员，而是职能重塑与能力跃迁（"The human will get superpowers"）。企业的管理心智必须彻底转变：不要去琢磨“如何用 AI 裁掉 50 名员工”，而要想方设法“为现有的 50 位业务骨干赋予超能力”，从而让企业有能力开拓原先无法涉足的新业务，创造远超以往的商业价值与降本成效。
```mermaid
flowchart TD
accTitle: 赋能模式与裁员模式的对比
subgraph Replacement["错误心智：替代裁员"]
R1["目标：裁撤 50 名员工"] --> R2["员工产生防御抵触"]
R2 --> R3["隐藏业务真实场景与痛点"]
R3 --> R4["AI 落地停留在表面/失败"]
end
subgraph Augmentation["正确心智：赋予超能力（CVS 模式）"]
A1["目标：给 50 名员工赋能"] --> A2["建立安全与信任环境"]
A2 --> A3["业务一线主动涌现高价值用例"]
A3 --> A4["释放繁琐杂务，回归核心价值<br/>（如临床医护专注服务患者）"]
end
```
Deepu Kumar 强调，这一理念正是 CVS 领导层自始至终推行的根本原则：AI 永远是“增强型”（Augmentative）的助手。在医疗健康领域，CVS 的目标是让 AI 替临床医护人员剥离繁琐、枯燥的机械事务，使医生和药剂师无需整天盯着系统修补数据，而是把最宝贵的时间留给患者。
这种对员工“绝不替代”的安全感承诺，直接催生了巨大的业务创新红利。Arijit 坦言，许多由 CVS 一线基层员工自主构想出的 AI 应用场景，技术专家团队哪怕想破脑袋也绝对设计不出来，因为外部专家永远不可能比一线人员更懂深层的业务流转与痛点。如果在企业内部无法构建这种心理安全感（Psychological Safety），员工就不会主动思考“如何用 AI 为公司创造价值”，企业就等于在智能化转型中白白浪费了最核心的战略资产——自己的团队。
- **核心心智切换**：坚决从“替代 50 人”的成本节约逻辑转向“赋予 50 人超能力”的价值倍增逻辑。
- **临床一线减负**：剥离系统维护等繁杂流程，将高素质人才的时间重新归还给医患面对面交流与高价值关怀。
- **激发隐性知识**：消除员工被替代的恐慌，才能激活基层对业务死角的洞察，挖掘出外部技术专家无法企及的高价值落地点。
---
### [16:27](https://www.youtube.com/watch?v=luyznCueMM8&t=987s) 跨职能协同与创新准入：不设门槛的全员参与模式
在第二个观众提问中，有人问及目前使用该平台的实际用户是否全都是非技术业务人员。Arijit 指出，不同组织在推进路径上略有侧重，例如内布拉斯加州（State of Nebraska）在落地时主要由 CIO 带领年轻实习生团队推进业务化开发；但在 CVS 内部，这完全是一场“全员参与的狂欢”（"They want everybody at the party"），呈现出高度融合的跨职能协同特色。
Deepu 解释道，在大型零售药房与健康服务体系中，长期深耕具体系统的技术工程团队往往比普通的业务运营团队更深刻地理解底层业务逻辑与功能边界。因此，最理想的实践永远是技术人员与业务人员紧密结对、高频协同，共同推导最佳方案。
在组织机制与系统权限上，CVS 采取了极其开放的策略：平台接入了企业统一单点登录（SSO），内部任何员工都可以随时登录并开始构建 Agent 试验原型。这是因为在**创新探索阶段**，企业绝对不应该对创意设立门槛或设置审批路障（"You're not trying to gate ideas"）；只有当原型验证成功、准备正式推向**生产部署阶段**（Production）时，才需要启动严密的安全审查与准入把关。这种“前端极度包容、后端严格准入”的机制，最大限度地释放了全员创新的活力。
- **跨职能无缝协同**：技术工程师（懂业务逻辑）与一线运营（懂实际痛点）深度结对，避免技术与业务脱节。
- **单点登录（SSO）零门槛探索**：全员皆可自由接入平台进行敏捷实验，杜绝在创意萌芽期设立繁琐的审批卡点。
- **创新与生产的门禁分离**：在创新探索期鼓励百花齐放，在生产上线期实施严谨把关与安全治理。
## Notable quotes
> [00:00](https://www.youtube.com/watch?v=luyznCueMM8&t=0s) AI is a waste of money — Arijit Sengupta
> [01:36](https://www.youtube.com/watch?v=luyznCueMM8&t=96s) 我知道为什么会这样了，让我们加进这个变量试一下 — Yogesh
> [04:26](https://www.youtube.com/watch?v=luyznCueMM8&t=266s) 去试吧，只要你能证明它创造了足够的业务价值，我就给资金支持 — Tony Ambrosie
> [07:53](https://www.youtube.com/watch?v=luyznCueMM8&t=473s) 如果你认为自己能彻底消除生成式 AI 的幻觉，说明你还没真正理解这项技术 — Arijit Sengupta
> [10:10](https://www.youtube.com/watch?v=luyznCueMM8&t=610s) training an AI like an intern not a pet — Arijit Sengupta
> [10:10](https://www.youtube.com/watch?v=luyznCueMM8&t=610s) Hey, you got this wrong. This is why you got it wrong. Don't do it again. — Arijit Sengupta
> [14:27](https://www.youtube.com/watch?v=luyznCueMM8&t=867s) The human will get superpowers — Arijit Sengupta
> [16:27](https://www.youtube.com/watch?v=luyznCueMM8&t=987s) They want everybody at the party — Arijit Sengupta
> [16:27](https://www.youtube.com/watch?v=luyznCueMM8&t=987s) You're not trying to gate ideas — Deepu Kumar
## Mentioned
Arijit Sengupta, Deepu Kumar, Tony Ambrosie, Tilak, Yogesh, Aible, CVS Health, BeyondCore, Salesforce Einstein Discovery, NVIDIA, Snowflake, Google BigQuery, State of Nebraska, AI is a waste of money
## Video description
Arijit Sengupta, Founder and CEO, Aible
Deepu Kumar, Vice President of Engineering, Pharmacy and Consumer Wellness, CVS Health
Abstracts
Join this session with Chief Digital & Technology Officer of CVS Health on how the retail leader empowers non-technical business teams to build and deploy AI agents within days to solve real-world business challenges.
Learn how CVS is transforming enterprise AI development through rapid prototyping and human-centered design. Discover their proven methodology for moving from concept to production in just 30 days, why they treat AI agents like interns who need training and feedback, and how reasoning models are creating more reliable and trustworthy AI systems. They also explore the future of autonomous agents, the importance of responsible AI in healthcare, and why the next wave of AI will focus on empowering humans rather than replacing them.
[Source](https://www.youtube.com/watch?v=luyznCueMM8)
Source quotes
[0:00] Arijit Sengupta: AI is a waste of money
[1:36] Yogesh: 我知道为什么会这样了，让我们加进这个变量试一下
[4:26] Tony Ambrosie: 去试吧，只要你能证明它创造了足够的业务价值，我就给资金支持
[7:53] Arijit Sengupta: 如果你认为自己能彻底消除生成式 AI 的幻觉，说明你还没真正理解这项技术
[10:10] Arijit Sengupta: training an AI like an intern not a pet
[10:10] Arijit Sengupta: Hey, you got this wrong. This is why you got it wrong. Don't do it again.
[14:27] Arijit Sengupta: The human will get superpowers
[16:27] Arijit Sengupta: They want everybody at the party
[16:27] Deepu Kumar: You're not trying to gate ideas
Original transcript
[0:00] Before
[0:04] this I started a company called
[0:05] BeyondCore which became Salesforce
[0:07] Einstein discovery. I actually wrote a
[0:09] book called AI is a waste of money based
[0:12] on a thousand AI projects I had done and
[0:14] yes I'm still proud of that book and I'm
[0:15] still right about it. The point there
[0:17] was most AI projects fail because of
[0:20] disconnect between data scientists and
[0:22] business users. That's the reason why
[0:23] they fail and that has continued to be
[0:25] the case today. Um today we are going to
[0:28] be talking through a practitioner. Deepu
[0:30] if you would like to come up. Um and the
[0:33] thing that we like doing and you're
[0:34] going to see this a little bit
[0:35] different. We will always start from
[0:37] something real that has been done and
[0:39] then we're going to discuss why that
[0:41] happened. But the what first and then
[0:43] the why because we don't want to talk
[0:44] about vaporware people.
[0:46] >> No drum rolls.
[0:47] >> No drum rolls. I'll do it for you.
[0:50] >> All right. Um hello everyone. Thank you
[0:53] for having me here and thank you all for
[0:54] joining. Um I'm Deepu Kumar. I lead
[0:58] pharmacy engineering for CVS.
[1:01] That's it. Okay. Deep and I have known
[1:03] each other for years. Like he was at
[1:04] Disney, he was at Baptist Health. Like
[1:07] the guy has had a very storied and broad
[1:09] career. Are you not going to add
[1:11] anything to that? Like okay like I
[1:13] hopefully he's going to speak more
[1:14] during the session. We'll see. So we are
[1:17] going to talk about three key things.
[1:18] One is how do to to succeed you need to
[1:21] iterate quickly and have rapid
[1:23] prototypes. Second is how do you empower
[1:26] your stakeholders? That idea of
[1:28] empowerment is very important. And then
[1:30] finally, how do you build trust between
[1:32] the user and the AI? These are the three
[1:34] key pillars to success. But let's start
[1:36] with rapid prototyping. And this is an
[1:38] example of a project we are not allowed
[1:40] to talk about any specific use cases. So
[1:43] if I accidentally say something, it is
[1:45] not from CVS. I'm always never going to
[1:47] talk about CVS case studies use cases.
[1:50] But the mechanics of it is very
[1:52] interesting. In the first two days of
[1:55] working on this project, the team did 20
[1:57] iterations. The first data set was 4.8
[2:00] billion rows of data evaluated across 10
[2:03] million rows of data. Second was 1.8
[2:06] billion evaluated at 550K. And you
[2:08] should be like why did the number go
[2:10] down? The point here is you don't want
[2:13] to get to the perfect data set. You
[2:14] can't get to a perfect data set in any
[2:17] real organization like CVS. So that's
[2:20] what the journey was. And how do you get
[2:22] to the business user review within days?
[2:24] How do you get to the executive review
[2:26] within days? And by day eight, it was
[2:29] already getting reviewed by the chief
[2:30] digital officer. That's the rapid
[2:33] iteration and rapid prototyping. Deep
[2:35] like broad picture thoughts. I think the
[2:38] question at hand is how do you scale AI
[2:40] use cases um through an organization the
[2:44] scale of CVS?
[2:47] We are committed in terms of responsible
[2:50] and um responsible use of AI. Hence we
[2:54] have the foundations well established. I
[2:57] believe the fundamentals are same for
[2:59] any enterprises looking to scale. You
[3:02] need to have a robust infrastructure.
[3:04] You need to have frameworks that
[3:06] abstract the governance and other
[3:08] crosscutting aspects from the user so
[3:11] they don't have to reinvent the wheel.
[3:14] You need to have cross collaboration
[3:15] between the business stakeholders and
[3:17] the the tech teams. And last but not
[3:20] least, you need to have test, iterate
[3:23] and improve mindset. What I will harp a
[3:27] little bit on the last point because you
[3:29] cannot wait for the stars to align or
[3:32] the data to be perfect to start. Many a
[3:35] times when you you need to have that
[3:37] prototype pilot uh production mindset.
[3:41] Many of times when you start doing the
[3:43] prototype you may uncover things that
[3:45] you never thought of which add
[3:48] incremental business value to to a
[3:50] certain extent. I don't know if you want
[3:51] to.
[3:52] >> No, absolutely. And to take this example
[3:54] of this project, there was this guy from
[3:56] uh from your organization, Yogesh. Like
[3:58] we were literally doing iterations. He
[4:01] was looking at the pattern. He's like, I
[4:02] know what that happened. Let's add this
[4:04] variable in there. Let's try this thing.
[4:05] Let's try that. And the entire team, his
[4:07] entire team would just go do another cut
[4:10] right then and there. That kind of
[4:12] mindset like you almost have to have a
[4:14] little bit of humility going into the
[4:16] project, right? Not that we know
[4:18] everything, but we will see what comes
[4:20] out. or we will learn from it and we
[4:21] will adjust. That was key that rapid
[4:24] iteration and that willingness to
[4:25] adjust.
[4:26] >> Correct. The other part of this is how
[4:29] do you get many stakeholders? You
[4:30] mentioned this a little bit but let's
[4:32] think about the process. The mistake
[4:34] people are making today in this market
[4:36] is they're trying to get to a good use
[4:39] case. Like literally yesterday at the
[4:41] training session here with HPE somebody
[4:43] asked us if I give you a perfectly
[4:45] defined agent spec what can you do with
[4:47] it? The problem is nobody has that.
[4:49] Nobody has a perfectly defined agent
[4:51] spec because your end stakeholders don't
[4:54] know what a perfect agent is yet. Right?
[4:57] So what you need to do is get many many
[4:59] shots at target. Our focus at CVS was
[5:02] how do we get many users on board in the
[5:05] first two months? 450 users building
[5:07] agents not consuming building agents 750
[5:10] projects 300 million plus variable
[5:13] combinations 100 billion rows of data.
[5:16] And then before you take it to anything
[5:17] real, you try many many projects, right?
[5:20] Most of those projects are not going to
[5:21] fail, not going to succeed. We don't
[5:23] have 750 production projects obviously
[5:26] in the first two months. You fail
[5:29] majority of the projects, but if you're
[5:31] failing very very quickly and you learn
[5:33] and adjust and you find the ones that
[5:35] matter, go through the validation with
[5:37] IT business and the chief digital
[5:39] officer, then you get to the agents used
[5:41] in production. Thoughts on that? Yeah,
[5:43] many times to solve meaningful problems
[5:46] in production, you need the right
[5:48] stakeholders from the onset. Um, so that
[5:50] typically is a businessme and a
[5:54] technical theme working in collaboration
[5:57] from the onset.
[5:58] While that's valid for production use
[6:01] cases, it's important for us to iterate
[6:03] and experiment in a sandbox environment
[6:06] with mockdub data that resembles real
[6:08] world scenarios so that ideas can flow.
[6:12] So I think that uh ability for users to
[6:15] experiment is is paramount in terms of
[6:18] how we take the business to the next
[6:19] level and the permission to fail from
[6:22] senior leadership. So Tony Ambrosie the
[6:24] chief digital officer of this business I
[6:26] have seen him interact with these team
[6:28] members and they'll come and say well I
[6:29] want to try this agent and Tony never
[6:31] says no. He's like okay try it then show
[6:34] me how it creates enough value. If it
[6:35] creates enough value I'll fund it.
[6:37] Right? That mindset of you don't just
[6:40] try. Let's see what happens. If
[6:42] leadership doesn't say it's okay to
[6:44] fail, people are going to be terrified
[6:46] of failing. Any thoughts on it? Because
[6:48] you don't don't get us wrong. We don't
[6:51] fail miserably in production, especially
[6:53] when dealing with efficients. This is
[6:56] all in a controlled environment, in a
[6:58] sandbox environment where teams have the
[7:00] liberty to experiment.
[7:01] >> Yeah. And absolutely only in
[7:03] prototyping, right? Like notice that
[7:05] validation with a big check mark. Very
[7:07] few projects go from the left to the
[7:09] right.
[7:10] The whole idea is innovate like crazy on
[7:13] the left but be extremely rigorous when
[7:15] you move it to the right. Does that make
[7:17] sense? You do not want to take risks on
[7:20] things going to production that is by
[7:22] design but if you try to make if you
[7:25] don't allow that kind of try many things
[7:28] like stop thinking about the perfect use
[7:30] case. Think about I'm going to create a
[7:32] thousand agents this week. Like the
[7:34] people who came to the training
[7:35] yesterday, if anybody's in the room, you
[7:37] guys were training agents live in the
[7:39] room while we were there. That's what
[7:42] the mindset needs to be. How do I build
[7:43] many prototypes in 30 minutes, an hour,
[7:46] 48 hours and then some of them will
[7:49] succeed. Fair? Yep. The other part, this
[7:53] is building trust between actually
[7:55] instead of going to the slides here, I
[7:57] actually wanted you talk about first and
[7:58] I'll go to the slides because they're
[7:59] product specific. Please.
[8:01] >> Yeah. I think the transparency and trust
[8:03] in how we use AI is paramount for us and
[8:06] we are not forcing tools into employees
[8:09] and thou shalt do it. It's more for
[8:11] their convenience and and so and hence
[8:16] um the logic in terms of how the
[8:18] decision process is being made is is of
[8:21] important because that's how the
[8:22] employees build trust and confidence.
[8:25] I'll I'll
[8:27] outline few uh things that we follow as
[8:31] an organization. Number one is as I
[8:33] mentioned the transparency in the
[8:34] algorithms. The a the AI has to explain
[8:38] how the decision process was made, what
[8:40] data is it used, what algorithms it
[8:43] used, what's the rationale behind the
[8:45] decision and we have that audit trail
[8:47] available for employees to review.
[8:51] Number two um is the uh feedback
[8:55] mechanism. The employees have the
[8:56] ability to provide feedback into the
[8:59] performance of the agent. Not just uh
[9:01] trivial yes or no but sophisticated uh
[9:05] feedback that in turn will improve how
[9:08] the agent performs next time onwards.
[9:10] Number three,
[9:12] um, and and I and I'm I'm big on this in
[9:17] terms of augmentative versus uh
[9:20] replacements. We always use AI in terms
[9:23] of augmenting our employees work, not in
[9:26] terms of replacing. And hence the final
[9:29] decision and review and judgment is with
[9:31] our employees, not with AI. Yeah. Our
[9:34] mission by the way the able tagline of I
[9:37] am able we always start with this idea
[9:39] of I our job is to empower the human it
[9:43] is not to replace the human right and to
[9:46] the first point you were making like an
[9:47] example of that validation is this thing
[9:49] called blue is true what we do is a AI
[9:52] will hallucinate by the way you cannot
[9:54] prevent a generative AI from
[9:55] hallucinating if you think you can
[9:57] you're actually not understanding the
[9:58] tech well enough but what we do is we
[10:01] have a second system automatically
[10:03] double double check the output of the
[10:05] generative AI trace back to the source
[10:07] data independently with all of that
[10:10] audit trail saved off in CVSS's
[10:13] environment so anybody can independently
[10:14] see it and users can just put their
[10:16] mouse on top of any blue highlighted
[10:18] text and see the chart that comes up any
[10:20] chart on the right hand side is only
[10:22] deterministic it literally has Python
[10:24] code backing it up so you can go see
[10:26] exactly where the data came from
[10:29] everything on the left is generative
[10:31] that's an example of how you do the
[10:33] first point DU was talking about on the
[10:35] reasoning steps. What we found is we
[10:37] call this training an AI like an intern
[10:39] not a pet. So here what's happening is
[10:41] you're detecting a situation and the AI
[10:43] got something wrong. You provide
[10:45] feedback on the reasoning steps. This is
[10:47] and we deliberately didn't use any CVS
[10:49] projects throughout. This is a lost
[10:51] baggage use case precisely because it
[10:53] has nothing to do with CVS. But you know
[10:55] the the AI saw the people in the
[10:57] background and said oh there are people
[10:58] there. This baggage is not lost. And the
[11:01] feedback the user gave is that the
[11:02] people are too far away. Right? That's
[11:05] how you have to think about it. Again,
[11:06] the people who came to the session
[11:07] yesterday, that was the kind of feedback
[11:09] you were giving the AI. The AI explains
[11:12] its reasoning. People give feedback on
[11:15] the reasoning steps giving useful
[11:16] information like this is our definition
[11:18] of gross margin instead of thumbs down.
[11:21] Because thumbs down could mean you're
[11:23] you are too verbose. You got the
[11:25] definition of gross margin. You looked
[11:26] at the wrong data set. It could mean
[11:28] many things. When you provide feedback,
[11:31] you build a trust bridge on both sides.
[11:33] The human trusts the AI because the AI
[11:36] is explaining itself and actually joke
[11:38] and say the AI trusts the human where
[11:40] the human instead of kicking it and
[11:42] saying, "Hey, you did wrong." Actually
[11:44] says, "Hey, you got this wrong. This is
[11:47] why you got it wrong. Don't do it
[11:48] again." That's how you build trust
[11:50] between the AI and the human. Anything
[11:52] you'd like to add to that?
[11:53] >> No, I think you covered it well.
[11:55] So um there's a couple of and by the way
[11:58] I think we should do some we have five
[12:00] minutes left. Okay. So um the two
[12:03] interesting sessions you want to talk
[12:04] about this because you guys might want
[12:05] to take the QR codes for these. Yeah, I
[12:07] think we have we have been outright
[12:10] talking about how CVS is looking at AI
[12:12] and Tony has uh podcast and other um so
[12:17] so does Tilak um please read and then
[12:20] you'll be able to understand our
[12:22] approach and how we are progressing and
[12:24] Tony is one of my favorite thought
[12:26] leaders by the way guys especially the
[12:27] left one which we did uh with Nvidia you
[12:30] should definitely go take a look at that
[12:32] um really makes a big difference thank
[12:34] you so much for doing this Deep let's
[12:36] show a little bit of how we do this as a
[12:38] product but I don't want to torture you
[12:39] by having you on stage.
[12:40] >> Thank you.
[12:41] >> Thank you. And we will bring you back
[12:42] for the Q&A if you have questions.
[12:44] >> Sure.
[12:44] >> Thank you so much Deepo.
[12:45] >> So very quickly on able what our focus
[12:48] is is how do we build these specialized
[12:50] AI agents on the customer's data on
[12:53] their business process on their
[12:55] functions. You can do a wide variety of
[12:57] agents because everybody has different
[12:59] idea of what agents are. You can have
[13:01] extremely simple to extremely complex
[13:02] agents. But these agents are all
[13:05] building on top of one shared platform.
[13:08] So if you're doing something on
[13:09] structured data, the right set of
[13:11] services come up. If you're doing
[13:12] something on unstructured data, the
[13:14] right set of services comes on. Your
[13:16] data never leaves where it was. If it
[13:18] was in Snowflake, if it was in BigQuery,
[13:19] it is staying there. When we did those
[13:21] billions of variable uh rows of data,
[13:24] millions of variable combinations, those
[13:26] queries were pushed to that underlying
[13:28] platform, right? And we do it very
[13:30] quickly. So think of it as you can ask a
[13:33] million questions. A AI can ask a
[13:34] million questions in the time a human
[13:36] can ask about seven questions. So you're
[13:39] still getting charged for seven
[13:40] questions because Snowflake, BigQuery,
[13:43] etc. actually charge you by the length
[13:44] of time the system is up answering
[13:47] questions. So you get charged for seven
[13:49] questions, but the AI asked a million
[13:50] questions on your behalf.
[13:53] And then you have things like this kind
[13:54] of a what's changed analytics looking at
[13:56] all of the different patterns. Like
[13:58] think of in the world of retail, think
[14:00] of in the world of logistics, any case
[14:02] where things are changing rapidly, being
[14:05] able to look at many many variable
[14:07] combinations, figuring out where things
[14:09] have changed, why things have changed,
[14:11] turning it into human answerable um
[14:14] questions, uh human understandable, uh
[14:17] responses and the ability to answer
[14:19] questions. And you guys can come to the
[14:21] booth and see live demos of this. Bring
[14:24] your own sample data and we can play
[14:26] around with it. I believe we have time
[14:27] for one question. Is your mic working?
[14:30] So the question is uh when I said AI is
[14:32] not going to replace humans, is it a
[14:34] position for today or is that going to
[14:35] be for the future? My point is going to
[14:38] be that what the human does will be
[14:40] different but the human will get
[14:43] superpowers. So don't think in terms of
[14:46] I will replace 50 humans. You should
[14:48] think in terms of I'm going to give my
[14:50] 50 stakeholders superpowers so the
[14:53] company can do things it can't do today
[14:55] and make a lot more money or save a lot
[14:57] more money than it's doing today. The
[14:59] mental model has to be how do we give
[15:01] people superpowers instead of how do you
[15:03] replace people? Is that a fair thing?
[15:05] Yes. Yes. Anything else you would like
[15:08] to add to that? That has been the stance
[15:10] if you have noticed our leaders coming
[15:12] out with um AI and and how we approach
[15:16] CVS has the stance that we are not going
[15:18] to use AI to replace humans. It's always
[15:20] going to be augmentative. And one thing
[15:22] you'll find is the combination of the
[15:24] thing he was talking about earlier where
[15:26] you get a bunch of people to take part
[15:28] right and telling people that and
[15:30] committing to people that you're not
[15:32] trying to replace them. What that does
[15:33] is you create real buyin. Like one of
[15:36] the things that fascinates me um though
[15:38] I can't talk about use cases and I'm
[15:39] sticking to it. There are use cases that
[15:42] CVS employees have brought up that my
[15:44] team would have never thought of because
[15:46] we don't understand their business well
[15:47] enough. Right? Experts will not
[15:50] understand as well as the actual end
[15:52] users about these processes. So if you
[15:55] don't create a safe environment where
[15:57] everybody on the team is saying how can
[15:59] I use AI to create value for CVS? You're
[16:02] just like leaving behind the most
[16:05] valuable resource you have which is your
[16:07] team
[16:08] >> and again there is value at work that
[16:10] the humans can provide and take away the
[16:13] mundane stuff that's what our approach
[16:15] has been always so that our clinicians
[16:18] have the time to spend with the patients
[16:20] not looking at the system and fixing
[16:22] things sorry I think we are definitely
[16:25] out of time now or do I have time for
[16:26] one more question one more question
[16:28] anybody has one more question it's a mix
[16:31] so the question was are the users all
[16:34] business users? So there are other
[16:36] organizations like for example the state
[16:37] of Nebraska um the CIO and bunch of
[16:41] interns they talked about they are
[16:42] mostly using business users at CVS it's
[16:45] a mix of all right like CVS again I
[16:48] don't want to speak for you but the I
[16:50] don't see this kind of line they want
[16:52] everybody at the party is it
[16:54] >> yeah I think it's mostly a crossf
[16:56] functional right the the tech teams who
[16:58] have been staying with the system long
[17:00] enough understands the business and the
[17:02] function much more than the business
[17:05] operations teams. So it's always a
[17:07] constant collaboration between the
[17:08] parties to make sure we get to the route
[17:11] right outcome. And the way we are set up
[17:14] by the way we don't distinguish between
[17:15] different kinds of users. So anybody has
[17:17] a single sign on they can come in and
[17:19] start using the platform. Again remember
[17:21] you're on the innovation side. On the
[17:22] innovation side you're not trying to
[17:24] gate ideas. You're gating ideas when you
[17:27] go into production.
