Strauss AI Chief Says Enterprise Deployments Stall on Quality Evals
Ai4
Most enterprise AI initiatives fail because organizations treat prompt generation as the complete skillset rather than enforcing quantitative benchmark evaluations and clean data governance.
## Background
Published on YouTube by Ai4 on August 21, 2026, this fireside chat features Ilai Soloducho (Chief AI Officer at Strauss Group) and Nurit Revah (Founder of EmbraceAI Global). The session outlines an executive playbook for closing the enterprise AI adoption gap by moving workforces beyond basic tool access into structured, measurable AI fluency.
## TL;DR
The speakers examine why enterprise AI adoption stalls post-rollout, emphasizing that prompt engineering accounts for only a fraction of genuine organizational fluency. They outline a dual-track strategy balancing off-the-shelf quick wins with transformative core initiatives, backed by rigorous validation practices such as golden datasets and unglamorous data infrastructure preparation. Addressing workforce anxiety through historical precedents like bank ATMs, they frame AI as an augmenting 'Superman suit' that elevates employees from first-drafters to orchestrators and curators. The session wraps with a concrete Monday morning plan focused on workflow audits, multi-stakeholder governance, and disciplined value-driven execution.
## Atmosphere & style
The presentation maintains a pragmatic, highly credible, and executive-level tone, drawing naturally upon the speakers' backgrounds across Google X, Palantir, and regulated industry environments. It feels like an unpretentious, real-world debrief rather than tech hype, opening with interactive audience polling and progressing through structured frameworks. The dynamic between Nurit and Ilai is collaborative and reassuring, methodically replacing corporate FOMO and employee anxiety with clear, actionable operational rigor.
## Key takeaways
- [02:01](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=121s) The fundamental enterprise AI bottleneck is no longer tool access or daily adoption, but the inability to systematically measure, benchmark, and trust output quality.
- [03:25](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=205s) To scale beyond failed pilots, leaders must pursue a disciplined two-track portfolio: low-barrier 'Quick Wins' for immediate productivity alongside strategic 'Big Stones' for core business transformation.
- [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) Prompting represents only 25% of true AI fluency; organizations must guide employees to become 'AI orchestrators' who master strategic delegation rather than mere first drafters.
- [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) Workflow redesign should maintain organizational humility by evaluating existing processes across three clear buckets: what to preserve, what to discard, and where to innovate.
- [08:36](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=516s) Individual validation tactics—such as requiring ingestion playback, stating underlying assumptions, and scoring confidence levels—are critical to preventing confident AI hallucinations.
- [08:36](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=516s) Enterprise AI quality requires transitioning from subjective intuition to standardized quantitative evaluation suites utilizing curated 'golden datasets' of benchmark prompt-output pairs.
- [12:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=729s) Resilient data architecture, transformation pipelines, and governance account for 80% to 90% of the effort required to power advanced RAG, fine-tuning, and agentic systems.
- [12:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=729s) Structuring fragmented organizational knowledge into interconnected context layers and knowledge graphs functions as an internal 'second brain' that accelerates onboarding and eliminates silos.
- [15:50](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=950s) Like the introduction of ATMs which expanded bank branches and shifted tellers into wealth advisory roles, AI acts as a 'Superman suit' that augments workers into higher-value strategic contributors.
- [18:01](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=1081s) Sustainable AI operationalization demands an immediate action plan: conducting structured workflow audits, pursuing measured baby steps, and establishing multi-stakeholder governance across IT, Legal, HR, and Finance.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=4s) Speaker Introductions and Technical Backgrounds
Ilai Soloducho introduces himself as the Chief AI Officer and Chief of Staff to the Group CEO at Strauss Group. His background in technology started exceptionally early: programming at age seven, launching a social network startup at age twelve, and completing his computer science degree by fifteen. After serving in Israeli intelligence and the Israel Prime Minister's office, he joined Palantir, where he collaborated with Fortune 500 executives and government leaders to drive data-informed decision-making. Following his tenure at Palantir, Ilai completed a master's degree at the Harvard Kennedy School in Boston, focusing on tech leverage within major institutions, and later worked at Google X (the Moonshot Factory) co-founding an AI-for-coding initiative that eventually graduated into Google Labs to build advanced AI agents and products.
Nurit Revah, Founder and CEO of EmbraceAI Global, brings over twenty years of experience as an AI architect and technical leader across diverse sectors including energy, manufacturing, and heavily regulated verticals like financial services and biotechnology. Holding both a master’s degree in AI and an Executive MBA, Nurit notes that rolling out AI across complex organizations is notoriously difficult and emphasizes her goal of providing practical, actionable pathways forward.
[02:01](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=121s) The AI Adoption Gap: From Daily Tool Use to Quality Measurement
To gauge the audience's current maturity with artificial intelligence, Nurit initiates an interactive room poll. Nearly every attendee raises their hand when asked if their organization has rolled out AI tools and whether teams utilize them on a daily basis. However, when asked if they can systematically measure the quality of what those AI tools produce, the vast majority of hands drop across the room. This drop-off highlights the fundamental disconnect between superficial tool adoption and true enterprise fluency.
```mermaid
flowchart LR
accTitle: The AI Adoption Maturity Funnel
subgraph Funnel["Enterprise AI Adoption Gap"]
A["1. Tool Rollout<br/><i>(Wide Access)</i>"] --> B["2. Daily Usage<br/><i>(Broad Adoption)</i>"]
B -.-> C["3. Quality Measurement<br/><i>(Significant Enterprise Drop-off)</i>"]
end
```
[03:25](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=205s) Organizational Traps, Overcoming Hype, and "Quick Wins" vs. "Big Stones"
Addressing the widespread industry buzz, Ilai explains why 80% of organizations fail to scale their AI initiatives beyond the pilot phase. Demonstrations, promotional videos, and startup pitch decks frequently display impressive capabilities, but large enterprises cannot operate in production without absolute trust and verifiable quality. Leaders often succumb to anxiety and FOMO, trying to inject AI into seventy or eighty different places at once without achieving dependable results anywhere.
Ilai advocates for a disciplined portfolio split into two distinct tracks:
- **Quick Wins:** Low-barrier, off-the-shelf capabilities (such as deploying standard $20/month SaaS licenses) that allow teams to execute routine tasks significantly faster without complex engineering.
- **Big Stones:** Strategic, long-term initiatives that fundamentally transform business economics. As Ilai stresses, "You will not reduce the cost of your company by 20% in a day or you will not increase revenue in in 50% in a week." Both tracks must be pursued simultaneously.
[05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) The Four Competencies: Shifting from First Drafters to AI Orchestrators
Nurit points out that merely handing employees access to ChatGPT is not genuine AI adoption. Prompting accounts for only 25% of the four key competencies required for true AI fluency; treating prompt engineering as the entire upskilling curriculum is a primary reason corporate adoption stalls.
Organizations must guide employees through a mindset shift from being an "AI first drafter" (using an LLM simply to generate a marginally better Word document) to becoming an "AI orchestrator." Referencing the Anthropic 4Ds AI fluency framework, Nurit explains that orchestration requires mastering *delegation*—deliberately determining what tasks stay under human-in-the-loop accountability versus what can be safely assigned to the machine. When redesigning workflows:
- Focus on the core business value chain rather than automating randomly for external PR or LinkedIn optics.
- Respect legacy processes that are already performing well, maintaining organizational humility.
- Categorize operations into three buckets: what to **preserve** (leave untouched), what to **discard** (outdated or improvable by at least 2x), and where to **innovate** (creating true 0-to-1 impact).
[08:36](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=516s) Validation Strategies: Curbing Hallucinations and Implementing Evals
Because language models speak with persuasive authority, humans naturally fall into the cognitive trap of believing confident-sounding errors. To prevent teams from mistaking "confident nonsense" for high-quality outputs, Nurit and Ilai detail concrete validation practices at both the individual prompt level and the organizational evaluation level:
- **Ingestion Playback:** Asking the model to explicitly restate what it ingested from an uploaded document before generating a summary substantially reduces hallucinations.
- **System Prompting for Assumptions & Confidence:** Instructing the AI upfront to declare its underlying assumptions and output a clear confidence rating reveals whether it is genuinely grounded in source data or merely guessing.
- **Critical Thinking as Curation:** Elevating human workers to "AI curators" who actively judge, verify, and critique model generations.
- **Quantitative Evals & Golden Datasets:** Ilai emphasizes standardizing evaluation suites using "golden datasets" of 15 to 20 representative prompt-output pairs to mathematically quantify accuracy (e.g., determining whether a pipeline operates reliably 90% or 60% of the time) rather than relying on subjective intuition.
- **Post-Implementation Impact Surveys:** Deploying structured internal surveys to systematically track financial ROI, time savings, and worker productivity against pre-defined metrics.
[12:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=729s) Data Architecture as the True Prerequisite: Beyond Vanilla LLMs
Addressing why clean data infrastructure remains the essential foundation of enterprise AI, Ilai recalls his time at Palantir. Stakeholders frequently requested flashy executive dashboards and advanced analytics, but creating the front-end interface represented only 10% to 20% of the effort; the unglamorous 80% to 90% consisted of building resilient data pipelines, transformation layers, and data governance frameworks.
Moving beyond generic, off-the-shelf "vanilla LLMs" toward advanced techniques like fine-tuning and Retrieval-Augmented Generation (RAG) demands pristine proprietary data. Clean data practices must extend down to daily employee habits—ensuring PDFs, spreadsheets, Word documents, and email archives are systematically structured so that agentic systems and natural language interfaces can query them seamlessly.
Connecting this to her master's research on breaking down enterprise data silos within the heavily regulated pharmaceutical sector, Nurit notes that structured data acts as an organizational "second brain" or context layer. By organizing unstructured enterprise knowledge and connecting it to knowledge graphs, companies can drastically shorten lengthy onboarding cycles, enabling new hires to instantly query company terminology and internal institutional knowledge.
[15:50](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=950s) Change Management, Workforce Anxiety, and the "Superman Suit" Analogy
When addressing fears that AI will trigger mass unemployment, Ilai reflects on the shift from public indifference during his Google X days to widespread panic following the November 2022 public release of ChatGPT. To put this anxiety into perspective, he draws a historical parallel to the introduction of Automated Teller Machines (ATMs) in the late 1960s and early 1970s:
```mermaid
flowchart LR
accTitle: Historical Parallel of Technology Augmentation - ATMs to AI
subgraph Historical["The ATM Precedent (1960s-1970s)"]
A1["Introduction of ATMs"] --> A2["Reduced Branch Operating Costs"]
A2 --> A3["Bank Branches Proliferated"]
A3 --> A4["Teller Headcount Rose & Roles Shifted to Wealth Advisory"]
end
subgraph Modern["The AI Augmentation Model"]
M1["Introduction of AI Tools"] --> M2["Automation of Routine Drafting"]
M2 --> M3["Worker Upskilling & Role Expansion"]
M3 --> M4["'Superman Suit': Higher Value Strategic Output"]
end
```
Instead of destroying bank teller jobs, ATMs lowered the cost of opening branches, causing branch locations to multiply and teller employment to grow faster than the broader economy. Crucially, the nature of the teller role shifted from manual check cashing and cash withdrawals toward high-value financial advisory and wealth management. AI serves a similar purpose—acting like a "Superman suit" that empowers workers to become superior versions of themselves rather than replacing them.
[18:01](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=1081s) The Monday Morning Action Plan: AI Audits and Enterprise Governance
To conclude the session, Nurit and Ilai offer concrete, pragmatic guidance for executives heading into their offices on Monday morning:
- **Conduct a Comprehensive AI Audit:** Resist the impulse to automate haphazardly. Map existing workflows, identify employee upskilling requirements, and discover internal AI champions to cultivate grassroots user communities.
- **Proceed in Measured Baby Steps:** Avoid rushing unvetted systems into production; prioritize automations grounded strictly in business value and clear ROI.
- **Establish Multi-Stakeholder Governance:** Recognize that AI implementation is not an isolated technical endeavor or a one-person show. Construct a holistic governance framework integrating Legal, IT, Finance, HR, and Sales to oversee compliance, evaluations, and sustainable scaling.
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=4s) rolling out AI across complex organizations is notoriously difficult — Nurit Revah
> [03:25](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=205s) You will not reduce the cost of your company by 20% in a day or you will not increase revenue in in 50% in a week. — Ilai Soloducho
> [03:25](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=205s) Big Stones — Ilai Soloducho
> [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) Prompting accounts for only 25% of the four key competencies required for true AI fluency — Nurit Revah
> [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) AI first drafter — Nurit Revah
> [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) AI orchestrator — Nurit Revah
## Mentioned
Ilai Soloducho, Nurit Revah, Strauss Group, EmbraceAI Global, Ai4, Palantir, Google X, Google Labs, Harvard Kennedy School, ChatGPT, Anthropic 4Ds AI fluency framework, Retrieval-Augmented Generation (RAG), Microsoft Word, LinkedIn
## Video description
Ilai Soloducho - Chief AI Officer & Chief of Staff to Group CEO, Strauss Group
Nurit Revah - Founder, EmbraceAI Global
Nearly every company has AI tools; far fewer have an AI-fluent workforce — and that gap, not access, is where adoption stalls. In a 20-minute fireside chat, Nurit Revah and Ilai Soloducho share a practical framework for moving teams from first-draft creators to AI orchestrators — the habits that compound, training on real use cases, and measuring quality so trust holds. Leave with an executive playbook for Monday morning.
Strauss Group Chief AI Officer Ilai Soloducho and EmbraceAI Global CEO Nurit Revah argue that enterprise AI maturity requires moving beyond daily tool usage toward rigorous quality measurement, systematic validation, and clean data infrastructure.
Bridging the AI Tool Adoption and Quality Measurement Gap (2:01)
While organizations broadly roll out artificial intelligence interfaces for daily use, systematic quality measurement remains rare. An interactive audience poll conducted by Revah demonstrated that nearly all attendees used enterprise AI tools daily, yet almost none could reliably measure the quality of the resulting outputs. According to Soloducho, roughly 80% of enterprise AI initiatives fail to transition beyond the pilot phase because corporate environments require verifiable trust rather than demo capabilities. To counter executive fear of missing out, he recommends dividing efforts into low-barrier 'Quick Wins' that accelerate routine work alongside strategic 'Big Stones' targeting fundamental business economics.
Expanding Fluency from Prompt Drafting to Workflow Delegation (5:09)
Treating prompt engineering as the entire upskilling curriculum stalls organizational progress, as Revah notes prompting covers only 25% of the four key competencies in Anthropic's 4Ds AI fluency framework. Organizations need workers to shift from 'first drafters' generating minor document improvements to 'orchestrators' who deliberately determine what remains under human accountability versus what can be delegated to automated models. Workflows should be categorized into operations to preserve, outdated processes to discard when improvable by at least two times, and high-impact areas to innovate.
Mitigating Confident Hallucinations Through Structured Evals (8:36)
Because language models deliver incorrect outputs with persuasive authority, Soloducho and Revah advise concrete validation protocols at both individual and organizational tiers. Workers acting as AI curators should instruct models to restate ingested source text, disclose underlying assumptions, and declare confidence levels before presenting finalized answers. At the enterprise level, Soloducho emphasizes standardized evaluation pipelines powered by 'golden datasets' of 15 to 20 benchmark prompt-output pairs to mathematically quantify reliability rates rather than relying on subjective intuition.
Underpinning Advanced Implementations with Resilient Data Architecture (12:09)
Flashy user interfaces represent merely 10% to 20% of enterprise AI engineering, whereas 80% to 90% relies on foundational data pipelines and governance frameworks, Soloducho observes from his background at Palantir. Transitioning from generic LLMs to fine-tuning or retrieval-augmented generation requires structured internal data, including standardized employee files, spreadsheets, and emails. Drawing on research in the pharmaceutical sector, Revah notes that connecting structured institutional knowledge into internal knowledge graphs reduces onboarding time and allows staff to query proprietary operations directly.
Addressing Workforce Anxiety Through Cross-Functional Governance (15:50)
Drawing an analogy to the introduction of bank automated teller machines (ATMs) in the 1960s and 1970s, Soloducho contends that automation does not inherently decrease employment. Just as lower branch operating costs multiplied bank locations and shifted tellers into higher-value financial advisory roles, AI can function as an augmentation tool that elevates employee strategic output. Translating this into practice requires multi-stakeholder governance across legal, IT, finance, HR, and sales divisions to manage audits, compliance, and ROI tracking without reckless deployment.
Soloducho and Revah emphasize that scaling enterprise AI sustainably depends on disciplined baby steps, structured governance, and grounded workflow mapping rather than hurried, indiscriminate automation.
精炼阅读摘要
施特劳斯集团（Strauss Group）首席人工智能官 Ilai Soloducho 与 EmbraceAI Global 首席执行官 Nurit Revah 指出，当前企业 AI 落地普遍存在“高日常使用率但缺乏质量度量”的脱节现象；要突破试点停滞的困境，企业必须构建跨职能治理架构、推行双轨投资策略、夯实专有数据底座，并将员工培养为精通任务委派的“AI 协调者”。
弥合应用脱节与平衡双轨推进策略 (2:01)
现场调研显示，几乎所有受访企业均已普及 AI 工具且团队日常使用率极高，但绝大多数企业无法系统化衡量产出质量。Soloducho 解释称，业内 80% 的 AI 方案止步于试点阶段，源于领导层在错失恐惧（FOMO）驱动下盲目铺开数十个项目，却缺乏在生产环境中必需的绝对信任与实证质量。对此，他提倡实施双轨投资组合：一是部署低门槛、即开即用的“快速成效”（Quick Wins，如人均每月 20 美元的标配 SaaS 许可），使团队无需复杂工程即可加速常规任务；二是推进直击核心商业经济模型的“重大基石”（Big Stones）级长期战略。Soloducho 强调，企业不可能单日削减 20% 成本或单周增收 50%，必须兼顾双轨并同步推进。
员工能力转型：从初稿撰写转向流程协调 (5:09)
Revah 认为，单纯向员工开放 ChatGPT 访问权限绝非真正的 AI 采纳。提示词工程仅占 AI 全面素养能力的 25%，将其视作完整培训体系是导致企业采纳停滞的主因。企业必须引导员工转变心智模式，从利用大模型润色文档的“AI 初稿撰写者”，转型为把控业务全局的“AI 协调者”。借助 Anthropic 4Ds 框架，Revah 指出协调的核心在于掌握“任务委派”能力，严谨界定哪些环节需保留人机回圈（Human-in-the-loop）的权责，哪些可安全交由算法执行。在业务重构中，企业应聚焦核心业务价值链并保持组织谦逊，将运营划分为三类：保留运转良好的成熟流程、淘汰或改造具备 2 倍提升空间的环节，以及开拓具备 0 到 1 突破潜力的创新领域。
质量验证机制：遏制模型幻觉与量化评估 (8:36)
为防止员工误信看似权威的“高置信度胡言乱语”，两位专家给出了微观与宏观相结合的质检方案。在提示词层面，建议采用“输入回放”机制（要求模型在生成摘要前复述上传文档内容），以此显著减少虚构事实；同时在系统提示中要求模型前置说明底层假设并输出置信度评分，促使员工作为批判性“AI 策展人”对产出进行审查。在企业级评估层面，Soloducho 主张建立由 15 至 20 组代表性样本构成的“黄金数据集”（Golden Datasets），以数学化方式量化流水线准确率（例如区分系统可靠度是 90% 还是 60%），摒弃主观直觉；同时辅以上线后的结构化内部调研，系统追踪投资回报率（ROI）、工时节约与全员生产率。
数据底座支撑：超越原生模型的知识中枢 (12:09)
针对企业从通用原生模型转向微调及检索增强生成（RAG）的需求，Soloducho 结合其在 Palantir 的经历指出，前端界面仅占项目总工作量的 10% 至 20%，其余 80% 至 90% 均为数据管道、清洗转换与数据治理框架等底层建设。高质量专有数据不仅依赖系统架构，更取决于员工日常对 PDF、表格、文档和邮件的规范化归档习惯，以便智能体系统无缝检索调用。Revah 基于其在制药行业的研究补充道，结构化数据构成了企业的“第二大脑”与上下文语境层；通过打破企业数据孤岛并结合知识图谱，企业能将专有术语与制度化经验全量贯通，不仅能大幅缩短新员工入职磨合期，更可支持自然语言即时查询内部核心知识资产。
变革管理与落地路径：ATM 隐喻与行动指南 (15:50)
面对 AI 引发的大规模失业焦虑，Soloducho 以 20 世纪 60 至 70 年代自动柜员机（ATM）的引入历史作类比。ATM 降低了网点运营成本并引发分行数量激增，最终促使柜员总用工人数以超越整体经济均速的节奏持续上升，且柜员工作重心也从机械的支票兑现转型为高价值的财富管理与咨询。AI 同样是为员工赋能的“超人战衣”，旨在帮助员工实现能力跃迁而非替代劳动力。为落实转型，两位专家建议管理者制定“周一行动计划”：首先开展全方位 AI 审计，梳理业务流程、技能缺口并挖掘基层倡导者；其次秉持小步快走原则，严守业务价值与 ROI 导向；最后组建由法务、IT、财务、人力资源与销售构成的跨职能治理委员会，统筹合规与可持续规模化。
施特劳斯集团与 EmbraceAI Global 的实践表明，企业跨越 AI 应用鸿沟的关键不在于追踪前沿技术的演示效应，而在于扎实的数据治理、以任务委派为核心的人才培养以及跨职能联合治理架构的落地。然而，如何在低门槛的轻量级工具与高投入的企业级数据基础设施之间维持动态的资源配置平衡，仍是管理层深化 AI 进程时需持续探索的核心命题。
Detailed notes — zh
## Background
本内容由 Ai4 于 2026 年 8 月 21 日发布在 YouTube 平台上。视频记录了施特劳斯集团（Strauss Group）首席人工智能官兼 CEO 幕僚长 Ilai Soloducho 与 EmbraceAI Global 创始人兼 CEO Nurit Revah 的炉边对谈，重点探讨企业如何跨越单纯分发工具的初级阶段，通过习惯沉淀、真实场景实训与质量度量，系统化打造一支具备 AI 熟练度的员工队伍。
## TL;DR
对话深入剖析了企业 AI 落地中“全员使用”与“质量掌控”之间的断层，指出高达 80% 的组织因无法建立信任机制而在规模化阶段受阻。两位讲者提出了兼顾“速赢项目”与“基石战略”的双轨落地法则，呼吁员工角色从被动的“初稿起草者”升级为端到端的“AI 编排者”。针对幻觉抑制与评估度量，他们分享了内容摄入先行呈现、置信度自评以及黄金测试集等工程方法，并强调日常数据素养才是构建企业“第二大脑”的核心底座。最后，讲者以 ATM 机的历史演变为例缓解失业恐慌，为管理者提供了一套涵盖 AI 审计、冠军社区培育与跨职能治理的周一早晨行动指南。
## Atmosphere & style
整场炉边对谈节奏紧凑、干货充沛，洋溢着务实理性的极客底色与资深企业顾问的实战洞察。开场的现场互动瞬间打破了技术狂热的泡沫，精准切中企业真实痛点；两位讲者配合默契、论述层层递进，既有对技术底层架构与工程评测的专业拆解，又有对组织变革管理和人性恐惧的温情关照，为观众营造出一种兼具战略前瞻性与落地可操作性的高效沉浸感。
## Key takeaways
- [02:01](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=121s) 绝大多数企业已普及底层 AI 工具，但普遍缺乏对 AI 输出质量的量化评估机制，形成了从日常使用到质量掌控的能力断层。
- [03:25](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=205s) 高达 80% 的企业在规模化落地 AI 时受挫，原因在于许多方案仅停留在炫酷的 Demo 阶段，进入生产环境后因缺乏信任与质量保障而无法转化为实际 ROI。
- [03:25](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=205s) 企业应推行双轨落地策略：通过低成本的速赢项目（Quick Wins）释放基础效率、建立信心，同时耐心推进基石项目（Big Stones）以实现长期的系统性降本增收。
- [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) 提示词工程仅占 AI 胜任力的 25%，企业必须推动员工从单纯利用工具生成内容的“AI 初稿起草者”升级为端到端重塑业务流程的“AI 编排者”。
- [06:38](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=398s) 流程重塑需遵循“保留、剔除、创新”法则：保留高效运转的成熟流程，果断剔除低效环节，在具备质变潜力的业务价值链核心卡位点全力引入 AI 创新。
- [08:29](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=509s) 防范模型幻觉需在提示词中内嵌验证机制，例如要求大模型先呈现对输入内容的理解要点，并在输出前明确列出假设与置信度自评。
- [10:41](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=641s) 企业需建立严谨的评估体系（Evals），通过构建包含代表性 Prompt 的“黄金数据集”量化模型准确率，并建立 ROI 跟踪闭环。
- [12:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=729s) 数据底座是 AI 落地的核心先决条件，企业 80% 到 90% 的精力应投入在底层数据治理、管道构建与安全性上，而非仅关注上层炫酷的可视化报表。
- [12:54](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=774s) 数据质量源于一线员工的日常素养，规范文档、表格与邮件等非结构化数据，是上层智能体（Agents）和 RAG 检索精准发挥效能的关键。
- [14:33](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=873s) 利用 LLM 与知识图谱构建企业“第二大脑”，能够打破内部数据孤岛并实现非结构化知识的自然交互，从而大幅缩短新员工入职培训周期。
- [15:43](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=943s) 技术自动化并不等于岗位消亡（如 ATM 普及后银行柜员总数反增并转型高价值服务），管理者应将 AI 塑造成赋能员工的“超人战衣”以消除焦虑。
- [17:53](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=1073s) 管理者周一即可启动的实操方案：开展全面的 AI 审计以梳理流程瓶颈，培养内部 AI 冠军社区，并联合财务、IT、法务与 HR 搭建跨职能治理体系。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=4s) 讲者背景介绍与职业履历
Ilai Soloducho 与 Nurit Revah 在开场向参会观众进行了自我介绍。Ilai Soloducho 现任 Strauss Group（施特劳斯集团）首席人工智能官（Chief AI Officer）兼集团首席执行官幕僚长（Chief of Staff to Group CEO）。在加入 Strauss 之前，Ilai 曾在 Google X（登月工厂，The Moonshot Factory）联合创立了 AI 辅助编程相关的研发团队，该团队随后孵化至 Google Labs，主导开发创新型 AI 产品与自主代理（Agents）。Ilai 拥有极其资深的极客背景：7 岁开始自学编程，12 岁创办社交网络初创公司，15 岁完成计算机科学学业。此后，他先后服役于以色列情报部门及以色列总理办公室，之后加入大数据巨头 Palantir，负责协助政府要员与《财富》500 强高管基于数据做关键决策；他在哈佛大学肯尼迪学院获得硕士学位，主要研究方向为大型机构与政府如何利用前沿技术。
Nurit Revah 是 EmbraceAI Global 的创始人兼 CEO。作为深耕行业超过 20 年的 AI 架构师与技术领袖，她拥有人工智能硕士学位及高级工商管理硕士（Executive MBA）学位。Nurit 的行业数字化转型经验覆盖制造、能源等传统领域，以及生物科技、金融服务等强监管行业。她直言企业落地 AI 充满挑战，而她此行的核心目标就是为参会管理者提供一套务实且清晰的推进路径。
- **Ilai Soloducho**：Strauss Group CAIO 兼 CEO 幕僚长；曾任职于 Google X、Google Labs、Palantir、以色列情报部门与总理办公室；拥有哈佛大学肯尼迪学院硕士学位。
- **Nurit Revah**：EmbraceAI Global 创始人兼 CEO；20 余年技术领袖与 AI 架构师经验，拥有 AI 硕士与 EMBA 学位。
---
### [02:01](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=121s) 现场互动：从“全员使用”到“质量掌控”的巨大断层
在正式分享前，Nurit 向全场观众发起了一连串举手互动，借此勾勒企业在 AI 应用上的真实切片。她首先请“团队在组织内已推广部署 AI 工具”的参会者举手，全场观众几乎悉数抬手；接着，她询问“团队中是否每天都在使用 AI”，台下依然举手如林，场面热烈。然而，当 Nurit 抛出第三个关键问题——“如果你们能够量化评估 AI 所生成内容的质量，请继续举手”时，举起的手臂瞬间寥寥无几。
这一鲜明的反差揭示了当前企业 AI 普及与技能提升（Upskilling）的真实困境：绝大多数企业已经完成了底层工具的采购和下发，员工也培养了日常调用的习惯，但在输出质量把控、业务成效评估这一核心环节上，组织内部存在着巨大的能力鸿沟。
---
### [03:25](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=205s) 企业规模化落地 AI 的常见陷阱与双轨策略
针对这一痛点，Ilai 指出，当前市场充斥着过度的炒作与狂热，真正的核心命题是如何把看似炫酷的技术转化为具有实际投资回报率（ROI）的业务成果。数据表明，高达 80% 的组织在尝试规模化落地 AI 时遭遇滑铁卢。许多在演示视频、产品 Demo 或概念验证（PoC）阶段看起来令人惊叹的方案，甚至在初创企业凭此融资数百万乃至数十亿美元之后，一旦进入大型企业的实际生产环境，往往因缺乏信任与质量保障而无法运行。
```mermaid
flowchart TD
accTitle: Enterprise AI Execution Strategy
subgraph Trap["常见误区与痛点"]
T1["盲目跟风: 试图铺开 70-80 个场景"]
T2["80% 企业止步于 Demo / 无法规模化"]
end
subgraph Strategy["聚焦与双轨落地法则"]
S1["聚焦核心: 集中精力做好 2-3 个关键项目"]
QW["速赢项目 (Quick Wins)<br/>• 低门槛、20美元工具授权<br/>• 迅速释放组织效率"]
BS["基石战略 (Big Stones)<br/>• 长期系统性重构<br/>• 驱动降本20%与增收50%"]
end
Trap --> Strategy
S1 --> QW
S1 --> BS
```
Ilai 强调，管理者最容易陷入的组织陷阱就是出于焦虑而试图“全面开花”——盲目开启七八十个项目，最终却没有一个能够建立起让人放心的信赖机制。他建议企业采取聚焦策略，集中做好两到三个高确定性项目，并在执行中严格执行“双轨法”：
- **速赢项目（Quick Wins）**：无需火箭级别的复杂技术，仅通过每人每月 20 美元的工具授权，就能立即帮助组织提速、释放基础生产力。
- **基石项目（Big Stones）**：面向长期的系统性工程。降本 20% 或增收 50% 不可能在一朝一夕间达成，必须给予长线资源与耐心。
- 企业应当**同时推进**这两类项目，用速赢建立信心，用基石构筑壁垒。
---
### [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) 走出工具误区：从“初稿起草者”升级为“AI 编排者”
Nurit 补充分析了阻碍落地深化的两大根因：工作流自动化的选型偏差与培训方式的滞后。很多企业误以为给员工分配 ChatGPT 账号就等于完成了 AI 落地，但“分发工具”与“建立能力”之间存在本质差距。她直言：“提示词工程（Prompting）绝不等于技能提升”，在员工亟需掌握的 AI 核心胜任力模型中，撰写提示词仅仅占到了 25% 的比重。
当前职场必须经历一场关键的认知升级——从“AI 初稿起草者”（AI First Drafter）进化为“AI 编排者”（AI Orchestrator）。Nurit 指出，引入大语言模型的目的绝不仅是帮员工“写出一份更好的 Word 文档”，而是将 AI 作为协作者整合到整个业务链路中，实现端到端的流程重塑。
---
### [06:38](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=398s) 业务重塑与合理授权：区分重构与保留的边界
讨论进而深入到“AI 编排”的核心机制——工作流委托（Delegation）。Nurit 引用了 Anthropic 的“4D AI 熟练度框架”（Anthropic 4D's AI Fluency Framework），将“委托”定义为：清晰界定哪些环节必须保留“人机协同 / 人在回路”（Human-in-the-loop）的把控责任，哪些环节可以完全交由 AI 执行。管理者在重塑流程时，绝不能单纯为了在 LinkedIn 上宣称自己是“AI-first 公司”而盲目自动化，必须全面梳理业务痛点，紧密围绕企业的“业务价值链”（Business Value Chain）进行优先级排序。
Ilai 对此深表赞同，并提出大型企业在拥抱技术时必须保持谦逊，尊重现有系统的既有成效。他提炼了一套极具操作性的业务审视法则：
1. **保留（Preserve）**：对运行良好、产生实际价值的成熟流程，切忌盲目改动；
2. **剔除（Discard）**：对低效、过时，或者能够借助 AI 带来至少 2 倍（2x）以上效率飞跃的环节果断淘汰重构；
3. **创新（Innovate）**：在能够产生从 0 到 1 质变、真正撬动业务增长的核心卡位点全力引入颠覆性技术。
---
### [08:29](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=509s) 质量校验与对抗幻觉：构建透明度与批判性思维
针对“如何教导员工甄别大模型自信胡说（Confident Nonsense）”这一普遍难题，Nurit 提出必须将输出验证机制内嵌到标准提示词结构中。由于人类天生倾向于相信看似笃定且权威的回答，员工的“批判性思维”（Critical Thinking）训练已成为当下紧迫的技能要求。Nurit 分享了两项经过研究验证且切实行之有效的实操技巧：
- **内容摄入先行呈现（Ingestion Summary）**：在让大模型处理或总结长文档时，先要求模型输出它所接收并理解的原始内容要点，然后再执行分析任务。研究表明，这一步骤能大幅压低模型幻觉率。
- **系统提示词透明化（Transparency & Confidence Rating）**：在系统级 Prompt 中预设约束，要求 AI 在交付最终成果前，明确列出其理解的前提假设；同时要求模型对自己的回复进行“置信度自评”（Confidence Level）。如果模型是在猜测，用户必须立刻知晓；如果模型十分确信，则必须提供明确的参考源与事实依据。
---
### [10:41](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=641s) 建立评估体系（Evals）与量化度量闭环
Ilai 强调了构建系统化评估体系（Evals）的不可替代性。企业不能停留在主观感性地评价“这个模型似乎比那个更好”，而必须引入诸如“黄金数据集”（Golden Datasets）等专业工程方法——构建包含 15 到 20 个高代表性 Prompt 的标准测试集，对照标准答案进行输出校验，从而得出“在 90% 的场景下稳定生效”或“准确率为 60%”等严密的量化指标，用数据定义 AI 质量。
Nurit 进一步从组织治理角度建议，企业在项目实施完成后必须建立持续的追踪问卷，围绕生产力提升、财务投资回报率（Financial ROI）等核心指标，对照实施前定义的基准进行定期测量。这种基于真实反馈的数据闭环，能够直观反映 AI 对各项业务的具体贡献。
---
### [12:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=729s) 坚实数据底座：AI 落地不可妥协的先决条件
当话题转向模型背后的根基时，Nurit 提问为何业界目光往往聚焦于前沿模型，但坚实的数据基础才是企业落地 AI 绝对不容妥协的前提条件。
Ilai 赞同道，“数据是王者”（Data is King），正如《经济学人》（The Economist）杂志在 2017 年所定义的“数据是新时代的石油”一样，这一论断在 AI 时代变得前所未有地关键。他回顾了早年在 Palantir 任职时的亲身经历：当他深入一线询问政企高管最迫切的痛点与挑战时，绝大多数负责人的第一反应往往是“我们需要一个炫酷的可视化仪表盘（Fancy Dashboard）”或是高大上的分析图表，却忽略了底座数据资产的真实质量与系统连通性。数据根基的缺失，正是众多企业技术构想沦为空中楼阁的根源所在。
### [12:54](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=774s) 数据治理与员工技能：从基础数据到智能体检索
构建上层应用和报表系统往往被误认为是核心，但实际上这只占整个工作量的 10% 到 20%。真正占据 80% 至 90% 精力、虽不光鲜却至关重要的脏活累活，是数据转换、数据管道构建以及确保数据的可靠性与安全性。一旦数据底座稳固安全，后续的所有工作都会事半功倍。在人工智能时代尤为如此——企业若想超越通用的基础大模型（Vanilla LLMs），实现真正的业务跃迁，就必须借助微调（Fine-tuning）和检索增强生成（RAG）技术，将公网预训练模型与企业内部私有数据深度结合。
这种结合的关键起点在于全员的技能提升（Upskilling）与培训。数据集成与管道搭建虽由 IT 等技术部门负责，但数据源头来自于每一位普通员工。如果员工日常处理的 Excel 表格、PDF 文档、Word 报告和电子邮件都能保持高度规范、清晰有序，上层的智能体（Agentic layer）或自然语言交互层才能精准理解内部提问并提取有效洞察。因此，数据质量不仅是 IT 部门的基础设施问题，更是全员培训和日常工作素养的核心组成部分。
```mermaid
flowchart TD
accTitle: Data Foundation to Agentic Layer Flow
subgraph DataHygiene["员工日常数据素养 (80-90% 底层工作)"]
A["规范化文档与表格 (Excel / PDF / 邮件)"]
B["数据清洗与安全管道 (IT / Data Pipelines)"]
A --> B
end
subgraph ContextLayer["企业上下文与智能层 (10-20% 上层应用)"]
C["知识图谱与 RAG 检索 / 微调"]
D["智能体与自然语言交互层 (Agentic Layer)"]
B --> C
C --> D
end
```
### [14:33](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=873s) 打破数据孤岛：构建企业“第二大脑”与加速入职培训
企业内部普遍存在数据孤岛问题。在制药等强监管行业中，合规要求所有流程必须详尽建档，导致新员工的入职培训周期极其漫长。如果企业不能将数据治理与员工培训、入职流程有机结合，就会错失巨大的提效机会。
在大语言模型时代，通过构建上下文层（Context Layer）与数据层，企业能够真正实现与非结构化数据的自然对话。新员工入职面对海量的专业术语表和内部流程时，可以直接向系统提问并获得精准解答。通过梳理规范数据并搭建知识图谱，企业便能沉淀出一套可交互的“第二大脑”（Company Brain），大幅缩短入职学习曲线。
- **监管与文档痛点**：高合规行业（如医药）文档体量庞大，新员工入职学习周期极长。
- **上下文层应用**：利用 LLM 将非结构化知识沉淀为企业知识库，支持自然语言即时答疑。
- **核心战略抓手**：打破部门孤岛，将数据资产转化为可交互的企业级“第二大脑”。
### [15:43](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=943s) 变革管理与消除恐惧：ATM 机效应与“超人战衣”赋能
面对“AI 将取代人类”的普遍焦虑，领导者需要用长远的历史视角来进行变革管理与内部沟通。在 Google X 工作期间，这项技术的早期潜力就已显现，但当时大众并不以为意；直到 2022 年 11 月 ChatGPT 问世，舆论又瞬间倒向恐慌。领导者需要引导团队理性看待技术演进，历史证明技术自动化并不等于岗位消亡。
20 世纪 60 年代末至 70 年代初自动柜员机（ATM）刚问世时，专家曾断言银行柜员将被彻底淘汰。然而事实恰恰相反，由于单店运营成本大幅降低，银行开设了更多网点，柜员人数增长速度甚至超过了普通岗位平均水平；与此同时，柜员的工作重心也转向了投资咨询与财富管理等更高价值的领域。
> “AI 应当被视为一件穿在员工身上的‘超人战衣’（Superman suit），它是在赋能员工，让你成为更优秀的自己。”
消除恐惧的关键在于通过技能重塑（Upskilling）赋予员工信心，让大家感受到管理层的深度参与，并在前沿的工作环境中实现自我进化。
| 历史/技术变革 | 初期公众担忧 | 实际业务演变 | 最终岗位与价值结果 |
|:--- |:--- |:--- |:--- |
| **ATM 机引入 (1960-70s)** | 存款、取款自动化将彻底淘汰银行柜员 | 单网点运营成本下降，银行分支机构数量激增 | 柜员总数反增，转型从事投资理财等高价值业务 |
| **生成式 AI 落地 (当前)** | 文本创作与流程自动化将导致大面积失业 | 释放繁琐重复劳动，催生全新智能协同工作流 | 员工转型为“AI 编排者”，作为“超人战衣”增强个人产能 |
### [17:53](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=1073s) 周一早晨行动指南：AI 审计、冠军社区与多方治理体系
针对管理者周一早晨即可启动的实战动作，核心建议是先做全面的“AI 审计”（AI Audit），切忌盲目追求全盘自动化。企业应系统性梳理现有业务流程，精准识别人才技能差距，并制定配套的变革管理与沟通计划。
落地推进应遵循“小步快跑”（Baby steps）策略，切忌操之过急：
- **开展 AI 审计与流程映射**：全面盘点现有业务流程、瓶颈环节与待赋能人员。
- **挖掘内部 AI 冠军（AI Champions）**：选拔业务骨干搭建内部实践社区，以点带面推动传播。
- **场景优先级排序与 ROI 评估**：依据审计结果严格筛选高价值自动化场景，建立结构化的评估指标。
- **建立跨职能治理体系**：AI 落地绝非单一技术专家的独角戏，必须联动财务（Finance）、IT、法务（Legal）、销售（Sales）和人力资源（HR）等多方利益相关者。
AI 转型是技术、人才与业务流程三者的综合体。管理者首要的任务是建立健全的治理框架与 ROI 跟踪机制，从而稳健享受技术带来的长期红利。
## Notable quotes
> [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) 提示词工程（Prompting）绝不等于技能提升 — Nurit Revah
> [05:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=309s) 分发工具与建立能力之间存在本质差距。 — Nurit Revah
> [06:38](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=398s) 大型企业在拥抱技术时必须保持谦逊，尊重现有系统的既有成效。 — Ilai Soloducho
> [12:09](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=729s) 数据是王者（Data is King） — Ilai Soloducho
> [12:54](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=774s) 真正占据 80% 至 90% 精力、虽不光鲜却至关重要的脏活累活，是数据转换、数据管道构建以及确保数据的可靠性与安全性。 — Ilai Soloducho
> [12:54](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=774s) 数据质量不仅是 IT 部门的基础设施问题，更是全员培训和日常工作素养的核心组成部分。 — Ilai Soloducho
> [15:43](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=943s) AI 应当被视为一件穿在员工身上的‘超人战衣’（Superman suit），它是在赋能员工，让你成为更优秀的自己。 — Ilai Soloducho
> [15:43](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=943s) 消除恐惧的关键在于通过技能重塑（Upskilling）赋予员工信心，让大家感受到管理层的深度参与，并在前沿的工作环境中实现自我进化。 — Ilai Soloducho
> [17:53](https://www.youtube.com/watch?v=ozUKqNIBtXE&t=1073s) AI 转型是技术、人才与业务流程三者的综合体。管理者首要的任务是建立健全的治理框架与 ROI 跟踪机制，从而稳健享受技术带来的长期红利。 — Ilai Soloducho
## Mentioned
Ilai Soloducho, Nurit Revah, Strauss Group, EmbraceAI Global, Google X, Google Labs, Palantir, 哈佛大学肯尼迪学院 (Harvard Kennedy School), Anthropic, ChatGPT, 《经济学人》 (The Economist), RAG (检索增强生成), Golden Datasets (黄金数据集)
## Video description
Ilai Soloducho - Chief AI Officer & Chief of Staff to Group CEO, Strauss Group
Nurit Revah - Founder, EmbraceAI Global
Nearly every company has AI tools; far fewer have an AI-fluent workforce — and that gap, not access, is where adoption stalls. In a 20-minute fireside chat, Nurit Revah and Ilai Soloducho share a practical framework for moving teams from first-draft creators to AI orchestrators — the habits that compound, training on real use cases, and measuring quality so trust holds. Leave with an executive playbook for Monday morning.
[Source](https://www.youtube.com/watch?v=ozUKqNIBtXE)
Source quotes
[5:09] Nurit Revah: 提示词工程（Prompting）绝不等于技能提升
[5:09] Nurit Revah: 分发工具与建立能力之间存在本质差距。
[6:38] Ilai Soloducho: 大型企业在拥抱技术时必须保持谦逊，尊重现有系统的既有成效。
[12:09] Ilai Soloducho: 数据是王者（Data is King）
[12:54] Ilai Soloducho: 真正占据 80% 至 90% 精力、虽不光鲜却至关重要的脏活累活，是数据转换、数据管道构建以及确保数据的可靠性与安全性。
[12:54] Ilai Soloducho: 数据质量不仅是 IT 部门的基础设施问题，更是全员培训和日常工作素养的核心组成部分。
[15:43] Ilai Soloducho: AI 应当被视为一件穿在员工身上的‘超人战衣’（Superman suit），它是在赋能员工，让你成为更优秀的自己。
[15:43] Ilai Soloducho: 消除恐惧的关键在于通过技能重塑（Upskilling）赋予员工信心，让大家感受到管理层的深度参与，并在前沿的工作环境中实现自我进化。
[17:53] Ilai Soloducho: AI 转型是技术、人才与业务流程三者的综合体。管理者首要的任务是建立健全的治理框架与 ROI 跟踪机制，从而稳健享受技术带来的长期红利。
Source quotes
[0:04] Nurit Revah: rolling out AI across complex organizations is notoriously difficult
[3:25] Ilai Soloducho: You will not reduce the cost of your company by 20% in a day or you will not increase revenue in in 50% in a week.
[3:25] Ilai Soloducho: Quick Wins
[3:25] Ilai Soloducho: Big Stones
[5:09] Nurit Revah: Prompting accounts for only 25% of the four key competencies required for true AI fluency
[5:09] Nurit Revah: AI first drafter
[5:09] Nurit Revah: AI orchestrator
[8:36] Nurit Revah: confident nonsense
[8:36] Ilai Soloducho: golden datasets
[12:09] Ilai Soloducho: vanilla LLMs
[12:09] Nurit Revah: second brain
[15:50] Ilai Soloducho: Superman suit
Original transcript
[0:04] So, before we begin, we would introduce ourselves.
So, hi everyone. Thanks for joining us today. My
[0:10] name is Eli Soloducho. I'm the chief AI officer
and the chief of staff to group CEO at Strauss
[0:17] Group. Before Strauss, I worked at Google X,
the moonshot factory, where I helped founding
[0:22] a team in the world of AI for coding. And
then my team graduated to Google Labs,
[0:28] focusing on developing innovative AI products
and agent for Google. My background is pretty
[0:34] technical. I was a developer since I was seven. I
had a startup social network at the age of 12 and
[0:40] finished computer science studies at the age of
15. Served in the Israeli intelligence and in
[0:44] Israel Prime Minister's office and then I worked
in Palantir, a big data company, where I work with
[0:50] key stakeholders from governments and Fortune
500 companies to help them make better decisions
[0:55] based on data. And then I did my masters
in Boston at the Harvard Kennedy School,
[1:01] focusing on how to leverage technology in large
institutions and governments. And with me, Nurit.
[1:08] Thank you so much. So, I'm Nurit. The founder and
CEO of Embrace AI Global. I'm an AI AI architect,
[1:18] sorry, and technical leader for over 20 years. I
hold a master in AI and executive MBA. I've been
[1:30] transforming industries for any industry
you can think of from manufacturing, energy,
[1:38] and also highly regulated environments
such as biotech and financial services
[1:46] lately. I've been around enough to know that
this is hard to roll out and I'm here to
[1:52] give you practical and clear the forward. So,
let's start. Uh we have a question for you guys
[2:01] and I would like we would like to map out the
room. So, first question we would like to know
[2:06] is uh raise your hand if uh
your team uses a AI daily.
[2:14] Okay. Wow, that's impressive. Also, keep it up. Um
keep it up if you Sorry, the question was first,
[2:21] my bad. Uh re- raise your hand if you rolled
out AI tools in the organization first.
[2:28] Yeah, let's correct this. And then
keep it up if they're used daily. Wow.
[2:34] Guys, it's a full room. Amazing. And keep it up if
you can measure the quality of what they produce.
[2:42] That's a gap, okay? So, uh just to reflect
back to you, uh the room was full. Uh everybody
[2:50] lifted their uh hands up when we asked the first
question, but the last question was actually um
[2:56] um there was less percentage that uh raised
their hands. So, this tells us the the story of
[3:04] the AI adoption and upskilling. Um Ilya, I would
like to ask you. You've seen AI integration from
[3:11] uh multiple angles. What are the most What
are the most uh common organizational traps or
[3:18] first draft draft mistakes company make when
trying to scale AI tools uh across their teams?
[3:25] Yeah, so look, I think there's a lot of hype, a
lot of buzz around AI, and I think that, you know,
[3:30] the the big question is how we take this amazing
technology and translate it into something that is
[3:37] real, with real ROI. And we already see that 80%
of the organization failed to scale, right? So,
[3:45] not everything that looks great in a video, in a
demo, in a pilot can actually be in production.
[3:51] And this is a huge gap between and we also see
some startups raising millions or billions of
[3:57] dollars or something that looks really good,
but in the end of the day when we have large
[4:01] organization, we really need trust and quality.
And you know, one trap that I see is leaders being
[4:08] very anxious about, "Hey, let's do everything. We
have AI, let's try to do it every place." And you
[4:15] know, my view is that we should focus. We should
I prefer doing two or three things really well
[4:21] than doing like 70, 80 things that but nothing
in a in something that we can really trust.
[4:27] And trust is something that is really important
when we're in scaling AI. And the way I see it is
[4:32] that we should divide it into two. The first one
is quick wins. Things that are not rocket science
[4:37] technology. Things that we can, you know, with a
license of $20, we can, you know, let organization
[4:44] run much faster. And the other side is what we
call big stones. Like things that are for the long
[4:50] term. You will not reduce the cost of your company
by 20% in a day or you will not increase revenue
[4:58] in in 50% in a week, right? So those things are
for the long term. And the way I see it is that we
[5:03] should do both simultaneously. Work on the quick
wins and the big stones. Amazing. I would like
[5:09] to add a couple of things. First is uh what you
choose to to automate like workflows and plus the
[5:17] training. Many organization give their employees
you know, tools such as ChatGPT and I think that's
[5:26] that's actually AI adoption. And there's a big
gap between providing tools to capability. So when
[5:33] you decide to give tools to to your employees, you
need also to provide the the the skills to use it.
[5:42] And with that saying, I mean that prompting is not
uh upskilling, okay? So, prompting is only 25% of
[5:52] actually of the four competencies that employees
need. And that also one of the reason that stalls
[6:01] adoption. Okay? Another thing I would like to
add is that we we are in a an era that we need to
[6:09] do a mind mindset shift from AI first drafter
to AI orchestrator. And that means that
[6:18] we are not really using AI or ChatGPT or any other
AI model to to do a better words document, okay?
[6:28] So, that's something that we need to understand
and we'll talk about what it means to be AI
[6:33] orchestrator in a second. Sure. And you know, we
talked a bit about that before the session, but
[6:38] delegation is choosing what to rebuild and what
to leave alone, right? So, when you coach teams to
[6:44] orchestrate, how do you tell a workflow that needs
reinventing from a legacy process that's already
[6:49] doing a job? Amazing. So, first let's let's define
what is delegation. According to Anthropic 4D's
[6:57] AI fluency framework, delegation is deciding
what to keep under the human in the loop
[7:03] responsibility versus versus what to delegate
to AI. Okay? And when you redesign workflows or
[7:11] or you decide to rebuild, you need to decide what
stays under the human responsibility. That is
[7:19] first part of the process. Plus, needless to say,
you need to list the pain points and understand
[7:26] and prioritize what needs to be rebuilt, right?
So, it's not just oh, I need to automate. I need
[7:32] to put a LinkedIn that we are an AI first company.
As you mentioned about the anxiety and the former
[7:38] to to be an AI company. So, you need to prioritize
and I and I would advise uh here, this is my tip,
[7:46] look at your business value chain and if if it's
possible to focus on that value chain and automate
[7:52] things and processes from those from this area of
focus. I may also add one sentence. Fully agree
[8:00] with what you're saying and you know, when we go
to large organizations, even small organization,
[8:04] we need to respect what's already working, what's
already already doing. We need to be very humble
[8:09] and in my view, we need to say, "Hey, what should
we preserve? What are the things that are working?
[8:14] Don't touch them. What are the things that we
should discard? Things that are not working
[8:19] or we can do it at least 2x better? And where
can we bring innovation? Where can we do this
[8:25] zero to one process to really move the needle?"
Uh [snorts] so, we've been talking a lot about
[8:29] the business value, but how do you teach people
uh to tell a good output from confident nonsense?
[8:36] Well, validation of the outputs needs to be part
of the process. Meaning, uh if we're going on a
[8:43] personal level, when you're prompting a model,
you need to include this in a in a in your prompt
[8:49] structure. And uh to be practical, okay? Uh
I'll give you a couple of tips. First of all,
[8:57] a when you ask a model, let's say you uploaded
a document, let's take the simplest use case,
[9:03] and you wanted to summarize it. So, according to
research, uh asking the model first to present to
[9:09] you what it was ingested, then to perform the
task at hand, then that reduces hallucination.
[9:17] Second thing, my personal uh day-to-day uh
practice is that I I use a system prompt
[9:24] where I told my AI to present me at the beginning
before anything delivered to me what it has
[9:32] uh and what it has been understood, what uh it
will and what is uh the assumption underneath it.
[9:42] And that gives you transparency with AI what
is actually going to be delivered first,
[9:48] second to rate the confidence level of
their of the response. If it's guessing,
[9:54] you need to know that. If it's uh confident,
it's only based on uh if it's grounded in with
[10:01] resources. So, the these are two things I highly
recommend to include in a in a daily basis. Um
[10:08] and going back to skills and upskilling,
uh today people needs to be, you know,
[10:15] upskilled in in a matter of critical thinking.
Because when AI uh uh provide you an answer,
[10:20] it's so confident and research shows that if you
have a confident answer, we tend to believe it
[10:26] as humans. So, we need to do this extra mile to
understand that basically uh we need to judge the
[10:34] outputs and know we're not producing and that's
this is part of the AI curator uh mindset shift.
[10:41] Yeah, I may also add like, do you know,
the importance of evals. I mean, this is so
[10:45] important and there are some methodologies such as
golden data sets when you actually build uh 15 or
[10:51] 20 examples of prompts and then you can actually
build the output of them and then you can quantify
[10:58] the quality and not just say, "Hey, I think this
AI model is better than the other one." or this
[11:02] tool is much better than the the rest. So, you
can actually say, "90% of the time it works. 60%
[11:08] of the time it works." And when you build those
evals, it's really important to really measure
[11:13] uh the quality of the of your AI. I would like to
add something organi- uh organizationally speaking
[11:20] is that, you know, when you implement and adopt
AI, post implementation, you would I would suggest
[11:27] to create a survey that uh includes metrics uh
whether it's productivity, whether it's, uh,
[11:35] the ROI, financial ROI, and measure that, uh,
based on metrics you you defined in the process.
[11:42] And that also will show you where AI
contribution is is, uh, is is good, right? Um,
[11:52] I saw this in different organization and it worked
very well because people filled in the information
[11:57] and then based on data you can actually
evaluate what is the outcome of your,
[12:02] uh, of your implementation. So, I have a question
for you, Eli. Are you ready? Of course, always.
[12:09] Uh, the conversation around AI, um, is
almost always focused on the models, right?
[12:15] Uh, why establishing a rock solid
data which you just talked about, um,
[12:21] is the absolute non-negotiable prerequisite for,
you know, a successful implementation of a AI
[12:28] in the enterprise? Yeah, so, you know, data is
king or the new oil as the Economist magazine
[12:34] defined it in 2017. And I think that in the AI
era it is even more crucial than ever. So, let
[12:40] me share with you a personal story. When I worked
in Palantir, I worked with leaders like you and
[12:45] when I would ask them, "Hey, what are your biggest
problems? What are your biggest challenges?" So,
[12:49] most of them would tell me, "Hey, Eli, we want
a fancy dashboard or we want an analysis or
[12:54] reporting system." But in my view, this is the
easy part. It's maybe 10% or 20% of the work.
[13:02] The 80 90% of the work, even though it's not sexy,
is doing the data transformation, the pipelines,
[13:08] making sure the data is reliable and safe. And
once the data is reliable and safe, that makes the
[13:13] rest of the work the rest of the work much easier.
And that's even more important in the world of AI
[13:18] because if we don't want to limit ourselves only
to vanilla LLMs, right? I know they're powerful,
[13:24] but in the end of the day, if we want to take our
organization to the next level, we will want to do
[13:30] things like fine-tuning. We'll want to use rag,
retrieval augmented generation. Uh so, we will
[13:36] want to leverage the internal data we have in the
organization combined with the powerful LLMs that,
[13:44] you know, in the post-training, they're already
trained on all the data or most of the data in
[13:48] the internet. But in order to really get insights
and and and this is something that starts with
[13:54] upskilling. This is something that starts with
training because obviously we have IT and other
[13:59] other department that are responsible for the
data integration and the pipelines, etc. But if we
[14:05] as users, as employees, we know that the our
Excel sheets and our PDFs and our Word documents
[14:12] and our emails are organized, are clean, then it
will make it much easier for the agentic layer
[14:18] or the natural language layer, if we
want to to ask in natural language
[14:22] prompts about the data in our organization, it
starts with our our learning. So, I think that
[14:28] it's both for the organization, but it's also
very important part of the upskilling part.
[14:33] Amazing. I would like to add like connect
the dots. In my thesis, my master's, I
[14:40] I research about data silos within organizations.
And because back then I was working in
[14:46] pharma industry, um in pharma, in a highly
regulated environment, everything needs to be
[14:52] documented and the onboarding process is forever.
And what Eli is actually saying that when you deal
[14:59] with the data, if you don't think that the data
is not connected to your onboarding process,
[15:04] uh you're you're wrong, okay? So, if you
organize the data, and now with the LLM era,
[15:12] to create a context layer and data
layer, that then you can chat with your
[15:18] unstructured data. So, when someone is onboarding,
for example, and there's a huge terminology list,
[15:26] he can he, she, they can, you know, type and
ask questions about the the data internally.
[15:31] And if you organize the data, that is possible.
You'll have knowledge graphs and everything that,
[15:37] you know, everybody's talking about having a
second brain, company brain. So, this is something
[15:43] I would put in my laser laser focus ahead. So,
the next question for you, and this is one of my
[15:50] favorite topics is communication and change
management. Um, we hear constant prediction that
[15:56] AI will replace us, right? Um, as a leader, how do
you drive an ambitious AI transformation without
[16:04] creating fear? So, [clears throat] when
I was at Google X, you know, I could see
[16:10] behind the scenes how amazing this technology
was. And when I used to tell my friends, "Hey,
[16:15] look, this is going to change the world." So,
everybody said, "Come on, Eli, we heard about
[16:19] AI. Leave us alone. We don't care." But then
something happened. ChatGPT emerged in November
[16:24] 2022. And then everybody said, "Wow, AI. It's will
replace us. It's going to change everything." So,
[16:31] you know, even now, I'm still saying that we
need to take it with proportion. In my view,
[16:36] you know, when we ask if it will replace us, I
will take you back in history to the late '60s,
[16:42] early '70s when when the ATMs, the automatic
teller machines, were introduced to the world. So,
[16:47] you know, people experts would say, "Hey, we don't
need bank tellers anymore because we have ATMs.
[16:52] We don't need bank tellers to deposit checks, to
withdraw cash." But in fact, what happened is that
[16:59] the growth of bank tellers was higher even than
the average employee, right? So, there were more
[17:04] and more bank tellers because first of all, it
was cheaper to build more branches because we had
[17:11] ATMs. And also, bank tellers started doing other
things like investment and wealth management.
[17:17] So, you know, I think that it's really important
to make sure the organization feel empowered.
[17:23] Like a Superman suit that you have that can make
a better version of yourself. Uh and then it will
[17:29] bring them the you know, the feeling that they're
working in a cutting edge environment that they
[17:35] that the leadership is engaged with this. Uh
so, you know, I think there will be new roles
[17:41] and I think that in order to eliminate fear, we
need to upskill them and give them the confidence
[17:46] that this will make them a better version
of themselves. I agree and keep communicate
[17:53] uh to the employees. Yeah, so maybe for to close.
Uh so, you know, what would you recommend one
[18:01] thing to do on Monday morning? Well, for leaders
here, I would say perform an AI audit. Don't jump
[18:09] straight into automating things. You know,
map your organization, your processes,
[18:15] uh the people that needs to be upskilled, and then
build the change management and the communication
[18:22] plan to roll it out. Uh you know, map
the people that can lead this, the
[18:29] AI champions within the organization as
the previous uh talker here before uh said,
[18:36] build communities. And then uh you prioritize what
you chose to, you know, according to the AI audit,
[18:43] you prioritize what you chose to to automate
and then measure the ROI. Do it very
[18:50] uh very organized, very structured, and also
uh in baby steps. Don't jump ahead of your
[18:57] of the game, you know. It it it doesn't it doesn't
Really seriously, it doesn't give you any value.
[19:03] Eventually, as Eli says, uh uh many people and
many companies are failing because of that.
[19:09] Wholly agree. Do you want to add something? Uh
no, I think you covered all. I would just say,
[19:14] you know, you as the leaders of AI, I would
say, think about how to build a system. Okay,
[19:21] a system that can track ROI, that can
do those evals. In the end of the day,
[19:26] it's not one-man show. And I think there are so
many stakeholders, finance, IT, legal, uh sales,
[19:34] HR. There's so many people involved. AI is not
just about technology. It's not just about people.
[19:40] It's not just about process. It's a mix of things
that can really transform our organization work.
[19:46] So, I think, you know, if you ask me about
what to do Monday morning, is make sure you
[19:50] have the right governance in place to make
it happen, and you will enjoy the ride.
