Soren AI Fine-Tunes Gemma Model to Replace Manual Legal Intake Tasks
Ai4
Enterprise AI pilots fail when forced into generic software, whereas fine-tuning open-weight models on proprietary data cuts infrastructure costs and secures domain workflows.
## Background
Published by Ai4 on YouTube on August 20, 2026, this presentation features Kevin Xie, CEO of Soren AI and an MIT computer scientist. The video description details a discussion on why most enterprise AI deployments fail, offering a framework for redesigning workflows, building proprietary AI agents, and leveraging enterprise data to create lasting competitive advantage.
## TL;DR
Kevin Xie explains that 95% of corporate AI pilots fail because organizations force generic off-the-shelf tools onto employees rather than tailoring AI around existing workflows and proprietary data. Using a case study of a midsize law firm, he demonstrates how mapping departmental processes and fine-tuning open-weight models like Google's Gemma can automate bottlenecks securely and cost-effectively. He highlights that while small teams benefit from frontier APIs, large enterprises achieve a stronger ROI and data sovereignty by running specialized in-house models in private clouds. Ultimately, sustained AI success relies on continuous parameter fine-tuning on high-quality internal data and periodically re-running training pipelines as base models evolve.
## Atmosphere & style
The presentation carries an authoritative, analytical, and practical tone, reflecting Xie's background in academic research and startup leadership. The delivery is structured and pragmatic, transitioning seamlessly from high-level enterprise statistics to granular case studies and technical audience Q&A. Listeners experience a clear intellectual progression from systemic industry failures to actionable strategic solutions, leaving an impression of grounded, data-driven expertise.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) 95% of enterprise AI pilots fail to achieve positive ROI despite massive industry investment, while only 5% succeed in building true competitive advantage.
- [01:40](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=100s) The primary cause of AI adoption failure is deployment friction—forcing teams to adapt to standard off-the-shelf software instead of molding AI around existing employee workflows.
- [01:40](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=100s) Public frontier models lack visibility into internal enterprise documents, institutional expertise, and business-specific operational logic.
- [04:45](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=285s) Workflow mapping across all employee roles and levels is essential to identify low-risk, tedious bottlenecks that serve as high-impact starting points for AI automation.
- [07:27](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=447s) Fine-tuning open-weight models like Google's Gemma on high-quality internal work products creates proprietary AI assets that continuously improve on domain-specific data.
- [11:00](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=660s) Open-weight models combined with Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) allow companies to update parameters directly, unlike proprietary closed models.
- [11:00](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=660s) For specialized niches and data-sensitive industries like legal or healthcare, smaller fine-tuned models (~30B parameters) outperform public frontier prompts in cost, accuracy, and data sovereignty.
- [11:00](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=660s) Enterprise data security and confidentiality require hosting fine-tuned models within on-premise infrastructure or the client's Virtual Private Cloud (VPC).
- [17:13](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1033s) Large enterprises benefit economically from hosting open-weight models on fixed infrastructure to eliminate usage-based API fees, whereas smaller organizations should stick to managed frontier APIs to avoid overhead.
- [18:28](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1108s) To handle parameter bias and maintain state-of-the-art capability, companies re-run custom automated fine-tuning pipelines whenever frontier labs release updated open-weight base models.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) Introduction and the AI Advantage Gap
Kevin Xie introduces himself as the founder and CEO of Soren, a San Francisco-based startup that recently went through Y Combinator. He highlights his background as an MIT computer science and mathematics double-major with years of AI research experience published in top journals and labs across MIT, Harvard, and Johns Hopkins. Setting the premise for his talk, he observes that while every enterprise now has access to frontier AI tools like ChatGPT, Claude, Microsoft Copilot, and Gemini, access alone does not guarantee a competitive moat.
He brings up a stark stat from an MIT article published the previous year stating that 95% of enterprise AI pilots fail despite hundreds of billions of dollars in industry-wide investment. He stresses that his focus is on studying the 5% of companies that are winning and achieving positive ROI from their AI investments.
- **Speaker Credentials:** Kevin Xie, Founder/CEO of Soren; double-majored in CS and Math at MIT; conducted research with labs at MIT, Harvard, and Johns Hopkins.
- **Company Profile:** Soren is a Y Combinator-backed startup headquartered in San Francisco.
- **Industry Stagnation vs. Winners:** 95% of corporate AI pilots fail to deliver positive returns despite billions spent; only 5% build true competitive advantage.
[01:40](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=100s) Two Key Observations: Deployment and Proprietary Data
Kevin lays out two foundational observations explaining why most AI deployments fail and how winning companies differentiate themselves. First, the gap in AI adoption is not model capability—modern models are already intelligent enough to discover novel mathematics and advance life sciences—but rather deployment and integration into day-to-day workflows. Most companies force their employees to adapt their existing habits to off-the-shelf software, whereas successful organizations mold AI around their team's established workflows.
The second observation centers on proprietary data. Foundation models like ChatGPT or Claude are trained on public internet data, but they lack visibility into a company’s internal documents, institutional knowledge, and operational expertise. Off-the-shelf models—even frontier releases like Claude 3.5 Sonnet or Opus—do not natively understand how an individual business operates.
```
+-----------------------------------------------------------------+
| TRADITIONAL APPROACH |
| Force Team ---> Adopt Off-the-Shelf Tools ---> High Failure |
+-----------------------------------------------------------------+
│
▼
+-----------------------------------------------------------------+
| SOREN APPROACH |
| Map Workflows ---> Custom AI & Internal Data ---> ROI & Edge |
+-----------------------------------------------------------------+
```
- **Capability vs. Deployment Gap:** Models are smart enough, but forcing teams to use standard, off-the-shelf AI creates friction and rejection.
- **Data Advantage:** Public AI models do not know a company's internal expertise or workflows; businesses must leverage their own proprietary data to build specialized value.
[04:45](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=285s) Case Study: Revamping AI Strategy at a Midsize Law Firm
To ground these observations in a real-world scenario, Kevin shares a case study of a midsize law firm with several hundred attorneys and employees. The firm had spent hundreds of thousands of dollars on AI initiatives, yet after six months, virtually no attorneys, HR staff, or finance team members were using the tools daily. The initiative left both the employees and the founding partners deeply frustrated over burned capital.
Soren stepped in and began by mapping out every workflow across every department—interviewing founding partners, junior attorneys, IT, HR, and finance personnel. From this long-form workflow map, they identified "low-hanging fruit" opportunities that were tedious and time-consuming but low risk. Their top operational bottleneck was client intake, where staff had to manually ask new clients for personal information and re-enter data across six separate software systems.
- **Law Firm Profile:** Midsize practice with a few hundred attorneys and staff.
- **Failed Initial Attempt:** Invested hundreds of thousands of dollars over 6 months with near-zero daily usage.
- **Diagnostic Mapping:** Interviewed staff across all levels and functions to document every operational process end-to-end.
- **First Automation Win:** Replaced manual six-tool data entry during client intake with a custom AI agent.
[07:27](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=447s) Custom Agents and Fine-Tuning Open-Weight Models
Beyond automating client intake, the firm faced a second challenge: privacy and internal domain knowledge. While models like Claude perform well on general legal tasks, they had no awareness of the firm's specific practice areas, historical case filings, or internal policies. Furthermore, the firm refused to upload sensitive client records to third-party public AI providers due to data security concerns.
To solve this, Soren helped the firm establish an in-house model built on Google's open-weight Gemma architecture. They fine-tuned this open-weight model using the firm's highest-quality past work products, such as drafted legal documents and attorney correspondence. This created a secure, proprietary AI model that continuously learned how the firm operated without risking data leakage, delivering higher quality legal outputs and domain-specific agents.
- **Privacy & Knowledge Barriers:** Commercial frontier models lack internal policy context and present data privacy risks for sensitive legal records.
- **Open-Weight Architecture:** Fine-tuned Google's Gemma model using high-quality internal documents and past email correspondence.
- **Closed-Loop Intelligence:** Built an in-house AI asset that continuously improves on internal data, creating a defensible moat against competitors using generic tools.
[09:30](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=570s) Core Takeaways for Enterprise AI
Kevin summarizes his talk with two main recommendations for business leaders aiming to turn AI into a multiplier. First, move away from general-purpose off-the-shelf software and focus on tailored deployments that integrate directly into existing operations—aiming to make "one Kevin do the work of five Kevins." Second, build and own internal intelligence by fine-tuning open-weight models on proprietary data assets currently sitting unused in enterprise databases.
- **Takeaway 1 (Workflow Integration):** Tailor AI around existing employee workflows rather than expecting teams to adapt to generic tools.
- **Takeaway 2 (Owned Intelligence):** Capitalize on proprietary data by training in-house, specialized models to retain full ownership of enterprise knowledge.
[11:00](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=660s) Q&A: Knowledge Injection, Training Techniques, and Privacy
Following the presentation, Kevin fields questions from the audience regarding fine-tuning mechanics, model specialized knowledge, and enterprise data security.
An audience member asks about knowledge injection versus process training during fine-tuning. Kevin clarifies that fine-tuning should not receive raw "garbage data," but rather curated, high-quality examples to prevent overfitting to specific writing styles while teaching the model how internal terminology and processes function.
Another participant asks whether "training" refers to parameter updates via Supervised Fine-Tuning (SFT) / Reinforcement Learning (RL) or prompt engineering. Kevin confirms that Soren performs actual parameter updates (SFT and RL) on open-weight models, noting that closed models like Claude do not support parameter modification.
When asked how to justify fine-tuning when some industry experts argue that frontier model prompting is sufficient, Kevin explains that while generic tasks benefit from public frontier models, specialized niches—such as unique legal practice areas or HIPAA-regulated healthcare workflows—require fine-tuned, smaller parameter models (e.g., 30-billion parameter models) for accuracy and data sovereignty.
Finally, addressing a question on confidentiality during the evaluation phase, Kevin emphasizes that enterprise deployments must occur either on-premise or within the client's private cloud (VPC) under strict legal guardrails to ensure sensitive data is never exposed or re-used outside the client's environment.
- **Fine-Tuning Methods:** Uses Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to update model parameters directly.
- **Prompts vs. Specialized Models:** Frontier models excel at generic prompts, but niche workflows and privacy-sensitive industries (e.g., legal, healthcare) see superior cost and quality performance with specialized ~30B parameter open-weight models.
- **Data Security:** Enterprise fine-tuning and deployment are hosted within the client's Virtual Private Cloud (VPC) or on-premise infrastructure.
[17:13](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1033s) The Economics of Open-Weight Models vs. Closed Frontier APIs
The discussion dives into the rapidly shifting landscape of open-weight models, which update on almost a quarterly basis. When weighing the economics of adopting open-weight models versus relying on proprietary frontier APIs (like Claude or ChatGPT), the organizational scale plays a decisive role. For smaller teams and businesses, attempting to train and maintain custom models creates unnecessary overhead; they are far better off relying directly on cloud-based frontier AI providers.
However, for large enterprises, running in-house open-weight models becomes dramatically more cost-efficient over time. While closed frontier APIs charge on a per-token usage basis—creating steep, continuous operational costs—in-house open-weight deployments decouple expense from usage by anchoring it to fixed infrastructure. The trade-off is the recurring maintenance burden required to continually retrain and update these systems to keep pace with state-of-the-art intelligence.
- **Large Enterprises**: Advantageous to host open-weight models locally to eliminate usage-based API billing, provided they can absorb the maintenance overhead.
- **Smaller Organizations**: Advised to stick with managed cloud frontier models and chatbots due to lower resource overhead and technical complexity.
- **Key Enterprise Use Case**: Specialized enterprise tasks where tailored functionality and economic sustainability align.
[18:28](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1108s) Overcoming Catastrophic Forgetting and Model Recalibration
When adapting an open-weight base model to proprietary corporate data, enterprises face the challenge of model calibration. Open-weight models represent a frozen point in time, pre-trained on billions of parameters. When fresh, client-specific datasets are introduced, the existing pre-trained weights can exert a significant bias over the newly introduced information, making fine-tuning and sensitivity calibration difficult.
To manage this bias without getting bogged down in complex continuous weight adjustment, the most practical approach leverages the massive compute budgets of major lab developers like Meta, Google, and open frontier labs in China. Because these organizations release updated open-weight base models on a frequent basis, enterprises can simply ingest the newest base model, re-run their existing automated training pipelines on top of it, and redeploy. This strategy relies on the frontier labs to handle the heavy lifting of foundational pre-training while allowing companies to keep their domain-specific fine-tuning aligned with state-of-the-art weights.
- **Pre-trained Parameter Bias**: Open-weight models are static snapshots; balancing pre-trained weight bias against newly introduced domain data requires deliberate calibration.
- **Pipeline Strategy**: Rather than attempting continuous online updates on aging models, companies re-apply their custom training and fine-tuning pipelines whenever frontier labs release updated base models.
- **Resource Efficiency**: Outsourcing massive foundational pre-training to well-capitalized labs like Meta or Google allows organizations to focus purely on domain specialization and rapid redeployment.
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) 95% of enterprise AI pilots fail despite hundreds of billions of dollars in industry-wide investment. — Kevin Xie
> [09:30](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=570s) one Kevin do the work of five Kevins. — Kevin Xie
## Mentioned
Kevin Xie, Soren, Soren AI, Y Combinator, MIT, Harvard, Johns Hopkins, ChatGPT, Claude, Microsoft Copilot, Gemini, Claude 3.5 Sonnet, Claude Opus, Google, Gemma, Meta, Ai4
## Video description
Kevin Xie - CEO, Soren AI
Every company now has access to powerful AI models. Yet only a handful are creating lasting competitive advantage. Join Kevin, an MIT computer scientist and CEO of Soren, to explore how leading organizations are redesigning workflows, empowering employees with AI agents, and building systems that compound intelligence across the business.
You will leave with a practical framework for identifying high-value AI opportunities, deploying agents responsibly, and building an organization that wins in the age of AI.
Soren AI 微调 Gemma 模型替代人工法律接案任务
强行套用通用软件会导致企业 AI 试点失败，而在专有数据上微调开放权重模型可削减基础设施成本并保障领域工作流安全。
Kevin Xie 解释称，95% 的企业 AI 试点项目之所以失败，是因为机构强迫员工使用通用的现成工具，而非围绕现有工作流和专有数据量身定制 AI。他以一家中型律师事务所为例，展示了如何通过梳理部门流程并微调 Google 的 Gemma 等开放权重模型，安全且经济高效地实现瓶颈环节的自动化。他指出，虽然小团队能从前沿 API 中获益，但大型企业通过在私有云中运行专用的内部模型，可以实现更高的 ROI 和数据主权。最终，AI 的持续成功取决于基于高质量内部数据进行持续的参数微调，并随着基础模型的演进定期重新运行训练流水线。
尽管行业投入巨大，但 95% 的企业 AI 试点未能实现正向 ROI，仅有 5% 成功建立了真正的竞争优势。 (0:04)
AI 落地失败的主要原因是部署阻力——强迫团队适应标准的现成软件，而非围绕员工现有工作流来定制 AI。 (1:40)
公共前沿模型无法接触到企业内部文档、机构专业经验以及特定业务的操作逻辑。 (1:40)
跨所有员工角色和层级梳理工作流至关重要，这有助于找出低风险、繁琐的瓶颈环节，作为 AI 自动化的高成效切入点。 (4:45)
基于高质量内部工作成果微调 Google 的 Gemma 等开放权重模型，可以打造出能根据领域特定数据持续改进的专有 AI 资产。 (7:27)
Reading summary
According to Soren CEO Kevin Xie, the widespread failure of corporate AI initiatives stems not from deficiencies in model intelligence, but from forcing generic off-the-shelf tools into existing operations. Xie argues that the small fraction of successful organizations achieve positive returns by embedding custom AI directly into established employee workflows and fine-tuning open-weight models on proprietary data.
Workflow Integration Over Generic Software (1:40)
Xie notes that modern foundation models are already intelligent enough to advance mathematics and life sciences, yet an MIT article published the previous year indicated that 95% of enterprise AI pilots fail despite billions invested. In his view, this failure rate occurs because organizations routinely attempt to force staff to adopt off-the-shelf software. Winning companies, representing the remaining 5%, succeed by mapping day-to-day operations and tailoring AI tools around their team's existing habits.
To demonstrate this distinction, Xie cites a midsize law firm of several hundred employees that spent hundreds of thousands of dollars over six months on AI initiatives with virtually zero daily adoption. By interviewing staff across legal, HR, and finance departments, Soren mapped operational workflows and targeted high-friction, low-risk bottlenecks. Specifically, they replaced a manual client intake process—which previously required staff to enter information across six separate software systems—with a custom AI agent that automated the data pipeline.
Fine-Tuning Open-Weight Models on Private Data (7:27)
Foundation models like Claude or ChatGPT are trained on public data and lack context regarding internal enterprise procedures, historical work products, and operational nuances. Furthermore, organizations handling sensitive materials—such as legal case files or HIPAA-regulated health data—frequently face privacy restrictions that preclude sending proprietary data to commercial third-party APIs.
To bridge this gap while preserving data sovereignty, Xie advocates fine-tuning open-weight architectures, such as Google's Gemma or models in the ~30-billion parameter range, within a client's Virtual Private Cloud (VPC) or on-premise infrastructure. Using Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) on curated internal correspondence and drafted documents rather than raw data dumps, companies can update actual model parameters to establish specialized, defensible domain intelligence.
Economics of Open Weights Versus Closed APIs (17:13)
The choice between commercial APIs and in-house open-weight models depends heavily on organizational scale. Closed frontier APIs operate on usage-based, per-token pricing. For high-volume enterprise tasks, these variable fees can accumulate into steep ongoing operational costs. Hosting open-weight models in-house decouples expenditure from usage by shifting costs to fixed infrastructure, creating long-term cost efficiencies for large enterprises.
Conversely, Xie advises smaller organizations and teams to continue utilizing managed cloud frontier APIs. For smaller entities, the fixed infrastructure investments and technical maintenance required to continuously update and host custom models generally outweigh the benefits of localized deployment.
Mitigating Calibration Bias Through Automated Pipelines (18:28)
Adapting an open-weight base model introduces calibration challenges, as frozen pre-trained weights can exert a strong bias against newly ingested enterprise datasets. Managing this delicate balance through continuous weight adjustments on aging snapshots is technically difficult and labor-intensive.
Rather than executing continuous online adjustments, Xie recommends taking advantage of the massive pre-training budgets of major labs such as Meta, Google, or open labs in China. When these organizations release updated base models on roughly quarterly schedules, enterprises can simply re-run their automated fine-tuning pipelines over the newest base weights, keeping internal domain expertise aligned with the broader market's state-of-the-art architectures.
Xie's framework suggests that long-term enterprise differentiation depends on owning internal models fine-tuned on proprietary data rather than relying purely on generic prompts. However, this strategy assumes an organization has the technical capacity to host models within private environments and maintain automated pipelines capable of re-tuning whenever underlying open-weight architectures are upgraded.
精炼阅读摘要
Soren 创始人兼首席执行官 Kevin Xie 指出，尽管当前企业普遍拥有前沿人工智能工具的使用权限，但由于通用模型与实际业务工作流脱节且未触及企业专有数据，高达 95% 的企业级 AI 试点未能产生正向投资回报；企业构建真实竞争壁垒的关键，在于围绕内部流程深度定制工具并基于自有数据微调开源模型。
业务工作流脱节与企业采纳阻碍 (1:40)
Kevin Xie 引用麻省理工学院前一年发布的一项数据说明，尽管全行业投入了数千亿美元，仍有 95% 的企业 AI 试点项目未能带来正向投资回报，仅有 5% 的企业构建了实质的竞争优势。他分析认为，当前行业瓶颈并非底层模型的能力不足——现有模型已具备发现新数学理论和推动生命科学发展的智力水平，真正的障碍在于部署方式与日常工作流严重脱节。大部分失败的企业往往采购现成的通用软件，并强行要求员工改变固有工作习惯去迎合软件，从而引发了强烈的基层阻力。
以一家拥有数百名律师及员工的中型律师事务所为例，该所在投入数十万美元推进 AI 举措六个月后，法务、人力与财务部门的日常实际使用率几乎为零。Soren 通过对各层级员工及创始合伙人进行全方位访谈，绘制出完整的业务流程图，并锁定了关键瓶颈：客户接待环节中，工作人员必须手动向新客户收集个人信息并在六个独立的软件系统中重复录入。通过针对这一低风险但耗时的流程定制 AI 代理，该律所才促成了首个切实落地的自动化成效，证明了围绕既有习惯设计系统而非强推通用工具的重要性。
专有数据主权与开源模型参数微调 (7:27)
以 ChatGPT 或 Claude 为代表的公有前沿模型均基于互联网公开数据训练，天然缺乏对单个企业内部文档、专业知识与业务运作机制的理解。在前述律所案例中，出于对敏感法律记录泄露的顾虑，该机构坚决拒绝将数据上传至第三方商业 AI 供应商。为平衡专业深度与数据安全，技术团队基于谷歌的开源 Gemma 架构建立了本地模型，利用该律所历史积累的高质量法律文书与律师往来邮件等专有资产进行微调，从而在避免数据外泄的前提下，使系统持续内化该机构的专属业务经验。
针对行业中“直接编写前沿模型提示词即已足够”的观点，Kevin Xie 澄清指出，通用模型确实长于普通泛化任务，但在独特的法律业务或受 HIPAA 严格监管的医疗场景中，基于清洗后的高质量内部数据、对约 300 亿参数（30B）的开源模型进行实际参数更新（包含监督微调 SFT 与强化学习 RL），在准确度与经济性上往往优于闭源大模型。同时，此类企业微调与推理部署必须完全运行在客户的本地机房或虚拟私有云（VPC）中，以确保敏感数据不会外流。
模型经济学与架构更新维护策略 (17:13)
在开源权重模型与封闭商业 API 之间进行选型时，组织的业务规模起着决定性作用。对于技术能力与调用量较小的小型团队，自行训练和维护模型的工程开销过于沉重，直接接入云端闭源前沿 API 是更理性的选择。然而对于业务调用庞大的大型企业而言，商业 API 按 Token 计费的模式会导致运营成本随使用量无限攀升；将开源模型部署于固定算力基础设施上，则能将成本与使用规模解耦，显著降低单位推理成本。
针对开源静态快照模型在引入专有数据时所面临的预训练权重偏差难题，Kevin Xie 建议企业不必陷入复杂的持续在线权重微调陷阱，而应建立自动化的微调流水线。鉴于 Meta、谷歌及中国开源实验室几乎按季度高频发布基座模型更新，企业明智的做法是专注于自身高质量专业数据集与训练流水线的建设；每当头部实验室推出更优的底座模型，企业只需重新运行既有管线完成微调即可。这种方式将庞大的基础预训练算力开销外包给了科技巨头，使企业能够以轻量化架构持续保持垂直专业优势。
Kevin Xie 认为企业 AI 竞争的胜负取决于能否将通用算力转化为由专有数据驱动的私有化智能资产，但这一策略的前提是企业必须具备严谨的数据治理能力，并有能力承担模型重新训练与自动化管线的持续运维负担，技术与运维门槛依然是阻碍广泛落地的现实变量。
Detailed notes — zh
## Background
本内容由 YouTube 频道 Ai4 于 2026 年 8 月 20 日发布，主讲人为 Soren AI 创始人兼首席执行官 Kevin Xie。视频旨在探讨在当今所有企业都能轻松接入通用 AI 大模型的背景下，领先企业如何通过重构业务流程、部署定制化 AI Agent 以及积累私有数据资产来构建持久的竞争优势与实际投资回报。
## TL;DR
Kevin Xie 指出当前 95% 的企业 AI 试点项目走向失败，其核心瓶颈不在于模型智商，而在于脱离业务实际的通用软件采购与落后部署方式。他结合 Soren 协助中型律所实现客户录入全自动化并基于 Google Gemma 微调专属模型的实战案例，论证了企业必须将 AI 深度嵌合进原生工作流并掌握专有模型主权。在问答环节中，他进一步深入拆解了垂直领域开源模型微调（SFT/RL）相比单纯提示词工程的不可替代性，并从本地化与 VPC 隐私部署、API 与自建算力成本经济学、以及基座模型快速迭代时的流水线重训策略等维度给出了系统性解答。
## Atmosphere & style
整场演讲呈现出典型硅谷技术创业者务实、冷静且直击痛点的工程与商业思辨风格。Kevin 摒弃了浮躁的技术造神论调，以严谨的数据和亲身落地案例剖析企业 AI 转型的真实困境，语言凝练且极具说服力。后半段的现场问答节奏紧凑、交锋深入，观众连续抛出关于微调本质、前沿模型替代论、隐私隔离与硬件经济学等极具实操深度的尖锐问题，Kevin 则以客观诚恳、毫无防备姿态的技术人视角逐一拆解，营造出一种干货密集、直面工程真相的高密度交流氛围。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) 95% 的企业 AI 试点项目走向失败，核心瓶颈并非基座模型的智商不足，而在于缺乏与具体业务流程深度咬合的交付与部署（Deployment Gap）。
- [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) 自上而下采购通用现成工具并强迫员工改变既有习惯必然引发抵触，企业 AI 的有效路径应当是“量体裁衣”地嵌入原生工作流程。
- [03:16](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=196s) 公有大模型受限于公开语料训练，无法理解企业内部特有的知识资产与运营规范，构建专有内部模型才能形成企业的竞争飞轮与护城河。
- [04:49](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=289s) 推进 AI 落地的关键起手式是跨部门业务流程全面测绘，精准锁定规则明确且高耗时的“低垂果实”（如律所跨 6 套系统的客户录入），避免盲目技术重构。
- [07:28](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=448s) 基于开源权重模型（如 Gemma）在私有环境中精细微调历史高质量业务范本，能在严守数据隐私合规的同时产出远优于公有模型的专业质量。
- [09:34](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=574s) 企业级 AI 的终极价值不在于日常零碎文本的修补起草，而是通过定制化系统与私有智能实现“让单个员工爆发数倍效能”的质变。
- [10:29](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=629s) 模型微调不是盲目塞入所有原始文档，而是精选高质量黄金范例并聚合跨部门样本，以防止模型对特定员工的风格产生过拟合。
- [12:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=724s) 垂直专业模型的打造本质上需要通过监督微调（SFT）或强化学习（RL）实质性更新底层参数权重，提示词工程无法替代参数级的理解构建。
- [13:57](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=837s) 在特定细分利基场景与高度机密的数据环境下，基于 30B 级别开源模型微调的效果和合规性显著优于无法改动参数的闭源前沿模型。
- [16:18](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=978s) 保障企业数据绝对机密的核心在于隔离部署，物理机房本地化部署（On-prem）或依托客户自有企业级 VPC 是兼顾安全与硬件成本的黄金方案。
- [17:15](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1035s) API Token 计费与自建开源存在成本曲线博弈：小体量适合按量调用 API，一旦跨越规模化调用拐点，自建开源垂直模型在财务与数据主权上更具长期优势。
- [18:32](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1112s) 应对开源底模时间冻结与代际跃迁的最佳策略，是沉淀可复用的专有数据清洗与微调流水线，借力顶级实验室算力在新基座上快速重训上线。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) 95% 的 AI 试点为何失败：模型能力过剩与落地差距
Soren AI 创始人兼 CEO Kevin Xie 拥有麻省理工学院（MIT）计算机科学与数学双学位背景，曾在顶尖期刊发表多篇人工智能论文，并与 MIT、哈佛大学及约翰斯·霍普金斯大学的前沿实验室保持紧密合作。在带领 Soren 顺利从 Y Combinator 毕业后，他将目光投向了一个困扰许多企业的核心矛盾：如今几乎每家公司都能接入 ChatGPT、Claude、Microsoft Copilot 或 Gemini 等前沿模型，然而当最顶尖的智能成为唾手可得的基础设施时，究竟什么样的企业才能在 AI 时代胜出？
Kevin 引用了 MIT 去年的一份调研数据指出，业界虽然向 AI 领域注入了数千亿美元，但高达 "95% 的 AI 试点项目都在走向失败"。绝大多数企业不仅没有看到正向的投资回报率（ROI），反而在推行过程中遭遇巨大阻力。他强调，这场分享的重点正是聚焦于那能够胜出的 5% 企业，探讨如何将 AI 转化为切实带来利润与生产力提升的竞争工具。
造成这种脱节的根源并非模型能力不足。现在的基座大模型已经能够推导全新的数学发现、深入生命科学与生物医药的前沿研究，其智力应对绝大多数日常办公业务早已绰绰有余。真正的鸿沟在于**部署（Deployment）**——即如何将 AI 真正整合、适配到企业实际的日常工作流程中。多数企业目前的做法是管理层在网上搜索第三方提供的现成工具（off-the-shelf tools），然后自上而下强迫员工改变既有习惯去迁就外部软件。这种削足适履的做法必然引发抵触，因为员工最清楚自己的日常工作细节，AI 的发展方向绝非强推通用工具，而是应当“量体裁衣”地嵌入现有工作流。
- **模型能力现状**：前沿模型已具备数学创新与生物科研级智能，基础推理能力远超大多数企业日常场景所需。
- **核心落地瓶颈**：问题不在模型的智商（Capability is not the gap），而在与业务流程深度咬合的交付与集成（Deployment gap）。
- **常见误区**：自上而下强推现成通用软件，逼迫团队改变原有工作习惯去适应外部工具。
---
### [03:16](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=196s) 企业专有数据与定制化内部模型的必要性
除了工作流适配问题，通用大模型的另一大局限在于知识鸿沟。像 ChatGPT、Claude 这类公有前沿模型是通过抓取全网公开数据训练而成的，OpenAI 和 Anthropic 无法访问任何企业的内部私有数据。然而，每一家企业赖以生存的核心资产，恰恰是深藏在内部文档、业务流程以及资深员工头脑中的专有知识与行业经验。
通用模型即使进化到最顶尖的级别，也无法真正理解一家具体企业的运营细节与内部规范。因此，企业若想将 AI 构建为自身的“竞争飞轮（Flywheel）”，就不能仅停留在消费通用公有智能的层面，而必须建立专属于自己的内部定制化模型（internal specialized models）。在硅谷的一线观察中，那些真正拉开差距的企业，正是通过结合自身专有数据构筑起独特的壁垒，而落后的企业则往往因通用工具答非所问而使员工倍感挫败。
```mermaid
flowchart TD
accTitle: 通用 AI 与专属定制 AI 的落地路径对比
subgraph General["通用公有 AI（95% 失败路径）"]
G1["公网数据训练的通用大模型"] --> G2["采购现成 SaaS / 通用工具"]
G2 --> G3["强迫员工改变既有流程"]
G3 --> G4["缺乏专有知识 / 员工抵触弃用"]
end
subgraph Custom["深度定制 AI（5% 成功路径）"]
C1["全员业务流程测绘与梳理"] --> C2["锁定高耗时低垂果实"]
C2 --> C3["结合私有数据微调专属模型"]
C3 --> C4["打造嵌入原生工作流的专用 Agent"]
end
```
---
### [04:49](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=289s) 律所实战案例：通过业务流程测绘挖掘低垂果实
为了具象化说明如何实现正向回报，Kevin 分享了 Soren 深度合作的一家中型律师事务所案例。该律所拥有数百名员工及专业律师，此前投入了数十万美元推行 AI 方案。然而在长达半年的时间里，无论是诉讼律师还是人力资源、财务部门，日常几乎无人使用这些工具。巨额投入打了水漂，管理层与合伙人倍感焦虑，团队内部也怨声载道。
Soren 介入后的第一步核心动作是**全面测绘（Map out everything）**。他们与该律所的创始合伙人、资深律师、新入职员工以及 IT、财务、HR 等各职能人员进行了全方位的深度访谈，将律所跨部门的每一条业务工作流细致地梳理成一份详尽的长文档。通过这份全局视图，团队能够精准筛选出最适合 AI 介入的高价值场景，即所谓的“低垂果实（low-hanging fruit）”——那些流程机械、耗费大量人力，但无需从零发明全新底层技术的环节，避免盲目地去“煮干整个大海（boil the entire ocean）”。
律所中最典型的痛点正是**客户录入流程（Client Intake）**。以往每当承接新客户时，工作人员需要通过问询收集客户的个人基本信息，并手动将姓名、出生日期、就职单位等数据重复录入到 6 种不同的内部业务软件中，为此律所甚至专门配置了一整个团队来处理这项繁琐的搬运工作。律所此前曾尝试使用现成 AI 工具辅助录入，但由于无法与特定工作环境深度整合而彻底失败。Soren 通过为其定制专属的自动化 Agent，直接无缝对接日常录入环境，成功实现了该流程的全面自动化。
- **流程梳理方法**：全方位访谈合伙人、一线律师、新员工及中后台职能，彻底摸清跨部门运营细节。
- **场景筛选原则**：优先寻找规则明确、耗时长、重复度高的“低垂果实”，拒绝盲目铺开技术重构。
- **客户录入痛点**：过去需要专属团队跨 6 套独立系统手动同步客户基础档案，现通过定制 Agent 实现了端到端自动化。
---
### [07:28](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=448s) 基于 Gemma 微调专属模型：安全闭环与竞争护城河
解决业务流程自动化的同时，该律所面临的第二个核心挑战是专业知识与数据隐私。法律行业对业务领域的专业度要求极高，虽然 Claude 等通用模型在通用法律文书撰写上表现不俗，但一旦涉及该律所特定的内部政策、办案偏好或历史办案案例，通用模型就会给出错误或不着边际的回答。与此同时，出于严苛的客户保密合规要求，律所坚决不能将高度敏感的案件材料与内部文档直接上传给公有云模型，这导致此前的内部尝试频频受阻。
为打破这一僵局，Soren 协助律所基于 Google 的开源权重模型 **Gemma** 搭建了本地化专属模型基础设施。技术团队精选了律所历史上最高水平的文书草稿、办案记录以及风格规范的往来邮件，对模型进行了针对性的监督微调（Fine-tuning）。
```mermaid
flowchart LR
accTitle: 专有模型训练与闭环构建流程
subgraph DataPrep["专有资产沉淀"]
D1["高质量历史法律文书"]
D2["合规政策与内部办案惯例"]
D3["标准往来邮件与文风范本"]
end
subgraph Training["模型定制化构建"]
T1["Google Gemma 开源权重基座"]
D1 & D2 & D3 --> T2["私有化环境精细微调 (Fine-tuning)"]
T1 --> T2
end
subgraph Value["形成企业护城河"]
T2 --> V1["懂业务运作的专属内部 AI"]
V1 --> V2["文书产出质量持续正向闭环"]
end
```
这种方案让律所拥有了一个既完全在安全可控范围内、又彻底掌握律所内部知识的专属模型。随着微调完成，不仅一线律师获得的法律文书生成质量大幅提升，运营端定制的 Agent 也能精准高效地执行复杂任务。Kevin 指出，这种专属于企业自身的“智能闭环（closing the loop）”是任何外部公有工具都无法赋予的，它构成了企业在同行面前无法被轻易复制的独特竞争壁垒。
---
### [09:34](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=574s) 核心总结：定制化落地与让一个员工发挥五倍效能
回顾前期的实践与思考，Kevin 提炼了企业在 AI 时代真正构建竞争优势的两大核心支柱：
1. **坚持围绕现有工作流进行定制化部署（Custom Deployments）**：公有通用工具绝非企业级 AI 的终局，在市场上到处物色与竞争对手一模一样的现成软件很难带来超额回报。企业应当选择自研定制系统，或与专业团队紧密合作，将 AI 技术无缝“塑形”并嵌合进已有的业务链路中。AI 的价值绝非仅仅停留在“辅助起草几封邮件”，其终极目标是实现质的飞跃——“让一个 Kevin 能够完成五个 Kevin 的工作量（how one Kevin can do the work of five Kevins）”。
2. **训练自有专有模型，将智能主权掌握在自己手中（Own Your Intelligence）**：企业内部蕴藏着巨大的专有数据金矿，通过开源模型进行私有化微调与定制，让 AI 真正懂得自身的业务逻辑与组织知识，才能将数据资产转化为长效增长的技术壁垒。
### [10:29](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=629s) 专有数据价值与问答开场：知识注入与风格拟合的边界
Kevin 结束了他对企业 AI 战略的主旨演讲，在收尾时再次强调：每家企业内部其实都沉睡着世界上独一无二的“专有智能”（proprietary intelligence），它们静静地躺在后端数据库里，根本没有被有效利用。他鼓励与会者回到各自团队去盘点这些高价值信息，而释放其潜力的起点，正是构建由企业自主掌控的基座模型。展示完联络方式后，Kevin 将剩余时间交给了现场观众提问。
第一位观众提出了一个直击微调痛点的关键困惑：“这或许是个有些幼稚的问题，但我常感到困惑。很多人说微调并不是用来塞具体知识的——比如微调不是为了把‘荷马·辛普森（Homer Simpson）说过的每一句名言’都喂进去，微调的结果只是让模型‘说话像荷马·辛普森’。那么你们在针对开源模型做微调时，到底是如何进行‘知识注入’（knowledge injection）的？仅仅是向模型展示处理流程的样例，还是有其他巧妙的方法将内部文档与业务流程注入模型？”
Kevin 解释道，做微调绝不能把所有内部资料一股脑塞进去，因为这必然会导致“垃圾进，垃圾出”（garbage in, garbage out），核心在于提炼出最高质量的黄金范例。提问者随即追问，除了利用合成数据训练流程，对于内部规章制度、政策与专属术语这类硬性知识，Soren 究竟是只展示其使用流程，还是另有门道。Kevin 承认，这正是模型微调中最具挑战性的环节：当从参数量较小的开源权重模型起步时，必须极力避免模型对某种特定写作风格产生过拟合（Overfitting）——例如绝不能让模型过度拟合某位特定员工的邮件行文习惯。对于大型机构而言，解决这一问题的有效路径是聚合来自跨部门、多角色的海量邮件与业务数据，通过多样化的样本帮助模型建立泛化能力，同时仅基于其中最优质的标杆案例进行训练。
---
### [12:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=724s) 参数更新的本质：从提示词工程走向 SFT 与强化学习
随后，另一位现场观众对“训练”（Training）的具体定义提出了更深层的技术疑问。他困惑地询问 Kevin，当 Soren 谈到打造专门化模型（specialized model）时，所谓的训练究竟是指“更新模型权重参数、生成一个新模型”，还是仅仅停留在高级“提示词工程”（prompting techniques）的范畴。
```mermaid
flowchart TD
accTitle: AI 模型落地路径对比
subgraph PromptEng["提示词工程 / Upskilling 路径"]
A["通用前沿模型 (如 Claude / GPT-4)"] --> B["精细化提示词 / Skills 文档"]
B --> C["上下文学习 (In-Context)"]
C --> D["权重冻结，无法修改参数"]
end
subgraph FineTune["深度微调与强化学习路径"]
E["开源基座模型 (如 30B 参数开源模型)"] --> F["专有业务数据标注 (好范例 vs 坏范例)"]
F --> G["SFT (监督微调) / RL (强化学习)"]
G --> H["参数权重更新，形成企业专属垂直模型"]
end
```
Kevin 明确澄清了技术界限：在 Soren 的语境中，“训练”指的是真正的监督微调（SFT）或强化学习（RL），其过程会实质性地修改和更新模型的底层参数权重。Kevin 指出，提示词工程固然重要，但它更多是用于企业内部员工的技能培训（upskilling），帮助那些不熟悉 AI 交互的员工学会如何提问。然而，要真正构建高精度的垂直模型，必须涉及数据标注工作——比如为模型明确标注在某一类特定法律案件或专业术语解释中，究竟什么是“优质输出范例”，什么是“低劣错误范例”。唯有通过调整参数，模型才能在底层深刻理解专业术语的真实内涵。
---
### [13:57](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=837s) 前沿模型 vs 垂直模型：为什么“只靠 Prompt”在垂直领域行不通
提问者紧接着追问，如果自己的业务希望直接使用 Claude，该如何将其改造成专属模型。Kevin 直言不讳地指出：“如果你今天使用的是 Claude，实际上你根本无法对它进行微调。你无法修改 Claude 或任何闭源前沿模型的参数。”相比之下，采用开源权重模型（open-weight models）的最大优势就在于企业拥有自主权，能够通过训练与微调直接改写底层参数。
此时，另一位观众提出了企业界普遍存在的争论：“我们公司的专家常说微调已经过时了，现在的闭源前沿模型能力极其强大，只要写好提示词、给出详尽的技能说明文档（Skills documents），就能搞定一切。对于从未亲手微调过模型、没见过实际效果的人，你该如何说服他们转变观念？”
Kevin 给出了非常务实客观的剖析：
- **通用场景看前沿模型**：对于绝大多数通用场景，确实应该直接依靠闭源前沿模型，毕竟像 Anthropic 和 OpenAI 这种顶级实验室“几乎每两周就在发布新模型”，追逐通用能力毫无意义。
- **垂直利基看微调开源**：微调带来显著超额收益的场景，集中在高度定制化的专门工作流或特定专业查询上。企业并不需要从头训练一个两万亿参数的庞然大物，而是基于一个 300 亿（30B）参数级别的开源模型，喂入极具针对性的高质量专有数据。
- **真实律所案例**：以 Soren 合作的一家律所为例，该所的业务领域高度垂直冷门（niche），导致通用 Claude 模型的生成质量差强人意。律所将资深律师数十年来积累的精深行业经验与优质过往案例输入 30B 模型进行微调，其生成效果立即拉开了与通用前沿模型的差距。
- **敏感合规限制**：在医疗健康、高端法务等领域，存在大量极度敏感的机密文档。企业既无法指望通用公有模型在此类私密数据上训练过，更在合规与心理层面绝不可能接受将这些机密文件上传至公共 AI 接口。
---
### [16:18](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=978s) 数据隐私与部署形态：本地化 On-Prem 与企业级 VPC 护栏
针对律所案例，现场观众立刻抓住了数据机密性这一痛点展开追问：“你提到律所因为担心隐私问题而不愿将内部数据接入 Claude，因而你们为他们定制了专属模型。那么，当他们与你们合作进行模型微调和评测时，你们又是如何确保他们数据的机密性与隐私安全的？”
Kevin 表示，无论是选择与 Soren 合作还是采购其他供应商的服务，核心原则在于绝对的部署隔离：
- **本地化部署（On-prem Deployment）**：如果预算与条件允许，首选将整套训练与推理直接部署在客户的物理机房内，并配套完备的法律与技术护栏，严格限定数据仅能用于该特定模型的训练和推理服务，绝不挪作他用。
- **企业虚拟私有云（VPC）**：考虑到当下的硬件市场状况极其严峻——“如今的算力硬件不仅贵得离谱，而且极难获取”，若纯本地机房成本过高，折中的黄金方案是在客户自有的企业级云环境（如 VPC）中部署。这种方案直接依托客户既有的 IT 业务设施构建闭环，既规避了数据外泄到公共 AI 工具的风险，又降低了自建机房的门槛。
---
### [17:15](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1035s) 成本经济学考量：API Token 计费与开源自建基础设施的曲线博弈
随后，观众抛出了一个现实的财务问题：开源权重模型更新迭代极为频繁，“几乎每个季度都在变”，企业花费高昂代价微调自己的模型，却要面对基座模型的快速更新，这种做法在经济效益（economics）上究竟是否合算？
Kevin 将这笔账拆解为两条成本曲线的博弈：
| 方案模式 | 成本结构特点 | 适用企业规模与业务场景 |
|:--- |:--- |:--- |
| **闭源前沿 API 模式**<br>(如 Claude / OpenAI) | 随 Token 消耗量与员工调用次数呈线性上升；无前期重度基建投入。 | **中小型机构或通用场景**：缺乏算法运维团队，调用规模有限，按 Token 付费更划算。 |
| **自建开源模型微调**<br>(如 30B 开源模型自部署) | 固定基础设施与微调运维成本；边际调用成本极低，无 API 计量账单。 | **大型企业与高频专业场景**：企业规模越大、调用量越庞大，自建基础设施成本优势越显著，且能深度掌控数据与专业任务。 |
Kevin 坦言，对于小型组织而言，自主训练与维护模型的负担确实过于沉重，直接依托通用前沿 API 是最优解；但对于拥有庞大业务量与特定垂直流程的大型企业，只要跨过调用量拐点，自建开源垂直模型不仅能建立防御壁垒，在财务可持续性上也完全能算得过来账。
---
### [18:32](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1112s) 权重偏置与模型迭代：如何应对“冻结时间戳”与持续校准
在全场最后一个问题中，一位观众提出了一个极具技术深度的难题：开源基座模型是凝固在特定时间截面上的产物（a frozen point in time），其 40B 或 80B 的权重内嵌了来自预训练语料的固有先验与偏置。当技术团队带着企业客户的新数据进入微调时，新数据与既有权重之间必然产生张力与偏置冲突，Soren 在实践中是如何校准新数据与模型原有权重之间的敏感度平衡的？
Kevin 赞许这是一个极佳的提问，并坦言这确实是开源微调最棘手的核心难题之一：
- **时间冻结与重新对齐**：正因为模型是在某一历史时刻被冻结的，大部分工程运维开销不可避免地来自于持续重训（continually retraining）以跟上最新的知识。
- **借力顶级实验室的预训练算力**：幸运的是，诸如 Meta、Google 以及部分优秀的中国 AI 实验室，正在以惊人的频率持续推出性能更强的新一代开源权重模型。
- **流水线化部署策略**：面对底模的代际跃迁，当下最经济高效的做法并非试图在老旧冻结权重上做无限度的数据嫁接，而是将企业沉淀下来的“数据清洗与微调流水线”直接迁移并运行在最新的开源基座上，重新跑一遍训练管线后直接上线部署。
“大厂拥有数十亿美元级的充沛算力和海量资源，预训练这件事交给他们就足够了，”Kevin 总结道。随着现场时间耗尽，他在观众的掌声中向全场致谢，结束了整场分享。
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) 95% 的 AI 试点项目都在走向失败 — Kevin Xie
> [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) Deployment — Kevin Xie
> [00:04](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=4s) Capability is not the gap — Kevin Xie
> [04:49](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=289s) Map out everything — Kevin Xie
> [04:49](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=289s) boil the entire ocean — Kevin Xie
> [09:34](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=574s) how one Kevin can do the work of five Kevins — Kevin Xie
> [09:34](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=574s) Own Your Intelligence — Kevin Xie
> [10:29](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=629s) garbage in, garbage out — Kevin Xie
> [13:57](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=837s) 如果你今天使用的是 Claude，实际上你根本无法对它进行微调。你无法修改 Claude 或任何闭源前沿模型的参数。 — Kevin Xie
> [16:18](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=978s) 如今的算力硬件不仅贵得离谱，而且极难获取 — Kevin Xie
> [18:32](https://www.youtube.com/watch?v=GRuyf2dZyQ0&t=1112s) 大厂拥有数十亿美元级的充沛算力和海量资源，预训练这件事交给他们就足够了 — Kevin Xie
## Mentioned
Kevin Xie, Soren AI, MIT, 哈佛大学, 约翰斯·霍普金斯大学, Y Combinator, ChatGPT, Claude, Microsoft Copilot, Gemini, OpenAI, Anthropic, Google, Gemma, Meta
## Video description
Kevin Xie - CEO, Soren AI
Every company now has access to powerful AI models. Yet only a handful are creating lasting competitive advantage. Join Kevin, an MIT computer scientist and CEO of Soren, to explore how leading organizations are redesigning workflows, empowering employees with AI agents, and building systems that compound intelligence across the business.
You will leave with a practical framework for identifying high-value AI opportunities, deploying agents responsibly, and building an organization that wins in the age of AI.
[Source](https://www.youtube.com/watch?v=GRuyf2dZyQ0)
Source quotes
[0:04] Kevin Xie: 95% 的 AI 试点项目都在走向失败
[0:04] Kevin Xie: Deployment
[0:04] Kevin Xie: Capability is not the gap
[4:49] Kevin Xie: Map out everything
[4:49] Kevin Xie: boil the entire ocean
[9:34] Kevin Xie: how one Kevin can do the work of five Kevins
[9:34] Kevin Xie: Own Your Intelligence
[10:29] Kevin Xie: garbage in, garbage out
[13:57] Kevin Xie: 如果你今天使用的是 Claude，实际上你根本无法对它进行微调。你无法修改 Claude 或任何闭源前沿模型的参数。
[16:18] Kevin Xie: 如今的算力硬件不仅贵得离谱，而且极难获取
[18:32] Kevin Xie: 大厂拥有数十亿美元级的充沛算力和海量资源，预训练这件事交给他们就足够了
Source quotes
[0:04] Kevin Xie: 95% of enterprise AI pilots fail despite hundreds of billions of dollars in industry-wide investment.
[9:30] Kevin Xie: one Kevin do the work of five Kevins.
Original transcript
[0:04] So I'm Kevin. Uh I'm the founder and
CEO of Saurin. Saurin. We recently
[0:09] went through Y Cominator. We're a startup
based in San Francisco. Uh before Saurin,
[0:13] I did my undergrad at MIT where I double majored
in computer science and mathematics. Um I've
[0:19] been in AI research for quite a few years now,
published in some of the best journals in the
[0:22] world. Uh worked with some really cool labs over
in MIT at Harvard, Johns Hopkins, and a few more.
[0:29] Um today I want to talk about a topic that has
been on my mind for quite some time. Um nowadays
[0:36] every company has access to AI tools. Uh you
might be using chatbt claude, Microsoft copilot,
[0:44] Gemini. Um but I think this really just begs
the question if everyone has access to frontier
[0:49] intelligence then really who is the winner in this
age of AI and how can your company stand out? So
[0:57] I want to start uh with a quick sorry I want
to start with a quick uh observation here.
[1:06] You might have seen uh an article last year by MIT
which posted that 95% of AI pilots are failing. Uh
[1:16] even though we've invested hundreds
of billions of dollars into AI,
[1:19] how come we are not seeing positive
returns on these tools on these usages? Um,
[1:26] I know there's 95% that are failing, but today I
really want to focus on the 5% that are winning
[1:31] and how you can take this back to your
company and turn AI into a tool that's
[1:35] going to give you positive returns. Okay, so
we're going to start with two observations.
[1:40] The first thing is that these models are way
smart enough for most of the tasks that we do.
[1:45] Um, these models, you've probably seen the
news recently. They're discovering new math,
[1:49] novel math discoveries. they're being applied to
the life sciences, biology. Um, and yet for a lot
[1:56] of the day-to-day tasks that we do, AI is not
really seeing too much real world applications.
[2:02] Um, so capability and the intelligence of
these models really is not the gap. Uh,
[2:08] the real gap is in the deployment of this AI. And
so when I say deployment, this really just means
[2:14] the application and the integration of these AI
tools to your real world day-to-day processes.
[2:20] Um nowadays if you're using AI tools
it probably looks something like this.
[2:25] You have a problem. You go on the internet you're
browsing for various products that other companies
[2:30] are providing. Uh and then you try and pilot
some of these tools and then you're forcing
[2:35] your team to use these tools. Um the issue is
that when you're forcing your team to use AI,
[2:42] you are forcing them to adapt their day-to-day
processes into a tool that someone else built.
[2:47] The future of AI is not enforcing your team
to use off-the-shelf AI tools, but rather
[2:53] to customize it. Your team, your employees, they
know how they do their job on a day-to-day basis.
[2:59] And hence, you should not be forcing them
to morph into some other workflow. Instead,
[3:04] AI should be applied to how they work. That's
the gap number one. These models are smart enough
[3:11] and yet you should not be forcing your
employees to adopt off-the-shelf tools.
[3:16] Okay, the second observation which I think
is very interesting, let's go back a slide.
[3:22] Uh is that your company actually
has a lot of very valuable data.
[3:26] Um if you're familiar with how these models
are trained, uh like a chat GBT or a claude,
[3:31] what they've done is they've scraped basically the
entire internet. Uh so they've scraped the entire
[3:36] internet. They're using that text, all that data
to train the models. But within your own company,
[3:41] you have a lot of proprietary data. You have a
lot of proprietary documents. uh expertise that's
[3:47] your employees know uh OpenAI Anthropic they
unfortunately don't have access to this data
[3:54] but you do and what this means is when you're
using these AI tools and trying to use it for
[4:00] your business workflows it doesn't know your
business right only you do only your employees do
[4:05] and so one really cool thing you can do as well
is develop your own internal specialized models
[4:11] so beyond applying AI to your existing workflows
these models models that are off the shelf today.
[4:17] Even the frontier ones like Opus 5, uh, they're
not going to know how your business operates. So
[4:21] sometimes there's a knowledge gap there. Um, I'm
going to jump into this a little bit later with
[4:26] a case study. So we can make it a little bit more
concrete. Um, but Saurin, you know, I've been in
[4:30] Silicon Valley for over a year now, and I think
what we've really seen is that some companies
[4:35] are doing very well in turning AI into their
competitive edge, a flywheel for how they operate.
[4:40] uh while others are sort of lacking behind their
team doesn't like AI because it just doesn't work
[4:45] for them. And so let's turn this let's just
examine a case study. So recently we worked
[4:50] with a midsize law firm. They have a few hundred
employees, a few hundred attorneys, few employees.
[4:56] Um they invested hundreds of thousands of dollars
into AI. Uh but over the course of 6 months,
[5:03] very few of their attorneys, their
HR, their finance teams, very few
[5:06] of them actually used AI on a day-to-day
basis. and their whole team was very upset
[5:10] and then their founding partners were also very
upset because they burned a few hundred,000.
[5:14] Now thinking to your company I want you
to think about okay so we have our team
[5:18] how many people are actually using AI today and
uh what has the general consensus been. Okay
[5:26] I think the very first thing that you need to do
when you go back and I think this is actually the
[5:31] most important part is to map out everything. What
this means this could be done in a few ways but
[5:37] how we do it is we interview as many employees as
possible from as many different teams as possible.
[5:43] So for this law firm, we talked to their founding
partners, we talked to their attorneys, we talked
[5:47] to people that recently joined the team, we talked
to their IT people, their finance, their HR,
[5:52] and then from there, you get a very comprehensive
overview of what their day-to-day operations
[5:57] looks like across all their teams, all their
functionalities. So basically do the best you
[6:02] can to map out every single workflow that you
have across every single team. And what this is
[6:07] going to allow you to do is you have now a very
long document of everything that you guys do.
[6:12] and you want to pick out the top opportunities.
So, these we like to call are lowhanging fruit.
[6:18] Um, ones that are simple that people
spend a lot of time on, but you're not,
[6:21] you know, trying to boil the entire ocean by
reinventing or inventing some new technology.
[6:26] So, start with those simple workflows. Uh, in this
case, uh, their simple example, their number one
[6:31] thing was their client intake. So, every time they
get a new customer, uh, they have to interview
[6:37] or not interview them, I guess, interview
them, ask about their personal information,
[6:40] and then fill out their name, their birth date,
uh, where they work across six different tools.
[6:45] Uh, and they have an entire team doing this to
handle all of their intake. Uh, and then from
[6:49] there, once you have these specific workflows, you
really want to think about, okay, what is the best
[6:55] solution for applying AI to this workflow? And
this goes back to what I talked about previously
[7:00] with custom AI deployments, off-the-shelf tools.
They tried Microsoft, sorry, not Microsoft,
[7:05] they tried Claude Co-work to try and fill out all
these tools for them. It was terrible. It didn't
[7:09] work for them. And as a result, they resulted back
to using uh their team members to do this. But in
[7:14] this case, because we mapped this out and because
they were interested in using a more deployed
[7:19] and applied and integrated into their day-to-day,
uh they were able to automate this with an agent.
[7:27] Awesome. Now, the next problem that they
had touches on the second point that we see
[7:32] is that their law firm, you know, they have their
own practice area specialized in what they do.
[7:37] Uh Claude, as we know, is actually very good for
legal work. But anytime one of their employees
[7:42] asked about an internal policy or how they did
something in the past, Claude would have no idea.
[7:49] And they weren't comfortable integrating
Claude with their internal documents because
[7:52] it's sensitive data. They don't want it leaking
outside of the organization. Uh and so in this
[7:56] case as a result a lot of times when they tried to
use cloud in the organization uh it actually could
[8:01] not provide the correct output and as a result
there was a lot of frustration and so what we
[8:06] did for them and thinking to your companies uh
how can you leverage the proprietary information
[8:13] the knowledge that's baked inside of your heads
but also your documents your policies you want to
[8:18] be able to train a model and have an AI model that
knows all of this and so what we did is we train
[8:23] helped them set up an openweight model So we built
on top of Google's Gemma models. We set up this
[8:28] openweight model, fine-tuned it on some of their
best examples, best data. So for example, some
[8:34] good documents that they drafted up in the past.
Uh good emails to learn from the writing style.
[8:39] Uh and then after some time, they now have an
in-house model that knows everything about their
[8:43] business. Uh and then over time, you can think of
this as closing the loop. You now have your own
[8:49] your company's personal AI model that knows how
your business operates, but no one else in the
[8:54] world has this, right? You are the only company
in the world that will have this intelligence.
[8:58] And what this means is that it will continue to
increase the quality of your outputs in the future
[9:02] and become your own sort of edge that you have
over other companies that are just using uh the
[9:07] tools that are available outside today. Um and so,
you know, I think this firm has been very happy.
[9:15] um their attorneys are seeing better legal
outputs. Uh on the operation side, they have
[9:20] custom agents that are automating the specific
workflows that they have. And none of this was
[9:24] possible with things I use that they used in the
past because it wasn't really built for them.
[9:30] Okay, awesome. And so I've talked a lot quite
a bit of stuff so far, but I think really the
[9:36] two main takeaways uh is number one, deployments.
The future of AI is not going to be these public
[9:44] general purpose tools. You should not be shopping
around for the same tool that someone else has.
[9:49] The best case and the best way that you're
going to get positive results is either
[9:53] one building in house something custom to you
guys or two working closely with some company,
[9:57] some expert that can tailor the AI around
how your company works. Um, it's totally
[10:02] okay to use offtheshelf tools for some. There's
definitely cases where it is more helpful. Um, but
[10:08] you want to see AI not as just some way to draft
emails, but rather how can one Kevin do the work
[10:15] of five Kevin. In order to do that, you want to
morph and mold AI around your existing processes
[10:20] as best as possible. And then number two,
I think this is gets more advanced is just
[10:25] you want to train your own models. You want to
own your intelligence. Your company has so much
[10:29] proprietary intelligence that no one else in this
world has right now. it's probably just sitting
[10:34] uh in your back end in your databases and
you're not really putting it to good use. Um,
[10:38] but I would go back to your teams and see, okay,
we have all this valuable information. How can we
[10:43] put it to good use? And it starts with a nice
foundational model that your company owns. Um,
[10:50] please feel free to get in touch. This is my
contact info. Um, and I'm very happy to I think we
[10:55] might still have a few minutes left, so I want to
open the floor up to some questions. I know this
[11:00] is a naive question, but a lot of times there's
confusion. So the whole thing about a fine-tune
[11:05] being not, you know, hey, I want to put every
quote by Homer Simpson in, but if you fine-tune,
[11:09] you'll make them respond like Homer Simpson. So,
but how are you adding doing knowledge injection?
[11:13] Are you just showing example processes or do
you also have some cool ways to inject knowledge
[11:18] through your finetunes on the open models? Yeah,
good question. So, um, how we think of this is
[11:24] you obviously don't want to shove everything that
you have into the fine-tuning process because
[11:29] garbage in, garbage out, right? So, you really
just want to take in the best examples. Um,
[11:35] I guess it's a question more so of how can it
not overfit on a certain type of writing style.
[11:40] It seems like you're doing both knowledge
injection and training processes. In the training
[11:45] processes, it makes a lot of sense because I'm
sure you're generating some really cool synthetics
[11:48] there. But as far as injecting knowledge like
internal documents, internal terminology,
[11:54] is that from that just the showing the processes
on how that knowledge is used or do you have other
[11:59] ways to like have him know documents and other
policies in there? Just wondering. Yeah. Yeah,
[12:04] it's a good question. I think that's like the
most challenging part of when you're fine-tuning
[12:08] something, right? um you're starting with a
smaller less parameter based model in these
[12:12] openweight models and you don't want it to
overfit on a certain kind like for example
[12:17] you don't want to overfitit on some a certain
person's emails I think here it just matters
[12:21] that you have a lot of data right for this larger
firm for example take emails for example if we
[12:27] accumulate emails from a bunch of different people
across the firm across different roles uh this
[12:31] helps it generalize a little bit better but also
train off the good examples if that makes sense
[12:35] like when you were uh talking about the training
right So training uh like you know when you are
[12:41] making the specialized model like are you
uh training with like you know updating the
[12:48] weights like making a new model or uh or
like you know you are uh training with
[12:54] the prompting like you know what what you are
actually uh meaning the training like you know
[13:02] did you get my question like you know are you
training the models I mean uh meaning uh updating
[13:07] uh uh parameters or like you know you are just
you know training with the prompting like you know
[13:16] oh I see um so the question is the question um is
the training itself like changing the parameters
[13:21] of the model versus just uh prompting techniques
yeah um so when I mention training here this
[13:29] is mainly uh doing SFT or doing RL on these
models such that it's updating it's updating
[13:35] the parameters of the model Right. Uh I mean
prompting techniques I think is something that
[13:39] it's like sort of upskilling the company. Some
people haven't worked with these AI models. They
[13:44] don't know what the best prompting techniques
are. Um but in these cases like you would have
[13:47] to label the data. For example, what is a good
example versus a bad example for a certain kind
[13:51] of legal case or maybe some kind of terminology.
What does it mean? Does that answer your question?
[13:57] Uh probably uh I need more clarification. Let's
say I want to uh use claude right for my business
[14:05] cases, right? So how can I make that specialized
you know? Mhm. Like by I mean can you can you uh
[14:13] give an example like you know? Yeah. Yeah for
sure. Like with if you're using cloud today you
[14:17] actually cannot fine-tune cloud at all. Right. You
are not able to modify the parameters of cloud or
[14:22] any of these closed source frontier models. Um
if you're using an openweight model the benefit
[14:27] there is that you can directly go in via training
and fine-tuning edit the parameters of the model.
[14:34] if that makes sense. Uh, so I've heard from
experts at the company I'm at um that basically
[14:40] fine-tuning isn't really necessary anymore that
the frontier models are so strong that you can
[14:45] prompt your way into uh whatever it is you want
them to do. Um whether it is through, you know,
[14:51] just really well- definfined skills documents
or, you know, prompting. Um I guess how would you
[15:00] how would you convince them otherwise? And and
I'll be honest, I haven't personally fine-tuned
[15:04] the model myself, so I haven't been able to see
the benefits myself. I I think they might be
[15:09] there, but I don't know. So, how do you how do you
how do you bridge that gap for someone who hasn't
[15:13] seen? Absolutely. I think actually for the most
part, if it's just a general purpose use case,
[15:20] you should just rely on the frontier models.
Enthropic OpenAI, they're releasing new models
[15:25] every other week. It feels like where we really
see companies seeing benefit is on like very
[15:29] specific specialized workflows or specific
kinds of queries. So often, you know, they're
[15:34] not training like a two trillion parameter model.
They're building on like a 30 billion parameter
[15:38] model, but feeding it very specific examples and
only using it on those specialized things that
[15:44] they do. So for example, for this law firm at a
high level, their practice area is a little bit
[15:48] niche, right? As a result, Claude's the quality
of the outputs isn't as great. So they gave it a
[15:54] lot of good examples of things they've done in
the past. Uh they took the knowledge that from
[15:58] the attorneys who've worked in the space for many
many years. I think if it's very specialized or if
[16:02] it's like a very sensitive for example if you work
in healthcare there's a lot of sensitive documents
[16:06] there. I think that's where you can really see
the benefits where claude maybe hasn't trained
[16:10] so well on it and you also wouldn't be comfortable
uploading these documents into a public AI tool.
[16:18] Uh it's like a follow-up question. So you
mentioned uh the law firms worry about the
[16:23] the privacy so they don't use cloud for their
like internal data. So you build a specialized
[16:28] model for them. Uh so how is confidentiality and
privacy is protected when they are working with
[16:35] you guys on the fine-tuning and the evaluation
of the new model. Yeah, I mean whether you work
[16:40] with us or some other vendor, you would want
to make sure they do it an on-prem deployment,
[16:45] right? You want to have the correct legal guard
guardrails in place such that you know that the
[16:50] data is not being used and used for other purposes
but solely for the training and deployment of the
[16:54] model. But you want on-prem deployment. I think if
on-prem is too expensive, I know hardware nowadays
[16:59] is ridiculous. It's hard to get the hardware,
it's very expensive, but doing something in
[17:03] your own cloud environment like a VPC also tends
to work out as well. So basically just deploying
[17:09] inside of their environment, something they've
already uh built their business on top of if that
[17:13] makes sense. Yeah. Um open weight models have been
changing constantly. I mean almost every quarter.
[17:20] What's the economics of um training your own model
and then having to change every every time there's
[17:25] an update? Yeah, it's a good question. I think
uh this open models are very interesting because
[17:30] it really varies depending on the size of the
company, right? For a lot of larger enterprises,
[17:35] it's actually more costefficient to run your own
open weight model in house because you're not
[17:39] paying the API bills of per usage. Correct. Um,
thinking to the future, I think uh the you can
[17:46] if you imagine like a graph with like two kinds of
curves like one being if you gave everyone clawed
[17:52] and then like doing it per usage token based
usage versus just purely the infrastructure. um
[17:58] as you get larger and larger uh it becomes cheaper
for companies to use their own in-house openweight
[18:03] models but then comes the overhead of okay we want
to keep this updated you know latest intelligence
[18:09] keeping it training um but I think if you're a
smaller organization training your own models
[18:13] probably a little bit challenging you should
just rely on cloud or these open AI chatb I say
[18:18] cloud I just mean any kind of frontier AI model
today um but thinking of your larger enterprise
[18:24] you want specialized tasks I think that's where
it can come in and be economically sustainable
[18:28] and like it makes sense to use it. Yeah. Um
with open model distillation and optimization
[18:36] um how are you going with calibrating
the new data that you have acquired from
[18:43] clients um into the existing model weights and
how do you calibrate the sensitiv sensitivity of
[18:49] new data coming in calibrating the model?
Uh that's a good question. I think um
[18:57] I think a lot of it comes down to Could you
actually could you clarify the question one more
[19:01] time please? Open models are a frozen point
in time. They have a pre-trained data set,
[19:06] right? They have four billion parameters,
80 billion, whatever the number is.
[19:10] You walk into company X with X amount of data
that needs to be the model needs to be trained on
[19:18] between the pre-existing weights of the model
and the new data that's being trained. Um the
[19:23] model going to have some type of a bias toward um
pre-trained data from the existing weights of the
[19:29] model uh in relationship to the new uh trained
data set. So as you calibrate the sensitivity of
[19:36] the new data versus the model weights, how are you
guys achieving that? Yeah, absolutely. This is a
[19:41] great question. Uh keep this short. Basically,
I think this is one of the uh most challenging
[19:46] parts. I think goes back to your question a little
bit. Um that it is frozen in time and hence most
[19:51] of the overhead comes to continually retraining
the model, keeping up to date. Um I know the
[19:55] frontier labs today like Meta, Google, some of
the Chinese labs, they are releasing openweight
[19:59] models quite frequently. In those cases, I think
it's actually easiest just to use their latest
[20:03] opening model and then run the same training
pipeline on that latest model and redeploying
[20:06] it. Um, let's say they stop training models one
one day. Uh, then there's another solution there
[20:11] to figure out. But for the most part, I think a
lot of it relies on hey like these open models
[20:15] and pre-training is done by these large billion
they have a lot of comput and resources that
[20:22] are already doing the pre-training. Yeah,
I think we're out of time. Thank you all.
