Ollama Token Volume Jumps 150x as Coding Agents Fuel Open AI Shift

代码智能体驱动开源AI转型，Ollama Token用量暴增150倍 随着自主编程智能体使廉价推理成为取代前沿闭源实验室的主要计算层，开放权重模型将占据高达90%的企业Token消耗量。 Jeffrey Morgan概述了在自主编程智能体以及超高效Flash架构海量Token需求的推动下，企业采用正如何坚决地向开放权重模型倾斜。他解释了Ollama如何作为操作系统层协调碎片化的硬件、推理提供商和开发者工具，从而实现零日无缝模型发布。讨论深入探讨了本地消费级硬件与弹性云集群之间新兴的混合计算分工，以及企业对模型溯源和安全性的评估。Morgan还回顾了Ollama历经两年、在经历失败的Kubernetes转型后最终获得病毒式传播的历程，并详细介绍了从Docker借鉴的关于开发者体验、商业化以及摒弃传统基础设施教条的关键经验。 Ollama已覆盖超过900万名开发者和85%的财富500强企业，主要在自主编程智能体的推动下，Ollama Cloud的总Token用量实现了150倍的激增。 (0:00) 企业Token消耗正迅猛转向开放权重模型，其中AT&T已有40%的总Token流量来自开源架构。 (0:00) 开放权重模型与顶尖闭源前沿实验室之间的能力差距已大幅缩小，预计落后时间已不足三个月。 (28:37) 工作负载正在分化为混合执行模式：本地工作站负责处理文档处理等零边际成本任务，而繁重的编程智能体则被分发至主要由中国实验室主导的云端托管开源模型。 (20:00)

Ollama co-founder Jeffrey Morgan argues that falling costs and rapid architectural parity are driving an enterprise transition toward open-weight AI models, turning raw tokens into an abundant commodity while shifting value toward orchestration, curation, and local execution. Enterprise Drivers and the Shift to Open Weights (0:00) While cost reduction serves as the initial justification for adopting open models, Morgan observes that the ultimate objective for engineering leaders is long-term autonomy over their intelligence stack. Ollama's footprint—spanning 9 million developers and 85% of Fortune 500 companies—shows enterprise workloads actively transitioning away from exclusive proprietary lock-in. For example, reporting highlighted that telecommunications provider AT&T transitioned 40% of its total token consumption to open models, actively evaluating frontier Chinese weights alongside Western releases. Autonomous Agents and the Surge in Token Demand (2:37) The rapid expansion of agentic systems has fundamentally altered token consumption profiles. Workflows like OpenClaw and Hermes Agent execute extended multi-step routines across software engineering, finance, marketing, and customer support, consuming millions of tokens per task. On Ollama Cloud, per-user consumption rose fivefold to over 15 million weekly tokens, while aggregate volume expanded 150 times from the start of the year. This consumption spike is supported by accelerated release cadences that outpace historical foundation lab schedules. Iterations of architectures like DeepSeek Flash have compressed release intervals from six months down to mere weeks. Consequently, developers increasingly run base models directly out of the box for agentic loops rather than investing in custom fine-tunes that risk swift obsolescence. Security, Specialization, and Chinese Model Adoption (6:50) Open-weight models provide practical operational advantages in cybersecurity auditing, vulnerability research, and penetration testing. Unlike closed commercial APIs that frequently issue false-positive refusals during defensive exploitation tests, standard open models like the GLM series process technical security contexts without blocking legitimate inquiries. Morgan notes that while enterprise adoption discussions initially raise governance and provenance concerns—particularly regarding models originating from Chinese laboratories—enterprises generally treat open weights like open-source software dependencies. Once sandboxing and validation controls are established, Fortune 500 companies deploy these models directly, resulting in Chinese open models dominating coding agent workloads on Ollama Cloud. The Emerging Split Between Local and Cloud Workloads (20:00) Enterprise AI deployment is settling into a hybrid architecture governed by latency, cost, and task complexity. Personal workstation hardware, particularly Apple Silicon and emerging desk-side units like the NVIDIA DGX Spark with 128 GB of unified memory, reliably powers local 20B to 120B parameter models at zero marginal cost per token. Companies leverage existing workstation fleets to handle document analysis and rapid iterations locally on premises. Simultaneously, massive reasoning tasks, long-horizon coding iterations, and complex test refactoring are dispatched to cloud-hosted infrastructure. Morgan projects that enterprise volume will eventually split roughly 80% to 90% toward open models for routine execution, while closed frontier models will primarily handle top-tier coordination, similar to senior partners delegating high-volume execution to junior associates. Systems Abstraction as the Defensible Layer (52:45) With token generation becoming heavily commodified, scarcity has relocated up the stack to orchestration, state management, and interface curation. Drawing from past engineering backgrounds at Docker and VMware, Morgan emphasizes that unbundled environments require unified execution layers to manage tokenizer quirks, parameter variations, driver dependencies, and elastic hardware pooling. In contrast to traditional infrastructure playbooks where platform-as-a-service layers were considered vulnerable, positioning higher up the stack in generative AI provides direct defensibility by insulating end developers from fragmented provider engines, variable hardware availability, and volatile GPU procurement markets. The long-term trajectory hinges on managing stateful memory, security boundaries, and reliable multi-agent coordination outside the model weights themselves. Although open architectures trail frontier proprietary models by an estimated latency of under three months, unresolved enterprise questions persist around long-term governance, training provenance auditing, and stabilizing dynamic hardware interconnects.

Ollama联合创始人兼CEO Jeffrey Morgan指出，开源模型正从成本替代方案演变为企业夺回底层智能架构自主权的核心驱动力。随着智能体驱动Token消耗量出现指数级爆发，企业工作负载正迅速转向以低成本开源权重执行批量任务、顶尖闭源模型负责高层规划的混合编排模式。 企业采用开源模型的驱动力演进 (0:00) 开源权重最初凭借极高的成本优势渗透进企业工程团队，但企业长期战略的核心目标是掌握智能技术栈的控制权与定制能力。根据Ollama的统计数据，其平台已触达全球900万名开发者，并进入85%的财富500强企业，美国与德国企业对其云端开源模型的调用尤为密集。例如《The Information》报道显示，电信运营商AT&T已将40%的组织Token流量切换至开源模型，目前以美欧开源权重为主，同时也在积极评测中国顶尖开源架构。 在安全审查领域，开源模型同样展现出结构性优势。商业闭源API因严格的内容安全防护机制，在处理攻防演练或漏洞分析时常出现误判性拒答。相比之下，GLM-4等开源架构能更准确地识别上下文意图，支持合规的白帽审计与防御性分析。针对中国实验室开源模型的顾虑，Morgan指出欧美企业在建立恰当治理机制评估实际风险后，多认为采用此类模型完全可行，且目前未在实际生产中发现任何植入后门或逻辑缺陷的恶意案例。 智能体触发Token消耗与架构编排转向 (2:37) 编程智能体（如Kimi、智谱GLM系列、MiniMax）与自主协同框架（如OpenClaw、Hermes Agent）的落地，推动Token需求爆发式增长。由于跨越财务、营销、销售的多步骤长流程需要反复调用工具、回传上下文并自纠错误，单次任务即可消耗数百万Token。Ollama Cloud单用户周消耗量激增约5倍并超过1500万Token，平台总Token体量增长达150倍。与过去依赖Cursor式微调不同，开发者已开始直接开箱使用具备前沿能力的基座模型。 技术周期加快使得静态微调权重极易在新基座发布后失效，开源生态迭代周期已从以往的6个月压缩至数周。面对海量调用需求，系统架构正从依赖单一全能模型转向由极低成本轻量模型（如DeepSeek Flash）协同工作的编排流水线。Morgan预测，未来企业80%至90%的Token流量将由开源模型承载，但仅占企业模型预算的10%至20%；顶尖闭源模型将转型为高阶决策者，通过智能路由机制与执行批量日常任务的开源模型协同作业。 本地计算与云端算力的混合分工 (20:00) 生成式AI的落地部署正重现云计算时代的混合模式。本地推理在硬件购置后具有边际调用成本为零及超低延迟的优势，适合文档处理等轻量化日常任务；需要复杂推理、自动测试套件与代码重构的高强度编程智能体则主要调度至云端集群。Ollama的调用分布呈现出鲜明的地理特征：本地轻量级模型调用在美、欧、中三地架构间分布均衡，而云端高负载编程智能体流量则高度集中在中国开源模型上。 端侧算力硬件的演进加速了这一混合架构的普及。当前主流配置的Apple Mac笔记本可顺畅运行20B至40B参数模型，部分高配环境可承载120B参数。新发布的NVIDIA DGX Spark提供128GB统一内存与高速互连能力，支持多台串联以在本地运行400B级别大模型。同时，企业原本用于CAD建模的RTX A6000等工程工作站也可无缝改造为本地推理节点，满足亚百毫秒级即时补全响应的需求。 模型解耦与系统层策展价值的重构 (13:17) 随着推理算力池化与Token供给过剩，底层调用的稀缺性正在向状态与协同层转移。Anthropic平台团队提出的三层框架将智能技术栈拆解为：知识连接、智能体协同调度，以及云端安全执行沙箱。由于模型训练周期固定且无法内置动态数据，状态存储、企业数据库连接以及凭证鉴权治理构成了不可被模型权重吞并的独立技术层。 面对算力供应链的价格波动与B200/B300等芯片的采购门槛，Ollama聚合多家独立推理服务商组建弹性算力池，屏蔽硬件配置与区域延迟调度。AI软件开发正打破传统系统工程对绝对确定性与PaaS脆弱性的教条，高层抽象反而更贴近用户核心业务价值。Ollama通过抹平底层供应商的格式错误、API差异并协同开源模型首发部署，在高度碎片化的模型与算力宇宙中提供标准化策展服务。 开源生态凭借极速迭代使开源与闭源前沿模型的性能代差收窄至大约三个月以内。然而，随着未微调基础模型的广泛铺开，企业在动态数据存储、跨智能体协作沙箱及跨国模型合规治理方面仍面临悬而未决的工程挑战。

## Background 本期内容由 Y Combinator（Lightcone 播客频道）于 2026 年 9 月 12 日在 RSS / 网页播客平台发布。节目简介指出，Ollama（YC W21）已被 900 万开发者以及 85% 的《财富》500 强企业采用，其联合创始人兼 CEO Jeffrey Morgan 携团队做客节目，深入探讨开源模型如何重塑 AI 经济学、Ollama Cloud 自年初以来 Token 消耗暴增 150 倍的驱动因素，以及团队历经两年迷茫探索最终打造出全球顶级 AI 开发者工具的创业历程。 ## TL;DR 节目围绕开源大模型对 AI 商业与技术生态的重塑展开深入探讨。Jeffrey Morgan 指出，在 Coding Agent 与协同智能体爆发的推动下，开源模型正以极低成本和开箱即用体验渗透企业工作流，并预测未来企业 80% 到 90% 的 Token 调用将流向开源模型，而闭源前沿模型则作为“高阶合伙人”处理硬核任务。对话还详细拆解了 Ollama 在底层算力硬件、模型提供方与应用脚手架之间充当“操作系统”的生态位，以及桌面级 AI 算力与云端推理结合的混合演进趋势。最后，Jeffrey 坦诚复盘了团队从早年过度设计 Kubernetes 安全产品、在旷野中迷茫探索两年，到极限抓住 LLaMA 契机实现爆发并完成商业化转变的心路历程。 ## Atmosphere & style 整场对话充满硬核系统工程师的务实与松弛感，主持团队与嘉宾之间宛如老友叙旧，谈及当年创业踩坑与在帕洛阿尔托会议室里破釜沉舟的决断时，既有风趣自嘲又有坦诚的共鸣。讨论深入到推理引擎适配、GPU 调度和非确定性软件工程时，语调严谨敏锐、节奏紧凑，呈现出对底层技术变迁的深刻洞见。听感上兼具一线工程实战的粗粝硬朗与战胜迷茫后的开朗释怀，生动展现了技术范式转移前夕极客生态的兴奋与野心。 ## Bulletin - `00:00` Ollama 已覆盖全球 900 万开发者并渗透进 85% 的《财富》500 强企业，2026 年初上线 Ollama Cloud 后全平台 Token 消耗量实现了 150 倍的爆发式增长。 - `02:30` Coding Agent 与协同工作智能体（如 OpenClaw、Hermes）带动单用户每周 Token 消耗增长约 5 倍，且开源模型直接以“开箱即用”形态被广泛调用。 - `16:40` 企业 AI 支出格局正在走向“二八分化”：预计 80% 至 90% 的 Token 调用量流向廉价开源模型，而 80% 至 90% 的预算依然流向前沿闭源实验室攻坚硬核任务。 - `26:15` 闭源前沿实验室与开源模型之间的能力代差已收窄至不到三个月，极端能效使得小模型可组成流水线协同处理绝大部分常规业务。 ## Key takeaways - `00:00` 降低成本是企业采用开源模型的短期最大痛点，但终极北极星指标是实现自主可控与深度业务定制。 - `00:00` 电信巨头 AT&T 已将其 40% 的 Token 消耗迁移至开源模型，全球范围内美国与德国是消耗开源模型 Token 最多的区域。 - `05:00` 开源模型在网络安全攻防与渗透测试中具备独特壁垒，能规避闭源模型过于严苛的安全拦截与合规误判。 - `10:00` Ollama 在生态中扮演类似操作系统的胶水角色，向下调优适配异构硬件与推理引擎，向上打通测试脚手架与应用运行时。 - `13:10` 模型之上的知识层、协调层与执行层正在被拆解（Unbundling），任何带状态（Stateful）的问题都无法被下沉进模型权重内部。 - `16:40` 企业 AI 负载呈现“合伙人-初级律师”分工模式：前沿闭源模型做高阶规划与复杂决策，庞大的开源模型群承接大批量流水线任务。 - `20:00` 本地与云端形成混合架构：20B 到 120B 参数量的模型在个人硬件上顺畅运行并处理轻量工作流，复杂 Coding Agent 仍依托云端算力集群。 - `22:30` DGX Spark 等桌面工作站级设备支持高速网络互连，使个人开发者能够在工位桌面上堆叠串联运行 400B 规模的超大模型。 - `30:00` 行业正迈向“无限 Token”时代，针对 80% 的常规业务场景，通过编排层组合低成本工作马模型比依赖单一全能上帝模型更具经济性。 - `33:14` 模型供应链安全本质上类似于传统开源依赖项风险，企业可通过端到端行为护栏与合规审查来化解潜在后门隐患。 - `40:00` Ollama 团队经历两年 Kubernetes 安全方向的探索与迷茫后，锁定本地模型运行体验并借力 LLaMA 2 发布窗口引爆社区。 - `45:00` 开源模型的无门槛、随处运行与无状态特性，促使其自下而上绕过采购审批迅速渗透至 85% 的《财富》500 强企业内部。 - `52:45` AI 时代推翻了传统 PaaS 的旧有思维定势，向架构上层靠近终端用户反而让中间层初创团队占据更有利的商业位置。 - `55:00` 在模型与算力提供商出现爆发式过剩的当下，消弭底层碎片化并提供开箱即用体验的“整合与筛选（Curation）”成为最具价值的生态位。 ## Detailed walkthrough ### `00:00` 开源模型的崛起与企业采用潮 开源大模型（Open Models）正在重塑整个 AI 领域的经济格局。Ollama 联合创始人兼首席执行官 Jeffrey Morgan（Jeff）指出，成本是开源模型能够直接解决的最大痛点，但对绝大多数企业而言，真正的北极星指标是对 AI 的掌控力与根据自身业务定制模型的能力。成本优势在短期内提供了吸引力，但更深远的意义在于它为企业解锁了针对特定业务场景定制模型的可行性。Ollama 最初是作为在 MacBook、NVIDIA、AMD、Intel 等硬件上本地运行开源模型的工具而诞生，如今已拥有 900 万开发者用户，并渗透进 85% 的《财富》500 强企业。2026 年初上线 Ollama Cloud 后，平台直接接入了海量的 Token 吞吐流，清晰展现出企业正在大规模转向开源模型。 ```mermaid flowchart LR accTitle: 开源模型采用与增长逻辑 accDescr: 展示开源模型如何通过短期降本进而推动企业自控与定制，并在编码与协作 Agent 推动下带来 Token 消耗爆炸。 subgraph Drivers["驱动因素"] A["短期痛点：大幅降低推理成本"] --> B["终极目标：自主可控与深度业务定制"] end subgraph UseCases["核心工作流"] B --> C["Coding Agents (Kimi / GLM / MiniMax)"] B --> D["协同工作 Agent (OpenClaw / Hermes)"] end subgraph Growth["平台爆发"] C --> E["单用户 Token 激增"] D --> E E --> F["Ollama Cloud Token 消耗激增 150 倍"] end ``` 这种向开源模型的迁移在企业端尤为显著，其中包含了美国本土模型以及来自中国实验室的模型。Jeff 观察到，虽然很多顶尖的开源权重模型源自中国，但调用这些模型的企业用户遍布全球，其中美国本土和德国是消耗开源模型 Token 规模最大的两个区域。正如 Jeffrey Morgan 所言：“Cost is by far the largest pain point that open models can jump in and solve. But every business has a vision of getting better control over AI and customizing it for their business. And that's really their North Star.” - **AT&T 的案例落地**：根据《The Information》的报道，电信巨头 AT&T 已经将其 40% 的 Token 消耗迁移至开源模型。目前主要运行美国和欧洲的开源模型，同时也在积极评估来自中国的模型。 - **驱动迁移的核心工作流**：企业开源 Token 消费的最主要推手是 Coding Agents（代码智能体）。 ### `02:30` Coding Agent 与 OpenClaw 带来的 Token 爆炸式增长 开源模型的应用在 2026 年经历了一场工作流驱动的爆发。年初阶段，Kimi、GLM 系列以及 MiniMax 等开源模型的发布，首次让开源阵营拥有了能够媲美闭源前沿实验室、能够稳定支撑 Coding Agent 复杂逻辑的能力，从而带动了第一波人均使用量的激增。随后在 3 月至 4 月间，OpenClaw 项目迅速爆发，紧接着 Hermes Agent 等协同工作（Co-work）类智能体项目也相继兴起。这一转变至关重要，它标志着自动化高强度复杂任务的能力不再局限于软件工程师，而是拓展到了财务、客户支持、市场营销和销售等非研发部门。 普通用户只需将一个棘手复杂的问题交给开源模型，Agent 便会在自主解决问题的过程中不断调用工具、循环验证，单次任务即可吞掉上百万的 Token。这使得 Ollama Cloud 上展示的并非仅是总用户量扩大的宏观图表，而是**每个独立开发者/用户每周平均 Token 消耗量**的指数级拉升。 - **单用户消耗阶梯式跃升**：在 Co-work 类应用爆发前，单个用户每周消耗大约在 1500 万 Token 以下；OpenClaw 等工具出现后，单用户每周 Token 消耗直接翻了 5 倍左右。 - **云端总体吞吐规模**：自年初以来，Ollama Cloud 全平台的整体 Token 使用量呈现出惊人的 150 倍爆发。 - **开箱即用（Out-of-the-box）模式的转变**：在 2024 至 2025 年间，开源大模型通常需要企业自行微调（例如 Cursor 对 DeepSeek 或 Kimi 进行定制）才能上线提供服务；而到了 2026 年初，开源模型直接以“开箱即用”的原生形态被大规模调用开始成为主流。 ### `05:00` 开源与闭源的迭代博弈：安全合规与定制化周期 针对“微调定制是否又进入了新一轮周期，抑或会被下一次模型发布彻底击垮”的行业疑虑，Jeff 指出，模型发布的节奏只增不减。闭源前沿实验室在加速，而开源/开放权重模型的迭代周期缩短得更为剧烈——以 DeepSeek Flash 为例，仅在短短一个夏天内就推出了三次迭代，彻底打破了以往 6 个月一更新的节奏。随着两者的能力差距以肉眼可见的速度抹平，频繁的模型更新确实给微调带来了维护压力，但日益成熟的微调工具链正在让团队能够更敏捷地跟进底层底座的升级。 与此同时，前沿闭源实验室正受到愈发严苛的 AI Safety（人工智能安全）与合规约束，面临对齐与遏制难题，其前进步伐可能会被迫放缓；而开源权重模型则在性能与专业领域一路狂奔。以智谱发布的 GLM-53 为例，其在网络安全（Cybersecurity）能力上的表现极其亮眼。 - **消除中国开源模型的顾虑**：欧美大型企业采用中国开源模型最大的障碍往往是安全和数据治理；但从 Ollama 与美欧客户的实际接触来看，只要能在安全框架上解决合规疑虑，海外企业对全面采用中国顶尖实验室的开源模型持完全开放且兴奋的态度。 - **网络安全攻防测试的刚需壁垒**：许多开发者常问开源模型在哪些场景具有绝对碾压闭源模型的优势，安全渗透测试（Penetration Testing）就是最典型的场景。如果命令闭源的 Claude 去对某项系统进行渗透测试，模型通常会直接触发安全审查机制并拒绝执行；而开发者在 Hugging Face 等社区既能找到完全解除限制的安全研究模型（Abliterated Models），也能直接利用理解上下文更为宽容、不轻易产生误判阻拦的开源基础模型。 ### `08:30` Ollama 的 Day Zero 模型首发工程手册 作为全球开源模型最关键的分发渠道之一，各大模型开发实验室通常会在公开发布前与 Ollama 团队提前接洽，协调联合首发（Day 0 Launch），这不仅让 Ollama 提前接触到最新前沿模型架构，也使其平台在每一次重大模型发布时都能迎来巨幅的流量跃迁。由于模型发布的频率不断攀升，Ollama 内部打磨出了一套标准化的“Day Zero 模型首发生态手册”。 ```mermaid flowchart TD accTitle: Ollama Day Zero 发布准备流程 accDescr: 描述在模型正式发布前 Ollama 完成推理引擎适配、测试框架打通与基准跑分的关键步骤。 A["模型实验室提前同步架构与权重"] --> B["推理引擎兼容性适配与性能调优"] B --> C["配套测试框架与工具链集成 (Harness Integration)"] C --> D["正式发布前针对最终产物跑 Benchmark 验证"] D --> E["Day Zero 同步全球上线并承载流量尖峰"] ``` 要实现完美的零日首发，工程挑战极其艰巨： 1. **推理引擎深度适配**：确保开源社区常用的主流推理引擎能在第一时间支持新模型架构。为了让模型在不同硬件上既跑得飞快、精度无损，同时又完全符合官方参考实现的规范，往往需要提前进行耗时数周的底层优化。 2. **测试框架与应用工具集成（Harness Integration）**：除了模型本身，还必须为开发者配套可落地的使用场景和调用工具。例如 DeepSeek 推出其首个大语言模型层面的多模态模型时（此前仅有一些轻量级 OCR 模型），不仅解锁了海量多模态应用场景，DeepSeek 官方还同步更新了专属 Harness 工具链以支持这一特性，Ollama 必须同步打通这些配套工具。 3. **最终产物的严苛基准验证**：由于每个模型的架构细节各不相同，最关键也是最艰巨的一步是在正式发布前，针对打包好的最终交付物进行严密的 Benchmark 跑分，确保端到端的输出质量与官方宣称完全一致。 ### `10:00` Ollama 在开源生态中的角色：像操作系统一样连接“测试脚手架”、模型与硬件 主持人指出，Ollama 在整个 AI 生态中扮演了一个非常关键的角色：它确立了一套清晰易懂的标准，确保针对每一个新模型、每一次新的 Tool Call（工具调用），开发者都能以一致且最优的方式去调用对应的 Harness（测试/运行脚手架）。Jeffrey Morgan 解释称，Ollama 本质上是将三个核心部分打包在一起： 1. **Harness（运行脚手架）**：无论是现成的 Harness 还是配合模型调用的 SDK。目前开源社区涌现了许多出色的 Harness，比如开源的 Codex Harness 以及在 Ollama 用户中极受欢迎的 OpenCode。 2. **Model（模型可用性与容量）**：确保模型开箱即用且高度稳定。尤其在云端场景下，发布当天（Day Zero）往往是流量激增最猛烈的时候，必须预备充足的推理算力。 3. **Hardware & Providers（硬件与云厂商适配）**：这涉及大量的底层协作。无论是与推理云厂商一起调优模型，还是与 NVIDIA、Apple Silicon 等硬件伙伴深度配合，都必须让模型飞速跑起来——如果一个模型能力极强但运行极慢，开发者的体验就会被彻底毁掉。 这三者往往必须在极短时间内拼装完成。模型通常最多提前几周开放测试，但绝大多数调优与适配工作都被迫挤在“发布前最后 24 小时”以内完成，现场宛如一场救火演习（Fire Drill）。主持人打趣称，Ollama 就像旧时代的操作系统，必须向下与所有底层硬件驱动精密咬合，向上还要在应用层确保各类软件顺畅运行，Ollama 就是串联一切的“胶水（Glue）”。 Jeffrey 对这一比喻深表赞同：“I do think the OS, which is generally a cliche analogy to use, is a good one because you've got the drivers for the hardware and the providers and the inference layer, but you also have the application runtime and making sure that the harness works.”（我认为操作系统这个比喻虽然有些陈词滥调，但非常贴切，因为你既有硬件驱动、云厂商和推理层，又有应用运行时，还得确保脚手架能够运转）。在组合爆炸（Combinatorially large）的技术栈中，Ollama 沉淀出了一套通用的开发、测试与发布流程，让任何模型都能平滑对接任何脚手架。 ```mermaid flowchart TD accTitle: Ollama 的技术集成架构 accDescr: 展示 Ollama 如何在应用层脚手架、模型本身以及底层硬件之间起到类似操作系统的枢纽作用。 subgraph AppLayer["应用与脚手架层 (Harness / App Layer)"] H1["开源脚手架 (OpenCode / Codex Harness)"] H2["开发者 SDK 与应用运行时"] end subgraph OllamaEngine["Ollama 核心集成枢纽 (The OS / Glue)"] O1["统一标准 API 与运行时调度"] O2["Day 0 快速打包与自动化测试"] end subgraph ModelLayer["模型提供层 (Model Layer)"] M1["开源模型权重 (Llama, DeepSeek 等)"] M2["云端弹性推理容量 (Cloud Capacity)"] end subgraph HardwareLayer["硬件与底层加速 (Hardware & Drivers)"] HW1["Apple Silicon (Metal 栈深度调优)"] HW2["NVIDIA GPU / DGX 集群"] end AppLayer <--> OllamaEngine OllamaEngine <--> ModelLayer OllamaEngine <--> HardwareLayer ``` - **底层技术基因**：针对团队如何建立深厚硬核工程功力的问题，Jeffrey 透露团队绝大多数成员并非 AI 研究员出身，而是来自 VMware、Docker 以及网络通信基础设施领域。经典的系统计算问题正在推理领域全面重演。 - **复刻闭源巨头的垂直整合**：英伟达 CEO 黄仁勋曾提出五层技术蛋糕理论（应用、模型、基础设施/推理、芯片、能源）。闭源前沿实验室在发布首日就把所有层打包搞定，而 Ollama 的目标正是帮助开源模型生态在保持开放的同时，完成这种跨层编排。 --- ### `13:10` 模型之上的“隐形层”：拆解还是捆绑？ 当被问及开源模型是否会把传统闭源的五层技术蛋糕进一步切分、催生出独立创业公司时，Jeffrey 给予了极其肯定的回答。他引用了 Anthropic 平台团队关于 Agent 系统架构的演讲，指出在模型之上存在三个极为核心但也面临巨大挑战的独立层级： 1. **Knowledge（知识）**：如何将企业的私有数据与上下文信息无缝喂给模型。 2. **Coordination（协调）**：当用户在 Claude App 或 Codex 中发起任务时，系统会在后台派生（Spin out）大量子代理（Sub-agents），部分运行在云端，部分运行在本地，跨环境调度极为复杂。 3. **Execution（执行/沙箱）**：随着越来越多的 Agent 迁往云端，如何为代码执行提供安全、弹性的 Sandboxes（沙箱），构成了庞大的云计算课题。 从云计算的历史规律来看，早期像 Heroku、Google App Engine 等产品试图将所有组件打包（Bundling），但开发者最终依然投票选择了每一细分领域的“最佳单品”（Best-of-breed）。如今，前沿闭源实验室试图将开发者牢牢锁在受管代理（Managed Agents）、私有上下文和记忆层的围墙花园中；而开源开发者和初创团队则竭力打破这种束缚，进行生态解构（Unbundling）。 ```mermaid flowchart LR accTitle: Anthropic 平台架构的三大核心层 accDescr: 展示模型之上逐渐独立出来的知识层、协调层与执行层。 subgraph AgentPlatform["模型之上的隐形架构层"] direction TB K["1. Knowledge (知识层)<br/>企业数据与上下文注入"] C["2. Coordination (协调层)<br/>云端/本地多子代理调度"] E["3. Execution (执行层)<br/>弹性代码运行沙箱 (Sandboxes)"] end M["开源模型 Token 泛滥"] --> AgentPlatform AgentPlatform --> A["终端开发者应用体验"] ``` 主持人指出，随着 Coding Agent 本身越来越强，软件构建门槛被大幅拉低，“软件工程的高壁垒构筑护城河”的旧逻辑可能正在瓦解。甚至可能出现只要跑一个 Cron 定时任务加一段读取 Markdown 文件的 TypeScript 脚本，再通过 MCP（Model Context Protocol）随时在不同记忆提供商之间热同步，任何形式的生态绑定（Vendor Lock-in）都将不复存在。 针对这一设想，Jeffrey 提出了判断功能归属的核心边界原则：**只要是带状态的（Stateful）问题，就永远不可能被吸收下沉进模型权重内部**。 - **存储与实时数据**：即便模型现在每隔一个月就重新预训练一次，它依然无法拥有最新的私有状态。数据存储是一个极为庞大且独立的软件问题。 - **凭据与安全性**：安全凭据管理（Credentials）以及企业级合规工具是开源模型原生缺乏、但企业落地必须配置的关键层级。 --- ### `16:40` 企业级 AI 支出的终局：开源占 Token 绝大多数，闭源吃预算大头 针对企业在闭源前沿模型与开源模型之间的预算分配趋势，主持人观察到两者的支出目前都在高速增长，好奇未来是否会维持五五开的稳态。Jeffrey Morgan 给出了一个鲜明而笃定的预测： > “The super majority of tokens, and this is our take, it will be open models within a business, call it 80, 90%. That doesn't mean 80, 90% of the budget will go to open models.” > —— Jeffrey Morgan Jeffrey 解释道，开源社区正在协同以惊人的速度将 Token 成本打下来。未来的典型格局是： - **Token 吞吐量**：企业内部 80% 至 90% 的 Token 调用都将流向开源模型。充沛且廉价的 Token 供应彻底改变了团队心智，管理层不再需要算计着扣减员工的 Token 额度，而是全面放开使用。 - **财务预算分配**：企业可能只需将 10% 到 20% 的实际预算支付给开源模型计算，剩下的绝大部分预算依然流向聚集了顶尖 AI 研究员的前沿闭源实验室，专门用来攻克最硬核、最复杂的任务。 ```mermaid flowchart TD accTitle: 企业 AI 负载的合伙人-律师模型 accDescr: 展示前沿闭源大模型扮演合伙人角色，开源模型扮演普通律师团队分发任务的架构。 U["企业业务与复杂任务请求"] --> R["智能路由 / 前沿闭源模型<br/>(扮演高阶合伙人 Partner)"] subgraph Allocation["企业内部负载分配比例"] R -->|"预算 80%-90%<br/>负责高难度决策/复杂调度"| Frontier["前沿实验室闭源模型<br/>(10%-20% Token 调用量)"] R -->|"预算 10%-20%<br/>负责流水线大批量具体任务"| OpenModels["开源模型集群 (Ollama)<br/>(80%-90% Token 调用量)"] end ``` 这种架构模式被主持人形象地比作律所的工作流：前沿模型就像律所的“高级合伙人（Partner）”，负责最难的案件框架制定与战略规划；而数量庞大的开源模型则是底层的“初级律师（Associates）”，高阶合伙人将拆解后的细碎任务分发出去，由庞大的开源模型群完成大规模的流水线作业。诸如 Sakana AI、OpenRouter 等项目的实践，已经验证了混合路由与高低搭配方案的巨大威力。 ### `20:00` 本地模型与云端模型的混合架构演进 从云计算的历史来看，专有软件与开源工具的混合搭配早已是行业常态，正如当年 AWS 既推出了自研的分布式数据库 DynamoDB，客户却往往将其与开源的 PostgreSQL 组合使用。在本地运行与云端托管 AI 模型的选择上，同样呈现出这种混合架构模式。Jeffrey Morgan 指出，本地运行最大的优势在于极低的延迟以及几乎为零的单 token 边际成本——企业只需在前期采购硬件设备，即可将其与云端的大模型协同工作。 随着近几年硬件性能的跃升，现代芯片在运行 20B 到 40B 参数量、甚至高达 120B 参数量的模型时展现出惊人的效率。例如 Qwen 系列（如 Qwen 2.5 32B 等中型模型）的表现已经能够匹敌顶尖闭源模型，而这类模型甚至不需要顶级配置，在市售“倒数第二低内存”配置的 MacBook 上就能顺畅运行。从实际应用场景来看，本地与云端呈现出清晰的分工：代码智能体（Coding Agents）由于需要处理写复杂测试、复杂工程排错等极高难度的长链路逻辑，目前绝大多数仍运行在云端大模型上；而相对简单的文档处理与轻量工作流，则完全可以在本地硬件上极高效率地执行，从而为企业节省海量云端开支。 ```mermaid flowchart TD accTitle: 混合模型执行流 accDescr: 展示用户请求如何在本地轻量模型与云端前沿大模型之间根据任务难度进行分流调度 subgraph User["用户开发环境"] REQ["开发与业务请求"] end subgraph LocalExec["本地硬件执行 (Mac / DGX Spark)"] L_TASK["文档处理 / 语法高亮 / 快速补全"] L_MODEL["20B - 120B 开源模型\n(Qwen / Nemotron / Llama 等)"] end subgraph CloudExec["云端大模型服务 (Ollama Cloud 等)"] C_TASK["复杂代码智能体 / 测试编写 / 深度推理"] C_MODEL["超大参数量前沿大模型\n(云端集群托管)"] end REQ -->|"简单 / 低延迟任务"| L_TASK --> L_MODEL REQ -->|"复杂 / 高难度长链路"| C_TASK --> C_MODEL ``` - **开源力量分布差异**：在 Ollama 观察到的数据中，本地模型的使用呈现中美并驾齐驱的平衡状态（美国、欧洲与中国开源模型平分秋色）；但在云端代码智能体调用中，中国开源模型（如 Qwen 等）占据了压倒性的主流比例。 - **降本逻辑**：企业使用开源模型不仅因为云端推理更便宜，“而是因为你现在可以在本来就要给业务采购的硬件上，几乎免费地运行它们。” --- ### `22:30` 桌面级 AI 算力与 NVIDIA、Apple 的硬件竞赛 硬件层面的飞速迭代正在彻底改变桌面端本地运行大模型的可能性。NVIDIA 近期发布了 Nemotron 系列模型，展示了其在软硬件协同上的独特打法：NVIDIA 并不依赖自建软件专有业务或靠兜售 token 获利，而是通过积极开源模型赋能整个生态，从而拉动其核心硬件的绝对领先地位。目前，NVIDIA 正在研发搭载 GB300 等顶级架构、可直接放置在工位桌面上且运转极安静的桌面工作站级设备。 很多传统行业（如银行、工业企业）早已为工程师的桌面标配了高端图形工作站 GPU（例如早期的 RTX A6000 系列）。而在 GTC 大会上，针对大模型优化的新一代桌面设备 DGX Spark 正式亮相，Ollama 与 Elon Musk 等人成为首批接收该测试设备的团队。DGX Spark 具备 128 GB 统一内存，能够直接在桌面端流畅运行 20B 至 120B 规模的模型。此外，这类设备支持极高带宽的高速网络互连，多台串联即可像“桌面微型数据中心机架”一样堆叠运行超过 400B 参数量的庞大模型。 - **多机互连扩展**：用户可以将多台高性能工作站堆叠串联，在个人桌面上运行 400B 级别的超大模型，体验类似极客玩家串联 Mac mini，但具备完全生产级的稳定性。 - **生态双雄竞争**：Apple 凭借成熟的 MLX 项目让 Mac Studio 和轻量 Mac 展现了卓越的本地 LLM 性能，而 NVIDIA 则有强大的 DGX Spark 堆栈与 CUDA 生态支撑，两者的性能表现都极具竞争力。 - **代码反馈回路的归宿**：Jeffrey Morgan 强调，虽然当前代码智能体因算力需求集中在云端，但未来终将回归本地桌面：“当你的桌上有一台 GB300，你希望获得最快的代码循环，就像运行测试或者在代码编辑器里改动一样快。”类似 GitHub Copilot 时代百毫秒级的即时自动补全体验，将在本地硬件算力提升后完全重现。 --- ### `26:15` 云端 GPU 供应链挑战与低成本 Token 创业建议 构建与扩展如 Ollama Cloud 这样的平台需要极其庞大的 GPU 算力资源，然而当前的 GPU 市场供需波动极大、价格变化极为剧烈。为保障充足算力，Ollama 与多家推理算力供应商建立了深度合作，构建动态 GPU 共享池以应对指数级增长的推理需求。这背后涉及极其复杂的工程调度问题：模型在何处加载、各区域网络延迟优化、请求路由与速度保证等。这也正是 OpenRouter、Ollama 和 OpenCode 等平台为开发者带来的核心价值——个人开发者无需亲自制定 24 个月的算力预算预测，也无需与供应商艰难谈判 B200、B300 采购合同，便能即开即用最新算力。 对于那些处于冷启动阶段、资金匮乏却需要大量消耗 token 的早期 AI 创业者，Jeffrey Morgan 给出的核心建议是瞄准新一代超低成本模型架构（例如类似 DeepSeek Flash 等高性价比模型）。这类模型不仅具备极低的单 token 成本，更关键的是“单任务成本（low cost per task）”极低。 - **走向无限 Token 时代**：正如初代 ChatGPT 让用户无需精打细算 token 数量一样，AI 交互体验最终必将重回“无限 token”的无感状态，而这依赖于底层专为高吞吐量设计的全新模型与架构训练。 - **开源与闭源代差收窄**：Jeffrey Morgan 对开源模型的发展速度做出明确判断：“我们缩小了差距。目前闭源前沿模型和开源模型之间的差距可能已经小于三个月了。”这一代差的迅速抹平，让早期初创企业完全可以依托廉价开源模型打造极具竞争力的产品。 ### `30:00` 极端能效与编排层：小模型如何重构 AI 经济学 Jeffrey Morgan 指出，在模型能力逐渐充沛之后，行业面临的下一个核心议题是“极端能效（Extreme Efficiency）”。例如在云端观察中，高性价比的模型（如 GPT 体系的轻量版本）大幅压低了调用门槛，使得客户团队能够全面铺开采用。开源生态目前正在全面复现这一趋势，其中 DeepSeek 的 Flash 级轻量模型表现尤为强劲。在 Ollama Cloud 的用量细分中，增长最为迅猛的正是 DeepSeek 体系，这很大程度上是由类似 Flash 的轻型模型推动的。这类模型性能足以应付 80% 的日常任务，同时具备极快的推理速度与极其低廉的成本，正在成为承担所有繁重苦差的“主力马模型（workhorse model）”。 ```mermaid flowchart TD accTitle: 模型分层与编排协作流程 accDescr: 展示从上层编排协调到轻量工作模型并行处理，再到顶层大模型攻坚的分工机制。 subgraph Orchestration["协调与编排层 (Orchestration Layer)"] A["复杂业务工作流 / 任务拆解"] end subgraph FlashTier["高效工作模型 (Flash / Workhorse Models)"] direction LR B1["任务 1: 数据清洗"] B2["任务 2: 结构化解析"] B3["任务 3: 代码片段生成"] end subgraph FrontierTier["前沿模型 (Frontier / God-tier Models)"] C["攻坚任务 / 复杂安全对抗 / 兜底推理"] end A -->|"批量并行调度 (极低成本 / 极高速度)"| B1 A -->|"批量并行调度"| B2 A -->|"批量并行调度"| B3 B1 & B2 & B3 -->|"遇到不可解的极难边界"| C ``` 当推理成本低到无需再精打细算 token 和请求数量时，开发范式发生了根本性转变。开发者不必将所有问题都塞给单一的全能大模型，而是可以通过编排层（coordination layer）将多个便宜、迅速的轻量模型串联成流水线，共同完成复杂的系统级任务。这种架构不仅可重复性高、容错能力强，在整体成本上也极具竞争力。关于早前 AI 领域常常争论的“是否存在一个能够搞定一切的单一全能上帝模型（God model）”，播客主持人和 Jeffrey 都认为，目前的现实并非走向唯一的上帝模型，而是走向多模型协同的分层架构。如 Jeffrey Morgan 所断言：“对于大多数客户用例而言，模型达到一个‘足够好’的水平后，他们就可以持续使用这种智能水平。”（“I think for most customer use cases, there's a level at which a model becomes good enough and then they can continue using that level of intelligence.”） - 80% 的常规业务用例并不需要最顶尖的前沿模型，开源模型在这些领域已经越过实用门槛。 - 极端的成本下降彻底释放了调用的自由度，促成初创公司、推理服务商及工作流工具深耕编排层架构。 - 尽管通用场景由小模型分流，前沿顶级模型依然在探索未知的硬核能力边界，两者并行演进。 --- ### `33:14` 地缘政治与开源供应链：模型安全与“满洲候选人”风险 随着来自中国团队的模型展现出前沿竞争力（如智谱 AI 的 GLM 系列，以及在网页开发基准上表现抢眼的 Kimi 模型），地缘政治与数据合规问题逐渐浮出水面。Jeffrey 指出，虽然外界媒体高度聚焦模型的研发国别，但企业客户的核心关注点往往更偏向落地细节：模型具体跑在哪里？运行环境是否封闭安全？不过客户群也明确分化为两类：一类只在乎模型托管与运行的基础设施是否合规；另一类则极其在意训练数据来源与血统，因为不同文化与训练语料会影响模型的语气、对齐风格及合规审查。 对于电网调度、公共设施等关键任务（例如芬兰有机构使用 Llama 检测电网电力激增以保障持续供电），模型的来源与透明度便成了核心诉求。NVIDIA 的 Nemotron 等模型允许用户自查和审计训练数据，因而广受欢迎。针对主持人抛出的敏感问题——“如何防止非本土模型内部潜藏后门，变成暗度陈仓的‘满洲候选人（Manchurian candidate）’”，Jeffrey 回应称，至今并未在业内听说过实质性的此类案例。 大型跨国企业与财富 500 强的 IT 安全团队对供应链投毒并不陌生。Jeffrey 解释道：“如果你想想一个普通的应用程序拥有成千上万个依赖项，这并不是一个新问题；任何一个依赖项出问题都会导致整个应用程序面临重大安全风险。”（“This isn't a new problem. And all it takes is one dependency for there to be a major security issue in the entire application.”）开源软件的依赖项投毒攻击已经存在了几十年，模型虽然内部权重复杂且带有一定的黑盒属性，但其推理流程是确定性的；企业只需建立起端到端的安全审查和行为护栏，这一风险在实际落地中是可以被工程手段有效化解的。 --- ### `36:12` Ollama 的起源故事：从 Kubernetes 迷局到开启开源模型狂潮 谈到 Ollama 宛如“坐在巨大油田之上”的成功现状，Jeffrey 回顾了漫长而曲折的创业历程。他和联合创始人此前在 Docker 共同打造过 Docker Desktop，积累了世界一流的开发者体验（DX）构建功底。然而在 2021 年入选 Y Combinator（W21 批次）时，团队申请的项目完全不同——当时他们试图在 Kubernetes 生态中复制成功，口号是做一款类似“面向 Kubernetes 的 Kitematic”或“Kubernetes 版 Docker Desktop”。 ```mermaid flowchart LR accTitle: Ollama 创业转型与演进历程 accDescr: 展现团队从 YC 申请、早期 Kubernetes 探索、完成 A 轮融资到最终锁定本地模型体验并推出 Ollama 的时间线。 YCW21["2021 年 (YC W21)<br/>项目：Kitematic / Docker Desktop for K8s"] Search["2021 - 2022 年<br/>过度设计、解决 SSO 问题<br/>持续寻找真正热爱的痛点"] SeriesA["2022 年底 (Benchmark A 轮)<br/>DALL-E 时代 / ChatGPT 前夕<br/>基于团队理念完成融资 (Pre-pivot)"] Launch["2023 年 7 月<br/>Llama 掀起开源模型浪潮<br/>正式发布 Ollama"] YCW21 --> Search Search --> SeriesA SeriesA --> Launch ``` 作为二次创业者，他们起初不自觉地陷入了对概念的过度设计中，甚至花精力去攻关像“Kubernetes 单点登录（SSO）”这样虽有商业价值却令团队缺乏真正激情的项目。从 2021 年参与 YC，到 2023 年 7 月正式推出 Ollama，中间经历了长达两三年的探索与调整。值得注意的是，他们在 2022 年就完成了由 Benchmark 领投的 A 轮融资（当时处于 DALL-E 走红、ChatGPT 尚未问世的阶段）。这次投资是在产品彻底转型（pre-pivot）之前完成的，很大程度上得益于 Benchmark 的合伙人 Peter 过去在 Docker 时期就对该团队能力的认可。 关于 Ollama 名字的由来，外界常常误以为它直接取自 Meta 发布的开源模型 LLaMA。Jeffrey 澄清并非单纯如此：“LLaMA”在社区语境中（例如 Reddit 的 LocalLLaMA 讨论区）本身就逐渐代指开放大语言模型体系，Ollama 的命名更像是“Open”与“LLM”的结合，同时赋予了软件一个亲切的羊驼动物形象。正是这种在两三年迷茫中对开发者体验极致打磨的坚持，让他们在 2023 年开源大模型浪潮爆发的瞬间抓住了契机。正如 Jeffrey 总结团队当年的执念时所言：“那段旅程的核心，其实是在不断寻找一个能够真正让开发者感到惊喜愉悦的客户问题。”（“That journey was one of really in search for a customer problem that could delight a developer.”） ### `40:00` 旷野中的迷茫：从安全工具到 Ollama 的诞生 在回顾 Ollama Cloud 随模型系列变化的 Token 使用量图表时，主持人笑称，如果光看数据曲线，整段故事简直就像是从 2026 年 2 月凭空爆发出来的；但实际上，团队早在 2021 年就已出发。谈及长达数年在“旷野中迷茫”、不断尝试却始终无法起飞的心理感受，Jeffrey Morgan 坦言那段时期非常可怕：“当你一次又一次给开发者或客户打电话，却发现事情始终无法引起共鸣时，那种感觉非常折磨；重点不在于我们有没有上新闻头条或产品是否爆发，而在于我们到底有没有真真切切地替别人解决问题。”他和联合创始人 Michael 此前曾创办过一家公司，创始团队极早就被 Docker 收购。他们的初心与北极星始终是“为开发者解决极其棘手的问题并创造绝佳体验”，但带着一支超过 10 人的团队在长达两年的时间里摸索问题所在，压力巨大。 在这段鲜为人知的转型期内，团队最初做的是针对 Kubernetes 的安全工具，随后转型为针对开发者桌面的安全工具，历经多次未公开的试错。转折点发生在大模型浪潮涌现之时——当他们亲手尝试在本地运行原始 LLaMA 模型时，发现流程极其痛苦、异常繁琐，但一旦跑通，其震撼力又无与伦比。那一瞬间成为了真正的“从 0 到 1”时刻。团队在帕洛阿尔托（Palo Alto）的会议室里围坐在一起，决定破釜沉舟：“如果我们要把一切推倒重来，就当作我们今天刚加入 YC 一样，我们会做什么？”当时摆在他们面前的有两条思路： 1. **统一模型网关**：构建一个类似“LLM 领域的 Segment”的路由网关，无缝访问和托管任何模型。 2. **专注于本地运行系统**：团队由来自 VMware 和 Docker 的系统底层老兵组成，最擅长的就是让复杂软件顺畅运行，因此决定聚焦于解决开源模型在本地的高效运行。 团队反思发现，此前在安全领域打转与他们真正的工程基因并不匹配，企业安全工具的销售与开发者工具截然不同。于是他们决定发挥极强的行动力，给自己定下两周的死磕期限去交付 Ollama 的第一个版本。就在这两周即将结束时，LLaMA 2 震撼发布。团队迅速抓住这一窗口将产品推向市场，两周内取得的用户规模瞬间超越了过去整整两年的总和。 ```mermaid flowchart TD accTitle: Ollama Pivot Journey accDescr: Diagram illustrating Ollama team pivots from Kubernetes security to local LLMs and cloud. subgraph Wilderness["旷野探索期 (2021-2023)"] A["Kubernetes 安全产品"] -->|"未完全契合需求"| B["桌面开发者安全产品"] B -->|"反思团队基因 (Docker/VMware)"| C["清空重来：聚焦开发者痛点"] end subgraph Eureka["顿悟与突破 (2023)"] C -->|"尝试本地运行 LLaMA 极其困难"| D["确立方向：两周极限开发 Ollama"] D -->|"遇上 LLaMA 2 发布窗口"| E["Ollama 正式开源上线"] end subgraph Growth["爆发与扩展 (2023-2026)"] E -->|"Reddit 极客自发狂热传播"| F["突破 10 万 GitHub Stars"] F -->|"免审批本地运行打入企业"| G["渗透 85% 财富 500 强"] G -->|"Coding Agent 爆发驱动大规模消费"| H["推出 Ollama Cloud 开启商业化"] end ``` - **转型历程的曲折性**：团队从 K8s 安全转向桌面安全工具，最终依托对开发者体验和底层系统的积累切入本地模型部署。 - **两周冲刺与历史契机**：团队给自己设定了两周倒计时上线首发版本，恰逢 Meta 发布 LLaMA 2，团队顺势发布，瞬间引爆社区。 - **隐形起跑线**：由于此前频繁调整方向，当 YC 团队（如 Jared 等合伙人）在楼梯间碰到穿着周边 T 恤的 Jeffrey 时，才惊呼原来在 Reddit 上引起狂热讨论的 Ollama 就是他们孵化的项目。 --- ### `45:00` 从 Reddit 极客到财富 500 强：开源模型的跨越与商业化探索 Ollama 上线后以惊人的速度突破了 10 万颗 GitHub Stars，增长速度甚至超越了当年的 Docker 和 Kubernetes。Jeffrey 描述那种感觉是创作者在撞上产品市场契合度（PMF）时的不知所措：“你作为创始人完全处于一种目瞪口呆的状态，因为你甚至无法完全解释这为什么会发生，这就是对 PMF 最真实的诠释。”主持人和 Jeffrey 谈到，这种从 Reddit 上捣鼓自家硬件的极客社区（“Homebrew Computer Club 式的极客”）在短短两年内一路席卷至 85% 的财富 500 强企业的历程，就像是将 PC 时代耗费十年才完成的普及速度压缩到了极致。 这种渗透的关键原因在于开源模型的两大内在特质：**零门槛免费使用**与**随处可运行**。对于大企业的 IT 部门与内部开发者而言，这意味着他们无需经过漫长的合规审查和管理层审批即可立即在本地尝试。加之 LLM 本质上是无状态的（Stateless），它比 MongoDB 等早期开发者数据库向企业端的演进更加顺畅无阻。 关于长期缺乏明确商业路径的疑问，Jeffrey 回忆称，在 2023 年 LLaMA 2 爆发后的头两年里，公司没有任何收入，这与当年 Docker 经历的发展路径如出一辙。但他并不盲目担忧，团队内部对商业化始终抱有两个清晰的假设： 1. **隐私安全驱动的专属产品**：立足于 Ollama 开源版最核心的本地化与隐私特性进行商业包装。 2. **云端算力与大模型协同**：早期开源模型能力不足，无法顺畅支持复杂的工具调用（Tool Calling），复杂的生产任务仍然被闭源模型垄断；团队坚信必须静待开源模型能力追平闭源前沿模型。 随着 2026 年编程智能体（Coding Agent）的大规模爆发，AI 消耗 Token 最大的场景终于与具备高阶能力的开源模型实现了交汇，Ollama Cloud 应运而生并顺利开启商业化变现。Jeffrey 同时强调了在享受开源红利时必须汲取的教训：“拥有爆火开源项目最大的风险之一，就是把你的用户群和客户群仅仅看作互联网上一团模糊的色块（just a blob on the internet）。”团队必须主动走出去面对面倾听真实客户在未来半年的真实业务诉求，而不是沉溺于虚荣指标。 - **GitHub 极速里程碑**：Ollama Stars 增长速度大幅超越 Docker 和 Kubernetes 早期，奠定了全球领先的开源开发者生态地位。 - **自下而上的企业渗透逻辑**：开源模型凭借“无需许可即可启动”的特性，使大企业工程师绕过采购审批直接落地，迅速在 85% 的财富 500 强企业中蔓延。 - **耐心的商业化时间窗口**：团队刻意等待模型能力演进，直到 Coding Agent 等高频强算力场景成熟，才正式推出 Ollama Cloud 承接大规模付费算力需求。 ### `50:00` YC 社区支持与“见识过成功样貌”的肌肉记忆 创业是一条极其孤独的旅程。Jeffrey Morgan 分享道，即便自己拥有极佳的合伙人——Michael（两人不仅是大学同窗、滑铁卢大学时期的室友，之前还一起创办过第一家公司）——创业过程中依然会感到孤独。当时他们是在疫情期间参与 YC 的，每周能与 Jared Friedman 以及其他五组创始人围坐在一起交流，对于缓解这种孤立感起到了至关重要的作用。当疫情散去、他们正式搬到湾区后，YC 庞大的校友网络展现出了惊人的能量。因为在滑铁卢大学就有许多在疫情前参加过 YC 的校友，他们早就提醒过“聚在一起才是核心价值”。 更关键的价值在于避免重蹈覆辙。在 YC 社区内部，创始人之间对彼此走过的弯路展现出了极高的透明度。Jeffrey Morgan 至今仍与 Docker 的创始人保持密切联系（后者也是 Ollama 的投资人），双方经常讨论上一代基础设施公司所经历的挑战，以确保新一代公司不会踩中致命陷阱。正如主持人所感慨的：“如果你仅仅是避开了其中一个可能扼杀公司的重大失误，那价值就是不可估量的。” 对于年轻创始人而言，曾经在优秀工程团队中建立的“肌肉记忆”有着巨大的倍增效应。主持人指出，很多 18、19 岁的年轻创业者常问该如何起步，其实哪怕先去一家优秀的团队待上三个月，见识过顶尖水准的产品开发标准和 Bug 处理流程，也能深刻理解“什么是好产品”。Jeffrey Morgan 表示，Ollama 团队中吸纳了来自早期 VMware 和 Nicira 的资深工程师，他们为团队注入了一套对工程质量高度自律的价值观。比如面对两年前发布的软件版本，如何确保即使开发者长达两年不升级，它在用户本地依然能够坚如磐石地稳定运行。 - Jeffrey Morgan 谈到创业的孤立感与社区价值：“创业是一次非常孤独的经历。即使你有一个出色的联合创始人……它仍然是孤独的。我认为拥有一群同行……每周与 Jared 以及其他大约五个创始人小组交流，真的能帮助你减少孤独感。”（“is a really lonely experience. Even if you have a great co-founder... it's still lonely. And I think just having a set of peers... just talking to Jared and like five other groups of founders every week really helped you feel less lonely.”） - 资深经验与标准带来的乘数效应：“在见过事物真正运作的完整流程（从最初的状态到明确‘这就是缺陷数据库该有的样子’、‘这是我们彼此要求的基准线’）之后，成功的几率就会成倍增加。” --- ### `52:45` 打破旧规则：从传统 DevOps 到 AI 软件工程的范式转移 虽然过去在 DevOps 和底层架构方面的工程积累至关重要，但进入 AI 时代后，大量来自云原生与基础架构 1.0 的旧经验已经失效。Jeffrey Morgan 指出，在过去“平台即服务”（PaaS，例如早期的 Heroku 或起步阶段的 Docker）的时代，行业普遍认为做“中间层”处于非常脆弱且易受挤压的地位；但在 AI 时代，这种观念被彻底推翻——在架构上往上走（向应用层和开发者体验靠拢）反而更具优势，因为你离终端客户更近。为了打造 Ollama，团队不得不主动打破很多以往引以为傲的工程本能。 另一个巨大的认知转变在于对“非确定性”的包容。在传统的系统工程领域，工程师要求一切都严格按照测试用例精确运行；但大语言模型（LLM）天生就是非确定性的。主持人笑称“这正是特性而非 Bug（That's a feature, not a bug）”，Jeffrey Morgan 对此深表认同。此外，团队在研发支持和云端架构的交付模式上也发生了剧烈改变。过去需要大量人员支持的环节，如今在 AI 辅助下变得极为精简。软件工程正在迈入一个“没有任何单一个体工程师能够完全掌握所有代码具体如何运作”的新范式，从 2000 年代的 Infrastructure 1.0、2010 年代的云计算，再到如今的 AI 浪潮，原有的软件开发法则正在被全盘重写。 - Jeffrey Morgan 阐述在 AI 时代推翻旧有思维定势：“作为处于其他层之上的中间层，初创公司会处于某种弱势地位——这在 AI 世界中完全不成立。事实上，往架构上层走有时甚至更好，因为你距离客户更近。”（“there's this concept that if you're a layer on top of something else, that you're in kind of a vulnerable position as a startup. Which is absolutely not true in the AI world. And in fact, going up the stack can sometimes be even better because you're closer to the customer.”） - 面对 LLM 非确定性的态度转变：“在大模型领域，它们绝不是完美的。在系统工程世界里，你希望一切都完全按照设计好的方式运行，经过充分测试与验证。但大语言模型从定义上就不是这样。”（“these LLMs are never perfect. And like in the systems world, you want everything to be exactly as it's designed to run. It's tested. It's validated. But LLMs by definition are not.”） --- ### `55:00` 开源生态的核心机遇：在模型爆发期提供“聚合与筛选” 主持人和 Jeffrey Morgan 深入探讨了 Ollama 在 AI 生态中所扮演的关键桥梁角色。底层的推理生态系统目前充斥着各种未文档化的参数异常、奇怪的 JSON 格式要求和深坑，若让开发者直接去调试这些碎片化的推理服务，往往会让人崩溃几个小时。Ollama 的价值在于它把复杂性封装在底层，不仅默默帮助底层推理服务修复了大量基础 Bug，还为终端开发者提供了一个干净、稳定、合乎逻辑且直接出 token 的统一 API。 Jeffrey Morgan 总结道，在当下的开源模型格局中，最大的商业机会之一就是“筛选与整合（Curation）”。面对由海量开源模型、异构推理框架和云端算力服务组成的碎片化宇宙，如何将其收敛为一个“拿来就能用（just works）”的开发工具，是价值极高的事情。开发者真正渴望的只是专心构建自己的应用和产品，而不是注册并管理上百个不同的模型服务供应商。正如聚合平台 OpenRouter 所做的那样，Ollama 通过提供统一的运行环境与标准化体验，消除了开发者的试错门槛。当模型与算力提供商出现爆发式过剩时，能够将其驯化并整合为可靠工具的产品，便占据了生态中最稀缺的生态位。 ```mermaid flowchart TD accTitle: Ollama生态价值链 accDescr: 展示碎片化底层模型与算力资源如何通过Ollama整合筛选，最终交付给终端开发者 subgraph Fragmentation["碎片化的底层世界"] A["碎片化开源模型 (Llama, DeepSeek等)"] B["异构推理引擎与框架"] C["未文档化的接口、异常参数与JSON报错"] end subgraph OllamaLayer["Ollama 核心价值"] D["统一整合与筛选 (Curation)"] E["底层Bug修复与环境对齐"] F["极简、确定性的标准化 API"] end subgraph Developers["终端开发者"] G["无缝替换尝试新模型"] H["专注构建业务逻辑与应用"] end A --> D B --> E C --> E D --> F E --> F F --> G F --> H ``` - Jeffrey Morgan 论述在资源过剩时代“整合与筛选”的稀缺性：“当模型和供应商出现过剩时，将它们整合为可用产品的能力就成了一种稀缺资源。”（“when there's a abundance of models and providers, now there's a scarcity in bringing that together into something that works.”） - 终端开发者的核心痛点：“终端开发者正如你所言，他们只是想构建自己的软件，他们只想把东西造出来，构建下一个公司、下一个应用，我认为这就是我们的用武之地。”（“the end developer to your point they just want to build their software right they just want to build stuff they want to build their next company their next application and I think that's where we come in”） ## Notable quotes > `26:15` 目前闭源前沿模型和开源模型之间的差距可能已经小于三个月了。 — Jeffrey Morgan > `30:00` 对于大多数客户用例而言，模型达到一个‘足够好’的水平后，他们就可以持续使用这种智能水平。 — Jeffrey Morgan > `33:14` 如果你想想一个普通的应用程序拥有成千上万个依赖项，这并不是一个新问题；任何一个依赖项出问题都会导致整个应用程序面临重大安全风险。 — Jeffrey Morgan ## Mentioned Jeffrey Morgan, Michael, Jared Friedman, Peter, Elon Musk, 黄仁勋, Ollama, Ollama Cloud, Y Combinator, Docker, Docker Desktop, VMware, Benchmark, Kubernetes, AT&T, Kimi, 智谱 AI, GLM 系列, MiniMax, OpenClaw, Hermes Agent, DeepSeek, DeepSeek Flash, Cursor, Hugging Face, Anthropic, Claude, Codex, OpenCode, Apple, NVIDIA, AMD, Intel, Nemotron, DGX Spark, GB300, RTX A6000, Qwen, Llama, LLaMA 2, Sakana AI, OpenRouter, Heroku, Google App Engine, DynamoDB, PostgreSQL, GitHub Copilot, Nicira, MLX, MCP ## Video description <p>Ollama (YC W21) is used by 9 million developers and 85% of the Fortune 500, giving co-founder and CEO Jeffrey Morgan a unique view into which AI models people are actually using and how that’s changing.Right now, the biggest shift he sees is toward open models, driven by coding agents, falling costs, and capabilities that are rapidly catching up to the frontier labs. On Ollama Cloud, that shift has driven a 150x increase in token usage since the start of the year.In this episode of the Lightcone, Jeff joins us to talk about the future of open models and the story behind Ollama, from two years of searching for the right idea to building one of the most widely used AI developer tools in the world.</p> [Source](https://anchor.fm/s/8c1524bc/podcast/play/125638035/https%3A%2F%2Fd3ctxlq1ktw2nl.cloudfront.net%2Fstaging%2F2026-8-12%2F431757948-44100-2-a60c62d3496b1.mp3)

[26:15] Jeffrey Morgan: 目前闭源前沿模型和开源模型之间的差距可能已经小于三个月了。 [30:00] Jeffrey Morgan: 对于大多数客户用例而言，模型达到一个‘足够好’的水平后，他们就可以持续使用这种智能水平。 [33:14] Jeffrey Morgan: 如果你想想一个普通的应用程序拥有成千上万个依赖项，这并不是一个新问题；任何一个依赖项出问题都会导致整个应用程序面临重大安全风险。

[52:45] Jeffrey Morgan: And there's this concept that if you're a layer on top of something else, that you're in kind of a vulnerable position as a startup, which is absolutely not true in the AI world. [52:45] Jeffrey Morgan: in the systems world, you want everything to be exactly as it's designed to run. It's tested. It's validated. But LLMs by definition are not. That's a feature, not a bug. [55:00] Jeffrey Morgan: when there's an abundance of models and providers, now there's a scarcity in bringing that together into something that works.

[0:00] Cost is by far the largest pain point that open models can jump in and solve. But, you know, every business has a vision of getting better control over AI and customizing it for their business. And that's really their North Star. You know, cost is something they can solve in the short term, but it then enables them to then go and customize these models for their unique use case. Early 2024, there was lots of interest in fine tuning your own custom models. Then it sort of went away or there will just be wasted effort or get stomped by the next model release. like it's coming back now. You have a front seat to all of it. Do you think we're going through like another cycle or is it here to stay this time? Welcome back to another episode of The Light Cone. Today, we're talking to Jeffrey Morgan, co-founder and CEO of Olama, the easiest way to run open source AI models locally and in the cloud. Olama is used by 9 million developers, is used by 85% of the Fortune 500, which means Jeff knows a lot about the state of the art of AI, what models score highest on benchmarks, and what developers actually download and keep using. Jeff, welcome to the light cone. Thank you for having me. We're down to here. What is the state of the art? What are you seeing out there? Well, I think the biggest thing we're seeing is a shift to open models, especially in enterprise. That's from a mix of US and Chinese by coding agents and also AI assistants, more co-work use cases like OpenClaw and Hermes. And because you sit in the token flow of like so many tokens, you have really good data on what models people are actually using and how it's changing. What are the trends that you're seeing? Yeah. You know, Olamas started as a way to run open models on your MacBook or other hardware, NVIDIA, AMD, Intel. And earlier this year, we launched Olamas Cloud. And what we're seeing models right now of Chinese origin, but they're being accessed by businesses all over the world, especially US and Germany is actually a big source of where... open model tokens are being accessed. Is it all about cost? Is it our enterprises coming because they just want to get the cost down or is there anything more to it? Cost is by far the largest pain point that open models can jump in and solve. But every business has a vision of getting better control over AI and customizing it for their business. And that's really their North Star. Cost is something they can solve in the short term, but it then enables them to then go and can name that has done this? I think there was a great article in the information yesterday from AT&T, and it ends up they've already shifted 40% of their token consumption to open models. And that's right now predominantly through US and Europe models, but they're also evaluating the Chinese models. What kind of workflows do they run? Predominantly coding agents. I think what we've seen just from the extreme growth and per developer or per user token usage has predominantly been from coding agents. And then earlier in March and April, we saw OpenClaw take off and subsequently the Hermes project, Hermes agent project take off, which has then opened up that ability to automate a huge chunk of work over a long span of time to non-developers too. whether it's like finance or support or marketing or sales. You had this actually very cool graph on the... Take off. exponential for OpenClaw. Yeah. So earlier this year, this is a graph of token usage by developer on Olamas cloud, the average amount of tokens they use per week. And we kind of start... This is per developer. So this graph looks like it should be an aggregate of Olamas growth, but this is actually the per user. Exactly. This is on an individual user basis. How many tokens run-up at the start of the year, which was driven by coding agents. So we saw Kimi, the GLM models, Minimax launch. Finally, we had open models that could power coding agents. And then in April, we saw this incredible growth from OpenClaw, really, which was then not just developers, but the rest of the world could take a hard problem, give it to an open model and let it go complete the task, which obviously consumes a ton of tokens as it's figuring out what tools to a million plus with open models. And so all that enabled this explosive growth. So it went from roughly 5x, so under somewhere around, I don't know, 15 million tokens before all these co-work type of use cases. I think that's about right for that open claw jump we saw in April. Obviously, in aggregate, it's, you know, in the 10 to 20x if not more as a whole through a llama's cloud we saw [5:00] 150x since the start of the year. And so it's just the big, you know, interesting thing here is this huge surge in demand for open models, right? Whereas open models, I'd say in 2024, 2025, from the large models being served, were mostly being served as custom models. So you'd taken off the shelf model like DeepSeek or Kimi, and you'd fine tune it for your use case. Like, for example, you know, Cursor had famously done and From there, you can serve that at scale. But seeing out-of-the-box open models being served, that really only took off at the start of this year. even since we started this podcast, these things sort of come in... cycles maybe like I feel like Early 2024, there was lots of interest in fine-tuning your own custom models. Then it sort of went away, and it's like, or that will just be wasted effort. It'll get stomped by the next model release. Seems like it's coming back now. You have a front seat to all of it. Do you think we're going through another cycle, or is it here to stay this time? I think the release of these models, the cadence is only speeding up, which makes it ever more harder to... you know stay on top of that and have a post you mean that the the latest closed frontier models are releasing faster than i think on the open source it's getting faster and faster you know this summer we've already seen three iterations of the deep seek flash model as an example um what used to be more of a six month cycle yeah so the gap is like closing it's closing and i think that makes it even harder to custom train models on the flip side i think the tooling is getting better and so it allows teams that want to fine tune their models to stay on top of it. I mean, we're also entering this moment where AI safety is becoming more and more of an issue at the Frontier Labs. So the Frontier may well slow down to figure out its alignment and containment issues. And then meanwhile, the open source models and open weight models are continuing to grow and... get better. Yeah. And, you know, we saw the announcement and the release of the GLM-53 model and its capabilities from a cybersecurity standpoint, you know, being extremely impressive. It creates a big opportunity for whether it's startups or existing businesses in the security and governance space to really jump in and help. Because I think if you look at, you know, that AT&T article I was speaking about, it largely, the blocker to adopting open models is largely around security and safety. But from our experience talking to customers, whether it's in Europe, whether it's here in the U.S., if you can solve the safety problems, by and large, adopting the Chinese origin model labs is completely on the table. And it's really exciting for these businesses. There's sort of this interesting moment right now where a hugging face had to use open weight models to actually even detect the hack from the frontier. A common question we get is like, well, where can I use open models that are leaps and bounds of an advantage over using a frontier closed model? And one of the key use cases is... security testing and making sure that your software is secure. Yeah, because if you try to get Claude to pen test your product, it will just refuse to do that. Correct. Yeah, by and large. Whereas there are literally obliterated security researcher models that you can find on Hugging Face that allow you to do it. That is true. There are ones that are custom trained to be even more liberal to go and attack these problems. But even the out-of-the-box models, they do come with safety training, but they're a little better understanding if you're doing this for a good you know, use case versus versus one that's more of a negative or you know malicious use case a cool thing about olama is that because you guys are such a key distribution channel for these models my understanding is that typically the model developers are contacting you before the general release to like coordinate launches and stuff like that and you often get sort of like previews of what's about to about to happen and you get these incredible growth spikes when a new model drops scale. Yeah, absolutely. And like I said before, the models are coming out faster and faster and faster. And so we've developed a playbook to what does a successful day zero model launch look like? And there are a lot of things to get right. There's making sure that it's supported and your favorite inference engine, which is generally sometimes a multi-week process to make sure it's fast, to make sure it's accurate, to make sure it's up to spec with the reference. There's also finding the right use cases and harnesses for developers and users to make use of this model. announced DeepSeq launched their first multimodal model from a large model LLM standpoint. They had previously had some smaller OCR models, which unlocks a whole bunch of use cases. But what they also changed was the DeepSeq harness, which launched recently so that it could support this capability. So step two is then to find the harnesses, make sure they're prepared to actually run this model and to do that effectively. But every model is different. They all have different is really hard. I think the key thing to do at the end of the day is to run benchmarks against the final product ahead of release and make sure that it's running [10:00] as the research team at the lab specified. So I think one very important part that you play in the whole ecosystem is you kind of create a very legible standard to be able to make sure you have the best way to use the harness for each new model, each new tool call, and all of it is consistent across all the different models, which is pretty hard to do. Yeah, I think there are three things really that we try to package together. One is harness, for example, whether it's an off-the-shelf harness or an SDK to help use the model, some existing harness that is designed for this. And what's great now, there's so many great open source harnesses. The Codex harness is open source. OpenCode's a great one and one of the most popular harnesses from Olami users. Step one's getting that right, but then you got to package it with the model, making sure that it's available, it's reliable. If it's in the cloud, that there's enough capacity for it because day zero tends to be the largest growth day, obviously. And then importantly, there's the hardware and the providers and that's actually where it's a lot of collaboration to be had whether it's like an inference provider optimizing the model or you know with some of our partners whether it nvidia or or you know for example working with the apple silicon stack making sure that it actually runs the model really fast because if the model is capable but it really slow that not a great experience So getting those three things packaged into a box and generally you get the model you lucky if there the ability to access the model a few weeks in advance. A lot of this stuff comes together in the last 24 hours before the model gets released. And so it's generally a fire drill. So you kind of almost like an operating system in the old world where you needed to really integrate very tightly with all the drivers, all the hardware, and then at the application level to make sure that all the apps were really tuned up well. And you are the glue for all of it, right? I do think the OS, which is generally a cliche analogy to use, is a good one because you've got the drivers for the hardware and the providers and the inference layer, but you also have the application runtime and making sure that the harness works. Gluing that together, it's a very combinatorically large problem to solve. And so doing that well is really difficult. And so, but, you know, over time you develop pieces that, you know, allow you to quickly develop that, test it, release it, and kind of have this common runtime that can match any harness to any model. And that's kind of the role we're really playing for developers. You guys are very hardcore engineers and you fine tune things all the way to from the origins with Apple Silicon all the way now to DJX How do you build such a deep technical bench with that I think a lot of our team you know we aren AI researchers by background We're from VMware and Docker and from other networking companies. And so the classic compute problems are kind of reinventing themselves in the inference land, whether and so largely, you know, that's where we like to focus our time. but I think at the end of the day, you know, it all comes down to the developer experience. What is it like when the developer makes a call to the API and gets tokens back? What happens? And there's more and more happening in that layer right now. And getting that right needs all the layers of the stack to work well together. And I think, you know, one of the challenges with open models has been that hasn't been happening at the rate of what a frontier model lab puts out where they have, you know, the classic five layer cake, right? That Jensen mentioned, which is like the apps. You have, you know, the model, You have the infrastructure and inference. You've got the chips and then you've got the energy. And they've got all that ready to go for developers on day zero. And that's really the thing that we're trying to reproduce for open models. Of course, we're not going to do every layer of the stack, but we can help orchestrate that. You layer the layers. Yeah. And maybe in open models there more than five layers Like that model layer actually has a lot to it right There the model weights but there also a lot of the orchestration components that each model is uniquely good at There this developer API layer There so many opportunities to build between the model and the application layer that are kind of hidden in today's, you know, five layer cake stack. That's interesting. Do you think any of those like hidden layers might get unbundled and become their own companies or providers? Absolutely. There was a really good talk from some of the Anthropic team, the platform team, and they talked about three big things. One was knowledge. How do you connect your company's data and context to the model? One is coordination. As you know, when you make a request to in your Claude app or Codex, it goes off and spins out a bunch of sub agents, some of those in the cloud, some locally. There's a coordination problem. And then lastly, there's an execution problem, which we talk a lot about as sandboxes. but these agents, more and more of them are moving to the cloud. There's a huge compute problem to be solved there. And if you look at what's happened classically and in the cloud business, open source has meant that there are best of breed companies for each of those problems, whereas you might've had kind of the, if you look back to the original generation cloud products, you have like the Heroku's of the world, you have Google app engine, where all those things were bundled together. But what developers ended up preferring is best of breed products, [15:00] each one. It's the novelism, which is all things are just bundling or unbundling. Exactly. The frontier model. Frontier model labs want you to be totally in their walled garden of managed agents and their context, their memory layer. And then meanwhile, little tech and all the founders out there and all the open source developers don't want to be caged in. So we're going to make all this other stuff and it'll be an interesting moment to figure out like what ends up winning. I mean, it'll probably be some mix of both. I think so. And you know, you've got this abundance of open model tokens that's being created. There are just dozens of open model providers that are able to serve these tokens. The new scarcity, the problems now are what's above the tokens, right? How do you orchestrate an agent from A to B? These are problems that are tons of new systems and engineering problems that are just really hard to solve for an individual dev. There's no way they're going to build all those layers. Well, the interesting thing now is because the coding agents themselves are getting a lot better and ostensibly this is the worst the models will ever be. The classic reason why there was a moat here was it was just too hard to have really well software that was properly tested that actually satisfied user need And what if that goes away Like we literally at this moment where actually maybe the age you know you just have a cron and it runs a markdown file on some TypeScript, and it'll just, you know, there's no lock-in anymore, right? Like, you could be using OpenAI's memory system one day, and then actually, you know, you could have an agent be constantly syncing that against your own memory system. And actually, it just works. It's fine. It works over MCP. There's plenty of runtime testing, and then there's no lock-in. Yeah, I think a lot of what we know of as harnesses today, a lot of those pieces will go down into the model. But as you kind of push a lot of the core loop of the model and hooks down to the model itself, there are these pieces that kind of come out. Memory is a great one. And the general guidance we're using is if it's a stateful problem, like there's storage involved, that's something that, you know, in the end can't go into the model because the model's trained and it's, you know, as we know, training runs are now happening on like a monthly basis, but it's still not up to date with the latest data. So generally storing data is like a huge problem space that I don think will ever make its way down into the model layer And managing credentials and that kind of stuff Security credentials safety Open models don have all the safety tooling that closed model providers give you out of the box But that's super important, especially for businesses to adopt them. I'm curious where you see sort of the end or future state for enterprise on the balance between sort of frontier closed models and open source model, especially on spend. like my it feels like you know initially it was just the everyone's just like allocating all of their budget to anthropic or um open ai uh my sense now is yes open a open source is clearly growing but like so are so is like anthropic spend so like the things seem to be growing together like does that continue or do you think there's like a steady state where it's like i don't know like half the budget's going to be on the closed source frontier model and half's going to be an open source or something different the super majority of tokens and this is our take it will be open models within a business call it 80 90 that doesn't mean 89 of the budget will go to models in fact i think what the open model community is doing incredibly well together is lowering the cost to make it more accessible and so maybe you'll only pay 10 to 20 percent of the cost towards open models but your most of your tokens will be going through the open models right which will enable a whole bunch of use cases on top because you have this abundance of tokens You not thinking about taking away token access from your team You giving more and more access I think for the hardest tasks, that's reserved for these frontier labs where a lot of the best researchers are. And then from there, there's a whole bunch of problems in the middle, right, where maybe it's a combination of open and closed models working together. I think the steady state is that most of the models are open. It's kind of an interesting idea because as the models get more powerful, ideally you just want to delegate to like your smartest model to figure out when to go to like an open model. But the labs who own the models presumably don't want that. And I think, look, I think that all the labs are aligned in many ways to one thing, which is how do you serve the customer? And I think it will be up to the customer to decide if I have a router where some of the scheduling and harder orchestration happens through a frontier model, so be it. But a lot of the kind of line item work can happen through open models and the collaboration of the two together. I think we've seen a ton of projects, whether it's from Sakana AI or Open Router, that have combined the two and seen really good results. This is not too dissimilar from a human organization, right? Like you have, you know, like a law firm, there's like a partner and then there's like a bunch of associates and like the partner farms out the work to the associates, right? It's like the same. [20:00] We saw the same thing with cloud computing, where it was really a blend of proprietary software, some of them provided by the cloud providers themselves. For example, you know, AWS had DynamoDB, which was kind of their proprietary scale-out database. But then a lot of customers use that in conjunction with PostgresDB. And in the end, you know, what we see is customers will use a combination of the two. I think it's a very common pattern. for let's say image processing, supposed to audio, that's been, or even go way back in the PC era, you had like your standalone audio card, right? graphics card and all that. Speaking of Apple Silicon, should we talk about local models? Because you're in a bit of a unique position because you have large businesses, both in cloud hosted models and locally hosted models that will run on your laptop. What are you seeing in those two worlds and what do you think is going to happen? I think it's incredibly exciting because it's similar to the closed versus open model question. It'll be a mix in our mind. And that's with lower latency and, of course, lower costs when it comes to the per-token costs. Ultimately, you're buying hardware up front. And you'll use that in conjunction with these cloud models. What's exciting about this next generation of hardware, which we've had for a few years now, is just how good they are at running the 20 billion parameter to 40 billion parameter range of models, sometimes up to 120 billion parameters. Yeah. QIN 3.838B is now as good as Opus 4.6 for incredibly exciting because you can run that on not the lowest memory MacBook, but the second lowest memory MacBook you can buy from the store. So it's incredible. And are you seeing your users do that? How are you seeing people use the local models versus the cloud hosted models for in practice? From the side of which models they're running, we're seeing a really solid mix of US and Chinese trained models being used for local. And we have the incredible models from the original for local. But when it comes down to use cases, coding agents by and large are most effective with the large cloud models. You're solving really hard problems. You're writing code tests. It's really difficult versus some of the document processing workflow use cases that run extremely well locally because they don't have as difficult of a task in the end-to-end problem you're trying to solve. And so that's where we see this hybrid execution model where some of the easier, more hey, we need to go to a large cloud model for this. And I think what that means for customers is that you're really dropping the costs even further when you're going from open models, not just because they're cheaper to run in the cloud, but because now you can run them effectively for free on the hardware you're buying for your business anyways. And what we're seeing ultimately from the cloud technology coding agent models, it is predominantly Chinese models being consumed today. And for the local models, it's a really strong blend of US, Europe, and Chinese origin models. Yeah, these two graphs are pretty stunning in comparison. Basically, for local models, the US and Chinese models are neck and neck. We're tied. And for cloud-hosted models, the US is recoloring the x-axis. It's 100% Chinese Is that what this graph is showing? Effectively. And with the launch of the Nemotron Ultra model, we're seeing kind of the first wave of that. And it's really exciting. NVIDIA as a company is so interesting because their moat is not, trying to start new software businesses or sell tokens. They seem to be quite interested in just releasing a lot of open source and helping the ecosystem. And then the fact that they do that then helps them stay ahead of the game on the hardware side. I think so. And ultimately, NVIDIA, what's so incredible is there's helping power an ecosystem around open models, whether that they're working on, which have a GB300 on your desk. How do we get on that list? That isn't deafening loud. Do you know the price point on that thing yet? I don't know it off the bat. It's got to be like Gary's buying it. I know, right? Well, I looked it up. I mean, you can probably run a frontier model for like, I mean, very slowly for like, $200,000, $300,000. Is that right? I think it's even more competitive than that. Yeah. And you can run more than a Frontier model at high speeds at a price point that isn't very far off what you can buy from a classic workstation computer. Oh, no way. If you think about quite a few of the customers we talked to, some of them are banks, for example, or industrial businesses, they already have these NVIDIA devices. workstation GPUs in every single engineer's desk, some of them, tens of thousands of [25:00] and so this is oh i want one They've been selling them for all the CAD work. Well, this is for the kind of original RTX A6000s, but this is the next generation. Yeah, yeah, yeah. I'm going to have to email Jensen. This was at GTC. So what we see here was at GTC. And we were one of the first people, along with Elon and a few others, to receive the DDX Spark as well, which sits on your desk and provides 128 gigabytes of unified memory to run that kind of 20 to 120B model range. But obviously, that's just the beginning of a whole new range of hardware that can run the biggest models. So did you say you can buy a bunch of these and chain them and actually run a 400B model? You can, absolutely. They have this really fast network link. And so you can stack them on your desk, almost like a miniature data center rack. The thing that people do with the Mac minis? Now this is like the production version of it. I think that's what's so exciting is you're seeing both from Apple and NVIDIA, this incredible, incredible lead to next generation hardware. That's... built for these models and is effective at running them. So you'd say this is the platform to get. You could make Apple Studios work, but... Like, if you want something that just... can work, get DGX Spark. From our testing, both are very competitive. Got it. So I think a lot of whatever you can get, what you can get. And then also the tech stack you're looking for. I think there's an incredibly mature tech stack through the MLX project with Apple, where they've done some amazing work to run LLMs on the Mac Studio, but also the smaller Macs. And of course, the DGX Spark stack's just incredible. We're super excited as partners with NVIDIA for that. And, you know, it's funny with Olama's journey. We started local. Clearly, the coding agent demand is in the cloud, but that's going to come back locally in our minds because the hardware will catch up when you have a GB300 on your desk and you want the fastest coding loop that's as fast as running your tests or as fast as making code editor change. We all remember the GitHub Copile experience of having the autocomplete come up in a few milliseconds, 100 milliseconds. That experience will make its way back to the desk, which has been a journey of starting We think that'll come back local and you'll end up using the two together. Speaking of like what you can get in order to run a llama cloud, you need like a shit ton of GPUs. What are you seeing in the GPU market? I think what we're seeing is ultimately the prices are changing very quickly and the supply and demand volatility is very high there. is very hard. Thankfully, there's a great set of inference providers building on top of that. And so we're seeing this extreme demand. Are you able to get all the GPUs that you need? Are you constantly growth limited by how many GPUs you can get your hands on? What's the current state? We're lucky in that we've partnered with quite a few providers to work together to pool a bunch of GPUs together, which allows us to stay on top of our demand. But that's a lot of work. spending time thinking through which model will get run where, how fast should it be, which region is it in, what will the latency be for the customer. There's a lot of hard problems to solve in that stack. And I think what's really exciting about products like Open Router, Olama, the Open Code project is for an end user developer, they can sign up and get access to this without having to go negotiate prices on a B200, B300. You know, think about their 24-month forecast in order to get access to some of these GPUs. YC's next batch is now taking applications. Got a startup in you? Apply at ycombinator.com/apply. It's never too early and filling out the app will level up your idea. Okay, back to the video. Suppose you were and you're building some AI company, and you haven't raised a lot of money, and so you want to use as many tokens as possible, like inexpensively, what would your advice be to that person about how they can get huge mileage with a limited budget? - This is new class of models, like DeepSeek Flash is a great example, and I think there'll be quite a few more, where it's ultra low cost per token. It's also low cost per task, which is a really important metric. That class of models, in my mind, will be the first ones that come down to this idea of unlimited tokens. We all remember ChatGPT. You didn't really have to think about how many tokens you were using. You would just use it every day. You had unlimited. Ultimately, I think we return to that, but it's going to take a lot of work in the model, the architecture, to be custom trained for high volume token usage. And if you think about the start of the year, we really want open models got to the frontier of intelligence. We bridged the gap. We're maybe like less than three months behind between the frontier closed models and the open model. it's [30:00] But the next problem to solve is extreme efficiency. Seeing, for example, the GPT Luna model become very, very price effective for customers has been a huge boom. We talk a ton of customers where that kind of pricing enables widespread adoption within a team. I think we're going to see that with open models. We already are seeing that with open models. I think the DeepSeq Flash model is leading that charge. If we go back to the model breakdown on Olamis Cloud, the highest growth area is definitely the DeepSeq model. And this is largely powered by the DeepSeq Flash adoption. So this new class of Flash models where they're good enough for 80% of the tasks, they're really fast and they're ultra cheap. This new class of model that I think will enable Flash. some of those use cases. Yeah, those are going to be like the workhorse model to do like all the grunt work. Exactly. Yeah. You won't have to be thinking about how many requests am I making, how many tokens. You'll be much more inclined to consume as much as you can because... you know, it's able to solve. the hardest not the hardest but you know difficult problems if you think back to the coordination layer we were talking about earlier too being able to coordinate these flash models together to do different tasks can also yield great results that a bigger model can so by having these cheaper models, not only are they more accessible, They can run faster and you can access them at higher volume, but you can start to chain them together and build new problems that are solved by orchestration on top. And that's a really exciting area for new startups, for existing inference providers, for some of the larger businesses today that solve workflow problems. Ultimately, they're going to be a big part of it. be able to chain these models together is going to be super helpful. And you won't have to think about the underlying costs. Yeah, I guess, you know, when we first started talking about AGI, even on this podcast, there was sort of debate about, you know, and I think a lot of AI researchers would come out and say, like, there's just going to be a giant God model, and it's going to do everything. But, you know, I think so far, like, it hasn't quite worked out that way. Like, obviously, you still have, you know, if you're, if you have to literally hack the NSA, maybe you of use cases, like you're talking about orchestration and you're talking about like smaller models, you know, that the task composition actually probably gives you a bunch of ways to make it more repeatable. It's more trustworthy. Like it actually does work at a cost that is like possible. So, you know, if it was going to be a God model versus like lots of, you know, smaller special purpose or even just. simpler models. It's turning out to be the latter so far. I think for most customer use cases, there's a you know, level at which a model becomes good enough. and then they can continue using that level of intelligence. Maybe the model will get faster, it'll have better architecture, it'll have new capabilities, but they won't have to reach for the god model. But I do think there are use cases where the most powerful models unlock them, and that'll continue to be a thing. It'll be really exciting sometimes scary as well on what they can do. But for the run-of-the-mill use cases where open models really shine, I think that's where we're hitting a point where You're not solving necessarily the hardest problems within the business, but they're hard enough where it's now unlocked by open models. Will there be an open model that becomes a God-tier model? I think it's possible. And we're seeing really exciting developments from ZPU AI and GLM where some of the tasks, they are frontier. And we all saw with the Kimi model how for web development, it became the best model. which I think makes all of this even much more exciting. This is a bit of a sensitive question, but what do you think about this and the geopolitics around it? I think, you know, a lot of the geopolitical... angles around this start with, you know, where the where the model's from. And the more we spend time with customers and users, a lot of it's actually how the models run, where it's run, how it's run. Is it run in a secure environment? And that starts to matter a lot more. But I do think, you know, look, it's super important that. A customer in the US can use a model trained in the U S and we have two kinds of classes of customers we speak to. One is they don't really care where the model's from, they care about where it's run. But for every one of those, there's you know, a customer that's saying, I really care about where the model's from because it's data, the way it, it's not even just a, a security issue as much as how does the model, you know, speak. We all go through and communicate. A lot of the models go through phases where they sound more robotic, they sound more friendly. And a lot of that matters too. But I think the highest sort of bit is obviously making sure that you have a model that end-to-end, you understand where the data's from, which is great from the Nemotron models, that you can go and introspect what made this model. Because if you're putting in a mission-critical task, which people are absolutely using open models for mission-critical tasks, there's a post online about how Lama powers to detect [35:00] surges in Finland to make sure that the lights stay on. That's where these models, the model origin really matters. For like critical... tasks like that, how do you ensure that a Chinese model even if it's hosted in the US, isn't basically booby-trapped to cause problems. The Manchurian candidate problem. Exactly. Have there been any known cases of the Manchurian candidate yet? I think not that I can think of off the top of my head. I feel like I would have heard about it. What you don't see a lot on some of the press articles is how robust some of the the IT and security teams are at the businesses that we know of, the top Fortune 500 businesses, they're really used to this already because open source software, if you think the average application has thousands of dependencies. This isn't a new problem. And all it takes is one dependency for there to be a major security issue in the entire application. Yeah, supply chain poisoning is insane. It's a thing. It's been a thing for decades. And it's not... new in that sense. It's a little more opaque because you can't dig into the model. It is deterministic. But it's deterministic. And if you screen the model properly with safety checks, by and large, at least what we're hearing from customers, is that can be solved. Do you want to talk about the origins of Ollama? You guys came up through the Docker ecosystem, and a lot of people watching would love to be in the position you're in, where you have this till the end of time. No, it's a very powerful situation to be in. You basically found yourself on top of a giant oil well, right? For those out there wildcatting, can you tell us that story? You were actually working with Jared in 2021. Yeah, you know, my co-founder and I previously built Docker Desktop while at Docker. So we really got an understanding of what makes a great developer experience. But I have to say the first few years of Olam as a company was really in search for what's the right problem to solve with this muscle we've built of trying to design a great experience for developers. You applied to YC with a very different idea, right? For sure. Do you remember what the tagline was when you guys applied to YC in Winter 21? I think it wasn't well defined. I think we realized, let's go back to building a really in Kubernetes. I remember what I wrote down on the application. It was a Kite-matic for Kubernetes or a Docker desktop for Kubernetes. Yeah, which was effectively Docker desktop. They had a great Kubernetes. I think, you know, it's one of the challenges as a second time founder that, you know, Michael and I have told ourselves, we tried to over-engineer the idea in many ways. And I think even the two to three years after, like Olama, we did YC in 2021 and Olama wasn't launched until... July of 2023, after we raised our Series A, obviously after we had done YC. That journey was one of... really in search for a customer problem that could delight a developer. In some ways, it was almost a good thing that We tried different ideas and pivoted until 2023 because that's when mom had LLAMA came out and started the Open Model Wave. Which is why it's called OLAMA. Not necessarily. Okay, no. Oh, really? LLAMA means generally from our experience, whether you think of local LLAMA, the subreddit, OLAMA really just stands for... open models you know as we were looking through the name it wasn't necessarily from an existing model it's more llm exactly it's like the animal plus llm yeah i think having that character was important so like what's a good name for a character a face you can put to the name because animal mascot sometimes yeah github yeah still hasn't taken my advice to have llamas come to actual olama oh my god i'm curious what your series a pitch was because you raised from like fantastic investor, but all of this, the future we're in now hadn't quite taken off in 2023. So what was like the pitch and the vision back then? Yeah. And we partnered with Benchmark in 2022. So it was Dolly days, but pre-ChatGPT. When it came down to the pitch, I think we weighed so much on like, hey, we're trying to build this great developer experience. We're solving this security problem. And we had known Peter, the partner at Benchmark from our previous lives, the Series A investor in Docker. And so a lot of it was weighted on the people and also why we exist. I think the what, I mean, solving SSO for Kubernetes, which is a real problem, wasn't really our passion. I think we were really lucky to find a partner that could could see us for what we stood for and what we were trying to do versus the point in time you know problem we were solving at that I see so you raised the a sort of pre-pivot correct okay I didn't realize that actually I was just [40:00] at this cloud tokens by model family graph. And like, basically, if you just look at this graph, it looks like the Olama story begins on in February, 2026. And it like explodes thereafter, which is like so funny, because like, of course, it actually goes back to like, 2021. How was it like to be like sort of lost in the wilderness for like many years working on stuff that was like, kind of working, but like not really taking off. And then all of a sudden, Like how did, how did it affect you and your co-founder psychology and the team and the employees? What was the experience like? It was definitely scary. And, and for a few reasons, you know, one is, um, Like when you're trying to solve a problem for devs or for a customer and you're just getting on the phone with them over and over again, and it's not totally clicking. That's, you know, it's less about the, are we in the headlines or is, is the project taking off the product we're building? It was just, are we truly actually solving a problem for somebody? And I think being lost in the wilderness, like what's your North star that customers are generally a great North star, but not seeing the North star is even scarier, right? Cause often, you know, what problem you want to solve, you just haven't figured out what problem. And I think the, you know, Michael, my co-friend and I, we started this company because we built a company in the and we ended up being acquired by Docker very early. It was just the founding team. And our North Star was saying, we want to go solve a great experience for developers with something they find really hard. But man, in the two years where we're just finding that problem, it's really scary. You know, we had a team of more than 10 people which made that really hard. And I'm so thankful to that team for staying by our side as we went through different ideas. And what's not really obvious is we went from this security for Kubernetes to then like security for developers on the desktop, which is like the pivot that we've never spoken about. And then we kind of took that form factor when models came out. We said, well, it was a leap, but it was, we knew kind of the kind of problem and the feeling a developer wanted to have, but Elem's finally made it realize. It was crystal clear at the point when We tried running the llama model and it was really hard. And we're like, okay, this is a problem. And it's really impressive when you get it working. And it's kind of just a... zero to one moment. I'm curious for the story of that pivot, because there are actually like many pivots in the Olama story, but probably like the most critical one was like the pivot to Olama to doing like locally hosted LLMs. Like, how did that come about? Were you just like tinkering with ideas on the side? And when you found the idea, was it really obvious to everyone in the company that that was the thing to do? Or was there like a big debate and it wasn't until it took off that it became clear? We, you know, sat a room together. I remember we were in Toronto Alto and now we're predominantly in Palo Alto. And, we were saying throw everything out like if we had to start from scratch and we were just joined yc right now what would we do and you know we had seen two big problems because we had talked to some users in lms we tried using open source lms ourselves which lms in general one problem was could you build a gateway to access any model and host that and make that really seamless. Back then, we were thinking of it as like the segment for LLMs. It's a good way to think about it. Which I think has become really this big router idea, which is only at the beginning. It's a massive opportunity. And the other problem was we were a bunch of ex-VMware, ex-Docker folks. We know how to make things run. And so let's help make things run with open models. And then we kind of, yeah, systems. And so we kind of tried to really introspect our team, which I wish we had done sooner because security is a very different team and sale than developer tools. And just by doing that, we gravitated towards saying, let's just try this thing. Let's give ourselves two weeks to launch the first version of Ollama. And then LLAMA 2 came out and we said, that was right at the end of the two weeks. We said, okay, we're launching it. And we just had a bias to action. If you think back, like, In two weeks, all of that happened, going from idea to shipping it to getting to more users than we had ever had with our previous stuff. And before that was two years of just, frankly, overthinking the customer, the product, and just not getting something out there. The first time I actually heard about Olama was on Reddit. I didn't realize it was you guys. I was on their logo. I just was interested in running local models. And it was on, I think, the local LLM subreddit or whatever. Everyone was just raving about Olama and how great it was. I was like, oh, it's a YC company. Yeah, I found out later, actually. We were called a different company. You weren't in our internal system. Yeah, I remember catching up with Jared and saying, oh, hey, by the way, there's all that security stuff. We have this Olama thing now. I think you were catching up with Jared, and then I bumped into you on the stairs, and I think you had your t-shirt or some swag or something. And I was like, you guys are Olama. I was like, meeting a rock star or something. It would have been really hard to time this, but I wish we had taken that leap much sooner. I mean, the best time to do it was during YC, possible. ILMs didn't exist. Lama didn't launch yet. I guess you were also one of the first GitHub projects that very quickly got to 100,000 [45:00] stars, right? Do you remember how long? It was like very quick. Yeah, I can't remember exactly how fast, but it was much faster than Docker and Kubernetes. To your point, things kind of just started working and started taking off. And You're really, as a founder, just beside yourself because you can't totally explain why. I think it's the best way to explain product market fit. And there are different levels of product market fit. We only started monetizing earlier this year with Ollama's Cloud. But just to see people fall in love with the product, it's such a zero to one moment that... um i wish we had done it during yc but in some ways it wasn't possible i also think it's just kind of wild to put into perspective like you sort of went from being in sort of like the cranks on reddit like interested in running their own like rigs at home to like 85 of the fortune 500 in like two years or something like that that's like a pretty that's the homebrew computer club to uh broad computer adoption like speed run that took 10 years for the pc yeah it took like One of the things that surprised us the most, because I think, look, I think Open Model C original users, very much hobbyists, just tinkering, oh my God, this is even possible. But very quickly, because Open Model C, two things. One is they're free to get started with, and you could run them anywhere. That is incredibly helpful to a Fortune 500 IT developer team, because they don't have to ask for permission to use it. And so it just happened what was really good for a hobbyist user translated very quickly to a developer within a business. It just happened to be a case where that was it. You know, for example, databases, we saw some of this too, where a database that started for devs like MongoDB very quickly also moved to enterprise. But because... LLMs are stateless. It made for such an easy transition. Now, moving to the cloud, there's a lot more in play. There's an economic question if you're a customer. There's obviously security. Where's the model running? But what's beautiful about open models that both hobbyists and IT developers loved is You can just get started. You didn't need permission. Can we talk about the monetization angle? Because this is interesting too. So like in 2023, Llama 2 takes off. All of a sudden you've got all these users, 100,000 GitHub stars. Like you've clearly found something. But it's basically like Reddit cranks who are using it. You're making no revenue. And there's no obvious path for how you will ever make any revenue from all these like cranks on Reddit. It was two years before you actually figured out a business model for it, which funny enough is exactly the position that Docker was in. How did you think about it during those two years? Were you worried about it? Was the team asking like, what's the business model going to be? How did you think about like figuring out how to make money from it? I think there's always two ways that we saw open models. being able to monetize in a way that's great for the company, great for the developer, and great for the customer. And one of them was a privacy-focused... product, which Ollama really started with that in its open source incarnation. But we always felt that there was this moment where you weren't using the Lama models, for example, with tool calling right away when they came out. So there were use cases where it was still reserved for the frontier models. And again, at the risk of overthinking it, we kind of saw that there wasn't the level of product market fit with open models that closed models had. And in some ways, philosophically, with when that happens, we want to be there to capture that. I think it happened this year with coding agents running with open models because you had the largest consumption of AI being matched with finally open models to be able to service that. There are a lot of opportunities along the way to do it privately, securely. Again, a lot of the Fortune 500 have already adopted Ollama. But we really asked ourselves, what would be the most important problem we could solve for a customer? And The local piece, while an important part of that story, never felt like the whole story, which was... how do you access open models? for the hardest problems. And so in some ways waiting, we knew we had to wait a little bit for the market to mature. At the same time, what are the risks of waiting? Well, you build a culture, if not careful, and we had learned a lot of this from our Docker days, where you don't think about monetization. It's not a priority. I think from our previous Battle Scars as a team, we kind of had, we knew about that. But I think the other component, which is really important, is making sure you keep in touch with your customers. Like one of the biggest risks of having an open source project that takes off is you consider your user base and customer base, like your customer, just a blob on the internet, which is a really risky way to think about customers because you want to meet them, figure out their needs. What are they doing? What do they want to do in six months? What's their story? And I think that's the thing I wish we had done a little more in the last few years. And we're doing a ton of that now. One thing I'm curious, when you went through IC, do YC, actually. You know, we went back and forth on this for a lot, which we shouldn't have. We should have just said, of course, we're doing YC. But by and large, starting a company [50:00] is a really lonely experience. Even if you have a great co-founder and Michael, my co-founder was the co-founder of my first company, he was my college roommate at University of Waterloo, but it's still lonely. And I think just having a set of peers, even though we did it during the pandemic, just talking to Jared and like five other groups of founders every week really helped you feel less lonely. And I think that's such an important part of it. And then, of course, when we finally moved down here and there was no more COVID, the network was just incredible. And the fact that we could meet founders building on open models, building on any kind of AI, we kind of knew that was going to happen because we had known so many founders from the University of Waterloo who had done YC pre-COVID. And they were like, it's really about getting together. And that was a big part of it. And we knew that was there. And, you know... I think that's what made it a no-brainer. But also just, I think... There are a lot of mistakes you can repeat that you don't have to. And what I love about the YC community is how transparent founders are with each other about those. And I still keep in touch with the founder of Docker, who's an investor in our company. And we're able to talk about some of these challenges we saw in the previous generation of companies that, you know, we... don't necessarily have to repeat or things that worked and we can bring, you know, into the future. Yeah. If you just don't repeat one of those mistakes that, you know, sometimes is the mistake that would have killed the company. Potentially. Yeah. Yeah. Our famous saying, you know, a bunch of our team is from companies that ended up working great. Docker is doing phenomenal now. But whether it's, you know, some of our team was early, early at VMware and, you know, It... There are always ups and downs. And I think just having a group of people around the table who have a collection of those and also what worked. Actually, what worked is actually even more important. Just being able to have that muscle memory is a big part of it. Oh, man. I was just thinking about this because we obviously hang out and work with a lot of 18-year-olds or 19-year-olds. And then sometimes they're always asking, like, well, what should I do? fit, like do that once, like even if it's a month, even if it's like three months, you would learn more in those three months because then you know what good looks like. And then without that, it's like, I mean, it's not like it's impossible. Like people at YC do figure it out because, you know, but it's that much harder. Like the difference between having seen something that actually works from like beginning to like some form of like, this is what the bug database looks like. And here's like the bar that we hold each other to. Having seen that, it just like multiplies the chance that people succeed. So it makes sense that, you know, starting off with a co-founding team that has seen a lot of that pretty powerful. Yeah, I think it provides you a set of values you can work around, especially when you have so much power in your hands with AI. They're just parts of it that I can help you with, but it won't hold you accountable to it. How does software work? And look at Olama, for example. I'm sure there's versions of Olama running in the wild from two years ago. How will your software work when somebody falls in love with it and continues using it for two years? Is it still going to be working well? Hopefully they update to the latest software or it's a cloud service. But I think you build that muscle memory. And we definitely have that from a lot of our more senior engineers on the team who are at VMware or Nysera, for example. But at the same time, I think there are a lot of lessons we learned in the previous generation of DevOps and infrastructure that aren't valid anymore in the AI world. Oh yeah, tell us about it. What have you found? What is not valid anymore? I think a good example that I classically used is there was this Generation of companies called platform as a service. And like Heroku's. The Heroku's of the world was a great example of this. I mean, Docker started out as that. Docker started out as a platform as a service. And there's this concept that if you're a layer on top of something else, that you're in kind of a vulnerable position as a startup. which is absolutely not true in the iWorld. And in fact, going up the stack can sometimes be even better because you're closer to the customer. In an infrastructure world, that's also the case. And that was like an analogy that we had to like so many of these muscles, we actually had to break building Ollama. Another one was, you know, these LMs are never perfect. And like in the systems world, you want everything to be exactly as it's designed to run. It's tested. It's validated. But LMs by definition are not. That's a feature, not a bug. Exactly. It's a feature. You want it to be a little non-deterministic, I suppose. It's that, you know, with AI now, there are just problems that you don't need to staff as heavily, whereas you did 10 years ago, right? If you think about what does your customer support pipeline look like? What does it look like to deliver a cloud service? Like it's a very different world with AI because how do you build a service where no engineer knows exactly how all the code works, which is obviously the case now. And so there's just new lessons we're learning going from like a, [55:00] Some of our team from infrastructure 1.0 in the 2000s to cloud in the 2010s to now the AI space, there are a lot of rules that break. You're probably actually doing an incredible service to both sides of the ecosystem. The end users get this very clean thing that just works, especially the tokens just come out and they're very clean and the API makes sense and it's rational and logical. oh, yeah, the underlying inference provider has a weird error for, you know, if you put this parameter in this way or it expects JSON and, you know, it's not documented. It's just like this insane minefield. Like, you know, the agents can kind of figure it out. But like you're going to like bang your head into the wall for like a couple hours before, you know, the agent figures it out. And in the meantime, you're like, this is a terrible experience, you know. And so you're like in there probably helping the inference providers fix all these fundamental bugs, too. of the big opportunities in the open model landscape is curation and taking a fragmented universe of models and inference technology and cloud services and harnesses and like making that actually just work is a really valuable problem because the end developer to your point they just want to build their software right they just want to build stuff they want to build their next company their next application and i think that's where we come in but it's where a ton of great services also come in and we saw you know open router obviously is a good example of that from a wide model selection so the developer doesn't have to sign up for you know 100 different providers they can just go to one they can pay in one place i think we've harness that integrates with any model. It's a really powerful experience for developers just looking to try the next model to see if it solves their use case better. So this curation and, you know, when there's a abundance of models and providers, now there's a scarcity in bringing that together into something that works. Thank you so much for joining us. That's all we have time for. Thank you guys for having me. Bye.