LendingUSA, Scrut Ditch Frontier AI to Cut Runaway Token Costs

LendingUSA与Scrut弃用前沿AI以削减失控的Token成本 小型企业将基础模型视为大宗商品化的公用工具，并将工作流路由至成本更低、规模适宜的架构，从而在与大型对手的竞争中建立运营优势。 技术专家们分享了在精简型组织中部署AI的实用策略，涵盖受监管借贷业务中的自主语音智能体、物流中的库存差异快速分析以及自动化安全分类分流。与会者强调了底层数据治理、持续模型评估（“evals”）以及严格适配模型规模，以避免失控的Token开支。他们还深入剖析了自研与外购之间的权衡，对“氛围编码”（vibe-coded）内部工具背后的运维负担提出了警告，并展示了自下而上的原型如何逐步实现传统企业架构的现代化改造。 数据治理与上下文层是 AI 落地的基础前提，将未经清洗的杂乱数据喂给模型只会更快得出错误答案，而严格的模型评估（evals）更是无法外包的核心工作。 (0:04) LendingUSA 通过将真实共情与后端工具调用相结合，利用自主语音 AI 代理承接了 40% 的呼入量，在严苛金融监管下实现了端到端业务办理。 (6:37) Conectiv 等中小企业利用 vibe coding 快速构建定制集成与粘合逻辑，但专家指出用其取代 SAP 或 Oracle 等核心企业级系统的“SaaS 末日论”并不现实。 (11:37) 企业正从盲目的“刷 token”实验转向严格的投资回报率（ROI）考核，生产架构普遍通过黄金数据集与模型路由将常规任务分流至轻量模型，从而压降成本。 (17:38)

According to technology leaders from Conectiv Supply Chain Solutions, Scrut Automation, and LendingUSA, small and medium-sized businesses achieve sustainable AI adoption not by pursuing expensive frontier models, but by enforcing strict data hygiene, right-sizing model infrastructure, and targeting high-friction operational workflows. Data Hygiene and Continuous Model Evaluation (0:04) Brian Docili of Conectiv emphasizes that feeding unrefined operational data into machine learning systems solves nothing: it merely produces "bad answers faster." Rather than expecting frontier models to compensate for fragmented information, Conectiv focuses on data lineage, clean pipelines, and a structured context layer to ensure downstream accuracy. At Scrut Automation, CISO Nicholas Muy regards internal evaluations ("evals") as the mandatory "sweat equity" required to validate governance, risk, and compliance agents across frameworks like SOC 2 and ISO 42001. To safeguard production environments, Scrut continuously benchmarks incoming model versions against proprietary "golden datasets" before deployment. Strict evaluation is equally vital in consumer finance; LendingUSA CTO Manoj Mathew notes that under Consumer Financial Protection Bureau oversight, customer interactions are subject to audits up to eight years later. Deploying non-deterministic models in regulated contexts requires architecting deterministic guardrails that satisfy stringent compliance and risk scrutiny. Measurable Operational ROI in Production Workflows (6:37) The panel distinguishes between soft productivity gains, such as automated meeting notes, and hard operational ROI tied to measurable business performance. LendingUSA integrated conversational voice AI to manage 40% of its 10,000+ daily inbound calls. Rather than functioning as static routing menus, these agents execute authenticated backend tool calls to process borrower payments and grant payment holidays. The agents also detect caller sentiment, offering empathetic responses to personal distress while maintaining full regulatory compliance. In physical supply chain management, Conectiv deployed the operational intelligence platform B/E across millions of square feet of warehouse space. The system reduced root-cause analysis for inventory discrepancies from three hours of manual spreadsheet exports to mere seconds. By isolating upstream transaction errors and training gaps, Conectiv achieved an immediate 20% labor reduction across its inventory analysis team within two weeks of rollout, demonstrating clear payback without an explicit downsizing objective. The Maintenance Trap and Build-versus-Buy Realities (13:03) AI-assisted "vibe coding" allows small teams to bypass integration platform bottlenecks and build custom connective tissue in days rather than months. However, Docili dismisses the notion of an imminent "SaaS apocalypse," asserting that lightweight coding assistants will not replace foundational enterprise resource platforms like SAP or Oracle. Muy introduces an operational ownership test for teams considering custom AI development: organizations must explicitly designate who will maintain the code, monitor uptime, and handle production errors. Non-technical departments such as revenue operations should not find themselves on midnight PagerDuty rotations simply to avoid commercial SaaS licensing fees. Mathew observes that executive disconnect often exacerbates this friction when leadership builds an unvetted weekend prototype and expects engineering to deploy it company-wide without accounting for edge cases, security, or enterprise SLAs. Model Right-Sizing and Commodity Infrastructure (22:27) Mathew warns against "token-maxing" cultures—popularized by large tech enterprises—where internal incentives reward high token consumption, often exhausting annual cloud budgets within months. SMBs hold an execution advantage because they can make cross-departmental decisions quickly, but they lack the capital to absorb runaway API invoices. Mathew illustrates this dynamic using the historical adoption of refrigeration: while dozens of compressor manufacturers emerged, enduring value was captured by consumer beverage brands like Coca-Cola. For businesses, underlying models are becoming a commodity utility, and lasting advantage comes from building reliable customer-facing workflows on top of cost-effective infrastructure. Consequently, standard workflows rarely require 1-million-token context windows or multi-trillion-parameter frontier models. While open-weight models like Qwen, GLM, or Kimi account for a growing share of inference, Muy points out that self-hosting massive models locally can demand prohibitive resources—such as 1.54 terabytes of GPU VRAM for Kimi K3. For regulated SMBs, using managed cloud infrastructure like AWS Bedrock or equipping developer machines with localized, quantized coding models balances data privacy with low operational overhead. The panelists conclude that sustainable AI adoption requires organizational discipline, where employees are empowered to build utilities within clear data boundaries but remain free to challenge unreliable tools. Lean teams can often drive digital transformation from the bottom up, using focused AI prototypes to bridge legacy architectures without triggering high-risk, top-down core migrations.

来自物流、合规与金融科技领域的中小企业（SMB）技术负责人指出，中小企业的核心优势在于敏捷的决策速度，但AI部署必须摆脱盲目的“刷Token”（Token-maxing）热潮，回归数据治理、严格评估与明确的投资回报率（ROI）。 数据清洗与评估严谨性奠定落地方案 (0:04) 部署AI的首要瓶颈在于底层数据质量而非模型参数规模。Conectiv供应链解决方案公司的Brian Docili指出，将混乱的数据喂给模型只会加速产生错误答案；该公司通过梳理数据血缘和构建结构化上下文层，并引入运营智能平台B/E，使原本长达3小时的人工Excel根因分析缩短至数秒，部署两周内使库存分析人工工作量减少20%。 在合规与安全领域，Scrut Automation的Nicholas Muy强调，对治理、风险与合规（GRC）智能体进行严密评估（Evals）是无法外包的基本工作。Scrut在团队从20人扩张到200人的过程中，利用内部智能体自动分流源码漏洞与安全告警，同时在生产中使用“黄金数据集”持续基准测试新模型，确保更新透明并符合企业合规要求。 在强监管业务中构建具共情力的自主系统 (6:37) LendingUSA的Manoj Mathew指出，尽管高盛调查显示76%的中小企业日常使用AI，但仅有14%将其整合到核心业务流程中。金融科技企业面临美国消费者金融保护局（CFPB）长达5至8年的严格审计要求，无法直接依靠不可控的大语言模型。 为此，LendingUSA建立了企业级智能体编排层，其语音AI目前自主承接全天40%的进线呼叫（公司日呼叫量超1万次，流经1至1.5TB数据）。该语音系统不仅能通过后端工具调用处理借款人还款和宽限期申请，还能对客户的失业或丧亲等情绪精准识别并给予共情回应，证明AI可在兼顾严苛监管的同时深入企业核心业务。 自研与外购平衡及“Vibe Coding”维护陷阱 (11:37) 针对自研还是采购商业软件，Docili表示AI辅助编程（Vibe coding）如OpenAI Codex能够绕过流程漫长的iPaaS集成服务商，将数周至数月的定制系统集成压缩至数天完成，但他明确否定了AI会迅速颠覆SAP或Oracle等传统核心企业SaaS的“SaaS末日论”。 Muy从IT运维角度提醒，非技术团队利用AI自研工具时往往忽视长期维护成本。如果销售运营或业务人员为了省下商业软件许可费而自建脚本，当工具在生产中崩溃时，业务团队将被迫承担半夜排障与PagerDuty告警的责任。组织必须在立项前明确所有权，避免低估自建软件的技术债务与隐性运营成本。 模型选型降本：右移轻量模型与开源权重应用 (22:27) 企业AI已从无节制消耗算力转向严格计算硬性与软性ROI。Docili指出日常场景无需百万级上下文窗口，塞入过量Token反而会增加输出的不确定性。Mathew警告，部分大型企业曾因盲目激励消耗Token导致年度IT预算在5月前耗尽，而中小企业必须对成本保持高度警惕。 在模型选型上，Mathew提出“冷藏技术类比”：制冷机最终会商品化，真正的长期价值属于可口可乐等构建在底层技术之上的品牌。由于顶尖闭源模型成本高出5至6倍，企业应根据吞吐量做出选择：低调用量直接采用托管云API，高并发下则在私有云中部署DeepSeek、Qwen等开源权重模型；但Muy补充提醒，直接自建超大规模未量化开源模型面临极其庞大的GPU显存门槛（如Kimi K3需约1.54TB显存），中小企业更务实的做法是在受控云沙箱或本地量化环境下运行适配模型。 打破周末原型幻想与自下而上的架构现代化 (37:44) 在企业内部落地中，管理层常因周末自行利用AI拼凑出Demo而向技术团队施压，Mathew称这种“周末原型与生产现实的脱节”极易演变为内部矛盾。粗糙的演示与符合高可用、低延迟及合规要求的生产系统存在巨大鸿沟，技术负责人必须依据行业失败案例守住架构与安全红线。 面对许多传统金融机构仍运行在COBOL大型机和早期RPC调用之上的现状，自上而下的核心系统重构往往因高风险而搁置。Mathew与Muy建议，精简的工程团队应利用明确的数据访问护栏，通过轻量级原型自下而上打通传统架构与现代AI接口，以可见的业务价值推动系统有机迭代，而非盲目推进全盘替换。 中小企业部署AI的制胜关键不在于追求最大的模型参数或盲目全盘自研，而在于严守数据底座、设定明确的维护权责并选择成本匹配的轻量基础设施。随着底层模型逐步商品化，如何在受控的合规边界内沉淀专属业务逻辑与客户信任，仍是中小企业需要持续解决的核心课题。

## Background 该内容由行业媒体与大会组织者 Ai4 于 2026 年 8 月 22 日发布在 YouTube 平台上。视频记录了一场题为《小公司，大工具：中小企业如何借助 AI 赢取先机》的圆桌论坛，来自 LendingUSA、Scrut Automation、Conectiv Supply Chain Solutions 等企业的技术管理者与《Inc.》杂志资深撰稿人共同探讨了中小企业如何以精简成本落地 AI 并跑出明确的 ROI。 ## TL;DR 圆桌嘉宾围绕中小企业（SMB）落地 AI 的实战路径展开深度探讨，指出数据质量与上下文层是发挥 AI 效能的前提。分享涵盖了内部安全运营自动化、全自主情感语音客服、实体仓储智能排障等真实落地场景，并深入剖析了自研与外采边界、模型分流选型策略（如闭源与开源开放权重模型权衡）及 ROI 考核。与会专家强调，中小企业的核心优势在于敏捷决策，应避开盲目堆砌 Token 和过大上下文的误区，警惕“周末 Vibe Coding”原型与生产级交付的鸿沟，通过自下而上的轻量试点在强监管与遗留系统中实现智能化破局。 ## Atmosphere & style 讨论整体务实、坦诚且极具工程实战色彩，彻底摒弃了市场上泛滥的空洞炒作。嘉宾们以幽默自嘲的口吻揭示了高管周末突击编程带来的管理阵痛以及大厂烧光 Token 预算的行业怪相，形成了一种同行间“传授血泪教训”的共鸣感。技术负责人们直言不讳地指出玩具原型与企业级合规交付的鸿沟，全场节奏紧凑、干货密集，洋溢着中小团队凭借敏捷性与清醒认知挑战行业巨头的实干底气。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=fW0-hckKB4U&t=4s) 数据质量与数据血缘是中小企业应用 AI 的基石，避免“垃圾进，垃圾出”，构建丰富的上下文层才能保证 AI 输出的准确度与业务价值。 - [04:04](https://www.youtube.com/watch?v=fW0-hckKB4U&t=244s) 为 AI 智能体构建严密的基准评测机制（Eval）是承受高风险运营与安全负荷的前提，不存在开箱即用的银弹。 - [06:37](https://www.youtube.com/watch?v=fW0-hckKB4U&t=397s) 在强监管的金融场景下，具备同理心感知与深层工具调用能力的 AI 语音客服已能实现 40% 通话的端到端全自动闭环处理。 - [09:46](https://www.youtube.com/watch?v=fW0-hckKB4U&t=586s) 供应链与实体运营通过引入 AI 运营智能平台，能将库存差异排查从 3 小时缩短至数秒并直击业务根因，显著释放人力负荷。 - [11:37](https://www.youtube.com/watch?v=fW0-hckKB4U&t=697s) AI 辅助编码能将系统集成周期从数月压缩至数天，但自研系统必须在前期明确后续的运维归属与生产环境告警响应机制。 - [15:14](https://www.youtube.com/watch?v=fW0-hckKB4U&t=914s) 中小企业落地 AI 正迎来四大技术红利：自主执行智能体、高性价比模型组合、标准化连接协议（MCP/A2A）以及拟真人语音技术。 - [17:41](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1061s) 企业正告别盲目追求 Token 消耗的狂热，转向务实的 ROI 考核，日常高频流量应优先路由至轻量低成本模型，生产环境模型需经黄金数据集严格评测。 - [22:27](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1347s) 中小企业无需盲目追求百万级的超大上下文窗口，决策链条短、敏捷落地与快速迭代是小团队压制大型企业官僚体制的核心竞争壁垒。 - [24:03](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1443s) 开源与开放权重模型调用量正在快速上升，在高吞吐与高合规场景下能显著降低长期边际成本并保障数据隐私。 - [31:10](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1870s) 底层基础模型正逐步大宗商品化（如同制冷技术），中小企业的破局之道在于打造垂直领域的解决方案与高价值商业应用（如同可口可乐）。 - [34:11](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2051s) 企业推行 AI 需建立双向信任文化，在鼓励业务部门探索的同时设立安全护栏，并包容对场景适用性与数据安全的理性质疑。 - [37:44](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2264s) 管理者单人利用 AI 拼凑出的周末玩具原型与正式投入生产环境有着巨大鸿沟，不可被单机 Demo 盲目带偏生产节奏。 - [41:30](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2490s) 面对传统大型机与老旧 RPA 系统的历史包袱，中小团队应依靠精悍特遣小组利用 AI 工具快速打样，以局部胜利自下而上推动现代化改造。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=fW0-hckKB4U&t=4s) 嘉宾介绍与中小企业部署 AI 的核心前提 本次圆桌讨论围绕中小企业（SMB）如何以聪明且具成本效益的方式应用 AI 展开。来自不同行业的运营管理者与技术负责人首先介绍了各自的业务背景及当前的 AI 落地战略重点。 ```mermaid flowchart TD accTitle: 中小企业部署 AI 的三大核心支柱 subgraph DataFoundation["数据与上下文基建"] A["数据血缘与数据治理"] --> B["构建丰富上下文层 (Context Layer)"] end subgraph GovernanceEval["治理与评估机制"] C["模型严格评测 (Eval)"] --> D["GRC 合规智能体落地"] end subgraph BusinessExecution["业务场景智能化"] B --> E["精准业务输出与决策分析"] D --> E E --> F["端到端智能客户服务与库存诊断"] end ``` 来自田纳西州孟菲斯的第三方物流与制造企业 Conectiv Supply Chain Solutions 的 IT 基础设施与运营副总裁 Brian Docili 指出，公司在实体媒介分销及各类产品供应链履约方面拥有庞大业务。在实践中他们深刻体会到：“如果把糟糕的数据放在 AI 面前，根本解决不了任何问题，只会让你更快得到错误的答案。”因此，Conectiv 当前的首要任务是夯实底层数据质量与数据血缘（Data Lineage），完善数据管道，并构建一个丰富上下文层（Context Layer），从而让 AI 模型在获取数据时能够输出更精准、更高质量的信息。 合规自动化平台 Scrut Automation 的首席信息安全官（CISO）兼平台工程副总裁 Nicholas Muy 介绍，其团队为 2000 多家中小型及中端市场企业提供 SOC 2、HIPAA 以及针对人工智能管理体系的 ISO 42001 等合规自动化支持。对于 Scrut 而言，过去三年他们持续评估各类模型，当前的核心工作是确保面向客户的治理、风险与合规（GRC）智能体拥有极高的质量水准，并构建必须亲力亲为、无法假手他人的上下文层。 消费金融科技公司 LendingUSA 的首席技术官（CTO）兼首席风险官（CRO）Manoj Mathew 分享了其横跨银行业与创业的丰富经历。他曾是全球核心银行系统 Finacle 的首席架构师之一，该系统服务全球超过 12 亿人口。LendingUSA 迄今累计撮合与处理了约 120 亿至 130 亿美元的贷款，发放了约 15 亿美元的消费贷款，合作商户超过 10,000 家，并正进军加密货币借贷业务。Manoj 强调，本次分享不带有任何推销目的，而是希望把中小企业在资源有限、经历多轮融资与试错后摸索出的“血泪教训”和实战心得分享给大众。 - **Conectiv 的首要原则**：避免“垃圾进，垃圾出”（Garbage in, garbage out），通过打磨数据管道与上下文层确保 AI 回答精准度。 - **Scrut Automation 的专注方向**：持续推进 ISO 42001 等 AI/GRC 合规标准，坚持对模型进行深度评测与上下文架构自研。 - **LendingUSA 的业务规模**：累计处理 120亿–130 亿美元贷款流水，支撑全美超万家商户与债务重组服务。 --- ### [04:04](https://www.youtube.com/watch?v=fW0-hckKB4U&t=244s) 内部运营防御：通过安全与告警智能体打破团队规模天花板 在谈及具体落地的 AI 项目时，主持人 Jennifer Conrad 提及 Scrut Automation 曾荣登《Inc.》杂志的 2024 年度最佳企业（Best in Business）与 2025 年度实力合作伙伴（Power Partners）榜单。Nicholas Muy 重点分享了 Scrut 如何利用 AI 强化内部运营安全。 Scrut 整个公司仅有约 200 人，保持着高度精简的组织架构。Nicholas 从团队只有 20 人时便加入公司，随着客户规模急剧攀升，各种基础设施安全、源代码安全审查、运行时 Web 应用漏洞排查，以及海量内部 IT 安全告警如潮水般涌来，这类事件的体量以往只有大型集团才会遇到。尽管采购了多款一流的安全工具，但极度精炼的安全团队依然难以招架。 为了在不大幅增加人手的前提下扩充产能，团队开发了一系列内部安全智能体，专门分担源代码安全检查、运行时缺陷分类与日常告警响应等工作。Nicholas 直言，起初大家抱着“直接把工具插上就能用”的幻想，但后来经历了漫长的测试与反思。他认为，给 AI 智能体建立严密的输出评测机制（Eval）就像是在为系统注入“汗水权益”（Sweat Equity），只有把基准评测做扎实，内部智能体才能真正承接起高风险的安全与运营负荷。 - **组织挑战**：从 20 人扩张到 200 人规模期间，业务与安全事件量呈指数级增长，团队编制却严格受限。 - **智能体覆盖场景**：源代码安全性审查、Web 运行时 Bug 处理、IT 安全技术栈告警分流与研判。 - **核心认知**：严密的内部 Eval 是保障智能体稳定输出的前提，不存在开箱即用的银弹。 --- ### [06:37](https://www.youtube.com/watch?v=fW0-hckKB4U&t=397s) 高度监管环境下的破局：全自动化、带同理心的端到端语音客服 Manoj Mathew 探讨了高度受规管的金融行业在引入非确定性（Non-deterministic）AI 工具时面临的巨大阻力。LendingUSA 每天需要处理超过 10,000 通客户电话，每天产生并流式传输逾 1.5 TB 数据——这相当于公司创业初期两年的数据总量。 金融科技行业受到美国消费者金融保护局（CFPB）和各州发卡银行的严格监管，任何外呼电话或客服对话在五到八年之后都有可能被监管机构调取审查。传统上，由于 AI 具备非确定性特征，医疗、保险、金融等行业的 AI 项目极易在委员会评审中被扼杀，或是上线后因违规风险被紧急叫停。中小企业必须用有限的预算与时间跑出明确的 ROI。 ```mermaid flowchart LR accTitle: LendingUSA 端到端全自主 AI 客服架构 User(["客户来电 (日均1万+通)"]) --> VoiceAgent["AI 语音客服智能体 (全天候工作)"] subgraph CoreEngine["实时决策与工具链"] VoiceAgent --> ToolCalling["工具调用 (Tool Calling) 核心系统"] VoiceAgent --> Empathy["情感感知与同理心回应机制"] end ToolCalling --> Actions["执行扣款 / 申请延期还款 / 查询账户"] Empathy --> Actions Actions --> Output(["端到端完成业务闭环 (覆盖率40%)"]) ``` 面对这一痛点，LendingUSA 成功构建并上线了分工明确的 AI 语音客服系统： - **全天候业务接管**：设有工作日班次与非工作时间班次的 AI 智能体，目前已全权处理公司 **40% 的客服通话**。 - **深层业务闭环而非简单路由**：智能体不仅仅负责问好或转接人工，而是通过工具调用（Tool Calling）直接深入核心业务系统完成实际服务，例如发起还款扣款、办理还款宽限期（Payment Holidays）以及查询账单流水。 - **情感计算与同理心表达**：当借款人在电话中透露个人困境（如“我母亲去世了”、“我失业了”或“我的宠物狗去世了”）时，智能体能够感知情绪并给出得体的同理心回应，随后自然引导回业务流程。 这一变革极大提升了用户体验。Manoj 透露，如今很多客户致电时甚至会主动要求：“我能直接和 Jen（AI 智能体名字）沟通吗？”他们反而更倾向于与没有情绪负担、响应敏捷的 AI 智能体交流。 - **数据通量**：日均处理电话 10,000+ 通，每日流式处理数据量超 1.5 TB。 - **自动化成效**：实现 40% 通话由 AI 全程闭环处理，支持直接收付款与延期申请。 - **用户反馈**：客户主动指名要求与 AI 客服沟通，形成体验正向循环。 --- ### [09:46](https://www.youtube.com/watch?v=fW0-hckKB4U&t=586s) 实体仓储与运营智能：利用 BE 平台破解库存差异与人力瓶颈 Conectiv 的 Brian Docili 分享了实体供应链领域的 AI 提效案例。Conectiv 运营着数百万平方英尺的仓储空间，货物堆满仓库，实时精准掌握货品位置与库存状态是企业运营的生命线。 在过去，仓储管理系统（WMS）虽记录了海量数据，但缺乏即时洞察。业务团队往往需要耗费大量人力从系统中导出报表、导入 Excel 并展开繁重的手工比对，为了排查某一处库存差异（Inventory Variance）的原因，单次分析常常耗费长达 3 个小时。 为了解决盘点与库存管理痛点，Conectiv 引入了一款名为 **BE**（字母拼写为 B/E）的 AI 驱动型运营智能分析平台： - **诊断耗时从小时级压缩至秒级**：原本需要 3 小时的复杂排查如今可在数秒内完成。 - **根因深度透视**：系统不仅能提示“库存数据错误”，更能直接指出背后的业务诱因——究竟是上游事务录入错误、盘点未完成，还是操作人员的操作培训存在漏洞。 - **立竿见影的效能释放**：项目上线前并未预设裁员目标，但在部署仅两周内，库存盘点团队便释放了 **20% 的工作量**。 Conectiv 计划将该分析平台进一步推广至客户服务管理以及整个制造运营的更多业务节点中。 | 评估维度 | 传统 WMS + Excel 分析模式 | 引入 BE (AI 运营智能平台) 模式 | |:--- |:--- |:--- | | **单次差异排查耗时** | 约 3 小时人工手工比对 | 数秒内全自动定位并输出分析 | | **诊断深度** | 仅能发现账实不符的表面现象 | 直击根因（上游交易异常、盘点残缺或人员培训缺失） | | **团队工作量变化** | 长期深陷报表下载与重复劳动 | 部署两周内库存团队工作负荷直接降低 20% | | **未来扩展场景** | 难以泛化至其他管理链条 | 快速向制造全流程与客户服务管理环节拓展 | ### [11:37](https://www.youtube.com/watch?v=fW0-hckKB4U&t=697s) 自研还是外采：Vibe Coding 的落地场景与运维隐患 面对市面上琳琅满目的现成商业软件（COTS）与新兴的 "Vibe Coding"（借助 AI 进行直觉式敏捷编码），企业应当如何在“自研”与“外采”之间划定界限？针对主持人 Jennifer Conrad 的提问，Conectiv Supply Chain Solutions 的 IT 基础设施与运维副总裁 Brian Docili 指出，决策的核心在于清晰定义业务诉求。在系统集成领域，过去企业不得不完全依赖 iPaaS 供应商，受限于服务等级协议（SLA）和繁琐的审批流，在不同系统间搭建一条完整的数据链路往往耗时数周甚至数月；如今，团队通过结合云端工具与 OpenAI 的代码生成能力（如 Codex），只需几天即可快速构建出易维护且具备自文档化特性的集成链路。不过，Brian 也明确反驳了业内流行的极端言论，直言所谓的“SaaS 灭绝论（SaaS apocalypse）”荒谬至极——现实中没有任何一家企业能单靠 Vibe Coding 就彻底抛弃并替换掉底层的 SAP 或 Oracle 核心系统。 ```mermaid flowchart LR accTitle: 自研与外采决策路径 subgraph S1["需求评估"] A["明确业务痛点与范围"] --> B{"是否有现成产品匹配?"} end subgraph S2["执行策略"] B -- "完全契合且性价比高" --> C["直接采购成熟 SaaS"] B -- "存在特定集成缝隙" --> D["采用 Vibe Coding 自研链路"] end subgraph S3["运维考量"] D --> E["明确维护权与告警责任"] end ``` Scrut Automation 的 CISO 兼工程副总裁 Nicholas Muy 进一步从基础设施与安全支持的角度展开分析。在营销、销售等非工程部门采购了大量第三方工具后，连接各系统的“粘合层（connective tissue）”依然严重匮乏。每当协助非技术部门评估自研工具时，Nicholas 都会抛出关键的“百万美元问题”：这项工具是仅供销售副总裁个人使用，还是支撑整个销售运营（Sales Ops）甚至营收运营（RevOps）？它需要全天候不间断运行吗？一旦系统崩溃，究竟由谁来修复 Bug、响应需求？他幽默地调侃道：“当工程部门之外的业务团队为了省下一万美元外采费用而自行搭应用，结果半夜系统挂掉、全员被 PagerDuty 告警叫醒时，那可真是一个‘美丽新世界’。” - **集成周期缩短**：借助 AI 辅助编码，原本需耗时数周至数月的系统集成流压缩到了数天以内。 - **自研评估准则**：自研前需明确核心三问——使用受众是谁、运行频次与持续性要求如何、后续由哪个团队承担运维及生产环境故障响应。 --- ### [15:14](https://www.youtube.com/watch?v=fW0-hckKB4U&t=914s) 中小企业落地 AI 的四波技术红利与认知鸿沟 LendingUSA 首席技术官 Manoj Mathew 引用了高盛（Goldman Sachs）针对全美 10,000 家中小企业的调查数据：虽然有 76% 的企业在日常工作中使用 AI 工具，但真正将其应用于核心主营业务的仅有 14%。为了弥合这一巨大的认知与应用鸿沟，Manoj 梳理了过去一年中正在赋能中小企业的四大技术演进方向： ```mermaid flowchart TD accTitle: 中小企业 AI 落地的四大技术支柱 subgraph G1["技术红利"] A["多步骤执行智能体<br/>(Agentic Nodes)"] B["高性价比模型组合<br/>(Haiku 等轻量模型)"] C["标准化连接协议<br/>(MCP / A2A 工具调用)"] D["拟真人语音 AI<br/>(24/7 自动化电话与预约)"] end subgraph G2["业务成效"] A --> E["实现复杂自动化业务决策"] B --> F["大幅降低推理与 Token 成本"] C --> G["打通多 Agent 记忆与数据交互"] D --> H["突破营业时间限制获取增量线索"] end ``` 第一是能够承载多步骤任务的自主智能体（Agentic nodes）。AI 不再只是被动问答的聊天机器人，而是能自主根据市场供需波动实时动态调价（宛如 Uber 的动态定价引擎），实现 24/7 自主运转。第二是模型性价比与智能成本的大幅下探，中小企业绝大多数业务场景根本不需要一味追逐最昂贵的前沿模型（Frontier models），轻量级模型（如 Claude Haiku 等）已完全够用。第三是连接器与交互协议的标准化，随着模型上下文协议（MCP）和 Agent-to-Agent（A2A）工具调用标准的普及，不同智能体之间的数据交换与跨系统记忆共享变得空前顺畅。第四则是拟真语音 AI（Voice AI）的突破，这使得传统的街角夫妻店或牙科诊所即便在晚间 6 点闭店后，也能通过极具真人质感的语音代理全天候自动接听咨询并完成预约挂号。 - **高盛调研数据**：全美 10,000 家中小企业样本中，76% 日常使用 AI，但仅 14% 将其用于核心业务流。 - **落地形态升级**：从单一 Prompt 对话转向全天候自主执行的多步骤 Agent，结合拟真人语音实现营业时间之外的业务拓展。 --- ### [17:41](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1061s) 告别盲目堆 Token：衡量 ROI 与基于黄金数据集的模型分级分流 讨论转向企业在 AI 上的预算分配与投资回报率（ROI）。主持人 Jennifer Conrad 提到，与半年前业内狂热追求“Token 最大化（token maxing）”不同，当前业界与《Inc.》杂志读者最关心的核心指标已变为切实可行的 ROI；此前甚至有报道称亚马逊在单月消耗达 5 亿美元 Token 后关停了特定 Leaderboard，微软、Salesforce 与 Meta 也在重新审视 AI 资本开支并倾向于将任务分流至轻量或低成本模型。 | 评估维度 | 生产力辅助工具（Productivity Tools） | 业务运营级系统（Operational Workflows） | |:--- |:--- |:--- | | **主要应用场景** | 项目会议纪要生成、Action Items 提取、日常邮件撰写 | 客户合规审查、核心业务路由、高频生产交易流 | | **评估考核指标** | 员工工时释放、主观工作效率提升（难以精确折算硬性美元） | 严格的硬性 KPI、基准测试（Benchmarking）、PoV 价值验证 | | **模型选型策略** | 严禁滥用顶尖大模型，优先指派轻量级小模型处理 | 经“黄金数据集”评测流水线严格筛选的高可靠模型 | | **项目存废准则** | 低成本覆盖日常流程，形成常态化办公辅助 | 定期追踪量化回报，若未达预期则果断弃用（Fail Fast） | 针对如何在不同代际和体量的模型间做权衡，Brian Docili 强调必须回归解决业务问题的本质，在立项之初做好 Baseline 基准测试并明确 KPI 指标。先通过价值验证（Proof of Value, PoV）检验其实际产出，若无法证明价值则必须践行“快速失败（fail fast）”原则果断放弃，切莫强行将 AI 塞进所有场景。他特别指出，生产力工具（如自动整理会议纪要与待办项）的核心收益在于为员工释放工时，很难折算为硬性美元 ROI；但对于业务运营系统，则必须设定硬性指标并严格追踪，行则规模化推广，不行则立刻叫停。 Nicholas Muy 则从安全与合规视角分享了 Scrut 的工程实践：对于每一个面向客户的关键场景，团队都建立了“黄金数据集（golden datasets）”评估流水线，持续对新发布模型的性能、安全性与合规性进行基准评测。由于公司承担着客户的数据治理与合规责任，任何推向生产环境的模型变动都必须在客户保障文档中明确标注其托管环境（如 AWS Bedrock 或自建部署）与输出防护栏（Guardrails）。在日常实际调用中，绝大部分流量都被严格路由到最小、最便宜且响应更快的模型。Nicholas 笑着透露，作为企业采购负责人，他经常提醒内部团队：“绝大多数日常外发邮件根本不需要动用顶级大模型（如 Opus），大家在个人使用时往往习惯默认最高配置，但在企业产品架构中必须保持极高的成本与性能纪律。” - **合规审计要求**：生产环境模型迭代需全链路追溯模型托管位置、控制权归属以及输入输出安全围栏。 - **流量分流纪律**：面向日常高频任务优先采用低延迟、低成本轻量模型，杜绝高端模型滥用。 ### [22:27](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1347s) 上下文窗口迷思与中小企业的速度红利 在探讨模型参数与上下文长度时，嘉宾们指出，解决日常客户问题或常规业务服务根本不需要动辄 100 万 token 的超大上下文窗口（Context Window）。盲目往提示词里塞进上百万 token 去回答简单问题，不仅无益，反而会增加模型输出不确定性与失控的风险。除非团队对预期结果有极致的清晰度并配合严格的强化学习（Reinforcement Learning），否则盲目迷信“越大越好”毫无收益。开发者完全可以在笔记本上打开 OpenRouter，亲手测试各种模型，无需大动干戈将其变成耗资巨大的全公司级工程。 ```mermaid flowchart LR accTitle: 中小企业与大型企业在 AI 决策上的对比 subgraph SMB["中小企业 (SMB)"] A["产生新想法"] --> B["走过走廊直接拍板"] B --> C["极速落地与迭代"] end subgraph Enterprise["大型企业 / 银行"] D["产生新想法"] --> E["陷入无数委员会评审"] E --> F["决策与交付严重滞后"] end ``` 中小企业（SMB）相较于大型银行或跨国巨头，最大的核心竞争壁垒正是“速度”。大企业往往受困于“死于上百万个委员会（death by a million committees）”的官僚体制；而小公司的团队只需穿过走廊就能当场拍板。AI 的普及成为了行业大平原上的均衡器（Equalizer），只要反应够快，小团队完全有能力与行业巨头正面对抗并推出颠覆性产品。 - **上下文窗口陷阱**：简单任务堆砌超大上下文只会增加输出漂移和不可控风险。 - **验证成本极低**：借助 OpenRouter 等聚合平台，单台笔记本即可完成全模型横向评测。 - **中小企业制胜核心**：敏捷、无阻碍的决策链条是压制大企业繁重官僚流程的唯一关键。 --- ### [24:03](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1443s) Token 消耗激励与开源/开放权重模型的崛起 企业管理中的激励机制往往直接决定员工行为。嘉宾提到，像微软（Microsoft）和优步（Uber）曾有高管将“Token 消耗量”作为考核指标，结果导致用量账单瞬间爆表，甚至有大厂在 4 月份就将整年的 IT 预算彻底烧光，不得不紧急切断对第三方 LLM 的调用。对于没有几亿美金 IT 预算的中小企业而言，防止每月凭空产生 4 万、5 万甚至 10 万美元的 Token 账单是生死存亡的问题。 ```mermaid flowchart TD accTitle: 闭源与开源开放权重模型的架构选型考量 subgraph US["前沿闭源模型 (US Frontier)"] A1["闭源 / 商业 API"] --> B1["开箱即用但调用昂贵"] B1 --> C1["轻量/低频调用场景"] end subgraph OpenWeights["开源与开放权重模型 (Open Weights)"] A2["开放权重 (Kimi K3 / GLM / DeepSeek / Qwen)"] --> B2["下载权重并本地/私有云托管"] B2 --> C2["数据安全合规 + 超高调用量场景"] end ``` 针对成本失控的痛点，开源与开放权重（Open-weight）模型展现了巨大价值。市场上既有美国的前沿闭源模型，也有开源、开放权重乃至采用 MIT 协议发布的模型（如 Kimi K3、GLM-4/5、DeepSeek 系列以及阿里巴巴的通义千问等）。根据 OpenRouter 的真实数据，仅仅一年时间，开源及开放权重模型的调用占比已从个位数的百分比一路飙升至近 60%，正在快速蚕食闭源模型的份额。 - **私有化与合规**：企业无需将敏感数据和 API 请求发送至境外，可直接下载模型权重部署在私有云或本地硬件中。 - **托管成本拐点**：如果仅使用数百万 Token，自建 GPU 节点并不划算；但在重度使用、高吞吐的高并发场景下，自建开源权重模型能大幅拉低边际成本。 --- ### [27:03](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1623s) 选型决策考量：硬件算力、AI 物料清单与供应链冗余 选择开放权重模型时，必须保持理性务实。以未经量化（Unquantized）的超大模型（如 Kimi K3）为例，本地完整加载甚至需要高达 1.54 TB 的 GPU 显存，这绝非在车库里摆几台 Mac Studio 或 DGX 工作站就能轻松解决的。企业应遵循“按需适配（Fit for purpose）”的原则，综合评估以下核心维度： ```mermaid flowchart TD accTitle: 企业 AI 选型与安全合规评估流程 A["评估数据走向与托管方 (Bedrock / 私有云 / 第三方推理)"] --> B["建立 AI 物料清单 (AI BOM) 与安全审计"] B --> C["明确客户与监管要求 (B2B / 强合规行业)"] C --> D{"用量与场景分流"} D -->|"研发日常代码补全"| E["开发者本地部署轻量模型"] D -->|"常规业务交互"| F["中轻量模型推理"] D -->|"复杂推理/多智能体"| G["高阶前沿大模型"] ``` 在日常研发中，许多中小团队的 CTO 倾向于为开发者配置本地运行的代码辅助模型，相比全量调用最新版本的 GPT 或 Claude Code，这种方案不仅大幅降低了成本，还摆脱了对外部平台的过度依赖。中小企业无法享受大型企业级 SLA 保障，在面对前沿实验室突然下线某项服务（如 Fable 5 关停事件）或政策变动时极其脆弱，因此必须在技术架构中制定后备方案（Backup plan）和缓解机制。 - **AI 物料清单（AI Bill of Materials）**：清晰记录模型来源、安全评估（Safety & Eval）负责人，向 B2B 客户与监管机构提供透明的安全背书。 - **技术与合同兜底**：通过合同条款与技术脱敏（Data sanitization）双重保障，确保数据流向可控。 --- ### [31:10](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1870s) “冰箱与可乐”隐喻：在模型商品化时代寻找商业价值 模型本质上就像工业革命初期的“制冷技术（Refrigeration）”。当年无数制造冰箱和制冷设备的公司不断涌现，但最终站在行业顶峰、攫取百倍商业利润的，却是利用制冷技术卖糖浆汽水并成长为数千亿甚至万亿巨头的可口可乐（Coca-Cola）与百事可乐（Pepsi）。 ```mermaid flowchart LR accTitle: 基础设施与应用层商业价值转化 subgraph Infra["AI 基础设施 (底层模型)"] A["大模型 / 开放权重 (制冷技术)"] end subgraph App["高价值商业应用"] B["垂直场景落地 / 解决方案 (可口可乐 / 百事可乐)"] end A -->|"转化为商品化底层能力"| B ``` 基础模型终将彻底成为大宗商品（Commodity）。中小企业不必迷失在底层复杂的技术竞赛中，除了涉及重度推理（Reasoning）和多智能体协同（Multi-agentic）的极其复杂查询外，绝大多数业务根本不需要 2.8 兆（2.8T）参数级别的超级模型。部分海外前沿模型的部署与调用成本往往是开源同级方案的 5 到 6 倍，企业必须进行严格的成本效益分析——在不同国家或渠道买一罐可乐，是否值得为同样的基础体验支付 5 到 6 倍的高昂溢价？选对模型架构、把控部署成本，打造出垂直领域的“可口可乐”，才是小团队实现商业成功的关键。 - **技术定位清晰**：认识到模型的工具属性，避免为用不上的算力与过剩参数买单。 - **成本效益优先**：根据具体业务深度量身匹配部署方案，以最优的单位经济模型（Unit Economics）赢得市场。 ### [34:11](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2051s) 平衡炒作与务实：构建双向信任的 AI 落地文化 在探讨如何在企业内部协调不同利益相关者、安全有序地推进 AI 工具时，Brian Docili 指出，关键在于避免走两个极端。当前市场上充斥着大量对 AI 的狂热炒作，许多人恨不得把所有业务场景都强行套上 AI，但这往往并非最优解。AI 落地的核心在于建立双向信任：如果一线员工不信任某个工具，他们就不会主动去用；而如果管理层缺乏信任，项目就根本拿不到预算支持。因此，团队必须明确究竟想解决什么业务问题，并在内部展开充分沟通。企业不仅要鼓励团队积极探索潜在用例，更要建立健康的“质疑机制”——允许员工坦率地反馈：“这个场景不适合用 AI”、“生成的结果不可靠”或是“这对公司的数据安全有风险”。只要企业在支持创新提案的同时，也包容并重视这些反对意见，AI 的采纳与落地自然会水到渠成。 Nicholas Muy 幽默地补充道：“把账单拿给他们看，通常就能解决 90% 的分歧。”他回顾了大约两年前公司刚起步推行 AI 时的情景：当时 CEO 恨不得所有人立即上手，公司迅速给每位员工开通了 Google Gemini、Claude Code 以及 Cursor Enterprise 等各类前沿工具的权限。然而，单纯分发账号并不能自动产生业务价值，关键在于这些工具究竟如何为业务带来实际成效。企业逐渐走过了早期盲目追求 API Token 消耗量的狂热阶段。Nicholas 观察到，最令人惊喜的创新往往来自于那些以前在技术支持上处于被动地位的业务部门——他们过去总要苦苦等待工程团队排期开发内部小工具，而如今在设定了明确数据连接边界与安全护栏的前提下，他们可以自主构建工具解决痛点。 Nicholas 进一步提到，企业对于 AI 的态度已经发生了成熟的转变。一两年前，很多公司还在热衷于举办黑客马拉松，甚至做大量“思想工作”去安抚员工，生怕大家觉得“饭碗要被抢走而想砸烂笔记本电脑”。如今，员工们在实际使用中体会到 AI 能够切实减轻琐碎负担，让他们专注于真正重要但此前受限于人力和预算而无法开展的工作。只要员工有合理的想法且不危害公司安全，管理层就会全力支持他们放手去尝试。 ```mermaid flowchart TD accTitle: AI 企业落地与信任闭环 subgraph InputStage["需求与探索"] A["业务部门自主探索用例"] B["自下而上反馈安全/可靠性风险"] end subgraph GovStage["管理与安全治理"] C["设立数据源与权限护栏"] D["聚焦真实业务价值与 ROI"] end subgraph OutputStage["规模化成效"] E["摆脱排期依赖，自主开发小工具"] F["员工聚焦高价值核心业务"] end A --> C B --> D C --> E D --> F ``` --- ### [37:44](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2264s) 原型与生产环境的鸿沟：应对来自管理层的“周末 Vibe Coding”压力 Manoj Mathew 指出了当前许多企业内部普遍存在的一大痛点：最对 AI 感到焦躁与急迫的，往往是 CEO 或销售负责人。高管们常常在周末利用最新的 AI 工具自行进行“Vibe Coding（氛围编程）”突击拼凑出一个演示模型，周一便直接甩在技术团队脸上质问：“看，我周末一个人就能做出来，为什么你们 20 个人干了这么久都做不到？”这种自上而下的巨大压力让许多技术人员难以招架。更棘手的是，这往往会演变成一种带有情绪色彩的博弈——因为这是管理层亲手花周末时间搭出来的成果，下属很少有勇气当面提出理性的反驳与质疑。 然而，Manoj 强调，“构建原型与正式投入生产环境之间有着天壤之别”。哪怕是对于首席技术官（CTO）或首席信息安全官（CISO）来说，要去向兴奋的 CEO 解释为什么这个周末原型一旦放进生产环境就会彻底崩溃、产生安全漏洞或无法扩展，也是一件极为艰难且吃力的事情。在金融、借贷等强监管行业尤为如此，现实远比单机 Demo 复杂得多。这正是参加行业大会交流的真正意义所在：企业必须从同行的惨痛失败教训中吸取经验，理性厘清在什么业务环节采用何种方案，切忌被不成熟的玩具原型盲目带偏生产节奏。 --- ### [39:24](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2364s) 现场问答：开源/权重开放模型与商业 Frontier 模型的权衡 在问答环节，一位经常为中小企业提供 AI 赋能咨询的观众提问：在推进落地策略时，由于成本和控制权等考量，是否有必要优先采用开源（Open Source）或权重开放（Open-weight）模型，而非直接接入昂贵的商业前沿（Frontier）模型？专家们在生产环境中部署开源模型的实际经验与成本表现究竟如何？ Manoj Mathew 明确回应，他们团队已经在生产环境中大规模部署了开源与权重开放模型。他建议企业采取务实的验证路径： - **依托云服务商快速测试**：充分利用 AWS Bedrock、Azure 或 Google Cloud 提供的托管 AI 服务，在这些平台上直接调用、对比不同开源模型与商业模型。 - **保护数据隐私与合规**：借助成熟云厂商的原生安全隔离环境，企业无需将敏感数据直接暴露给外部的前沿模型实验室，即可完成私有数据的微调（Fine-tuning）与推理验证。 - **警惕隐性推理成本**：切勿想当然地认为开源/权重开放模型一定比前沿大模型便宜。实际上，私有化部署和算力微调的综合持有成本，在特定场景下可能反倒劣于轻量级商业 API。 - **精细化横向评测**：企业需要将诸如 Claude 3 Haiku 等低成本商业模型，与智谱 GLM、月之暗面 Kimi 或开源权重模型进行同台推理效果与综合成本比对，以找到实际价值最高的平衡点。 --- ### [41:30](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2490s) 现场问答：打破大型机与 RPA 旧系统的枷锁——自下而上的敏捷突围 现场另一位观众提出难题：在大型传统或强监管企业中，核心业务逻辑高度依赖底层遗留系统、RPA 自动化流程以及老旧的数据转储机制，数据流通困难重重；在尝试将语音 AI（Voice AI）等前沿交互接入老旧 RPA 时，集成过程堪称“一场噩梦”，想了解专家有何破局策略。 Manoj Mathew 剖析道，美国企业在数字化进程上面临着“先发优势带来的后发劣势”——大量银行和保险巨头至今仍运行在 IBM 大型机（Mainframe）之上，核心逻辑写在 COBOL 这种古董语言里，市场上极度缺乏懂这类架构的现代开发者。许多内部接口甚至连 REST 规范都没普及，还停留在老旧的 RPC 调用或 SOAP 协议阶段。在面对这套深层历史包袱时，若一味等待高管发起全盘自上而下的数字化重构往往不切实际，因为管理层深知其中的系统性风险，没有人愿意押上自己的职业生涯去冒险推翻核心底座。 破局的关键在于“自下而上的敏捷突围”： - **组建精悍特遣小组**：依靠少数技术骨干组成敏捷突击队，在隔离的边缘环境或微服务层展开试点。 - **利用 AI 工具快速打样**：一线主管和架构师可以借助现代 AI 编程与自动化工具快速完成原型验证，展示在不破坏底层稳定的前提下如何取得同等或更优的业务成果。 - **用局部胜利建立全员信心**：通过一系列小规模但成效显著的轻量级现代化改造案例，逐步证明新架构的敏捷性与可行性，从而形成示范效应，自下而上地推动整个庞大企业的智能化演进。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=fW0-hckKB4U&t=4s) 如果把糟糕的数据放在 AI 面前，根本解决不了任何问题，只会让你更快得到错误的答案。 — Brian Docili > [04:04](https://www.youtube.com/watch?v=fW0-hckKB4U&t=244s) 给 AI 智能体建立严密的输出评测机制（Eval）就像是在为系统注入“汗水权益”（Sweat Equity），只有把基准评测做扎实，内部智能体才能真正承接起高风险的安全与运营负荷。 — Nicholas Muy > [11:37](https://www.youtube.com/watch?v=fW0-hckKB4U&t=697s) 现实中没有任何一家企业能单靠 Vibe Coding 就彻底抛弃并替换掉底层的 SAP 或 Oracle 核心系统。 — Brian Docili > [11:37](https://www.youtube.com/watch?v=fW0-hckKB4U&t=697s) 当工程部门之外的业务团队为了省下一万美元外采费用而自行搭应用，结果半夜系统挂掉、全员被 PagerDuty 告警叫醒时，那可真是一个‘美丽新世界’。 — Nicholas Muy > [17:41](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1061s) 绝大多数日常外发邮件根本不需要动用顶级大模型（如 Opus），大家在个人使用时往往习惯默认最高配置，但在企业产品架构中必须保持极高的成本与性能纪律。 — Nicholas Muy > [22:27](https://www.youtube.com/watch?v=fW0-hckKB4U&t=1347s) 中小企业（SMB）相较于大型银行或跨国巨头，最大的核心竞争壁垒正是“速度”。大企业往往受困于“死于上百万个委员会（death by a million committees）”的官僚体制；而小公司的团队只需穿过走廊就能当场拍板。 — Manoj Mathew > [34:11](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2051s) 把账单拿给他们看，通常就能解决 90% 的分歧。 — Nicholas Muy > [37:44](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2264s) 看，我周末一个人就能做出来，为什么你们 20 个人干了这么久都做不到？ — Manoj Mathew > [37:44](https://www.youtube.com/watch?v=fW0-hckKB4U&t=2264s) 构建原型与正式投入生产环境之间有着天壤之别。 — Manoj Mathew ## Mentioned Manoj Mathew, Nicholas Muy, Brian Docili, Jennifer Conrad, LendingUSA, Scrut Automation, Conectiv Supply Chain Solutions, Inc. Magazine, Finacle, BE, CFPB, Goldman Sachs, Claude Haiku, Claude Code, Opus, OpenRouter, DeepSeek, Kimi K3, GLM, 通义千问 (Qwen), Cursor Enterprise, AWS Bedrock, Azure, Google Cloud, Google Gemini, SAP, Oracle, PagerDuty, Coca-Cola, Pepsi, IBM Mainframe, COBOL ## Video description Manoj Mathew - CTO, LendingUSA Nicholas Muy - CISO / VP of Engineering, Scrut Automation Brian Docili - Vice President, IT Infrastructure & Operations, Conectiv Supply Chain Solutions Jennifer Conrad - Senior Writer, Inc. Magazine Join a group of nimble SMB operators and tech enablers as they share how small companies are using AI to punch above their weight. From virtual assistants and automated quoting to AI-driven marketing campaigns, we’ll dive into specific tools, costs, and outcomes. Panelists will share real lessons—not just hype—on what it takes to deploy AI in a lean business and still see measurable ROI. Perfect for any entrepreneur looking to work smarter, not harder. [Source](https://www.youtube.com/watch?v=fW0-hckKB4U)

[0:04] Brian Docili: 如果把糟糕的数据放在 AI 面前，根本解决不了任何问题，只会让你更快得到错误的答案。 [4:04] Nicholas Muy: 给 AI 智能体建立严密的输出评测机制（Eval）就像是在为系统注入“汗水权益”（Sweat Equity），只有把基准评测做扎实，内部智能体才能真正承接起高风险的安全与运营负荷。 [11:37] Brian Docili: 现实中没有任何一家企业能单靠 Vibe Coding 就彻底抛弃并替换掉底层的 SAP 或 Oracle 核心系统。 [11:37] Nicholas Muy: 当工程部门之外的业务团队为了省下一万美元外采费用而自行搭应用，结果半夜系统挂掉、全员被 PagerDuty 告警叫醒时，那可真是一个‘美丽新世界’。 [17:41] Nicholas Muy: 绝大多数日常外发邮件根本不需要动用顶级大模型（如 Opus），大家在个人使用时往往习惯默认最高配置，但在企业产品架构中必须保持极高的成本与性能纪律。 [22:27] Manoj Mathew: 中小企业（SMB）相较于大型银行或跨国巨头，最大的核心竞争壁垒正是“速度”。大企业往往受困于“死于上百万个委员会（death by a million committees）”的官僚体制；而小公司的团队只需穿过走廊就能当场拍板。 [34:11] Nicholas Muy: 把账单拿给他们看，通常就能解决 90% 的分歧。 [37:44] Manoj Mathew: 看，我周末一个人就能做出来，为什么你们 20 个人干了这么久都做不到？ [37:44] Manoj Mathew: 构建原型与正式投入生产环境之间有着天壤之别。

[0:04] Brian Docili: Putting bad data in front of AI doesn't really solve any problems. It just gets you bad answers faster. [22:27] Manoj Mathew: If you look at a bank or a large enterprise, it's death by a million committees, right? Versus a small company, you can just walk across the hallway and make a decision. And I think that is a great equalizer. [31:10] Manoj Mathew: These models are going to become a commodity... and what's important for each one of us to understand is who becomes the Coke. And with AI, we have the opportunity to become the Coke or the Pepsis of the world to build on these things and not get lost into all the complication that goes on.

[0:04] Thank you and welcome to our panel on how  small businesses can use AI in both smart [0:10] and coste effective ways. Um so to start,  thank you for those great introductions. [0:15] I would love to just go down the line and hear  from each of you a little bit about your roles [0:19] and what your priorities are for using AI  right now. Brian, can you start? Sure. So, [0:25] hi, I'm Brian Dile. I run IT infrastructure  and operations for Connective. Connective is [0:30] a thirdparty logistics and manufacturing company  based in Memphis, Tennessee. We're most known for [0:34] our strong presence in the physical media space,  but we all offer all types of fulfillment and [0:39] supply chain services across all types of products  and industries. Now, our top priority right now, [0:45] I'll tell you, we've kind of worked for a little  while and we've noticed that putting data, [0:49] bad data in front of AI doesn't really solve any  problems. It just gets you bad answers faster. [0:54] So our top priority right now is focused  on really improving our data data lineage, [0:59] data quality, improving our data pipelines,  getting that right. We want to add a rich context [1:05] layer so when we add AI on top of that data,  we get better answers. We get more accurate [1:11] information. So that's really the main focus we're  working on right now. We want to get data right. [1:18] Sure. Yeah. So Nicholas Moy here, CISO for ScrUm.  We're uh we have about 2,000 plus customers in [1:26] the SMB mid-market space that we do compliance  automation for. Um we're doing everything like [1:32] you know SOCK 2 HIPPA and others and ISO 4201  especially and one of our biggest priorities [1:39] has been you know especially for me as a CISO and  VP of engineering for platform is helping the team [1:45] the product team and the engineering team make  sure that what we're putting out there for our [1:49] customers getting governance risk and compliance  agents that have extremely high bar for like the [1:56] quality and getting the eval right and making sure  that what they're getting you know we looked at [2:02] many models and we've been doing it for the last 3  years and using them for you know our new releases [2:08] and seeing the the huge difference that makes  and I I think to your point about the context [2:14] layer and we have to continue to do that and we're  seeing that that investment is something that is [2:20] it's not something someone else is going to do for  us and so that's a big thing where we're really [2:25] focused I think that quality and making sure that  what our customers are getting is exactly what [2:30] they need. Thank you, Nicholas. Um, I'm Manoj  Matthew. I'm the chief uh technology officer [2:37] and the chief risk officer at uh a fintech lending  USA. We have uh trust me, none of us are trying to [2:45] sell our services. I think these are hard lessons  learned that we want to share with the group here. [2:51] Um so we are a fintech. We have uh originated  about 12 uh process about 12 to 13 billion [2:57] dollars worth in loans. um given out about uh $1.5  billion in consumer loans. We are we work with [3:04] about 10,000 plus merchants across the country  almost all states and uh we work with almost all [3:11] debt settlement companies. Um our latest launch  is going to be on crypto lending. U so I think [3:17] um there's there's a lot that goes on on a  day-to-day. My background is from banking. [3:22] I've been one of the chief architects for one of  the largest core banking platforms in the world [3:27] called Phin. It's being used by 1.2 billion  people across the globe. Um being a serial [3:34] entrepreneur. Uh this is my third gig. Uh so a  lot of mistakes, a lot of lessons learned, a lot [3:39] of funding rounds back and forth. Um so I'm I'm  sure I'm looking forward to this session. Um and [3:45] sharing our insights of our uh uh valleys and our  and our events. Thank you. Great. Um and so Nick, [3:52] I should mention that Scrut Automation was on the  Inc. power partners list in 2025 and our best in [3:59] business list in 2024. So with that, why don't we  start with you? I'd love to hear from each of you [4:04] about a specific AI project that you've launched  or in the process of launching. Um and if you tell [4:09] us a little bit about the problem it's addressing  and um its impact. Yeah. So um a big thing for for [4:18] my team has been how do we actually make use of  this for our internal operations whereas you know [4:25] the my colleagues are focused on what we're  doing for our customers. Um we have even for [4:32] us you know we're about a 200 person company.  We're not the biggest company in the world and [4:37] we have to run a very lean organization  to scale to support our customer base. [4:42] One of the big things that we have to deal with  is security for our infrastructure, especially as [4:47] our customers are asking us about how we're using  AI in our product and in the company internally [4:54] both by our users for productivity and for our  product. You know, we're having to handle a volume [5:02] of security issues and events that used to be in  the realm of just big companies. And so we had to [5:10] build internal utilities to handle a lot of the  back and forth that even though we're leveraging [5:15] a lot of say like best-in-class products  and platforms to expand our team's capacity, [5:22] we had to build agents internally to make sure we  can handle our product security problems. So like [5:27] our the security of our source code uh bugs in the  web in the web application at runtime um agents [5:35] that can handle all the alerts that come from  our internal IT security uh stack and having to [5:41] do that with a very small team like I don't have a  huge team. I joined the company when we were only [5:47] 20 people uh and to 200 people now and have barely  grown the team but you know the work hasn't you [5:54] know the work has grown a lot more than the team  and and that's something that has taken some time [5:59] and it's been a lot of test and learn because you  know in the beginning we were like hey we could [6:05] just plug this in and it should be fine that's  what everyone's telling us and and we've gone back [6:10] and learned a lot from that um fortunately and  and and I think that's been a big part and a lot [6:16] of that has been doing doing the evals even for  our internal uh use cases and and really having [6:23] you know we we have to put in that work. I feel  like the eval for a lot of the output you get from [6:28] your agents is the sweat equity you're doing for  your work, right? Um we we've been in business for [6:37] almost 10 years. We started as a we were a startup  just a handful of people with an idea and today we [6:43] are close to 200 people and we handle about 10,000  plus calls a day um process about over a terabyte [6:53] terabyte and a half of data a day um a lot of  these things are streamed um that amount of data [7:00] was generated when we started maybe in 2 years  now it's processed in a day so when you scale [7:07] and we had our call centers, customer service call  centers um outside of the US and we are highly [7:15] regulated. We are regulated by the CFPB. Uh we or  our loans are issued by uh a state uh originated [7:22] bank. So any calls that we make, anything that we  say can be questioned 5 6 7 8 years down the line. [7:31] So especially using AI in a in a non in a  which is non-deterministic how do you use a [7:39] non-deterministic tool in a regulated environment  and that is a huge part of the question that we [7:46] want to solve right when it comes to finance when  it comes to insurance when it comes to healthcare [7:51] these questions are going to regurgitate and  there's going to be apprehensions and many of [7:56] these projects are either going to be killed by  committees or never sees the Okay. Or someone [8:01] launches it and it gets shut down. So the question  that each one of us has to ask is when a small to [8:07] mid-size business has limited resources, limited  time, limited budget, how do we make sure that the [8:13] intent that we have, the ROI, the effort that  we put brings you success. And today I'm happy [8:20] to share right we have 40% of our calls being  handled by AI. We have a a AI agent which works [8:30] uh during office hours, business hours and an AI  agent which works non-b businessiness hours and it [8:36] cers. It's not just taking calls and saying  hello let me transfer you to an agent. It's [8:40] actually providing service. It takes payments.  It it gives you payment holidays, gives you [8:46] information. It dips into the system goes through  a tool calling. If the customer comes and says, [8:52] "Hey, I lost my mom or I I I I lost my job or my  puppy died." There's empathy across there's some s [9:01] of sense of sentient coming along and then it goes  back to the path. So I think there's a lot that [9:07] goes into um as my my my colleagues were saying  um building that layer starting from the data [9:15] layer to building context the orchestration  piece of it and how do you bundle all these [9:21] things to create that kind of an experience which  um an AI uh agent can deliver to a end human and [9:30] then what happens today is customers call us  today and say hey we can I talk to Jen. They [9:36] would rather prefer talking to the AI agent than a  human. And that transformation is super critical. [9:46] So I'll tell you about a tool we've recently been  working on called BI. And for anyone who's writing [9:50] that down, it's B/E. So we're a fulfillment  place, right? We have warehouses, huge, millions [9:56] of square feet. We have products everywhere. It's  critical that we know where they are. So one of [10:00] our challenges was around inventory management and  cycle counting and we deployed this tool called [10:05] BI which is an operational intelligence platform  powered by AI and we kind of let me tell you the [10:12] problem in the past. So the problem in the past  was we have lots and lots of data out of our [10:15] WMS and warehouse systems and we didn't get real  insights out of them. We had large teams having to [10:21] constantly download reports put them in Excel do  a lot of analysis to figure out why inventory was [10:26] wrong. And sometimes this took a while. We could  have three hours spent on a single analysis just [10:32] to figure out what was causing these inventory  variances. And when we put this BI tool in place, [10:37] now the same information could be available in  seconds. So it's made a huge difference to our [10:41] operation. Now we not only know that inventory  is wrong, we can start to see why. Is it an [10:46] upstream transaction? Is it an incomplete count?  Is there a training issue that we need to get our [10:51] team reworking on? And this is providing real  results so far for us. We didn't even go into [10:57] this thinking about headcount reductions, but I'll  tell you that right away within first two weeks, [11:02] we realized we could make a 20 cent 20% reduction  effort on the inventory team because of this tool. [11:09] So, it's providing real value to us and we're  really excited about this case, but we're also [11:13] excited to unleash this tool in other cases  around customer service management. Um, [11:19] and there's a million places how we run our  manufacturing operations. There's going to [11:22] be a lot of places where this tool is going to  provide value and we're very excited about it. [11:26] Great. Um, and I think I'd love to hear from all  of you about which tools to use when because we're [11:32] in a moment where, you know, you walk the expo  floor and there's just, you know, hundreds of [11:37] different off-the-shelf tools you could buy. You  can spin up your own apps with Vibe Coding. Um, [11:44] so when do you when do you build your own things?  When do you buy? And how do you decide what tools [11:49] you're going to use? Brian, do you want to start?  Sure. So I think it really comes down to what are [11:56] you trying to accomplish and you have to have  that clearly down. Sometimes the off-the-shelf [12:01] tool is the perfect solution but sometimes if  the requirements are simple and there's not [12:05] something that's a perfect fit. Vibe coding can  be great. So we've been using vibe coding a lot [12:09] for integrations. So in the past we had to handle  all integrations to our iPass vendor and because [12:15] of the way the SLAs's are and the way just the  overall process flows this can take a while. [12:19] It could take weeks to sometimes months to have a  fully deployed integration between a couple of our [12:24] systems. And now the team using a combination  of cloud tools and vibe coding, which we're [12:29] using codecs from OpenAI, um we've been really  successful at getting integrations built within [12:34] just a few days and they're not hard to support.  They document themselves really easily and we're [12:39] having good success with that. Now, that said,  like I said, I don't believe vibe coding is the [12:44] answer to everything. I think the SAS apocalypse  that everyone talks about is ridiculous. I don't [12:48] think everyone's going to vibe code their way out  of their SAP and Oracle systems and just replace [12:52] everything. But I do think for certain use cases,  um, building it yourself, having your teams use [12:57] vibe coding or an AI agent of some type to do that  is a great use case and you should investigate. [13:03] Thank you. Yeah, for for us I I think you know so  there's the example we've built a lot of security [13:09] utilities internally uh for a lot of departments  because I also support our IT internal IT and [13:17] infrastructure uh a lot the marketing team  the sales team there's a lot of commercial [13:23] off-the-shelf uh products that they're using  whether they're other AI agents or something else [13:28] but then a lot of the connective tissue we've had  to provide internally I think to Brian's point on [13:33] the integ tegration between a lot of these things  still kind of lacking oddly enough in today which [13:39] is weird and then on when what we decide there  there's a lot of things like a big I'm a big fan [13:45] of um for for most of our departments especially  in the non-engineering or like I usually have [13:52] to consult them like hey what are you trying to  solve is this an internal only use case is this a [13:56] utility for either is this for the VP of sales or  is this for the entire sales or is this for sales [14:01] operations who is this or how frequently does this  have to run? Does this have to run continuously [14:07] and who's going to maintain it is kind of the  milliondoll question like and even for us as a [14:13] smaller company, you know, who's going to maintain  it? Who who actually is going to do the bug fixes, [14:19] feature request, is that someone internally? Is  that the sales ops or revops team? Do they want [14:25] to be up and get paged when this goes down? Is it  going to be a production issue? I I was thinking [14:31] the other day, you know, it's going to be a  brave new world when everyone outside of the [14:35] engineering org gets on pager duty and gets paged  when their vibe coded apps go down because they [14:41] didn't want to pay 10K to someone else. So, you  know, and which I I think there's a lot of things [14:47] that are very specific to each company's kind of  context and I think question is is this important [14:52] for you? Is this important for your customer?  Does this help you help your customers more? Um, [14:57] but there are a lot of things that are quite  affordable to just buy off the shelf. I'm a [15:02] big consumer of a lot of that. And I think though  where I would like to see more is like putting it [15:07] together is still a lot harder than it should be.  Thank you. Good point. Um, there was a survey done [15:14] by Goldman Sachs. Uh, they surveyed about 10,000  small to midsize businesses and 76% of them said [15:22] they use AI in their daily daily lives. But only  14% of them use it for their core businesses. So [15:29] there's in in the US there's still a big gap and  this knowledge gap and I I see this room full. I [15:36] think it's important it is critical that we know  that there's a gap to fill and this question [15:41] is really pertinent and if you look at just the  last one year what has shifted what has happened [15:47] there are agents there are agents that can hold  multi-step tasks. What that means is it's not [15:53] just a chat bot. You ask chat GPT a question,  it would respond. You can give it a task. Hey, [16:01] sell this candy at the most optimum price. You  can increase go up and down the price based on [16:09] uh demand, right? It will behave like an Uber  and run 24/7 for you. And these agentic nodes [16:18] are feasible today for your businesses. um the  price versus performance or price or intelligence [16:26] has come down and I think I'm sure we'll talk  more about the uh Chinese models and where and [16:32] how and when we can use I think we can go a lot  more into debt u we may all not everyone may need [16:39] the latest and greatest frontier models like  the fables of the world and things um we may [16:44] be okay with using haikus or other models in our  businesses number three connectors and standard [16:50] standardization the MCPs and A2As the kind of tool  tool calling that's available today and how one [16:57] agent can talk and exchange information and share  memory has standardized now so that is an amazing [17:04] uh direction that the industry is moving  towards and number four is and and one of [17:09] the most important and that can impact uh small  businesses let's say a small dentist office or [17:16] um a small mom and pop show uh store they close  at 6 p.m. but they can be taking calls and [17:21] appointments 24/7. You don't have to shut your  store. The voice AI sounds really like a human [17:29] and take those appointments and do more for you  today. And that is something that we all should [17:33] take off from this this this conference. Great.  So, um I would love to stay on the conversation [17:41] of budget a little longer. Um, you know, I think  if we were having this conversation 6 months ago, [17:46] maybe we'd still be talking about token maxing  and how to get AI into everything. But the term [17:52] I've heard more than anything else, both at this  conference and from Inc. Magazine readers, is ROI, [17:59] return on investment. And when are AI projects  actually worth it? Um, we've seen in recent months [18:04] that Amazon shut down its leaderboard reportedly  after they spent half a billion dollars in one [18:10] month on tokens. Um and Microsoft, Salesforce,  and Meta are all also reportedly rethinking some [18:17] of their spending. Um the companies are starting  to shuttle work to older models. Um so I'd love [18:24] to hear from all of you how you're thinking  through which projects are worth it and what [18:30] level of budget you're going for. Um you know,  when you use a newer model, a frontier model [18:35] versus an older model. Um Brian, would you like to  start? Sure. So, I don't think anyone's surprised [18:42] that we don't necessarily see perfect ROI from  everything AI. AI is very exciting technology, [18:47] but it's also full of hype. So, really, you got  to get back to the basics. Focus on what is the [18:52] business problem you're trying to solve. Make  sure to do benchmarking of that first. You know, [18:56] what what are the things you're going to be  able to improve? Find the KPIs, measure them, [19:01] then try your proof of value, see how it works,  see what's improved it, and see if it really [19:05] did justify the value. And if it doesn't, you  can't be afraid to abandon it. Move on. Fail [19:11] fast. Don't waste time trying to make AI fit into  everything. It's not always the right answer. So, [19:17] um, and that's a big part of our philosophy and  we've tried to use that diligently and I think it [19:23] really makes a difference. Um, I don't have a lot  of comment on different models per se. We're still [19:28] kind of focused on the easy use cases, but I will  say that you have to measure them differently. [19:34] If you're talking about productivity tools, you  can't give them hard dollar figure ROIs. These are [19:39] things about giving employees more time, but these  things do have benefits. Anyone who's used them [19:43] for their project management meetings now sees  that the days of having to take constant diligent [19:47] notes and have gone away. AI does that for you.  It gives you the action items. These are these [19:52] are real benefits even if they're not hard dollar.  But for anything that's operational, you got to [19:57] measure something and you got to have a plan for  what it specifically it's going to deliver. You [20:00] got to measure that and if it doesn't, abandon  it. if it does scale it. These are the things [20:04] that are going to make a difference for your  business. Back to you on [clears throat] Yeah. [20:10] So for every important uh customerf facing use  case that we have, we have golden data sets [20:16] uh for our evals to benchmark the performance of  different models as they get released. So then we [20:21] have an entire pipeline constantly testing, you  know, one model versus another for the ones that [20:26] we've chosen to use. uh because any new ones we  do choose to push to production, we have to update [20:34] our our customer assurance kind of documents  for our customers. uh we have obligations to [20:40] our customers especially because we're supporting  their governance versus compliance on you know our [20:45] entire model pipeline like where is that coming  from where is it hosted who controls it is it [20:51] you know is this in bedrock is this directly with  the labs is something we're hosting locally and we [20:56] have to be able to provide all that and then who's  who's actually providing kind of the guard rails [21:01] on and all that and doing the safety evaluations  not not necessarily on what the model's actually [21:06] doing but on our outputs as Well, and then we're  looking at that and and something we've tried to [21:11] do from the beginning. In the early days, you  know, it used to be like, okay, there's only [21:15] so many models. Let's hit it with this um and see  how it performs and then do the benchmarking. Now, [21:21] we're really trying to for most of I I'd say  the majority of calls on a day-to-day basis, [21:27] we can handle that with the smallest and the  cheapest models. Um we do not try to hit our [21:34] customers. I I think it goes back to like what  do the customers need and what does that business [21:38] problem need most of the time they don't need opus  5 it just doesn't make sense like most of your you [21:46] know the emails you write don't need opus 5 is I  I joke with our internal teams who are I look at [21:51] the usage cuz I I'm the one responsible for all  our enterprise procurement on like all the to the [21:57] you know for every model that we're using I'm like  most people don't need the sales team doesn't need [22:02] fable to like you know send out outbound emails  but it it it's something you'd be surprised like [22:09] people take for granted they just they're like  well this is the default so I I just use this in [22:14] the product we're actually way more disciplined  than we are as individuals where no most of this [22:19] is getting routed to way smaller models they're  also a lot faster and you know these a lot of [22:27] day-to-day product problems for your customers  or even if for services they don't need like 1 [22:33] million context windows. It just you don't need a  million tokens to shove in there to answer simple [22:39] questions. And and I think that actually creates  more risk for like the range of outputs that [22:43] you're going to get. And unless you have like just  topnotch clarity on what it is you're expecting to [22:50] get and how to do the reinforcement learning for  that or having someone do that for you, you know, [22:58] I think you're asking for a lot more like it.  I have not seen a lot of upside to just put [23:02] bigger is better for everything. And I hope  you know I mean and I I think it's something [23:07] though the nice thing is you can go pull up open  router go hit every model on your own laptop and [23:13] see for yourself. I think seeing for yourself  is important here. You don't have to make it a [23:17] whole companywide project. Thank you Brian. Um I  think one of the biggest advantages that smaller [23:25] midsize businesses have is speed. um if you look  at a bank or a large enterprise, it's debt by a [23:32] million committees, right? Versus a small company,  you can just walk across the hallway and make a [23:38] decision. And I think that is it is a great  equalizer. And I think all of us have to be [23:44] aware that this is the time where a small company  can really grow and fight against the giants when [23:50] a new product that is that is an idea can be  built and and produced at a at at speed which [23:56] a large enterprise cannot. Right? So and number  two what I would say is uh incentives drives [24:03] behavior. Um I think Jennifer just talked about um  what happened with token maxing is Microsoft and [24:09] Uber had a situation where the CEO came and said  hey or the CTO came and said you will be measured [24:19] based on the amount of tokens you burn or use. And  what happened is their bills went off the roof. [24:27] um they had to shut down literally Microsoft just  shut down usage of you use uses of third party [24:32] LLMs because but the IT budget was was vanished  by April and that is not a shock that anyone of [24:41] us want to have. You don't have $100 million $200  million IT budgets. How do you ensure that you're [24:48] not spending 40 50 100k a month on tokens? Um  there are options there. So going back to the [24:54] question is I think it's important to understand  that um the US frontier models compared to the [25:00] Chinese models there's a difference between what  we have most of them are closed source models [25:06] um closed weight models and some of the Chinese  ones are open- source and some are open weight [25:13] models and there's a difference between these  three and some are MIT licensed and we if the more [25:18] we know about this the better we can use for our  enterprise And um when it comes to risk is you may [25:26] not want to basically go and put your queries in  or APIs in where it's hosted in China, right? Your [25:32] rights you you're giving away your rights. You may  have better rights with the US models, but you can [25:38] still download them, download the source, download  the the way open weights and host it and deploy it [25:45] in your own enterprise in your own cloud provider  and use it and not risk your data. So that is [25:51] something that you need to know. But there's a  time and a place for it. If you're just using [25:56] let's say a few million tokens, it doesn't make  sense to host it. Host an open source openweight [26:01] model on on enterprise. The cost of GPU may not  be cutting it. But if you're a heavyweight user, [26:07] um you're using it heavily, then it one day it  starts making sense to host it on your own own [26:13] set of GPUs and maintain it. And you have options.  Um if you see Kimik K3 that is competing with some [26:19] of the front frontier models there's GLM52 um  deepseek has uh has their options um Alibaba's [26:27] models have their options so there's I think  it's important that we understand and if you [26:32] see the data the stats coming in from open router  um just a year ago uh the open source models open [26:40] weight models were about two singledigit usage now  it's heading to about 60% set compared to the the [26:47] closed uh closed or closed source uh US models.  So there's a shift happening and it's important to [26:53] keep our ears and eyes open towards that in terms  of which one is better suited for your purpose. [26:59] Right. Yeah. I'd love to stay on the Chinese  openweight model topic for a little bit because [27:03] as you mentioned with those open router numbers um  this isn't just theoretical um a lot of people are [27:09] using them. Um and so Nick maybe we can start with  you. Can you talk us through some of the um cost [27:15] benefits and some of the risks um and how people  should think through whether this is a good idea? [27:21] Yeah, there's no one-sizefits-all answer for  everyone in this room and you know anyone who [27:26] ever hears this. the the way I would look at  it just personally as a practitioner and how [27:31] we look at it as a company has been uh you know  where is the data going is who's hosting this [27:38] model is it hosted in bedrock is this hosted  by some inference provider is this hosted by [27:46] yourself and where are you hosting it you know the  cost just just the hardware requirements for an [27:52] unquantized K3 to locally host that you would need  like 1.54 terab terabytes of GPU memory. You know, [28:01] that's going to be a lot of Mac Studios if you  know or DGX Sparks you're trying to put in your [28:05] garage. Like it just it I was looking at it  for fun. It doesn't make sense. Um but there [28:11] are so many other models and there's so many other  options and and I think what is fit for purpose. [28:16] So you have to know where the data is going. And  then I think even more important or as important [28:22] as that is who are your customers cuz I I would  say that will dictate a lot more. you know, [28:26] you'll you'll hit this the reality really quickly,  whether it's risk committees, if you're in a [28:32] highly regulated sector or even if you're not and  you're just in B2B or something else, it who your [28:38] customers are going to dictate, okay, where is  this model coming from? Do you know where it's, [28:43] you know, the source? Can you vouch for that? You  have that in, you know, your kind of AI bill of [28:50] materials. So, it's like what is the model source?  Who's responsible for the safety and eval of this? [28:55] who's doing that? Are you paying someone else for  this? Are you getting this directly from a lab [29:00] overseas? And and I think that's going to drive  a lot of your decision- making too in addition to [29:06] like is this cost effective and how much do you  really need to use it and at what volume? Um or [29:12] is it something that maybe internally you know I I  have a lot of friends CTO's for small engineering [29:19] teams who are very successful using local models  for their engineering team on their developer [29:25] machines and that's been working out really well  for them and it saved them quite a bit of cost [29:30] versus just going straight to like say codeex with  you know the latest GPT or going to claude code [29:36] for everything. Um especially for SMBs, you know,  we're not even always up to the like enterprise, [29:43] you know, fortunately, but the the problem I've  seen personally using like non-enterprise plans, [29:49] which are all just built purely on usage, is you  don't there's a lot of control you don't have than [29:55] we don't have over when the frontier labs are,  you know, having to let's say uh cope with uh [30:03] regulatory uncertainty and circumstances and you  know, I I think we were all here when like Fable [30:09] 5 was just shut off. And so for us that was a  wakeup call like okay what is our uh backup plan? [30:18] What what what are our mitigation options if we  needed to rely on that? Unfortunately, you know, [30:23] most of our customerf facing calls don't have  to go to such large models, but I do think it's [30:31] um you know, it's not never, but it's not always.  And and I think that's a big question, but I think [30:37] there's a lot to learn, a lot to see, and I  think all of us and the entire industry would [30:43] be at a disadvantage if we don't actually look at  everything as an option and keep everything on the [30:48] table. But I think how is super important because  is this model somewhere that you understand where [30:54] this is being hosted and you have asurances  whether contractually or you have technical [30:59] safeguards in place around you know how much of  your data and how that's getting sanitized where [31:03] that's going and what you're expecting to see too.  So do either of you want to add anything to that? [31:10] Um the the analogy that I give is um always if  you think about um refrigeration as your models [31:21] um there was a lot of refrigeration companies  that came up uh and they're still there but there [31:26] was a syrup company which came on top called Coke  and they are a 100 $250 billion4 trillion dollar [31:35] company today. Pepsi is a $200 billion company  today. So I think what we have to understand [31:41] as small businesses is uh these models are going  to become a commodity and we have to understand [31:48] and the analogy they would be similar to the  refrigeration companies and what's important for [31:53] each one of us to understand is who becomes the  Coke and with AI we have the opportunity to become [32:01] the Coke or the Pepsis of the world to build  on these things and not get lost into all the [32:08] complication that goes on but it's important to  understand what kind of refrigeration is required [32:13] where right so there's a there's a lot that coming  in not majority of the people here unless you're [32:19] doing a lot of inferencing and reasoning and  multi-agentic complex queries you don't need the [32:24] latest and greatest models out there right uh for  instance the largest model right now Kim K3 is the [32:33] uh is about 2.8 8 trillion parameter model you  a small business cannot afford to even deploy it [32:42] today they the costs are going to be very similar  to what an opus is what we have to understand is [32:48] the US labs compared to some of the Chinese models  that we have if you deploy it it's about 56 times [32:56] what a typical other models would cost and what  we have to understand from a cost perspective is [33:03] it is it worth paying 56 times for a bought can  of Coke in the US compared to somewhere outside. [33:11] Great. When we deploy these things, we have to  think through from a cost perspective. We have [33:18] to figure out which models and it's important to  understand that you get the right guidance. Which [33:22] models is better suited for my purpose and from a  costbenefit analysis. If you are running at scale, [33:28] if you pick on the right models, pick on the  right deployment framework and architecture, [33:33] I think you have a winner there. And it's we the  great part is we have options on the refrigeration [33:40] side. Do you know which ones to pick and how do  you become a winner is going to be the key to [33:46] success. Great. Um, we have about 10 minutes left,  so I'm going to do one more question, then we're [33:52] going to throw it to the audience. Um, but I'd  love to hear from each of you a little bit about [33:56] how you have these internal conversations about  how to use AI. And I'm thinking specifically, [34:01] sometimes you have leadership that wants to throw  AI at everything, think it's going to save time, [34:06] think it's going to save money, or maybe you have  younger employees who want to use these tools, [34:11] but you're not sure if you can deploy them in a  safe way. And I think this is a really relevant [34:15] question for people in this room. How do you  bridge these different stakeholders and how you're [34:20] deciding to roll out AI? Brian, would you like to  start? Sure. So I think the key is avoiding both [34:26] extremes. Obviously there's a lot of hype about  AI and people want to deploy it for everything, [34:30] but it's not always the right example. Now you  need people to buy into it. If the employees [34:36] don't trust the AI tool, they're not going to use  it. And if the management doesn't trust the tool, [34:41] they're not going to fund it. So you need to be  clear on what you're trying to accomplish and is [34:45] that tool the right answer for you? And have those  conversations internally. You want your team to be [34:50] fully engaged in trying to come up with use cases  to suggest AI were appropriate. But you also want [34:55] them to be able to push back. You want them to  tell you when this isn't the right example. It's [34:59] not trustworthy. I'm not getting good answers.  This isn't safe for the company. If as long as you [35:03] fund both sides of that, you know, you make sure  that people are okay giving you the good ideas, [35:07] but also pushing back when it's a bad idea. I  think that'll make adoption handle by itself. [35:12] As long as the you're focusing on the business  value, I think that's the right conversation [35:16] to have and I think that's when it's going to  work out for you. Okay. Next. You show them the [35:21] bill that that usually solves like 90% of it.  Yeah. I mean, our CEO was like, "Hey, everyone, [35:28] you know, this is like 24 months ago and needing  to make sure everyone's like using it." You know, [35:33] we had given employees access to Gemini, to  Cloud Code, to Cursor Enterprise. It's like, [35:41] okay, everyone has it. I I don't think the point  is just giving people access to this. It has to [35:46] mean something to like how are they expected to  like make a difference and and actually make an [35:52] impact in the business. And then we saw the huge  evolution past token maxing all that nonsense. And [35:58] you know, we're seeing great I'm seeing great  stuff coming from all parts of the company, [36:04] especially like I think teams where they always  felt like they were at someone else's mercy to [36:10] get things deployed to get utilities for their  own team to solve some of their use cases. [36:17] um giving it to them and then saying like hey  you know creating some guard rails around that [36:21] obviously and what data sources they can connect  things with and we have controls around that but [36:26] we're we're we're fairly um let's say open with  the team in terms of like have at it and they I [36:35] think they're finding that a lot more valuable  than just hey go use it as much as possible [36:38] or you'll get fired and I mean obviously no  management team has ever said that out loud I [36:44] guess but you know so I I I I I [laughter] think  that that has made a big difference and like no [36:51] I mean I I I don't know about I'm sure people  in this room saw a you know kind of year and [36:56] a half two years ago where it's kind of like oh  we need to go have hackathons we need to like do [37:01] this propaganda stuff and convince people it's not  going to take their jobs or they won't use it and [37:06] people are otherwise going to burn their laptops  and stuff. Um, I think we've come around a bit, [37:11] at least I've seen for for us and that evolution  and especially talking to my peers about no, [37:16] once people have gotten the tools, they can  actually make their jobs easier. They can focus [37:20] on things that they wanted to do. They never had  they never had the headcount they always felt they [37:25] needed anyways. They never had the budget they  felt like they needed. And at this point in time, [37:30] we're giving some some of these resources to like,  hey, you know, if you have an idea and it makes [37:36] sense and it's not going to destroy the company,  then like let's go. Thank you. Um, one of the [37:44] things um that I'm sure a lot of you are facing  is this. The CEO tends to be the most frustrated [37:52] person in the company with AI or the head of  sales tends to be the most frustrated company. [37:56] He or she goes in wipe codes all through the  weekend creates something slaps it on your face [38:01] and say look if I can do it why can't 20 of you  do it [clears throat] and that that's where starts [38:09] the pressure and each of you are going to face  that pressure and how you deal with that or picks [38:15] someone picks up a vendor they go to a conference  and say hey take take a look at the demo and let's [38:19] deploy it it comes top down pressure no one has  the guts to counter question. Ask a rational [38:27] question because now it becomes an emotional  piece because I sat all through the weekend built [38:33] something and shared it with you. Why can't  you deploy it? There's a difference between [38:37] prototyping and going in production. And it's  difficult even for the chief technology officer, [38:43] the CISO or anyone to go and explain it to a CEO  why this will fail and fall flat in production. [38:51] It doesn't work that way. I wish life was that  easy, right? And I'm I'm talking in a regulated [38:57] industry. I'm even in a nonregulated industry. It  doesn't work that way. Life is not easy. I think [39:04] it's important and there's a reason why we spend  the money and the time to attend these conferences [39:08] so that you can learn from the hard failures  that people have had. So it is important to [39:13] understand that like we we we need to figure  out which ones to use where, how to use it, [39:19] learn from some of the successes and the failures  they have and deploy them in your enterprise. [39:24] Great. Um, I think we have time for a couple  of questions. I know there's one right here. [39:32] Thank you. Thank you. So, I uh I've worked with a  lot of SMBs in doing their AI enablement strategy [39:38] and all that and uh this thing always comes up and  I always think that you know we should always try [39:44] open source models first before actually  going to the frontier models because of [39:48] the same things that Manoj mentioned. But I see  there are other points too. So I just wanted to [39:53] actually see if you guys have deployed something  open source open wage and uh tried that and how [39:59] has your experience been like I I I understand  the costing thing and that's very important on a [40:04] recurring basis. So if you have tried something  please and other ones absolutely um we have and [40:11] we have deployed open source openweight models  at scale. Uh what I would say is test it out. [40:18] So if you go if you're with any cloud provider,  this could be AWS or Azure or Google cloud, [40:24] each of them have their own um cloud hosting  um the um the uh AI hosting there. So Amazon [40:34] is Bedrock, right? So you can access these  models using Bedrock. Um you will you will [40:42] actually you can compare and contrast the and you  can build inference. you can actually train these [40:47] models using your data using any of these  cloud providers and that's the first step [40:52] that I would do right and then just because  you're using open source open weight models [40:57] doesn't mean your co cost may be worse than the  US frontier models if you use for instance haiku [41:03] maybe compare with a GLM or a kimmy also you have  to understand where you're getting the most value [41:10] um so how you train the model matters and  you have an ability today without exposing [41:16] osing your data even to the frontier labs in  the US to use these cloud providers to your [41:22] benefit and test your inference and training  out and your output output out using APIs. [41:30] Does anyone else have any questions? We still do  have a bit of time, so don't be shy if you've got [41:35] a burning question. Oh, okay. Here we go. um you  you have experience in working with legacy systems [41:42] which are RPAs and you know that sort of stuff  you know because you you're you're only as good [41:49] as the data that you get access to and especially  in regulated industries you're behind these you [41:55] know dumps and you know how do we how do these  thing work together well um I had experience [42:01] with voice AI working through some of those RPA  sort of system it was a nightmare to get it done [42:08] and so I just want to know any guidance you can  give on that. Got it. Um want to take it? Um so [42:16] I I I'll rephrase the question right US has the  advantage and the disadvantage of being the first [42:23] to digitize major a large number of insurance and  banks are still running mainframes right there [42:30] are IBM mainframes running they're running on um  cobalt it's a dinosaur you don't get developers [42:38] today you're running uh RPC calls you're you're  running those very old API which is not even an [42:46] API it has not even passed soap forget about rest  and all those kind of things right how do you plan [42:52] towards a digital transformation in the AI age  and it is an important question it is it is super [42:58] critical and it is I think what what the benefit  today is it if it does not come from the top down [43:07] and I'm sure there is some top down pressure  these evolutions can come from the bottom up [43:12] so it it requires a small team, an army of team,  a small team trying to build and showcase that [43:21] hey I can g get similar results in a different  environment and we can be much more nimble uh even [43:29] at at a large enterprise scale. So if it doesn't  start from the top down the tops won't change [43:35] sometimes because they don't want to change. No  one wants to risk their careers by asking for a [43:40] change and sometimes these changes can come from  the bottom up even at a managerial level at a lead [43:46] level. Build those prototypes, share it and build  that ecosystem where now uh building through wipe [43:52] coding uh using these latest AI tools um u is  encouraged and build it build that confidence [44:00] that can change the enterprise. I've seen those  things happening in a number of enterprises.