Tines Warns AI Code Adoption Triggers Security Vulnerabilities

Everyone's a Builder Now. Is Anyone Watching? With Tines — Ai4

Ai4

Unmonitored AI usage by non-technical teams creates severe security risks that require embedded runtime safeguards rather than outright bans.

AI-assisted summary of Everyone's a Builder Now. Is Anyone Watching? With Tines — original source. Check the original for full context.

Source published:

Detailed notes

## Background This presentation was published on YouTube by the channel Ai4 on August 15, 2026. The video features Rosie Halpin, Product Manager at Tines, discussing the commoditization of building through AI platforms, the governance blind spots it creates, and how Tines adapted its software to support an AI-native workforce.

## TL;DR Rosie Halpin outlines how the widespread adoption of AI tools enables non-technical employees to independently build solutions, introducing ungoverned "wild code" and creating a significant governance gap [00:04, 03:46]. She details the three waves of AI adoption and highlights hidden risks, such as exposed credentials, unmonitored maintenance costs, and compliance vulnerabilities in department-led automations [01:51, 07:22]. Rather than banning AI or allowing unchecked usage, Halpin advocates for a framework built on four pillars—freedom, compliance, control, and monitoring—to make the safe path the easiest path for builders [11:22]. She presents practical technical safeguards, including credential proxies, containerized execution, and AI-assisted maintenance, to maintain both speed and security [15:29].

## Atmosphere & style The presentation carries a professional, insightful, and practical tone, addressing the real-world tension between rapid enterprise productivity gains and security oversight. Halpin speaks as a collaborative product leader, shifting the dynamic between IT and business teams from blocker to enabler. Watching the talk feels structured, pragmatic, and engaging, guiding attendees through an urgent corporate challenge toward a constructive, actionable roadmap [15:29].

## Key takeaways - [00:04](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=4s) The commoditization of AI tools has enabled non-technical employees across departments like Marketing, Legal, and HR to become active software builders. - [00:04](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=4s) A severe AI governance bottleneck exists, with 77% of organizations reporting that employee AI adoption is currently outpacing their governance capabilities. - [01:51](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=111s) AI adoption occurs in three waves, progressing from developer code assistance to embedded SaaS features, and finally to independent business units purchasing or building custom AI point solutions. - [03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) While large language models solve the capability problem of writing code, they fail to solve the accountability problem because they lack organizational, security, and compliance context. - [03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) Tines defines "wild code" as ungoverned, AI-generated software created by well-intentioned non-technical workers operating below IT visibility. - [07:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=442s) Wild code introduces hidden enterprise risks, including redundant token costs, exposed credentials in code, and unmonitored workflow breakage. - [11:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=682s) Banning AI tools entirely is ineffective and drives shadow IT deeper, while allowing unchecked usage exposes the enterprise to major security vulnerabilities. - [11:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=682s) A sustainable governance strategy relies on four core pillars: Build with Freedom, Design for Compliance, Run with Control, and Monitor at Scale. - [15:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=929s) Technical safeguards such as runtime credential proxies and containerized execution environments enable safe, vendor-neutral AI usage. - [15:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=929s) Organizations should proactively audit non-technical teams to uncover existing wild code and establish a unified, governed path for AI deployment.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=4s) The Rise of the Non-Technical Builder and the AI Governance Gap

Rosie Halpin, Product Manager at Tines, introduces the central theme of her talk: "Everyone, and I really do mean everyone, is a builder now, but is anyone watching?" With modern AI tools and large language models (### Everyone Is a Builder Now

Rosie Halpin opens her presentation by introducing the fundamental shift happening across organizations today: "Everyone, and I really do mean everyone, is a builder now, but is anyone watching?" With the sudden explosion of accessible AI tools, non-technical teams and employees who are not software engineers by trade—ranging from marketing and legal to HR—are actively using AI and LLMs to build solutions and solve their own operational problems. This is inherently empowering because the people who best understand these internal domain processes can now solve their issues directly without going through tedious requirements-gathering cycles or back-and-forth iterations with engineering teams.

However, the rapid speed of AI adoption has created a severe governance bottleneck. Halpin points out that this is not a future, hypothetical concern, but an active crisis: 77% of organizations report that their employees' AI adoption is currently outpacing their governance capabilities. This leaves leaders caught in a difficult trade-off between mandating company-wide AI usage while remaining strictly compliant with organizational risk profiles and governance rules.

- 77% of organizations report AI adoption outpacing governance. - Non-technical business functions (Marketing, Legal, HR) are actively writing solutions via LLMs.

[01:51](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=111s) The Three Waves of AI Adoption

AI adoption across companies typically manifests in three distinct waves. The first wave fits neatly into existing governance structures: technical teams and software engineers use LLMs for code assistance, which seamlessly flows through standard code reviews, security testing, and IT oversight. The second wave occurs when non-technical business units are exposed to embedded AI features within the SaaS platforms they already use every day, such as a new AI interface or widget inside a marketing tool.

The third wave presents a far more complex challenge. In this phase, teams actively purchase or build custom, vertical-specific AI point solutions in-house. While it is crucial for teams to advocate for themselves and solve domain problems, this wave can lead to an unmanageable environment where an organization might run 80 different AI point solutions across 80 different teams—many of which IT and security teams do not even know exist.

- **First Wave:** Developers and software engineers using LLMs within established security and testing pipelines. - **Second Wave:** Non-technical teams adopting embedded AI widgets inside existing vendor software. - **Third Wave:** Business units independently building or buying disparate, vertical AI point solutions (e.g., 80 distinct solutions across 80 teams).

[03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) The Rise of "Wild Code" and the Capability vs. Accountability Gap

Building software is no longer restricted to technical experts. In traditional software development, engineering teams provided a natural safety choke point with architectural controls built directly into systems. Today, because LLMs remove the technical barrier to entry, anyone can generate code without adhering to security tests, compliance requirements, or governance frameworks. This has created a direct trade-off where organizations trade safety for speed.

While LLMs excel at generating functional code, they completely lack organizational context. Halpin emphasizes that while LLMs solve the "capability problem," they do not solve the "accountability problem." An LLM might draft a working script, but it remains oblivious to whether the solution overexposes sensitive data or violates company access controls. Tines coins this invisible, ungoverned, AI-generated software as **"wild code."**

Wild code operates like an iceberg: traditional software built by engineering teams forms the visible tip above water, fully understood and controlled by IT. Wild code lives below the surface, unseen by security teams until a major incident occurs and leadership has to step in.

- "LLMs are solving this capability problem, but they're not solving the accountability problem." - Wild code is created with good intentions by workers untrained in software development and security practices. - The Iceberg Analogy: Known enterprise software sits above the water; ungoverned wild code hides beneath the surface.

[07:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=442s) Hidden Costs and Real-World Examples of Wild Code

Wild code carries severe hidden operational and security risks. First is the technical debt and token waste incurred when ten different people independently build duplicate tools to address the same process. Second is the danger of exposed credentials: connecting custom tools to proprietary systems requires credentials, which non-technical builders often hardcode or store insecurely. Third is the ongoing maintenance burden, as workflows silently break over time without designated owners, alerting systems, or error tracking.

To illustrate these dangers, Halpin shares concrete examples across different business functions: - **RevOps:** A RevOps analyst builds a CRM lead routing workflow using an LLM. While it saves time, it silently overshares confidential customer data with unauthorized personnel without IT's knowledge. - **HR:** An HR analyst automates onboarding and offboarding workflows. The productivity gains are huge, but sensitive employee personally identifiable information (PII) is routed through un-audited systems. - **Finance:** A finance associate connects multiple internal platforms to generate automated reports, inadvertently introducing a critical supply chain vulnerability that security never vetted.

These examples highlight that wild code is not driven by malicious actors, but by well-intentioned employees focused on solving daily tasks faster without realizing the underlying compliance vulnerabilities they create.

- **Three Hidden Costs:** Redundant token costs/technical debt, exposed credentials in code, and untracked maintenance/breakage costs. - Wild code stems from well-meaning employees, not malicious insiders.

[11:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=682s) Architecting a Governed Path for the AI Workforce

Restricting AI usage entirely is an unrealistic solution that only forces wild code deeper underwater as employees bypass rules. Conversely, allowing AI usage to run completely unchecked exposes the enterprise to severe risk. The real solution lies in building an environment where safety is architected directly into the development experience, making the safe path the easiest path for builders.

Instead of asking "Can we build this?"—which is now almost always a yes—organizations must ask: "How do we protect builders who don't know the risks?" and "How do we allow LLMs to move freely where appropriate while maintaining firm boundaries elsewhere?" A lasting framework rests on four core pillars: 1. **Build with Freedom:** Empower every team to use AI and innovate. 2. **Design for Compliance:** Structure systems so non-compliant actions cannot be built in the first place. 3. **Run with Control:** Execute processes reliably with secure credential handling and scoped access controls. 4. **Monitor at Scale:** Provide IT and security with centralized visibility across all departmental builds.

- Neither total AI bans nor unmonitored AI usage are viable strategies. - The four pillars of governed AI adoption: Build with Freedom, Design for Compliance, Run with Control, and Monitor at Scale. - Security and IT shift from being perceived as "blockers" to becoming productivity "enablers."

[15:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=929s) Technical Safeguards and Practical Next Steps

In a modern, governed AI environment, employees can build using their preferred interfaces—whether through a terminal, Model Context Protocol (MCP), or custom user interfaces—using any LLM provider without vendor lock-in. Safety is enforced automatically at runtime through architectural constraints: - **Credential Proxies:** Sensitive credentials are never embedded or exposed in raw code; instead, they are dynamically inserted at runtime. - **Containerized Executions:** AI-generated workflows execute within isolated containers every single time they run. - **AI-Assisted Maintenance:** Automated agents notify non-technical users when workflows fail and recommend targeted fixes or improvements.

Halpin closes with an actionable takeaway for attendees: speak directly with non-technical teams like Marketing, Legal, HR, and RevOps to discover what they have already built using LLMs. Inspect those custom workflows for unmitigated security risks and begin building a unified, governed path together.

- Real-time protection relies on credential proxies and containerized runtime environments. - Attendees are encouraged to visit the Tines booth (booth #630, marked by live flowers matching the "wild code" theme) in the expo hall for t-shirts and further discussion. - "The future really isn't to stop using AI or restrict AI within your organization. Instead, it's to create a system that allows and architects for both speed and security."

## Notable quotes > [00:04](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=4s) Everyone, and I really do mean everyone, is a builder now, but is anyone watching? — Rosie Halpin

> [03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) LLMs are solving this capability problem, but they're not solving the accountability problem. — Rosie Halpin

> [15:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=929s) The future really isn't to stop using AI or restrict AI within your organization. Instead, it's to create a system that allows and architects for both speed and security. — Rosie Halpin

## Mentioned Rosie Halpin, Tines, Claude Code, Codex, LLM, CRM, RevOps, HR, Finance, Marketing, Legal, Model Context Protocol (MCP)

## Video description Rosie Halpin - Product Manager, Tines

The commoditization of building via AI-assisted platforms or AI coding agents (Claude Code, Codex, etc.) is solving problems across the business faster than ever before. One team's productivity gain is another team's blind spot. Often companies are spinning off ICs into AI councils and governing bodies, which solves part of the problem – helping people deploy their work more safely. It still lacks scale.

How do you build a lasting model for this new world? What considerations do you need to make? Where do you trust the LLM vendor vs. a third party?

Join Rosie Halpin, Product Manager at Tines, as she talks through what Tines learned in their journey to power the world’s most important workflows. She’ll share how our customers led them to rebuild their software from the ground up to address our mission in the context of an AI-native workforce.

[Source](https://www.youtube.com/watch?v=bMoCwOzGUFY)

中文简报

Tines 警告:引入 AI 代码引发安全漏洞 非技术团队在未受监管的情况下使用 AI 带来了严重的安全风险,需要的是嵌入式运行时防护措施,而非直接彻底禁止。 Rosie Halpin 概述了 AI 工具的广泛普及如何赋能非技术员工自主构建解决方案,进而引入不受监管的“野生代码”(wild code)并造成巨大的治理缺口 [00:04, 03:46]。她详细阐述了 AI 普及的三次浪潮,并着重指出了部门主导的自动化所带来的隐性风险,例如凭据泄露、未受监控的维护成本以及合规漏洞 [01:51, 07:22]。Halpin 既不主张彻底禁止 AI,也不赞成放任其不受控制地使用,而是倡导建立一个基于自由、合规、控制和监控四大支柱的框架,让安全路径成为构建者最容易走通的捷径 [11:22]。此外,她还提出了包括凭据代理、容器化执行和 AI 辅助维护在内的实用技术防护措施,以兼顾交付速度与系统安全 [15:29]。 随着 AI 工具普及,营销、法务和人力等非技术部门正迅速成为应用构建者,但 77% 的组织表示员工对 AI 的采用速度已超出其治理能力。 (0:04) Tines 将非技术人员脱离 IT 监管生成的应用定义为“野代码”(wild code),指出大语言模型虽能解决代码生成能力,但缺乏组织上下文与合规问责机制。 (3:46) “野代码”给企业带来了隐形风险与成本,包括重复的 token 消耗、代码中的敏感凭据暴露,以及缺乏告警追踪的工作流静默损坏。 (7:22) 全面禁用 AI 只会加剧影子 IT 泛滥,企业应依托自由构建、合规设计、受控运行与规模化监控四大支柱构建可持续的治理框架。 (11:22) 通过在底层架构中内置运行时凭据代理、容器化隔离执行和 AI 辅助运维机制,企业能够在赋能业务创新的同时自动保障安全合规。 (15:29)

Reading summary

Rosie Halpin, Product Manager at Tines, argues that while generative AI empowers non-technical employees to build operational solutions, this rapid adoption has outpaced enterprise governance, creating invisible and unmonitored security risks termed 'wild code.' The AI Adoption Waves and Accountability Gap (1:51) Enterprise AI adoption is progressing through three distinct waves. The first wave integrated LLM code assistants into traditional software engineering workflows, fitting neatly within existing code reviews and IT controls. The second wave brought embedded AI features into standard SaaS platforms used by business units. The third and most disruptive wave involves non-technical teams independently purchasing or developing vertical-specific AI point solutions, which can lead to an uncoordinated sprawl of dozens of isolated tools across an enterprise. Halpin cites data indicating that 77% of organizations report employee AI adoption is outpacing their governance capabilities. Although non-technical workers in functions such as marketing, legal, and HR can now solve operational problems without engineering backlogs, Halpin emphasizes that while LLMs solve the 'capability problem,' they fail to address the 'accountability problem.' Models generate functional code but lack organizational context, remaining unaware of data exposure risks, security testing standards, and compliance frameworks. The Operational Dangers of 'Wild Code' (7:22) Tines defines 'wild code' as invisible, ungoverned software built by well-intentioned employees outside the supervision of IT and security teams. Operating like an iceberg, visible enterprise software approved by IT represents only the surface, while unmanaged wild code remains submerged until an operational failure or compliance breach occurs. Wild code introduces three primary enterprise costs: technical debt and redundant token expenditures from duplicate departmental builds, exposed credentials hardcoded into custom scripts, and unmonitored maintenance failures when unowned workflows break. Concrete risks appear across business functions: RevOps analysts may build lead-routing workflows that inadvertently expose CRM customer data to unauthorized personnel; HR staff automating onboarding can inadvertently expose sensitive personally identifiable information (PII) to un-audited tools; and finance professionals linking platforms for reporting can introduce unvetted third-party supply chain vulnerabilities. Architecting Guardrails Around Four Core Pillars (11:22) Outright bans on AI tools are counterproductive, as restrictions merely push wild code deeper underground, while unregulated use exposes enterprises to unacceptable liabilities. Rather than asking if a tool can be built, organizations must architect environments where the secure pathway is the easiest pathway for non-technical creators. Halpin outlines a framework based on four core pillars: Build with Freedom (allowing cross-functional teams to experiment and innovate), Design for Compliance (engineering systems so non-compliant actions cannot be deployed), Run with Control (enforcing secure credential management and granular access controls during runtime), and Monitor at Scale (giving central IT and security teams complete visibility across all departmental applications). Through this approach, security teams transition from perceived operational blockers into productivity enablers. Runtime Technical Safeguards and Discovery Actions (15:29) A governed development framework allows employees to work across diverse interfaces—including command-line terminals, custom interfaces, or the Model Context Protocol (MCP)—with any preferred model provider. Security must be enforced automatically at execution through technical safeguards such as credential proxies, which prevent raw credentials from being exposed in code by injecting them dynamically at runtime, and containerized executions that isolate AI workflows during each run. In addition, AI-assisted maintenance agents can monitor workflows, flagging failures and suggesting targeted fixes directly to non-technical creators. Halpin advises organizations to immediately initiate direct conversations with non-technical departments—including HR, Legal, RevOps, and Marketing—to uncover what custom AI workflows are already in active use, assess those scripts for latent security liabilities, and migrate them onto a unified, governed framework. Halpin concludes that sustainable AI strategies do not restrict adoption, but instead embed security directly into the development runtime to protect non-technical builders from unmonitored risk while preserving organizational velocity.

精炼阅读摘要

Tines 产品经理 Rosie Halpin 指出,大语言模型正在促使非技术员工广泛成为软件构建者,但 77% 的企业反映 AI 应用速度已超越治理能力。这种由缺乏专业工程训练的员工出于善意编写且脱离 IT 监管的“野蛮代码”(wild code)带来了严重的数据与合规隐患,企业必须通过内嵌安全的系统架构而非全面封禁来化解风险。 采纳浪潮下的治理断层与野蛮代码 (0:04) 现代大语言模型大幅降低了开发门槛,使市场、法务和人力资源等非技术部门无需等待繁琐的工程排期,即可自行编写提示词或脚本解决业务痛点。然而,这种自主创新引发了严重的治理脱节。根据调研,77% 的受访企业表示员工采纳 AI 的步伐已快于现有的合规监管能力。AI 在企业内的渗透经历了三波浪潮:第一波是工程师在已有的代码审查与安全测试流程中应用代码辅助工具;第二波是非技术人员使用现有 SaaS 软件中内置的 AI 模块;第三波则是各业务团队自行采购或搭建专用的垂直 AI 单点工具。这种碎片化扩展可能导致单个企业内出现几十个互不相通且未被 IT 部门察觉的独立方案。 Halpin 认为,大语言模型虽然解决了“能力问题”,却无法解决“问责问题”。模型能够生成功能可用的脚本,但并不具备企业的组织上下文,也无法判定代码是否过度暴露了敏感数据或违反了权限规则。Tines 将这类在缺乏安全测试与合规框架下生成的软件定义为“野蛮代码”(wild code)。在冰山比喻中,工程团队构建的受控软件仅是露在水面上的小部分,而野蛮代码则潜藏在水面之下,往往直到发生重大安全漏洞时才被管理层察觉。 善意构建引发的隐性成本与业务风险 (7:22) 野蛮代码的出现并非源于内部恶意攻击,而是来自希望提升工作效率的普通员工,但这造成了三大隐性成本。第一是技术债务与资源消耗,多个团队各自开发重复工具解决相同问题,造成了冗余的开发负担与 Token 浪费;第二是凭证暴露风险,非技术人员在连接内部专有系统时,经常将访问密钥硬编码或不安全地存储在脚本中;第三是后期维护成本,这类自制工作流缺乏明确的维护人、错误告警与追踪机制,容易在系统迭代中无声损坏。 Halpin 列举了各业务场景中的真实隐患:在收入运营(RevOps)中,分析师利用大语言模型构建了 CRM 销售线索路由工作流,虽节省了时间,却在 IT 未知情的情况下将机密客户数据违规同步给未授权人员;在人力资源(HR)部门,员工为了加速入职和离职流程,让敏感的个人身份信息(PII)流经未经安全审计的外部工具;在财务部门,业务人员跨内部平台自动汇总报表,无意间引入了未受安全团队审查的供应链脆弱点。 兼顾创新与安全的四项治理支柱 (11:22) 面对野蛮代码的蔓延,Halpin 指出全面禁止 AI 并不可行,严苛的限制只会迫使员工绕开规则,让影子开发进一步转入地下;而完全放任自流同样会使企业暴露在合规与安全风险中。解决之道在于构建一种将安全性直接内嵌于开发体验的机制,让合规路径成为构建者最容易选择的路径,推动安全与 IT 部门从“阻碍者”转变为“赋能者”。 该治理体系由四大支柱构成:一是自由构建(Build with Freedom),鼓励各业务部门自主利用 AI 工具进行业务创新;二是合规设计(Design for Compliance),在系统结构层面设置约束,从源头上阻止不合规逻辑的生成;三是受控运行(Run with Control),通过受控的执行环境与严格的凭证管理保障流程可靠运行;四是规模化监控(Monitor at Scale),为 IT 和安全团队提供跨部门集中可见性,实时掌握全公司的 AI 自建项目情况。 运行时技术防护机制与落地排查 (15:29) 在技术架构层面,受治理的开发环境支持员工使用终端、模型上下文协议(MCP)或自定义界面自由对接各类大语言模型,同时在运行时通过底层架构实现强制安全防护。具体机制包括:使用凭证代理(Credential Proxies),使敏感密钥绝不暴露在原始代码中,而是在运行时动态注入;采用容器化执行环境(Containerized Executions),确保 AI 生成的工作流每次都在隔离容器内运行;配置 AI 辅助维护功能,在自动化流程发生故障时向非技术用户告警并提供精准的修复建议。 Halpin 为企业管理者提出了具体的落地建议:安全团队不应被动等待事故发生,应立即主动与市场、法务、HR 及 RevOps 等部门沟通,全面排查业务人员目前已经使用大语言模型搭建的自制工作流,厘清潜在漏洞,并协同建立受统一治理的合规使用轨道。 Halpin 总结强调,企业面对生成式 AI 的正确策略并非限制或封禁,而是通过系统设计兼顾速度与安全;当务之急是打破 IT 与业务部门的隔阂,让潜藏在水面下的自建工作流重回安全监管视野之中。

Detailed notes — zh

## Background 本视频由 Ai4 频道于 2026 年 8 月 15 日在 YouTube 平台发布。视频记录了 Tines 产品经理 Rosie Halpin 的主题演讲,围绕“全员构建者时代”下企业如何应对 AI 编码工具普及所带来的安全盲区,并分享了 Tines 在重构软件与赋能 AI 原生员工队伍过程中的治理经验与思考。 ## TL;DR 演讲指出,大语言模型让非技术人员也能轻松编写代码和搭建自动化流程,但这也导致 77% 的企业面临 AI 采纳速度远超治理能力的困境。大量缺乏安全规范与所有权归属的隐形资产形成了危险的“野蛮代码”(Wild Code),埋下了凭证泄露和数据暴露的深层隐患。面对这一挑战,企业不应一刀切禁止 AI,而应探索内嵌安全机制的治理路径——立足于“自由构建、受控运行、规模化监控”三大支柱。通过引入凭证代理、容器化隔离执行与智能运维,企业既能释放全员生产力,又能让 IT 与安全团队重塑为创新的赋能者。 ## Atmosphere & style 整场演讲风格务实、敏锐且极具同理心,讲者始终肯定业务一线员工利用 AI 解决痛点的积极初衷,而非将其视作安全威胁。通过“冰山模型”与“野蛮代码”等生动形象的隐喻,演讲在揭示失控风险的严峻现实与指明前行方向的笃定建设性之间达成了良好平衡。整体叙事条理清晰、节奏紧凑,给听众带来兼具宏观认知重塑与落地实践启发的专业视听体验。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=4s) 大语言模型击碎了开发门槛,促使“全员成为构建者”,但有高达 77% 的企业表示内部 AI 采纳速度已超越自身的治理与管控能力。 - [01:51](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=111s) 企业 AI 采纳正处于“第三波浪潮”:非技术团队开始绕过传统流程自建垂直 AI 工具,导致企业内部涌现出大量 IT 团队不知情的影子单点应用。 - [03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) 传统开发中工程团队扮演的“安全关卡”已被瓦解;LLM 虽然解决了构建“能力问题”,但因缺乏组织上下文而无法解决安全与合规的“问责问题”。 - [06:06](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=366s) 企业正面临庞大的“野蛮代码”(Wild Code)冰山危机:大量隐蔽、缺乏维护且未纳入测试管线的 AI 生成代码潜藏于业务流程中。 - [06:06](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=366s) 野蛮代码会带来三大隐形成本:重复建设导致的 Token 资源浪费、内部专有凭据暴露外泄风险、以及工作流静默崩溃时的维护盲区。 - [09:06](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=546s) 野蛮代码并非由恶意攻击者造成,恰恰来自于最积极、最懂业务且渴望提升效率的优秀员工,但他们普遍缺乏网络安全与合规意识。 - [11:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=682s) 企业治理不应陷入“一刀切全面封杀”或“彻底躺平放任”的两极误区,而应寻找在系统架构层面内嵌安全机制的“第三条道路”。 - [12:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=749s) AI 时代的核心思考已从“能否构建”转向“如何保护无安全意识的构建者”,并确保安全合规路径成为全公司阻力最小、最易用的首选。 - [13:30](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=810s) 企业 AI 治理框架依赖三大支柱:自由构建(底层防呆机制)、受控运行(细粒度权限与凭证隔离)以及规模化监控(全局可视与统一纳管)。 - [15:38](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=938s) 通过凭证代理(动态注入密钥)、容器化沙箱隔离执行与智能化运维监控等技术手段,可以在保障安全底线的同时赋予构建者充足的自由度。 - [15:38](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=938s) IT 与安全团队正在迎来角色重塑,从以往被视作业务创新的“阻碍者”转型为赋能全员高效安全使用 AI 的“推动者”。 - [17:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=1049s) 建议安全与 IT 团队主动走出技术部门,深入市场、法务、HR 等非技术业务团队了解其 AI 构建现状,及时排查隐患并重塑治理体系。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=4s) 全员构建者时代的到来与治理脱节的困境 Tines 的产品经理 Rosie Halpin 开门见山地指出,在当今的 AI 时代,“每个人——真的是每一个人——都成为了构建者(Builder)”,但这引发了一个关键疑问:“有谁在监督这一切吗?”随着大语言模型(LLM)的普及,解决业务问题的门槛被彻底击碎。市场营销、法务、人力资源(HR)等非技术背景或传统上不从事软件开发的业务团队,现在都能利用 AI 和 LLM 独立解决自身面临的痛点。 这种转变表面上看令人振奋,因为业务流程的实际专家无需再费时费力地撰写冗长的技术需求文档、排队等待工程团队排期,并在反复沟通迭代中耗费数周时间。然而,AI 采纳与落地的狂飙速度带来了一个极为严峻的问题:企业治理能力完全跟不上工具扩散的步伐。Rosie 强调,这绝非虚构的假想风险或对未来的推测,而是当下正在发生的真实痛点——数据显示,目前有高达 77% 的企业组织表示,其内部 AI 的采纳速度已经远远超出了自身的治理掌控能力。 这一现状将管理层置于极其两难的处境:一方面,管理层背负着在全公司范围内加速推进 AI 落地、赋能业务生产力的硬性指标;另一方面,他们又必须对企业的合规要求、数据风险和治理规范负全责。在当前复杂的组织风险格局下,同时平衡这两项冲突的诉求变得异常艰难。 - **核心矛盾**:业务端快速利用 AI 自主建构 vs 安全合规团队治理手段的严重滞后。 - **关键数据**:77% 的企业面临 AI 采纳速度超越治理能力的现实困境。 ```mermaid flowchart TD accTitle: AI Adoption Dilemma subgraph Mandates["管理层面临的双重冲突目标"] M1["业务驱动:全面推进 AI 采纳与效率提升"] M2["风险底线:严格履行合规审查与治理责任"] end subgraph Reality["企业现实瓶颈"] R1["77% 组织治理脱节:AI 扩散速度 >> 治理监管能力"] end Mandates --> Reality ``` --- ### [01:51](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=111s) 企业 AI 采纳的三波浪潮与碎片化蔓延 Rosie 详细拆解了企业引入和采纳 AI 的三个典型演进阶段。第一波浪潮天然契合现有的治理流程:软件工程师和技术开发团队开始借助 LLM 辅助编写代码。由于这一过程依然运行在既有的工程轨道上,代码依旧需要通过传统代码审查、安全测试以及 IT/安全团队的可见性监控,因此风险整体可控。紧接着到来的第二波浪潮,是非技术团队在日常使用的现有 SaaS 工具中接触并使用 AI。例如营销团队常用的平台新增了内置的 AI 交互界面或小组件,员工在原有平台边界内体验到了 AI 的赋能。 最值得警惕且引人深思的是目前正在爆发的“第三波浪潮”。当业务人员习惯了 AI 的便利后,他们不再满足于现有工具的内置功能,而是开始主动采购或完全自主搭建面向各自垂直业务领域的独立 AI 单点解决方案(Point Solutions)。市场、法务、HR 等各个部门都在自建专属工具。 虽然各团队利用前沿 AI 技术为自身争取效率本无可厚非,但这正在催生一个难以收拾的治理黑洞:一家企业内部可能会瞬间冒出针对 80 个不同团队的 80 个独立 AI 单点应用,甚至存在大量 IT 与安全团队根本毫不知情的“影子应用”。这种无序蔓延使得企业在规模化维度上几乎不可能实现统一的可视化监控与治理。 ```mermaid flowchart LR accTitle: Three Waves of AI Adoption subgraph Wave1["第一波浪潮:技术团队"] W1["工程师使用 LLM 辅助编码"] W1 --> G1["纳入既有安全测试与治理管线"] end subgraph Wave2["第二波浪潮:现有工具嵌入"] W2["业务团队使用现有 SaaS 的 AI 组件"] W2 --> G2["风险受限于供应商既有边界"] end subgraph Wave3["第三波浪潮:自主构建与采购"] W3["各部门自建/采购垂直 AI 单点方案"] W3 --> G3["爆发 80+ 影子应用,IT/安全失去可见性"] end Wave1 --> Wave2 --> Wave3 ``` --- ### [03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) 构建门槛瓦解:能力提升与责任缺失的矛盾 第三波浪潮之所以能够席卷各部门,根本原因在于“软件构建不再是一项纯粹的技术专长”。在传统开发模式下,企业存在一个天然的“关卡(Choke Point)”:任何业务需求都必须经过软件工程团队。工程师会在系统架构中天生植入安全审查与合规机制,从而在架构层面确保业务流程受到严密监管。 但在当前现状下,技术壁垒已被彻底抹平。任何人都可以通过与 LLM 对话来编写代码并构建系统,然而这些新兴的构建者并不会走传统开发流程去跑安全测试或合规审计。这种模式实际上是在“用安全换取速度(trading off speed for safety)”,安全不再是系统构建过程中的固有属性。 Rosie 直言,LLM 确实极度擅长编写代码,但它只解决了“能力问题(Capability Problem)”,完全无法解决“问责问题(Accountability Problem)”。LLM 缺乏对特定企业组织背景的理解:它或许能为业务人员生成一个运行完美的解决方案,但它根本不知道该流程涉及极度敏感的数据,更不会意识到自己生成的逻辑可能导致数据过度暴露或赋予过高权限。整个业务流程建立在一个完全不知道潜在后果、所有权和法律责任的模型之上,这构成了巨大的系统性风险。 --- ### [06:06](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=366s) 什么是“野蛮代码”?冰山之下的隐形技术债务与安全成本 为了精准描述企业内部广泛存在的隐形、失控且由 AI 生成的代码资产,Tines 提出了一个专属术语——**“野蛮代码”(Wild Code)**。野蛮代码并不集中存放于单一代码库,它极其分散、难以追踪,通常由缺乏系统安全训练的业务人员怀揣良好初衷编写,完全暴露于长达数十年的凭证安全漏洞和权限攻击风险中。 Rosie 形象地使用了“冰山模型”进行比喻: - **水面之上的冰山一角**:代表传统软件工程团队编写的代码。这是 IT 与安全团队清晰可见、建立了既定控制措施并能够量化风险的部分。 - **水面之下的庞大冰体**:正是潜藏在企业各个角落的“野蛮代码”。企业管理层甚至不知道它有多大、到底存在于何处,往往直到恶性安全事件爆发、高层介入时才被迫浮出水面。 ```mermaid flowchart TD accTitle: Iceberg Model of Wild Code subgraph Visible["水面之上:传统可见代码(冰山一角)"] V1["工程团队按标准规范开发"] V2["IT / 安全团队具备完全掌控力与测试管线"] end subgraph Invisible["水面之下:野蛮代码 Wild Code(潜在危机)"] I1["非技术人员借助 LLM 零散构建"] I2["硬编码凭证 / 权限过度暴露"] I3["无告警、无维护、无所有权归属"] I4["直到安全事件爆发才被察觉"] end Visible --- Invisible ``` 野蛮代码的存在会带来三大深层隐形成本: 1. **重复建设的技术债务与 Token 成本**:不同部门的 10 个人可能在信息孤岛中利用 AI 搭建了 10 个几乎一模一样的流程,导致巨大的 API Token 浪费、协作脱节以及昂贵且低效的维护开销。 2. **凭证暴露与外泄风险**:任何有价值的业务流都必须对接内部核心专有系统,这必然涉及访问凭证(Credentials)。野蛮代码中凭证如何存储?一旦代码失控或遭到越权访问,极易发生灾难性的凭证外泄。 3. **维护缺失与故障盲区**:这些流转着核心业务的工作流一旦因接口变动发生静默崩溃,系统根本没有任何错误告警机制,缺乏明确的运维责任人。 --- ### [09:06](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=546s) 善意初衷下的安全陷阱:三大业务场景实例 为了使抽象的概念具象化,Rosie 列举了三个来自真实业务场景的“野蛮代码”案例: * **收入运营(RevOps)分析师**:借助 LLM 自行搭建了一套极其出色的 CRM 线索路由工作流。该方案大幅提升了销售团队效率,但由于缺乏数据访问控制意识,导致海量核心客户数据被过度共享给了无关人员,而 IT 和安全团队对此毫不知情。 * **HR 分析师**:针对极其繁琐耗时的员工入职与离职流程自建了自动化工具。团队生产力大幅提升,但员工的个人身份信息(PII)却在未经合规审计的系统之间肆意传输流转。 * **财务专员**:为了生成深入的财务洞察报表,自行通过脚本连接了多个不同业务系统,却在不知不觉中引入了安全团队完全未曾评估过的软件供应链脆弱性。 Rosie 特别指出,野蛮代码的根源**绝不是因为内部存在具有恶意企图的攻击者(Bad Actors)**,恰恰相反,它来自于那些最敬业、最具业务创新意识的优秀员工。他们是自身业务流程的顶级专家,渴望利用 LLM 更快地解决问题,但他们不是网络安全与合规治理的专家,最终导致高危漏洞伴随着生产力的提升一同扩散。 --- ### [11:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=682s) 拒绝两极化:寻找内嵌安全机制的治理第三条路 面对野蛮代码的威胁,企业往往容易陷入两种极端的治理误区: * **极端一:全面封杀 AI(一刀切禁止)** 试图通过全面禁用 AI 来规避一切风险。Rosie 认为这不仅无法奏效,反而会适得其反——业务人员出于效率需求依然会私下使用,这只会把所有野蛮代码“推向冰山更深处”,变成更加不可控的影子 AI(Shadow AI)。 * **极端二:彻底放任不管(全面躺平)** 任由野蛮代码野蛮生长,直接承受其带来的隐患。但正如前文所述,缺乏可见性和监管的代码将带来凭证泄露、数据违规和系统崩溃等巨大负面代价。 因此,摆脱困境的唯一出路是开辟第三条道路:构建一种**系统架构本身即内嵌安全性(Safety inherently within it)**的治理框架。这种机制既能赋能每一位业务构建者继续安全、自由地使用 AI 解决问题,又能在底层架构上从根本上阻止违规、越权或非安全行为的发生。 ### [12:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=749s) 构建新范式:从“能否构建”转向“如何保护构建者与设定边界” 过去企业在评估一个项目时,通常会问三个核心问题:“我们能把这个做出来吗?”、“我们有足够的资源去构建它吗?”以及“这是当前的高优先级事项吗?”然而在 AI 时代,这些问题已经不再适用,因为答案显然是肯定的——现在的技术能力几乎可以让任何人构建出任何东西。 当下的核心问题已经彻底转移。首先,企业必须思考:“我们该如何保护那些根本不知道自己构建的应用潜藏着何种风险的构建者?”其次,一个非常关键的权衡在于:在哪些场景下我们能放心地让 LLM 自由运作,而在哪些地方必须划清红线,坚决限制 LLM 的自由度?各家企业必须为自身业务明确界定这种取舍。最后,如何让“安全合规的路径”成为“最显而易见且阻力最小的选择”至关重要。如果安全流程充满摩擦、繁琐难用,即便安全规范存在,员工也不会遵守;合规路径必须是全公司最简单、大家最乐意使用的途径。 ```mermaid flowchart LR accTitle: AI 时代构建思维的转变 subgraph Old["过去的思维模式"] direction TB O1["我们能构建这个吗?"] O2["有足够资源吗?是优先级吗?"] end subgraph New["现在的核心问题"] direction TB N1["如何保护缺乏安全意识的构建者?"] N2["在哪里放行 LLM,在哪里划定红线?"] N3["如何让安全路径成为阻力最小的首选?"] end Old -->|AI 大模型普及驱动| New ``` --- ### [13:30](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=810s) 治理路径的三大支柱:自由构建、受控运行与规模化监控 每个人都能亲手解决业务问题固然是一件好事,但也催生了全新的治理挑战。为了在这个新世界建立持久的运作模式,企业需要共同踏上一条规范治理的道路。虽然每家公司的具体落地形式会有所不同,但底层都离不开三大核心支柱: ```mermaid flowchart TD accTitle: 企业 AI 治理的三大核心支柱 subgraph Pillar1["1. 自由构建 (Build with Freedom)"] P1A["所有业务团队皆可使用 AI"] P1B["系统底层防呆:无法构建违规/高危应用"] end subgraph Pillar2["2. 受控运行 (Run with Control)"] P2A["高可靠性与可扩展性"] P2B["敏感系统凭证安全隔离与细粒度鉴权"] end subgraph Pillar3["3. 规模化监控 (Monitor at Scale)"] P3A["IT 与安全团队拥有全局可视性"] P3B["摆脱数十个工具孤岛,统一纳管"] end Pillar1 --> Pillar2 --> Pillar3 ``` - **自由构建(Build with Freedom)**:让组织内的每一个团队都能无门槛地接触 AI,自由构建业务所需的原型与应用;但在系统架构层面做好防护,使构建者从根本上无法创建出不合规或高风险的流程。 - **受控运行(Run with Control)**:确保员工构建的应用在生产环境中能够可靠、稳定且弹性地扩展;严格纳管访问敏感系统所需的凭据,确保权限精准分配,杜绝凭证泄露。 - **规模化监控(Monitoring at Scale)**:赋予 IT 与安全团队覆盖全公司的全局可见性,避免企业内充斥着 80 多个各自为政且不可见的单点 AI 工具;提供统一视窗,无论团队之间是否存在部门墙,都能实时洞察所有正在构建的项目。 --- ### [15:38](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=938s) 安全技术实践与 IT 角色重塑:让合规路径成为最容易的选择 拥抱 AI 的未来绝不是在企业内部禁用或严格限制 AI,而是构建一套在架构层面兼顾“速度”与“安全性”的现代化系统。这意味着所有团队(无论技术背景深浅)都能通过大模型快速将想法转化为可用方案;安全与 IT 团队可以在应用构建的进行时获得实时洞察与控制,而非事后被动追责;同时企业能够自由切换底层 LLM 模型供应商,避免被单一厂商深度绑定。这也为 IT 和安全团队带来了角色重塑的良机——他们不再是被业务视作阻碍业务创新的“绊脚石”,而是驱动全公司生产力提升与安全用 AI 的“赋能者”和“企业英雄”。 在具体架构实践上,构建者可以使用任何自己顺手的工具,例如终端、MCP(Model Context Protocol)、可视化界面或任意模型。但在运行与维护阶段,系统会在底层注入硬核安全机制: - **凭证代理(Credential Proxy)**:彻底解决代码硬编码密钥的风险。即使代码被他人查看,里面也不会有任何敏感凭证,所有密钥与鉴权信息仅在运行时通过代理动态注入。 - **容器化隔离执行(Containerized Executions)**:每次工作流执行都在独立的容器沙箱中进行,既让 LLM 能够在其边界内自由运行,又完全被安全架构兜底。 - **智能化运维监控**:当业务工作流发生故障时,系统会自动发出警报,并引入 AI Agent 针对报错主动推荐修复方案与优化建议,帮助非技术背景的业务人员轻松维护关键流程。 --- ### [17:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=1049s) 行动呼吁与总结:走出安全团队,看非技术团队用 AI 做了什么 在演讲尾声,Rosie Halpin 向参会者提出了一个切实可行的行动建议:第二天回到公司后,主动去和那些非技术团队聊一聊——无论是市场营销、法务、人力资源还是营收运营(RevOps)团队。 去看看这些业务人员已经在用大模型自己构建了哪些工具、解决了什么实际业务痛点,然后仔细审视这些自建应用是否在不知不觉中带来了未经评估的安全与合规隐患。如果现有的安全治理体系无法覆盖这些潜在风险,那就到了重构组织治理路径的时候。 - **展位互动**:Rosie 欢迎大家在演讲结束后前往 Expo 展厅的 Tines 展位(630 号展位)深入交流。她风趣地提到,展位非常好辨认,现场布置了大量鲜花以契合本次“Wild Code”的主题,欢迎大家前去领取 T 恤及精美周边。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=4s) 每个人——真的是每一个人——都成为了构建者(Builder),但有谁在监督这一切吗? — Rosie Halpin > [00:04](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=4s) 77% 的企业组织表示,其内部 AI 的采纳速度已经远远超出了自身的治理掌控能力。 — Rosie Halpin > [03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) 软件构建不再是一项纯粹的技术专长。 — Rosie Halpin > [03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) 这种模式实际上是在用安全换取速度(trading off speed for safety),安全不再是系统构建过程中的固有属性。 — Rosie Halpin > [03:46](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=226s) LLM 确实极度擅长编写代码,但它只解决了能力问题(Capability Problem),完全无法解决问责问题(Accountability Problem)。 — Rosie Halpin > [09:06](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=546s) 野蛮代码的根源绝不是因为内部存在具有恶意企图的攻击者(Bad Actors),恰恰相反,它来自于那些最敬业、最具业务创新意识的优秀员工。 — Rosie Halpin > [11:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=682s) 全面禁用 AI 不仅无法奏效,反而会适得其反——只会把所有野蛮代码推向冰山更深处,变成更加不可控的影子 AI(Shadow AI)。 — Rosie Halpin > [11:22](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=682s) 开辟第三条道路:构建一种系统架构本身即内嵌安全性(Safety inherently within it)的治理框架。 — Rosie Halpin > [12:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=749s) 如何让安全合规的路径成为最显而易见且阻力最小的选择至关重要。如果安全流程充满摩擦,员工就不会遵守;合规路径必须是全公司最简单、大家最乐意使用的途径。 — Rosie Halpin > [15:38](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=938s) IT 和安全团队不再是被业务视作阻碍创新的绊脚石,而是驱动全公司生产力提升与安全用 AI 的赋能者和企业英雄。 — Rosie Halpin > [17:29](https://www.youtube.com/watch?v=bMoCwOzGUFY&t=1049s) 第二天回到公司后,主动去和那些非技术团队聊一聊——去看看这些业务人员已经在用大模型自己构建了哪些工具、解决了什么实际业务痛点。 — Rosie Halpin ## Mentioned Rosie Halpin, Tines, Claude Code, Codex, MCP (Model Context Protocol), Ai4 ## Video description Rosie Halpin - Product Manager, Tines The commoditization of building via AI-assisted platforms or AI coding agents (Claude Code, Codex, etc.) is solving problems across the business faster than ever before. One team's productivity gain is another team's blind spot. Often companies are spinning off ICs into AI councils and governing bodies, which solves part of the problem – helping people deploy their work more safely. It still lacks scale. How do you build a lasting model for this new world? What considerations do you need to make? Where do you trust the LLM vendor vs. a third party? Join Rosie Halpin, Product Manager at Tines, as she talks through what Tines learned in their journey to power the world’s most important workflows. She’ll share how our customers led them to rebuild their software from the ground up to address our mission in the context of an AI-native workforce. [Source](https://www.youtube.com/watch?v=bMoCwOzGUFY)

Source quotes

[0:04] Rosie Halpin: 每个人——真的是每一个人——都成为了构建者(Builder),但有谁在监督这一切吗? [0:04] Rosie Halpin: 77% 的企业组织表示,其内部 AI 的采纳速度已经远远超出了自身的治理掌控能力。 [3:46] Rosie Halpin: 软件构建不再是一项纯粹的技术专长。 [3:46] Rosie Halpin: 这种模式实际上是在用安全换取速度(trading off speed for safety),安全不再是系统构建过程中的固有属性。 [3:46] Rosie Halpin: LLM 确实极度擅长编写代码,但它只解决了能力问题(Capability Problem),完全无法解决问责问题(Accountability Problem)。 [9:06] Rosie Halpin: 野蛮代码的根源绝不是因为内部存在具有恶意企图的攻击者(Bad Actors),恰恰相反,它来自于那些最敬业、最具业务创新意识的优秀员工。 [11:22] Rosie Halpin: 全面禁用 AI 不仅无法奏效,反而会适得其反——只会把所有野蛮代码推向冰山更深处,变成更加不可控的影子 AI(Shadow AI)。 [11:22] Rosie Halpin: 开辟第三条道路:构建一种系统架构本身即内嵌安全性(Safety inherently within it)的治理框架。 [12:29] Rosie Halpin: 如何让安全合规的路径成为最显而易见且阻力最小的选择至关重要。如果安全流程充满摩擦,员工就不会遵守;合规路径必须是全公司最简单、大家最乐意使用的途径。 [15:38] Rosie Halpin: IT 和安全团队不再是被业务视作阻碍创新的绊脚石,而是驱动全公司生产力提升与安全用 AI 的赋能者和企业英雄。 [17:29] Rosie Halpin: 第二天回到公司后,主动去和那些非技术团队聊一聊——去看看这些业务人员已经在用大模型自己构建了哪些工具、解决了什么实际业务痛点。

Source quotes

[0:04] Rosie Halpin: Everyone, and I really do mean everyone, is a builder now, but is anyone watching? [3:46] Rosie Halpin: LLMs are solving this capability problem, but they're not solving the accountability problem. [15:29] Rosie Halpin: The future really isn't to stop using AI or restrict AI within your organization. Instead, it's to create a system that allows and architects for both speed and security.

Original transcript

[0:04] I'm Rosie Ellison. Let's go ahead and get  started talking about the fact that everyone, [0:08] and I I really do mean everyone, is a  builder now, but is anyone watching? [0:15] So, with AI, more than ever now, people are able  to solve their own problems themselves. We have [0:23] less technical teams, non-technical teams, people  who are not software developers by trade, people [0:30] in marketing, legal, HR, that are now able to  use AI, to use LLMs, to solve their own problems. [0:39] And that's great, right? We're able to empower  people who are experts in their own processes [0:45] to solve their own problems without having to to  find requirements and get technical engineering [0:50] help and go back and forth and iterate. However,  what's not so great is that the sheer speed [0:57] of that AI adoption is really fast, and it's  hard to keep up from a governance standpoint. [1:04] And this isn't a hypothetical risk. This isn't  something I'm predicting in the future. This is [1:09] a pain that's happening right now. In fact, 77%  of organizations are reporting this pain today, [1:17] that their AI adoption is outpacing that of their  governance capabilities. And that creates a really [1:25] tough trade-off for a lot of leaders, right?  They're being mandated to give an enable AI [1:31] adoption throughout their organization, but they  still have to be liable and compliant with all [1:38] of their governance and compliance mandates and  risks in the context of the risk profile of their [1:44] organization. And those two are are two very hard  mandates to to follow and uphold at the same time. [1:51] So, if we think about how AI adoption works  at organizations, it tends to come in waves. [1:57] The first wave is one in which works with the  existing governance processes. You have technical [2:04] teams, software engineers, developers that are  now using LLMs to help them write code. And that's [2:10] still going through the existing processes, the  existing security testing, governance efforts, [2:16] and visibility that IT and security have. Then we  have that second wave. And in that second wave, [2:23] you have your less technical teams that are now  beginning to use and be exposed to AI through the [2:30] solutions that they already use. So, the solution  marketing uses every day has introduced a new AI [2:37] interface or widget or some way to interact  with AI in that platform, and they're getting [2:42] more exposure there. What's really interesting  to me is this third wave that's happening, [2:47] where now that people are used to AI and beginning  to adopt it more, they're looking to purchase [2:54] or maybe even build themselves in-house AI point  solutions for their vertical. So, marketing has [3:01] one, legal has one, HR has one. And nothing's  wrong with that. It's really important for teams [3:07] to be able to advocate for themselves and solve  their own problems with the tools that they need [3:12] aligning with what's happening right now in in  AI advancement, but it's creating a new problem [3:19] that is really interesting to me in that now  you might have 80 different AI point solutions [3:27] for 80 different teams in your organization. Or  you might have ones you don't even know about, [3:32] teams that have built something themselves that  IT and security is not even aware of. And that's [3:38] really hard to manage at scale. That's really  hard to have visibility and governance over. [3:46] And so, the reason that third wave of  adoption is able to occur is because building [3:52] is really no longer a technical  skill. In our traditional state, [3:56] prior to LLMs being really good at coding  and and something everyone can use, [4:01] you had a really natural choke point where you  had to work with a software engineering team [4:07] who had processes in place and safety  embedded architecturally in that system [4:13] to ensure that you were compliant and ensure  that your processes were appropriately governed. [4:20] Fast forward to our current state, and that's not  necessarily the case. In that current state, we're [4:26] seeing that because there's no technical barrier  to entry, everyone can build. Everyone can code [4:33] using an LLM, but not everyone who's doing that is  going through that existing traditional process to [4:39] make sure that security tests are being followed,  to ensure that that code is compliant, to ensure [4:44] that you are still adhering to those compliance  and governance requirements for your organization. [4:52] And so, there's a trade-off here that's  happening where in the current state, [4:56] it's a lot faster to build, but you're trading  off speed for safety. Safety is no longer [5:02] an inherent part of that current state process.  And the reason that building is no longer [5:09] a technical skill is because LLMs are really good  at it, right? And they are, to be very clear. [5:16] I use LLMs to write code all the time. But an  LLM doesn't have a context of your organization. [5:24] And so, LLMs are solving this capability  problem, but they're not solving the [5:29] accountability problem. An LLM could produce  a really great solution for my problems, [5:37] but it doesn't necessarily know that that's a very  sensitive process and that this is overexposing [5:43] data that shouldn't be, or that there's more  access here than security or IT would like. And so [5:50] that's a really big risk when you have when your  new system is inherently dependent on an LLM that [5:57] isn't aware of the consequences and the ownership  and the liability of the risk that's occurring. [6:06] And so the word that we've coined for  for this for that invisible ungoverned [6:12] AI-generated code that different teams across  your organization are are using is wild code. [6:19] And wild code is not in one particular place. It's  not that easy to find. It's scattered. It's It's [6:26] built with the best intentions,  but it's built by people who [6:30] are not trained in software development, who are  not aware of decades of credential security and [6:37] attacks and understanding what  potential liability exists in code. [6:43] I really like to think of it like an iceberg  analogy. So if you think about that traditional [6:49] state and and what your software development teams  create, that's the tip of the iceberg. That's what [6:54] you can see above the water. And that's what IT  and security is aware of, the risks that they have [7:02] understood and have controls in place to adhere  to. But where wild code lives is beneath the water [7:10] and you don't actually know how big it is and and  you don't even know if how much it exists until [7:16] probably something bad has happened and and  that's why leadership has gotten involved. [7:22] So you might be thinking at this  point, all right, all right, Rosie. So [7:26] wild code probably exists in my organization.  It's going to be hard to find. What are the [7:31] real risks? What are the real hidden costs  of that wild code? And there there are a few [7:37] I really want to make sure we're thinking about.  The first is around the technical debt of having [7:44] 10 different people build the same thing. And  that's a lot of token costs. That's a lot of [7:50] missed opportunity for collaboration. And now you  have 10 different existences of basically the same [7:56] process, maybe within a team, maybe across team  silos. That's that's a lot of money and that's [8:02] not only inefficient, but but costly to maintain.  The second is around exposed credentials. [8:09] So, if you're building anything that's that's  particularly valuable in an organization, it's [8:13] likely going to be connected to your proprietary  systems. And in order to connect to those systems, [8:19] you need a credential. And how is that credential  being stored in the code? And if someone is able [8:24] to gain access to that code, is there the ability  or the risk there of credential exfiltration? [8:31] And finally, it's the maintenance cost. These  are ideally workflows and processes that are [8:39] being created to help teams in what matters most  for them, their most important work. But are they [8:44] being alerted to any failures? A lot of things  change, a lot of things break. Maintenance and [8:51] continuing to iterate and build and troubleshoot  and being alerted of those errors is a really [8:57] big part of building anything. And is there true  ownership there? How are people being notified? [9:06] I've talked pretty high level. I'm a big  fan of of getting down into the details. [9:10] Let's make it actually real and let's look at  some real examples of what wild code looks like [9:16] in the wild, I guess. So, first, think  about a RevOps analyst. This person has [9:23] built a really phenomenal, with an LLM, built  a really phenomenal CRM lead routing workflow. [9:31] And that's great. It's saving the team a lot  of time. It's providing a lot of value. And [9:36] everyone's really happy it. But there's now a  lot of customer data that's being overshared [9:44] to too many people or people that shouldn't have  access to that and security and IT aren't aware. [9:51] Or take for example, an HR analyst who has built  a really great new solution for a very manual and [9:59] time-consuming process of their onboarding and  offboarding workflow. Major time saver for the [10:04] team, major productivity gains, but now employee  personal information PII is getting transferred [10:12] through systems that it shouldn't be and those  systems aren't being audited appropriately for [10:18] the fact that they now maintain PII. Or finally,  think about a finance associate. Someone who has [10:26] been able to connect up a few different systems,  create really insightful reporting for the team, [10:31] but those connections through those systems  creates a supply chain vulnerability that security [10:36] wasn't in the loop of. And so the the important  thing to call out here and what I really love [10:40] about these examples is it clarifies that wild  code is not about your bad actors, your malicious [10:48] uh people with malicious intent. It's about really  well-intentioned people that have now been given [10:54] tools through LLM usage and access to solve their  own problems. These people are able to get their [11:00] work done faster and solve their own problems and  are fully focused on what's best for their team [11:06] in the process that they're an expert  in. But what they're not an expert in [11:10] and what's creating this risk of wild code  is all of these security vulnerabilities [11:15] and governance and risk and compliance  that's coming with what's being created. [11:22] And so at this point, you might be thinking,  all right, well then maybe I just need to [11:26] completely restrict AI usage and and say no  one uses AI, we can't bear with the risks. [11:32] Um but that's not really a good solution. That's  at one extreme and it's really just going to make [11:38] all of that wild code sink deeper beneath the  water, right? It's just going to make it even [11:44] harder to find because people are going to use it  regardless. On the on the flip side, you might be [11:49] thinking, "Yeah, I agree. So, let's just let it  run wild. Let's let everyone use AI and accept [11:55] the risk of wild code." But as we just covered,  there are actually a lot of really negative [12:00] trade-offs to that, to not having that insight  and visibility over what people are building [12:07] when they don't necessarily know the risks.  And so, neither of these extremes is really the [12:12] answer. We need an answer that allows us to create  a system that has safety inherently within it. [12:21] That that allows people to build safely so  they can't create these non-compliant actions. [12:29] And so, we used to be asking, "Can we build this?  Do we have the resources to build this? Is it a [12:35] priority?" But that question isn't really relevant  anymore, right? The answer is yes, you can build [12:40] pretty much anything. The The questions to be  asking now are "How do I protect the builders [12:47] who don't actually know the risks of of what  they've built?" One that's really interesting [12:52] to me and that I'll come back to you later is "How  do we allow LLMs to move really freely in certain [13:00] situations that I'm comfortable with and where  do we make that trade-off or create that line of [13:05] no, I don't want an LLM to move freely here?"  Where do you decide that that trade-off for [13:10] your organization? [snorts] And the last one is  really around how do you make the safe path the [13:15] obvious one? Because if it's filled with friction  and it's not easy enough, even if it exists, [13:22] people won't follow it. It has to be the easiest  one. It has to be one that people want to use. [13:30] And so I think we're all as we as we look at  this new problem together that's coming from [13:34] something really great, which is the ability for  people to be able to solve their own problems, [13:39] I think we're all going to be journeying down this  governed path together. And I think it'll look a [13:44] bit different for everyone's organization, but  I think there will be a few main pillars there [13:50] that exist throughout. And so those pillars  are to build with freedom, to let every team [13:57] have access to AI to build what they need to with  freedom. However, we need to design that system [14:03] so that what they're building, they aren't  able to build something that's not compliant. [14:08] They aren't able to build something that's  that's risky. And then what they've built [14:14] can actually run with control. It can  run reliably. It can scale effectively. [14:18] It can make sure that any of those credentials  in those sensitive systems being accessed [14:24] are appropriately secured and governed. The people  who have access should have access. And finally, [14:31] monitoring at scale. So IT and security have  that oversight, that visibility across the [14:36] entire organization and what they're building.  Not having 80 point solutions with no way to see [14:42] into all of them what people are building,  but one really nice one really nice broad [14:48] area that you can see everything that people are  building across all of your teams, siloed or not. [14:54] And so the future really isn't to stop using AI  or restrict AI within your organization. Instead, [15:01] it's to create a system that allows and  architects for both speed and security. [15:08] And that means that every single team, technical  or not, will have access to go from an idea [15:15] to a solution using an LLM. That also means that  you're you're to have that visibility and insight [15:22] and control into what's being built as it's  being built, not after the fact retroactively. [15:29] And finally, it means that you  have the ability to choose between [15:34] to choose between LLMs or model  providers and not be locked into one. [15:38] It's also a really exciting way to reframe the  role of security and IT in an organization. [15:44] They are absolute heroes and often times are  seen as blockers, but here they can be seen [15:50] as enablers. They can be seen as enablers to  productivity and AI usage and productivity from [15:56] AI usage within an organization. And so to get  a bit more clear, this means that you can build [16:03] however you'd like. We've already cemented that  building is easy to do. You can use your terminal, [16:10] your MCP, you can use an interface, you can  use whatever models you'd like to build in. [16:15] But the way that you're building is now going to  be architected with security and safety in mind, [16:20] so that when you run, your credentials are not  exposed. They're nowhere in the code to be found. [16:26] And it's happening through an isolated  container every single time it runs. [16:31] And finally, when you monitor and you  have think about that maintenance cost, [16:35] you're actually able to get notified when your  workflow is breaking. You're able to use agents [16:41] that are recommending fixes to errors, that are  recommending improvements, and that's helping [16:47] those non-technical teams continue to maintain  those really important processes. And so, coming [16:54] back to that question I flagged earlier of where  do you let LLMs move and run freely and where do [17:00] you restrict them, this is an example of what  I'm talking about. In this system, for example, [17:06] you could use something called a credential proxy  and that's going to make sure that even if someone [17:11] accesses your code, there's no credential  to be found there because the credential [17:16] is inserted at runtime. Containerized executions  is another example. So, you're able to let an LLM [17:22] move freely, but within the bounds of a system  architected and backed by security and safety. [17:29] I said that this was really top of mind for me at  the beginning of this talk. And I hope that it's [17:37] maybe a little more top of mind for you, too, now.  But if there's anything I'd love for you to take [17:42] away from this and maybe tomorrow to do, it would  be to go talk to your less technical teams. Talk [17:48] to marketing, talk to legal, talk to HR, RevOps.  I'll stop naming teams now. You know the teams to [17:55] talk to. And see what they built. See what they've  been using LLMs to build and what problems they've [18:00] been solving. And then maybe look and see if  there were any risks that were created from [18:07] what they've built. And if there were risks that  are not covered by your existing governance and [18:11] compliance structures, maybe it's time to consider  going down that new governed path together again. [18:18] Thank you all so much for your time and attention.  I hope this was really valuable and and maybe [18:24] sparked some new thoughts for you around uh  just this new era we're in and and everyone [18:29] being able to to build and solve their own  problems themselves and the risks that creates. [18:34] I'd love to continue the conversation. You can  find me at the Tines booth in the expo hall. [18:39] It's booth number 630, but it's a  lot easier to recognize it by the [18:44] big flowers. There's lots of live flowers all  over the booth to go with our wild code theme. [18:49] Um so, I'd love to meet some of you there and  chat more and give you a t-shirt or some of our [18:54] other cool swag, but I hope you have a great  day and and thank you all so much. [applause]