Visa, Meta Engineers Shift to Architecture as AI Commoditizes Code

AI将代码商品化，Visa与Meta工程师转向系统架构 随着自动化模型将代码生成变成低成本的通用商品，企业软件的完整性取决于将未经核实的Agent输出隔离在沙盒中，同时将开发者的精力转向高阶系统设计。 专题研讨小组探讨了AI编程工具如何使语法生成平价商品化，从而将真正的工程瓶颈转移到验证、人类架构直觉和零信任沙盒机制上。与会嘉宾分享了实用的企业案例，包括自动化处理非结构化医疗理赔裁决、Visa开源自动化漏洞测试框架，以及通过严格的Token预算上限来管理平民开发者（citizen developer）层级。与会者强调要从基础提示词转向技术上下文工程，并指出计算机科学教育和产品管理必须向系统设计、任务拆解和快速功能原型开发演进，而非单纯依赖手动编码。 代码语法生成已基本解决，但人类直觉、大规模架构设计和自动化验证仍是AI辅助工程的核心瓶颈。 (1:38) 自主Agent集群无需定制OCR流水线，即可处理扫描的历史遗留文档和医疗理赔等复杂、非结构化的现实输入，为企业带来立竿见影的投资回报。 (3:08) 应用安全范式必须从单纯更快发现漏洞，转变为直接在开发者环境内部实现自动化修复和代码打补丁。 (4:58) 针对生产级软件的AI提示需要全面的上下文工程，并起草具有明确不变量的细粒度技术规范，而非随意的对话式提示词。 (7:03) 平民开发者计划赋能非技术员工快速构建内部工具，但要推广至企业级应用，仍需要正规的产品管理流程并交接给工程团队进行维护。 (11:11)

While AI coding assistants have commoditized raw syntax generation and accelerated prototyping, engineering leaders emphasize that operational bottlenecks have shifted to system verification, architectural design, and automated security governance. Verification Bottlenecks and Shifting Engineering Roles (1:38) Nishant Gupta of Meta argues that generating code syntax is largely a solved problem; the central bottleneck is verifying that code adheres to organizational standards, handles high concurrency, and scales to millions or billions of users without failures. Human engineering intuition gained from hard-won failures remains difficult to replicate. Aartee Simran Dhomeja of FedEx observes that while AI assistants do not necessarily free up time, they drastically lower the "cost of exploration" by summarizing documentation instantly. This allows engineers to redirect cognitive effort into auditing architectural choices, pruning over-engineered directory structures, and cross-checking outputs across multiple models. This shift is reshaping engineering education. Justin Johnson of Renaissance Benefits argues that university curricula should pivot away from teaching manual coding syntax toward distributed systems, database indexing, and multi-server architecture, noting that syntax generation costs pennies while architectural problem-solving commands high value. Although Dhomeja contends that baseline literacy in languages like Python and SQL remains indispensable for debugging and iterating on AI proposals, panelists agree that architectural competence now supersedes rote code writing. Security Guardrails and Zero-Trust Sandboxing (4:58) Accelerated development cycles and "vibe coding" introduce vulnerabilities at an unprecedented rate while enabling adversaries to automate exploits. Daniel Fernandez Coviella of Visa notes that passive educational programs failed to curb insecure practices or shadow AI. In response, Visa deployed automated controls such as gateways, proxies, and the open-source Visa Vulnerability Agentic Harness (VVAH/VBA) to automate both vulnerability discovery and code remediation directly within developer environments. Gupta stresses a zero-trust model: enterprise systems must treat AI-generated code as untrusted third-party code that must execute within isolated sandbox environments and pass automated security policy checks before committing to production. At FedEx, Dhomeja integrates AI tools directly into established Software Development Life Cycle (SDLC) governance and prioritizes building modular, pre-vetted agent frameworks to ensure security compliance while preventing costly architectural duplication across teams. Citizen Development Tiers and Cost Management (11:11) Organizations are increasingly extending AI tools to non-technical staff while enforcing strict operational boundaries. Johnson outlines a three-tier model for citizen development at Renaissance Benefits: applications built for personal use, tools designed for specific teams, and company-wide software. Once an internal application crosses into enterprise-wide usage, dedicated product management and professional engineering teams assume long-term support, infrastructure monitoring, and deployment pipelines. To prevent cost overruns—where unmonitored model runs can consume up to $800 in a single workflow—Johnson recommends strict tiered governance. Renaissance Benefits implements an enterprise Claude base seat ($20 monthly per user) alongside monthly consumption caps: $10 to $20 for casual query users, and $75 to $100 for active citizen developers and engineers. At FedEx, Dhomeja addresses scalability by cultivating localized "AI literacy," training edge business units to build initial prototypes using their own data. This enables teams to identify compliance blockers early and filters out non-viable projects before core data science teams commit months to dead ends. Technical Specifications and Spec-Driven Prototyping (7:03) To generate production-ready code, panelists argue that prompting must evolve from casual conversational dialogue into rigorous technical specification. Gupta advocates drafting explicit technical specifications that define unchangeable constraints, invariants, target KPIs, and organizational standards, prompting the model to generate multiple options and perform self-critiques before human review. Johnson adds tactical practices: dictating prompts verbally, keeping feature requests modular, and instructing models to generate Markdown documentation inside repositories so future maintainers understand the architecture. AI coding tools are also transforming cross-functional collaboration. Gupta notes that product managers at Meta use AI assistants to generate functional proofs of concept (POCs) rather than authoring static Product Requirement Documents (PRDs), handing working prototypes to engineers for production hardening. Furthermore, engineering leaders are expanding AI beyond code generation into cross-functional QA automations, such as linking Jira requirements directly to models like Claude for automated functional verification. The panel concludes that as AI coding tools commoditize code creation, industry scrutiny must shift from what models generate to how rigorously outputs are evaluated, sandboxed, and architected. Key operational challenges remain unresolved, including closing the velocity gap between accelerated engineering teams and slower adjacent business units, and managing the hidden technical debt of AI-generated systems.

来自Meta、FedEx、Visa与Renaissance Benefits的技术专家指出，AI编程助手已将代码语法生成转变为低成本基础能力，但企业落地的核心瓶颈已转向系统架构验证、自动化安全防护以及跨部门研发流程的重塑。 验证瓶颈与上下文工程重塑 (1:38) Meta主任软件工程师Nishant Gupta指出，编写基础代码语法已基本解决，真正的工程阻碍在于验证：如何确保模型生成的代码符合企业规范、避免安全漏洞并在隔离沙箱中安全投产。同时，AI往往倾向于编写全新的冗余逻辑，而非复用现有的内部或开源框架。在面对支撑数十亿用户或高并发请求的大规模系统架构时，人类工程师长期积累的失败经验与直觉依然无可替代。 在提示词与上下文构建方面，Nishant主张将交互从随意的对话转变为编写详尽的技术产品规范，明确不变约束条件与业务指标。FedEx资深数据科学家Aartee Simran Dhomeja强调，行业正从基础提示词工程过渡到全面的“上下文工程”，因为不合格的系统设计通常源于业务背景输入的缺失而非底层模型本身的缺陷。Justin Johnson则建议采用语音听写输入、保持模块化细粒度需求，并在代码库中保留自解释的Markdown文档，以规避长期维护时的认知断层。 零信任安全防护与自动化治理 (4:58) Visa资深应用安全工程师Daniel Fernandez Coviella指出，“氛围编程（vibe coding）”大幅提升了代码交付速度，但同时也加速了安全漏洞的产生与利用。Visa为此开源了内部的“Visa Vulnerability Agentic Harness（VVAH/VBA）”，推动应用安全从单纯的“快速发现漏洞”转向由自主代理直接修补代码的“快速修复”模式。他指出，由于员工通常不会主动阅读培训材料，仅靠通识教育无法遏制“影子AI”，企业必须通过代理网关和自动化过滤策略实现技术层面的刚性拦截。 Nishant补充强调，AI生成的代码必须被视作未经信任的第三方外部代码，强制在隔离沙箱环境中运行并通过全部安全合规检查。在成本控制与非技术人员治理方面，Justin介绍了Renaissance Benefits的阶梯控制机制：普通查询用户的月度API硬上限为10至20美元，活跃开发者设为75至100美元；尽管行业预估全天候编码可能消耗300至400美元，但其内部全职工程师的实际月均Token消耗在150至200美元之间。企业需严防未经监控的大模型流水线单次执行烧毁多达800美元的失控风险。 公民开发者分级与业务原型分流 (11:11) Renaissance Benefits设立了“公民开发者计划”，为非技术员工提供Claude Code及预设工具栈，使其能自主开发部门工具，但后续的长期运维、部署流水线与基础设施监控由专业工程团队接管。该模式建立了三级升级管理架构：个人级工具、团队级工具与企业级工具。一旦工具晋升为企业级软件，产品管理团队将介入并指派专属代表进行产品化治理。 关于非技术人员是否需要理解底层代码，专家们呈现出不同视角。Aartee基于FedEx的实践认为，即便是产品经理也必须掌握数据结构、数据库关系与逻辑流的基础知识，否则无法迭代和审核AI生成的脆弱代码。Nishant则介绍了Meta的做法：产品经理无需掌握深层工程细节，其核心目标是服务客户与关键业务指标；他们正广泛利用AI工具迅速搭建动态可运行的概念验证原型（POC）来替代传统的静态PRD文档，再由专业软件工程师重构为符合生产标准的稳健系统。 工程认知转移与教育范式重构 (23:32) Aartee指出，AI编程助手并未凭空创造“闲暇时间”，而是彻底重构了认知努力的分配方式。模型大幅降低了探索成本，工程师不再需要耗费大量时间翻阅冗长文档，而是将节省出的时间投入到架构评估、多模型交叉对比（如并行比对Codex、Cursor与GitHub Copilot）以及修正模型过度设计等问题上。AI工具经常会在轻量原型中凭空生成过于复杂的文件目录结构，需要工程师人工介入进行精简剪枝。 在计算机科学教育领域，Justin强调高校课程应从死记硬背编程语法转向教授分布式系统、系统设计与数据库索引等宏观架构能力，因为“代码本身已变得廉价”，高价值在于构建复杂系统的思维。Aartee与Daniel则认为，Python、SQL等基础语言素养仍然不可放弃，缺乏底层逻辑理解将导致工程师丧失调试、分解复杂安全问题与有效迭代AI输出的能力。 讨论总结指出，AI工具链带来的不仅是纯粹的Token消耗，还带来了工具碎片化、隐性技术债务以及上下游职能步调不一致等组织管理成本。当前行业普遍关注AI能实现什么产出，但对生成过程的具体评估机制仍缺乏足够关注，建立严谨的跨领域评估框架与零信任沙箱验证是确保软件生产级质量的核心未决议题。

## Background 该视频由 YouTube 频道 Ai4 于 2026 年 8 月 22 日发布。内容汇集了来自 Renaissance Benefits、Visa、FedEx 以及 Meta 的工程技术领袖与安全专家，探讨 AI 辅助编程带来的生产力提升、代码常见故障模式、规模化部署经验、治理与可解释性，以及 AI 工具如何重塑工程组织文化。 ## TL;DR 本次圆桌讨论深入剖析了 AI 编码助手在企业实际落地中的机遇与挑战。嘉宾们分享了利用智能体处理非结构化数据、推行“全民开发者计划”、降低探索成本以及利用 AI 快速构建端到端 PoC 原型的实战经验。同时，专家们着重强调了零信任安全防护栏、沙箱验证机制、上下文工程的重要性，并指出随着基础编码被商品化，系统架构设计能力与严密的输出评估框架正在成为开发者的核心竞争力。 ## Atmosphere & style 整场圆桌氛围轻松幽默且充满实战锋芒，主持人的自嘲式串场与台下观众的积极提问带动了热烈的互动节奏。来自一线大厂的技术领袖们知无不言，既有对“氛围编码（Vibe Coding）”与非技术人员写代码的直率调侃，也有对生产级安全、Token 预算失控与过度设计的清醒审视。从技术实操到组织文化的碰撞层层递进，既接地气又极具启发性。 ## Key takeaways - [01:38](https://www.youtube.com/watch?v=kL6o44eBxOc&t=98s) AI 编程助手能快速处理基础代码与测试用例，但在高并发海量架构设计所需的工程直觉与生产环境验证上，仍依赖人类经验把关。 - [05:03](https://www.youtube.com/watch?v=kL6o44eBxOc&t=303s) 安全防御重心正从传统的“快速发现漏洞”转向“快速修复漏洞”，Visa 正在开源其自研的 VBA 漏洞智能体框架以加速安全闭环。 - [06:50](https://www.youtube.com/watch?v=kL6o44eBxOc&t=410s) 编写 Prompt 需像撰写严谨的技术规格说明书，行业正从单纯的提示词工程迈向注重业务问题边界与上下文供给的“上下文工程”。 - [10:36](https://www.youtube.com/watch?v=kL6o44eBxOc&t=636s) 企业推行“全民开发者计划”赋能非技术人员自建内部工具，但生产级交付、安全审计与底层基础设施运维仍需专业工程师接盘兜底。 - [13:03](https://www.youtube.com/watch?v=kL6o44eBxOc&t=783s) 通过普及“AI 素养”赋能业务前线自主打磨原型，能够在早期快速锚定技术断点并及时止损，避免中心化团队长周期的研发内耗。 - [16:03](https://www.youtube.com/watch?v=kL6o44eBxOc&t=963s) 企业需针对轻度问答、全民开发与全职工程人员实施分级 Token 预算和模型路由控制，防范重型自动化工具链带来的成本激增。 - [18:10](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1090s) 仅靠合规培训无法杜绝安全漏洞，企业必须在底层网络架构中架设统一的安全代理与访问网关，实现无感且强制的防护。 - [21:09](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1269s) 对 AI 生成代码应采纳零信任与沙箱隔离机制，所有策略检查通过后方可部署，并通过沉淀可复用架构框架平衡合规与交付成本。 - [23:32](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1412s) AI 的核心价值在于大幅压降“探索成本”，采用多模型并行与交叉验证技巧，可有效识别并纠正 AI 对简单原型的过度架构设计。 - [25:59](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1559s) 研发团队效能提升后，整体瓶颈往往转移到产品与运营等协作环节，企业需将 AI 工具与方法论向周边部门推广以消除木桶短板。 - [27:11](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1631s) 随着基础代码编写日益商品化与廉价化，高校与工程师的核心竞争力正全面转向高层系统架构设计与底层运行机制把控。 - [32:46](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1966s) 产品经理可借助 AI 将枯燥的 PRD 文档直接转化为可运行的概念验证（PoC）原型，大幅缩短沟通鸿沟并交由工程团队重构为生产系统。 - [35:55](https://www.youtube.com/watch?v=kL6o44eBxOc&t=2155s) AI 应用应突破写代码的边界并向 QA 验收等跨流程环节渗透，同时借助管理层的定期问责建立全员探索 AI 的组织文化。 - [37:31](https://www.youtube.com/watch?v=kL6o44eBxOc&t=2251s) 工程师应善用全套 AI 工具链极速拼装端到端原型以替代文档沟通，同时行业亟需建立跨业务领域的系统化模型输出评估框架。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=kL6o44eBxOc&t=4s) 圆桌嘉宾介绍与开场 主持人 David Savage 幽默地开启了对话，自嘲是全场“唯一的有用笨蛋”（useful idiot），因为他根本不会写代码，甚至从 16 岁起就放弃了数学。但他拥有 19 年的科技从业经验，并从事播客和纪录片制作，因此鼓励台下观众时刻保持警惕，准备随时提出更有深度的问题。随后，四位来自一线的技术专家依次进行了自我介绍： - **Aartee Simran Dhomeja**：FedEx 生成式 AI 卓越中心（Generative AI Center of Excellence）的高级数据科学家，负责处理 FedEx 内部各种类型的数据。 - **Justin Johnson**：Renaissance Benefits 的 AI 与工程副总裁，同时也在初创孵化机构 Overclock Accelerator 担任 AI 工程与 AI 运营的讲师。 - **Daniel Fernandez Coviella**：Visa 应用程序安全团队（AppSec）的高级网络安全工程师，主要职责是协助开发者在保障系统安全的前提下更快、更顺畅地编写代码。 - **Nishant Gupta**：Meta 的主任软件工程师（Staff Software Engineer）兼技术主管（Tech Lead），近几个月来专注于构建训练与推理基础设施以持续提升大模型的能力。 --- ### [01:38](https://www.youtube.com/watch?v=kL6o44eBxOc&t=98s) 编写代码中的痛点与“混乱之处” David Savage 将第一个问题抛给了身旁的 Nishant Gupta，询问当前工程师在使用 AI 智能体（Agents）协助编程时试图解决什么问题，以及现实中最“混乱”（messy）的盲点究竟在哪。Nishant 总结了过去 9 到 10 个月中团队借助编程助手的真实体验。在他看来，基础工作其实已经很轻松：理解现有代码库、自动生成测试用例、甚至做空间探索分析备选方案。真正棘手的是人类通过反复试错、手写代码所沉淀下来的“工程直觉”（intuition）。面对高并发、海量用户架构以及扩展性设计时，AI 仍旧无法替代人的经验判断。此外，**验证（Verification）** 构成了另一大难关——如何确保生成的代码符合团队特定规范、个人代码风格，且没有安全隐患？如何打通沙盒执行环境并将代码落地到端到端具备反馈回路的生产环境中，让模型真正了解哪些代码管用、哪些失效？ 针对“混乱”的业务场景，Justin 从医疗与保险领域的视角分享了经验。在高度合规的医疗场景中，向大模型传输数据必须极其严谨。他半开玩笑地透露了一个保密项目：Renaissance Benefits 正在构建一个智能体集群（Fleet of Agents），用于自动化处理传统理赔裁决团队（Claims Adjudication）繁琐的业务流程。理赔数据极度碎片化且非结构化——可能是一封普通邮件、一张医疗 X 光片，或者一份扫描件 PDF。智能体需要自主理解混乱的非结构化输入，并做出合理的裁量决策。 ```mermaid flowchart TD accTitle: 医疗理赔非结构化数据自动化处理流 subgraph Inputs["非结构化输入数据"] A1["理赔邮件"] A2["X光影像片"] A3["扫描件 PDF / OCR"] end subgraph AgentFleet["智能体集群 (Agent Fleet)"] B["多模态理解与解析"] C["智能裁决与决策逻辑"] end subgraph SystemActions["业务执行与验证"] D["自动化理赔裁决处理"] E["安全隔离与审计流"] end A1 --> B A2 --> B A3 --> B B --> C C --> D C --> E ``` 针对这种非结构化扫描数据，Aartee 深有感触地补充道，在 FedEx 这样历史悠久的大企业中，以前必须手工编写复杂的 OCR 数据清洗算法与流程，甚至专门训练处理模型。现在，智能体的多模态理解能力直接帮工程师免去了编写这些繁复手工胶水代码的负担。 --- ### [05:03](https://www.youtube.com/watch?v=kL6o44eBxOc&t=303s) Visa 开源漏洞安全工具与架构复用权衡 针对医疗数据合规的高压，Daniel 笑着表示作为应用安全专家，听到这类敏感数据确实让人心惊肉跳，庆幸自己是在金融领域的 Visa。最近几周各类模型密集迭代，AppSec 工程师面临着全新态势：虽然开发者写代码的速度成倍提高，但安全漏洞也在激增，攻击者利用 AI 发动攻击的速度同样在大幅加快。 为了扭转防御端被动的局面，Daniel 透露 Visa 正在将其内部自研的安全工具进行开源，命名为 **Visa 漏洞智能体框架**（Visa Vulnerability Agentic Harness，简称 **VBA**）。传统安全的思路一直是“如何更快发现问题”，而 VBA 的核心使命在于促成思维的代际转变——**“如何更快修复问题”**（how can we fix stuff faster），从而极大缩短端到端的脆弱性消除闭环。 在 Daniel 分享完之后，Nishant 补充了开发中最棘手的另一个“混乱之点”：工程师必须清楚**何时不要重新发明轮子**。依靠敏锐的架构直觉在“现有的外部框架或解决方案”与“自研定制”之间做出恰当取舍，是当前极为困难但必须面对的挑战。 --- ### [06:50](https://www.youtube.com/watch?v=kL6o44eBxOc&t=410s) Prompt 工程、上下文工程与协作心法 面对主持人关于“如何写出好 Prompt 帮助团队生成高质量代码”的提问，各位专家给出了极具操作性的建议： 1. **化身产品经理（Product Manager）思考**：Nishant 指出，写 Prompt 绝不能像聊天一样随意，而要将其视作一份严谨的技术规格说明书（Technical Specification）。工程师需要明确系统假定、不变量（Invariants）、优化的 KPI 指标、组织历史最佳实践规范；让模型给出多套方案并自我批判（Critique），由人类工程师保留最终裁量权。 2. **拒绝手打输入与模块化拆解**：Justin 犀利地调侃道：“现在已经是 2026 年了，如果你还在手动打字输入 Prompt，请立刻停下。”目前优秀的语音输入工具完全可以解放双手。在编写代码任务时，简短且结构化、有测量尺度的 Prompt 远胜过宽泛的大词（比如“帮我写一个估值百万美元的初创项目”）。 3. **自动生成代码知识地图**：Justin 建议在 Prompt 中加入显式指令，要求模型在 `documentation` 目录下生成一份 Markdown 说明文档，用自然语言分步解析类似结账功能的业务逻辑，并列出核心逻辑存放的文件名。这样在数月之后系统需要迭代维护时，开发者只需把该文档喂给编程助手作为上下文索引。 4. **从 Prompt 工程迈向“上下文工程”（Context Engineering）**：Aartee 指出，目前行业前沿正快速向上下文工程演进。代码写得不对往往不是底层工具（无论使用 Codex、Cursor 还是 Claude）的问题，而是业务问题的定义上下文出现了偏差。智能体即便读遍了代码仓，若缺少准确的业务问题边界与背景，依然无法交付符合实际商业需求的代码。 ### [10:36](https://www.youtube.com/watch?v=kL6o44eBxOc&t=636s) 上下文工程与“全民开发者计划”：让非技术人员写代码 在探讨技术演进时，Aartee 回忆起自己最初构建对话式聊天机器人的经历：当时产品上线仅三个月便迅速迭代过时，但由于团队未曾保留详尽的 Markdown 架构说明文档，导致后续新项目不得不一切从零重构。她感慨道，如今“上下文工程（Context Engineering）已然成为新的提示词工程（Prompt Engineering）”。随后主持人 David Savage 现场发起举手调查，发现会场内绝大多数观众皆为软件工程师，仅有少数非技术背景从业者。这随即引出了 Renaissance Benefits 正在实践的激进创新——副总裁 Justin Johnson 详细介绍了他们推行的“全民开发者计划”（Citizen Developer Program）。 Justin 坦言，这套模式初听起来会让传统软件工程师感到“毛骨悚然”，但实际运转效果却出奇良好。公司为所有员工（不论是否具备技术背景）开通 Claude Code 账号，并预置了一批定义好核心技术栈架构的“技能包”（Skills），赋能大家自由构建针对个人、团队乃至全公司的内部工具。然而对于台下的工程师而言，真正的“噩梦”往往在于交接环节——一旦非技术人员借助 Claude 把应用搭建成型，随后的运维支持、代码维护、生产部署以及底层基础设施，便会全盘移交给专业工程团队兜底接盘。 更有趣的现象体现在不同角色与 AI 沟通的心智模型差异上。Justin 观察到，诸如 C-Suite 高管等偏向宏观愿景导向（Vision-based）的用户，对编程助手的交互极为干脆直白：“我需要这个功能，直接做出来”；相反，资深软件工程师在提示时则充满审慎与条条框框，不仅严格限定实现路径，还会反复确认边缘情况并强制生成单元测试。 ```mermaid flowchart TD accTitle: 全民开发者计划交付流程 subgraph S1["需求与构建阶段"] A["非技术人员 / 高管"] -->|"愿景式提示词"| B["Claude Code 辅助开发"] B --> C["企业预置技能包 / 技术栈脚手架"] end subgraph S2["测试与分级阶段"] C --> D{"应用层级判定"} D -->|"个人级 / 部门级"| E["团队自用与快速实验"] D -->|"公司级 (Tier 3)"| F["引入产品管理团队 (PM) 介入"] end subgraph S3["交付与运维阶段"] F --> G["移交专业工程团队"] G --> H["合规审计、自动化测试、CI/CD 与基础设施运维"] end ``` --- ### [13:03](https://www.youtube.com/watch?v=kL6o44eBxOc&t=783s) 联邦快递的“AI 素养”赋能与去中心化原型验证 针对非专业人士能否真正成为值得信赖的开发者，David Savage 提出了自我怀疑：正如他制作纪录片必须仰赖专业摄影师 Chris 一样，缺乏对写代码的狂热激情，外行是否注定无法写出媲美专业工程师的高质量系统？对此 Justin 给出了解释与宽慰。他指出，在很多庞大组织内部，员工常因惧怕犯错而陷入瘫痪，担心“第一次如果没提示好，以后就再也改不回来了”；但现代 AI 辅助开发的本质在于极速迭代，就像在短短二十分钟的圆桌讨论期间，一款正在构建的 App 便足以推倒重来调整六七次之多，因此完全不必心怀畏惧。 随后，联邦快递（FedEx）的 Aartee Simran Dhomeja 分享了他们在大型企业中推行的“AI 素养”（AI Literacy）计划。在 FedEx 组建首个生成式 AI 团队之初，数据科学团队曾承担着沉重的研发过载——他们需要为各个业务部门耗时数周甚至数月制作产品原型，且必须在前期处理安全、治理以及权限审批流程。然而，这种中心化模式暴露了严重的弊端：大量耗时耗力的原型因无法触及特定业务数据，或在业务端缺乏真实落地价值而最终胎死腹中。 FedEx 通过 AI 素养培训将主动权交还给业务前线，实现了流程再造： - **数据自主与技能上行**：赋能各小型业务团队自主掌控专属数据，并利用 AI 辅助自行打磨初期原型。 - **精准锚定技术断点**：业务团队在亲手实践中明确了“系统能做什么”与“系统缺失什么”，进而带着极度明确的技术拼图缺口向专业 AI 团队寻求攻坚支持。 - **及时止损机制**：一旦在早期交互中发现断点无法弥合，业务侧即可在起步阶段果断将项目搁置，避免了专业技术团队徒耗三至五个月人力后才不得不放弃的巨大内耗。 --- ### [16:03](https://www.youtube.com/watch?v=kL6o44eBxOc&t=963s) 现场答疑：非技术人员实验的 Token 成本与项目管理边界 在现场提问环节，前排观众首先提出了一个直击财务痛点的现实问题：当全公司非技术人员都在使用 AI 进行自由实验时，管理层应如何精准监控并收敛因盲目调用带来的飞涨成本？ Justin 拆解了 Renaissance Benefits 的阶梯式预算与模型分流机制： - **基础席位与模型分配**：采用 Claude Enterprise 企业账户，基准费用为每席位每月 20 美元（最低 20 至 25 个席位起订，不含实际 Token 用量）。 - **日常问答轻度用户**：针对仅把 AI 当作日常问答助手的普通员工，系统限制其模型选用范围，并将每月 Token 预算上限硬性设定在 10 至 20 美元之间。 - **全民开发者用户**：对于投身“全民开发者计划”构建内部工具的员工，企业开放更高级别的顶尖模型，并将额度放宽至每月 75 至 100 美元。 - **全职开发人员预算现实**：业内许多研究推算，若软件工程师全天 8 小时全负荷编写代码，每月 Token 消耗约在 300 至 400 美元；但 Justin 指出这只是理论峰值，实际平均支出通常仅为该数字的一半。他同时特别警示，若未经约束直接调用高昂的自动化代理或重型工具链（如使用 Fable 构建复杂应用），单次任务可能瞬间烧掉 800 美元，因此企业必须在底层实施严密的分级控制。 紧接着，另一位现场观众追问：“在全民开发的全新格局下，项目管理（PM）团队应该处于怎样的坐标？”Justin 回应道，在其组织架构中产品团队与工程团队属于平行部门。企业根据应用影响范围设立了三级分类（个人级、团队级、全公司级）。一旦某款工具跨越边界上升为“全公司级应用（Tier 3）”，产品团队便会正式指派代表进驻接管，将该内部产品纳入正规的产品生命周期与需求演进路线中。 --- ### [18:10](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1090s) 安全底线与组织防线：从规章培训转向自动化防护网关 针对 AI 自动生成的代码究竟能否令人放心这一议题，Visa 的资深应用安全工程师 Daniel Fernandez Coviella 幽默地指出，在安全专家的眼里，企业对安全维度的重视“永远都嫌不够”（Never enough）。随着越来越多缺乏安全底线的非技术人员涌入编码领域，企业绝不能指望使用者具备代码审计能力，而是必须抢先一步在系统层架构起牢固的“防护栏”（Guardrails）。 Daniel 强调，面对 AI 辅助开发，企业必须坚守安全基本盘，恪守“相信但必须验证”（Trust but verify）的核心原则： - **提示词与代码安全前置**：投入大量精力打磨输入提示词与安全过滤规则，力求让 AI 模型在绝大多数情境下输出原生安全的代码结构。 - **双向边界防护机制**：对输入端（防止敏感、机密商业数据外泄至模型）与输出端（防止引入 SQL 注入、硬编码凭据等漏洞）建立端到端的防护，让开发者能够心无旁骛专注于业务逻辑本身。 - **全流程自动化验证**：维持甚至提升现有的安全扫描与授权测试强度，避免对自动化生成物产生盲目信任。 针对主持人提出的高成本是否会诱发员工绕开安全框架催生“影子 AI”（Shadow AI）的隐患，Daniel 分享了 Visa 早期治理策略的惨痛教训。他坦诚表示，团队最初寄希望于通过“搭建学习路径与安全培训课程”来规范开发者行为，但事实证明这种教育导向的尝试“彻底走错了路”，因为企业根本无法保证每位开发者都会逐条研读并严格执行繁复的合规守则。最终的破局之道依然在于硬性技术治理——企业必须在底层架构中架设统一的安全代理与访问网关（Security Proxies & Gateways），将防护栏与拦截策略无感且强制性地内嵌在网络层与基础设施当中。 ### [21:09](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1269s) 零信任安全模型与可复用框架治理 Daniel Fernandez Coviella 强调，在面对 AI 生成的代码时必须保持一种“悲观主义”视角，即假定所有外部输入和代码本质上都是不安全的，必须遵循“先验证再信任”（Trust, verify）的原则，并在整个交付链路中叠加层层安全过滤。Nishant Gupta 完全赞同这一观点，指出应当将编码系统的产出视作来自不受信任的第三方。为了建立真正的工程信任，必须构建包含沙箱执行环境（Sandbox Execution Environment）与零信任（Zero Trust）执行模型的自动化工作流——所有由安全系统制定的策略检查必须全绿（All green）通过，否则严禁合并或部署到生产环境中执行。 ```mermaid flowchart TD accTitle: AI代码沙箱与零信任安全验证流程 subgraph AI_Input ["AI 生成层"] A["AI 编码助手 / Agent"] --> B["待验证代码输出"] end subgraph Security_Gate ["治理与隔离环境"] B --> C["沙箱执行环境 (Sandbox)"] C --> D{"零信任策略检查"} end subgraph Production_Deploy ["生产交付"] D -- "全部通过 (All Green)" --> E["合并并部署至生产环境"] D -- "策略违规 / 失败" --> F["拦截并打回重构"] end ``` Aartee Simran Dhomeja 从 FedEx 的实践出发补充道，安全是企业级交付的首要考量，AI Agent 必须严格嵌在现有的系统化治理框架内，传统的软件开发生命周期（SDLC）、严格的代码审查以及测试流程绝不能因引入 AI 而被绕过。针对主持人 David Savage 提出的降本问题，Aartee 指出团队的核心策略是构建“可复用且可重复”（Reusable and repeatable）的成熟框架。许多业务团队都会提出定制对话式 Agent 的需求，与其每次从头构建，不如在已通过严格安全合规验证的既有框架上进行二次开发，这在保障安全底线的同时大幅压降了研发与试错成本。 - **零信任策略拦截**：未经沙箱严格测试并符合所有预设安全策略的代码，一律不得进入代码仓库和生产集群。 - **治理与 SDLC 一体化**：AI 编码工具必须服从既有的工程治理规范，不能破坏现有的测试和 Review 机制。 - **资产复用降本**：通过沉淀标准化的对话 Agent 框架，实现安全基线与交付效率的平衡。 --- ### [23:32](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1412s) “探索成本”降低与双模型交叉验证技巧 讨论转向 AI 是否真正为开发者带来了“更多空闲时间”。Aartee 坦言事实并非如此，时间并没有凭空变多，而是“利用时间的方式发生了根本变化”。AI 最大的价值在于大幅降低了“探索成本”（Cost of Exploration）——以往需要耗费大量精力翻阅冗长文档和历史代码，如今这些低效工作可以直接委托给 AI，从而腾出脑力用于核心逻辑的推导与审视。 在实际操作中，Aartee 透露了她的个人工作流：绝不单纯依赖单一模型，而是采用并行运行（Parallel Run）与交叉验证（Cross-check）的方式。例如将 Codex 生成的代码放入 Cursor 中重构，或者在 Cursor 与 GitHub Copilot 之间进行比对。此外，她指出 AI 经常出现过度设计或“幻觉”： > “很多时候你会觉得它们在产生幻觉，为一个简单的原型应用（Prototype）生成了过于超前、结构过度复杂的架构。这时候你就像是在跟它说：‘老兄，你扯太远了，快收一收。’” Aartee 指出，通常扫读 5 到 15 分钟就能凭借直觉判断出 AI 是否把简单原型做成了庞然大物，进而手动将其拉回正轨；只有面对真正复杂的企业级系统时，才会投入更多时间深入研判其架构设计。 --- ### [25:59](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1559s) 研发效能提升 25% 与跨部门协作的“木桶短板” Justin Johnson 幽默地回应 Aartee 的观点，直言如果这是在教堂，他会大喊一声“阿门”（Amen）以示彻底赞同。在谈到效能指标时，Justin 调侃道：“前几天我开玩笑说工程团队的生产力提升了 25%，然后有人接茬问：那 25% 的人到底长啥样？是从膝盖以下还是从肩膀以上算的？” 笑声过后，Justin 严肃地指出了当前企业普遍面临的新瓶颈：虽然研发团队的效率借助 AI 获得了立竿见影的提升，但周围的配套业务团队（如产品团队、运营团队）的工作节奏仍然停留在前 AI 时代。研发写代码的速度变快了，但上游需求分析和下游交付协同没有跟上，导致整体商业流转速度受限。因此，他们正在将 AI 工具与方法论向周边协作部门推广，以消除组织协作中的“木桶短板”。 --- ### [27:11](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1631s) AI 时代高校计算机教育重心的转移 在现场提问环节，一位来自大学的计算机系教师提问：随着 AI 编码助手的普及，计算机专业的学生应该重点学习什么、放弃什么？ Justin Johnson 结合自身经历分享道，他的二儿子目前在北卡罗来纳大学夏洛特分校（UNC Charlotte）攻读新开设的“AI 工程”（AI Engineering）学位。他认为在传统计算机科学领域，学生不仅要掌握扎实的 Python 编程功底，更必须把核心重心转移到**系统架构（System Architecture）与系统设计（System Design）**上： > “现在的初级工程师不需要再一行行手写代码了，因为代码已经变得极其廉价，写代码现在只要几分钱；但我愿意花成千上万美元雇佣一个懂得构建整体系统的大脑。” 学生必须理解 AI 生成的代码在底层是如何运转的，比如它如何与外部系统互联、底层数据库表是否需要建立索引、多服务器环境下高可用如何保障等宏观架构问题。 Aartee 补充了自己在研究生期间担任助教的经历：自 ChatGPT 问世以来，学生高度依赖 AI 生成代码，但如果自身不掌握 Python、TensorFlow、SQL 等底层核心技术，就无法对 AI 的输出进行审校，更无法在面对业务管理层时解释代码逻辑。此外，软件工程的精髓在于“持续迭代”（Reiteration），完全盲信 AI 就会丧失推敲和迭代的能力，最终无法交付优秀的产品。 --- ### [30:37](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1837s) “氛围编码”（Vibe Coding）与非技术人员的实践困惑 现场主持人抛出领导层面对技术快速迭代的焦虑感，并提到 Daniel 在会前曾自嘲为“业余氛围编码者”（Amateur vibe coder）。Daniel 表示在当前技术浪潮下技术管理者没有退路，只能主动迎难而上。对于网络安全团队而言，AI 工具极大地加速了内部防御工具的构建。但他坚信，自身的计算机专业背景（理解代码语法、软件设计原则和高层设计）是能够驾驭 AI 助手的决定性优势。 随后，前排一位白衬衫观众接过麦克风提问，指出自己是一名完全不懂编程的产品经理（PM），但热衷于使用生成式 AI 工具进行原型开发。针对前述“不懂代码就无法保证准确性”的观点，他提出了困惑：产品经理的专长在于挖掘痛点、理解用户终极价值并进行端到端结果验证，是否只要能验证应用在预定时间内达成了最终目标，即便不懂底层代码也可以胜任？ - **传统代码的商品化**：单纯的基础编码技能价值被大幅摊薄，高层系统设计与架构互联能力成为核心溢价点。 - **氛围编码与工程底蕴**：即使是“Vibe Coding”，具备扎实 CS 功底的工程师在代码质量把控和系统稳定性上也具备显著优势。 ### [32:46](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1966s) 产品经理与代码基础之争：持续精进底层 vs. 快速构建概念验证（PoC） 面对现场关于“大模型未来是否会强大到让人不再需要审查代码、非技术人员是否无需理会底层逻辑”的探讨，Aartee Simran Dhomeja（联邦快递高级数据科学家）首先给出了回应。她呼应了当天上午主旨演讲的核心观点——“在谈论 AI 时，人类绝不是来被取代的（humans are not here to be replaced）”。Aartee 认为，即便作为产品经理（PM）能够洞察企业价值、业务痛点和用户诉求，持续的自我技能提升（upskilling）仍然不可或缺。以数据科学家使用代码智能体（Coding Agent）构建应用为例，如果不理解数据库关联逻辑或迭代需求，就无法把控质量。回顾计算机科学中的 Python 和 SQL 学习，这些基础正是理解系统的基石，没有人天生就会 Debug，但打好地基才能真正具备把控应用落地的能力。 ```mermaid flowchart TD accTitle: 产品经理赋能与研发交付路径对比 subgraph PM_Traditional["传统研发交付"] direction TB A["业务诉求与客户画像"] --> B["撰写传统 PRD 文档"] B --> C["交接给工程团队开发"] end subgraph PM_AI_Assisted["AI 驱动的原型交付 (Meta 实践)"] direction TB D["业务诉求与客户画像"] --> E["借助 AI 工具自主构建 PoC 原型"] E --> F["直接展示可运行系统"] F --> G["工程师补足底层并构建生产级系统"] end ``` 随后，Nishant Gupta（Meta 超级智能实验室技术主管）提出了截然不同但极具实战性的视角。他主张产品经理即使不懂底层技术细节，也完全可以利用 AI 释放巨大生产力。产品经理最核心的职责是链接工程团队与终端客户，推动 KPI 增长。在 Meta 的实际工作流中，产品经理不再局限于只撰写枯燥的 PRD（产品需求文档），而是直接使用 AI 工具独自将想法转化为可交互的概念验证原型（Proof of Concept, PoC）。随后，软件工程师再依托其扎实的技术基本功，将这个原型重构为健壮的生产就绪（Production-ready）系统。 - **技能沉淀与调试学习**：Aartee 指出，理解底层基础需要长期积累，面对智能体生成的代码，必须具备判断数据库关系与系统架构合理性的能力。 - **Meta 内部产品交付模式**：Nishant 分享道，利用 AI 将 PRD 转变为直观的 PoC 原型，不仅大幅缩短了跨部门沟通鸿沟，还能在早期阶段精准验证产品构想。 --- ### [35:55](https://www.youtube.com/watch?v=kL6o44eBxOc&t=2155s) 容易被忽视的关键议题：跨流程自动化与全员 AI 问责文化 在圆桌会议进入尾声之际，主持人 David Savage 向 Justin Johnson（Renaissance Benefits AI 与工程副总裁）抛出一个核心问题：“在这个领域中，有哪些至关重要但常被埋没、参会者更应该带回去深入探讨的议题？” Justin 认为，除了已经被广泛讨论的成本问题之外，最大的盲区在于**工程研发之外的端到端业务流程赋能**，例如 QA 测试与产品验收环节。 Justin 举例说明，团队完全可以通过集成工具将 Jira 与 Claude 等大模型直接打通，让模型根据业务需求文档自动执行功能性验收测试，核对交付内容与原始需求是否吻合。这种跨流程的自动化协同，在许多企业中尚未得到充分重视。此外，Justin 强调了在组织内部推动“压力测试同行”的必要性，软件工程师和各部门成员应主动交流彼此的 AI 使用方法。他分享了一个生动的企业管理细节：他们公司的 CEO 在每周全员例会（Staff Meeting）上面对管理团队时，总会直截了当地发问：“你们这周是如何使用 AI 的？（How have you used AI this week?）”在当下的环境下，如果答不上来或者没有使用，那就是一个“极差的答案”，每个人都必须对此给出交代。 - **突破研发边界**：AI 不仅应停留在辅助写代码，更应渗透到 Jira 需求追踪、QA 功能测试验证等跨部门协作环节。 - **强有力的 AI 组织文化**：高层管理者的直接问询推动了团队主动拥抱工具，使得探索 AI 融入日常工作成为每个人的刚性指标。 --- ### [37:31](https://www.youtube.com/watch?v=kL6o44eBxOc&t=2251s) 极速端到端验证与模型评估机制的未来思考 紧接着 Justin 的分享，Nishant Gupta 补充了他经常分享给正在辅导的工程师以及家人的核心理念：**借助现代工具最大程度减少在初期的“人力依赖”**。如今在 UI/UX 设计（如 Figma）、数据分析面板、前端界面搭建以及后端服务等各个细分领域都有大量 AI 工具可用。面对复杂业务问题，工程师应具备深度拆解的能力——将问题拆解为 UI 交互、数据指标大盘与后端架构，并迅速利用工具链拼装出端到端的全套 PoC 原型。 Nishant 强调，最高效的沟通绝不是靠空谈或翻看静态文档，而是“展示真正能运行的代码原型（showing something that's working）”。只有快速迭代出工作原型，才能以最高效率拉入领域专家（Domain Experts）进行深度打磨并平滑过渡到生产环境。 在全场最后 30 秒的结语中，Aartee Simran Dhomeja 提出了关于行业发展的重要警示：当前业界绝大部分注意力都在关注“AI 能做什么（what AI can do）”，却严重缺乏对“AI 是如何做到的（how it is doing it）”的深度审视。她呼吁全行业应重点投入到不同领域和架构下对 AI 输出内容的系统化评估（Evaluation Frameworks），确保 AI 生成结果的可靠性与严谨性。最后，主持人 David 带领全场观众向四位嘉宾致谢，圆桌讨论在热烈的掌声中圆满结束。 - **最小化人力依赖与敏捷交付**：通过综合运用设计、分析与后端 AI 工具，单兵即可快速串联全栈 PoC，让原型替代传统文档成为最高效的沟通媒介。 - **亟待深化的评估框架**：面对生成式 AI 的广泛应用，建立跨业务领域的评测基准与输出评估机制是保障落地质量的核心前提。 ## Notable quotes > [06:50](https://www.youtube.com/watch?v=kL6o44eBxOc&t=410s) 现在已经是 2026 年了，如果你还在手动打字输入 Prompt，请立刻停下。 — Justin Johnson > [10:36](https://www.youtube.com/watch?v=kL6o44eBxOc&t=636s) 上下文工程已然成为新的提示词工程。 — Aartee Simran Dhomeja > [18:10](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1090s) 在安全专家的眼里，企业对安全维度的重视“永远都嫌不够”。 — Daniel Fernandez Coviella > [23:32](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1412s) 很多时候你会觉得它们在产生幻觉，为一个简单的原型应用（Prototype）生成了过于超前、结构过度复杂的架构。这时候你就像是在跟它说：‘老兄，你扯太远了，快收一收。’ — Aartee Simran Dhomeja > [25:59](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1559s) 前几天我开玩笑说工程团队的生产力提升了 25%，然后有人接茬问：那 25% 的人到底长啥样？是从膝盖以下还是从肩膀以上算的？ — Justin Johnson > [27:11](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1631s) 现在的初级工程师不需要再一行行手写代码了，因为代码已经变得极其廉价，写代码现在只要几分钱；但我愿意花成千上万美元雇佣一个懂得构建整体系统的大脑。 — Justin Johnson > [32:46](https://www.youtube.com/watch?v=kL6o44eBxOc&t=1966s) 在谈论 AI 时，人类绝不是来被取代的（humans are not here to be replaced）。 — Aartee Simran Dhomeja > [35:55](https://www.youtube.com/watch?v=kL6o44eBxOc&t=2155s) 你们这周是如何使用 AI 的？（How have you used AI this week?） — Justin Johnson > [37:31](https://www.youtube.com/watch?v=kL6o44eBxOc&t=2251s) 最高效的沟通绝不是靠空谈或翻看静态文档，而是展示真正能运行的代码原型（showing something that's working）。 — Nishant Gupta > [37:31](https://www.youtube.com/watch?v=kL6o44eBxOc&t=2251s) 当前业界绝大部分注意力都在关注‘AI 能做什么（what AI can do）’，却严重缺乏对‘AI 是如何做到的（how it is doing it）’的深度审视。 — Aartee Simran Dhomeja ## Mentioned David Savage, Justin Johnson, Daniel Fernandez Coviella, Aartee Simran Dhomeja, Nishant Gupta, Renaissance Benefits, Visa, FedEx, Meta Superintelligence Lab, Nash Squared, Ai4, Overclock Accelerator, 北卡罗来纳大学夏洛特分校 (UNC Charlotte), Visa 漏洞智能体框架 (VBA), Claude Code, Claude Enterprise, Codex, Cursor, GitHub Copilot, ChatGPT, Figma, Jira, Fable, 全民开发者计划 (Citizen Developer Program), 上下文工程 (Context Engineering), 氛围编码 (Vibe Coding), 零信任模型 (Zero Trust), AI 素养 (AI Literacy) ## Video description Justin Johnson - VP of AI & Engineering, Renaissance Benefits Daniel Fernandez Coviella - Senior Application Security Engineer, Visa David Savage - Technology Evangelist, Nash Squared Aartee Simran Dhomeja - Senior Data Scientist, FedEx Express Corporation Nishant Gupta - Staff Software Engineer, Tech Lead, Meta Superintelligence Lab This panel brings together engineering leaders and technical experts at the intersection of AI and coding who are redefining how software gets written with the help of AI. Topics will include productivity gains from AI pair programming, common failure modes in AI-generated code, and lessons learned from deploying coding assistants at scale. We’ll also touch on trust, explainability, and governance—plus how AI tools are reshaping engineering culture. [Source](https://www.youtube.com/watch?v=kL6o44eBxOc)

[6:50] Justin Johnson: 现在已经是 2026 年了，如果你还在手动打字输入 Prompt，请立刻停下。 [10:36] Aartee Simran Dhomeja: 上下文工程已然成为新的提示词工程。 [18:10] Daniel Fernandez Coviella: 在安全专家的眼里，企业对安全维度的重视“永远都嫌不够”。 [23:32] Aartee Simran Dhomeja: 很多时候你会觉得它们在产生幻觉，为一个简单的原型应用（Prototype）生成了过于超前、结构过度复杂的架构。这时候你就像是在跟它说：‘老兄，你扯太远了，快收一收。’ [25:59] Justin Johnson: 前几天我开玩笑说工程团队的生产力提升了 25%，然后有人接茬问：那 25% 的人到底长啥样？是从膝盖以下还是从肩膀以上算的？ [27:11] Justin Johnson: 现在的初级工程师不需要再一行行手写代码了，因为代码已经变得极其廉价，写代码现在只要几分钱；但我愿意花成千上万美元雇佣一个懂得构建整体系统的大脑。 [32:46] Aartee Simran Dhomeja: 在谈论 AI 时，人类绝不是来被取代的（humans are not here to be replaced）。 [35:55] Justin Johnson: 你们这周是如何使用 AI 的？（How have you used AI this week?） [37:31] Nishant Gupta: 最高效的沟通绝不是靠空谈或翻看静态文档，而是展示真正能运行的代码原型（showing something that's working）。 [37:31] Aartee Simran Dhomeja: 当前业界绝大部分注意力都在关注‘AI 能做什么（what AI can do）’，却严重缺乏对‘AI 是如何做到的（how it is doing it）’的深度审视。

[0:04] David Savage: probably the useful idiot in the room because I cannot code [1:38] Nishant Gupta: Generating code is I think mostly easier, but how do you know if the agent followed the organizational standard practices or my personal style of coding? How do you verify the solution, make sure there's no security leakage, get access to the sandbox execution environments, and take that piece of generated code into production through this end-to-end feedback loop? [7:03] Justin Johnson: build me a million-dollar startup [11:11] Justin Johnson: I need this business outcome, make it happen [18:10] Daniel Fernandez Coviella: never enough [18:10] Daniel Fernandez Coviella: trust, but verify [23:32] Aartee Simran Dhomeja: dude, you've gone too far, come back. [27:11] Justin Johnson: code is cheap and code is easy now—we're paying pennies for that, but I will pay thousands for the mind that can build the system. [36:25] Justin Johnson: How have you used AI this week?

[0:04] Uh, my name is David Savage. Uh, I am probably the  useful idiot in the room because I cannot code. I [0:09] stopped doing maths at 16, but I'm a podcaster and  a documentary filmmaker who's worked in tech for [0:15] 19 years. So, you may well have better questions  than I do. So, please stay alert and we might come [0:21] out to the audience as we go and uh see what  questions you have for our panel. But look, [0:25] let's introduce the experts to go through this  subject. Um let's start on the end. RT quickly [0:30] tell us who you are, who you work for. Um so  I work as a senior data scientist for FedEx [0:35] uh in their under the generative AI center of  excellence. So I work with everything all kind [0:42] of data at FedEx. Um I think you would know  more about me as we go through but that's a [0:47] short introduction of me. Excellent. Justin. So  I'm Justin Johnson. I am VP of AI and engineering [0:53] at Renaissance Benefits and then I am also an  instructor for AI engineering and AI operations [0:59] for a startup called Overclock Accelerator. Danny.  Awesome. I'm Daniel Fernandez. I'm a senior cyber [1:07] security engineer at Visa. Um I'm part of the  appsite team. So, you know, my day-to-day is [1:13] just how trying to help developers code faster  without better, more secure, without, you know, [1:21] making their lives easier. So, fantastic. And  Nishant. Hi everyone. My name is Nishan Gupta. [1:26] I'm a staff software engineing tech lead at Meta.  And uh for the last couple of months, my focus [1:32] has been on building the training and inference  infrastructure to help make our models better. [1:38] Perfect. And as you're sat next to me, I'll pick  on you. um where you're using using agents to [1:42] help you code at the minute. What problems are you  trying to solve? Where is it getting messy? Let's [1:46] kind of have a roll your sleeves up moment. Right,  that's an interesting question. So, personally, [1:52] I've been using coding assistance to help make  me and the team better and more productive for [1:57] the last I would say 9 10 months. I think the easy  part as I think if we have used coding systems in [2:02] our day-to-day has been first understanding the  codebase and being able to do test generation [2:09] being able to analyze the space right understand  various options on the table but the messy part is [2:15] still the intuition the intuition you get from you  know doing things by hand by failing by learning [2:21] the architecture how the system can scale when  there are billions of users or when there are [2:26] thousands of concurrent requests which are coming  in picture and um that has been the difficult part [2:32] lately. So and the last import another important  difficult part which I have observed personally is [2:37] the verification. So generating code is I think  mostly easier but how do you know if the agent [2:44] followed the organizational standard practices  or my personal style of coding or you know how [2:50] do you verify the solution make sure there's  no security leakage right how do you get access [2:54] to the sandbox execution environments how do you  take that piece of generated code into production [2:59] through this end to end feedback loop make sure  that coding assistant is able to understand what [3:03] worked what didn't work and seeing that impact  just I imagine as a VP, you have a a slightly [3:10] different lens on what is messy perhaps to some  of your your some of your staff. Yeah, I think [3:15] a lot of it's messy because when we look at AI  specifically in like a healthcare lens, you have [3:21] to be very very careful about what you're passing  to and from the model and specifically in the [3:26] insurance space. But one of the ways that we're  getting our hands dirty that I really like and I'm [3:30] probably not supposed to talk about it, but don't  tell anybody. I'm going to talk about it anyway. [3:36] When a medical claim comes in, there's a  certain amount of steps that, you know, [3:40] the claims adjudication team has to take when they  work through a medical claim. And we're actually [3:45] looking at building a fleet of agents that handles  a lot of that automatically, but they're also able [3:50] to make smart decisions based upon the data that  they're given. And so this data could come in the [3:55] form of an email. It could come in the form of  an X-ray, in the form of a PDF. So we don't know [4:00] what the data is. It's unstructured if you're  not familiar with that term. And so we have to [4:05] figure it out so to speak and then let the agents  act on that data. So it's it's a very interesting [4:10] project. And again, don't tell anybody I told you  about that. Arthur, you look like you were willing [4:15] to jump in there. Yeah. So just to add to what  Justin said, I think when it comes to messy data, [4:21] uh when we were dealing with data, we do not have  one structured format and a company that old u a [4:28] lot of that could be just scan data. I remember  working with OCR documents and everything. I think [4:33] one good thing that with agents is it at least  eases your life out of writing manual codes for [4:40] processing that OCR. So I think um that has made  my life easier. I'm no more writing algorithms, [4:47] get data and then process it and then  write another model to work on it. So yeah, [4:52] that was something we've kind of been helped with  recently. And Danny, very quickly, when you hear [4:58] someone talk about medical data, you probably have  heart palpitations and you're glad that you work [5:03] for Visa, right? Yeah, pretty much. Um, yeah, I  think I mean, as everybody in the room can relate, [5:08] it's it's been a couple of eventful weeks as an  appseac practitioner, right, with all these new [5:13] models. Um, we're also obviously leveraging all  these new um, you know, coding uh, assistants. [5:20] Um we're particularly focusing on um a new tool  which we build and we're now open sourcing so [5:29] hopefully everybody is able to use it. It's called  Visa vulnerability agentic harness or VBA. Um, [5:35] and what we're trying to do with this, you  know, we've identified that, um, you know, [5:39] with VIP coding, with all these new models, we're  coding much more. We're coding much faster, but [5:45] also the models are finding more vulnerabilities  and the bad guys will also exploit us out faster, [5:51] right? Um, so what we're really leaning into is uh  making a push for tools like this. um give them to [5:59] uh you know for everybody to use um and leverage  and make your own um to really just dig into you [6:06] know whether you are in the medical field or you  are in meta you know like how can we find a stuff [6:13] faster which is always been the theme but now I  think we're shifting more towards how can we fix [6:19] stuff faster right so how can we shorten this this  kind of like end to end so I think that's you know [6:25] that's what's on top of our mind And you know  hopefully everybody in this room look everyone [6:31] on this panel has a slightly different Sorry.  Do you want to jump in? Um no I agree with you [6:35] Danny. Right. I think one thing to add is what I  discovered the messy part is figuring out when not [6:40] to invent something new rather than reusing the  existing whatever framework solutions outside. [6:46] Right. So that has been a really difficult  part which requires a lot of intuition. Yep. [6:50] Cool. Well, look, we've obviously got a range  of different views on this panel and different [6:54] expertise, which is fantastic to have. I'd imagine  that Danny, you might have a slightly different [6:59] view on this next question to everyone else. But  again, Nishan, as you're sat next to me, writing a [7:03] good prompt is almost becoming a skill set in its  own right. How are you helping your colleagues, [7:10] your members of your team to do that in a way  that creates good code, especially when some [7:15] of those jobs are quite hard, right? Uh I think  the way I think about this is just acting like [7:22] putting out a PM hat and uh making sure you know  you know what are your stakeholders what you're [7:29] trying to solve clearly laying out a technical  specification for the coding assistant right [7:35] okay this is what we're trying to build these  are your assumptions these are your invariance [7:38] you should not change this this is the metric or  the KPI we're trying to improve and uh these are [7:44] the organization best practices this is the past  examples of the work, me or the team or whatever, [7:49] right? Some of the better products and systems  have already been implemented and then asking [7:54] the agent to suggest multiple options on the table  to critique its own options, right? And not use [8:00] it as a chat assistant like you are communicating  with someone but treating it as someone who can do [8:04] a lot of research but you have to make that final  call. So the input needs to be very very detailed [8:09] in a such a format that it's able to understand  what it is able to touch what it is able not to [8:14] change what are the whatever tools available to  it. So at least in the last couple of projects I [8:20] have worked on and the team has worked on I think  this has been my go-to approach. So I I think I [8:27] would add to that a couple things. Um one is 2026  and so if you are typing prompts please stop. [8:35] There are voice programs that will just listen  to you and you can just talk to it and put it in [8:39] there because who wants to type anymore, right?  But a few things that I like to tell folks when [8:44] when they're looking at prompts when it comes  specifically to building software applications, [8:49] there's a couple things. I've gotten better  results from short, more structured prompts [8:54] than I have bigger prompts. So instead of build  me a million-doll startup, which is how it works, [9:00] you know, of course, I would say I want to build  a startup. I wanted to do this feature and then I [9:05] would break that feature down a little smaller and  give it more measurable prompts. The second thing, [9:11] you're going to build your app, you're going to  update your existing app, whatever you're going [9:14] to do, and then x amount of months later, you're  going to come back to it and you're going to say, [9:17] I have no idea what what this code is and  what it does. So, as part of the prompt, [9:22] a lot of times I like to say, hey, make a markdown  file in a documentation folder that explains to [9:27] me how the checkout feature works. give me a  bunch of steps so that I can understand it from [9:32] a business perspective and then tell me where the  core logic lives like just give me the file names [9:37] and then that way I can actually tell my coding  assistants to reference that file if I have to go [9:43] make changes to the system after I've already  built it I think um just to add on the on top [9:51] of what Nishant and Justin said um I think he's  right you know it's 2026 you're mostly speaking [9:57] your promps but I think one thing that I very  recently and I would not say it's very old. [10:01] One thing I've recently discovered is there's  a layer of context engineering on top of prompt [10:06] engineering now. So you build around your context.  It's no more about do this for me. It's about the [10:12] business problem. So if your coding assistant  or your coding agent is not developing something [10:19] um right, it might have the best repository,  but is it solving your business problem? No. [10:25] And it's not probably that you're using codeex  versus cursor versus claude. It's basically [10:31] the problem lies in your definition.  So I think building that context and [10:36] um Justin said you know keep a markup file  that is uh you'll thank yourself when you do [10:44] that as engineers. I think once AI came in,  I remember building the first conversational [10:51] uh chatbot and then once it was done 3 months  later it was obsolete but we did not have like [10:57] a markup file explaining why we did what so for  another project we did everything from scratch. [11:02] So I think I learned my lesson then but yes  I think context engineering is the new prompt [11:07] engineering now just interested here show of  hands because we don't know what the audience [11:11] is right AI4 is everyone in here put your hand  up if you're a software engineer most people [11:20] okay is there anyone who's non-technical there's  a couple okay Justin you've got people who are [11:26] non-technical in your organization using coding  assistants to build software, right? Yes. Yes, [11:33] we do. And and software engineers, you're going to  be terrified when I tell you about this, but trust [11:37] me, it's worked out okay so far. We have what we  call our citizen developer program. And what we do [11:41] with that is we enable anyone, so non-technical  or technical folks, the ability to have access [11:47] to our claude code uh account uh to have access  to several skills that we set up which give the [11:54] uh let's call it core setup for the tech stack  that they would use. And we say, "Hey, go build [12:00] whatever you want. You can build internal tools.  You can build uh just for your team. You can [12:05] build for the company as a whole." And here's the  great part. Software engineers in the room. Once [12:10] someone who is non-technical builds an app with AI  assistance, in our case, Claude, they then turn it [12:16] over to my team and we have to manage the support  and the deployments and all the infrastructure [12:21] around it. And all the software engineers in the  room are like, "Oh, right." Like, you understand [12:26] what I'm saying? But it's it's been an interesting  journey because when you get folks that are, [12:33] let's say, vision based, like I I actually have a  lot of our seale executives will will do this type [12:40] of work. It's very interesting to see how they  will talk to a coding assistant versus someone [12:46] that writes software. So the ones that that are  more vision based will just say, you know what, [12:50] I need this thing. Go make it happen. And the  engineers were like, well, I need this thing, [12:55] but it has to work this way, and I want to make  sure that this happens, and I got to have tests. [12:58] And so, it's an interesting dichotomy between the  two with how they use it. Just indulge me for a [13:03] second. I am not a videographer. You might notice  there's a guy standing there with a camera. When [13:08] I make documentaries, Chris from Sound and Color  makes them because he's a videographer and he has [13:13] that passion. I don't have a passion for editing.  It's the same that if I was I think if I was to [13:18] build a piece of software code because I don't  have that passion I would struggle to do it well [13:23] and see what I'm prompting in the way that Nishan  might for example. Can someone like me genuinely [13:28] become an expert and someone trusted? Absolutely.  Because I think you bring up a good point where [13:33] you say hey I don't have the passion. I don't  know if I can do it right. And what I see with [13:38] a lot of people both technical and non-technical  is, you know, depending on your organization, some [13:44] organizations are huge and and change is hard and  so you're scared that if I don't prompt this right [13:49] the first time or do it right the first time,  I'm never going to be able to change it. I mean, [13:53] we could have changed the app. I think we've been  going 20 minutes so far. So, we could have changed [13:57] the app we're building probably six or seven times  in that 20 minutes. And so I would say don't be [14:02] afraid of that because you can make changes as you  go very quickly. Sorry. Um just to I think um how [14:10] they have a program um at Renaissance I think with  FedEx we have something called the AI literacy. So [14:16] as the first generative AI team we spent a lot of  time prototyping products for different teams. Um [14:24] and then because we have the security layer and  the governance layer, we would do that first to [14:29] ensure that okay everything is in line and now  this you know we could get the approvals for [14:35] uh getting that into production. But I think that  was a big overload for us to just do prototypes [14:41] for a lot of um use cases that would not make  into production or for that matter as an AI [14:48] team or as a DS team we would not have access to  certain kind of data. I think with AI literacy [14:55] um the the bridging gap is they're more uh they're  in control of their own data. They have that skill [15:02] set. So upskilling them for off obviously a very  drastically changing industry but at the same [15:08] time this enables those smaller teams to build  their own prototypes. They know exactly that okay [15:17] um it does this it does this but it does not do  that right. they can come and talk to us say okay [15:22] here's what we think is the missing piece of the  puzzle do you guys have something that you can put [15:27] in and fix that and then we can talk about it but  if it's something that oh you know for whatever [15:33] reasons we not able to bridge that gap then it is  easier shelving it um from that you know point of [15:41] interaction instead of us spending four months 5  months 3 weeks whatever time and then say oh you [15:47] know this is not going to happen I think saves a  lot of um time but also upscales those individual [15:54] teams. We we do have a roving mic and I'm kind  of aware that if we leave it till the end for [15:59] any questions. Anything that feels relevant now  might not feel relevant later. Does anyone have [16:03] any questions at this there is a question right  down at the front here. Sorry that's a very long [16:07] way for our roving mic. Um do you want to  ask the question? I'll repeat it. Save. So, [16:12] so the question was as a non as a non- techy  when you're experimenting how do you manage the [16:16] cost of you know how does how does the management  team manage the cost of everyone is experimenting [16:21] especially if they're non techie. So when I'm  doing it at the company or when I would be doing [16:24] it. Okay. So yeah, at the company we basically  have uh an enterprise cla account which is $20 [16:30] a seat and I think you have to do 25 seats at a  minimum or 20 seats at a minimum and that does not [16:36] include usage, right? That's just for the the base  part of it. So we have for folks that are just [16:43] going to use it as you know, hey Claude, what is  XYZ? We usually set a lower budget like 10 to 20 [16:49] a month on that. And we actually when you have the  enterprise account you can control the models and [16:54] then for the developers or the citizen developers  we give them access to some better models and we [16:59] give them a little bit larger budget you know  75 to 100. A lot of the studies that I've seen [17:05] typically say if you are a software engineer  and you're eight hours a day writing code, [17:10] which let's be fair, none of us are doing  that eight hours a day, but if you are, [17:14] because it maths better, they say expect about  300 to 400 a month in token usage. I've typically [17:21] seen that be probably half of that. But one of  the big things is model selection. You know, [17:26] if you went out with Fable and told Fable to build  an app, you know, there goes like 800 bucks right [17:31] there. So, you know, just be mindful of that.  Before we move on, do we have any other Sorry, [17:35] there's another one right down at the front. Um,  I can I can repeat the question for everyone here. [17:39] Where does the project management team land in  that structure? Right. Yeah. So, organizationally, [17:44] they're a peer team to mind, but the way that it  works for the citizen development piece is we have [17:49] a several tiers, you know, like I said, so it's  this is just an app for me, this is just an app [17:54] for my team, this is an app for the company. And  so when it hits that third tier, we then go to [18:00] the product. For us, it's product management.  We say the product management team, we need a [18:05] representative to help us represent this product  even though it's an internal product. Let's move [18:10] on. Danny, let's talk about security. When AI  hands you a suggestion, you know it's safe. [18:17] How much consideration's actually been given to  security? Obviously, from a security practitioner, [18:23] never enough [laughter] to deal with, right?  Uh no, I think this links really well with the [18:27] previous question. Um I think coding senses are  great. Um they also have a lot of risks obviously, [18:35] but from an enterprise perspective, I think we  have a great opportunity to build garels around [18:43] everything, right? So we're having a lot of not in  technical people that are coding. So for example, [18:48] we spend a lot of time figuring out what's the  way that we can make um prompts give you safe [18:55] code pretty much all the time, right? So getting  ahead in all these questions, you know, putting [19:00] guar into how nontechnical people are using it, I  think is very important. And I think this really [19:06] ties into that, right? We don't know that whatever  code we're getting, whatever information we're [19:11] getting is secure or safe, uh mentioned safe. Um  so I think it's very important that we stick to [19:18] the basics right whatever security programs that  you have whatever checks that you have in place [19:23] um trust but verify always um you know we want  to make sure that we're doing more testing or [19:29] you know the same at least amount of testing that  we could do authorize um there's also a different [19:37] um side to it which is um the I don't know  confidential information right or like the [19:44] safety of the data that that's been sent. Um, and  that's also another side of it into these guards [19:49] that that you know as an enterprise you need to uh  you need to keep in mind. Um, how do we secure the [19:55] input? How do we secure the output to make sure  that you know we're making again uh developers [20:00] life as as simple as possible and they just have  to worry about coding. Just just building on [20:05] what you're saying and following on from from the  question that we had at the front, organizations [20:10] are feeling the heat when it comes to cost at  the minute and especially where shadow AI is [20:13] concerned that is possibly pushing developers and  nontechnical staff towards um bad behaviors going [20:20] outside of governance working around frameworks.  How do you try and build a culture where you have [20:25] champions who act in the right way or or make  sure that that is part of the culture of the [20:30] organization? Yeah, that's a great question. So um  at first we thought that learning was going to do [20:36] it. So set up learning paths training uh you know  give this information to all developers was going [20:42] to work. Uh we got it completely wrong. Uh you  know you cannot just try to pretend that you're [20:48] going to give this information to all developers  and they will be able to like everybody's going to [20:52] review that information. So again I think that's  where those um safety guils or like all you know [20:59] like how do you build that program around it? um  I don't know do you have a gateway do you have a [21:03] proxy in which you can put these controls um will  really play a role right um again we we need to [21:09] try to pretend that everything is insecure  you can tell that I'm a pessimistic in this [21:14] perspective right um but you know trust verify  and make sure that everything that you're doing [21:21] um you put kind of like those those filters  on top of it to make sure that you know yeah [21:25] I agree with you I think you have to treat the  output of coding systems as you know Someone [21:30] some an untrusted external whatever someone gave  you that code and you have to at least the thing [21:36] the workflow which has given me trust is you  have to always verify it in a sandbox execution [21:43] environment and the security like yourself right  you define all the policies you have a zero [21:48] trust execution model and unless all those policy  checks are passed you should not commit that code [21:53] to production and it should not go to you know it  should not be executed. So executing the code from [22:00] HTT systems and in execution sandbox environment  and the policies from security system you know [22:06] being all green until that happens you cannot  trust the code I think so it needs to be part [22:11] of your engineing workflow we ke just a bit to  add on that is um I think with FedEx security is [22:22] uh one of our foremost concerns so although we  do use coding assistance uh or AI agents but they [22:29] still sit somewhere between that systematic  governance uh layer. So as the software [22:35] development life cycle does not change. Um there  again the testing reviews all of that happened. [22:41] Um and then coming back to your point David how do  we cut the cost? I think uh one thing that we as [22:48] a team have been very consciously trying to do is  whatever we build that's reusable and repeatable. [22:55] uh because until unless your organizational  use case or data changes significantly [23:02] uh a lot of teams do come to us with uh a lot  of conversational agent requirements. So of [23:08] course they differ but I think if there's there's  existent framework that you know is tried tested [23:15] um is within the security protocols and  everything I think building on top of that [23:20] uh or reutilizing it into some capacity I think  that also reduces the cost that we were talking [23:25] about. Um so yeah I think that's one thing that  we've been trying to do. Just stick with you for [23:32] a minute. I'm going to pick up on something that  Nishan said because you mentioned the word got to [23:35] verify the work that's coming out of agents. We we  talk about productivity gains a lot where you're [23:42] utilizing coding assistants and agents. Is it  actually giving you more free time or do you find [23:48] yourself managing ever greater number of agents?  Um I think how you utilize time has changed. It [23:56] is not I wouldn't say it's giving me free time.  uh but I think instead so my cost of exploration [24:02] is what I call it has reduced so I'm spending less  time uh you know reviewing existing documentation [24:09] or code a coding agent does that best for me I  think I would put that time into when I what I'm [24:16] building on top of it so I would okay why did you  do this you know if you chose this approach why [24:21] this and then uh a personal approach that I use I  do not go by one coding agent so I usually do do [24:28] a parallel run. So if I get something from let's  say codeex, I would put it through cursor. If I'm [24:34] using cursor and then the GitHub copilot, then I  would just cross check between the two. Another [24:40] point that I think uh with time is sometimes when  um coding assistants you know give you uh a file [24:47] structure or a solution there are two ways to  it. One it is very optimized very structured. [24:54] Sometimes you have to cross check because a lot of  times I feel that they hallucinate into giving you [25:00] a way advanced structure or opinion than that was  needed for a prototype application. Right? So now [25:07] you're basically you know using your time and  brain into something else rather than reading [25:13] the documentation. How long does it take to build  that instinct to know when to look in more detail [25:19] because you must be dealing with an ever greater  number of of tasks, right? Um so this is I if you [25:27] if you skim read through it I think once your  application's generated I usually personally [25:31] spend maybe five 10 maximum 15 minutes 15 minutes  is the maximum you'll know that oh that detailed [25:39] file structure that kind of a repository I don't  think I need it for a prototype application. So [25:44] you're like you know dude you've gone too far  come back. [laughter] So I think that's but [25:49] if it's a very complex application then I would  probably spend a little more time understanding [25:54] the architecture. Just I can see you nodding along  and kind of do you recognize us in your own or I [25:59] mean if we were at church I would say amen after  that. Like I I fully agree. So I I think look when [26:05] when you get in there to the to the what's the  first part of the question you answered I fully [26:10] agreed with it and now I can't forget what you  said. you remembered what you said 30 seconds ago, [26:14] right? Well, I fully agreed with it, so it's  okay. [laughter] But but no, when when it comes [26:20] to productivity gain, like we've definitely seen  it. Uh, you know, I said I made the joke the other [26:25] day, shocker, that uh I've seen a 25% increase  in my engineering team. And then the joke, [26:32] which was amazing, was what is 25% of a person  like? Is it knee down? Is it shoulders up? It [26:37] was cost of exploration. Oh, yeah. I love that.  But the joke, was it funny? No. Got it. Okay, [26:43] moving on. So, I have seen that come in my  engineering team, but what I'm seeing is the [26:48] surrounding teams, they are not coming up to speed  as quickly as the engineers are. And so, you know, [26:55] the product side still operates like the product  side still did before uh AI. This, you know, [27:01] other team over here still does. This other team  over here still does. So, we've we've now expanded [27:05] our work out to those additional teams to help  drive the productivity even more. We have any [27:11] questions at this point in the room? Just out of  interest. It's got a couple over here. Sorry. Can [27:15] we get a microphone over this side? Sorry, I  appreciate it's a huge room. [laughter] Yeah, [27:21] I have a question in regards to a computer  science program at the college I teach at. [27:27] I'm just wondering with Rise of Coding assistants,  what you would recommend as far as what students [27:35] should learn and what they should know and what  they don't have to know in regards to all this. [27:40] So I can take that one two ways. One, it it hits  close to home. I have a uh college junior and he [27:46] is actually or excuse me, sophomore, but he is  actually taking a degree at UNC Charlotte that [27:53] is called AI engineering and it's the uh in Sep or  August. It's the first time they're ever offering [27:59] that degree and I think it's it's pretty powerful  and I would recommend that. But in a traditional [28:05] computer science world, when you get into the AI  space, just all the Python you can muster, right? [28:12] Um I I personally think that junior engineers  coming out of college now need a bigger focus [28:18] on system architecture and system design because  they don't need to write code anymore but they [28:25] need to understand how is it how the thing that  this AI coding assistant wrote how does it work [28:31] how does it interconnect to other systems you  know does it need to do indexes on a database [28:36] table does it need to have a uh you know  multi-server environment like how what is [28:41] architecture outside of code and I think that's  going to be a much bigger focus because code is [28:48] cheap and code is easy now we're we're paying  pennies for that but I will pay thousands for [28:53] the mind that can build the system amazing sorry  um just to add a little bit to what Justin said [29:00] I think uh with coding assistance and then AI  literacy programs and everything but I think [29:05] um when I was graduating out of school my final  year chart GPT had already been launched. So I was [29:12] a research assistant uh and then I was helping  my professor just review papers and stuff like [29:17] that and I could actually see that undergrads were  very heavily and charg back in 2023 was not uh as [29:26] powerful at is as it is today but still there was  a very heavy usage but I think in today's world [29:32] where code will be generated you know coding is no  more the problem but I think just to reiterate to [29:40] Justin said it'll only help if you understand  it. Um, and then we do things here now. I think [29:48] uh when I'm looking so I get the code right. I  mean I still prefer coding by myself but for me to [29:54] understand what a coding assistant is generating  is I know Python, I know TensorFlow, I know SQL. [30:01] Uh I just cannot go and execute and just put it in  front of a board member and say hey this is what [30:05] it is. It's not supposed to be like that. But I  think for you to understand so if you know your [30:10] documentation if you know what retrieval are you  looking for uh and then with coding there's a lot [30:17] of reiteration required so if you're just trusting  a coding assistant you will never reiterate if you [30:25] never reiterate you never improve so if you never  improve I don't think you have the best product so [30:31] I think that is not the goal I think that was  a fantastic question let me just very quickly [30:37] because if we can bring the microphone to the  front, there's a there's a gentleman in the white [30:41] shirt. And whilst the microphone comes over, do  you feel any anxiety yourselves to keep up with [30:48] the latest trends? Because Danny, I know on our  prep call, you describe yourself as an amateur [30:52] vibe coder, right? Leaders themselves must be  under a fair amount of pressure given how quickly [30:57] this is moving. Yeah, I think I mean at this point  I feel like you don't have a choice. you have to [31:04] um you know the way that we've seen in the  enterprise you know everybody it it it helps [31:09] understands the problems better as well when you  start breaking it up and like putting it together. [31:14] So I think like from our perspective right uh  from a cyber it's helped us tremendously build [31:19] our own internal products um it's helped us uh  you know make everything easier faster in a more [31:27] reliable way. Um so you know again you have to  have those garl and you have to have that standard [31:35] which I completely agree that you will not be  able you will be much faster successful if you [31:40] have that background right if you have your high  level design right if you have uh if if you have [31:48] that understanding of the code so I think I'm a  computer science major so that kind of like helped [31:53] build myself into that I shifted to to cyber but I  definitely think that having that background as to [32:01] you know what is Python or what is Java or what is  whatever really helped uh build now whatever I'm [32:07] building with with u sorry you had a question yeah  so I think this might be directed for RT um so I'm [32:16] a product manager and I can't code but I love  using these tools to build software and you had [32:22] mentioned that if you can't read the code well um  it'll never be accurate and I think as a product [32:27] manager were trained to identify problems and  understand the end value that the user is looking [32:33] for and to hopefully create some like enterprise  value. And so I've been trying to test to ensure [32:39] that my end goal is sufficient that may be things  operating within a period of time or maybe solve [32:46] the end user's problem. And so my bet is that  these models will be good enough to the point [32:51] where that reviewing process I don't have to worry  about. just compare from codeex to cloud code etc. [32:58] So I I kind of wanted to just push back on that  and just get your additional thoughts on if it's [33:04] really you you're just out of luck or if there are  some intricacies to that. I think one takeaway I [33:11] had from a key from the keynote this morning is  uh humans are not here to be replaced when when we [33:17] talk about AI and I think I got some I don't know  but yeah just to come back to your point I think [33:26] um as a product manager I'm assuming when you're  talking about enterprise value you know what the [33:30] business problem is you know the enterprise what  value does it want from it you know your consumers [33:37] um so my suggestion would uh and I think I've  seen a lot of people do that and I think all [33:43] of us although we are in tech space it's  not like we've stopped upskilling ourselves [33:48] uh that could be for the work we do but even  outside of our work just to uh keep up with [33:54] the changing environment I think so if you can  just learn and when I say you will not be able to [34:01] uh you know review the code but as a data  scientist if I'm building a coding uh I'm using [34:06] a coding agent to build an application but if I do  not know you know it needs a reiteration or what [34:12] are the you know database relations and everything  then I'm not doing my job well right um so as a [34:19] product manager I think if you have that kind of  instance that you want to develop but I would say [34:24] start learning you know but before you build an  application u and how Daniel said you know what [34:30] what we learned in computer science python SQL  that's just helped us b um build the base I think [34:36] once you know the base and it's not like any of us  I think these people are way more experienced than [34:40] I am. Uh and if you ask them I don't think any of  us learned to debug the code the first day we saw [34:47] it. It took us a whole lot of time but if you pick  that as that uh first piece as the first step I [34:54] think that that's a very good start to even think  about. I don't know if I answered your question. [34:59] I have a little bit different opinion and take  on it. I think my personal suggestion is even [35:05] though if you don't know fundamentals right as  a PM you still know your customers you know your [35:09] stakeholders you know your engineers and uh I  think most important job of PM is to connect [35:14] engineering and the customers right to help build  the product so that of course customers use it [35:18] and whatever KPI you're trying to increase it  improves so you can use AI without understanding [35:23] the fundamentals to create a proof of concept to  demonstrate your idea whatever idea you have in [35:28] your mind right instead of just writing the PRD  product requirement docu ment right at least this [35:32] has been what all the PMs and meta have been doing  right which has been pretty successful right you [35:37] have your you know your customers you know all the  stakeholders you are the engine person connecting [35:44] everything together so you create a proof of  concept right you don't need anyone else and using [35:47] the proof of concept then engineers of course  can use the fundamentals to build the actual [35:51] production ready whatever system or platform so  this is at least my personal suggestion we've [35:55] got a couple of minutes left so I'm just going to  ask one more question and uh maybe one or two of [36:00] you will jump in but Justin I'll go to you with  this where this conversation is concerned. There [36:05] are probably quite a lot of obvious um themes to  talk around. What are the conversations that get [36:12] buried the people in this room should walk away  talking about more frequently? Oh wow, that's a [36:18] great question. Um let's see. Cost is a good one.  We've already brought up cost. I think that's [36:25] that's an interesting one. I I will go back to a  previous answer I gave where where it was about [36:30] the processes that are outside of engineering and  so that can be QA that can be product you know as [36:36] as we were answering this question I'm sitting  here thinking okay well there's a process there [36:41] that what we could say is let's connect Jira  to Claude and tell Claude to go do the testing [36:47] from a functional perspective and make sure that  it matches the requirements right that convers [36:52] that type of conversation doesn't happen at every  organization and so I think What everyone needs to [36:57] do regardless of what role you're in is you need  to like stress test your peers and say especially [37:04] the software engineers and say I'm using this  but how are you using this? I know. And then [37:09] last thing I'll say here is is our CEO when we're  in staff meetings will look at all of us and say, [37:17] "How have you used AI this week?" And like if  you can't answer, that's a bad answer. Like you [37:22] have to have an answer. And I would just suggest  folks to start asking that question not only to [37:28] yourselves but to your peers. And just to a little  bit add on to it, I think the analogy I even give [37:34] to my brother and you know any engineer  I'm mentoring is to try to have as minimal [37:40] dependencies as possible in terms of humans right  of course you know there are domain experts right [37:45] but there are various tools available for design  whatever right Figma UI UX right various tools [37:50] available for data analysis right various tools  available for of course UI creation right or for [37:55] backend services right so try to understand the  problem very very deeply Okay, you try to break it [38:00] down. Okay, this is the UI component. This is the  whatever use UX wireframe you need. This is the [38:06] data analytics aspect you need. These are the data  dashboards you need for leadership or customers, [38:10] right? And you try to build everything into it as  a proof of concept. If you can do that and you can [38:14] iterate really fast, you can reach a production  ready system and loop in the domain experts really [38:20] quickly because I think the most important  thing is to communicate very effectively, [38:24] not just through words or through documents, but  through actual showing something that's working. [38:29] after we've got 30 seconds left. A final  thought. Yeah, just one thing uh and I [38:34] agree with what both of what uh they said. One  thing is I think we've uh focused a lot on what [38:39] AI can do but we I think I haven't heard a lot  of the conversation about how it is doing it. [38:44] So for me evaluation in different frameworks  across different organ I mean across different [38:50] domains is I think that is one piece that uh I  haven't heard a lot about. So I would focus on [38:56] evaluating those responses and in any domain and  space where AI is being utilized. Amazing. Look, [39:02] please show your appreciation for our panel  today. Arty, Justin, Danny, and Nishan. And [39:07] thank you for coming along and joining us. Enjoy  the rest of the conference. Thank you, David.