eBay, Audible Deploy AI Coding Agents to Accelerate Software Delivery

eBay与Audible部署AI编程智能体加速软件交付 企业工程正从手动编写语法转向AI智能体编排；深厚的领域专长能带来巨大的生产力提升，而初级人员不加批判的依赖则会导致代码缺陷成倍增加。 本场圆桌探讨了从手动、确定性编程向智能体驱动的软件开发及非确定性上下文工程的转变。工程负责人分享了利用Claude Code、Cline和Cursor等工具加速企业框架迁移、自动化缺陷分流以及自愈测试自动化的实际落地案例。对话强调，尽管AI显著成倍提升了资深开发人员的研发效率，但关键任务系统仍严格依赖确定性架构、稳健的评估和人工护栏。最后，与会嘉宾还讨论了工程专业方向向产品工程的演化，以及向初级开发者牢固灌输核心调试基础的必要性。 生成式 AI 通过在编码前检测导致约 37% 下游缺陷的含糊需求，推动测试真正“左移”，并能跨 Playwright、Cypress 和 Selenium 等框架自动生成完整测试套件。 (2:33) 在研发全流程引入 Agent 工作流带来了可量化的效能提升，例如 eBay 的 PR 评审合并耗时缩短了 17%，JDK 8 升级至 21 等老旧运行时迁移提速达 25% 至 30%。 (5:53) 针对解析任务的自动化生产管线借助多模型多数投票机制来防范幻觉，并在目标网站布局变动时触发自愈程序重新生成解析指令，以保障系统稳定性。 (11:39) 软件工程正从确定性的语法编码转向基于记忆层与 MCP 协议的上下文工程，这种转变极大放大了资深工程师的产能，但也可能让缺乏底层调试能力的初级开发者引入更多缺陷。 (19:38)

Engineering leaders from eBay, Audible, Qualitest, and Oxylabs report that generative AI is transforming software engineering from manual syntax authoring into agent orchestration, automated modernization, and requirements validation. However, realizing sustained productivity gains requires deep domain expertise, strict deterministic architecture, and rigorous evaluation frameworks to prevent code regressions and security risks. Accelerated Testing, Code Migration, and Triage (2:33) AI tools are significantly reducing the operational overhead of software testing, maintenance, and defect triage. Vikul Gupta from Qualitest notes that roughly 37% of late-stage software defects originate from defective or ambiguous user stories; deploying large language models to validate requirements early enables shift-left defect elimination and automated test script generation across frameworks like Playwright, Cypress, and Selenium. For routine modernization, Senthil Padmanabhan reports that eBay achieved a 25% speedup migrating enterprise software from JDK 8 to JDK 21, and a 30% speedup for Spring Boot migrations using an internal agent framework. Across daily development, eBay measured a 17% reduction in pull request merge times and a 12% reduction in commit-to-production lead times. Production incident management has seen comparable acceleration. By feeding Claude Code repository context and historical incident workflows, eBay reduced defect triage from a two-to-three-day manual investigation down to one or two hours, automatically filtering out non-code configuration issues. Similarly, Ian Moraes highlights that Audible leveraged Amazon Q Developer alongside Cline CLI tools to execute migrations across thousands of JUnit 4 and 5 test cases and legacy Java runtimes, systematically removing routine engineering toil. Context Engineering and Multi-Agent Orchestration (17:59) Software development is undergoing a paradigm shift from deterministic, syntax-driven programming to non-deterministic context engineering. Developers increasingly steer models by managing short- and long-term memory, system prompts, and tool protocols like the Model Context Protocol (MCP) rather than authoring raw syntax in isolation. Senthil Padmanabhan describes an emerging lifecycle where specialized autonomous agents handle distinct phases of software delivery: architect agents draft plans from user stories, implementer agents generate code, tester agents verify functionality, and deployment agents manage CI/CD pipelines. Within this architecture, the human software engineer transitions from a manual coder into a manager and orchestrator responsible for providing precise context, steering system behavior, and handling edge cases. Ian Moraes argues this evolution elevates user empathy, product design, and creative problem-solving, freeing developers from documentation and routine dependency upkeep while transforming traditional roles into product engineering. Senior Leverage Versus Junior Debugging Pitfalls (19:38) While generative assistants offer substantial operational leverage, panel findings demonstrate that value creation remains heavily dependent on foundational engineering competency. In an internal AI hackathon at eBay where tooling was equalized across participants, the winners were domain experts and senior engineers whose deep systems knowledge and concise communication allowed them to steer models effectively. Conversely, in teams with recent college graduates lacking mature debugging foundations, junior developers accepted AI-generated suggestions blindly, resulting in four new bugs appearing for every single defect closed. Democratization also creates architectural tension through citizen developers. Although non-technical stakeholders can rapidly produce functional prototypes without waiting on core engineering teams, they often overestimate the production-readiness of raw LLM outputs. Engineering teams encounter operational friction explaining why proof-of-concept scripts cannot be deployed to production without comprehensive architectural redesigns for security, maintainability, and scalability. Deterministic Architectures, Rigorous Evals, and Security (35:21) Because AI model outputs are probabilistic, enterprise software architecture is separating into two distinct tiers: a non-deterministic AI layer and a deterministic control layer. Panelists emphasize that large language models cannot reliably enforce mission-critical boundaries such as authentication, secret storage, role-based access, and rollbacks, which must remain governed by traditional rule-based code. To prevent hallucinations and performance drift, Qualitest benchmarks relevance, toxicity, ROUGE/BLEU scores, and model bias, driving the emergence of new specializations like Data Science Engineers in Test (DSIT). Similarly, Andrius Kūkšta notes that Oxylabs mitigates single-model hallucination through multi-model consensus voting and self-healing parsing routines. Enterprise security and latency constraints further dictate architectural deployment strategies. For latency-critical workflows like eBay's live auction bidding, or strictly regulated, air-gapped environments in banking and healthcare, organizations cannot route sensitive operational telemetry through external hosted APIs. Instead, teams deploy open-source models locally using runtimes like Ollama, preventing proprietary data leakage while eliminating internet network latency. While AI tooling dramatically accelerates code generation, testing, and legacy migration, the panel underscores that foundational computer science skills remain irreplaceable. Long-term enterprise success hinges on mentoring junior engineers in manual debugging, establishing rigorous evaluation frameworks for probabilistic code, and preserving deterministic software guardrails around security and access controls.

来自Audible、eBay、Qualitest和Oxylabs的技术管理者指出，生成式AI正在将软件工程的核心范式从确定性的语法编写转变为非确定性的上下文工程与多智能体编排。工程效率的根本壁垒不再是样板代码的产出速度，而取决于开发者的底层系统理解、上下文引导能力与确定性架构防线。 左移测试与研发流程重构 (2:33) Qualitest首席技术官Vikul Gupta指出，在测试周期后期发现的缺陷中，约有37%可直接追溯至存在缺陷、模糊、重复或缺乏明确验收标准的用户故事。企业通过利用大语言模型在需求阶段预先检验文本质量，实现了真正的开发“左移”，能在编写代码前拦截下游缺陷。一旦需求验证完成，生成式AI工具可直接生成涵盖Selenium、Playwright和Cypress的测试场景与自动化脚本，而预测性AI模型则评估测试套件的失败风险，优先在CI/CD流水线中运行高风险用例以保护发布节点。 在代码维护与运维层面，eBay平台与基础设施工程副总裁Senthil Padmanabhan透露，公司通过内部A/B测试证实AI工具使PR代码审查合并时间缩短17%，代码提交到生产部署的交付时间缩短12%。针对常规技术债与遗留系统升级（曾占据10%至15%的工程产能），eBay自主研发了由Claude与GPT模型驱动的编排框架eBay Coder，使得JDK 8升级至JDK 21的速度加快25%，Spring Boot 2.7升级至3的速度加快30%。对于线上缺陷排查，集成历史事故上下文的Claude Code将原本需2至3天的故障调查时间压缩至1到2小时内，可自动将非代码配置问题重新分派给运维人员或直接为开发者生成待合并PR。 信任机制与确定性控制防线 (11:39) 针对开发者对代码幻觉与质量下降的疑虑，Oxylabs研发工程师Andrius Kūkšta介绍了其在OxyCopilot产品中采取的多数投票共识机制。针对网页数据解析中DOM微调即可导致提取失败的痛点，系统将解析任务广播给多个异构模型，通过挑选频次最高的输出降低单模型幻觉风险，并在生产环境成功率下跌时自动以失败页面触发自愈流程，重新生成并验证解析规则。在企业级评测方面，Vikul Gupta强调必须建立双重机制：业务专家保留最终人工审查与覆盖权限并以此对齐模型；技术层面上持续追踪幻觉率、毒性、ROUGE/BLEU评分，并通过动态路由算法将工作负载分派给匹配的底层架构。 未来软件系统的稳固性依赖于双层架构的彻底分离：一层是由AI模型构建的非确定性层，另一层是由传统面向对象编程构建的确定性控制层。由于语言模型在机密隔离上表现薄弱，系统必须依靠传统的确定性规则来兜底基于角色的访问控制、版本回滚、凭证保管与数据隔离，避免跨租户数据泄漏。高标准的自动化评估（evals）与确定性边界构成了防范法律诉讼与运行风险的必要屏障。 技能分化与核心专业知识杠杆 (19:38) 开发者的核心日常正由手动编写语法行转向引导由架构智能体、实现智能体、测试智能体及自主部署智能体构成的Agent生命周期。虽然AI能快速完成约90%的样板实现，但剩下的10%关键问题对高级领域经验提出了极高要求。在eBay举行的全员AI黑客松中，最终获胜的并非善于实验提示词的新手，而是具备深厚技术底蕴与清晰表达能力的资深架构师；相反，在某应届毕业生占比10%的工程团队中，初级工程师因缺乏底层排错能力而盲目合并AI生成的补丁，导致每关闭一个缺陷就产生四个新缺陷。 Audible技术副总裁Ian Moraes与Andrius Kūkšta强调，软件工程师的角色正从基层编码演变为产品工程师，需将精力投入至用户同理心与系统架构设计中。但在新人培养上，技术主管一致主张必须先扎实掌握数据结构、算法、手工断点调试及日志分析等计算机底层基础，再利用AI加速探索。企业数据显示，实习生与应届生使用AI助手能极快熟悉大型遗留代码库并编写补充测试用例，但若跳过底层训练直接依赖AI生成代码，将导致团队丧失长期自愈与创新能力。 企业治理摩擦与本地部署挑战 (24:09) 低代码与生成式AI的普及使产品经理等非技术人员能够独立构建原型，但这也引发了系统架构上的摩擦。业务人员容易高估裸模型输出的生产成熟度，误将孤立的原型脚本等同于可投入生产的系统，由此迫使工程团队在系统可扩展性、安全合规与可维护性上投入大量二次重构成本，给核心工程组织带来维护负担。 在企业规模化落地中，Audible通过高管赞助跨职能培训与认证报销、举办全员科学博览会以及开展每周点对点技术分享来消除基层对被取代的焦虑。在技术选型上，团队采取随模型基准迭代的自适应机制，避免一刀切的硬性采购。针对医疗、金融等存在严格物理隔离（Air-Gapped）的网络环境，或类似eBay实时竞价这类对亚毫秒级网络延迟极度敏感的场景，企业无法直接调用公有云托管API，必须通过Ollama等运行时部署开源本地模型，以确保自研知识产权不外泄并满足超低延迟运行要求。 生成式AI在消除软件工程常规技术债与缩短测试周期方面展现出明确的量化效益，但工具的普及使代码编写门槛归零，软件成败的重心完全转向了严谨的确定性安全防线、非确定性评估体系以及工程师自身的系统架构鉴别力。如何在加速业务原型的同时避免初级开发者调试能力的退化和不可维护代码的膨胀，仍是工程组织面临的关键课题。

## Background 该内容由 YouTube 频道 Ai4 于 2025 年 8 月 29 日发布，记录了一场题为《AI-Powered Development: The Future of Software Engineering》的圆桌研讨会。来自 Qualitest、eBay、Audible、Devon Energy 和 Oxylabs 的多位技术高管与专家围绕 AI 如何重塑软件工程全流程展开对话，探讨了从自动化代码生成、测试左移、DevOps 工作流优化到代码质量管控与人机协同监督等核心议题。 ## TL;DR 圆桌讨论深入剖析了 AI 在现代软件工程生命周期中的前沿落地实践。嘉宾们展示了 Qualitest 将测试套件构建从数月压缩至数分钟的测试左移方案，以及 eBay 和 Audible 借助 Claude Code、Cline 和 Amazon Q 等工具实现 JDK 升级与线上 Bug 极速分流的真实案例。针对代码可靠性难题，研讨会探讨了 Oxylabs 的多模型多数投票自愈机制及企业级算法集成评测体系。最后，与会领袖展望了软件工程师从繁琐编码向“Agent 编排者”与“上下文工程师”的范式转变，并就初级开发者过度依赖 AI 的隐患、运维日志智能排障及物理隔离内网部署给出了务实应对策略。 ## Atmosphere & style 整场研讨会气氛务实且极富现场感染力，主持人开场以“走廊尽头角落会场”和“午餐前最后一关”的幽默自嘲迅速调动起观众热情，互动调研更是频频引发台下工程师的真实共鸣。对话风格硬核落地，嘉宾们毫不避讳技术光环背后的真实挑战——如初级程序员修复一个 Bug 却衍生四个新 Bug 的窘境、模型幻觉带来的治理难题等，既展现出大厂工程实践的严谨深度，又洋溢着同行在技术巨变前夕惺惺相惜的坦诚与求索精神。 ## Key takeaways - [02:33](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=153s) AI 正在推动测试流程全面左移（Shift-Left）：Qualitest 通过 AI 清洗需求缺陷，从源头消除 37% 隐患，并将原本耗时数月的测试套件构建缩短至 2.8 分钟。 - [05:53](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=353s) eBay 建立了涵盖 IDE、终端/云端及探索三层的 AI 智能体体系，实测使 PR 合入时间缩短 17%、变更前置时间缩短 12%，JDK 及 Spring Boot 大版本升级提速 25%-30%。 - [05:53](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=353s) 借助终端智能体（如 Claude Code 搭配 CLAUDE.md）进行线上 Bug 自动分流与排查，eBay 将线上故障排障周期从 2-3 天大幅压缩至 1-2 小时。 - [10:08](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=608s) 大模型在消除工程杂务（如自动生成单元测试）和遗留系统现代化中成效显著，可对无文档的老旧代码进行逆向工程以提取原始业务需求。 - [11:40](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=700s) 为确保 AI 输出可靠性，Oxylabs 采用多数投票机制与指标下滑自动触发的自愈流程，Qualitest 则通过专家人工覆盖反馈与算法集成动态路由来筑牢质量防线。 - [16:05](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=965s) 软件工程范式正从确定性的语法编写转向非确定性的“上下文工程”（Context Engineering），涵盖记忆管理、MCP 协议集成与提示词设计。 - [19:37](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1177s) 未来工程师正转型为“智能体管理者与编排者”，核心竞争力在于对底层运作机制的深度理解以及精准的业务意图表达能力。 - [21:12](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1272s) 内部黑客松表明 AI 收益高度偏向资深工程师与领域专家；初级工程师因过度依赖 AI 且缺乏基础调试能力，容易陷入“修一个 Bug 产生四个新 Bug”的困境。 - [29:29](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1769s) 大模型极大降低了原型验证的边际试错成本，但机械使用模型会导致产出回归均值，人类的创造力、精细上下文与业务共情（Human Variance）成为最终壁垒。 - [33:09](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1989s) 随着 AI 引入非确定性，测试领域正诞生如 DSIET（数据科学测试开发工程师）等新岗位，专注于偏见测试、幻觉评估及非确定性软件的质量保障。 - [42:22](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2542s) 针对生产支持与运维排错痛点，AI 正在被用于关联日志、指标与部署记录，自动化识别工单背后的系统性瓶颈并动态反哺监控告警规则。 - [44:00](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2640s) 在强监管物理隔离网络（Air-Gapped）及在线拍卖等毫秒级极致低延迟场景下，基于 Ollama 和开源生态的本地私有化模型部署成为不可或缺的刚需方案。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=4s) 圆桌开场与嘉宾背景介绍 主持人 Bryce Herbranson 以轻松幽默的方式为圆桌讨论拉开序幕。他打趣道，主办方把他们的会场安排在走廊最尽头的角落，而且这场讨论还是大家去吃午餐前的“最后一道关卡”，非常感谢现场观众的热情参与。Bryce 自身担任 Devon Energy 新兴技术团队的首席 AI 软件开发专家，并依次邀请在座的四位行业技术领袖进行自我介绍： * **Ian Moraes（Audible 消费者领域技术副总裁兼负责人）**：负责 Audible 的面向消费者核心功能，包括用户订阅注册与内容购买链路。Audible 作为行业领先的有声叙事内容平台，持续为听众打造前沿技术与原创音频体验。Ian 笑着分享了一个插曲：这几天遇到不少听众问他最近在听什么，他正在听《列奥纳多·达·芬奇》的传记，并感叹达·芬奇绝非只是画了《蒙娜丽莎》，而是一个极其博学多才的奇才。 * **Vikul Gupta（Qualitest 首席技术官）**：介绍 Qualitest 的核心业务聚焦于两大方向——一是“利用 AI 实现更智能、更安全的工程实践”，涵盖开发、测试、运维全流程智能化；二是“AI 本身的工程化”，覆盖从模型数据采集、构建、测试、部署、流水线集成到模型漂移监控的全生命周期。 * **Senthil Padmanabhan（eBay 平台与基础设施工程副总裁）**：主管 eBay 的底层平台与基础设施架构，同时兼管全公司的“研发效能与交付速度”（Engineering Velocity），这也是他探索并推广 AI 辅助开发的核心动力。 * **Andrius Kūkšta（Oxylabs 研发工程师）**：来自立陶宛的数据采集解决方案提供商 Oxylabs，所在团队专注于研发、验证并落地前沿创新功能，为客户提供更高效的数据抓取支持。 --- ### [02:33](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=153s) Qualitest：以 AI 赋能“测试左移”与需求质量把控 在正式提问前，Bryce 现场调查了观众中的软件工程师比例，发现开发者人数远超预期，反映出 AI 在工程一线的火爆程度。Bryce 随即抛出首个核心议题：各位嘉宾的团队如何在软件研发平台与流程中深度集成 AI？究竟解决了什么痛点？带来了哪些工作流上的实质改变？ ```mermaid flowchart TD accTitle: Qualitest AI 驱动的智能测试左移流程图 subgraph S1["需求分析与左移验证"] A["用户故事 / 业务需求"] --> B["AI 需求分析引擎"] B --> C{"检测缺陷与歧义"} C -->|消除 37% 需求隐患| D["高质量结构化需求"] end subgraph S2["自动化生成资产"] D --> E["AI 自动化测试生成器"] E --> F["测试场景 / Feature 文件"] E --> G["测试脚本: Selenium / Playwright / Cypress"] end subgraph S3["风险预测与执行优化"] F & G --> H["AI 失败风险预测模型"] H --> I["高风险测试用例优先执行"] end ``` Vikul Gupta 首先从 Qualitest 深耕的智能测试领域切入。传统的自动化测试、性能测试、安全测试与无障碍测试，通常需要资深领域专家（SME）花费大量时间梳理需求、编写测试场景、撰写 Feature 文件并编写自动化脚本。过去 27 年里，Qualitest 积累了庞大的 SAP、Salesforce（SFDC）、银行及医疗理赔等复杂业务的测试用例库，但整个过程依然极度依赖人力，耗时动辄数月。 大语言模型与生成式 AI 的出现彻底颠覆了交付效率。Vikul 指出，过去需要几个月才能完成的测试套件构建，现在仅需 **2.8 分钟** 即可生成。更为关键的是，研发后期发现的缺陷中有 **37%** 根源在于需求本身的质量缺陷（如定义模糊、重复、缺乏结构或缺失验收标准）。Qualitest 利用 AI 实现真正的“测试左移”（Shift-Left），直接在需求阶段进行 AI 校验与清洗，从源头消灭这 37% 的隐患。 - **一键资产生成**：在清洗后的需求基础上，AI 能一键自动生成测试用例、BDD Feature 文件以及基于 Selenium、Playwright、Cypress 的自动化测试脚本。 - **失败预测与用例优先级调度**：利用 AI 预判哪些测试用例更有可能失败并识别故障风险，从而在测试流水线中优先执行高风险用例，确保工程师尽早修复并按期完成交付。 --- ### [05:53](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=353s) eBay 的三层 AI 智能体体系与效能飞跃 Senthil Padmanabhan 详细拆解了 eBay 在软件研发体系中部署 AI 的三大层次分类：**IDE 智能体（IDE Agents）**、**终端与云端智能体（Terminal & Cloud Agents）** 以及 **探索智能体（Discovery Agents）**。 ```mermaid flowchart LR accTitle: eBay AI 研发工具分类与应用架构 subgraph IDE["1. IDE 智能体"] A1["GitHub Copilot"] A2["Cline<br/>(Plan & Act 模式)"] A3["Cursor"] end subgraph TC["2. 终端与云端智能体"] B1["eBay Coder<br/>(Claude + GPT 编排)"] -->|框架/语言升级| B2["JDK 8 至 JDK 21 / Spring Boot 3"] B3["Claude Code + CLAUDE.md"] -->|线上 Bug 分流| B4["生成修复 PR 或转派运维"] end subgraph DA["3. 探索智能体"] C1["代码检索与团队依赖协同"] end ``` 在日常编码的 IDE 层面，eBay 两年前开始全面推广 GitHub Copilot，如今已全员覆盖。近期开源插件 Cline 因其具备“规划与执行”（Plan and Act）的 Agentic 工作流，在工程师群体中引发了极高人气，Cursor 的使用率也在迅速爬升。eBay 内部持续开展 A/B 测试评估工具收益，实测数据显示： - **代码审查合入时间（PR Merge Time）** 缩短了 **17%**； - **变更前置时间（Lead Time to Change，从代码提交到生产上线）** 缩短了 **12%**。 在应对技术债务（Tech Debt）方面，eBay 每年投入约 10% 至 15% 的工程算力用于底层系统现代化的“基础工程”（Essential Engineering）。为了让工程师从枯燥繁重的版本升级中解脱出来，eBay 搭建了基于 Claude 和 GPT 模型编排的终端与云端 Agent 平台 **eBay Coder**。其升级提速效果极为显著： - 整个老旧应用套件从 **JDK 8 升级到 JDK 21**，耗时缩短了 **25%**； - **Spring Boot 2.7 升级到 3.0**，耗时缩短了 **30%**。 针对线上缺陷排查（Production Bug Triage），eBay 引入了终端智能体 Claude Code。工程师耗时数周编写了详尽的 `CLAUDE.md` 上下文规则，喂入历史变更与排障会话数据。很多反复出现的线上问题其实只有 50% 真正涉及代码修改，其余大多属于配置变更或缓存刷新。现在线上 Bug 被打上标签后，Claude Code 会自动执行分流排查：若是配置问题则直接转派给相应运维人员，若是代码问题则自动生成 PR 供开发者审核，使排障周期从过去的 **2 到 3 天** 骤降至 **1 到 2 小时**。 --- ### [09:30](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=570s) Audible：以大模型消除冗余杂务与技术栈迁移 Ian Moraes 对 Senthil 的观点深表赞同。他强调，在业务高并发、交付周期极度压缩且背负大量遗留代码的环境下，测试周期往往会被迫压缩。为了消除这些枯燥重复的“杂务”（Toil），Audible 团队深度利用了生成式工具： - **测试框架自动化升级**：借助 Amazon Q Developer，团队在极短时间内将数以千计的测试用例从 **JUnit 4 平滑迁移至 JUnit 5**。 - **底层运行环境演进**：结合 Cline 与命令行 AI 工具，无缝完成了从 **JDK 8 到 JDK 17 及 JDK 21** 的多应用跨代迁移，大幅降低了人工改造遗留系统的阻力。 ### [10:08](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=608s) 单元测试生成与遗留系统现代化的 GenAI 落地 在探讨如何减少开发者的繁琐劳作（developer toil）时，嘉宾们分享了多个已在企业中被验证的生成式 AI（GenAI）核心落地场景。现场向开发者观众抛出了一个互动问题：“你们当中有多少人真正喜欢写单元测试用例？”台下响应寥寥无几。正是看到这一痛点，团队将生成式 AI 视为巨大的“空白机会”，投入精力利用 GenAI 针对 JUnit 和 NUnit 等测试框架自动生成单元测试用例，从而将工程师从机械的测试编写中解放出来。 另一个极具价值的场景是遗留系统现代化（Legacy Modernization）。在重构老旧系统时，工程团队常面临“看得到代码、却找不到原始业务需求”的尴尬困境。借助 GenAI 对遗留代码进行逆向工程，能够精准推导并梳理出底层的业务需求，再将这些需求输入到后续的现代化重构流程中，大幅加速了老旧架构的升级迭代。随后，主持人 Bryce 现场调研了与会者日常使用 AI 编程助手的习惯，涵盖了 GitHub Copilot、Cline、Claude Code、Gemini CLI 以及 Codex 等工具，展示了当前 AI 编码工具在工程一线的快速渗透。 - **单元测试自动化**：基于 JUnit、NUnit 框架利用 GenAI 自动生成测试用例，解决开发者最不愿手动编写重复测试的痛点。 - **逆向工程需求**：从无文档的遗留代码中自动提取业务逻辑与系统规格，为现代化重构提供需求输入。 - **一线工具调研**：现场调查了 GitHub Copilot、Cline、Claude Code 及 Gemini CLI 的实际使用情况。 --- ### [11:39](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=699s) 建立代码信任：多数投票机制与 Oxycopilot 自愈体系 针对工程师普遍担忧的“AI 生成代码、测试或部署脚本是否可信并符合质量规范”这一问题，Oxylabs 的研发工程师 Andrius Kūkšta 强调了“人机协同”（Human-in-the-Loop）在防范技术债方面的必要性。在实际业务中，Oxylabs 推出了名为 Oxycopilot 的产品，专门用于自动化生成网页数据解析指令。由于网页数据解析极其脆弱，HTML 结构哪怕出现极细微的变化，也会导致整个解析流程崩溃，最终使用户无法获取数据。 为了确保 AI 生成指令的高可用与准确率，团队采用了“多数投票”（Majority Vote）技术：将相同的解析任务同时分发给多个不同的模型，并选择出现频率最高的输出结果，从而最大程度压低模型幻觉和犯错的几率。此外，平台支持客户将解析指令保存在基础设施中通过 ID 调用，后台会持续追踪成功率指标；一旦监控到成功率出现下滑，系统会立即触发“自愈流程”（Self-healing Process），利用最新失败的页面样本重新生成并测试指令，验证成功率回升后无缝替换旧指令，实现对最终客户完全透明的自动修复。 ```mermaid flowchart TD accTitle: Oxycopilot 多数投票与自愈流程架构 subgraph S1["解析指令生成与校验"] A["输入待解析网页"] --> B["分发至多个大模型"] B --> C["多数投票算法"] C --> D["筛选最高频优质指令"] end subgraph S2["运行时监控与自愈"] D --> E["存入基础设施 (ID调用)"] E --> F{"成功率监控"} F -- "正常" --> E F -- "指标下滑" --> G["提取失败样本触发自愈"] G --> B end ``` --- ### [14:18](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=858s) 质量评估体系与模型集成（Ensemble）策略 Qualitest 的 CTO Vikul Gupta 从企业治理与技术评测两个维度补充了如何构建对 AI 输出的信任机制。在流程层面，无论是业务需求的校验还是生成代码的审核，企业高管（C-level）一致要求必须保留专家介入（SMEs in the loop）。当人工审核者对模型的输出进行覆盖（override）修改时，这一修正数据会反哺并触发自动化学习流程，确保系统未来不会重蹈覆辙。 在技术评测层面，团队建立了一整套严格的量化指标体系，对每次调用的机器学习或大语言模型进行实时打分和透明发布，包括相关性得分（relevancy score）、幻觉率、毒性（toxicity）、偏见测试（bias tests），以及传统的 ROUGE 和 BLEU 得分。此外，他们采用了算法集成（Ensemble of algorithms）策略，让多个模型针对同一任务并行生成结果，并依据当前数据的具体特征，由评测系统智能决定选用最适合的模型输出，从而在底层技术上构筑质量防线。 - **人工覆盖与闭环学习**：专家在审核环节纠正模型输出，修正记录直接作为反馈数据用于模型自适应优化。 - **多维度度量指标**：在生成链路中持续监控并公布相关性、幻觉、偏见、毒性及 ROUGE/BLEU 评分。 - **算法集成动态路由**：结合模型打分评估（Model-graded evaluation），根据输入数据特征动态选择表现最佳的模型。 --- ### [16:05](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=965s) 角色重塑与能力跃升：上下文工程与开发平民化 Audible 的副总裁 Ian Moraes 指出，AI 正在为软件的架构设计、开发、部署和运维带来一场“地震般的范式转变”（seismic shift）。软件工程师的角色正在从繁琐的事务性编码中解放出来，不再被文档撰写、常规升级和运维琐事消耗精力，从而能够将更多的时间倾注于创意、想象力以及对客户需求的共情上。 ```mermaid flowchart LR accTitle: 软件工程师技能范式转变 subgraph Old["传统确定性开发"] A["编程语言<br/>(Java / C++ / Python)"] B["语法与正则<br/>(Syntax & Regex)"] A --> B end subgraph New["现代非确定性开发"] C["上下文工程<br/>(Context Engineering)"] D["记忆与工具调用<br/>(MCP / Memory / Prompts)"] C --> D end ``` 这一转变对工程师提出了截然不同的技能要求。过去几十年，计算机科学训练工程师通过 Java、C++、Python、Shell 脚本和正则表达式与机器进行确定性的精确交互；而在新范式下，开发转向了“上下文工程”（Context Engineering），工程师必须熟练驾驭短期记忆与长期记忆管理、通过 MCP（Model Context Protocol）协议进行工具集成、设计系统提示词与用户提示词。同时，AI 工具极大地加速了“软件构建的平民化”（democratization），非技术背景的业务伙伴也能直接参与到开发流程中，为产品带来更加多元的视角与灵感。 --- ### [19:37](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1177s) 智能体编排时代：工程师转型为“Agent 管理者” eBay 基础设施副总裁 Senthil Padmanabhan 预判，软件工程的未来正在演进为一个“软件工程师成为智能体（Agent）管理者”的时代。整个软件产品开发生命周期（PDLC）都将被 Agent 驱动：当一个新的用户故事（User Story）提交时，系统会唤醒**架构师 Agent**制定技术方案，**实现 Agent**根据方案编写代码，**测试 Agent**执行验证确保功能无误，最后由**部署 Agent**（如智能 CI/CD）自动化发布至生产环境。 在这种模式下，人类软件工程师的角色转变为“总指挥与编排者”（Orchestrator），核心职责是确保各个 Agent 各司其职、输出符合预期，并负责提供精准的上下文、工具集以及微调评测标准。Senthil 强调，未来工程师的核心竞争力取决于两点： 1. **对底层运作机制的深度理解**：“AI 或许能帮你完成 90% 的工作，但剩下的 10% 才是产生 100 倍杠杆效应的关键”，只有深刻理解底层运行逻辑的人，才能在边缘场景下解决故障、把控全局。 2. **极度精准的思想表达与沟通能力**：在人机协同与复杂系统编排中，清晰准确地传递指令与业务意图是决定系统效能的核心分水岭。 ### [21:12](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1272s) 黑客松启示与初级工程师的“AI 依赖症” 嘉宾们分享了企业内部应用 AI 编程工具的真实经历。在硅谷风格的内部 AI 黑客松上，规则要求参赛者必须使用 AI 生成代码而非纯人工编写。原本组织者普遍认为，既然大家都能使用同样强大的 AI 工具，经验与技能的壁垒就会被抹平，任何人都有可能夺冠。然而比赛结果却截然相反：最终获胜的依然是那些最顶尖的软件工程师和具备深厚业务领域知识（Subject Matter Expertise）的专家。其关键在于，这些顶尖工程师真正理解业务领域，并且能够极其清晰、精准地与 AI 进行沟通。AI 确实让工具使用“民主化”，使所有人都能变强，但专业能力带来的增量杠杆反而被成倍放大了。 与此同时，这种技术普及也给初级工程师带来了隐患。团队中有约 10% 的应届新员工（freshers），他们同样拥有 AI 工具，但团队很快发现了一个异常现象：每修复一个 bug，就会随之产生四个新 bug。究其原因，初级工程师过度依赖 AI，自身的代码调试（debugging）基本功严重不足。他们对底层代码缺乏理解，一旦 AI 给出修复方案，便不加辨别地直接合并代码，根本不清楚改动的具体影响。因此，企业观察到明显的生产力提升主要集中在资深员工身上；初级员工由于缺乏对 AI 输出结果的理解与判断力，在正确使用 AI 方面反而陷入了挣扎。 - **黑客松冠军画像**：精通领域知识、沟通表达清晰、架构理解深厚的资深工程师。 - **初级工程师的困境**：过度依赖 AI 生成代码，缺乏调试能力，导致修复一个 bug 衍生出四个新 bug。 - **生产力分化**：AI 带来的实质性效率收益目前高度偏向资深工程师（Senior folks）。 --- ### [22:47](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1367s) 全民开发者浪潮与工程落地的现实挑战 随着大语言模型（LLM）的普及，产品经理（PM）和团队主管等非纯技术人员也开始积极使用 AI。这一方面减轻了数据分析师和工程师处理临时琐碎任务（ad hoc tasks）的负担，但另一方面也催生了新的工程隐患。非技术人员往往容易高估 LLM 的实际能力，经常拿着 LLM 随手生成的原型演示，就兴冲冲地要求将其直接转化为新功能或新产品。 这种“全民开发者”（citizen developers）虽然能够在缺乏专属开发团队的情况下快速把想法跑起来，但却给正式的开发团队带来了巨大的维护与架构风险。工程师不得不花费精力向他们解释，这些由 AI 快速生成的简单原型如果放到大规模高并发（large scale）等严苛的生产环境中，将面临极高的技术债务和支撑难度。 ```mermaid flowchart TD accTitle: 非技术人员使用 AI 带来的机遇与工程挑战 subgraph NonTech ["业务与非技术人员 (PM / Lead)"] A["使用 LLM 生成原型 / 代码"] --> B["减少数据分析与工程临时琐碎支持"] A --> C["高估 LLM 实际落地能力"] end subgraph Challenge ["工程团队面临的现实挑战"] C --> D["提出将 AI 原型直接作为产品功能"] D --> E["工程师介入评估与技术把关"] E --> F["指出大规模高并发与架构风险"] end ``` --- ### [23:53](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1433s) 企业落地 AI 的三大支柱：Audible 的实战经验 面对如何将 AI 工具深度整合进现有开发流程、技术栈与团队文化的挑战，来自 Audible 的 Ian Moraes 总结了成功落地 AI 的三大核心经验： 1. **高管层的坚定支持（Executive Support）**：Audible 首席执行官亲自推动并承诺支持全员技能提升与再培训，设立跨职能的研发与培训项目（不仅限于技术部门），并由公司全额报销 AI 培训费用。在技术领导层层面，团队举办内部“科学博览会”（Science Fairs），广泛展示和分享各部门使用 AI 工具带来的实际业务成效。 2. **倡导自发探索与技术好奇心（Organic Learning & Curiosity）**：践行 Audible “从文化与技术中学习并汲取灵感”的用人原则，公司并不强制或统一指定某一款特定工具，而是鼓励团队针对具体业务痛点自主挑选最适合的工具。考虑到市面上 AI 工具层出不穷且成熟度演进极快（例如评估时的新工具、新版本），保持工具选择的灵活性至关重要。目前团队及研发负责人广泛采用命令行工具（如 Q CLI）并获得了显著的效率提升。 3. **建立常态化同行互助学习机制（Peer Learning）**：团队每周自发组织一小时的实战学习会，拒绝单向的“大师布道”，而是由不同岗位的同事轮流上台演示。大家现场打开笔记本电脑，配置工具链（如各类 MCP 工具等），手把手演示如何用 AI 工具更高效地解决日常工作中的具体问题。通过“以教促学”（By teaching we learn），极大地加速了团队整体的技能跃迁。 --- ### [27:20](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1640s) 组织层面的顾虑、合规与技术架构卡点 从全员推广与实施的角度来看，企业在推行 AI 时面临着文化心理与技术架构的双重关卡： 在员工心理层面，许多员工一方面担忧 AI 会取代自己的岗位，产生信任危机；另一方面甚至会有“使用 AI 完成工作是在作弊”的道德顾虑。企业需要自上而下地明确传达：AI 的本质是赋能和增强员工角色，将人类从枯燥的事务性工作中解放出来，从而转向更高价值的智能工作。 在公司架构与合规层面，当员工准备就绪后，企业面临一系列关键技术决策与合规审查： - **模型与部署形式选择**：选择哪种 LLM？采用托管云服务、SaaS 模式还是私有化本地部署（On-prem）？这需要经过严格的架构评审委员会（Architectural Review Board）评估。 - **知识产权与数据外泄风险**：AI 生成代码是否暗含侵权或非法复制开源代码的风险？员工在使用 ChatGPT 等公共工具时，是否会无意上传敏感商业数据导致公司核心知识产权（IP）泄露？企业必须在安全合规与敏捷创新之间建立起严密的防护体系。 --- ### [29:29](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1769s) 试错成本坍塌与未来差异化：从 GenAI 迈向 Agentic AI 谈及对未来 AI 辅助开发的趋势展望，Senthil Padmanabhan 提出了两个令人振奋的核心观点： 第一，**探索机会空间的边际成本正在归零**。决策理论表明，当不确定性增加时，推迟决策的价值虽高但伴随着巨大的机会成本。过去，产品经理验证 10 个想法或架构师尝试 5 种框架需要耗费大量时间，等待数据分析师调研或设计师绘制原型，试错极其昂贵；如今“一切尽在 Prompt 之中”，原型可以瞬间生成验证，机会成本大幅坍塌至趋近于零，工程师得以集中精力进行真正的生产级落地。 第二，**“人类差异化”（Human Variance）将成为核心竞争壁垒**。在当前范式下，普通人花 20 或 200 美元就能获得与全球财富 500 强企业完全一致的前沿基座模型。如果仅仅机械、平庸地直接使用模型，输出结果只会迅速“回归均值”（Regress to the mean）。唯有凭借人类自身的创造力、构建高度精细化的上下文、注入明确的业务使命感以及与产品产生情感共鸣，才能创造出真正的价值。 纵观技术演进路径，软件开发正经历从“传统机器学习（用于分析与预测）”到“生成式 AI（用于内容与代码生成）”，再到“行动驱动型 AI（Action AI / Agentic AI）”的深刻跨越。 | 发展阶段 | 核心技术范式 | 主要应用场景与特征 | |:--- |:--- |:--- | | **第一阶段** | 传统机器学习 (Traditional ML) | 侧重于数据分析、分类与业务预测 | | **第二阶段** | 生成式 AI (Generative AI) | 侧重于代码辅助生成、原型快速构建与内容创作 | | **第三阶段** | 行动与智能体 AI (Action AI / Agentic AI) | 具备自主规划、工具调用与任务闭环执行能力的新一代范式 | ### [31:35](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1895s) 智能体架构与意图驱动的软件测试变革 讨论转向了智能体 AI（Agentic AI）对软件工程、测试与运维流程的深远影响。演讲者以内部构建的平台“Coco Copilot”为例，阐述了正在演进的全新智能体架构。在未来的开发场景中，工程师不再需要按部就班地手动编写测试场景、制定具体用例、或者分别下达安全测试与性能测试的指令，而是只需用自然语言传达高层“意图”（Intent）——例如“获取最新构建版本并进行全面测试，确保没有缺陷遗漏（Bug Leakage）”。 ```mermaid flowchart TD accTitle: 智能体驱动的测试执行流 subgraph Input["用户输入层"] U["用户自然语言意图<br/>（如：测试最新版本并防止Bug泄漏）"] end subgraph Engine["智能体拆解与调度"] P["意图解析引擎"] A1["功能测试智能体"] A2["安全审计智能体"] A3["性能压测智能体"] end subgraph Output["交付层"] R["自动化测试报告与质量门禁"] end U --> P P --> A1 P --> A2 P --> A3 A1 --> R A2 --> R A3 --> R ``` 随后，系统会自动将该意图拆解为一系列具体动作，并分发给专门的子智能体在预定测试周期内并行完成任务。此外，工程师们天生具有极强的好奇心，AI 展现出的“深度研究（Deep Research）”能力能够根据使用者现有的理解水平因材施教，极大地加速新技术与新概念的学习，这将重塑整个技术教育体系。 --- ### [33:09](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1989s) 角色重塑、非确定性软件与 DSIET 的诞生 随着大语言模型和上下文工程（Context Engineering）的兴起，行业内涌现出了许多半年前根本不存在的全新岗位（如 Prompt Engineer）。AI 自动接管调优警报、日志分析和文档编写等重复繁琐的工作后，工程师能够将更多精力投入到需要“同理心与创造力”的产品创新中。这也促使计算机科学教育重新审视其培养目标——传统的计算机教育过度聚焦于编写代码语法，而当今软件工程师需要更宽广的综合能力。 与技术演进相伴的是传统岗位的进化。正如十年前自动化测试工程师（Automation Engineer）以及随后出现的 SDET（软件测试开发工程师）一样，Qualitest 如今设立了全新的 **DSIET（Data Science Engineer in Test，数据科学测试开发工程师）** 岗位： * **测试对象的范式转移**：从传统的“确定性系统”（如银行贷款审批，结果非 0 即 1）演进为“AI 赋能的非确定性系统”（如电商推荐，上午和晚上的登录偏好不同会导致推荐结果截然不同）。 * **新型质量挑战**：传统自动化测试无法直接应对大模型输出的非确定性与随机波动，偏见测试（Bias Testing）和幻觉评估成为全新必修课题。 --- ### [36:07](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2167s) 确定性与非确定性的双轨架构及角色跨界 未来软件系统将明确划分为“非确定性部分（AI 生成）”与“确定性部分（人类治理与传统规则引擎）”。大模型不擅长保守机密且具有幻觉风险，因此传统面向对象编程（OOP）、硬性规则引擎等确定性架构在身份验证、权限隔离（防止用户 A 看到用户 B 的数据）、回滚机制与高标准评测（Evals）中依然不可或缺。高质量的评测是防止 AI 系统演化为企业法律与合规责任（Liability）的核心防线。 与此同时，AI 工具正在大幅降低技术跨界的门槛： * **全栈转型加速**：纯后端工程师借助 AI 辅助能够在毫无前端经验的情况下，独立为“AI Studio”平台开发出高质量的前端界面。 * **行政类职能淘汰**：单纯记录会议纪要等低附加值工作正被 Microsoft Teams Copilot 等工具迅速取代。 * **角色向产品工程师迁移**：开发者的主要价值不再是编写语法，而是转向设计可扩展、安全且可持续的系统架构。 --- ### [38:52](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2332s) 新生代工程师的培养之道：筑牢基础与善用生成式 AI 在探讨应届毕业生的培养与入职引导（Onboarding）时，导师们强调必须警惕对 AI 的过度依赖。年轻学生借助 AI 可以在 10 秒内理解并快速交付任务，但在遇到深层 Bug 时往往缺乏查看日志、单步跟踪等传统调试能力。 培养新一代工程师的共识策略包括： * **打牢第一性原理与底层基础**：先通过传统方式理解数据结构、微积分和物理基础（正如学习费曼的治学方法），在具备扎实基本功后，再引入 AI 作为效能放大器。 * **利用 GenAI 加速代码库上手**：在实习生入职的前几周，引导他们利用生成式 AI 快速通读复杂系统的历史代码。 * **反哺团队代码资产**：实习生通过 AI 快速上手后，可以反向为项目补充高质量的测试用例与工程文档，企业监控指标上甚至常出现实习生入职带来的“AI 对话调用量暴涨”效应（Intern Effect）。 ### [41:49](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2509s) 影响研发团队生产力的 Top 2 工具盘点 在讨论老牌资深开发者面临退休浪潮、团队可能陷入维护困境的背景下，现场观众抛出了一个极具实操性的快问快答问题：在过去一年中，究竟哪两款工具对各位嘉宾所在开发团队的生产力影响最大？几位技术负责人的回答高度集中在当前最前沿的 AI 编程辅助工具上： 第一位嘉宾毫不犹豫地提到了 Claude Code 与 Cline；紧接着，第二位嘉宾则指出了 Amazon 体系的利器——Amazon Q Developer（包括 Q CLI）以及同样备受青睐的 Cline；随后的嘉宾也一致将选票投给了 Cursor 和 Claude Code。在推荐 Cline 时，嘉宾特别强调了其专属上下文维护机制，郑重向现场工程师建议：“尤其是搭配 Memory Bank 使用的 Cline。如果你用 Cline 还没配上 Memory Bank，一定要去查查了解一下（look it up）。” - **现场嘉宾高频推荐的 AI 研发工具清单**： - **Cline**：受到多位嘉宾力荐，特别提及其与 Memory Bank（内存库机制）结合使用时，能为复杂项目提供极佳的持久上下文与指令记忆能力。 - **Claude Code**：作为新兴的高性能终端与编程智能体工具，被多家团队视为生产力跃迁的关键支柱。 - **Cursor**：深度集成的 AI IDE 代表，稳居多家企业开发流程的核心。 - **Amazon Q Developer / Q CLI**：在企业级云环境和命令行终端作业中展现出显著的辅助研发效益。 --- ### [42:22](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2542s) AI 在生产运维排错与日志分析中的实战与 Agent 化应用 随后，一位现场观众提出了一个许多工程团队都会面临的痛点问题：软件工程不仅仅包含写代码，现实中大量工程师的精力其实都被绑死在生产支持（Production Support）和运维排错上。工程师通常要花费大量时间去翻查、检索纷繁复杂的系统日志，定位偶发故障的根因并进行修复。他想知道，各位专家目前是否已经引入了 AI 工具来辅助日志调试？或者是否正在探索某种智能体（Agentic）落地方案来实现全自动化的日志分析与故障排查？ ```mermaid flowchart TD accTitle: AI-Powered Production Support and Incident Analysis accDescription: 生产支持中利用 AI 进行指标关联、日志排查与根因分析的自动化流程。 subgraph Monitoring ["监控与告警层"] A["生产告警与指标异动"] --> B["自动化事件触发"] C["部署系统记录 (最后发布时间)"] --> B end subgraph AIAgent ["AI 诊断与总结层"] B --> D["AI 聚合分析引擎"] E["跨系统海量日志数据"] --> D D --> F["关联分析与事件摘要生成"] end subgraph Resolution ["运维与决策层"] F --> G["向值班人员呈现问题全貌"] F --> H["定位系统性根因 (内存瓶颈/发布缺陷)"] H --> I["反哺系统：调整告警阈值与规则"] end ``` 嘉宾给出了非常肯定的回答，明确表示团队已经在生产支持环节深度应用了 AI。他进一步梳理了生产支持的实际工作场景：这不仅仅是看单条日志，而是要理清各种日志之间的交错关系、研判监控指标与报警异动，并对生产故障（Incident）进行智能提炼与摘要总结。 目前团队借助 AI 已经能做到： 1. **自动定义与抽象故障**：迅速从海量日志中筛选并抓取对应上下文的关键信息； 2. **时序与部署联动分析**：在值班工程师（On-call）介入的第一时间，直接为其呈现结构化报告，例如清晰标注“上一次部署发生在何时，且该时间点与生产故障的爆发高度重合”； 3. **识别工单背后的系统性隐患（Systemic Issues）**：在处理跨多张工单的日常琐碎问题时，单靠人工极难发现规律，而 AI 工具能帮助挖掘出底层共性的生产瓶颈——无论这起故障是由特定部署版本引起的隐性 Bug，还是系统长期面临的某种内存资源受限（Memory Constraint）问题。此外，AI 的分析结果还会被进一步用来反哺监控体系，动态调整报警规则与阈值设置，防止告警风暴或误报。 --- ### [44:00](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2640s) 物理隔离环境（Air-Gapped）与超低延迟场景下的本地大模型部署 最后一位观众提出了安全敏感情境下的落地疑问：在严苛的物理隔离网络（Air-Gapped Networks）或高密安全内网中，是否有使用这些 AI 工具的实战经验？未来在安全网络环境中应用 AI 工具的前景又该如何看待？ 嘉宾指出，这种需求在医疗健康（Healthcare）与银行金融（Banking）等强监管行业的客户中“几乎无时无刻不在被提及”。针对无法连接外网的物理隔离环境，核心解法就是摆脱对云端托管商用大模型（Hosted LLMs）的单一依赖，转向自建私有化部署。嘉宾特别提到了 Ollama 和 Hugging Face 社区生态： - 团队一直在生产环境中持续使用 Ollama 这类本地运行时，而且这些开源/本地工具“每周都在飞速迭代进化（getting better week by week）”，完全能支撑内网环境下的各种特定任务。 来自 eBay 的 Senthil 随即补充道，除了高安全性物理隔离环境之外，同样的本地化私有部署逻辑在**极致低延迟（Ultra-low Latency）**的业务要求下同样是必选项。例如在 eBay 的核心在线拍卖（Auctioning）业务场景中，“每一毫秒都至关重要（every millisecond matters）”。在这种毫秒级定胜负的高并发关键路径上，任何跨网络调用远程大模型 API 带来的网络延迟开销都是完全无法接受的，因此团队必须在本地基础设施上自行托管部署模型，以确保在极短时延内完成实时推理计算。 整个研讨会在此热烈而硬核的 Q&A 交流中圆满画上句号，主持人和嘉宾们向全场观众致谢致意。 ## Notable quotes > [02:33](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=153s) 过去需要几个月才能完成的测试套件构建，现在仅需 2.8 分钟即可生成。 — Vikul Gupta > [02:33](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=153s) 研发后期发现的缺陷中有 37% 根源在于需求本身的质量缺陷，Qualitest 利用 AI 实现真正的‘测试左移’，直接在需求阶段进行 AI 校验与清洗，从源头消灭这 37% 的隐患。 — Vikul Gupta > [19:37](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1177s) AI 或许能帮你完成 90% 的工作，但剩下的 10% 才是产生 100 倍杠杆效应的关键。 — Senthil Padmanabhan > [21:12](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1272s) 每修复一个 bug，就会随之产生四个新 bug。 > [23:53](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1433s) 通过‘以教促学’（By teaching we learn），极大地加速了团队整体的技能跃迁。 — Ian Moraes > [29:29](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1769s) 一切尽在 Prompt 之中，原型可以瞬间生成验证，机会成本大幅坍塌至趋近于零。 — Senthil Padmanabhan > [29:29](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1769s) 如果仅仅机械、平庸地直接使用模型，输出结果只会迅速‘回归均值’（Regress to the mean）。 — Senthil Padmanabhan > [31:35](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=1895s) 获取最新构建版本并进行全面测试，确保没有缺陷遗漏（Bug Leakage）。 > [41:49](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2509s) 尤其是搭配 Memory Bank 使用的 Cline。如果你用 Cline 还没配上 Memory Bank，一定要去查查了解一下。 > [44:00](https://www.youtube.com/watch?v=E37-VgrzMKQ&t=2640s) 在 eBay 的核心在线拍卖业务场景中，‘每一毫秒都至关重要（every millisecond matters）’。 — Senthil Padmanabhan ## Mentioned Bryce Herbranson, Ian Moraes, Vikul Gupta, Senthil Padmanabhan, Andrius Kūkšta, Qualitest, eBay, Audible, Devon Energy, Oxylabs, Oxycopilot, eBay Coder, Coco Copilot, GitHub Copilot, Cline, Cursor, Claude Code, Amazon Q Developer, Q CLI, Gemini CLI, Codex, ChatGPT, Ollama, Hugging Face, Microsoft Teams Copilot, SAP, Salesforce, Selenium, Playwright, Cypress, JUnit, NUnit, Spring Boot, JDK, MCP, DSIET, SDET ## Video description Vikul Gupta, CTO, Qualitest Senthil Padmanabhan, Vice President, Platform & Infra Engineering, eBay Inc Ian Moraes, VP and Head of Consumer Domains Tech, Audible Bryce Herbranson, Sr. AI Software Developer, Devon Energy Andrius Kūkšta, R&D Engineer, Oxylabs Abstract This session will dive into how AI is reshaping the software engineering landscape, from automating code generation to optimizing development workflows. Panelists will discuss the technical advancements in AI-driven tools that are streamlining tasks such as debugging, testing, and deployment. Learn how AI is being integrated into DevOps practices, enhancing productivity, and enabling engineers to focus on more complex and creative problem-solving. The discussion will also cover challenges in adopting AI tools, such as ensuring code quality and managing the balance between automation and human oversight in software development. [Source](https://www.youtube.com/watch?v=E37-VgrzMKQ)

[2:33] Vikul Gupta: 过去需要几个月才能完成的测试套件构建，现在仅需 2.8 分钟即可生成。 [2:33] Vikul Gupta: 研发后期发现的缺陷中有 37% 根源在于需求本身的质量缺陷，Qualitest 利用 AI 实现真正的‘测试左移’，直接在需求阶段进行 AI 校验与清洗，从源头消灭这 37% 的隐患。 [19:37] Senthil Padmanabhan: AI 或许能帮你完成 90% 的工作，但剩下的 10% 才是产生 100 倍杠杆效应的关键。 [21:12] 每修复一个 bug，就会随之产生四个新 bug。 [23:53] Ian Moraes: 通过‘以教促学’（By teaching we learn），极大地加速了团队整体的技能跃迁。 [29:29] Senthil Padmanabhan: 一切尽在 Prompt 之中，原型可以瞬间生成验证，机会成本大幅坍塌至趋近于零。 [29:29] Senthil Padmanabhan: 如果仅仅机械、平庸地直接使用模型，输出结果只会迅速‘回归均值’（Regress to the mean）。 [31:35] 获取最新构建版本并进行全面测试，确保没有缺陷遗漏（Bug Leakage）。 [41:49] 尤其是搭配 Memory Bank 使用的 Cline。如果你用 Cline 还没配上 Memory Bank，一定要去查查了解一下。 [44:00] Senthil Padmanabhan: 在 eBay 的核心在线拍卖业务场景中，‘每一毫秒都至关重要（every millisecond matters）’。

[0:04] Ian Moraes: did more than just paint the Mona Lisa. [0:04] Vikul Gupta: engineering of AI [2:33] Vikul Gupta: shift left [5:53] Senthil Padmanabhan: essential engineering [16:01] Ian Moraes: seismic shift in terms of how we design, how we develop, how we deploy and how we support software. [24:09] Ian Moraes: study and draw inspiration from culture and technology. [29:29] Ian Moraes: human variance [38:38] Senthil Padmanabhan: the intern effect

[0:04] Hello. Hello. Thank you for coming. They [0:06] put us uh all the way down at the [0:07] hallway at the very very end. So glad [0:09] you guys make it. And I know we are the [0:12] last group probably between you and [0:13] lunch. So thank thanks for coming again. [0:16] Uh my name is Bryce Herbronson. I'm the [0:19] lead AI software developer at Devon [0:21] Energy on the emerging technology team. [0:23] Um I'll let the panelists introduce [0:25] themselves. Ian, you want to go first? [0:27] >> Yes. I'm Ian Maras and I'm with Audible [0:31] and I lead the consumer domains tech [0:33] team where we support uh consumerf [0:36] facing functions such as when you sign [0:38] up for a subscription or make a [0:39] purchase. Um, and uh, in terms of [0:43] Audible, if you may not be familiar with [0:45] Audible, we're a leading provider and [0:47] producer of premium audio storytelling [0:51] and uh, we deliver cutting edge tech and [0:54] create pioneering audio content for our [0:56] listeners. Um, I know I've met a number [0:59] of our listeners here over the last few [1:00] days and they've asked me what I'm [1:02] listening to and I'm listen currently [1:04] listening to Leonardo da Vinci. [1:07] interesting individual and he did more [1:09] than just paint the Mona Lisa. [1:13] >> That's interesting. Uh my name is Vicl [1:15] Gupta. I'm the CTO with a company called [1:17] Qualest. Um what does Qualitest do? Two [1:21] broad areas we play in. One is how do [1:24] you use AI to do smarter, safer uh and [1:28] intelligent uh engineering. So when I [1:30] say engineering, dev, test, ops, how do [1:34] you do it smarter? And the other space [1:36] we work in is uh engine engineing of AI. [1:40] So starting from collection of data for [1:44] AI models, building AI models, testing [1:46] AI models, deploying AI models, [1:49] integrating them in pipeline and then uh [1:52] monitoring them for drifts. We do it [1:54] all. So we use we build AI models and we [1:58] use AI to do smart engineering. [2:02] >> Hello everyone, my name is Santal [2:03] Batman. I run platform and infra at eBay [2:06] and since I run platform I'm also [2:07] responsible for engineering velocity in [2:09] the company and that's also the reason [2:10] I'm here. Thanks. [2:12] >> Hi, my name is Andreas. Uh I'm R&D [2:15] engineer at Oxlabs, a company which [2:18] offers data gathering solutions. Uh we [2:21] are based in uh Lithuania and I'm on a [2:23] team which basically is responsible to [2:26] like create and test it out new features [2:28] which we could introduce and provide to [2:31] our customers. [2:33] Thank you. So just curious how many [2:36] software developers, software engineers [2:37] do we have out here? I know this is wow [2:39] way more than I thought. Know this is [2:41] primarily like a data science space but [2:44] uh big turnout. So AI is increasingly [2:47] becoming part of development workflows [2:50] um daily integrations from code [2:52] generation to testing and deployment. [2:55] Can you guys share a specific example [2:57] when you or your teams have integrated [2:59] AI into your software engineering [3:01] platforms and processes? Um, and what [3:03] did it solve and how did it change the [3:05] way you work? Cool. You want to lead us [3:07] off? [3:07] >> Sure. So, like I explained in the [3:10] introduction, we use AI to do smart [3:13] testing. I'll take that as an example. [3:16] So, when we talk about testing, right, [3:18] there's automated testing, there's [3:20] performance testing, there's security [3:22] accessibility testing. Now to do any [3:26] testing you need to have [3:29] uh same requirements [3:31] create test scenarios create test cases [3:34] create feature files create automation [3:37] scripts [3:38] all of this requires a lot of time a lot [3:40] ofmemes a lot of knowledge [3:44] till 2 and a half years back we hadmemes [3:46] who were doing it the way we [3:49] differentiated ourselves was we've been [3:50] doing it for 27 years if you talk about [3:53] SAP testing SFDC testing, banking [3:56] applications, healthcare or claim [3:58] portals. We had written test cases, [4:01] automated them and we had a library. But [4:04] then it still required a lot of human [4:07] effort. [4:09] And then LLMs came, geni came. What used [4:13] to take months, we can do it in 2.8 [4:18] minutes. [4:20] So a lot of you have been to a booth [4:23] where we have shown a live demo where [4:25] you get user stories. 37% of defects we [4:29] find later in testing are because of bad [4:32] requirements. [4:33] They could be they could be ambiguous. [4:37] They could be duplicate. They could be [4:39] unstructured. They could have not no [4:40] acceptance criteria. We use AI to [4:43] validate your requirements. [4:46] People talk about shift left. We use AI [4:49] to shift left. We're eliminating those [4:52] 37% of defects which will otherwise be [4:54] found much later in the cycle. So now we [4:58] have good requirements. With a click of [5:00] click of a button using AI, we can [5:02] autogenerate test scenarios, test cases, [5:06] feature files and automation scripts. [5:10] Automation scripts in Selenium, in [5:11] Playright, in Cyprus and so on. [5:15] Again I'll repeat what used to take [5:17] months we can do it in minutes. That's [5:21] how we are using AI. We can also use AI [5:24] to predict what test cases will fail. [5:28] Identify the risk of failure for those [5:30] test cases so that when you are [5:31] prioritizing your test set you can run [5:34] these earlier in the life cycle so that [5:36] they fail you can fix and still complete [5:38] your testing on time. This is just one [5:42] way we are using it from a testing [5:43] standpoint. [5:45] Yeah, I imagine using AI in testing and [5:47] testing AI hits pretty close to home [5:48] with qualest. Uh Sentil, did you have [5:52] any thoughts on this too? [5:53] >> Yeah, so so at eBay we can broadly [5:55] divide this into three categories, [5:57] right? One is ID agents. This is your [5:59] day-to-day uh software coding like where [6:01] engineers are spending time in their ID [6:02] to create software and the next I will [6:04] classify as a terminal and cloud agents, [6:06] right? These are workflows where these [6:08] they they give outcomes based on [6:10] specific tasks and lastly discovery [6:12] agents. We all know engineers spend only [6:13] a portion of that time coding. Most of [6:15] that time goes in discovery, right? [6:17] Waiting on peers and things like that. [6:19] So when you come to ID agents um in eBay [6:21] like GitHub copilot is the one most [6:23] commonly used one like all our [6:25] developers across the board use GitHub [6:26] copilot right it has been used uh when [6:28] we started it two years back and we have [6:30] been gradually ramping it up and almost [6:31] all developers have access to it but in [6:34] the last 10 years I would say client an [6:36] open source plug-in has gained immense [6:37] popularity among our developers right [6:39] it's an agentic framework it has a plan [6:41] and act mode and engineers allowing it [6:43] and we also started ramping up cursor [6:45] which is another tool that's gaining a [6:46] lot of popularity So what we observed is [6:49] right on and off we run some AB testing [6:51] uh we observed that for folks who use [6:53] these tool see a 17% reduction in PR [6:56] merge times that is code review merge [6:58] times and a 12% reduction in lead time [7:00] to change that's the time from the time [7:02] your software gets committed to the time [7:04] it reaches production so both were like [7:06] really promising numbers when you come [7:08] to these terminal or cloud agents by [7:10] terminal agents I mean the command line [7:12] uh agents right and the cloud agents are [7:13] those that are hosted on a URL um so you [7:16] know all software companies have to keep [7:18] up with software upgrades, right? Like [7:20] all technology companies have to do [7:22] constant software upgrades or else they [7:24] left behind because tech depth [7:25] accumulates and tech is one of the [7:26] reasons that holds people back, right? [7:28] It's it makes your velocity slower and [7:31] addressing them makes you go fast but [7:33] nobody wants to do that work. It's a [7:34] very grunt boring work um and um and in [7:37] eBay itself we spend around 10 to 15% of [7:39] our engineering capacity on something [7:40] called essential engineering which is to [7:42] modernize the software and make it up to [7:44] date. Um so this is another area where [7:46] we thought we can uh we can optimize [7:48] right and that's why we created a [7:49] terminal and a cloud agent called eBay [7:51] coder which is nothing but a agent [7:53] orchestration framework built on top of [7:55] cloud cloud as well as GPTs and that has [7:58] enabled us to do many upgrades within [8:00] the company for example last year we [8:02] upgraded our old application suit from [8:04] JDK8 to JDK 21 I think and the whole [8:07] process was 25% faster it was earlier [8:09] last year right uh and later spring boot [8:11] 2.7 to3 right and that was like 30% [8:14] faster So every time we do this the [8:16] upgrades are getting faster and faster [8:17] and the new models are making it better [8:18] and better. Um and lastly there is also [8:21] production use production bug use cases [8:22] right a lot of big companies deal with a [8:24] lot of production bugs. Um and there are [8:26] scenarios we didn't solve it across the [8:27] board but there is a very specific flows [8:29] that we are taking within the companies [8:30] and trying to solve it. Um for examp and [8:33] for that we use cloud code. Cloud code [8:34] is another terminal agent which [8:36] extremely popular. Um so we created a [8:38] full cloud MD context file where [8:40] engineers spent a couple of weeks [8:42] creating the whole workflow on how this [8:44] bug is trashed right starting from okay [8:45] this is the scenarios and these are all [8:47] the past code changes and past session [8:49] history and the beauty about this is [8:52] only a portion of this bug is actually a [8:54] code change most of the time this bug [8:55] repeatedly comes in only like maybe 50% [8:57] of it is a code change most of the time [8:59] it's either an agent who may change the [9:01] setting with our seller or it is a cash [9:03] refresh so now when the bug is getting [9:06] uh filed We tag it with clot code and [9:08] that goes and does the triage and it [9:10] either reassigns the bug to the agent or [9:12] it creates a PR for the developers to go [9:14] and do some stuff. So, so all this is [9:16] again a massive increase in [9:18] productivity. Right. This was usually [9:19] taking two to three days to triage and [9:21] now it's like within an hour or two cla [9:23] does the triage and it gives a PR or it [9:25] reassigns the bug to the right agent. So [9:26] again massive savings in times. [9:28] >> Cool. Thank you. [9:30] >> Yeah, if I could add I I'd echo what [9:32] Senthol pointed out. you know, in a high [9:34] velocity development environment where [9:37] you have tight timelines and you have [9:40] legacy code, uh, test cycles t typically [9:43] get compressed. And so we used, uh, Q [9:46] developer to be able to hit to go ahead [9:48] and migrate from JUnit 4 to JUnit 5 very [9:51] quickly. And we migrated thousands of [9:53] test cases to do so. And we also used [9:55] uh, client and CLI 2 to migrate from [9:59] JDK8 to JDK17 or JDK 21. and we did so [10:03] very seamlessly. So in terms of it helps [10:06] reduce toil which is routine and mundane [10:08] tasks. So if you're beginning your AI [10:10] journey, these are good use cases and [10:12] solid use cases that have been used in [10:15] multiple organizations to be able to go [10:17] ahead and deliver for our organizations [10:21] while reducing developer toil. [10:24] >> Two other use cases and I agree with [10:26] both Sentinel and I um we see I heard a [10:30] lot of developers are here. How many of [10:32] you love writing unit test cases? [10:35] >> Not many. [10:36] >> The best. So we saw this as a as a white [10:39] space. So one of the things we've [10:41] invested time is how to use GI to [10:44] autogenerate uh JUnit NUnit test cases [10:48] using that framework. So that's one [10:50] area. Second thing from a development [10:51] standpoint we find out is legacy [10:54] modernization. [10:55] A normal question when you are look [10:58] modernizing legacy code is what were the [11:01] requirements you have access to the code [11:03] but then you don't have access to the [11:05] business requirements and so on. So um [11:07] using ji to reverse engineer from legacy [11:10] code requirements now which can feed [11:12] into uh the modernization effort that's [11:15] another area we see genia being used a [11:17] lot more. So, the panelists dropped [11:19] several names of a few agent decoders, [11:21] uh, coding assistants. How many people [11:23] daily drive copilot? [11:26] Yeah. How about Klein? [11:28] Klein's where it's at. What about Claude [11:30] Code? Gemini CLI. [11:34] Codeex, not so much. [11:36] >> Q. I haven't tried that one. All right. [11:39] Uh, we'll move on here. One concern we [11:41] often hear from engineers is trust. How [11:43] do you ensure that AI generated code [11:45] tests or deployment scripts meet quality [11:47] standards and don't introduce new risks? [11:50] Uh what processes or checks have you put [11:52] in place and balance automation with [11:54] human oversight? Um Andreas, I'm going [11:57] to pitch this over to you. See what you [11:58] think. Uh so yeah basically uh we start [12:02] from like micromanaging and basically [12:05] encouraging our colleagues uh to that is [12:09] a must to have this human in the loop [12:12] and check what AI offers and what uh [12:14] code it generates it for you because it [12:17] can create like a technical depth if you [12:19] are unaware of what's happening in your [12:22] pro project. uh but in more uh [12:25] particular case and as an example I can [12:28] uh uh say that we have a product called [12:32] Oxycopilot where we basically generate [12:35] uh data parsing instructions and uh it's [12:38] really like sensitive field because uh a [12:41] small mistake or a small change in HTML [12:44] file can break the whole process and the [12:47] clients won't get data parsed basically. [12:50] So uh what we are doing we we are using [12:54] uh AI to generate these uh parsing [12:57] instructions and we use this technique [13:00] called majority vote where we simply [13:02] send the same tasks to multiple models [13:06] and uh select the response which is the [13:08] most frequent one basically. So that way [13:11] we like minim minimize the chance of AI [13:14] making a mistake or hallucinating [13:18] and uh moreover we give the ability for [13:21] clients to like save these instructions [13:24] in our infra. So meaning that they are [13:27] simply using by providing the ID and [13:30] that way we are able like to track uh [13:32] the success rates how they perform over [13:35] time and once we notice that uh the [13:38] success rates drops we simply like [13:41] trigger self-healing process but this [13:43] time we are using not the initial uh uh [13:48] documents to generate these instructions [13:50] but once uh they were failing on and [13:54] after whole process We simply test it [13:56] out. If the success rate like increased, [13:59] we simply replace it uh in a infra. And [14:03] what's like cool about it is that [14:05] clients don't have I mean they don't [14:08] even notice it, but the success rates uh [14:11] like goes up. So yeah, [14:15] >> what are your thoughts on on trusting [14:16] this new code? It's [14:18] >> a very common question we get we get [14:21] asked. [14:22] So human in the loop is critical. Every [14:25] sea level we've talked to, they want [14:27] their thememes to be in the in the loop [14:29] because this is a critical part of the [14:31] process. So whether it is validated the [14:33] requirements, we validate, we tell you [14:35] what is wrong, what is not wrong, but [14:36] then the output screen has an a human [14:40] review process. human can override the [14:43] results of of a bot uh of a model and [14:47] then once it overrides that goes into [14:49] the process of autolearning the bot so [14:51] that it doesn't do the same mistake [14:52] again that's one aspect that's the [14:54] process aspect human in the loop from a [14:57] technology perspective we also test AI [14:59] models so every time we use an AI model [15:02] whether it is a machine learning based [15:05] or large language based we publish uh [15:08] scores relevancy scores hallucinations [15:11] scores uh whether whether there's [15:13] toxicity uh traditional models like we [15:16] we we publish rogue scores, blue scores, [15:19] we test our results for bias tests. So [15:22] we test we publish that at every step so [15:26] that uh as part of a review process they [15:29] can evaluate [15:31] and um I think you talked about uh model [15:34] graded evaluation. So we use this [15:36] concept of insemble of algorithms. We [15:39] constantly uh use multiple LLMs or uh u [15:44] ML models to to create the same output [15:47] but then have a review process as to the [15:50] current kind of data is it better suited [15:52] for this versus this versus this and [15:54] then the system has that intelligence to [15:56] switch. So that's the other aspect. [16:01] >> Thank you. Um, so we also hear AI [16:05] promises to boost developer [16:06] productivity, whether it's reshaping [16:08] roles and workflows or um, you know, [16:12] where have you guys seen the biggest [16:14] productivity gains? Uh, are there places [16:16] where AI has introduced new bottlenecks [16:18] or new friction? Um, you want to lead us [16:20] off here? [16:21] >> Yes, thank you Bryce. So AI does [16:24] represent [16:26] a seismic shift in terms of how we [16:29] design, how we develop, how we deploy [16:31] and how we support software [16:34] and and I think it's an exciting time [16:36] for software development. And there are [16:39] three points I wanted to touch on which [16:41] are the first is that it's redefining [16:44] roles and it redefining the role of a [16:46] software developer. If we think about [16:48] what a software developer does, a [16:51] software developer ideates with UX and [16:53] product, helps design codes, will go [16:58] ahead and then validate the solution, [17:01] deploy the solution, support the [17:03] solution through through customer uh [17:06] incidents if if need be. The other thing [17:09] uh developers also have to do is [17:11] documentation and be able to go ahead [17:13] and do knowledge transfer um upgrade [17:16] software the routine and mundane tasks [17:19] that that need to be done that were [17:20] discussed previously [17:23] and I think AI has has can help us and [17:26] we've used AI for all those different [17:29] use cases using a variety of different [17:30] tools and so that's important because it [17:33] enables developers to be able to focus [17:36] on creative use their creativity, their [17:39] imagination, and their empathy to be [17:41] able to develop solutions for our [17:44] customers. That's really important. The [17:46] second as the second uh comment I have [17:48] on it, it's it's about acquiring new [17:51] skills, right? And if you think about [17:54] developers, we've been trained for as [17:57] computer scientists to communicate with [17:59] a machine using languages such as Python [18:03] and C++ and Java and shell scripts and [18:06] GP and reg regular expressions etc. [18:09] Right? And then in this new model, [18:13] we're having to deal with context [18:15] engineering where we've got to manage [18:16] short-term memory, long-term memory, [18:18] tool usage via via MCP uh integration, [18:23] uh integration with in terms of system [18:26] prompts and user prompts and all of that [18:29] needs to be presented as information to [18:31] a contact as context to an LLM to [18:34] perform a defined task for us. In the [18:37] former example where it's very precise [18:39] and and exact and syntax based, it's [18:43] rather deterministic. In this new model, [18:45] it's rather it's not as deterministic, [18:48] right? And so it's going to require a [18:51] change in terms of how we as software [18:54] engineers operate in this new world. So [18:58] that's the second skill. The last one I [18:59] think which is exciting is that AI tools [19:02] have helped democratize the building of [19:05] software. We're able to now, for [19:07] example, have our non- tech partners be [19:10] able to participate in the software [19:13] development process. And that's exciting [19:16] because it brings up a broader set of [19:19] voices and a broader set of great ideas [19:21] to be able to help us develop software. [19:24] And so I think those three areas are [19:28] important and have been impactful in us [19:30] in terms of being able to uh impact [19:33] development productivity. [19:34] >> I absolutely agree. U Sento thoughts on [19:37] this. [19:38] >> So I I think the the future the future [19:40] will evolve into a state where software [19:42] engineers become managers of agents and [19:44] I think that's already happening. Right. [19:46] Right. like there is these agents to do [19:48] PR reviews and things but I think the [19:49] whole product development life cycle [19:51] will become an a agent-driven uh life [19:54] cycle like for example there's a user [19:56] story that's coming we are going to tag [19:58] or trigger a architect agent which is [20:00] going to come and implement a plan for [20:01] it and then there will be an implement [20:03] agent which takes that plan and [20:05] implements the code and there will be a [20:06] tester agent which tests and verify [20:08] whether the functionality is happening [20:09] and lastly there will be a deploy agent [20:11] like a smart CI/CD which deploys it to [20:13] production um so in this case the human [20:15] software engineer becomes the [20:16] orchestrator. So they make sure that all [20:18] of them are doing the thing that they [20:19] supposed to do and that the outputs are [20:21] doing what it is supposed to do and they [20:23] also evaluate what good is and they [20:25] fine-tune the system they provide the [20:26] right right context provide the right [20:28] tooling and this transition will [20:30] introduce a new kind of productivity um [20:33] right because here we are not authoring [20:34] software like every line of software [20:35] rather we are the ones who are steering [20:37] the system fixing edge cases and [20:39] optimizing the loop and that's [20:40] altogether a different world and [20:42] secondly I also think uh the two two [20:45] skills that I I I I I believe are [20:47] critical going forward is one you should [20:49] deeply understand how things work. Only [20:51] people who deeply understand how things [20:52] work can put all these fixes, the these [20:55] pieces together and develop a system or [20:57] develop a product that adds value to [20:58] your customer. Right? Yeah, AI may take [21:00] you to 90% but the remaining 10% is [21:02] where you get 100x leverage and people [21:04] who do that 10% are the ones who are [21:06] going to differentiate themselves. And [21:08] secondly, um you should also able to [21:11] communicate your thoughts with absolute [21:12] clarity and eloquence. Um I had an [21:14] example recently. We had a hackathon in [21:16] our company and it's an AI hackathon. [21:18] very similar to a lot of people in in [21:19] Silicon Valley do it where you need to [21:21] author software from AI and not human [21:24] authored software. Um so we generally [21:26] thought that okay now everybody has [21:28] access to these tools these powerful [21:30] tools. Um so experience or skill does [21:32] not matter because anybody can win the [21:34] hackathon. But apparently that did not [21:36] happen and the people who won the [21:38] hackathon were the best software [21:39] engineers and the best theme subject [21:40] matter expertise. Why? Because they [21:42] really understood the domain and they [21:44] were able to communicate very clearly [21:45] and that combo made made the difference [21:48] right yes it it democratizes things [21:50] everybody can become good but the delta [21:52] has such a massive leverage going [21:54] forward. Just to build on the last point [21:57] that's so true 10% of my engineering [21:59] team were freshers fresh out of college [22:03] um everyone has access to AI tools they [22:06] also had and we started seeing that we [22:09] have bugs bugs are getting close four [22:12] more are getting created why because [22:15] they were so dependent on AI their [22:17] debugging skills were really really not [22:20] up to the mark so they were using AI for [22:23] everything they didn't understand the [22:25] code enough. So if the AI said this is [22:27] the fix, they'll just make it without [22:29] understanding the impact. [22:32] So in our case, we saw definitely a [22:34] productivity gain, but only with senior [22:37] folks. Junior folks struggled with use [22:41] of AI because they didn't understand the [22:43] output clearly to use it in a right way. [22:47] >> Yeah. And I would like to add uh that [22:50] what we see is uh less technical people [22:54] uh specifically like uh product [22:56] managers, team leads are also trying to [22:59] use that which is good because I mean it [23:01] reduces uh the amount of ad hoc tasks [23:04] that uh like data analysts uh engineers [23:06] uh gets. Uh but what they tend to do [23:09] they tend to like overestimate uh LLMs [23:13] and often often they come like uh with [23:16] new ideas that what they just generated [23:19] with LLM can become a new feature or a [23:21] product and uh yeah we as uh like [23:24] engineers and developers have to like [23:27] prove to them uh that yeah it will be [23:30] hard in the future if we are thinking [23:32] for instance about large scale or [23:33] something. So yeah, that's like uh a [23:36] struggle uh to do that. [23:38] >> Absolutely. [23:40] >> You get uh creative citizen developers [23:42] who don't maybe have a dedicated dev [23:44] team to help them with their product and [23:45] they can kind of get it going, but yeah, [23:47] it does introduce new risks and and [23:49] strains on the actual dev team to [23:51] support it. [23:53] Um so adopting AI and engineering isn't [23:56] exactly just plug-and-play. What's the [23:58] biggest challenge for your organization [23:59] or your customers in integrating these [24:01] AI powered tools into existing [24:03] development workflows? Uh your tech [24:05] stacks or even your team cultures. [24:09] >> Thank you. Um, so a number of [24:12] organizations and I've talked with a [24:14] number of you this week are beginning [24:16] your AI journey and [24:20] facing the same challenge that we did [24:22] when we began ours, which is answering [24:24] the question, how do we successfully [24:27] enable AI into our organization to [24:31] better position our business for the [24:33] future? And um, you know, there are [24:35] three learnings I would sh I would [24:37] share. The first is that executive [24:40] support is needed in terms of we had our [24:44] audible CEO [24:46] commit and encourage and promote [24:49] upskilling and retraining and created [24:53] learning and development programs for a [24:55] crossf functional team not only for the [24:57] technology organization and also our [25:00] audible went ahead and supported [25:02] reimbursing of training programs for AI. [25:07] So that was important from a from a [25:09] technical leadership perspective. We [25:12] went ahead and encouraged science fairs [25:14] in terms of sharing innovations in terms [25:16] of sharing use of AI tools that were [25:19] impactful across the organization. And [25:21] so that's been positive. Second uh [25:25] learning is that we promoted organic [25:29] learning and curiosity. One of Audible's [25:31] people's principles is study and draw [25:33] inspiration from culture and technology. [25:36] And this was a wonderful example of just [25:38] that. We did not prescribe or mandate [25:40] the use of a specific tool. We [25:43] encouraged the team to select an [25:45] appropriate tool to solve an opportunity [25:47] and address an opportunity identified by [25:49] a team. In retrospect, it was a great [25:51] call because of the number of new tools [25:53] that have come out in the last few [25:55] months. Even uh we had Kira come out [25:58] last month and uh also the maturity [26:02] curve of different AI tools is changing. [26:06] So when you evaluate an AI tool in [26:09] January, you may want to revisit it a [26:12] few months later and I see a lot of nods [26:13] because they learn and get better, [26:16] right? Um for example, right now a lot [26:19] of our developers and and myself [26:21] included, we use QCLI which is great. [26:23] It's been very impactful for us. So [26:25] that's the second uh learning and then [26:28] the third learning I would share would [26:30] be that peer learning is extremely [26:33] important. Uh we have an organic [26:36] session. It's set up by the by the team [26:39] every week where it's a learning. We [26:41] have different presenters. So we don't [26:42] necessarily just have one guru or a few [26:45] AI developers. We have different [26:47] presenters get up and we spend an hour. [26:50] We open up our laptops. we set up our [26:53] the configuration required for a tool [26:55] whether it's MCP tools or others and we [26:58] go ahead and we learn how to solve a [27:00] familiar problem uh using an AI tool and [27:04] and our team have found that it's that [27:06] it's extremely valuable to learn from a [27:08] fellow colleague about how to solve a [27:10] familiar problem in a faster and easier [27:13] way using an AI tool and so that's been [27:16] that's been important and by teaching we [27:19] learn [27:20] >> absolutely So I'll add on this. So from [27:23] two perspective one um as the end user [27:26] the employee [27:28] um acceptance and adoption two aspects a [27:32] lot of employees are worried that their [27:34] job is at stake they don't trust AI it's [27:37] the same thing when automation first [27:39] came right people were worried so [27:41] helping them understand this is not [27:43] about AI replacing this is about in AI [27:45] enhancing their role they don't have to [27:48] do I think central used the word mundane [27:50] work they can do intelligent work [27:52] So first is helping them understand, [27:55] accept and then adopt. Adoption follows [27:58] the same principles, right? [27:59] Certification, curriculums, giving them [28:02] access to learn is one aspect. U now [28:05] when the employees are ready, the [28:07] company's ready, then what are the [28:09] hurdles there? [28:11] Which AI, which to which LLM to use? [28:16] um whether it'll be hosted, whether [28:19] it'll be SASbased on-prem, those are the [28:22] hurdles you have to go through. There's [28:23] an architectural review board. Um what [28:26] happens to data? What happens to IP? [28:27] What happens to using AI to code, but [28:30] then you have copied code from somewhere [28:32] where you're not supposed to copy, [28:34] right? They have as an organization, you [28:36] have to worry about that. Second is [28:38] security. Am I uploading something into [28:43] chat GPT because people have access [28:44] where I'm going to lose IP? I'm going to [28:46] lose access to stuff which I normally [28:49] want to protect. So companies have to [28:52] answer these architectural and security [28:54] aspects. [28:56] So we see we help a lot of companies [28:58] adopt AI through tools, through services [29:00] and so on. But these are some of the [29:02] common challenges I see them struggle [29:04] with. [29:06] I think it really helps um with the [29:08] adoption and and usage coming from the [29:11] top down. I've I've met with a lot of uh [29:14] developers and uh individual [29:16] contributors who feel like at times [29:18] using AI is is cheating to do their job. [29:21] Um so that that coming from the top down [29:23] really helps them feel okay with with [29:25] using it to empower their work. [29:29] Uh so looking forward what's one trend [29:31] in AI powered development that you're [29:33] personally excited about or just curious [29:35] about Sento? Yeah. Uh I think I'm [29:38] genuinely interested in the fact that [29:40] I'm able to explore the opportunity [29:42] space very rapidly especially if I'm [29:44] making one way door decisions right u [29:46] and there's this thing in desian theory [29:47] which states that if uncertainty [29:49] increases the value of deferring work [29:51] also increases but there's a cost to it [29:52] which is opportunity cost so to give an [29:54] example right well let's say in the past [29:56] like a PM has 10 a product manager has [29:58] 10 ideas or a architect has wants to try [30:01] five frameworks they have to choose one [30:03] because validating each took time for [30:05] for a for a PM they need to wait on [30:08] analysts, they need to do some deep [30:09] research. They have to wait for [30:10] designers to create prototypes um and [30:12] all those things. So exploring the space [30:13] was super expensive. But now everything [30:15] is a prompt away, right? If you want to [30:17] try and validate ideas, you can quickly [30:19] create prototypes and validate them and [30:21] choose the one that you really are going [30:23] to put it in production. Valid engineers [30:24] come in and do the real work, right? Uh [30:26] so that that's super. So which means [30:28] what opportunity cost is collapsing, [30:29] right? It's trending to zero. The second [30:31] thing I'm very excited is also the the [30:34] fact that uh I believe that human [30:36] variance will be the differentiator in [30:37] the future. See, this is one of those [30:39] few paradigm shifts where everybody in [30:41] the world has access to the same [30:43] frontier models on at the same time, [30:44] right? You can pay 20 bucks a month or [30:46] $200 a month and get access to the most [30:48] powerful model in the world that Fortune [30:50] 500 companies also have. Which means [30:53] that if you just use the model out of [30:54] the box very naively, you are going to [30:56] get baseline results, right? An average [30:58] result. So, you regress to the mean. So, [31:00] but if you use your creativity to [31:02] distinguish yourself and you create the [31:04] context that's well refined uh that has [31:07] true purpose that emotionally connects [31:08] with your product that's where the magic [31:10] happens. I think that I I I truly [31:12] believe in that sense that where humans [31:14] will add the value where they they come [31:15] and close the gap. [31:18] So I think um if you look at the journey [31:20] right you had traditional ML algorithms [31:23] which were mostly used for analysis and [31:26] prediction. Then came generative AI [31:29] which was more about generating stuff [31:31] and the next wave is going to be action [31:32] AI right agentic AI is a step in the [31:35] right direction I'm very excited about [31:37] how will agentic AI change how we do [31:40] engineering in general testing in [31:43] general uh operations in general right [31:45] imagine um we have a platform called [31:48] coco copilots of copilot in there we are [31:50] building this whole agentic architecture [31:52] where a user can just come in and say [31:55] just take the latest build and test it [31:58] uh to make sure that there is no buck [32:00] leakage. That's all he's saying. He's [32:02] not saying go create test scenarios, [32:04] test cases. Uh I want security testing. [32:06] I want performance testing. He's going [32:09] to share his intent. That intent I think [32:13] someone central used the word agents. th [32:16] those agent those intent will be broken [32:18] down into a set of actions which will be [32:21] uh passed on to these individual agents [32:24] do the job and then within that time [32:27] period of testing complete the so I [32:29] think that's that's that's something I'm [32:31] very very excited about the agentic AI [32:34] what will it drive from a use case [32:36] perspective [32:37] >> yeah I know a lot of engineers and uh [32:39] developers are are very inquisitive by [32:41] nature and I get a lot of value out of [32:42] deep research um helping me learn new [32:45] new technologies, new concept. Um, and [32:48] because AI can meet you where you are in [32:50] your understanding of a concept. It's [32:51] it's going to make a lot of changes to [32:52] education and how people learn new [32:55] technologies. Um, I'm excited about that [32:57] direction of it. Um, how about the topic [33:00] of of reshaping new roles or we [33:03] mentioned new skills like context [33:04] engineering. What do you guys you have [33:06] any thoughts on these new roles and new [33:07] skills? [33:09] I think you know for example we didn't [33:12] have prompt engineering roles or new [33:15] roles six months ago did we Bryce right [33:18] and I think you go look around now I [33:20] think this new this new um AI reshaping [33:26] of roles is is creating new roles uh to [33:30] begin with and I think I'm also excited [33:31] about the fact that we'll be able to [33:34] enable human beings smart smart folks [33:37] like in this room to be able to focus on [33:40] creativity and using empathy to be able [33:43] to create better products by getting [33:45] away from the routine and mundane tasks [33:48] whether it be adjusting alarms or [33:51] dealing with logs or creating [33:53] documentation. I think all of that could [33:55] be done faster. I think one of the [33:57] aspects that hasn't been discussed is [34:00] a reframing of computer science [34:03] education to be able to leverage these [34:06] new skill sets and new tools and to be a [34:09] because right now if you think about [34:11] coding when computer science grads come [34:14] out the focus is on coding but there are [34:16] much more broader [34:19] capabilities that are required to be [34:21] able to be effective in this new world [34:23] and I'm excited about that. [34:25] So I think um roles diff like prompt [34:28] engineers would definitely come up but [34:31] the existing roles the scope of what [34:33] they do also will change. I was in [34:36] conference where we were talking about [34:38] how marketing has changed with AI. [34:41] How buyers are using chat GPT to make [34:44] decisions on what is good, what is bad. [34:49] 10 years back automation engineer from a [34:51] testing world was a new thing. Six, [34:54] seven years back sats [34:57] right software design engineer in test [34:59] was a new thing. Today in our company [35:02] we've introduced this role of DSIT which [35:04] is data science engineer in test. [35:08] Even when you're doing testing you have [35:09] to have AI knowledge [35:12] because guess what you're not testing [35:15] plain vanilla applications anymore. [35:17] You're testing AI infused applications. [35:19] Why does it matter? A banking [35:21] application you put a loan application [35:22] you get approved or you don't get [35:24] approved. That's it. One or zero. [35:27] Apple Amazon shopping card. You go in [35:30] the morning, it gives you a different [35:32] recommendation. If your wife is using [35:34] your same login and what god knows what [35:36] she's bought in the evening, it gives a [35:38] total different recommendation. AI [35:40] infused apps have nondeterministic [35:43] output. How do you test for that? [35:46] Concept of bias testing didn't even [35:48] exist. How do you teach normal [35:51] automation engineers those concepts? [35:53] That's why new roles will come in. I [35:56] think often too um there's a lot of [35:59] confusion between AI agentical workflows [36:01] and a deterministic workflow. They often [36:03] get uh interchanged when they're pretty [36:05] different. [36:07] >> Yeah, I think that that to follow up on [36:08] that right there. Exactly. Both of them [36:10] are right. The future will also be two [36:12] sections, right? The deterministic part [36:13] and the nondeterministic part. [36:14] Nondeterministic part is the AI [36:16] generated by the AI and the [36:17] deterministic part is the part that [36:19] human oversight and make sure it's [36:21] doing. And even deterministic software [36:22] will manifest in many ways. uh it can [36:24] manifest in a in a traditional [36:26] rule-based object to programming right [36:27] it can ensure authenticity uh it ensures [36:30] roll back authorization right it makes [36:31] sure user A doesn't see user B's content [36:33] it'll keep it'll keep your secrets uh [36:35] safe because LM are not good in keeping [36:37] secrets so these are the things where a [36:39] deterministic uh nature of system is [36:41] needed and eval right we talked a little [36:43] bit about evals I think high quality [36:45] wells evaluations are the ones which [36:48] will keep your AI from becoming a [36:49] liability right or else you you are [36:52] highly liable in these in these [36:53] situations. So that's where that's the [36:55] role that's going to come up where [36:57] humans provide that touch point on [36:58] making sure that I mean it's getting [37:00] better day by day, right? GPD5 has the [37:01] least hallucination score at this point [37:03] in time. Uh but also your baseline keeps [37:05] improving. That's why I keep going back [37:06] the thing, right? Because everybody has [37:08] access to this now. So so that that's [37:10] where the human comes into play. [37:12] >> Yeah. And just a quick thought, I think [37:14] not talking about the new roles, but the [37:16] current roles, I think that AI uh lets [37:20] you to like switch roles faster. I would [37:22] say for instance if you're like a [37:24] back-end developer it's like a lot [37:26] easier now to be become a full stack [37:29] developer and for instance now um just [37:32] recently I was able to like build a [37:35] front end for a new platform of ours AI [37:39] studio you could check it out it looks [37:41] really nice but I had zero experience in [37:43] front-end developer development and uh [37:46] yeah so it's like exciting uh to see [37:50] >> but if you had a role of just taking [37:52] minute uh meeting minutes that role is [37:54] going away. I can tell you that [37:56] >> like Microsoft Teams co-pilot does a [37:59] fabulous job, right? Using AI to capture [38:02] the meeting minutes, the action items, [38:04] even assigns if required. [38:07] >> There's going to be a lot of change in a [38:08] lot of roles. Um shifting and and we got [38:11] to be able to be agile and move with [38:12] these changes. Um I I feel like my you [38:16] know role as a software developer is [38:18] shifting more towards the role of a [38:21] product engineer. um because these [38:23] language models are getting so much [38:25] better, so much quicker that they are [38:27] going to be the best at writing syntax [38:29] and writing syntax is going to be a [38:31] smaller portion of my job um versus [38:33] architecting a scalable, sustainable, [38:36] secure solution. [38:38] >> Um so, um how are new engineers that are [38:41] going to be, you know, like new grads [38:42] when they're coming in, what's the [38:44] pathway for them? Like, you know, where [38:46] do you see them that they need to be [38:48] trained on when when we're onboarding [38:50] them? [38:52] is in final year college [38:54] and um [38:57] I remember when I was growing up and I [38:59] was learning data structures and [39:00] everything you learn and then there's a [39:02] learning period and then you start [39:04] implementing stuff for him probably it's [39:07] 5 seconds 10 seconds I gave him some [39:10] task and 10 minutes later he has [39:12] completed the task so they're going to [39:15] use AI [39:17] but if I were to be the mentor of those [39:21] young graduates I would tell them use AI [39:25] only for the right areas don't use AI to [39:27] do basic stuff which like basic coding [39:31] learn the basics first and once you [39:33] learn that that otherwise you're too [39:35] dependent on AI the same son he [39:39] struggled when he had to debug because [39:41] they have they don't teach you how you [39:43] enable logs you debug you go step by [39:45] step they depend a lot on AI to do that [39:49] job So just for them to learn I would [39:53] say do it the traditional way once you [39:55] become comfortable start using AI and [39:58] that's where productivity would come in [40:00] >> spot on I think you should be deep like [40:02] as I already mentioned to right you [40:03] should clearly understand what you're [40:04] doing if you don't understand it it's [40:06] not going to make a difference right [40:07] that's I mean I I do the same to a lot [40:09] of advice to kids is right be like [40:11] understand everything right do the [40:12] foundation right if you're into physics [40:14] read Richard Fain man like all the [40:16] things are needed and that's not going [40:18] to change at all you need to understand [40:19] differential calculus, you need to do [40:20] all the basic stuff or else you will I [40:22] mean these things will help you but [40:24] you're not going to be a differentiator [40:26] at all. [40:27] >> Yeah, we tried um we we ran um we had [40:29] interns recently join our teams and um [40:34] in order for them to come up to speed [40:35] and learn about the system and learn [40:37] about the code, we had them use Gen AI [40:41] in their first couple of weeks to be [40:43] able to do that. And they found that [40:45] very productive by the way to be able to [40:47] use Gen AI to understand the code so [40:50] that they enhance their understanding [40:51] right and also what they did is they [40:54] helped contribute new test cases and [40:56] they helped contribute to the to the [40:58] codebase and the documentation which was [41:01] a win-wall. So bringing in these new [41:05] grads and who have this fresh [41:07] perspective, they can leverage their [41:09] expertise and we can use Gen AI to [41:12] leverage their creativity, [41:14] enhance our test suites and enhance our [41:17] codebase, but still allowing them to [41:19] come up to speed more quickly, I think [41:20] is a win for everyone. Yeah, our summer [41:23] interns that come in are some of the [41:25] most inquisitive bunch and and our [41:27] metrics on our enterprise AI platform [41:29] will see the the chatbot usage just [41:31] skyrocketing like well there's the [41:32] intern effect. [41:33] >> Exactly. And [41:34] >> I think you need to foster that that [41:35] that young innovation and challenge [41:38] because you know these these systems [41:40] these AI models at sometimes people will [41:43] think they perform at a junior or fresh [41:45] developer level. But um if you don't [41:47] keep that influx you're just going to [41:49] have senior developers. They're going to [41:50] retire and you're going to be in a bind. [41:52] >> Uh just a quick question the last year. [41:54] What do you think would be the top two [41:55] tools for each of you that have impacted [41:57] your dev team's production? I would say [42:00] cloud code um and client [42:03] >> I would say uh Q developer uh QCLI and [42:07] client [42:09] >> I would say uh cursor and cloud code. [42:12] Yeah. [42:12] >> Yeah. Cursor and cloud code for us too. [42:15] >> Klein specifically with memory bank. If [42:18] you're not using memory bank with client [42:19] look it up. [42:22] >> Hi. Uh good afternoon everyone. Yeah. So [42:24] you're broadly covered on the software [42:26] development aspects right. So but uh our [42:28] engineers mostly uh spend time on [42:31] production support. So currently are you [42:34] using any tools AI tools to debug the [42:37] logs? So most of the time spent on [42:40] debugging the logs and identifying the [42:42] root cause of the issue and fixing it [42:44] right. So are you using any tools [42:47] currently or else like are you exploring [42:48] any kind of agentic implementation uh to [42:51] identify the logs and trouble [42:53] troubleshoot the issue? Thank you. [42:54] >> I would say um yes we are. Are you uh so [42:57] to reframe your question, you're [42:58] referring to production support in terms [43:00] of debugging production support, [43:02] understanding what the the different [43:04] logs, looking at the metrics and alarms [43:06] and being able to summarize the [43:07] incidents. And we are using AI tools to [43:09] do that. Uh to be able to both define [43:13] what the problem is, gather the [43:14] appropriate information from logs and be [43:16] able to present the on call with here's [43:19] when the last deployment happened. It [43:21] coincides with the when the production [43:24] issue occurred. We do have uh we have [43:27] used tools for that and we've also used [43:29] AI tools to be able to help us identify [43:32] systemic issues underlying different [43:34] tickets. So because that's also takes [43:37] time in terms of understanding what are [43:38] those different underlying production [43:40] issues whether it be a specific [43:42] deployment or a um a memory constraint [43:46] that we're having to deal with. So [43:49] overall, we've used it for production uh [43:51] issues and also for understanding root [43:54] causes that have contributed to those [43:56] production issues and including [43:58] adjusting alarms and thresholds too. [44:00] >> I'm just curious if any of you have any [44:02] experience on using these tools on air [44:05] gap networks and where you see the [44:07] future of that on using AI tools and [44:10] secure networks. [44:12] >> This this comes up all the time in [44:14] healthcare customers and banking [44:16] customers. [44:18] Um, so you don't have to use a hosted [44:21] LLM all the time. You can use something [44:24] like Olama. [44:25] We've used that consistently. It's [44:28] getting better week by week. We've used [44:30] it. Um, uh, so depending on the use [44:34] case, there are plenty of other options, [44:36] right? Hugging face and others which you [44:39] could use. So yes, the answer is uh for [44:42] air gap networks where you can't go out [44:45] um uh you can use local hosted LMS there [44:49] and [44:49] >> the same applies to low like [44:51] requirements where latency becomes key [44:53] right um so we we also do that uh same [44:56] setting where uh we want to host it on [44:57] oursel uh where latency is super [44:59] critical I mean in our scenario like [45:01] auctioning and things like that where [45:02] every millisecond matters uh going over [45:04] the network and doing stuff it's not uh [45:07] it's not possible Right. [45:10] I think that's it. Thank you guys. [45:11] >> Thank you everyone.