Airbyte Uses Autonomous AI Agents to Fix Broken Data Connectors
Ai4
Building production trust in autonomous AI agents requires strict validation loops, declarative code limits, and geofenced progressive rollouts rather than human-in-the-loop prompt engineering.
## Background
Published by Ai4 on YouTube on August 16, 2026, this presentation features Airbyte Senior Software Engineer Pedro Lopez detailing how the company maintains 600+ data connectors. The session covers how Airbyte empowers AI agents to autonomously detect, diagnose, and repair production breakages across thousands of live customer pipelines.
## TL;DR
Maintaining hundreds of integrations against third-party API changes requires shifting focus from AI code generation to building strict validation and trust systems. Airbyte uses continuous triggers and a Triage Agent to diagnose issues, passing hypotheses to a Fix Agent constrained by a declarative schema framework. The pipeline enforces safety using Proof-Fix Agents, automated reviewers, system evals, and progressive rollouts to safely automerge low-risk patches without human intervention.
## Atmosphere & style
The talk is technical, pragmatic, and grounded in real-world engineering realities rather than AI hype. Lopez presents a clear, authoritative narrative that draws insightful parallels between software autonomy and autonomous vehicles. The session feels like an authentic insider look at moving agents from fragile demos into production systems where failure is not an option.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) Airbyte maintains over 600 connectors that frequently break due to unannounced upstream API updates, deprecations, and schema changes.
- [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) The primary barrier to autonomous AI engineering isn't code generation, but establishing trust that automated fixes won't break existing customer pipelines.
- [05:07](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=307s) Self-healing architectures rely on designing automated system loops that iterate on fixes using CI feedback rather than using manual direct prompting.
- [05:06](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=306s) Dual incoming signals—changelog monitoring and production error alerts—trigger a Triage Agent to analyze logs and build root-cause hypotheses for the Fix Agent.
- [05:06](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=306s) Constraining agent output using a standardized, declarative connector framework prevents code sprawl and unmaintainable fixes.
- [08:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=500s) A Proof-Fix Agent verifies that the pre-patch version fails and the post-patch version succeeds before code is passed to automated reviewers and system evals.
- [08:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=500s) System evaluation requires both deterministic assertion test suites and non-deterministic LLM-as-a-Judge review stages to ensure architectural reliability.
- [11:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=680s) Autonomy is granted progressively using geofencing, automerging low-risk changes like security updates while keeping humans in the loop for complex refactors.
- [11:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=680s) Progressive deployment incrementally rolls out connector patches to customers, automatically pausing and re-triggering triage loops if regressions are detected.
- [17:59](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=1079s) Triage agent hallucinations are mitigated by providing rich production log access and allowing downstream Fix Agents to dynamically adapt hypotheses during execution.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) Maintenance Burden of 600+ Connectors
Pedro Lopez, Senior Software Engineer at Airbyte, opens the talk by setting the stage on how open data infrastructure works and why connector maintenance is such a massive operational headache. Airbyte helps data teams connect all their disparate data sources to feed both traditional analytics pipelines and, increasingly, AI agents. However, because this data resides in third-party SaaS platforms outside Airbyte's control (such as Salesforce, HubSpot, and Marketo), maintaining over 600 connectors is like tending to a living organism. Static integrations simply do not exist because upstream vendors frequently push breaking API updates, deprecate endpoints, or alter schemas without prior announcement.
When attempting to solve this with AI, Lopez notes that while AI agents are generally capable of producing code solutions given the right context, the primary obstacle is **trust**. The core question isn't whether an agent can write a patch, but how engineers can trust that the fix is correct and won't break existing functionality across thousands of customer pipelines.
- Airbyte maintains over 600+ connectors in its ecosystem, with the number constantly growing.
- Upstream vendors (e.g., Salesforce, HubSpot, Marketo) frequently introduce unannounced breaking API changes and endpoint deprecations.
- The fundamental engineering challenge shifts from AI code generation to building validation systems that grant trust and confidence.
---
[00:307] Designing Agent Loops Over Direct Prompting
Lopez highlights a viral tweet by Peter Steinberger (creator of OpenClaw) that garnered 8.5 million views and 20k likes, which declared that engineers should stop prompting coding agents directly and instead design **loops** that prompt the agents. Instead of having a human engineer manually guide an AI agent through iterative bug-fixing, loop engineering constructs an automated system where the agent iterates on its own work based on feedback, validation checks, and strict constraints.
- **Loop Philosophy:** Replaces manual human-in-the-loop prompting with automated system loops.
- **Initial Baseline Validation:** The simplest loop triggers an agent fix, runs CI (linting and test suites), and routes failed builds directly back to the agent for auto-correction before marking a PR as ready.
---
[05:06](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=306s) Triggers, Triage, and Architectural Constraints
To run autonomously without human intervention, the self-healing pipeline relies on dual incoming signals. The first trigger comes from documentation and changelog monitors that scan vendor APIs for changes. The second comes from production monitoring alerts that capture unannounced breaking failures. When a signal arrives, it is processed by a **Triage Agent** (a planner/explorer) equipped with log-access tools and debugging skills. This agent gathers evidence and establishes a root-cause hypothesis, which serves as the direct prompt for the downstream **Fix Agent**.
To limit "code slob" and unexpected side effects, Airbyte leverages its standardized, declarative connector framework. By constraining the output space to predictable building blocks, the agent simply configures predefined components rather than writing arbitrary, chaotic code.
- **Dual Triggers:** Ingests documentation/changelog updates and real-time production failure logs.
- **Triage Agent:** Analyzes system logs and gathers evidence to generate a root-cause hypothesis.
- **Declarative Constraints:** Restricting code generation to a standardized connector schema limits output variability and prevents code sprawl.
---
[08:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=500s) The Validation Chain: Proof-Fix, Review, and Evals
Once the Fix Agent generates a pull request, Airbyte subjects the code to a strict chain of automated validators before considering it safe. First, a **Proof-Fix Agent** executes a test harness comparing the old connector version against the new patch; it verifies that the old version fails as expected and the new version succeeds. Next, an **Agent Reviewer** checks the code for test coverage, hygiene standards, and backward compatibility. Any validation failure feeds directly back into the Fix Agent loop.
To validate the agents themselves, Airbyte runs continuous **Evals**. These include deterministic assertion test cases on agent outputs as well as non-deterministic LLM-as-a-Judge evaluation stages.
- **Proof-Fix Agent:** Compares pre-patch and post-patch executions in a test harness to confirm the precise bug is resolved.
- **Agent Review:** Enforces repository style guidelines, test coverage thresholds, and backward compatibility.
- **System Evals:** Uses continuous assertion suites and LLM-driven judges to keep the agentic architecture on track.
---
[11:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=680s) Earning Autonomy: Geofencing and Progressive Rollouts
Drawing a parallel to the autonomous vehicle industry (referencing Waymo in San Francisco and Zoox in Las Vegas), Lopez notes that autonomy must be *earned* progressively across standardized levels (0 to 5). Software teams should not merge every AI-generated PR instantly; instead, they should establish **geofences** (constrained operational scopes) where autonomous merging is permitted without human intervention.
```
[Level 0: Manual] -> [Level 1: AI Assist] -> [Level 2/3: Geofenced] -> [Level 5: Full]
|
+---------------+---------------+
v v
Automerge Scope Human Review
- Vulnerability Bumps - Complex Schema Shifts
- Safe Endpoint Additions - High-Risk Refactors
```
Airbyte currently operates between Levels 2 and 3, automatically merging updates within low-risk geofences—such as security dependency bumps and minor endpoint additions—while requiring human approval for higher-risk changes. Furthermore, post-merge execution utilizes **progressive rollouts**, gradually pushing updates to small user percentages and routing real-time failure signals back into the triage loop if regressions occur.
- **Automotive Analogy:** Compares agent rollout stages to SAE autonomous driving levels (0 to 5) and localized testing geofences.
- **Current Airbyte Scope:** Operates between Level 2 and Level 3, automerging security patches and minor feature additions while expanding boundaries based on success metrics.
- **Progressive Rollouts:** Deploys connector updates incrementally across customer pipelines, pausing and triggering self-healing loops if production regressions appear.
During the Q&A session, an audience member asks how to handle situations where a triage agent begins hallucinating or deviating on a large codebase, creating an incorrect solution spec. Lopez clarifies that the triage phase merely formulates an initial hypothesis, much like a human engineer starting a task without full visibility into every edge case.
To reduce triage errors, developers should focus on improving the agent's available context, skills, and tools—specifically exposing deep production logs and runtime telemetry. Crucially, the downstream Fix Agent is designed to evolve or discard the initial triage hypothesis as it works through the problem and encounters actual harness validations.
- Triage hypotheses are treated as flexible starting prompts rather than strict execution mandates.
- Hallucinations are mitigated by providing rich production log access and well-defined agent skills.
- The downstream Fix Agent dynamically adapts its strategy as validation feedback unveils the true state of the codebase.
## Notable quotes
> [00:00](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=0s) The hard problem turned out to be trust, not code generation. — Pedro Lopez
> [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) maintaining over 600 connectors is like tending to a living organism. — Pedro Lopez
> [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) The core question isn't whether an agent can write a patch, but how engineers can trust that the fix is correct and won't break existing functionality across thousands of customer pipelines. — Pedro Lopez
> [05:07](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=307s) engineers should stop prompting coding agents directly and instead design loops that prompt the agents. — Peter Steinberger
> [05:06](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=306s) By constraining the output space to predictable building blocks, the agent simply configures predefined components rather than writing arbitrary, chaotic code. — Pedro Lopez
> [11:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=680s) autonomy must be earned progressively across standardized levels (0 to 5). — Pedro Lopez
> [11:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=680s) Software teams should not merge every AI-generated PR instantly; instead, they should establish geofences (constrained operational scopes) where autonomous merging is permitted without human intervention. — Pedro Lopez
> [17:59](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=1079s) the triage phase merely formulates an initial hypothesis, much like a human engineer starting a task without full visibility into every edge case. — Pedro Lopez
## Mentioned
Pedro Lopez, Airbyte, Salesforce, HubSpot, Marketo, Peter Steinberger, OpenClaw, Waymo, Zoox
## Video description
Pedro Lopez - Senior Software Engineer, Airbyte
We gave AI agents the authority to fix our production connectors on their own, while thousands of customer pipelines depended on them not getting it wrong. Upstream APIs change constantly, so we built a system that keeps 600+ connectors healthy autonomously: AI agents detect an issue, diagnose it, and ship the fix automatically. The hard problem turned out to be trust, not code generation. I'll show how we evaluate whether a fix actually works, where we keep a human in the loop and where we don't, and the progressive rollout system that lets an agent's change reach production without risking the whole product. If you're moving agents from demos into production, come see what it takes to let them act autonomously and still sleep at night.
Airbyte 利用自主 AI Agent 修复故障数据连接器
在生产环境中建立对自主 AI Agent 的信任，需要严格的验证闭环、声明式代码约束以及渐进式发布，而非依赖人工介入的提示词工程。
面对第三方 API 变更维护数百个集成，需要将重心从 AI 代码生成转向构建严格的验证与信任系统。Airbyte 利用持续触发机制和分诊 Agent（Triage Agent）来诊断问题，并将假设传递给受声明式 Schema 框架约束的修复 Agent（Fix Agent）。该流水线通过验证修复 Agent（Proof-Fix Agent）、自动化审查器、系统评估和渐进式发布来确保安全性，从而在无需人工干预的情况下安全地自动合并低风险补丁。
Airbyte 维护着 600 多个连接器，这些连接器常因未提前通知的上游 API 更新、弃用和 Schema 变更而损坏。 (0:04)
自主 AI 工程的主要障碍并非代码生成，而是建立确信自动化修复不会破坏客户现有流水线的信任机制。 (0:04)
自我修复架构依赖于设计利用 CI 反馈迭代修复的自动化系统闭环，而非依赖人工直接提示。 (5:07)
更新日志监控与生产错误告警的双重输入信号，会触发分诊 Agent 分析日志并为修复 Agent 构建根本原因假设。 (5:06)
通过标准化、声明式的连接器框架约束 Agent 输出，可防止代码无序膨胀并避免产生难以维护的修复方案。 (5:06)
Reading summary
Pedro Lopez, Senior Software Engineer at Airbyte, argues that sustaining hundreds of external API integrations requires replacing direct human prompting with self-correcting agent loops, strict declarative constraints, and tiered validation chains.
The Maintenance Burden and Loop Engineering (0:04)
Maintaining Airbyte's catalog of over 600 connectors resembles tending a living organism because upstream vendors like Salesforce, HubSpot, and Marketo routinely deprecate endpoints, alter schemas, and issue unannounced breaking API changes. Lopez argues that the central bottleneck in applying AI to this maintenance workload is not code generation capability, but establishing trust that a generated patch will not compromise thousands of production pipelines.
To solve this, Airbyte adopts loop engineering—a model popularized by Peter Steinberger advocating that developers build systems that prompt agents rather than prompting them directly. Under this design, an agent receives a trigger, writes a patch, and runs CI and lint tests. Any failure is immediately routed back into the agent loop for self-correction without requiring manual human steering.
Dual Triggers and Declarative Constraints (5:06)
The self-healing architecture initiates via two distinct signals: documentation and changelog monitors that detect announced vendor changes, and production monitoring alerts that capture active runtime failures. These signals feed into a Triage Agent equipped with log-access and debugging tools, which gathers evidence to establish a root-cause hypothesis for a downstream Fix Agent. Lopez notes during Q&A that this hypothesis is treated as an adaptable starting prompt rather than a rigid mandate, allowing the Fix Agent to discard or evolve the theory as it encounters harness feedback.
To prevent unstructured code sprawl and unintended side effects, Airbyte restricts the Fix Agent's output space using its declarative connector framework. By constraining the agent to configuring standardized, predefined building blocks rather than authoring arbitrary code, the system limits behavioral unpredictability.
The Multi-Layered Validation Chain (8:20)
Airbyte subjects every candidate pull request to an automated validation sequence before code is approved. First, a Proof-Fix Agent runs a comparative test harness to verify that the unpatched connector fails as expected and the new patch resolves the specific issue. An Agent Reviewer then inspects the pull request to enforce repository hygiene, backward compatibility, and test coverage thresholds. Any validation defect routes back into the Fix Agent loop for automated revision.
To audit the agents themselves, the pipeline runs continuous system evaluations. These evals combine deterministic assertion test suites against agent outputs with non-deterministic LLM-as-a-Judge assessment layers to ensure the operational architecture remains reliable.
Geofenced Autonomy and Progressive Rollouts (11:20)
Drawing an analogy to autonomous vehicle deployments by companies such as Waymo and Zoox, Lopez asserts that software engineering autonomy must be earned gradually across levels 0 through 5. Instead of enabling full auto-merging across all scenarios, Airbyte defines operational boundaries, or 'geofences.' The system currently functions between Level 2 and Level 3 autonomy: low-risk updates, such as security vulnerability bumps and safe endpoint additions, are auto-merged, while complex schema changes and high-risk refactors still require human review.
Once merged, updates are introduced through progressive rollouts that deploy fixes incrementally across small percentages of pipelines. If runtime regressions appear, live failure telemetry routes back into the triage loop to trigger automated self-healing.
Airbyte's model demonstrates that autonomous self-healing hinges on constraining the solution space and building multi-tiered validation checks rather than relying on raw code generation. Unresolved challenges remain in handling complex, high-risk schema refactors, keeping current autonomous production merging restricted to geofenced, low-risk operational boundaries.
## Background
该视频由频道 Ai4 于 2026 年 8 月 16 日发布在 YouTube 平台上，由 Airbyte 高级软件工程师 Pedro Lopez 主讲。简介指出，本次演讲主要探讨 Airbyte 如何构建自愈系统，让 AI Agent 在保障数千条客户数据管道正常运行的前提下，自主检测、诊断并修复 600 多个连接器的生产故障，并详细介绍了建立信任、人机协同与渐进式发布等核心机制。
## TL;DR
面对 600 多个连接器频繁因上游 API 变更而损坏的痛点，Airbyte 发现代码生成并非瓶颈，建立对修复方案的“信任”才是核心难题。团队从单纯写 Prompt 转向闭环工程（Loop Engineering），构建了涵盖 Triage 分诊、Fix 修复、Proof 验证和审查的多 Agent 自主循环，并通过声明式架构收敛输出空间。此外，系统借鉴自动驾驶的“地理围栏（Geo-fence）”理念实现自主权分级，仅对安全范围内的修复执行自动合并，并结合渐进式灰度发布构建出高可靠的端到端生产自愈闭环。
## Atmosphere & style
整场演讲呈现出务实、严谨且极具一线实战价值的工程技术氛围，摒弃了空洞的 AI 概念炒作，直击模型落地生产环境时的信任与稳定性痛点。主讲人以清晰从容的逻辑和生动的自动驾驶类比展开分享，将复杂的系统设计拆解得层次分明。聆听过程既能感受到面对海量接口维护难题时的真实工程挑战，又能体会到通过严密验证环路实现系统自愈后的从容与确定性，为探索 Agent 生产级落地的开发者带来了极具启发性的实操参考。
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) 生产环境自愈的核心瓶颈在于建立信任而非代码生成能力；模型往往能生成代码，但必须有机制确保修复不破坏既有功能与向后兼容性。
- [03:07](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=187s) 从编写单次 Prompt 转向设计闭环回路（Loop Engineering）：构建能够自动捕捉报错、提供反馈并自主迭代的验证闭环，是实现软件维护自动化的关键。
- [04:59](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=299s) 通过主动监控上游变更日志与被动捕获生产告警的双轨触发机制，实现无需人工发现与介入的端到端自主响应流程。
- [04:59](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=299s) Triage Agent 利用日志访问与调试工具排查根本原因，并生成附带完整证据链的诊断报告，作为下游修复步骤的输入。
- [06:20](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=380s) 采用声明式构建块（Declarative Building Blocks）标准化代码架构，将 Agent 的解空间收敛为标准组件装配，从根本上防止垃圾代码与逻辑失控。
- [08:14](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=494s) 引入 Proof Fix Agent 执行差异化双向验证：在独立测试台中同时验证旧版本确实重现故障且新版本在相同场景下完全恢复正常。
- [09:00](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=540s) Agent Review 与多层循环把关：审查 Agent 针对代码整洁度、向后兼容性与测试覆盖率进行严格评估，任何不合格项均会打回 Fix Agent 重新迭代。
- [10:00](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=600s) 引入确定性断言与 LLM-as-a-judge 构建完备的 Evals 评估体系，持续监控多 Agent 系统本身的执行表现，防止工作流发生偏航。
- [11:12](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=672s) 借鉴自动驾驶分级理念建立“自主权是挣来的”机制：根据成熟度从有人类兜底逐步过渡到限定“地理围栏”（Geo-fence）内的全自动合并上线。
- [13:00](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=780s) 通过持续积累验证数据逐步拓宽地理围栏：现阶段允许 Agent 对安全漏洞依赖修补、受限声明式端点扩充等高确定性场景直接自动合并。
- [16:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=964s) 渐进式发布（Progressive Rollouts）提供线上安全熔断：新版本以极小流量灰度上线，若生产指标异常则自动暂停放量并重新进入分诊修复循环。
- [17:10](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=1030s) 将分诊输出视为动态进化的“初始假设”而非绝对真理：Fix Agent 在编码与测试中遇到阻碍时可动态修正假设，结合丰富上下文降低幻觉影响。
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) 维护 600+ 连接器的痛点与 AI 信任危机
Pedro Lopez 作为 Airbyte 的软件工程师，开门见山地介绍了本次演讲的核心主题：Airbyte 如何构建自愈连接器（Self-Healing Connector）管道，让 AI Agent 能够全自主地修复生产环境中的 Bug，并保证系统的极高可靠性。Airbyte 是一家开源数据基础设施公司，主要帮助数据团队打通各类外部数据源，并将数据同步至分析管道或 AI Agent。然而，绝大多数数据都存储在不受 Airbyte 控制的外部第三方 SaaS 平台中（例如 Salesforce、HubSpot、Marketo 等）。
生态系统中维护着超过 600 个连接器，且这一数字每天都在攀升。每个集成方案都像是一个“活的有机体”（living organism），绝不可能一劳永逸地构建一次就再也不碰。上游服务商可能在任何时间更新 API、废弃端点，甚至进行未提前通知的非破坏性/破坏性变更。工程师团队因此背负着极其沉重的日常维护负担。
尽管团队发现现有的 AI Agent 在拥有充足上下文与指导的情况下，已经具备解决这些具体问题的代码生成能力，但真正的核心瓶颈在于“信任”（Trust）：“我们如何确信 Agent 提出的修复方案是正确的？又如何保证这个修复不会破坏现存正常运行的功能？”因此，工程的重心并非写 Prompt，而是设计出一套能让修复安全上线并建立信心的系统。
- **生态规模**：Airbyte 目前维护着 600+ 连接器，数量仍在持续增加。
- **故障根源**：外部上游供应商 API 频繁变更或静默弃用（API deprecation）。
- **核心矛盾**：代码生成能力已达标，但将修复推向生产环境面临极高的信任与安全风险。
---
### [03:07](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=187s) 从 Prompting 到 Loop Engineering：构建闭环自愈系统
Pedro 引用了 OpenClaw 创作者 Peter Steinberger 一条引发全网轰动的推文（获得了约 850 万次浏览和 2 万点赞）：“你不该再去给编码 Agent 写 Prompt 了，你应该设计 Prompt 你的 Agent 的循环（Loops）。”虽然当时很多人觉得概念抽象，但其本质正是软件维护的自动化核心：不再是由工程师一步步在终端或对话框里引导 Agent，而是将整个流程抽象为一个能根据反馈自主迭代的闭环系统。
在这样的自动化闭环中，系统可以无缝嵌入各类校验规则与硬性约束（Validations and Constraints），从而在机制上确立对输出结果的信任。最基础的循环形态可以非常直观：Fix Agent 生成修复方案并提交 Pull Request（PR）后，系统触发 CI 流程进行 Lint 检查和测试套件运行；一旦 CI 报错，错误信息会重新作为输入扔回给 Fix Agent 继续迭代修改，直到测试完全通过后标记 PR 为就绪状态。
```mermaid
flowchart TD
accTitle: Basic Self-Healing Loop Architecture
accDescr: Diagram showing triggers feeding into triage, iterative fixing with validation layers, and rollout.
subgraph Triggers["1. 信号触发 (Triggers)"]
T1["上游文档 / 变更日志监控"]
T2["生产环境监控告警"]
end
subgraph TriageFix["2. 分诊与修复 (Triage & Fix)"]
TR["Triage Agent (根因分析与证据链)"]
FX["Fix Agent (声明式构建块生成修复)"]
end
subgraph Validation["3. 验证环路 (Validation Loop)"]
V1{"CI 检查 (Lint / Tests)"}
V2{"Proof Fix Agent (前置/后置行为比对)"}
V3{"Agent Review (代码规范 / 兼容性)"}
end
subgraph Deployment["4. 发布上线 (Deployment)"]
GF{"地理围栏判定 (Geo-fence / Scope)"}
PR["合并 Pull Request"]
RO["渐进式发布 (Progressive Rollout)"]
end
T1 --> TR
T2 --> TR
TR --> FX
FX --> V1
V1 -- "失败" --> FX
V1 -- "通过" --> V2
V2 -- "失败" --> FX
V2 -- "通过" --> V3
V3 -- "失败" --> FX
V3 -- "通过" --> GF
GF -- "超出范围" --> PR
GF -- "安全范围" --> PR
PR --> RO
RO -- "生产异常" --> TR
```
---
### [04:59](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=299s) 信号触发、Triage Agent 与声明式约束
为了实现真正的端到端自主，系统不能依赖人类手动发现 Bug 后去唤起 Agent。Airbyte 引入了两类自动触发信号：
1. **主动监控**：爬取并分析上游厂商的 API 文档与更新日志（Change logs）；
2. **被动监控**：捕获生产环境中突发的异常指标与告警。
这些信号首先流入第一道关卡——**Triage Agent（分诊 Agent）**。Triage Agent 扮演着探索者与规划者的角色，配备了访问生产日志、调试工具与排查技能，其职责是搜集所有上下文，推导出故障的根本原因（Root Cause），并附带推导所依据的完整证据链。这份带有证据的诊断假设，直接转化为下一个环节 Fix Agent 的输入 Prompt。
在代码实现层面，Airbyte 通过将连接器架构设计为标准化的**声明式构建块（Declarative Building Blocks）**，从架构上约束了输出空间。工程师在维护数百个集成时最忌讳出现成百上千种不同的语法与代码风格，声明式格式让 Agent 的任务简化为“拼装标准组件”，有效杜绝了传统大模型生成的散乱垃圾代码（Code Slob）以及跨模块破坏现有逻辑的问题。通过严格限制 Agent 的输出解空间，大幅收敛了输出的不确定性与变异度。
---
### [08:14](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=494s) 多层校验循环：Proof Fix 与 Agent Review
生成了 PR 并不代表真正修好了问题。为了检验修复的有效性，系统在 CI 之外增加了两个专门的校验 Agent：
1. **Proof Fix Agent（修复证明 Agent）**：专门负责制定“验证计划”。它在一个专属测试测试台（Harness）中同时运行 Bug 出现前后的两个版本，一方面验证旧版本确实如预期那样报错失败，另一方面验证新版本在相同场景下能完全恢复正常、解决痛点，以此证明修复有效。
2. **Agent Review（审查 Agent）**：对照代码库的各项工程标准进行代码审查，评估内容涵盖测试覆盖率是否达标、代码整洁度（Code Hygiene）、以及是否严格遵守向后兼容性（Backwards Compatibility）。
如果 CI、Proof Fix 或 Agent Review 中有任何一个环节未通过，上下文和失败信息就会被打回 Fix Agent，重新进入修改循环，直到所有断言和规范全部满足。为了保证这套由多个 Agent 组成的系统自身不发生偏航，团队引入了 **Evals（评估体系）**，通过确定性的断言测试用例以及 LLM-as-a-judge 等方式，持续评估并保障各个 Agent 的执行效果。
---
### [11:12](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=672s) 自主权是挣来的：借鉴自动驾驶的分级与“地理围栏”
在 Airbyte 看来，"自主权应当是通过验证逐步赢得的"（Autonomy should be earned）。不能因为循环跑通了就随意全量自动合并。Pedro 借鉴了自动驾驶汽车（如旧金山的 Waymo、拉斯维加斯的 Zoox）的发展路径：自动驾驶遵循从无自动化、人类安全员在座接管、到限定区域（Geo-fence，地理围栏）全无人驾驶、再到全场景自动化的分级演进。
软件工程的自主修复同样依循此阶梯渐进：
| 演进阶段 | 驱动方式 | 人类角色 | 自主程度 |
|:--- |:--- |:--- |:--- |
| **Level 1** | 人类工程师主动 Prompt AI | 人类端到端主导并修正每一步 | 辅助编码 |
| **Level 2** | 触发器驱动 Loop 自动修复 | 人类必须审查最终的 Pull Request | 自主提 PR |
| **Level 3** | 地理围栏（Geo-fence）内自主推进 | 人类仅审查围栏外的复杂变更 | 围栏内全自动合并 |
| **Level 4+**| 全场景端到端自主修复与上线 | 仅负责顶层监控与评估体系迭代 | 完全自动化 |
Airbyte 目前正处于 Level 2 向 Level 3 迈进的阶段，并不断拓宽其安全“地理围栏”。目前允许 Agent 无需人工审批、直接自动合并上线的场景包括：
- **依赖安全漏洞修补**：纯粹升级存在 CVE 漏洞的依赖版本；
- **特定范围的新端点拓展**：在声明式框架下为已有连接器增补受限的新 API 端点功能。
团队通过持续监控围栏之外的失败用例、编写 Evals 并改进流程；一旦发现围栏外的某些任务类型成功率极高且稳定，便将其纳入地理围栏，进一步扩大自动合并的范围。
---
### [16:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=964s) 渐进式发布与 QA：从根因假设到动态迭代
整个自愈系统并没有在合并 PR 时终结，而是深度延伸到了**渐进式发布（Progressive Rollouts）**体系中。新版本的连接器首先以极小的流量百分比灰度推送给部分用户，如果在灰度期间监控系统捕捉到生产异常，渐进式发布会自动暂停，并将最新异常信息送回自愈循环中重新分诊修复；若指标平稳，则逐步提升流量配比直至 100% 全量发布。
在演讲最后的问答环节，现场观众提问：“如果代码库极其庞大，Triage Agent 开始出现幻觉、偏离方向或者提出了错误的方案，系统在何处将其打回重来？”
Pedro 解释道，Triage Agent 给出的并不是绝对真理，而仅仅是一个**初始假设（Initial Hypothesis）**。要减少幻觉，首要是为 Triage Agent 提供更丰富的技能工具和更充足的生产日志上下文——大多数幻觉往往源于信息缺失。在实际工程中，从“自以为找到了根因”到“动手修复”本就会遇到新发现，人类排查也是如此。Triage Agent 的输出只是 Fix Agent 的初始上下文，当 Fix Agent 带着假设进入编写与测试验证时，如果发现跑不通，这一假设会在后续的迭代与多重校验中被不断修正与进化。
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) 我们如何确信 Agent 提出的修复方案是正确的？又如何保证这个修复不会破坏现存正常运行的功能？ — Pedro Lopez
> [03:07](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=187s) 你不该再去给编码 Agent 写 Prompt 了，你应该设计 Prompt 你的 Agent 的循环（Loops）。 — Peter Steinberger
> [11:12](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=672s) 自主权应当是通过验证逐步赢得的（Autonomy should be earned）。 — Pedro Lopez
> [16:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=964s) Triage Agent 给出的并不是绝对真理，而仅仅是一个初始假设（Initial Hypothesis）。 — Pedro Lopez
> [16:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=964s) 大多数幻觉往往源于信息缺失。 — Pedro Lopez
> [16:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=964s) 从“自以为找到了根因”到“动手修复”本就会遇到新发现，人类排查也是如此。 — Pedro Lopez
> [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) 工程的重心并非写 Prompt，而是设计出一套能让修复安全上线并建立信心的系统。 — Pedro Lopez
> [00:04](https://www.youtube.com/watch?v=aKtpWWeBWDw&t=4s) 每个集成方案都像是一个“活的有机体”（living organism），绝不可能一劳永逸地构建一次就再也不碰。 — Pedro Lopez
## Mentioned
Pedro Lopez, Airbyte, Peter Steinberger, OpenClaw, Salesforce, HubSpot, Marketo, Waymo, Zoox
## Video description
Pedro Lopez - Senior Software Engineer, Airbyte
We gave AI agents the authority to fix our production connectors on their own, while thousands of customer pipelines depended on them not getting it wrong. Upstream APIs change constantly, so we built a system that keeps 600+ connectors healthy autonomously: AI agents detect an issue, diagnose it, and ship the fix automatically. The hard problem turned out to be trust, not code generation. I'll show how we evaluate whether a fix actually works, where we keep a human in the loop and where we don't, and the progressive rollout system that lets an agent's change reach production without risking the whole product. If you're moving agents from demos into production, come see what it takes to let them act autonomously and still sleep at night.
[Source](https://www.youtube.com/watch?v=aKtpWWeBWDw)
Source quotes
[0:04] Pedro Lopez: 我们如何确信 Agent 提出的修复方案是正确的？又如何保证这个修复不会破坏现存正常运行的功能？
[3:07] Peter Steinberger: 你不该再去给编码 Agent 写 Prompt 了，你应该设计 Prompt 你的 Agent 的循环（Loops）。
[11:12] Pedro Lopez: 自主权应当是通过验证逐步赢得的（Autonomy should be earned）。
[16:04] Pedro Lopez: Triage Agent 给出的并不是绝对真理，而仅仅是一个初始假设（Initial Hypothesis）。
[16:04] Pedro Lopez: 大多数幻觉往往源于信息缺失。
[16:04] Pedro Lopez: 从“自以为找到了根因”到“动手修复”本就会遇到新发现，人类排查也是如此。
[0:04] Pedro Lopez: 工程的重心并非写 Prompt，而是设计出一套能让修复安全上线并建立信心的系统。
[0:04] Pedro Lopez: 每个集成方案都像是一个“活的有机体”（living organism），绝不可能一劳永逸地构建一次就再也不碰。
Source quotes
[0:00] Pedro Lopez: The hard problem turned out to be trust, not code generation.
[0:04] Pedro Lopez: maintaining over 600 connectors is like tending to a living organism.
[0:04] Pedro Lopez: The core question isn't whether an agent can write a patch, but how engineers can trust that the fix is correct and won't break existing functionality across thousands of customer pipelines.
[5:07] Peter Steinberger: engineers should stop prompting coding agents directly and instead design loops that prompt the agents.
[5:06] Pedro Lopez: By constraining the output space to predictable building blocks, the agent simply configures predefined components rather than writing arbitrary, chaotic code.
[11:20] Pedro Lopez: autonomy must be earned progressively across standardized levels (0 to 5).
[11:20] Pedro Lopez: Software teams should not merge every AI-generated PR instantly; instead, they should establish geofences (constrained operational scopes) where autonomous merging is permitted without human intervention.
[17:59] Pedro Lopez: the triage phase merely formulates an initial hypothesis, much like a human engineer starting a task without full visibility into every edge case.
Original transcript
[0:04] All right. Hello everybody. So, my name is Pedro.
Um, I work as a software engineer at Airbet.
[0:09] And today I'm going to be speaking uh about how
we have built our self-healing connector pipeline
[0:16] or self-healing software um at Airbet. And
this is all about how we can get agents to
[0:22] fix production bugs auton autonomously and the
systems that make that work uh reliably. So a
[0:30] bit of context um for how this fits into airbite.
So, Airbite is an open data infrastructure company
[0:37] and we help data teams and uh connect to all
of the places where their data lives and make
[0:43] that available either for analytics pipelines as
well as for agents uh more increently as of late
[0:50] and most often this data lives in external
systems that are outside of our control. So we're
[0:56] maintaining a lot of integrations to um these SAS
platforms that marketing teams and sales teams use
[1:04] such as like Salesforce, HubSpot, Marquetto,
etc. And what this means is hey, we have to
[1:10] maintain a lot of integrations. We have over 600
connectors in our ecosystem and that number just
[1:17] keeps growing and growing each day. And so the
maintenance of all these really becomes a burden
[1:24] especially because all of these integrations
are their own little uh living organism and
[1:32] you can't really have a static integration
that you build once and never touch again
[1:37] because we depend on these upstream vendors that
can change their API at any point in time. And
[1:44] so we have to be really mindful of what are these
changes that they're making and be reactive uh to
[1:51] those uh those those changes on their side and
make sure we can we can adapt uh to that. So that
[1:57] can come from breaking changes API deprecations
whether announced or not by the vendors.
[2:03] So all of this really becomes just a lot of
maintenance burden for our uh our engineers.
[2:11] And so naturally the the question becomes how can
we best leverage agents and AI to help with this
[2:17] maintenance issue and keep up with the realities
of integrating with all these third party systems
[2:23] that that we have no control over. And so what
we found out is you know agents are good enough.
[2:29] They given the right context and the right
guidance they can come up with a solution
[2:36] to these problems that are coming up. But the
bigger question then becomes that of trust.
[2:41] So how do we trust that the fix that we are making
is actually the correct fix or that the fix that
[2:48] we're making isn't going to break other things
that uh that are uh that are currently working.
[2:53] And so the a lot of the work turns out to be
how we can build these systems that make the fix
[3:02] safe to ship and give you trust
and confidence in that process.
[3:07] You might have seen this tweet uh going around.
This Peter Steinberger uh creator of Open Claw
[3:13] and he famously, you know, had this tweet that go
viral about how you shouldn't be prompting coding
[3:19] agents anymore. You should be designing loops
that prompts your agents. And the internet went
[3:23] crazy. This was like 8.5 million views, 20K
likes. Um this like very vague what what he's
[3:30] talking about with loops. He went on to clarify
a little bit um after. And really what what this
[3:36] he's talking about how he maintains his open claw
uh project by having codec spin up and look at
[3:44] issues that have been happening um evaluate
the the the PRs that are going on and really
[3:51] automatically come in and and and fix things. And
really what this boils down to is all this loop
[3:57] engineering talk is that instead of you having an
engineer directly be the one uh guiding the agent
[4:03] and iterating and working with it, you think about
it more as a system that you can get the agent to
[4:10] iterate on itself given certain feedback. And as
part of this loop, as part of this system, we can
[4:17] introduce these validations and constraints that
really start to give us trust in that process.
[4:23] So let's build a loop. What does this look like
for us in maintaining connectors? This is the very
[4:29] simple like most uh straightforward version of
the loop that you can start with. So you have an
[4:36] agent that does a fix. The initial thing you would
validate uh once the agent comes up with that that
[4:41] fix and comes up with a PR is okay, let's run CI
and see the linting and the tests. See if they
[4:48] fail. And if that fails, then we send it back
to the fix. And that creates our initial loop.
[4:54] Um, and if everything is is good to go, then we
just mark the PR as ready. But we want agents that
[4:59] can autonomously do this. We don't want to have to
be prompting or having someone kick off this fix.
[5:06] So in our self-healing connector journey, we're
going to need some signals or some triggers
[5:11] to kick this off. So in that um like I mentioned
before a lot of this comes from changes that are
[5:21] made by our upstream vendors and
so we have a set of monitors that
[5:25] basically look at the documentation uh that that
these these vendors provide and and change logs
[5:31] and that becomes a trigger or a signal into the
system that's going to kick off our loop. That's
[5:37] one way that things can happen. But the reality
is that a lot of times they don't tell you when
[5:42] they're making changes when you have all these
integrations. And so we also have to have some
[5:46] production monitoring in here to make sure that
when something does go wrong in production that we
[5:51] didn't know about, we can also trigger this whole
cycle. And then once we do have that ingest of
[5:58] the the triggers that come in, we need to process
them somehow. We call this the triage agent or the
[6:03] first step of our process. And really what this
is a a planner or an explorer that is meant to
[6:10] look at the the signal that is coming in or
the issue that's being um highlighted by by
[6:16] our triggers and gather all the right context, the
right information in order to come up with what is
[6:22] the root cause for the issue that is going on. So
that looks like giving it tools to access logs,
[6:29] giving it skills to be able to debug things um and
really just come up with the um that root cause
[6:36] along with the evidence that led it to to to
believe that this is the cause. And then this in
[6:42] our loop becomes the prompt for the next cycle or
the fix that we already talked about. So instead
[6:48] of having an engineer be the one that prompts this
fetch agent, we already have the signals from the
[6:55] triggers and the triage with a hypothesis
of the root cause be what kicks this off.
[7:03] An interesting thing about air bite is that
constraints um are something we've been
[7:08] thinking about for a long time in the context
of maintenance of all of these integrations.
[7:14] The this problem of having a ton of integrations
is not new to us. We've been around for
[7:18] a couple years and in order to make it easier to
maintain all these connectors where you can't have
[7:25] hundreds of different ways of implementing
the same feature, we have this standardized
[7:30] connector format that really simplifies the way
that we do integrations into building blocks. And
[7:35] the agent then when working on the fix often only
has to really put those building blocks together
[7:42] um in order to uh implement the connector. And
so we don't have to deal with a bunch of like
[7:49] code slob that ends up breaking in other parts
of the system or is repeated all over the place
[7:54] because of this. So this I think aside from you
know us having the the um this this connector
[8:03] declarative format I would I would uh suggest
to think about how you can constrain the output
[8:09] space of agents that you're building so you can
reduce kind of the variability that you can get
[8:14] out of that. So, okay, we have a fix out there.
We have a PR, but how do we know that that fix
[8:20] actually addresses the problem that we set out
to fix? How can we trust this part of the system?
[8:26] And here we can introduce another part to our
loop. Another validator. This time the proof
[8:31] fix agent is what we call aptly named because
all it does is prove that the thing was fixed.
[8:38] So this comes up with a plan um for how to
what evidence that we need to see in order to
[8:45] prove that the issue is actually fixed.
And we do this by having a harness that
[8:50] runs previous version and compares against the new
version. And it's able to evaluate how we can um
[8:58] that the one the old version actually failed
in the way that we expect it to fail and two
[9:03] that the new version works as expected
and the issue was actually addressed.
[9:10] Now we have a PR that actually fixes the problem
but how can we ensure that it meets our standards
[9:17] that uh we want to uphold in our codebase that's
just another validator another point in our loop
[9:23] which is the agent review part and this just
evaluates whether the PR meets specific criteria
[9:29] upholds things like making sure that the the
change that it was that that made had proper test
[9:35] coverage the PR has proper hygiene code hygiene
according to our standards backwards compatibility
[9:40] is respected, etc. And if anything goes wrong
there, it just goes back to the fix along with the
[9:48] other three um the other three validators of our
loop. So this just becomes a selfiterating process
[9:55] until we have a pull request. And once we
have all these, we think to ourselves, well,
[10:01] we have a bunch of agents that are doing all these
things. How can we trust all these agents that
[10:06] we just built uh to make sure we can trust the
system itself? And there is where eval come in.
[10:14] So eval are nothing more than test cases for these
agentic systems. Um this is a whole topic on its
[10:20] own. Um that can be its own talk. But uh the way
that we see this is we have eval agent uh in this
[10:29] process. And the it it just ensures that we're
adhering to the goals of each agent. On the right,
[10:37] you can probably barely see that, but this is
an example of eval fix stage where we come up
[10:43] with test cases of the kinds of things we want
to make sure we are catching and each of those
[10:49] becomes a run um and we uh assert on the output.
That is one kind of eval. But there's other evals
[10:57] that are more less deterministic. um that use
an LLM to judge the outputs of the the result.
[11:04] Um so yeah, there's a infinite way of uh of doing
this that I won't dive too much into. [snorts]
[11:12] Once we can trust it, we have a PR. So we should
go ahead and and ship that pull request, right?
[11:20] Not so fast. Uh because autonomy for us is really
something that should be earned. We don't want to
[11:26] just willy-nilly start merging every single
change that is coming out of all of these um
[11:33] out of the the loop. And the way we think about
this is if you've seen these autonomous cars
[11:39] driving around, um we're in San Francisco where
we're based where there's a ton of these Whimos
[11:45] uh here in Vegas there's zuks uh running around.
And if you notice, these don't start off by
[11:52] just being let loose uh anywhere. These
start generally in a pretty controlled
[11:59] test phase like Zuks is out here in Las Vegas,
but it's not in in a lot of other places. And so
[12:06] they've kind of standardized this into the
levels of driving automation. This is like a
[12:12] a pretty a standard in the the automotive industry
with how they think about self-driving. And it
[12:17] starts on the left with no automation at all
all the way to full automation on the far right.
[12:23] And what you see is like there's steps along the
way here. There's a step where initially there
[12:29] is a driver in the car that is actually there
to make sure that if the if the the car steers
[12:35] the wrong way or does something weird, it the
human can come in and and correct it as needed.
[12:41] And then you can move on to okay maybe we don't
need the human but we're only allowing this this
[12:47] to happen autonomously within a geoence or a
restricted area uh that we have trusted and and
[12:55] you know can uh rely on that the the car will
operate uh the way that we we expect it to.
[13:01] And so this has been mapped to software factories
in in a couple different ways. The way that we
[13:07] think about this um very similarly no automation
on the left there just humans shipping code
[13:12] and then full automation on the right um for
you know just agents are doing everything um
[13:19] but really there's this this gradual gradual
process here so we start with you know a human
[13:25] prompting the AI to make the fix and driving
every step of the way um but then as we graduate
[13:32] maybe an agent autonomously makes the
fix using the loop that we just described
[13:37] kicking kicking it off via these these
triggers and these sensors, but still a
[13:41] human has to ultimately look at the pull request
and evaluate if that's something worth shipping.
[13:46] But then as you push towards agent ship the fix,
you can start introducing a similar concept like
[13:52] these geoences um where agents can autonomously
drive the fix from the the trigger all the way to
[14:00] to pushing that pull request to production.
um but just in a in a constrained scope.
[14:06] So for us um what that means is we allow certain
kinds of changes to be automatically merged
[14:15] um and instructed by the agent. A kind of change
that we allow for example is uh changes that are
[14:22] bumping versions for uh for dependencies that have
security v vulnerabilities. That's something that
[14:29] an agent can pretty easily do and we are confident
we can validate that still works correctly
[14:34] and we can allow to fix without a human ever
looking at that. Another example is adding new um
[14:40] new endpoints to to the to the connectors, new
features um a specific subset of new features
[14:47] rather and that is also a set of of scope that
we feel comfortable about merging um all the way
[14:54] through. And so you can start to really define
what are the things that that we're comfortable
[14:59] with um and and uh letting the agent take take
hold on that and then everything else the human
[15:06] still has to look at. This is kind of the stage we
are at at Airbet is between two and three pushing
[15:12] more on the three um where we're expanding that
set of of automergeable and shippable scopes.
[15:18] And so how do we gain confidence in expanding
that geoence um and expanding those scopes um is
[15:25] really about closely monitoring that behavior
that's happening outside of the current geoence
[15:30] and driving improvements to the process via the
evaluation frameworks that that we talked about.
[15:36] So, it's taking a look at, you know, what what are
the failures that we're seeing um and and writing
[15:42] eval to make sure it it it works as expected. And
then we can track and identify these areas that
[15:48] are outside the boundary that do have high success
rate and we can expand that um that geoence tour
[15:56] outside of the um the shipping the fix. So, now
we can get to merge the PR if we're within this
[16:04] geoence. There's another part of this loop that
comes after after the fix. So we started thinking
[16:10] about the whole loop and the whole system not just
as the authoring authoring of the initial pull
[16:18] request and just leaving it there. We actually go
past that into the into the release. So I think a
[16:25] big way to gain confidence in expanding it and
to mitigate that risk that comes with letting an
[16:31] agent ship to production is thinking about your
release processes. For us that looks like doing
[16:37] progressive rollouts for all of these connector
changes that are happening. So we start with
[16:43] a slow uh you know small percentage of users
that get uh every new change that we make
[16:49] and then if anything goes wrong that's just
another part of this loop that goes back to
[16:55] pausing the roll out and you know going
again to triage what happened fix it etc.
[17:01] And if everything goes well with the the change
we keep increasing the percentage and eventually
[17:07] that fix goes out to to production. So this
is really the what the self-healing loop
[17:12] looks like at Airbet in a summarized way um
today including from triggering the problem
[17:20] looping through and iterating the through
the fix and all the validation steps that
[17:25] we have along the way along through to the uh
progressive flow ads and release of that change.
[17:31] And so back to what we asked ourselves initially,
you know, can we get an agent to help maintain
[17:36] these connectors? And for us, the answer is is
yeah, a lot as long as we have those layers in the
[17:42] system and in our loops that give you trust and
confidence in that process. And that is how we're
[17:48] able to to get to to maintain maintenance of these
connectors. And that's it. Thank you so much.
[17:59] If you had a large code base, the
triarch agent starts to hallucinate,
[18:06] where would you put and it starts deviating or
creates a new plan or idea that is not the right
[18:12] or the right solution, where would you
implement to go back and start the spec again?
[18:18] Good question. So for the triage um
this is really an initial hypothesis
[18:25] of what the root cause could be. Um, and the
way you make that better is by having more
[18:33] more like working on your skills, working on the
the context that you're able to give to the triage
[18:38] to be able to solve that as well as the tools.
Uh, so think about how you can expose production
[18:43] logs and more information that it's usually
because it's missing some kind of information
[18:47] that it would, you know, kind of go wrong.
But this is ultimately a first hypothesis of
[18:53] the problem that it thinks the the root cause is
because it still hasn't really like actually gone
[18:57] through and and worked on it from when you
think you know what the problem is to when
[19:02] you actually go through you know a lot of things
can happen um that you didn't discover initially.
[19:07] Even when humans like working on a problem uh you
might think one thing is the problem but then it
[19:11] turns out to be another thing. So really when you
do that just serves as an initial prompt initial
[19:17] context for the fix agent and then the fix agent
is really the one that has to iterate on on that
[19:24] initial hypothesis and that hypothesis can evolve
as it works on uh through through this process.
