Steinberger Automates OpenClaw Development With Custom Agent Tooling

Build the thing that builds the thing | BRK245 — Microsoft Developer

Microsoft Developer

Software engineering must shift from manual coding to building disposable infrastructure and feedback loops that empower autonomous agents to verify, repair, and ship software end-to-end.

AI-assisted summary of Build the thing that builds the thing | BRK245 — original source. Check the original for full context.

Source published:

Key ideas

  • Peter Steinberger argues that software engineering has inverted: developers should stop optimizing their own coding speed and instead build tools, feedback loops, and verification environments that empower AI agents to operate autonomously end-to-end. (0:06)
  • OpenClaw handled an issue backlog exceeding 10,000 items using ClawSweeper, an automated triage system running OpenAI Codex over GitHub Actions that evaluated tickets against a core vision.md file and closed roughly 15,000 issues. (3:02)
  • To bypass the 5,000 requests-per-hour personal GitHub API cap during multi-agent sweeps, Steinberger deployed Octopus, a Cloudflare Worker shim that routes read traffic through a 15,000 req/hr GitHub App while reserving personal tokens for mutating actions. (17:09)
  • Visual and cross-platform verification was solved through CrabBox and Mantis, which spin up remote cloud environments across operating systems, record before-and-after video proof via VNC, and allow agents to visually confirm fixes before PR merging. (24:48)

Detailed notes

## Background Published on YouTube by Microsoft Developer on June 4, 2026, this breakout session from Microsoft Build 2026 (BRK245) features Peter Steinberger discussing the rapid transformation of software development. The presentation highlights the custom ecosystem of tools his team created to build and scale OpenClaw faster and with greater confidence using autonomous AI agents.

## TL;DR Peter Steinberger presents a paradigm shift in software development, arguing that engineers should stop optimizing their own coding speed and instead design feedback loops that empower autonomous coding agents to work end-to-end. Facing a deluge of tens of thousands of GitHub issues and pull requests on OpenClaw, Steinberger engineered an interdependent suite of disposable internal tools—such as ClawSweeper for continuous Codex-driven issue triage, Octopus for bypassing API rate limits, CrabBox for cross-platform cloud virtualization, and Mantis for automated video bug verification. Across the session, he illustrates how using personal developer annoyance as an automation signal allows a small team to build compounding toolchains, conduct whole-codebase audits, and bypass traditional operational bottlenecks.

## Atmosphere & style The session has an irreverent, fast-paced, and highly candid atmosphere, with Steinberger deliberately avoiding standard corporate slide decks in favor of pragmatic developer war stories and live dashboard walk-throughs. His delivery is conversational and wry, filled with self-deprecating humor about runaway recursive agent loops, 'clankers,' and accidentally coding a full backup chat application out of frustration with Cloudflare captchas. The talk feels less like an executive conference keynote and more like an animated hackathon retrospective that leaves engineers energized to build scrappy, high-leverage automation.

## Bulletin - [00:06](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=6s) Peter Steinberger argues that software engineering has inverted: developers should stop optimizing their own coding speed and instead build tools, feedback loops, and verification environments that empower AI agents to operate autonomously end-to-end. - [03:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=182s) OpenClaw handled an issue backlog exceeding 10,000 items using ClawSweeper, an automated triage system running OpenAI Codex over GitHub Actions that evaluated tickets against a core vision.md file and closed roughly 15,000 issues. - [17:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1029s) To bypass the 5,000 requests-per-hour personal GitHub API cap during multi-agent sweeps, Steinberger deployed Octopus, a Cloudflare Worker shim that routes read traffic through a 15,000 req/hr GitHub App while reserving personal tokens for mutating actions. - [24:48](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1488s) Visual and cross-platform verification was solved through CrabBox and Mantis, which spin up remote cloud environments across operating systems, record before-and-after video proof via VNC, and allow agents to visually confirm fixes before PR merging.

## Key takeaways - [00:06](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=6s) The primary role of a modern developer is no longer writing code faster, but closing the autonomous feedback loop so agents can verify and correct their own work. - [03:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=182s) Incoming pull requests and issues are functionally identical signals ('prompt requests') indicating user dissatisfaction, and generating clean fixes from issues is often faster than reviewing messy third-party LLM pull requests. - [06:45](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=405s) High-velocity open source repositories require a continuous sweeping mechanism, running weekly or daily sweeps where agents close obsolete tickets patched by recent main branch commits without human intervention. - [03:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=182s) A standard repository specification like vision.md is essential to ground autonomous triage models on explicit feature boundaries and project scope. - [13:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=789s) Correlating live community chatter across Discord with GitHub trackers allows automated agents to prioritize the top user pain points rather than merely triaging chronological backlogs. - [15:19](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=919s) Internal developer tooling does not need rigorous design standards or line-by-line review; it can be entirely 'vibe coded' and treated as disposable scripts to eliminate operational friction. - [17:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1029s) API rate limits from commercial developer platforms can severely throttle multi-agent workflows, prompting custom proxy shims like Octopus to split read and write token quotas. - [20:00](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1200s) Offloading heavy parallel test suites to ephemeral multi-cloud runners (CrabBox) prevents local machine lockups and catches OS-specific regressions across Windows, macOS, and Linux. - [24:48](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1488s) Automated visual testing via Mantis captures side-by-side bug reproduction and verification videos, shifting human code review toward inspecting recorded visual proof. - [26:29](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1589s) Running automated reviews with fresh sub-agent context in an isolated CLI process prevents context bloat and eliminates circular review loops. - [29:25](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1765s) Agent instructions and invariants in agents.md should be authored and refactored by LLMs themselves rather than manually drafted by developers. - [31:46](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1906s) Requiring sanitized prompting transcripts to be attached to pull requests allows maintainers to distinguish rigorous engineering dialogue from low-effort AI-generated 'slop.' - [34:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2042s) Large codebases exceeding one million lines can be audited by tools like ClawPatch, which divide the repository into discrete functional modules for parallelized, fresh-context agent passes. - [37:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2222s) Commercial anti-bot defenses against automated headless Linux containers can force teams to build independent internal testing infrastructure like the ClickClack messaging platform. - [38:43](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2323s) Personal annoyance is the best signal for automation; addressing friction points creates an interlocking toolchain that compounds productivity for small teams. - [39:51](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2391s) AI models excel at cross-repository pattern matching, allowing developers to point agents at previous solutions and Git history instead of maintaining fragile, rigidly coupled tool architectures.

## Detailed walkthrough ### [00:06](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=6s) Rethinking Software Engineering: Helping Agents Close the Loop

Peter Steinberger opens the session by cautioning the audience not to expect a standard corporate slide deck. His talk, titled "Build the thing that builds the thing," is intended to reframe how developers view the future of software construction. In his view, coding agents have only existed for less than a year, have only truly been effective for about six months, and their rapid evolution is rendering traditional software development practices obsolete. He asks the room if anyone actively maintains open source, following up with a wry query: “How are you handling the pain?”

Addressing media narratives around his recent work, Steinberger laughs off articles asserting that he launched 43 failed projects before finally striking gold with OpenClaw. None of those projects were failures; rather, each was a targeted internal tool he spun up to accelerate his own workflow and remove friction for his autonomous coding agents. He stresses that modern engineering requires an inverted mindset: human developers should stop focusing on how to code faster themselves and instead focus on how to empower AI agents to work faster and complete tasks end-to-end.

To illustrate what he calls "closing the loop," Steinberger recounts spending several months building a SaaS product the previous year. Initially, an agent would write code, announce it was finished, and leave Steinberger to manually verify the user interface in a browser, spot errors, and prompt the model again in a tedious back-and-forth. The breakthrough came when he realized the agent could simply take its own screenshots and inspect the visual rendering directly. By eliminating repetitive human interventions, developers can design feedback mechanisms that allow agents to autonomously verify and fix their own outputs.

As Peter Steinberger put it: “Anytime you build software now your job is no longer to, to figure out how you build software faster. Really your job should be, what can I do to help my agent to build software faster and even more?”

[03:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=182s) Taming the Influx: Prompt Requests and ClawSweeper

When the OpenClaw client suddenly gained widespread traction, the repository was flooded with an unmanageable spike in issues and pull requests. Steinberger discovered firsthand that GitHub's web interface masks totals once they hit 5,000, obscuring a backlog that had surged past 10,000 entries. Because users and their automated agents file tickets indiscriminately, open-source repositories are inundated with low-effort contributions. While many maintainers vent their frustration at this deluge of AI-generated content, Steinberger takes a pragmatic stance: developers cannot fight the inevitable tide of technology and must adapt their infrastructure instead.

He redefines GitHub's standard vocabulary, arguing that incoming pull requests are essentially just "prompt requests." To him, issues and PRs represent identical signals: a user is dissatisfied with the current software behavior and wants a change. In fact, raw issues are often preferable to pull requests. Reviewing external pull requests frequently forces a maintainer's agent to ingest third-party LLM code, untangle it, and rewrite it—consuming far more time than simply having an agent implement a fresh fix directly from an issue description.

To tackle the backlog, Steinberger experimented with commercial automated code review platforms like CodeRabbit. While competent at commenting, they failed to group or close duplicate and invalid items automatically. Rather than waiting for a third-party vendor to build a solution, Steinberger created ClawSweeper. Powered by GitHub Actions, ClawSweeper triggers an instance of OpenAI Codex in the cloud whenever a ticket or PR is submitted. The workflow injects a dedicated repository markdown file—`vision.md`—into the agent's core prompt, defining explicit boundaries for features the project actively wants versus those it explicitly rejects. Armed with these project rules, ClawSweeper evaluated the backlog and closed roughly 15,000 issues.

As Peter Steinberger observed: “You can't fight the world. You have to figure out a way how to deal with that.”

```mermaid flowchart TD accTitle: ClawSweeper Automated Issue Processing accDescr: Pipeline showing how incoming GitHub issues and PRs are evaluated against vision.md using Codex to comment or close tickets.

subgraph Ingestion ["GitHub Activity"] A["New Issue / Pull Request"] B["Scheduled Weekly Re-sweep"] end

subgraph Evaluation ["ClawSweeper Engine"] C["GitHub Actions Runner"] D["OpenAI Codex Instance"] E["vision.md (Project Rules)"] end

subgraph Resolution ["Automated Actions"] F["Close Obsolete / Out-of-Scope Ticket"] G["Post Clarifying Comment / Status"] H["Discord Notification to Maintainer"] end

A --> C B --> C C --> D E --> D D --> F D --> G D --> H ```

- When backlogs reach 5,000 items, GitHub's user interface stops incrementing the public counter, masking the actual volume (which reached over 10,000 open items for OpenClaw). - Operating under the assumption of "infinite tokens," ClawSweeper ran automated triage passes that successfully closed approximately 15,000 extraneous issues and PRs. - Every repository maintained by Steinberger now standardizes on a root `vision.md` specification to steer autonomous triage decisions.

[06:45](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=405s) Continuous Sweeping and Discord Notifications

Triage in an ultra-high-velocity repository cannot be treated as a one-time static audit. At OpenClaw's scale, everyday code commits landing on the `main` branch regularly fix bugs and edge cases that the maintainers have never personally read, simply because thousands of tickets sit uninspected in the backlog. To address this lag, Steinberger configured ClawSweeper to execute sweeping runs over every open issue and pull request at least once a week—a cadence that can be dialed up to daily runs depending on token expenditure.

This background sweeping loop links directly to Steinberger's Discord workspace. Codex inspects aging tickets against recent commits on `main`, resolves those whose root causes have already been patched, and posts automated feedback to the issue author explaining that the bug is resolved in the latest build. Steinberger regularly receives Discord pings noting that three-month-old tickets have been automatically cleared without requiring a second of his manual attention.

Demonstrating the live operations dashboard, Steinberger shows 1,818 Codex background workers concurrently analyzing tickets. The system surfaces a consolidated activity stream that provides maintainers with structured summaries, contextual arrays of issues requiring targeted fixes, and detailed progress logs.

Highlighting the reality of high-volume repository maintenance, Peter Steinberger explained: “Any time a fixed lens on main, there's a good chance that I solved an issue that I've never read because there's like many thousand issues open.”

[08:30](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=510s) Maintainer Dashboards, Custom Issue Browsers, and Gorg

As the OpenClaw ecosystem accelerated, Steinberger onboarded community maintainers rapidly and liberally. However, contributor engagement varied wildly: while some individuals proved exceptional, others contributed for a few days before vanishing entirely. To regain governance and visibility across the team, Steinberger built an internal maintainer dashboard. Recognizing that open-source leadership encompasses far more than pushing git commits, the dashboard synthesizes GitHub repository analytics with Discord communication metrics, ensuring contributors actively assisting users in discussions receive full visibility.

To feed Discord activity into the dashboard, Steinberger wrote a custom Discord crawler using quick agent prompts. He points out that creating such micro-tools is trivial with modern LLMs; the true difficulty lies in sustaining and maintaining them over time. Steinberger manages his workload at an abstraction layer above single repositories: he navigates to a root directory containing all his projects and instructs Codex to triage across every workspace simultaneously, surfacing an aggregate action list of open issues.

```mermaid flowchart LR accTitle: Maintainer Dashboard Data Ingestion accDescr: Data flow consolidating GitHub activity and Discord community discussions into a unified internal maintainer portal.

subgraph Sources ["Activity Signals"] GH["GitHub Commits & PRs"] DC["Discord Discussions & Chat"] end

subgraph Pipeline ["Processing Layer"] CR["Custom Discord Crawler"] TX["Multi-Project Codex Triage"] end

subgraph Portal ["Maintainer View"] DB["Maintainer Analytics Dashboard"] IB["Fast Issue Browser UI"] end

GH --> DB DC --> CR --> DB TX --> IB ```

Because review time remains the ultimate bottleneck, Steinberger eliminated small interface latencies that slowed down triage. When reviewing aggregate issue lists from Codex, opening multiple browser tabs proved sluggish. Furthermore, different AI models and LLM setups—which he jokingly refers to as "Clankers"—output markdown lists and issue references in erratic, non-standardized formats. To bypass this friction, he built a dedicated, web-based issue browser. The custom tool parses any copied text block regardless of formatting irregularities and renders a clean interface where he can rapidly approve or discard items. He generally directs the agent to execute autonomous fixes for eight out of ten selected tasks while reserving his personal focus time for the remaining two complex issues.

When existing platform tooling fell short for other agent workflows, Steinberger adopted the same philosophy. Frustrated that Google lacked a comprehensive CLI for programmatic agent access, and dissatisfied with built-in integrations that locked agents to a single Google account, he created **Gorg**—a dedicated command-line interface enabling agents to interact seamlessly with multi-account Gmail and Calendar data.

As Peter Steinberger emphasized regarding developer efficiency: “The critical thing that you are is like, your time is limited. So you need to build the tools to build the tools.”

- High-level maintainer triage operates out of a centralized master projects directory rather than inside individual Git repositories. - Steinberger's custom issue browser incorporates flexible parsing heuristics to handle formatting quirks produced by diverse LLM agents. - **Gorg** was developed as an open-source CLI utility to allow coding agents to manage multi-account Gmail drafts, messages, and Google Calendar events without human browser intervention.

[12:33](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=753s) End-to-End Autonomous Issue Verification

Peter Steinberger explains the profound leverage of having an autonomous agent operate across an entire software stack. Because he and his agent control the complete environment end-to-end, any missing component or bug can simply repair itself in place without requiring external intervention. He describes the empowering sensation of owning the whole problem space, remarking: “My agent controls the whole stack. So it can just fix itself like so like you own the whole universe. Anything is fixable.”

When new issues or feature requests come in, tools like Codex do far more than merely produce source code. Steinberger highlights that because he maintains demo accounts on external platforms such as GOG, the agent can autonomously implement the feature and run comprehensive end-to-end verification. Consequently, the maintainer's role is reduced to simply reviewing the finished artifact, giving approval, and deploying to production with total confidence.

- Agents with full stack access can self-diagnose and repair missing dependencies or runtime failures autonomously. - Live integration accounts (e.g., GOG demo accounts) allow the AI to test end-to-end user flows rather than stopping at mock unit tests. - Maintainer involvement shifts from manual implementation to reviewing verified pull requests and clicking "yes" to ship.

---

[13:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=789s) Mining Maintainer Reports: Correlating Discord and GitHub

Maintaining a large open-source project with thousands of incoming issues makes prioritization an overwhelming challenge. To combat issue fatigue, Steinberger built an automated crawling system connected directly to Codex. The system ingests chatter across Discord community channels and cross-references it with open GitHub issues and pull requests, surfacing the problems generating the most user friction.

```mermaid flowchart LR accTitle: Discord Crawl to Agent Triage Flow accDescr: Discord chat logs are crawled by a bot, correlated with GitHub PRs and issues by Codex, and matched against a vision document to select tasks. subgraph Ingestion["Community Ingestion"] A["Discord Channels"] -->|"Bot crawl"| B["Serialized DB Backup on GitHub"] end subgraph Triage["Autonomous Triage"] B --> C["Codex Correlation Engine"] D["GitHub Issues & PRs"] --> C E["Vision Document"] --> C end subgraph Execution["Task Execution"] C -->|"Top 5 pain points"| F["Autonomous Worker Agents"] end ```

The crawl system aligns issues directly with high-level project goals. Steinberger prompts the agent with a targeted query: “What are people screaming the most correlated to open issues and PRs and find me the top five things I should work on.” The agent reconciles high-volume community complaints against a core vision document, filters for self-contained tasks that can be addressed independently, and spins up parallel background tasks for hours of autonomous remediation while keeping human maintainers informed of high-risk, nuanced code paths.

- Triage agents correlate raw community chatter with open tracker items to surface top user pain points. - The system checks candidate tasks against an architectural vision document to prevent agents from wandering off-scope. - Maintainers maintain a high-level overview, letting agents handle routine PRs while reserving human review for delicate or architectural components.

---

[14:22](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=862s) Distributing Community Crawl Data via GitHub

Once the Discord crawling tool proved effective, other project maintainers requested access to the aggregated data. However, Discord lacks a simple public export mechanism, and sharing a privileged bot token across multiple maintainers posed severe security and operational risks.

Steinberger resolved the distribution bottleneck by leveraging GitHub as a decentralized database. Instead of provisioning token infrastructure, the bot continuously serializes the Discord message database into flat files stored in a private GitHub repository. Maintainers without Discord API bot tokens can pull the structured data directly from GitHub, giving them an instant, secure query backend without administrative overhead.

---

[15:19](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=919s) The Release Bar Dashboard and Vibe-Coding Internal Tools

Managing numerous open-source repositories often leaves maintainers blind to which projects require urgent releases, dependency bumps, or review. Steinberger created "Release Bar," a custom lightweight dashboard designed to track days elapsed since each project's last release and identify stagnant repositories requiring maintenance. Addressing the continuous upkeep required by modern software, he warns: “If you don't constantly garden your software, it's dead because dependencies, everything needs updating.”

He draws a sharp philosophical contrast between core user-facing software and internal productivity tooling. While public code in projects like OpenClaw demands meticulous review, internal tools can be purely "vibe coded" with agents without manual line-by-line inspection. If a disposable tool works, it provides immediate leverage; if it fails, it can be iterated on or thrown away at near-zero marginal cost.

- Release Bar monitors projects by tracking the number of days since the last release and flagging unmerged PRs. - Internal developer tooling can be fully generated by AI without strict code reviews because it is not mission-critical. - Successful internal utilities can be incrementally refined if they prove sticky, eliminating upfront engineering costs.

---

[17:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1029s) Bypassing GitHub Rate Limits with Octopus

Running ten or more agentic workflows concurrently quickly exhausted personal GitHub API rate limits. Steinberger describes the frustration of running large automated sweeps across related issues, only to hit the individual developer cap of 5,000 API requests per hour. He jokingly recalls relying on an internal GitHub contact who had to manually push a button every hour to reset his quota, a setup that collapsed whenever she had to sleep.

Reflecting on his development style, Steinberger remarks: “I find being annoyed is the very best way to solve problems.” To permanently overcome the 5,000-request wall, he created "Octopus," a routing proxy that shims GitHub API calls through Cloudflare Workers.

```mermaid flowchart TD accTitle: Octopus GitHub Token Proxy Architecture accDescr: API requests pass to a Cloudflare worker which routes read-only calls to a 15k token GitHub App and mutating calls to a 5k personal token. A["Agent Requests"] --> B["Octopus Shim (Cloudflare Worker)"] B -->|Non-mutating read on public repo| C["GitHub App Token (15,000 req/hr)"] B -->|Mutating / private action| D["Personal PAT Token (5,000 req/hr)"] C --> E["GitHub API"] D --> E ```

- Individual personal access tokens (PATs) cap out at 5,000 requests per hour. - GitHub Apps grant an elevated tier of 15,000 requests per hour. - Octopus inspects HTTP traffic, automatically shunting read-only public requests through the GitHub App while reserving the personal PAT strictly for state-mutating actions. - Autonomous agent stacks (combining advanced reasoning models and browser automation) handled the Cloudflare infrastructure configuration directly.

---

[20:00](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1200s) Crab Box: Cloud Test Execution and Cross-Platform Matrixes

As project scope expanded and bug reports multiplied, the test suite bloated from taking 10 seconds to 30 seconds, then 2 minutes, and eventually over 5 minutes. Steinberger optimized the test suite locally for his high-spec Mac Studio down to 60 seconds, but this caused teammates' machines to overheat and freeze under the heavy parallel load. Early experimentation with remote test-runner startups like Blacksmith showed promise, but recurring daily downtime frequently paralyzed team pull requests.

In response, Steinberger developed "Crab Box," an open infrastructure abstraction tool named in keeping with crustacean convention. Crab Box instantly spins up ephemeral remote cloud VMs across more than 20 supported providers (including AWS and Azure), mirrors local repository changes, and executes heavy CPU-bound test suites off the developer's local machine.

```mermaid flowchart TD accTitle: Crab Box Cross-Platform Execution Architecture accDescr: Crab Box synchronizes code to multiple cloud instances running macOS, Linux, and Windows with VNC remote inspection. A["Local Repo / Agent Session"] -->|"Sync changes"| B["Crab Box Orchestrator"] subgraph RemoteVMs["Ephemeral Matrix Workers"] B --> C["Linux VM (Red Hat / Ubuntu)"] B --> D["macOS Worker"] B --> E["Windows Worker + VNC UI"] end RemoteVMs -->|"Aggregated Test Results"| F["Agent Confidence Verification"] ```

Beyond raw performance offloading, Crab Box resolved stubborn cross-platform inconsistencies. Node.js environments frequently introduce platform-specific edge cases, such as EEPROM issues on Windows. Rather than manually booting virtual machines at 3:00 AM, Steinberger integrated cross-platform matrix testing into Crab Box, enabling agents to fan out tests across Windows, Red Hat, and other operating systems simultaneously. To eliminate the final hurdle of diagnosing OS-specific visual regressions on Windows web browsers, he embedded VNC capabilities directly into the boxes, enabling full remote graphical debugging.

- Local parallel test runs overheated developer laptops, necessitating isolated remote runner environments. - Crab Box provides failover fallback across cloud infrastructure providers (AWS, Azure, and alternatives) to prevent CI interruptions. - Multi-architecture runners spin up simultaneously to validate platform-specific behavior across Linux distros and Windows without requiring manual host setups. - Built-in VNC remote desktop access allows agents and maintainers to inspect browser and desktop UI bugs directly on remote instances.

[24:48](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1488s) UI Automation with CrabBox and Visual Verification with Mantis

Peter Steinberger explains that remote debugging using VNC in CrabBox was an enormous step forward: an agent could spin up an exact environment containing a browser, a terminal, and messaging apps like Telegram to recreate end-to-end issues. However, having to manually interact with the VNC session remained a chore. Driven by the philosophy that “laziness is good,” he equipped CrabBox with computer vision capabilities and lightweight tools for taking screenshots, clicking, and typing. This allowed coding agents to interact directly with graphical user interfaces using automated visual controls.

Yet watching an agent slowly navigate an interface was still tedious. Steinberger wanted finished proof rather than live monitoring, which prompted the creation of Mantis. When an issue arises, Klaus can be triggered to generate a pull request, while Mantis spins up sandboxed environments to record video evidence of the bug, apply the patch, capture video confirming the resolution, and analyze the resulting footage to ensure correctness.

```mermaid flowchart LR accTitle: Mantis Automated Bug Fix and Verification Pipeline accDescr: Shows the automated flow from issue detection to Klaus writing a fix, Mantis verifying via video, and one-click merging.

subgraph Trigger["1. Issue & Patch"] A["Issue Reported"] --> B["Klaus Generates PR"] end

subgraph Verification["2. Mantis Visual Testing"] B --> C["Spin up CrabBox Container"] C --> D["Record Bug Video"] D --> E["Apply Fix & Record Verification"] E --> F["Agent Inspects Video Artifacts"] end

subgraph Resolution["3. Maintainer Review"] F --> G["Maintainer Reviews Clip"] G --> H["Merge PR"] end ```

- CrabBox added native tools for screenshots, clicks, and keystrokes to let models drive UI workflows directly. - Mantis records dual videos (before-fix bug reproduction and after-fix resolution) and evaluates them automatically. - “And that's all built based on me being annoyed,” Steinberger noted regarding the motivation behind building these visual testing systems.

---

[26:29](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1589s) Eliminating Review Thrash with Automated Review Chains

Steinberger recounts the frustration of relying on manual `/review` commands when working with Codex. A prompt would run, followed by a review revealing three genuine issues. After spending twenty minutes fixing tests, running the review a second time would reveal two previously unmentioned issues due to context limits. Repeating this cycle over ten times while implementing support for the Line messaging platform consumed an entire day: “I don't even use Line, but like somebody was very passionate about it. I wanted to get it in. And then all day I did this review and I was like, is it enough? Is enough? Like, like, I feel like I'm the clanker.”

To stop acting as the manual intermediary, he built `auto-review`. While he initially considered writing complex hooks or forking the open-source Codex codebase, modern models follow system prompts reliably enough that a simple rule inside `agents.md` was sufficient. When instructed to run an auto-review before committing or landing a pull request, the agent spins up a fresh CLI sub-agent session with clean context to review its own modifications. Although an early recursion bug caused Codex to invoke nested instances endlessly, the stabilized implementation now automatically executes sequential review passes without human intervention.

- Relying on human reviewers to spot regressions with "fresh eyes" does not scale, but spawning fresh agent processes provides genuine context isolation. - Early prototypes ran into runaway process loops where Codex recursively called itself until guarded limits were enforced.

---

[29:25](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1765s) Refining Agent Guidelines and Pruning Invariants

To make autonomous reviews practical, Steinberger had to teach the agent which issues were real and which were irrelevant noise. Without explicit domain knowledge, agents default to excessively defensive code—such as adding guards against plugin files being modified live on disk while the process is active, an edge case the project intentionally ignores. Because agent memory resets at the start of every session, project-specific invariants must be documented explicitly in the instructions.

He advises developers never to write these agent configuration documents by hand. Instead, developers should describe their goals conversationally and have the model author the instructions in its own terminology. Periodically, maintainers should enter a "meta mode" to ask the model whether any rules inside its instruction file are contradictory or confusing. Ambiguous instructions waste context tokens and degrade generation quality.

- Maintainers must document project-level assumptions in `agents.md` to prevent models from generating boilerplate defensive code for scenarios that never occur. - Instruction files should be authored and refactored by the models themselves rather than written manually by humans. - “Try auto review and try CrabBox. It totally revolutionized how we build software like my, my trust in in prompting and shipping is much, much higher with that stuff,” Steinberger emphasized as his primary advice.

---

[31:46](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1906s) Extracting Prompts and Mitigating Pull Request "Slop"

A persistent problem with open-source contributions in the AI era is determining the engineering rigor behind large submissions. A one-sentence prompt can generate ten pages of code, making it difficult to differentiate low-effort code generation from thoughtful, thoroughly iterated contributions. Early efforts asking contributors to manually paste their prompting transcripts failed because contributors simply bypassed the request.

To solve this, Steinberger developed an OpenClaw skill that intercepts the pull request submission process. The agent asks the contributor for permission to sanitize and attach their session history, isolates the relevant portions of the disk transcript, scrubs sensitive information, and appends the prompting log directly to the PR description. This provides maintainers with immediate visibility into whether the contributor had a substantive problem-solving dialogue with the model or simply ran a four-word command like "fix issue number."

- The OpenClaw skill automatically detects, redacts, and attaches local conversation transcripts to pull requests when authorized. - Transcripts allow maintainers to distinguish low-effort AI "slop" from carefully verified, high-context agent workflows. - “Oftentimes we're not limited anymore by intelligence. The models are good enough now. Everything that we are limited of is our own imagination,” Steinberger reflected.

---

[34:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2042s) Whole-Codebase Auditing with ClawPatch and CrabFleet Multiplayer

After witnessing how many subtle defects auto-review caught on new pull requests, Steinberger realized that existing code committed without review was full of undetected mistakes. However, running a standard review across a massive code base with over a million lines of code is impossible because context windows become overwhelmed and models lose focus.

To address this challenge, he built ClawPatch. The tool breaks large repositories down into roughly fifty isolated functional subsections, analyzes each module independently with fresh agent contexts, and surfaces actionable refactoring patches. Finally, Steinberger introduced CrabFleet, an ongoing project addressing the inefficiency of human teammates acting as copy-paste relays between their local coding agents and internal chat channels. By running Codex inside shared CrabBox virtual environments, developers can collaborate within the same live agent session through a shareable link instead of manually shuttling snippets back and forth.

| Tool | Purpose | Mechanism | |:--- |:--- |:--- | | **ClawPatch** | Large-scale repo auditing | Splits a 1M+ line codebase into ~50 discrete components, running dedicated review agents across each slice. | | **CrabFleet** | Multiplayer agent workspaces | Hosts Codex directly inside containerized CrabBox instances, allowing teammates to access and steer a shared agent session via URL. |

- “You're just intermediary. I want to talk to your clanker,” Steinberger noted, explaining the frustration that led to CrabFleet's collaborative environments. - CrabFleet turns isolated local agent sessions into multi-user collaborative sandboxes accessible via a web link.

[37:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2222s) Accidental Infrastructure: Building "ClickClack" to Escape Captchas and Bot Blocks

Peter Steinberger describes how their internal developer setup expanded into full collaborative tooling. With remote instances and VNC sessions, maintainers can inspect an active session, talk through it, and take control in what he humorously calls "multiplayer clanking." However, their ambition to perform true end-to-end integration testing quickly collided with commercial messaging platforms. Testing complex APIs across multiple channels required verifying visual output directly inside an authentic user interface. Whenever their automated agents spun up headless, Linux-based instances in containers to interact with these external services, defense mechanisms immediately triggered. Security systems interpreted automated Linux boxes with no logged-in user as an active attack:

> “And I built this automation, but man, anytime I logged in, there was like a Cloudflare captcha or like, like all these, all these companies really don't like it if you spin up something in a box, especially not if it's Linux based and there's nobody logged in and it just feels like it feels like someone's trying to hack the system.”

Rather than continually wrestling with bot detection, IP bans, and captchas, the team took the extreme step of reinventing their own real-time chat infrastructure. Steinberger admits that the project spun out of scope, leaving them with an entirely functional backup messaging platform:

> “So we just built our own messaging platform to like test to like, have real end to end tests with something that we have under control. And it got a little bit out of hand. And now we kind of have a backup chat system that works. Whenever our team is down, it's called ClickClack.”

```mermaid flowchart TD accTitle: Internal Chat and Test Infrastructure Architecture accDescr: Flowchart showing why automated headless testing against commercial platforms led to building ClickClack.

subgraph ExternalBlocks["Commercial Chat Platforms"] A["Headless Linux Container"] -->|"Automated End-to-End Test"| B["Cloudflare Captchas & Bot Detection"] B -->|"Fails / Blocks Connection"| C["Brittle Test Automation"] end

subgraph InternalSolution["In-House Replacement"] D["Multiplayer VNC Sessions"] -->|"Direct UI & API Testing"| E["ClickClack (In-House Messaging Platform)"] E -->|"Bypasses Captchas"| F["Reliable End-to-End Automated Tests"] E -->|"Failover Channel"| G["Team Backup Chat"] end ```

---

[38:43](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2323s) Annoyance as an Automation Signal and the Compounding Toolchain

Reflecting on the sprawling ecosystem of specialized crawlers, dashboards, and virtualized runners built for OpenClaw, Steinberger shares his overarching philosophy for deciding what to automate. Rather than following rigid roadmaps or top-down mandates, developers should pay close attention to daily friction points. Personal frustration serves as the most reliable diagnostic indicator that a manual workflow is ready to be handled by an agent:

> “Listen to your body. Listen when you're annoyed. Annoyance is an amazing signal for, we can automate this.”

When internal tools are created to alleviate specific annoyances, they begin compounding in unexpected ways. Steinberger notes that individual systems do not exist in isolation; each new utility builds upon the outputs and abstractions of the ones that preceded it. The release reports exist only because data was first extracted using Discrawl, while runner orchestration via CrabFleet depends on the base virtualization provided by CrabBox. Despite what he admits are eccentric internal names, these tools form an interdependent scaffolding that allows a small engineering group to maintain and iterate across a massive, cross-platform codebase.

- **Compounding internal toolchain:** Specialized tools feed directly into higher-level workflows (e.g., Discrawl data powers report dashboards, while CrabBox isolates instances for CrabFleet orchestration). - **Team leverage:** Custom automation enables a very small maintainer team to manage issue triage, cross-OS testing, and releases for an expansive open-source ecosystem.

---

[39:51](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2391s) Q&A: Tool Dependencies, Cross-Repo Pattern Matching, and Background Codex Building

Opening the floor to questions, an attendee asks whether assembling a sprawling, enterprise-style ecosystem of interconnected internal tools creates friction, where dependencies between custom utilities end up blocking future improvements. Steinberger responds that this bottleneck hasn't materialized. Rather than acting as rigid dependencies, existing tools serve as concrete templates and reference points for coding agents. When instructing an AI model, referencing existing internal solutions drastically reduces prompting overhead:

> “When you work with agents, if you've solved something once, you can just tell the agent, look there, I already solved this there. They are really good at pattern matching. So I would actually say it really compounds.”

He illustrates this with an issue he faced when building token management. The initial prototype ran locally on his workstation, which proved clunky and awkward to integrate with a GitHub App. Instead of crafting elaborate new prompts from scratch, he had his agent inspect the Git history, discard the broken prototype, salvage functioning logic from an existing companion tool, and assemble the revised architecture. Because agents navigate Git commit history and project context with high competence, prompt engineering shifts from detailed specification writing to high-level delegation.

Steinberger also highlights their habit of letting models listen directly to team design conversations:

> “Sometimes we just talk in the team about it and I just press the button and Codex will just listen in on the conversation and just start building while we've been discussing, even though there's like side chatter and blah blah, but like it's clever enough to understand like, oh, this comment was a joke and this is important. So you just like, you can just do stuff.”

---

[42:08](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2528s) Q&A: Overcoming the "Too Busy" Trap with Disposable Internal Tooling

A solutions engineer named Divya steps up to the microphone, asking how someone with a massive workload can realistically find time to design and build custom agents and tools when their daily schedule is already completely full. Steinberger answers by recalling a classic stone-age comic:

> “Do you notice, do you notice meme and like Stone age where they had this car with rectangular, and and then the guy comes along with like the round one and they're like, oh, we don't have time. We're so busy.”

His central argument is that developers must lower their standards regarding the architectural elegance of internal helpers. Small utility tools do not require rigorous design reviews, clean abstractions, or preferred languages; they can be quickly spun up as disposable scripts:

> “So, so the beauty is all these tools, they can be massive slop. It's like you can literally, you can really tell your Codex, hey, look at this session and see the pain that I had to do. Build me a tool that reduces the pain.”

Steinberger points out that he regularly lets models output Go code despite not particularly enjoying the language himself, simply because agents write Go quickly and reliably. By running small, low-stakes agent experiments in an auxiliary terminal window alongside daily tasks, developers reclaim both their operational efficiency and the joy of playful engineering.

## Notable quotes > [00:06](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=6s) Anytime you build software now your job is no longer to, to figure out how you build software faster. Really your job should be, what can I do to help my agent to build software faster and even more? — Peter Steinberger

> [31:46](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1906s) Oftentimes we're not limited anymore by intelligence. The models are good enough now. Everything that we are limited of is our own imagination — Peter Steinberger

> [38:43](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2323s) Listen to your body. Listen when you're annoyed. Annoyance is an amazing signal for, we can automate this. — Peter Steinberger

> [15:19](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=919s) If you don't constantly garden your software, it's dead because dependencies, everything needs updating. — Peter Steinberger

## Mentioned AWS, Azure, Blacksmith, ClawPatch, ClawSweeper, ClickClack, Cloudflare, Cloudflare Workers, CodeRabbit, Crab Box, CrabFleet, Discord, Discrawl, Divya, GitHub, GitHub Actions, GOG, Google Calendar, Gorg, Gmail, Klaus, Line, Mantis, Microsoft Build 2026, Node.js, Octopus, OpenAI Codex, OpenClaw, Peter Steinberger, Release Bar, Telegram, VNC

## Video description The way software is built is changing rapidly. Learn how we build a whole ecosystem of tools to build OpenClaw faster and with more confidence.

Seating for this session is first-come, first-served. Add it to your schedule to plan your day and arrive early to secure a spot.

To learn more, please check out these resources: * https://aka.ms/build26-next-steps

𝗦𝗽𝗲𝗮𝗸𝗲𝗿𝘀: * Peter Steinberger

𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻: This is one of many sessions from the Microsoft Build 2026 event. View even more sessions on-demand and learn about Microsoft Build at https://build.microsoft.com

BRK245 | English (US) | Agents & apps

Breakout | (200) Intermediate

#MSBuild

Chapters: 0:00 - Scaling Automation to Regularly Review and Resolve Issues 00:08:27 - Building Maintainer Dashboards Integrating GitHub and Discord Data 00:13:09 - Creating crawl system for open source issue management 00:14:22 - Distributing Discord crawl data via GitHub for maintainers 00:15:19 - Developing Release Bar dashboard for open source maintenance 00:22:56 - Expansion to cross-platform testing to handle OS-specific bugs 00:24:37 - Addition of VNC integration for full UI bug reproduction and remote debugging 00:33:37 - AI models no longer limited by intelligence but imagination 00:39:26 - Q&A: Balancing ecosystem dependencies and agent interoperability

[Source](https://www.youtube.com/watch?v=o5IQMijn-Ks)

Reading summary

Peter Steinberger argues that modern software engineering requires shifting developer focus from writing code faster to creating infrastructure, feedback loops, and disposable tools that empower autonomous AI agents to construct and verify software end-to-end. Closing the Autonomous Feedback Loop (0:06) Steinberger contends that human developers frequently remain an inefficient bottleneck by manually inspecting interfaces and shuttling context back and forth to language models. Closing the feedback loop requires providing agents with direct self-verification capabilities, such as automated visual controls to capture screenshots, inspect renderings, and test against live integration accounts. This eliminates the need for maintainers to verify basic user flows by hand. Autonomous Backlog Triage and Project Invariants (3:02) Faced with over 10,000 open issues and pull requests on the OpenClaw repository, Steinberger redefines incoming PRs as mere "prompt requests," noting that review costs often exceed the effort of having an agent write a fresh fix from a bug report. To manage this influx, he deployed ClawSweeper, a GitHub Actions workflow using OpenAI Codex that closed approximately 15,000 tickets by evaluating backlog entries against a project specification file, `vision.md`. To keep autonomous triage accurate, Steinberger advises documenting domain-specific constraints in an `agents.md` file to stop models from generating redundant defensive code for edge cases the project intentionally ignores. He recommends having models author and refactor these guideline documents themselves, while using automated crawlers to correlate real-time community chat on Discord with open GitHub issues to identify top user pain points. Dedicated Scaffolding for Agent Execution (20:00) When standard tooling and platform quotas break down under agentic workloads, Steinberger advocates building custom infrastructure tailored specifically to autonomous execution. To bypass individual GitHub rate limits of 5,000 requests per hour during multi-agent issue sweeps, he created Octopus, a Cloudflare Workers proxy that routes read-only queries to a 15,000-request GitHub App token while reserving personal tokens for state mutations. To prevent heavy test suites from overheating developer laptops and stalling pull requests, Steinberger built CrabBox to spin up ephemeral cloud virtual machines with VNC remote access across multiple operating systems. Building on CrabBox, he introduced Mantis to record and evaluate video evidence of bug reproductions and their subsequent patches, and ClickClack—an in-house messaging platform created when automated headless testing against commercial chat services repeatedly triggered bot detection and captchas. Annoyance as an Automation Signal (38:43) Steinberger asserts that personal frustration is the most reliable diagnostic indicator that a routine workflow should be automated. He urges engineers to adopt a lower quality standard for internal utilities compared to production software: internal helpers do not require clean design reviews or preferred programming languages and can be rapidly "vibe coded" by agents as disposable scripts in side terminal sessions. In his experience, specialized tools compound rather than create blocking dependencies. Because modern models navigate Git commit history and perform effective cross-repository pattern matching, previously resolved internal tools serve as direct structural templates, allowing developers to delegate complex architectural tasks with minimal prompt overhead. Steinberger's approach demonstrates that autonomous agents can massively amplify the leverage of a small maintainer team, provided engineers invest in self-verifying test environments and clear boundary documents. However, this workflow relies heavily on continuous token expenditure and accepting disposable, agent-generated scaffolding that may require ongoing pruning as projects evolve.

Source quotes

[0:06] Peter Steinberger: Anytime you build software now your job is no longer to, to figure out how you build software faster. Really your job should be, what can I do to help my agent to build software faster and even more? [31:46] Peter Steinberger: Oftentimes we're not limited anymore by intelligence. The models are good enough now. Everything that we are limited of is our own imagination [38:43] Peter Steinberger: Listen to your body. Listen when you're annoyed. Annoyance is an amazing signal for, we can automate this. [15:19] Peter Steinberger: If you don't constantly garden your software, it's dead because dependencies, everything needs updating.

Original transcript

[0:06] afternoon. All right, let's see [0:16] if we get this working here. [0:18] Yeah. So don't expect a regular [0:21] presentation here. My talk is [0:24] about build the thing that [0:26] builds the thing. I want to [0:28] inspire you a little bit. How [0:31] to approach building software [0:33] in the future. You know, it's [0:36] been not even a year since we [0:39] have coding agents. And it's [0:42] kind of like half a year since [0:44] they got good. And there's no [0:48] stopping done. So a lot of the [0:51] ways we build software in the [0:54] past, I feel are obsolete. And [0:58] we we haven't really figured it [1:01] out yet. I'm curious here, is [1:05] anyone here who maintains open [1:07] source? Yes. How are you [1:12] handling the pain? So. I kind [1:19] of stumbled into into open claw [1:22] people. I saw this all these [1:27] articles where people say, oh, [1:28] Peter made 43 projects that [1:31] failed. And then finally, like [1:33] one worked. And I had to laugh [1:38] at this because none of these [1:41] projects failed. All of these [1:43] projects are things I built for [1:45] myself to work faster to like. [1:54] Mostly actually to like help my [1:55] agents to work faster. Like [1:58] anytime you build software now [2:00] your job is no longer to, to [2:02] figure out how you build [2:04] software faster. Really your [2:05] job should be, what can I do to [2:08] help my agent to build software [2:10] faster and even more? So it's [2:13] all about closing the loop. You [2:16] know? Last year I built, I [2:21] spent a few months building a [2:22] SaaS. And the agent would write [2:29] code and then it would like, [2:30] tell me it's done. And then I [2:32] would look at the browser, I [2:33] would at the agent. It would [2:35] like go in a loop, it would fix [2:37] it. And then eventually we [2:38] would be there. And that was [2:42] until I figured out like, hey, [2:43] no, the agent just can't make a [2:46] screenshot, you know, just like [2:47] humans would like anything that [2:50] where you feel. This is [2:54] something I do repeatedly, you [2:56] should start thinking, how can [2:58] I automate this? How can I, how [3:00] can I help my agent to close [3:02] the loop? So when I guide you a [3:05] little bit through all my [3:06] failed projects. And give you a [3:10] little bit of the, the, the [3:11] thought process, I put on this [3:14] website a little bit. It's pure [3:17] slop, but I think it's kind of [3:18] nice for that. All the little [3:21] tools that we built to build [3:24] Opencore, and I don't think [3:26] it's even complete. So my, my [3:33] first problem was. That open [3:38] cloud client exploded. And then [3:40] the issues and PRS started [3:43] increasing. And I didn't know [3:43] that this number actually, once [3:46] it reaches five K, it stays at [3:48] five K and kind of masks it. At [3:50] some point we had like more [3:51] than 10,000 issues because [3:53] people don't they don't look, [3:54] they just like, oh, I mean, [3:57] people, the agents don't look, [3:59] you know, it's like, let's not [4:00] pretend any of that is written [4:02] by humans anymore. And I know a [4:05] lot of open source people get [4:06] really mad at all the slop [4:08] they're getting, but I feel. [4:12] You can't fight the world. You [4:14] have to figure out a way how to [4:15] deal with that. So I see [4:18] differently. Yes, there's a lot [4:19] of slop, but the real problem [4:22] is that it's still called pull [4:25] requests. I call it prompt [4:27] request because what you really [4:29] want is anytime someone opens [4:31] an issue, a PR, it's just a [4:33] signal that somebody is not [4:34] happy with what you built and [4:37] is asking for a change. [4:39] Actually, like these two things [4:40] are like interchangeable. Like [4:43] mostly doesn't matter if it's [4:45] an issue or PR. Sometimes an [4:46] issue is actually nicer because [4:47] I can work fresh if it's a pull [4:50] request. And like somebody used [4:51] Claude and like I, it's [4:54] probably takes me more time to [4:55] like make my agent read the [4:57] code and then like at the code [4:58] and then rewrite the code. But [5:02] the first thing, the first [5:03] thing I had to had to solve was [5:05] how do I get this number down? [5:07] And I tried, obviously you try [5:09] tools, right? I tried code [5:12] Revit, I tried, I tried a bunch [5:13] of tools and they were nice, [5:16] but they ultimately would just [5:17] give me a review. I wanted more [5:20] I reviews are nice, but I kind [5:22] of want something that [5:24] automatically groups and closes [5:26] the issues and I couldn't find [5:27] anything good. And in this day [5:29] and age you don't actually need [5:31] to wait anymore to like find [5:33] stuff. You can just build your [5:36] own stuff. So I built Cloud [5:37] Sweeper again, totally sloppy [5:40] website because the website [5:41] doesn't matter. But it's [5:47] actually really simple to build. [5:49] Like, I don't know what all [5:50] these tools were their motives [5:52] because ultimately, anytime [5:54] someone opens an issue or a [5:57] pull request, I just start a [5:59] codex in the cloud. We are a [6:02] GitHub action and have a nice [6:07] prompt that explains the codex. [6:08] Like what is, what's the goal [6:11] of the project? What's our [6:13] vision? I do this now for any [6:15] of my open source projects. I [6:17] have a vision.md, and part of [6:20] my standard prompt is that my [6:22] agent reads that file. And in [6:23] that file is kind of like what [6:25] stuff that yes, we totally want [6:27] what stuff that definitely we [6:29] don't want. And based on those [6:31] rules. Klaus Reaper will run. [6:35] And we'll just either comment [6:37] on the issue or close the issue. [6:40] And through that, I was able to [6:41] like close around 15,000 issues. [6:45] I mean, yes, I also operate [6:46] under the assumption that have [6:47] infinite tokens. You could [6:49] definitely do this a little bit [6:50] more efficient, but we didn't [6:52] stop there. Once we had a [6:55] system that could close issues. [6:58] Of course, we also wanted to [6:59] review issues. And it's not a [7:06] static process right at our [7:09] scale. Any time a fixed lens on [7:15] main, there's a good chance [7:17] that I solved an issue that [7:19] I've never read because there's [7:20] like many thousand issues open. [7:22] So what we do is actually we [7:24] rerun Klaus Reaper on every [7:26] issue and pull request at least [7:28] once a week, depending on how [7:30] many tokens you want to spend. [7:32] You could do it every day. So [7:34] what regularly happens is, is [7:36] that I, I have an integration [7:39] on discord. And then and then [7:41] the report will tell me, hey, [7:42] close this issue from three [7:44] months ago that I've never seen [7:46] just because, because Codex [7:48] would run on that issue. And [7:50] then a person that wrote an [7:51] issue will get a, will get a [7:53] response. Hey, we fixed this on [7:55] main. And again, it's something [7:56] I don't have to deal with it. I [7:58] automated it, it solved the [8:00] problem. I also built a nice [8:04] dashboard. So see, right now [8:07] there's only 1818 Codex working [8:11] right now. And this is how it [8:15] looks. It's just one long [8:18] comment that explains what it [8:19] does. And it gives people array [8:23] of issues that that they need [8:25] to solve with typical stuff. [8:30] The next thing I, I had to [8:32] solve now, now the project [8:34] gained momentum. I added more [8:36] maintainers and some people [8:39] turn out to be really good. [8:41] Some people, they kind of [8:43] showed up for a few days and [8:45] then disappeared. And I was [8:47] very liberal in the beginning [8:49] of adding people and I lost a [8:51] little bit overview. So I [8:54] wanted a dashboard. So I just [8:56] built a dashboard. So now we [8:58] have reports of open cloud. [8:59] This is not public for everyone [9:01] because it because you want a [9:03] little bit more data. Actually, [9:05] I'm not just looking at GitHub [9:07] issues. I also look at all our [9:08] discord stuff. So to me, being [9:12] a maintainer doesn't just mean [9:14] you push code. Domain also [9:16] means that you participate in [9:17] discussions, which we use [9:19] discord for that. So so this [9:21] system integrates all data from [9:22] discord. Now that brought me to [9:26] the next problem. How do I get [9:27] the data from discord? Well, [9:32] let's just build this crawl. [9:36] Like in this day and age, any [9:37] problem is just a prompt away. [9:39] Like building this thing is [9:41] never the hard thing. [9:42] Maintaining this thing is [9:43] actually the hard thing. But [9:46] again, even even on even now [9:48] when I, when I maintain stuff, [9:51] I don't think about the project [9:54] level anymore. I. It looks like [9:59] this. I basically, I go one [10:02] level beyond. I go into my [10:04] projects folder and ask Codex, [10:05] hey, let's triage. What are the [10:07] things that I should work on? [10:09] And it just finds a list of all [10:11] the, all my projects and open [10:13] issues. And then I basically [10:14] say, yes, do all of them [10:16] because I already have. My [10:20] slope is better than your slope. [10:24] No, I don't do all of them [10:26] because there's still sometimes [10:28] people report weird stuff and I [10:30] don't fully trust my agents. So [10:32] I want to I want to read that. [10:35] Right? So again, I'm a lazy [10:38] person. I could click on each [10:40] of those URLs, but it's kind of [10:42] like slow. So I built something [10:45] where you just select a text [10:48] and I built myself an issue [10:49] browser because again, it was [10:52] like, like there was like just [10:54] a few clicks too much. It [10:55] annoyed me a little bit too [10:57] much. And the critical, the [10:58] critical thing that you are is [10:59] like, your time is limited. So [11:01] you need to build the tools to [11:03] build the tools. So again, I [11:04] built this little slot, right [11:05] issue browser where I can copy [11:09] by now any format because it [11:11] turns out there's Clankers [11:13] given an almost unlimited [11:16] amount of versions, how to [11:17] format things. So I anytime [11:19] something didn't work, I just [11:21] fixed it to make it work. And [11:23] now I basically parses all the [11:24] formats that it ever seen. And [11:26] I can very quickly click [11:28] through this list. In many [11:31] cases, the agents are already [11:33] so good because I have this [11:34] vision in D file that the [11:36] selection is mostly right. And [11:38] I usually say, do those eight [11:39] don't do those two because I [11:41] know this needs actual time. [11:43] You know, anytime you work with [11:46] agents, you want to optimize [11:49] what they can do autonomously. [11:51] And it doesn't matter how long [11:52] it takes. You want to, you want [11:54] to really manage the focus time [11:58] where you want to be in the [11:59] loop for the things that are [12:01] important and all the things. [12:03] What is that even gmail draft [12:05] create? Yeah, that's probably a [12:07] nice feature. Oh yeah. Let's [12:09] look at this. This is a nice [12:11] feature on Gorg. Another thing [12:14] that I built because I got [12:15] annoyed that Google doesn't [12:17] have a CLI, and I want my [12:18] agents to access all my Google. [12:20] Yes, like Codex has gmail [12:23] integration and now. Calendar [12:27] integration, but only for one [12:29] account against an annoyed me. [12:31] So I just built a CLI that can [12:33] do everything anytime [12:33] something's missing because I, [12:35] I control the whole stack. My [12:38] agent controls the whole stack. [12:39] So it can just fix itself like [12:41] so like you own the whole [12:42] universe. Anything is fixable. [12:44] And this is all these issues [12:46] are super nice because this [12:48] would eventually add a feature. [12:51] Codex can not only build it. [12:55] But also end to end verify it [12:57] because I of course have like a [12:59] demo account on GOG so it can [13:01] end to end test the whole [13:02] feature. And I basically have [13:03] to do nothing other than saying [13:05] yes, and I can ship it and I [13:07] can ship it with confidence [13:08] because it's end to end tested. [13:15] So then I built this role for [13:16] the maintainer reports. Turns [13:18] out this is like this solves a [13:21] much bigger problem because [13:23] when you do open source. And [13:27] you have multiple thousands of [13:28] issues, it's very hard to know [13:30] what's actually the thing that [13:31] I should work on. So now I, I [13:35] hooked up this crawl to Codex [13:40] and it will, it will go through [13:42] all my channels and I can [13:43] basically ask it, what are [13:44] people screaming the most [13:47] correlated to open issues and [13:49] PRS and find me the top five [13:50] things I should work on. It [13:54] should like match it with my [13:55] vision document, pick the stuff [13:57] that you can work autonomously, [13:58] and then basically it can run [14:00] off and work for doesn't matter [14:02] how many hours you can, you can [14:05] parallelize it. And then what [14:11] you always should know when you [14:13] build, even when you highly [14:14] automate with agents, is you [14:15] should have this. The you [14:17] should have the overview of [14:18] your system so you will know [14:20] which issues are like. Probably [14:21] fine, probably fine. Oh, this [14:23] is hairy. I need to look at [14:24] that more specifically. So I [14:28] built this crawl and then of [14:30] course all my maintenance [14:31] wanted this crawl. But again, [14:33] it there's no, there's no [14:34] public way to get all this data. [14:36] So I used a bot multi that was [14:38] in discord to like use him to [14:41] access all the data. But then [14:43] how do you distribute it to, to [14:45] the other maintainers. Well. [14:49] What's the, what's the simplest [14:51] backup system there is GitHub. [14:56] So, so all the, all the data [14:59] from our discord is not just in [15:02] some discord store, just in [15:06] some, some format that [15:09] serializes the database. And [15:11] anyone that uses discord can [15:13] just use this as an alternative [15:15] data server if they don't have [15:16] a bot token, because I didn't [15:18] want to give my bot token to [15:19] all the people, this is good [15:21] enough. Again. Problem solved. [15:24] Now, if you're like me and [15:28] you're maintaining a lot of [15:30] open source projects, what I [15:32] always wanted was a way to have [15:37] this little dashboard of like, [15:39] what actually needs my [15:40] attention? Where are the most [15:41] pull requests? Should I do [15:43] another release? But there [15:44] wasn't really anything good. So [15:47] I just built this bar and now I [15:49] can filter on this. [15:52] Specifically how many days [15:53] since the last release. So [15:55] whenever I do like my little [15:56] fixes and software, if you [15:58] don't constantly garden your [16:02] software, it's dead because [16:05] dependencies, everything needs [16:06] updating. Systems change. So [16:08] now I have a nice dashboard to [16:09] know when should I do another [16:11] release? And I can basically [16:12] filter on stuff that need [16:15] attention. And again, the [16:18] beauty of all these little, all [16:19] these little tools that you [16:22] build to help you build the [16:23] thing, they're not system [16:26] critical. So all these tools [16:29] are. Yes. I actually didn't [16:31] look at any of that code [16:32] because it doesn't matter, like [16:34] for, for stuff that a lot of [16:38] people run open claw, I read [16:40] more code than people would [16:41] think. But for all these little [16:44] helper tools. Vibe anyway, and [16:48] if maybe you use them a lot, [16:50] then you can iterate on them. [16:51] Maybe they don't work out and [16:53] no cost. Yeah, we talked about [16:58] a little browser here. There's [17:01] another thing now that I. I use [17:04] more and more to work on things [17:06] in parallel. I hit another [17:09] problem. There's only so many [17:12] GitHub tokens that you can use. [17:14] And because we have so many [17:16] issues, and I instructed my, my [17:17] agent to look at find related [17:21] issues, I always ran out of [17:23] tokens. And I know that I, I [17:26] don't know if the GitHub people [17:27] are here like they are good [17:28] friends and I have someone [17:30] literally clicking a button [17:31] once an hour to reset Peter's [17:33] rate limits. But the problem is [17:36] these people need sleep. It's [17:37] not clankers. So so when I was [17:39] working weird hours, like I [17:41] wasn't around, I couldn't ping [17:42] her to reset my tokens. Again. [17:49] I find being annoyed is the [17:51] very best way to solve problems. [17:55] So I built octopus and you're [17:58] like, my feelings were like [17:59] transported into this this [18:00] things feelings. But the [18:05] principle is the principle is [18:07] very, very easy. Octopus. Let's [18:13] see if this actually works. But [18:18] again, the principle is very [18:20] easy. You as a as an individual [18:24] have 5000 API tokens per hour. [18:27] But if you install a GitHub app, [18:29] it gives you 15,000 tokens per [18:31] hour. So I just built this [18:33] thing that shims GG and then [18:38] calls this thing on Cloudflare. [18:40] If it's a read only if it's if [18:42] it's a non mutating read only [18:43] action from a public repository, [18:45] it can use the GitHub app and [18:48] then use the token from the [18:49] GitHub app and not my token [18:51] only if it's something that's [18:52] mutating or it doesn't fit that [18:54] that match it will use my token. [18:56] Bam! Again. Problem solved. And. [19:03] All the tools are so good now, [19:07] especially especially [19:08] Cloudflare, I didn't have, I [19:10] just explained the agent what I [19:12] want. And then with the [19:17] combination of really smart [19:20] models, browser use, you don't [19:22] have to do anything anymore. [19:23] Like literally, you could [19:25] configure Cloudflare itself. [19:26] You probably still want to have [19:27] a look at it because you want [19:30] to review if it actually [19:31] doesn't mess up your security [19:33] settings. But the idea is any [19:39] time, any time you feel you [19:40] feel pain, think of go, go, go, [19:44] go out and think about what, [19:45] what can I do to like get rid [19:47] of that pain? And then usually [19:51] it will not only help you, it [19:52] will help your whole team. And [19:54] it's, it's a and it compounds [19:56] all the little tools are built. [19:57] Help me to build faster. [20:00] Another, another problem I had. [20:04] How are we on time? Actually, I, [20:06] I think we're good. Who here [20:13] had this problem? Who's crazy [20:15] enough to run like more than, [20:16] let's say ten coding sessions [20:18] except for two? Yeah. You guys [20:22] need to level up. So. The [20:29] project grew. It kind of [20:32] exploded. The code and the [20:34] features grew. People reported [20:37] bugs, our test case grew and [20:40] the tests were becoming more [20:42] and more pain. You know, that's [20:44] that's the stuff kind of creeps [20:45] in. It takes like, it took like [20:47] 10s and it took 30s and took a [20:49] minute. Then it took two [20:51] minutes, then it took five [20:54] minutes. Then I optimized it [20:55] for my Mac studio to work in [20:57] one minute. And everybody [20:58] complained that that like, [20:59] their computers were melting [21:01] because they didn't have like [21:02] 40 cores. So we had this shared [21:06] painting again. And around that [21:10] time, I saw that. Oops, I saw [21:15] that. What's it called? [21:18] Blacksmiths. They, they started [21:20] something. Test box concept. So [21:26] the idea is. Your computer [21:32] power is limited. So you want [21:34] to give the agent a little box [21:36] where it can run the tests or [21:38] do whatever it needs to be CPU [21:41] intensive. And what's beautiful [21:45] for a while. But the problem [21:46] was their stuff was very new [21:48] and it was like. It was down [21:51] for at least two hours a day. [21:53] And by then, our team was so [21:54] dependent on it that like, [21:55] nothing would get done anymore [21:57] in those two hours because CI [21:58] would break down and then the [22:00] test would break down. So I was [22:01] like, this is fine. I'll just [22:03] build something that wraps it [22:04] with alternatives. And that's [22:06] how Crab Box was born, because [22:08] you have to keep it with the [22:10] weird crustacean names. Again, [22:13] very simple, a very simple [22:15] concept. You spin up a little [22:16] infernal machine in the cloud, [22:20] and then it syncs your changes [22:22] in there. Your reports are [22:23] already checked out. It runs [22:24] the tests in there, and then [22:27] you pay the all the CPU stuff [22:29] that you would need to run the [22:31] test on your machine. You can [22:32] use really beefy hardware. You [22:34] can massively parallelize. [22:36] Everyone's happy. And then this, [22:40] this tool quickly became like [22:44] universally loved because it [22:47] would it would use our main [22:50] provider. And if that runs down, [22:52] it would just use AWS or Azure. [22:55] And then people started making [22:56] pull requests. And now it [22:57] supports like 20 providers. But [23:01] it didn't stop there because I [23:04] recognized this is actually way [23:05] more useful because oftentimes [23:07] we have bugs where, oh yeah, on, [23:10] on windows there is a Eprom [23:12] issue, even though we use node [23:14] and it should be cross-platform, [23:15] there's like always little [23:17] weird details where [23:19] cross-platform is a lie. And [23:21] then in the past, I would just [23:23] sometimes just like wipe it [23:25] because I'm too lazy to spin up [23:27] my windows computer, I could [23:29] spin up a VM, but then how do I [23:31] get it into the VM? And then [23:33] kind of like have to do get [23:34] cortex into the VM because I [23:36] can't, I can do git anymore [23:38] without an agent because it's 3 [23:39] a.m. and I'm tired. You know, [23:41] you have to fight friction like, [23:44] like ultimately we are being [23:46] lazy is good. So I noticed that [23:50] I'm, I'm pushing slop because [23:53] I'm too lazy to come up with a [23:56] good testing system. So I added [24:00] cross-platform support to grab [24:01] box. And now anytime there's [24:03] something that's hairy, I just [24:05] tell my agent, hey, test it on [24:07] all the races. And it will not [24:09] just create one box, it will [24:10] create three boxes. Or maybe, [24:12] maybe there's a weird issue [24:13] that's only on Red hat. Sorry, [24:15] Sally, I love Red hat and it [24:17] will spin up four boxes and you [24:18] can like, you can scale it [24:20] infinitum and like increase [24:22] your confidence that your code [24:23] actually works. And it can do [24:25] that in parallel, which is [24:27] really amazing. And then once I [24:30] built that got me thinking like. [24:34] I have this bug, but it's like [24:37] actually only on windows on [24:39] this browser. So I kind of need [24:41] UI for that. People already [24:43] solved that. Just add VNC into [24:45] it. So now I have a crap box [24:48] that like can recreate exact [24:50] scenario, and then my agent [24:52] opens a browser with VNC and in [24:54] there there's already a browser [24:55] waiting in a terminal and maybe [24:57] telegram or whatever I need so [24:59] I can recreate the issue end to [25:01] end and test it. That was [25:03] amazing. But still, like I had [25:06] to click myself. Oh, you know, [25:12] like laziness is good. So that [25:15] got us the idea. Wouldn't it be [25:17] cool if the Clanker could do it? [25:19] Of course they can do it. They [25:20] have computer vision models [25:21] could really good. So so now [25:25] Crap box also has a little [25:26] little tools for screenshot and [25:28] click and type. And now I can [25:31] actually, my agent can have its [25:32] own little box with UI that it [25:35] can use computer automation in [25:36] to control the UI. I was still [25:41] not good enough. Like I don't [25:45] want to watch the clanker. I [25:47] just want to see video. So what [25:49] we built next was mantis, where [25:50] now we have an agent that I can [25:53] ping on a, on a pull request [25:56] that'll spin up boxes, make a [25:59] video of the bug, fix it, make [26:01] a video of the fix and actually [26:03] look at the video and verify it. [26:05] And then all I get, I can [26:07] basically see an issue. I can [26:09] ping Klaus to fix it, and it'll [26:12] create the PR mantis will do [26:14] the visual verification, and [26:15] all I have to do is like, look [26:17] at the video and press merge. [26:21] And that's all built based on [26:23] me being annoyed. The next [26:29] thing I noticed. Like you would, [26:35] you would use Codex, you would [26:36] like do the prompt and then you [26:39] would do slash review because [26:41] you're like, you're kind of [26:42] unsure. Maybe maybe I should [26:43] review it. And then Codex would [26:45] find three issues and they're [26:47] like, damn, they're like, all [26:48] of them sound real. So I fix it [26:51] and run off for ten, 20 minutes, [26:54] run all the tests again, and [26:55] you're like, oh, is it really [26:57] all fixed? Let's just let's [26:59] review again and will run for [27:01] ten minutes. It would find two [27:03] new issues because it can't [27:05] show me all the issues at once. [27:07] It's just like, I can't do it. [27:09] You type fix again, you wait 20 [27:12] minutes, you do something else, [27:14] go back and like, yeah, but is [27:16] it really fixed this? Let's [27:17] review again. It would run for [27:20] 20 minutes, 30 minutes. It [27:21] would find another issue. And I [27:23] don't know, have you done this? [27:25] Like I did this. And then there [27:27] was one time I did this ten [27:30] times. It was the whole day. [27:32] Like I added support for line. [27:35] I don't even use line, but like [27:38] somebody was very passionate [27:39] about it. I wanted to get it in. [27:41] And then all day I did this [27:42] review and I was like, is it [27:44] enough? Is enough? Like, like, [27:47] I feel like I'm the clanker. So [27:51] I built auto review and in the [27:54] beginning I thought, oh, this [27:56] is kind of annoying. I probably [27:57] need hooks or like, maybe I for [27:59] Codex because Codex is open [28:00] source. I could totally do that. [28:02] But that turns out it's [28:03] actually way easier than that [28:05] because agents are really good [28:06] at following instructions for [28:08] last at least 4 or 5 months. So [28:11] now there is one line in my [28:14] agents.md that says, before you [28:17] commit or land a PR, if you [28:20] haven't done auto review it, [28:22] run auto review. And again, you [28:25] know, review. Some people say, [28:27] look at the code with fresh [28:29] eyes. Yeah, good luck with that. [28:31] You know, like context doesn't [28:33] really work that way. But. But [28:44] you can just call. I don't know [28:47] if you find the but you get the [28:50] idea. You can just call the [28:52] agent that you use as CLI with [28:55] the commands you should do. So [28:57] you get a fresh context, but [28:58] just calling like I Codex calls [29:02] Codex to like run auto review. [29:05] There was a bug sometimes. [29:06] Sometimes Codex was calling [29:07] Codex, calling codex and [29:08] calling colleagues. And it took [29:09] very long. I fixed that, but [29:12] now I have a very reliable [29:15] system that automatically runs [29:18] many rounds of review and. I [29:25] also tuned the file to explain [29:29] the agent. Not every review [29:32] thing is valid. You know, [29:35] sometimes. They would find very [29:39] weird edge cases. So it's a [29:41] little bit up to you to explain [29:43] the agent, the, the, the, the [29:48] conditions, what you think is [29:50] valid running, for example, in [29:51] our case, we don't expect that [29:54] you randomly edit plugin files [29:56] while the system is running. [29:57] The agent doesn't know that. So [29:59] like by default it would write [30:00] very defensive code to like [30:02] deal with all of those things. [30:04] So, so it's always a good idea [30:06] to write all these invariants [30:08] into your agent file. That's a [30:11] good idea in general because [30:13] you know that, but the clanker [30:16] doesn't. And every time you [30:17] start a new session, Clanker [30:19] memory is pretty much gone. So [30:22] you the more you tell it about [30:26] your project, the better the [30:27] time you'll have. I went all in [30:30] from very long agent files to [30:32] like very short ones, and now [30:34] mine are kind of long again, [30:35] but with meaningful stuff. [30:39] Every time you do that, it's [30:41] also don't don't write the text [30:44] yourself. You know, none of [30:45] this is written yourself. You [30:47] don't know how to write to the [30:48] client in a way that Clanker [30:50] writes to a clanker. So like [30:51] you're going to explain what [30:52] you want from the agent, and [30:54] the agent will write it in its [30:56] own words. And then every once [30:57] in a while, go into meta mode [30:59] and just ask the agent, hey, is [31:01] there anything that's confusing [31:02] in your agent file? And then [31:04] they will usually tell you, [31:04] yeah, well, this is confusing [31:06] because here it says this and [31:07] here it says this and then [31:09] cleaned it up. Clean up any [31:10] confusion because that's just [31:11] token burner. And we'll build a, [31:13] we'll give you a worse results. [31:17] But I think if, if you only [31:20] take about two ideas from this [31:22] talk, try auto review and try [31:23] grab box. It totally [31:25] revolutionized how we build [31:27] software like my, my trust in [31:29] in. Prompting and shipping is [31:34] much, much higher with that [31:36] stuff. There's another thing [31:46] when we get PRS. I would love [31:52] to know how people came to that [31:55] PR like. The good and the bad [32:00] thing with agents is like a one [32:02] line prompt can create ten [32:04] lines, ten, ten, ten pages of [32:07] content. And you don't always [32:10] know how much, how much work [32:11] was put in. So very early on in [32:14] our contributor guidelines, I [32:15] asked people to please send me [32:18] the prompts and nobody would do [32:20] it because there was just no [32:21] good system. You know, I was [32:24] fighting against people's [32:25] laziness. Of course they will [32:26] not do it. That one took me a [32:29] long time to figure out, but [32:30] now I built a skill. If you use [32:33] open claw and you contribute, [32:34] you send a pull request to open [32:36] claw. Your agent will ask you, [32:38] hey, do you want to send a a [32:41] privacy filtered version of the [32:43] prompt? If you do that, we will [32:45] look at your PR faster and then [32:48] the user can type yes. And then [32:50] the agent will upload, will, [32:52] will, will look on the disk, [32:54] will find its own transcript, [32:56] will cut the transcript to the [32:58] interesting parts, and then [33:01] we'll add those transcript to [33:03] the pull request. And suddenly [33:05] I have a much more interesting [33:07] data point about how much [33:09] effort was put in, because [33:10] that's ultimately what you're [33:12] after. Like I can, somebody can [33:14] say, fix issue number and send [33:17] PR and it's like four words, [33:20] five tokens. And or somebody [33:22] could actually spend a few [33:23] hours and like have an actual [33:25] discussion and you don't always [33:26] know. So this has been [33:27] extremely helpful in, in, in [33:29] figuring out what's total slop [33:32] and what's kind of good slop. [33:35] And again, like, I could have [33:37] done this half a year ago, just, [33:40] I just didn't come to mind that [33:42] actually it's that easy. [33:44] Oftentimes we're not limited [33:46] anymore by. By intelligence. [33:51] The models are good enough now. [33:53] Everything that we are limited [33:54] of is our own imagination. How [33:57] what you do with it and how to [33:59] build things. Now, I built auto [34:02] review and I was like, oh, this [34:03] is amazing. And it made me [34:05] really bad. I felt really bad [34:07] for all the code that I [34:08] committed without review. Now [34:11] that I, I saw how much stuff [34:14] the agent finds each time and I [34:16] saw GPT five is good, but and [34:18] yes, it is really good, but [34:21] still, there's so many boo boos [34:24] like the. Especially on large. [34:27] There's just mistakes and like [34:28] difficult things. The agent [34:29] will only find. After doing [34:32] multiple rounds of reviews. So [34:34] it got me worried like, oh, I [34:36] wish I had done this earlier, [34:39] but you can't really. There's [34:41] no slash review for a project [34:42] that has a million lines of [34:43] code. There's too much like [34:46] what should the agent focus on? [34:48] There's too much for, for. The [34:53] context. It annoyed me. You [34:59] know what, what, what happens [35:01] when I'm annoyed? So I built [35:03] cloud Patch. And the principle [35:06] is very simple. Like, oh, you [35:07] have a big project. Let's take [35:09] it apart into like 50 little [35:12] sub things, subsections by [35:14] feature or whatever, whatever [35:15] else you can, like, you can [35:17] separate it and then have the [35:19] agent review each of those [35:21] things separately. And then you [35:24] can, you can download this, run [35:25] this on, on your project. And I [35:27] guarantee you you'll find a lot [35:28] of interesting things. It's not, [35:33] it's not quite as perfect as [35:36] paired change because you would [35:38] have to probably scale it up to [35:40] thousands of code sessions, [35:44] even with unlimited token. [35:45] That's a problem. But it's [35:47] really good at like cleaning up [35:49] code bases. This is something [35:57] that's not fully finished yet. [36:02] But, you know, I got I was so [36:04] into crap box. The more I [36:11] worked with the team, the more [36:14] did you had this where you ask [36:17] your teammate and your teammate [36:18] will pass stuff from your from [36:19] the agent into chat. And then [36:21] you would you would comment on [36:23] it and then they would pass [36:24] stuff into into the coding [36:25] agent and they would pass stuff [36:27] back into the chat. I'm like, I [36:29] don't like, like, like I, [36:33] you're just intermediary. I [36:34] want to talk to your clanker. [36:37] So, so that's the last thing I [36:40] built this, this crap fleet, [36:41] which is basically we already [36:44] have crap box where the tests [36:45] run. Couldn't Codex run in [36:47] there as well. Turns out, yes, [36:49] it can. And then you have [36:52] multiplayer codecs. So so so [36:57] now if I work on something and [36:58] I want a second opinion, I can [37:00] just send a link to one of my, [37:02] my maintainers and like, he can [37:04] look at the session, talk to [37:05] the session, take over the [37:07] session. It's like multiplayer [37:10] clanking. There's more stuff. [37:18] We kind of rebuild slack. Yeah, [37:22] I don't know what happened [37:23] there, but we ran into this [37:26] issue where we had all these [37:31] model message channels, and you [37:36] often want to test it end to [37:38] end, but there was sometimes [37:41] features are very difficult to [37:44] understand with all the complex [37:45] API. You kind of want to see [37:46] see it in the UI. And I built [37:49] this automation, but man, [37:51] anytime I logged in, there was [37:52] like a Cloudflare capture or [37:54] like, like all these, all these [37:57] companies really don't like it [37:58] if you spin up something in a [38:00] box, especially not if it's [38:01] Linux based and there's nobody [38:02] logged in and it just feels [38:04] like it feels like someone's [38:05] trying to hack the system. [38:06] We're just trying to like make [38:08] it work. But I can see why [38:10] people don't like it or why [38:11] their systems would freak out. [38:13] So we just built our own [38:14] messaging platform to like test [38:16] to like, have real end to end [38:18] tests with something that we [38:20] have under control. And it got [38:21] a little bit out of hand. And [38:23] now we kind of have a backup [38:25] chat system that works. [38:27] Whenever our team is down, it's [38:30] called click clock. There's so [38:43] much more. Where were we? I [38:49] think I showed you, I showed [38:50] you a good part. Yes, there's [38:52] of course there's way more [38:54] crawlers. But my, my big, my [38:58] big takeaway for this is listen [39:02] to your listen to your body. [39:03] Listen when you're annoyed. [39:04] Annoyance is an amazing signal [39:06] for. We can automate this. And [39:09] then anytime you automate [39:10] something, it compounds all [39:12] these little tools. They're [39:13] working together, right? I, I [39:15] can only do the reports because [39:16] I use disk crawl and I could [39:18] only use crab fleet because I [39:21] have crab box. Even though all [39:22] of these are weird names, they [39:24] are incredibly helpful in [39:26] helping us ship a massive [39:28] codebase with a very small team. [39:30] And with that, I do think we [39:33] have a few minutes of Q&A. [39:34] There is a microphone. There's [39:38] a microphone here, so should. [39:51] As you're building. So. It's [40:00] making some noise. Is it [40:02] working? Yes. There we go. [40:03] Sorry. As you're building this [40:06] ecosystem and becoming more [40:07] like enterprise, quote unquote, [40:09] like, do you feel that there [40:11] are times where you want to [40:14] build something, but the [40:17] dependency on the other things [40:19] get in the way of you improving? [40:25] Hasn't happened so far yet. [40:29] It's less dependencies. It's [40:30] more like all those little [40:31] tools compound and make [40:33] building new tools faster in [40:36] many ways. Sometimes I use one [40:38] tool in another tool many other [40:41] times, and that's a general [40:42] trick. When you work with [40:43] agents, if you've sold [40:45] something once, you can just [40:45] tell the agent, look there, I [40:47] already solved this there. They [40:48] are really good at pattern [40:50] matching. So I would actually [40:51] say it really compounds. [40:52] Sometimes something doesn't [40:53] work out, but then you learn [40:55] from it like, like the the [40:57] token thing. Where was it? Too [41:02] many things. The token thing, [41:06] the first version I built was [41:08] just on my machine. That was [41:14] really awkward with the GitHub [41:17] app. So I iterated a bunch on [41:19] it, and eventually it was like, [41:20] I ripped it all out and I just [41:22] told my agent, remove this [41:23] feature here, rebuild this with [41:26] this new idea, but take that [41:28] part from that tool and like [41:30] push it into your tool. And [41:31] then like, basically you save [41:33] the prompting, right? Because [41:34] you already, you can just tell [41:36] the agent, look around this, [41:37] look at the git history. [41:38] They're very smart at figuring [41:40] out the context. Even better, [41:41] if you just press the, the, the [41:44] button and you just like, you [41:45] just talk about it sometimes. [41:47] Sometimes we just talk in the [41:48] team about it and I just press [41:50] the button and Codex will just [41:52] listen in on the conversation [41:53] and just start building while [41:55] we've been discussing, even [41:56] though there's like side [41:57] chatter and blah blah, but like [41:59] it's clever enough to [42:00] understand like, oh, this [42:02] comment was a joke and this is [42:03] important. So you just like. [42:07] You can just do stuff. Thank [42:08] you. All right. Hi, Peter, this [42:14] is Divya. My biggest question [42:17] is this is awesome. But imagine [42:20] you have a billion things on [42:22] your plate as a solutions [42:24] engineer. And I love the [42:26] concept of building my own [42:27] agents, but I simply don't have [42:29] the time to actually go ahead [42:32] and build tools to help me be [42:34] more efficient and save time. [42:36] Do you notice, do you notice [42:38] meme and like Stone age where [42:40] they had this car with [42:42] rectangular. And and then the [42:46] guy comes along with like the [42:47] round one and they're like, oh, [42:49] we don't have time. We're so [42:50] busy. So, so the beauty is all [42:55] these tools, they can be [42:56] massive slop. It's like you can [42:59] literally, you can really tell [43:00] your cortex, hey, look at this [43:02] session and see the pain that I [43:04] had to do. Build me a tool that [43:06] reduces the pain. This is the [43:07] rough idea. You don't even have [43:09] to give it a programing [43:10] language. Nothing matters for [43:11] like those little tools like I [43:12] use go. I don't even like go. [43:14] Like I know, I know that my [43:16] agents are fast and go, so who [43:18] cares? And then just like, do [43:19] it on the side, you know, it's [43:21] like there's always space [43:22] somewhere in the corner for a [43:23] little window for like [43:24] experiments. And it also takes [43:26] a fun a little bit back. And [43:27] then it, it really compounds.