Microsoft Developer
Software engineering must shift from manual coding to building disposable infrastructure and feedback loops that empower autonomous agents to verify, repair, and ship software end-to-end.
AI-assisted summary of Build the thing that builds the thing | BRK245 — original source. Check the original for full context.
Source published:
Key ideas
- Peter Steinberger argues that software engineering has inverted: developers should stop optimizing their own coding speed and instead build tools, feedback loops, and verification environments that empower AI agents to operate autonomously end-to-end. (0:06)
- OpenClaw handled an issue backlog exceeding 10,000 items using ClawSweeper, an automated triage system running OpenAI Codex over GitHub Actions that evaluated tickets against a core vision.md file and closed roughly 15,000 issues. (3:02)
- To bypass the 5,000 requests-per-hour personal GitHub API cap during multi-agent sweeps, Steinberger deployed Octopus, a Cloudflare Worker shim that routes read traffic through a 15,000 req/hr GitHub App while reserving personal tokens for mutating actions. (17:09)
- Visual and cross-platform verification was solved through CrabBox and Mantis, which spin up remote cloud environments across operating systems, record before-and-after video proof via VNC, and allow agents to visually confirm fixes before PR merging. (24:48)
Detailed notes
## Background Published on YouTube by Microsoft Developer on June 4, 2026, this breakout session from Microsoft Build 2026 (BRK245) features Peter Steinberger discussing the rapid transformation of software development. The presentation highlights the custom ecosystem of tools his team created to build and scale OpenClaw faster and with greater confidence using autonomous AI agents.
## TL;DR Peter Steinberger presents a paradigm shift in software development, arguing that engineers should stop optimizing their own coding speed and instead design feedback loops that empower autonomous coding agents to work end-to-end. Facing a deluge of tens of thousands of GitHub issues and pull requests on OpenClaw, Steinberger engineered an interdependent suite of disposable internal tools—such as ClawSweeper for continuous Codex-driven issue triage, Octopus for bypassing API rate limits, CrabBox for cross-platform cloud virtualization, and Mantis for automated video bug verification. Across the session, he illustrates how using personal developer annoyance as an automation signal allows a small team to build compounding toolchains, conduct whole-codebase audits, and bypass traditional operational bottlenecks.
## Atmosphere & style The session has an irreverent, fast-paced, and highly candid atmosphere, with Steinberger deliberately avoiding standard corporate slide decks in favor of pragmatic developer war stories and live dashboard walk-throughs. His delivery is conversational and wry, filled with self-deprecating humor about runaway recursive agent loops, 'clankers,' and accidentally coding a full backup chat application out of frustration with Cloudflare captchas. The talk feels less like an executive conference keynote and more like an animated hackathon retrospective that leaves engineers energized to build scrappy, high-leverage automation.
## Bulletin - [00:06](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=6s) Peter Steinberger argues that software engineering has inverted: developers should stop optimizing their own coding speed and instead build tools, feedback loops, and verification environments that empower AI agents to operate autonomously end-to-end. - [03:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=182s) OpenClaw handled an issue backlog exceeding 10,000 items using ClawSweeper, an automated triage system running OpenAI Codex over GitHub Actions that evaluated tickets against a core vision.md file and closed roughly 15,000 issues. - [17:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1029s) To bypass the 5,000 requests-per-hour personal GitHub API cap during multi-agent sweeps, Steinberger deployed Octopus, a Cloudflare Worker shim that routes read traffic through a 15,000 req/hr GitHub App while reserving personal tokens for mutating actions. - [24:48](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1488s) Visual and cross-platform verification was solved through CrabBox and Mantis, which spin up remote cloud environments across operating systems, record before-and-after video proof via VNC, and allow agents to visually confirm fixes before PR merging.
## Key takeaways - [00:06](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=6s) The primary role of a modern developer is no longer writing code faster, but closing the autonomous feedback loop so agents can verify and correct their own work. - [03:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=182s) Incoming pull requests and issues are functionally identical signals ('prompt requests') indicating user dissatisfaction, and generating clean fixes from issues is often faster than reviewing messy third-party LLM pull requests. - [06:45](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=405s) High-velocity open source repositories require a continuous sweeping mechanism, running weekly or daily sweeps where agents close obsolete tickets patched by recent main branch commits without human intervention. - [03:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=182s) A standard repository specification like vision.md is essential to ground autonomous triage models on explicit feature boundaries and project scope. - [13:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=789s) Correlating live community chatter across Discord with GitHub trackers allows automated agents to prioritize the top user pain points rather than merely triaging chronological backlogs. - [15:19](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=919s) Internal developer tooling does not need rigorous design standards or line-by-line review; it can be entirely 'vibe coded' and treated as disposable scripts to eliminate operational friction. - [17:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1029s) API rate limits from commercial developer platforms can severely throttle multi-agent workflows, prompting custom proxy shims like Octopus to split read and write token quotas. - [20:00](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1200s) Offloading heavy parallel test suites to ephemeral multi-cloud runners (CrabBox) prevents local machine lockups and catches OS-specific regressions across Windows, macOS, and Linux. - [24:48](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1488s) Automated visual testing via Mantis captures side-by-side bug reproduction and verification videos, shifting human code review toward inspecting recorded visual proof. - [26:29](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1589s) Running automated reviews with fresh sub-agent context in an isolated CLI process prevents context bloat and eliminates circular review loops. - [29:25](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1765s) Agent instructions and invariants in agents.md should be authored and refactored by LLMs themselves rather than manually drafted by developers. - [31:46](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1906s) Requiring sanitized prompting transcripts to be attached to pull requests allows maintainers to distinguish rigorous engineering dialogue from low-effort AI-generated 'slop.' - [34:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2042s) Large codebases exceeding one million lines can be audited by tools like ClawPatch, which divide the repository into discrete functional modules for parallelized, fresh-context agent passes. - [37:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2222s) Commercial anti-bot defenses against automated headless Linux containers can force teams to build independent internal testing infrastructure like the ClickClack messaging platform. - [38:43](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2323s) Personal annoyance is the best signal for automation; addressing friction points creates an interlocking toolchain that compounds productivity for small teams. - [39:51](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2391s) AI models excel at cross-repository pattern matching, allowing developers to point agents at previous solutions and Git history instead of maintaining fragile, rigidly coupled tool architectures.
## Detailed walkthrough ### [00:06](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=6s) Rethinking Software Engineering: Helping Agents Close the Loop
Peter Steinberger opens the session by cautioning the audience not to expect a standard corporate slide deck. His talk, titled "Build the thing that builds the thing," is intended to reframe how developers view the future of software construction. In his view, coding agents have only existed for less than a year, have only truly been effective for about six months, and their rapid evolution is rendering traditional software development practices obsolete. He asks the room if anyone actively maintains open source, following up with a wry query: “How are you handling the pain?”
Addressing media narratives around his recent work, Steinberger laughs off articles asserting that he launched 43 failed projects before finally striking gold with OpenClaw. None of those projects were failures; rather, each was a targeted internal tool he spun up to accelerate his own workflow and remove friction for his autonomous coding agents. He stresses that modern engineering requires an inverted mindset: human developers should stop focusing on how to code faster themselves and instead focus on how to empower AI agents to work faster and complete tasks end-to-end.
To illustrate what he calls "closing the loop," Steinberger recounts spending several months building a SaaS product the previous year. Initially, an agent would write code, announce it was finished, and leave Steinberger to manually verify the user interface in a browser, spot errors, and prompt the model again in a tedious back-and-forth. The breakthrough came when he realized the agent could simply take its own screenshots and inspect the visual rendering directly. By eliminating repetitive human interventions, developers can design feedback mechanisms that allow agents to autonomously verify and fix their own outputs.
As Peter Steinberger put it: “Anytime you build software now your job is no longer to, to figure out how you build software faster. Really your job should be, what can I do to help my agent to build software faster and even more?”
[03:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=182s) Taming the Influx: Prompt Requests and ClawSweeper
When the OpenClaw client suddenly gained widespread traction, the repository was flooded with an unmanageable spike in issues and pull requests. Steinberger discovered firsthand that GitHub's web interface masks totals once they hit 5,000, obscuring a backlog that had surged past 10,000 entries. Because users and their automated agents file tickets indiscriminately, open-source repositories are inundated with low-effort contributions. While many maintainers vent their frustration at this deluge of AI-generated content, Steinberger takes a pragmatic stance: developers cannot fight the inevitable tide of technology and must adapt their infrastructure instead.
He redefines GitHub's standard vocabulary, arguing that incoming pull requests are essentially just "prompt requests." To him, issues and PRs represent identical signals: a user is dissatisfied with the current software behavior and wants a change. In fact, raw issues are often preferable to pull requests. Reviewing external pull requests frequently forces a maintainer's agent to ingest third-party LLM code, untangle it, and rewrite it—consuming far more time than simply having an agent implement a fresh fix directly from an issue description.
To tackle the backlog, Steinberger experimented with commercial automated code review platforms like CodeRabbit. While competent at commenting, they failed to group or close duplicate and invalid items automatically. Rather than waiting for a third-party vendor to build a solution, Steinberger created ClawSweeper. Powered by GitHub Actions, ClawSweeper triggers an instance of OpenAI Codex in the cloud whenever a ticket or PR is submitted. The workflow injects a dedicated repository markdown file—`vision.md`—into the agent's core prompt, defining explicit boundaries for features the project actively wants versus those it explicitly rejects. Armed with these project rules, ClawSweeper evaluated the backlog and closed roughly 15,000 issues.
As Peter Steinberger observed: “You can't fight the world. You have to figure out a way how to deal with that.”
```mermaid flowchart TD accTitle: ClawSweeper Automated Issue Processing accDescr: Pipeline showing how incoming GitHub issues and PRs are evaluated against vision.md using Codex to comment or close tickets.
subgraph Ingestion ["GitHub Activity"] A["New Issue / Pull Request"] B["Scheduled Weekly Re-sweep"] end
subgraph Evaluation ["ClawSweeper Engine"] C["GitHub Actions Runner"] D["OpenAI Codex Instance"] E["vision.md (Project Rules)"] end
subgraph Resolution ["Automated Actions"] F["Close Obsolete / Out-of-Scope Ticket"] G["Post Clarifying Comment / Status"] H["Discord Notification to Maintainer"] end
A --> C B --> C C --> D E --> D D --> F D --> G D --> H ```
- When backlogs reach 5,000 items, GitHub's user interface stops incrementing the public counter, masking the actual volume (which reached over 10,000 open items for OpenClaw). - Operating under the assumption of "infinite tokens," ClawSweeper ran automated triage passes that successfully closed approximately 15,000 extraneous issues and PRs. - Every repository maintained by Steinberger now standardizes on a root `vision.md` specification to steer autonomous triage decisions.
[06:45](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=405s) Continuous Sweeping and Discord Notifications
Triage in an ultra-high-velocity repository cannot be treated as a one-time static audit. At OpenClaw's scale, everyday code commits landing on the `main` branch regularly fix bugs and edge cases that the maintainers have never personally read, simply because thousands of tickets sit uninspected in the backlog. To address this lag, Steinberger configured ClawSweeper to execute sweeping runs over every open issue and pull request at least once a week—a cadence that can be dialed up to daily runs depending on token expenditure.
This background sweeping loop links directly to Steinberger's Discord workspace. Codex inspects aging tickets against recent commits on `main`, resolves those whose root causes have already been patched, and posts automated feedback to the issue author explaining that the bug is resolved in the latest build. Steinberger regularly receives Discord pings noting that three-month-old tickets have been automatically cleared without requiring a second of his manual attention.
Demonstrating the live operations dashboard, Steinberger shows 1,818 Codex background workers concurrently analyzing tickets. The system surfaces a consolidated activity stream that provides maintainers with structured summaries, contextual arrays of issues requiring targeted fixes, and detailed progress logs.
Highlighting the reality of high-volume repository maintenance, Peter Steinberger explained: “Any time a fixed lens on main, there's a good chance that I solved an issue that I've never read because there's like many thousand issues open.”
[08:30](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=510s) Maintainer Dashboards, Custom Issue Browsers, and Gorg
As the OpenClaw ecosystem accelerated, Steinberger onboarded community maintainers rapidly and liberally. However, contributor engagement varied wildly: while some individuals proved exceptional, others contributed for a few days before vanishing entirely. To regain governance and visibility across the team, Steinberger built an internal maintainer dashboard. Recognizing that open-source leadership encompasses far more than pushing git commits, the dashboard synthesizes GitHub repository analytics with Discord communication metrics, ensuring contributors actively assisting users in discussions receive full visibility.
To feed Discord activity into the dashboard, Steinberger wrote a custom Discord crawler using quick agent prompts. He points out that creating such micro-tools is trivial with modern LLMs; the true difficulty lies in sustaining and maintaining them over time. Steinberger manages his workload at an abstraction layer above single repositories: he navigates to a root directory containing all his projects and instructs Codex to triage across every workspace simultaneously, surfacing an aggregate action list of open issues.
```mermaid flowchart LR accTitle: Maintainer Dashboard Data Ingestion accDescr: Data flow consolidating GitHub activity and Discord community discussions into a unified internal maintainer portal.
subgraph Sources ["Activity Signals"] GH["GitHub Commits & PRs"] DC["Discord Discussions & Chat"] end
subgraph Pipeline ["Processing Layer"] CR["Custom Discord Crawler"] TX["Multi-Project Codex Triage"] end
subgraph Portal ["Maintainer View"] DB["Maintainer Analytics Dashboard"] IB["Fast Issue Browser UI"] end
GH --> DB DC --> CR --> DB TX --> IB ```
Because review time remains the ultimate bottleneck, Steinberger eliminated small interface latencies that slowed down triage. When reviewing aggregate issue lists from Codex, opening multiple browser tabs proved sluggish. Furthermore, different AI models and LLM setups—which he jokingly refers to as "Clankers"—output markdown lists and issue references in erratic, non-standardized formats. To bypass this friction, he built a dedicated, web-based issue browser. The custom tool parses any copied text block regardless of formatting irregularities and renders a clean interface where he can rapidly approve or discard items. He generally directs the agent to execute autonomous fixes for eight out of ten selected tasks while reserving his personal focus time for the remaining two complex issues.
When existing platform tooling fell short for other agent workflows, Steinberger adopted the same philosophy. Frustrated that Google lacked a comprehensive CLI for programmatic agent access, and dissatisfied with built-in integrations that locked agents to a single Google account, he created **Gorg**—a dedicated command-line interface enabling agents to interact seamlessly with multi-account Gmail and Calendar data.
As Peter Steinberger emphasized regarding developer efficiency: “The critical thing that you are is like, your time is limited. So you need to build the tools to build the tools.”
- High-level maintainer triage operates out of a centralized master projects directory rather than inside individual Git repositories. - Steinberger's custom issue browser incorporates flexible parsing heuristics to handle formatting quirks produced by diverse LLM agents. - **Gorg** was developed as an open-source CLI utility to allow coding agents to manage multi-account Gmail drafts, messages, and Google Calendar events without human browser intervention.
[12:33](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=753s) End-to-End Autonomous Issue Verification
Peter Steinberger explains the profound leverage of having an autonomous agent operate across an entire software stack. Because he and his agent control the complete environment end-to-end, any missing component or bug can simply repair itself in place without requiring external intervention. He describes the empowering sensation of owning the whole problem space, remarking: “My agent controls the whole stack. So it can just fix itself like so like you own the whole universe. Anything is fixable.”
When new issues or feature requests come in, tools like Codex do far more than merely produce source code. Steinberger highlights that because he maintains demo accounts on external platforms such as GOG, the agent can autonomously implement the feature and run comprehensive end-to-end verification. Consequently, the maintainer's role is reduced to simply reviewing the finished artifact, giving approval, and deploying to production with total confidence.
- Agents with full stack access can self-diagnose and repair missing dependencies or runtime failures autonomously. - Live integration accounts (e.g., GOG demo accounts) allow the AI to test end-to-end user flows rather than stopping at mock unit tests. - Maintainer involvement shifts from manual implementation to reviewing verified pull requests and clicking "yes" to ship.
---
[13:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=789s) Mining Maintainer Reports: Correlating Discord and GitHub
Maintaining a large open-source project with thousands of incoming issues makes prioritization an overwhelming challenge. To combat issue fatigue, Steinberger built an automated crawling system connected directly to Codex. The system ingests chatter across Discord community channels and cross-references it with open GitHub issues and pull requests, surfacing the problems generating the most user friction.
```mermaid flowchart LR accTitle: Discord Crawl to Agent Triage Flow accDescr: Discord chat logs are crawled by a bot, correlated with GitHub PRs and issues by Codex, and matched against a vision document to select tasks. subgraph Ingestion["Community Ingestion"] A["Discord Channels"] -->|"Bot crawl"| B["Serialized DB Backup on GitHub"] end subgraph Triage["Autonomous Triage"] B --> C["Codex Correlation Engine"] D["GitHub Issues & PRs"] --> C E["Vision Document"] --> C end subgraph Execution["Task Execution"] C -->|"Top 5 pain points"| F["Autonomous Worker Agents"] end ```
The crawl system aligns issues directly with high-level project goals. Steinberger prompts the agent with a targeted query: “What are people screaming the most correlated to open issues and PRs and find me the top five things I should work on.” The agent reconciles high-volume community complaints against a core vision document, filters for self-contained tasks that can be addressed independently, and spins up parallel background tasks for hours of autonomous remediation while keeping human maintainers informed of high-risk, nuanced code paths.
- Triage agents correlate raw community chatter with open tracker items to surface top user pain points. - The system checks candidate tasks against an architectural vision document to prevent agents from wandering off-scope. - Maintainers maintain a high-level overview, letting agents handle routine PRs while reserving human review for delicate or architectural components.
---
[14:22](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=862s) Distributing Community Crawl Data via GitHub
Once the Discord crawling tool proved effective, other project maintainers requested access to the aggregated data. However, Discord lacks a simple public export mechanism, and sharing a privileged bot token across multiple maintainers posed severe security and operational risks.
Steinberger resolved the distribution bottleneck by leveraging GitHub as a decentralized database. Instead of provisioning token infrastructure, the bot continuously serializes the Discord message database into flat files stored in a private GitHub repository. Maintainers without Discord API bot tokens can pull the structured data directly from GitHub, giving them an instant, secure query backend without administrative overhead.
---
[15:19](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=919s) The Release Bar Dashboard and Vibe-Coding Internal Tools
Managing numerous open-source repositories often leaves maintainers blind to which projects require urgent releases, dependency bumps, or review. Steinberger created "Release Bar," a custom lightweight dashboard designed to track days elapsed since each project's last release and identify stagnant repositories requiring maintenance. Addressing the continuous upkeep required by modern software, he warns: “If you don't constantly garden your software, it's dead because dependencies, everything needs updating.”
He draws a sharp philosophical contrast between core user-facing software and internal productivity tooling. While public code in projects like OpenClaw demands meticulous review, internal tools can be purely "vibe coded" with agents without manual line-by-line inspection. If a disposable tool works, it provides immediate leverage; if it fails, it can be iterated on or thrown away at near-zero marginal cost.
- Release Bar monitors projects by tracking the number of days since the last release and flagging unmerged PRs. - Internal developer tooling can be fully generated by AI without strict code reviews because it is not mission-critical. - Successful internal utilities can be incrementally refined if they prove sticky, eliminating upfront engineering costs.
---
[17:09](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1029s) Bypassing GitHub Rate Limits with Octopus
Running ten or more agentic workflows concurrently quickly exhausted personal GitHub API rate limits. Steinberger describes the frustration of running large automated sweeps across related issues, only to hit the individual developer cap of 5,000 API requests per hour. He jokingly recalls relying on an internal GitHub contact who had to manually push a button every hour to reset his quota, a setup that collapsed whenever she had to sleep.
Reflecting on his development style, Steinberger remarks: “I find being annoyed is the very best way to solve problems.” To permanently overcome the 5,000-request wall, he created "Octopus," a routing proxy that shims GitHub API calls through Cloudflare Workers.
```mermaid flowchart TD accTitle: Octopus GitHub Token Proxy Architecture accDescr: API requests pass to a Cloudflare worker which routes read-only calls to a 15k token GitHub App and mutating calls to a 5k personal token. A["Agent Requests"] --> B["Octopus Shim (Cloudflare Worker)"] B -->|Non-mutating read on public repo| C["GitHub App Token (15,000 req/hr)"] B -->|Mutating / private action| D["Personal PAT Token (5,000 req/hr)"] C --> E["GitHub API"] D --> E ```
- Individual personal access tokens (PATs) cap out at 5,000 requests per hour. - GitHub Apps grant an elevated tier of 15,000 requests per hour. - Octopus inspects HTTP traffic, automatically shunting read-only public requests through the GitHub App while reserving the personal PAT strictly for state-mutating actions. - Autonomous agent stacks (combining advanced reasoning models and browser automation) handled the Cloudflare infrastructure configuration directly.
---
[20:00](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1200s) Crab Box: Cloud Test Execution and Cross-Platform Matrixes
As project scope expanded and bug reports multiplied, the test suite bloated from taking 10 seconds to 30 seconds, then 2 minutes, and eventually over 5 minutes. Steinberger optimized the test suite locally for his high-spec Mac Studio down to 60 seconds, but this caused teammates' machines to overheat and freeze under the heavy parallel load. Early experimentation with remote test-runner startups like Blacksmith showed promise, but recurring daily downtime frequently paralyzed team pull requests.
In response, Steinberger developed "Crab Box," an open infrastructure abstraction tool named in keeping with crustacean convention. Crab Box instantly spins up ephemeral remote cloud VMs across more than 20 supported providers (including AWS and Azure), mirrors local repository changes, and executes heavy CPU-bound test suites off the developer's local machine.
```mermaid flowchart TD accTitle: Crab Box Cross-Platform Execution Architecture accDescr: Crab Box synchronizes code to multiple cloud instances running macOS, Linux, and Windows with VNC remote inspection. A["Local Repo / Agent Session"] -->|"Sync changes"| B["Crab Box Orchestrator"] subgraph RemoteVMs["Ephemeral Matrix Workers"] B --> C["Linux VM (Red Hat / Ubuntu)"] B --> D["macOS Worker"] B --> E["Windows Worker + VNC UI"] end RemoteVMs -->|"Aggregated Test Results"| F["Agent Confidence Verification"] ```
Beyond raw performance offloading, Crab Box resolved stubborn cross-platform inconsistencies. Node.js environments frequently introduce platform-specific edge cases, such as EEPROM issues on Windows. Rather than manually booting virtual machines at 3:00 AM, Steinberger integrated cross-platform matrix testing into Crab Box, enabling agents to fan out tests across Windows, Red Hat, and other operating systems simultaneously. To eliminate the final hurdle of diagnosing OS-specific visual regressions on Windows web browsers, he embedded VNC capabilities directly into the boxes, enabling full remote graphical debugging.
- Local parallel test runs overheated developer laptops, necessitating isolated remote runner environments. - Crab Box provides failover fallback across cloud infrastructure providers (AWS, Azure, and alternatives) to prevent CI interruptions. - Multi-architecture runners spin up simultaneously to validate platform-specific behavior across Linux distros and Windows without requiring manual host setups. - Built-in VNC remote desktop access allows agents and maintainers to inspect browser and desktop UI bugs directly on remote instances.
[24:48](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1488s) UI Automation with CrabBox and Visual Verification with Mantis
Peter Steinberger explains that remote debugging using VNC in CrabBox was an enormous step forward: an agent could spin up an exact environment containing a browser, a terminal, and messaging apps like Telegram to recreate end-to-end issues. However, having to manually interact with the VNC session remained a chore. Driven by the philosophy that “laziness is good,” he equipped CrabBox with computer vision capabilities and lightweight tools for taking screenshots, clicking, and typing. This allowed coding agents to interact directly with graphical user interfaces using automated visual controls.
Yet watching an agent slowly navigate an interface was still tedious. Steinberger wanted finished proof rather than live monitoring, which prompted the creation of Mantis. When an issue arises, Klaus can be triggered to generate a pull request, while Mantis spins up sandboxed environments to record video evidence of the bug, apply the patch, capture video confirming the resolution, and analyze the resulting footage to ensure correctness.
```mermaid flowchart LR accTitle: Mantis Automated Bug Fix and Verification Pipeline accDescr: Shows the automated flow from issue detection to Klaus writing a fix, Mantis verifying via video, and one-click merging.
subgraph Trigger["1. Issue & Patch"] A["Issue Reported"] --> B["Klaus Generates PR"] end
subgraph Verification["2. Mantis Visual Testing"] B --> C["Spin up CrabBox Container"] C --> D["Record Bug Video"] D --> E["Apply Fix & Record Verification"] E --> F["Agent Inspects Video Artifacts"] end
subgraph Resolution["3. Maintainer Review"] F --> G["Maintainer Reviews Clip"] G --> H["Merge PR"] end ```
- CrabBox added native tools for screenshots, clicks, and keystrokes to let models drive UI workflows directly. - Mantis records dual videos (before-fix bug reproduction and after-fix resolution) and evaluates them automatically. - “And that's all built based on me being annoyed,” Steinberger noted regarding the motivation behind building these visual testing systems.
---
[26:29](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1589s) Eliminating Review Thrash with Automated Review Chains
Steinberger recounts the frustration of relying on manual `/review` commands when working with Codex. A prompt would run, followed by a review revealing three genuine issues. After spending twenty minutes fixing tests, running the review a second time would reveal two previously unmentioned issues due to context limits. Repeating this cycle over ten times while implementing support for the Line messaging platform consumed an entire day: “I don't even use Line, but like somebody was very passionate about it. I wanted to get it in. And then all day I did this review and I was like, is it enough? Is enough? Like, like, I feel like I'm the clanker.”
To stop acting as the manual intermediary, he built `auto-review`. While he initially considered writing complex hooks or forking the open-source Codex codebase, modern models follow system prompts reliably enough that a simple rule inside `agents.md` was sufficient. When instructed to run an auto-review before committing or landing a pull request, the agent spins up a fresh CLI sub-agent session with clean context to review its own modifications. Although an early recursion bug caused Codex to invoke nested instances endlessly, the stabilized implementation now automatically executes sequential review passes without human intervention.
- Relying on human reviewers to spot regressions with "fresh eyes" does not scale, but spawning fresh agent processes provides genuine context isolation. - Early prototypes ran into runaway process loops where Codex recursively called itself until guarded limits were enforced.
---
[29:25](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1765s) Refining Agent Guidelines and Pruning Invariants
To make autonomous reviews practical, Steinberger had to teach the agent which issues were real and which were irrelevant noise. Without explicit domain knowledge, agents default to excessively defensive code—such as adding guards against plugin files being modified live on disk while the process is active, an edge case the project intentionally ignores. Because agent memory resets at the start of every session, project-specific invariants must be documented explicitly in the instructions.
He advises developers never to write these agent configuration documents by hand. Instead, developers should describe their goals conversationally and have the model author the instructions in its own terminology. Periodically, maintainers should enter a "meta mode" to ask the model whether any rules inside its instruction file are contradictory or confusing. Ambiguous instructions waste context tokens and degrade generation quality.
- Maintainers must document project-level assumptions in `agents.md` to prevent models from generating boilerplate defensive code for scenarios that never occur. - Instruction files should be authored and refactored by the models themselves rather than written manually by humans. - “Try auto review and try CrabBox. It totally revolutionized how we build software like my, my trust in in prompting and shipping is much, much higher with that stuff,” Steinberger emphasized as his primary advice.
---
[31:46](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1906s) Extracting Prompts and Mitigating Pull Request "Slop"
A persistent problem with open-source contributions in the AI era is determining the engineering rigor behind large submissions. A one-sentence prompt can generate ten pages of code, making it difficult to differentiate low-effort code generation from thoughtful, thoroughly iterated contributions. Early efforts asking contributors to manually paste their prompting transcripts failed because contributors simply bypassed the request.
To solve this, Steinberger developed an OpenClaw skill that intercepts the pull request submission process. The agent asks the contributor for permission to sanitize and attach their session history, isolates the relevant portions of the disk transcript, scrubs sensitive information, and appends the prompting log directly to the PR description. This provides maintainers with immediate visibility into whether the contributor had a substantive problem-solving dialogue with the model or simply ran a four-word command like "fix issue number."
- The OpenClaw skill automatically detects, redacts, and attaches local conversation transcripts to pull requests when authorized. - Transcripts allow maintainers to distinguish low-effort AI "slop" from carefully verified, high-context agent workflows. - “Oftentimes we're not limited anymore by intelligence. The models are good enough now. Everything that we are limited of is our own imagination,” Steinberger reflected.
---
[34:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2042s) Whole-Codebase Auditing with ClawPatch and CrabFleet Multiplayer
After witnessing how many subtle defects auto-review caught on new pull requests, Steinberger realized that existing code committed without review was full of undetected mistakes. However, running a standard review across a massive code base with over a million lines of code is impossible because context windows become overwhelmed and models lose focus.
To address this challenge, he built ClawPatch. The tool breaks large repositories down into roughly fifty isolated functional subsections, analyzes each module independently with fresh agent contexts, and surfaces actionable refactoring patches. Finally, Steinberger introduced CrabFleet, an ongoing project addressing the inefficiency of human teammates acting as copy-paste relays between their local coding agents and internal chat channels. By running Codex inside shared CrabBox virtual environments, developers can collaborate within the same live agent session through a shareable link instead of manually shuttling snippets back and forth.
| Tool | Purpose | Mechanism | |:--- |:--- |:--- | | **ClawPatch** | Large-scale repo auditing | Splits a 1M+ line codebase into ~50 discrete components, running dedicated review agents across each slice. | | **CrabFleet** | Multiplayer agent workspaces | Hosts Codex directly inside containerized CrabBox instances, allowing teammates to access and steer a shared agent session via URL. |
- “You're just intermediary. I want to talk to your clanker,” Steinberger noted, explaining the frustration that led to CrabFleet's collaborative environments. - CrabFleet turns isolated local agent sessions into multi-user collaborative sandboxes accessible via a web link.
[37:02](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2222s) Accidental Infrastructure: Building "ClickClack" to Escape Captchas and Bot Blocks
Peter Steinberger describes how their internal developer setup expanded into full collaborative tooling. With remote instances and VNC sessions, maintainers can inspect an active session, talk through it, and take control in what he humorously calls "multiplayer clanking." However, their ambition to perform true end-to-end integration testing quickly collided with commercial messaging platforms. Testing complex APIs across multiple channels required verifying visual output directly inside an authentic user interface. Whenever their automated agents spun up headless, Linux-based instances in containers to interact with these external services, defense mechanisms immediately triggered. Security systems interpreted automated Linux boxes with no logged-in user as an active attack:
> “And I built this automation, but man, anytime I logged in, there was like a Cloudflare captcha or like, like all these, all these companies really don't like it if you spin up something in a box, especially not if it's Linux based and there's nobody logged in and it just feels like it feels like someone's trying to hack the system.”
Rather than continually wrestling with bot detection, IP bans, and captchas, the team took the extreme step of reinventing their own real-time chat infrastructure. Steinberger admits that the project spun out of scope, leaving them with an entirely functional backup messaging platform:
> “So we just built our own messaging platform to like test to like, have real end to end tests with something that we have under control. And it got a little bit out of hand. And now we kind of have a backup chat system that works. Whenever our team is down, it's called ClickClack.”
```mermaid flowchart TD accTitle: Internal Chat and Test Infrastructure Architecture accDescr: Flowchart showing why automated headless testing against commercial platforms led to building ClickClack.
subgraph ExternalBlocks["Commercial Chat Platforms"] A["Headless Linux Container"] -->|"Automated End-to-End Test"| B["Cloudflare Captchas & Bot Detection"] B -->|"Fails / Blocks Connection"| C["Brittle Test Automation"] end
subgraph InternalSolution["In-House Replacement"] D["Multiplayer VNC Sessions"] -->|"Direct UI & API Testing"| E["ClickClack (In-House Messaging Platform)"] E -->|"Bypasses Captchas"| F["Reliable End-to-End Automated Tests"] E -->|"Failover Channel"| G["Team Backup Chat"] end ```
---
[38:43](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2323s) Annoyance as an Automation Signal and the Compounding Toolchain
Reflecting on the sprawling ecosystem of specialized crawlers, dashboards, and virtualized runners built for OpenClaw, Steinberger shares his overarching philosophy for deciding what to automate. Rather than following rigid roadmaps or top-down mandates, developers should pay close attention to daily friction points. Personal frustration serves as the most reliable diagnostic indicator that a manual workflow is ready to be handled by an agent:
> “Listen to your body. Listen when you're annoyed. Annoyance is an amazing signal for, we can automate this.”
When internal tools are created to alleviate specific annoyances, they begin compounding in unexpected ways. Steinberger notes that individual systems do not exist in isolation; each new utility builds upon the outputs and abstractions of the ones that preceded it. The release reports exist only because data was first extracted using Discrawl, while runner orchestration via CrabFleet depends on the base virtualization provided by CrabBox. Despite what he admits are eccentric internal names, these tools form an interdependent scaffolding that allows a small engineering group to maintain and iterate across a massive, cross-platform codebase.
- **Compounding internal toolchain:** Specialized tools feed directly into higher-level workflows (e.g., Discrawl data powers report dashboards, while CrabBox isolates instances for CrabFleet orchestration). - **Team leverage:** Custom automation enables a very small maintainer team to manage issue triage, cross-OS testing, and releases for an expansive open-source ecosystem.
---
[39:51](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2391s) Q&A: Tool Dependencies, Cross-Repo Pattern Matching, and Background Codex Building
Opening the floor to questions, an attendee asks whether assembling a sprawling, enterprise-style ecosystem of interconnected internal tools creates friction, where dependencies between custom utilities end up blocking future improvements. Steinberger responds that this bottleneck hasn't materialized. Rather than acting as rigid dependencies, existing tools serve as concrete templates and reference points for coding agents. When instructing an AI model, referencing existing internal solutions drastically reduces prompting overhead:
> “When you work with agents, if you've solved something once, you can just tell the agent, look there, I already solved this there. They are really good at pattern matching. So I would actually say it really compounds.”
He illustrates this with an issue he faced when building token management. The initial prototype ran locally on his workstation, which proved clunky and awkward to integrate with a GitHub App. Instead of crafting elaborate new prompts from scratch, he had his agent inspect the Git history, discard the broken prototype, salvage functioning logic from an existing companion tool, and assemble the revised architecture. Because agents navigate Git commit history and project context with high competence, prompt engineering shifts from detailed specification writing to high-level delegation.
Steinberger also highlights their habit of letting models listen directly to team design conversations:
> “Sometimes we just talk in the team about it and I just press the button and Codex will just listen in on the conversation and just start building while we've been discussing, even though there's like side chatter and blah blah, but like it's clever enough to understand like, oh, this comment was a joke and this is important. So you just like, you can just do stuff.”
---
[42:08](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2528s) Q&A: Overcoming the "Too Busy" Trap with Disposable Internal Tooling
A solutions engineer named Divya steps up to the microphone, asking how someone with a massive workload can realistically find time to design and build custom agents and tools when their daily schedule is already completely full. Steinberger answers by recalling a classic stone-age comic:
> “Do you notice, do you notice meme and like Stone age where they had this car with rectangular, and and then the guy comes along with like the round one and they're like, oh, we don't have time. We're so busy.”
His central argument is that developers must lower their standards regarding the architectural elegance of internal helpers. Small utility tools do not require rigorous design reviews, clean abstractions, or preferred languages; they can be quickly spun up as disposable scripts:
> “So, so the beauty is all these tools, they can be massive slop. It's like you can literally, you can really tell your Codex, hey, look at this session and see the pain that I had to do. Build me a tool that reduces the pain.”
Steinberger points out that he regularly lets models output Go code despite not particularly enjoying the language himself, simply because agents write Go quickly and reliably. By running small, low-stakes agent experiments in an auxiliary terminal window alongside daily tasks, developers reclaim both their operational efficiency and the joy of playful engineering.
## Notable quotes > [00:06](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=6s) Anytime you build software now your job is no longer to, to figure out how you build software faster. Really your job should be, what can I do to help my agent to build software faster and even more? — Peter Steinberger
> [31:46](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=1906s) Oftentimes we're not limited anymore by intelligence. The models are good enough now. Everything that we are limited of is our own imagination — Peter Steinberger
> [38:43](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=2323s) Listen to your body. Listen when you're annoyed. Annoyance is an amazing signal for, we can automate this. — Peter Steinberger
> [15:19](https://www.youtube.com/watch?v=o5IQMijn-Ks&t=919s) If you don't constantly garden your software, it's dead because dependencies, everything needs updating. — Peter Steinberger
## Mentioned AWS, Azure, Blacksmith, ClawPatch, ClawSweeper, ClickClack, Cloudflare, Cloudflare Workers, CodeRabbit, Crab Box, CrabFleet, Discord, Discrawl, Divya, GitHub, GitHub Actions, GOG, Google Calendar, Gorg, Gmail, Klaus, Line, Mantis, Microsoft Build 2026, Node.js, Octopus, OpenAI Codex, OpenClaw, Peter Steinberger, Release Bar, Telegram, VNC
## Video description The way software is built is changing rapidly. Learn how we build a whole ecosystem of tools to build OpenClaw faster and with more confidence.
Seating for this session is first-come, first-served. Add it to your schedule to plan your day and arrive early to secure a spot.
To learn more, please check out these resources: * https://aka.ms/build26-next-steps
𝗦𝗽𝗲𝗮𝗸𝗲𝗿𝘀: * Peter Steinberger
𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻: This is one of many sessions from the Microsoft Build 2026 event. View even more sessions on-demand and learn about Microsoft Build at https://build.microsoft.com
BRK245 | English (US) | Agents & apps
Breakout | (200) Intermediate
#MSBuild
Chapters: 0:00 - Scaling Automation to Regularly Review and Resolve Issues 00:08:27 - Building Maintainer Dashboards Integrating GitHub and Discord Data 00:13:09 - Creating crawl system for open source issue management 00:14:22 - Distributing Discord crawl data via GitHub for maintainers 00:15:19 - Developing Release Bar dashboard for open source maintenance 00:22:56 - Expansion to cross-platform testing to handle OS-specific bugs 00:24:37 - Addition of VNC integration for full UI bug reproduction and remote debugging 00:33:37 - AI models no longer limited by intelligence but imagination 00:39:26 - Q&A: Balancing ecosystem dependencies and agent interoperability
[Source](https://www.youtube.com/watch?v=o5IQMijn-Ks)