Microsoft announced the General Availability 1.0 of the GitHub Copilot SDK across Node.js, Python, .NET, Rust, and Java, accompanied by a rewrite of the Copilot CLI server runtime from Node.js to Rust that drops memory consumption from ~120 MB to sub-10 MB. (57:52)
Microsoft introduced MAI-Code-1-Fast (MAI-Code-Flash), its first purpose-built coding model trained from scratch—featuring a 137B MoE architecture activating 5B parameters per token pass—optimized specifically for GitHub Copilot's harness and native tooling. (6:41)
The TypeScript compiler team spent 1.5 years porting its scanner, parser, checker, and emitter from JavaScript/TypeScript to Go, achieving a 10X compilation speedup and cutting VS Code's clean build time from 97 seconds down to 9–10 seconds under the upcoming TypeScript 7.0 (tsgo). (129:40)
Addy Osmani outlined an agentic engineering framework grounding custom agent skills in the SDLC stages using markdown-based context and red flag tripwires, emphasizing that non-deterministic AI generation requires deterministic automated quality gates. (15:10)
Detailed notes
## Background
Published by the Visual Studio Code channel on YouTube on June 5, 2026, this video documents a live broadcast stage session from Microsoft Build 2026 in San Francisco. According to the show description, the program delivers live demos, engineering discussions, and product reveals directly from the teams building GitHub, Copilot, VS Code, and Visual Studio.
## TL;DR
Hosted by Burke Holland and Pierce, the broadcast covers major advancements across Microsoft's developer tooling, emphasizing the shift from raw syntax generation to disciplined agentic engineering workflows. Key announcements include the General Availability of the GitHub Copilot SDK, the Copilot CLI runtime rewrite into Rust, and the debut of Microsoft's custom, ground-up coding models such as MAI-Code-Flash. Guests like Addy Osmani demonstrate structured agent skills grounded across the SDLC, while engineering leads break down prompt harnesses, offline evaluation suites, and multi-session CLI workflows. The session culminates with Anders Hejlsberg and Daniel Rosenwasser unveiling the native port of the TypeScript compiler from JavaScript to Go, unlocking multi-core parallelism and delivering roughly 10x faster build times.
## Atmosphere & style
The broadcast maintains an energetic, fast-paced conference vibe that pairs relaxed host banter with rigorous technical deep dives. Rather than indulging in uncritical AI hype, the presenters adopt a candid, pragmatic tone, openly discussing cognitive overload, prompt non-determinism, and the need for deterministic security gates. The pacing shifts fluidly between hands-free voice coding demos in the terminal and detailed compiler architectural breakdowns, giving viewers an intimate look at production developer tools under active reinvention.
## Bulletin
- [57:52](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3472s) Microsoft announced the General Availability 1.0 of the GitHub Copilot SDK across Node.js, Python, .NET, Rust, and Java, accompanied by a rewrite of the Copilot CLI server runtime from Node.js to Rust that drops memory consumption from ~120 MB to sub-10 MB.
- [06:41](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=401s) Microsoft introduced MAI-Code-1-Fast (MAI-Code-Flash), its first purpose-built coding model trained from scratch—featuring a 137B MoE architecture activating 5B parameters per token pass—optimized specifically for GitHub Copilot's harness and native tooling.
- [02:09:40](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7780s) The TypeScript compiler team spent 1.5 years porting its scanner, parser, checker, and emitter from JavaScript/TypeScript to Go, achieving a 10X compilation speedup and cutting VS Code's clean build time from 97 seconds down to 9–10 seconds under the upcoming TypeScript 7.0 (tsgo).
- [15:10](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=910s) Addy Osmani outlined an agentic engineering framework grounding custom agent skills in the SDLC stages using markdown-based context and red flag tripwires, emphasizing that non-deterministic AI generation requires deterministic automated quality gates.
## Key takeaways
- [03:54](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=234s) Managing 3 to 4 concurrent agents represents a realistic cognitive limit for developers, debunking social media hype around running dozens of terminal agents simultaneously.
- [06:41](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=401s) MAI-Code-1-Fast is Microsoft's first ground-up coding model rather than a fine-tune, built with adaptive solution length and positioned in the fast tier alongside Claude Haiku.
- [15:10](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=910s) Agent skills act like developer dotfiles, structuring executable domain context across SDLC stages (Define, Plan, Build, Verify, Review, Ship) with explicit red flag tripwires and verification checks.
- [31:43](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=1903s) Pre-specifying target files when generating implementation plans limits an agent's blast radius and prevents unintended modifications in existing codebases.
- [39:52](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2392s) Vibe coding suits solo greenfield prototyping, but enterprise production requires disciplined agentic engineering with strict, deterministic verification gates rather than relying on markdown guardrails alone.
- [51:31](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3091s) Model tiering—leveraging fast models like Flash for planning and high-capacity models like Claude Opus or GPT-5.5 for implementation and verification—optimizes token cost and runtime performance.
- [55:08](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3308s) Progressive skill loading conserves context window tokens compared to MCP tool definitions by loading only high-level summaries until specific instructions are required.
- [01:00:26](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3626s) GitHub Copilot SDK reached General Availability 1.0 stable, backed by an architecture rewrite into Rust that slashes server memory usage to below 10 MB.
- [01:12:15](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4335s) Multi-model verification workflows, such as implementing architecture with Claude Opus and performing independent verification with GPT-5.5, reduce hallucinated flaws.
- [01:29:09](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5349s) Copilot CLI incorporates local speech models like NVIDIA Nemotron (~700 MB) for on-device voice dictation without incurring cloud latency or sending audio externally.
- [01:47:05](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=6425s) Model integration into VS Code requires tailored harnesses; public benchmarks like SWE-bench serve merely as regression tests, while proprietary suites like VSCbench evaluate true coding capability.
- [01:58:39](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7119s) MAI-Code-Flash uses a 137-billion parameter Mixture of Experts (MoE) architecture with only 5 billion active parameters per token pass, trained directly against Copilot's production harness.
- [02:09:40](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7780s) Single-threaded JavaScript execution and dynamic shape lookups in V8 placed a hard ceiling on TypeScript's self-hosted compiler performance, motivating a complete rewrite in Go.
- [02:22:59](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=8579s) The native Go TypeScript compiler (@typescript/native-preview / tsgo) achieves 10X faster compilation speeds via multi-core parallelism and is slated to become TypeScript 7.0.
- [02:26:40](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=8800s) Early production adopters of the native TypeScript compiler—including VS Code, Slack, Notion, and Figma—report major productivity wins and hundreds of saved CI hours per month.
## Detailed walkthrough
### [00:30](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=30s) Day 2 Kickoff, Roles, and Schedule Highlights
Burke and his colleague Pierce welcome the audience to Day 2 of Microsoft Build first thing in the morning, immediately bantering about their ambiguous job titles. Pierce explains that he works on the Visual Studio Code team as well as across GitHub Copilot, Copilot apps, and the CLI—summarizing his role as simply working on "things." Burke shares a similar remit across Copilot, the CLI, and VS Code, while producing YouTube videos and hanging out with Pierce. They alert attendees that the current stage will transition to streaming Windows-focused content after 2:15 PM, alongside simultaneous broadcasts on YouTube, X, GitHub, and Microsoft Developer channels.
```mermaid
flowchart TD
accTitle: Microsoft Build Day 2 Stage and Session Schedule
accDescr: Schedule breakdown of live sessions and special stages across Microsoft Build Day 2.
subgraph Stages["Stage Programming"]
A["Broadcast Stage"] -->|"After 2:15 PM"| B["Windows Announcements"]
C["Keynote Stage"] --> D["Copilot Game Show<br/>(Vibe coding with Kent C. Dodds)"]
E["Expo Floor Booth"] --> F["'Fish Tank' Live Coding<br/>(Pierce: 2-hour build)"]
end
subgraph Sessions["Breakouts & Streams"]
G["Breakout 204<br/>(10:15 - 11:00 AM)"] --> H["VS Code AI Adoption Story<br/>(Pierce & Josh)"]
I["VS Code Channel"] --> J["Monthly Release Live Streams"]
end
```
The hosts run through the day’s featured programming, pointing out the glass-enclosed live coding booth on the expo floor that resembles a human fish tank. Pierce notes he is scheduled to be locked inside for two hours with no predetermined plan of what to build, recalling that Anthony spent ten minutes in the booth the previous year assembling an entire programming language. Additional highlights include:
- **Breakout 204 (10:15–11:00 AM):** Led by Pierce and Josh from the VS Code team, addressing the practical reality of team-wide AI usage. As Pierce puts it: “The basic concept is like, OK, you use AI, now what? There's a lot of things that break.”
- **The Copilot Game Show:** Taking place on the keynote stage, co-hosted by Kent C. Dodds, where the team will attempt to "vibe code" an application live in 45 minutes.
- **VS Code Monthly Release Live Streams:** Recommended by Burke as the only viable mechanism for developers to keep pace with rapid deployment cycles where new features ship virtually every fifteen minutes.
---
[03:54](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=234s) The GitHub Copilot App and Multi-Agent Workflows
Turning to the major announcements from Day 1, Burke and Pierce discuss the new GitHub Copilot app, emphasizing that it is neither an editor nor a CLI. Pierce characterizes it as an “agent native desktop experience for building things.” While developers are accustomed to session managers that spin up isolated agents running simultaneously across Git worktrees, the Copilot app extends beyond code generation into the broader context of engineering work: triaging backlogs, deciding which issues to pick up, and steering pull requests across the finish line once automated changes are submitted.
Burke polls the live audience on how many developers run multiple agents at once, observing that while a handful run two or three, the idea of running fifty concurrent terminal agents is mostly performative social media hype. Pierce agrees, reflecting on what manageable productivity looks like when writing performance reviews:
> “I personally don't think I can go beyond 3 or 4 because I really start like losing track of what's actually going on and I have to batch the task together.”
Beyond concurrency, Pierce explains that the app intentionally pushes raw source code into the background. Rather than forcing the user into code review immediately, it exposes an embedded browser with one-click execution so developers can evaluate functional outcomes first. Burke concurs with this design philosophy, admitting that when code remains hidden, developers are spared from getting "nerd sniped" into nitpicking syntax, shifting their mindset strictly to verifying whether the agent's output actually functions.
---
[06:41](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=401s) The GitHub Copilot SDK and MAI-Code-1-Fast Coding Model
The discussion moves to the underlying architecture of GitHub Copilot and the rationale behind releasing a dedicated Copilot SDK. Pierce explains that every GitHub Copilot surface relies on an "agent loop"—the combination of system prompts, tool invocations, and context gathering that drives the model's interaction. Historically, these components were implemented independently across different products until they were unified inside the GitHub Copilot CLI. The new Copilot SDK extracts this battle-tested core runtime, which undergoes rigorous offline evaluation and lab testing, enabling developers to package that exact agent loop into arbitrary third-party software. Burke highlights that users with a Copilot subscription can use the SDK to instantly one-shot their own custom assistants, contrasting it with open-source agent tooling like OpenClaw or Hermes Agent.
```mermaid
flowchart LR
accTitle: Evolution of GitHub Copilot Agent Architecture into the SDK
accDescr: Historical fragmentation of Copilot agent loops unifying into the CLI and exposed as an SDK.
Pierce also introduces **MAI-Code-1-Fast**, which he describes as the very first ground-up coding model developed by Microsoft rather than a derivative fine-tune:
> “It is the first coding model we built from scratch here at Microsoft.”
- **Model Class & Benchmarking:** Positioned in the lightweight, fast-tier category alongside models like Claude Haiku, optimized for rapid, cost-effective execution.
- **Specialized Training:** Specifically trained on GitHub Copilot's internal tools, harnesses, and agent trajectories, ensuring superior performance within Copilot workflows compared to generic models.
- **Adaptive Solution Length:** Trained to dynamically adjust verbosity to the user's prompt—avoiding long diatribes when responding to brief conversational inputs.
- **Lineage Distinction:** Unlike earlier offerings such as Raptor Mini—which was a fine-tuned variant of GPT-5 Mini—MAI-Code-1-Fast was trained completely from scratch.
- **Availability:** Rolling out incrementally to VS Code developers via the model picker dropdown and through automatic model selection ("auto" mode).
[09:54](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=594s) Prompting VS Code with Voice via the Speech Extension
A quick demonstration highlights how voice interaction can streamline working with autonomous coding agents in the editor. By installing the official VS Code Speech extension, developers can initiate prompts verbally rather than manually typing out long instructions. In the demo, the presenter works on a community contributor website designed to track and celebrate top contributors for the latest VS Code release. The site already features a mechanism allowing visitors to award "kudos" to contributors, but the presenter wants to enhance the celebratory experience with visual flair: “Add a confetti animation. When a user gets their first kudos, we'll send that prompt off.” Clicking the microphone icon inside the Copilot chat input converts the spoken instruction directly into an agent task, and within seconds the agent writes and applies the entire animation implementation without a single keystroke.
- The workflow relies on the VS Code Speech extension integrated directly into the chat input bar.
- The demonstration updates a web interface to trigger a confetti burst the moment a contributor receives their first kudos.
---
[11:02](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=662s) Welcoming Addy Osmani and the "Dotfiles" Analogy of Agent Skills
The stage turns to welcoming guest Addy Osmani, Director at Google Cloud AI, author, and longtime voice in developer experience and web performance. The host jokes about recently spotting Addy in a Bloomberg television feature on artificial intelligence and trying to convince his wife that he actually knew him: “I told my wife I was like, I know that guy she's like no you don't I was like I do. She didn't believe me.” The conversation drifts into shared web-development nostalgia dating back to the jQuery era before shifting toward modern AI agents, skills, and Addy’s open-source projects that recently trended on Hacker News.
To introduce agent skills, Addy shares a retro "vibe-coded" demo interface featuring a classic Windows desktop floating above a macOS dock, complete with Clippy. The host playfully warns him about Microsoft's strict mascot etiquette, noting that someone onsite actually wears a Clippy suit under an ironclad rule never to speak aloud. Demystifying the terminology, Addy explains that skills are essentially a standardized packaging mechanism for providing AI models with domain expertise and executable context beyond raw system prompts: “If you're not really sure what skills are, they're basically a standardized way to give AI agents new capabilities and expertise.” He compares them to developer dotfiles and custom shell configurations that engineers used to trade like secret magic tricks. Rather than overloading every interaction with massive prompts, skills reside in structured markdown files containing a name, description, and execution steps, loading dynamically only when the agent determines a task matches that specific capability.
```mermaid
flowchart TD
accTitle: Agent Skill Lifecycle
accDescr: Diagram showing how an agent matches an incoming user task to an on-demand skill using lightweight metadata before loading full instructions.
subgraph Input ["Task Arrival"]
Task["User Task / Prompt"]
end
subgraph Discovery ["Skill Discovery (Lightweight)"]
Index["Skill Catalog"]
Match{"Match Name & Description?"}
end
subgraph Execution ["Execution (On-Demand)"]
Load["Load Full Skill Instructions"]
Guard["Evaluate Guardrails & Red Flags"]
Run["Execute Steps & Verify Output"]
end
Task --> Index
Index --> Match
Match -->|"Yes"| Load
Load --> Guard
Guard --> Run
Match -->|"No"| Task
```
- Addy compares AI skills to the historical culture of developers sharing their personal dotfiles and shell trick configurations.
- Unlike one-off prompts, skills are designed to be reusable across an entire engineering team and multiple codebases.
- Skills operate on an on-demand loading model: the agent only checks the high-level name and description initially, loading full instructions solely when invoked by a relevant task.
---
[15:10](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=910s) Grounding Agent Skills in the Software Development Life Cycle (SDLC)
Reflecting on his decision to publish his agent skills repository, Addy explains that he had previously stepped back from open source because maintaining projects demands immense dedication: “As you know, you create an open source project, it's a lot like having a kid. You have to maintain it.” However, after seeing Y Combinator's Garry Tan release G-Stack, Addy felt compelled to share his own methodology for disciplined, AI-assisted engineering. Rather than treating AI code generation as an unguided free-for-all, Addy grounds his skills directly across the six core stages of the traditional Software Development Life Cycle (SDLC): Define, Plan, Build, Verify, Review, and Ship.
Each stage includes specialized skills equipped with clear rationalizations, explicit guardrails, red flags, and verification checks. In the Define phase, an interview skill helps developers shape vague shower ideas into concrete specs. Planning skills break broad architectural goals into granular implementation tasks. The Build phase standardizes front-end UI and API interface design, while the Verify phase automates headless browser testing and debugging routines. The Review phase applies automated code simplification and security scans before handing off to Ship skills that handle git workflows, CI integration, documentation, and launch tasks. All of these workflows can be triggered either autonomously by the agent or explicitly via slash commands inside developer environments. Addy concludes by opening a live project scaffolded with his skill set inside the Copilot sidebar, running on Google's Gemini 3.5 Flash model.
| SDLC Phase | Key Skills & Capabilities | Primary Objective |
|:--- |:--- |:--- |
| **Define** | Idea interview skill, spec-driven development | Turn fuzzy, unformed ideas into concrete functional specifications. |
| **Plan** | Task decomposition, roadmap generation | Translate software specs into actionable, sequential engineering tasks. |
| **Build** | Front-end UI crafting, API and interface design | Enforce consistent design patterns and high-quality implementation. |
| **Verify** | Automated browser testing, automated debugging | Validate running browser UI and diagnose runtime failures automatically. |
| **Review** | Code simplification, security scanning | Refactor generated code, strip dead weight, and audit for vulnerabilities. |
| **Ship** | CI pipeline config, Git workflow, release docs | Package code, generate launch documentation, and handle deployment mechanics. |
- Addy states his philosophy clearly: “I ground my agent skills around the software development life cycle.”
- Every skill file adheres to a four-part structural template: an overview, rationalizations for when and when not to invoke it, red flag tripwires, and post-execution verification checks.
- The skills integrate directly with VS Code and Copilot interfaces as slash commands, allowing developers to invoke specific phases on demand while using models such as Gemini 3.5 Flash.
[19:32](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=1172s) Model Selection, VS Code Agents View, and TDD Skill Architecture
The presenter sets up their environment using Gemini 3.5 Flash, highlighting it as an ideal model for agent tasks and agentic coding workflows. While Visual Studio Code now includes dedicated agent views populated with custom slash commands—such as `/refine` and commands for code simplification—the presenter opts to stick with the primary view to clearly display what is being generated at each stage. Before diving into live coding, they inspect the contents of their custom skill configuration files, starting with test-driven development (TDD).
```mermaid
flowchart TD
accTitle: Agentic Workflow from Concept to Spec
accDescr: Flowchart illustrating the steps of refining an idea into a concrete specification using agent skills.
The presenter stresses the importance of curating context window content deliberately. While some developers might find expansive context files overwhelming and wish to trim them back, explicit definitions ensure predictable behavior:
- **TDD Lifecycle:** Encodes strict definitions of what constitutes a failing test, a passing test, and how refactoring cycles should be structured.
- **Enterprise Engineering Principles:** Incorporates software patterns inspired by engineering practices at Google, including the classical test pyramid and decision-making guides.
- **Testing Anti-Patterns:** Outlines explicit anti-patterns to avoid, browser testing approaches, debugging methodologies, and security boundaries.
Explaining the necessity of this upfront rigor, the presenter remarks: “I try to be explicit just because I think that there's value to being intentional with how you approach these things.”
---
[22:03](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=1323s) Initiating the Habit Tracker Concept with `/refine`
Drawing from personal experience, the presenter mentions their longstanding, imperfect quest to optimize personal productivity and habits, proposing an app based on a ubiquitous developer visual: the GitHub contributions activity graph. Rather than writing a comprehensive specification upfront, the presenter enters an intentionally minimal, high-level prompt into the `/refine` slash command: a GitHub-inspired habit tracker that operates purely in the browser without any database backend. When the co-host asks whether they plan to explicitly specify the contribution graph in the prompt, the presenter chooses to hold back: “I could say contribution graph, but that's being explicit.”
The agent processes the request, loads the refinement skill, and immediately engages the user with focused architectural and scope-clarifying questions:
- **Target Audience:** Clarified as a general developer audience, such as software engineers attending Microsoft Build or viewers watching along online.
- **Storage Strategy:** Confirmed as simple browser `localStorage` rather than IndexedDB or session storage.
- **GitHub Analogies:** Despite the agent asking whether git metaphors like "committing a habit," branching variations, or pull requests should be included, the presenter elects to keep the scope constrained strictly to the visual contribution grid.
- **MVP Expectations:** Prioritizing fast load times, a clean responsive user interface, and zero immediate need for offline-first service worker architecture.
---
[25:10](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=1510s) Evaluating Directions, Cognitive Debt, and Scoping the "Streak" MVP
After synthesizing the answers against the fresh project directory, the refinement skill generates three distinct product directions:
- **Direction A (Dev Profile Configurator):** A minimalist, high-fidelity take on the GitHub profile view where habits are monitored similarly to repositories.
- **Direction B (Keyboard-Driven Habit Terminal):** A terminal user interface (TUI), prompting the co-host to joke that terminal interfaces are trending heavily across developer tools right now.
- **Direction C (Markdown/SVG Exporter):** A plain-text data management tool oriented toward raw habit data entry and export.
The presenter selects Direction A. Observing that the model did not pre-select an option, the co-host reflects on how models typically nudge users toward specific choices, leading to a quick exchange about developer psychology. The co-host notes how easy it is to automatically accept an AI's recommendation under the assumption that the model knows best. In response, the presenter quipped: “I have so much to say about cognitive debt and cognitive surrender.”
The agent outputs a structured product breakdown for the app, dubbed **Streak**. It details validation assumptions (frequency of habit tracking, mobile responsiveness versus desktop configuration), sets explicit non-goals (no cloud synchronization, no advanced social features), and prompts to save the resulting artifact directly to the project files.
---
[28:11](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=1691s) Transitioning from Refinement to Detailed Specification via `/spec`
Once the refinement output file is saved to the workspace, the presenter prepares to transition from the raw product concept to an engineering specification. Addressing different developer workflows, the presenter acknowledges that while some engineers prefer jumping straight from a high-level thought to writing code, breaking the lifecycle into modular stages provides greater control over output quality. The co-host playfully contrasts their own style: “You want the plan, the spec, and then the implementation. I'm a plan implement guy myself.”
The presenter triggers the next agent skill by executing `/spec`. The agent reads the refinement document generated in the previous step, ingesting the core assumptions, constraints, and success criteria. It then begins systematically scaffolding the complete architectural specification, defining the target technology stack, concrete interface contracts, and implementation milestones.
[29:13](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=1753s) Refining Questions and Finalizing the Specification
The demonstration moves into reviewing the open questions surfaced by the agent. The system responds rapidly, prompting a side remark to simply run Gemini 3.5 Flash or whatever leading model is active at the time, given how fast AI shifts every few hours.
```mermaid
flowchart TD
accTitle: Spec Generation and Plan Decomposition
accDescr: Flow diagram showing inputs to spec creation and subsequent task breakdown.
subgraph SpecPhase["1. Specification Generation"]
A["Open Questions (Grid tech, defaults)"] --> B["Agent Generates Spec"]
B --> C["Performance metrics & tech stack defined"]
end
subgraph PlanPhase["2. Plan Decomposition"]
C --> D["/plan command"]
D --> E["Scaffold boilerplate & task list"]
E --> F["Granular phases with TDD acceptance criteria"]
end
```
The agent prompts for architectural decisions:
- Whether to render the GitHub-style contribution grid using dynamic SVG group structures or CSS grid layouts (the speaker decides to let the agent pick what it thinks is best).
- What default habits to pre-populate for first-time visitors; after joking about doom-scrolling, the hosts settle on hydration, exercise, and meditation.
- Where to place the generated file (left to agent discretion).
All of this prompting happens directly inside the Copilot sidebar without context switching. The generated spec establishes success criteria, performance metrics (such as Largest Contentful Paint), project structure, code style, and a default tech stack of vanilla JavaScript with Playwright for end-to-end testing. The speaker notes that stack choices can either be encoded directly into custom skills or adjusted iteratively with the agent.
[31:43](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=1903s) Generating Structured Plans and Limiting Blast Radius
Once the specification is locked in, the next step is breaking down the project using `/plan`. For tiny projects, jumping directly into implementation makes sense, but for larger efforts, dividing the work into granular tasks supports test-driven development (TDD) by keeping each piece independently verifiable.
The agent scaffolds the initial boilerplate, adds `package.json`, and outputs a structured task list under `plan/`. Phase 1 focuses on pure domain logic:
- TypeScript interfaces and pure functions for computing streaks.
- Calculating total contributions over the trailing 365 days.
- LocalStorage integration without touching the UI.
- Strict verification gates requiring test suites to pass before proceeding.
The speaker highlights the value of specifying the exact files to touch in advance: “If you've been doing any vibe coding or agentic engineering for a while, you know that one of the ways in which we try to limit our blast radius is by saying, well, I'm just going to target these specific files because especially if you're working on an existing project, you don't want it to start overwriting or touching files that maybe have nothing to do with the task at hand, right?”
[34:34](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2074s) Granular Execution vs. Autonomy and Bypassing Approvals
Executing the plan can be done in fine-grained steps, such as `/build task 1`, which loads the project's test-driven development skill and begins installing dependencies. Burke Holland points out a setting at the bottom of the UI to bypass approvals, which the speaker enables to keep momentum going.
```mermaid
flowchart LR
accTitle: TDD Build Loop
accDescr: Shows the red-green test cycle executed by the autonomous agent.
A["Failing Target Test (Red)"] --> B["Implement Minimal Logic"]
B --> C["Pass Test Suite (Green)"]
C --> D["Advance to Next Task"]
```
The hosts discuss the tension between granular step-by-step guidance and letting the agent run completely end-to-end:
- For complex software, verifying slices one by one gives peace of mind that earlier layers remain intact.
- However, requiring human approval introduces friction: “It's such a trade-off though because then you know the back in the loop, like once again, you're the problem. You're slowing the agent down. It's tough. I struggle with this myself.”
- Teams must decide whether to invest heavily upfront in specs, rely on test-driven development throughout, or generate large swathes of code and review afterward.
The agent demonstrates the red-green TDD pattern by introducing a failing unit test for streak logic before implementing the clean code necessary to pass it.
[37:28](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2248s) Monitoring Agent Trajectories and Mental Models
As the build loops through subsequent tasks autonomously, the conversation shifts to how engineers actually supervise agents while code is being written. Instead of context-switching to other work or attempting to read every single line of generated code, the speaker pays close attention to the agent's thought trajectory:
- Reviewing reasoning tokens and intermediate notes to understand why the agent chose a particular path.
- Catching subtle misalignments early before they compound.
- Burke admits that he often finds himself just watching the agent spin instead of multitasking.
For throwaway personal scripts or quick tools needed for only an hour, deep code reviews may be unnecessary, but on long-term production systems, understanding the agent's logic remains vital.
[38:41](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2321s) Throwaway Code vs. Long-Term Maintenance and the TDD Cycle
The discussion starts with the trade-offs between understanding generated code versus letting an agent take complete control. When writing throwaway code, trusting an agent completely is fine: if something fails, the agent can simply fix it. But when building something intended to be maintained for weeks or months, reading through core files and understanding the overall architecture is essential so that a developer can step in when the agent runs off on tangents or fails to resolve a bug.
```mermaid
flowchart LR
accTitle: Test-Driven Development Slice Cycle
accDescr: Simple loop where tests are written failing then made to pass as slices are implemented.
subgraph TDD["TDD Phase"]
A["Write failing tests (Red)"] --> B["Implement slice (Green)"]
B --> C["Verify and complete"]
end
```
Meanwhile, the build agent steadily progresses through its Test-Driven Development (TDD) phases, stepping slice by slice from red to green. One speaker remarks that they had never encountered the concept of red-green testing until Spec Kit came out, joking that finding out about writing failing tests first at forty-five years of age was “kind of sad.” The other host quickly replies that they do not look over forty-five at all, sparking a brief, playful exchange about birthdays—revealing they turned forty-five just the day before.
[39:52](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2392s) The Agentic Engineering Spectrum: Solo Founders vs. Enterprise Reality
The speakers pivot to critique how best practices are communicated across the agentic engineering ecosystem. Advice on Twitter and social media often applies to solo founders or early startups operating on greenfield repositories without users, where giving in to "vibes" and letting the model fix errors on the fly is acceptable. That workflow falls apart in enterprise legacy codebases with real users. Developers find themselves wondering whether they need agent skills, Superpowers, G-Stack, or another tool altogether, but the underlying role has shifted from writing lines of code to orchestrating functional workflows. One speaker jokingly asks if software development has turned into management consulting: “Like we all work at McKinsey now.”
The presenter outlines that developers now operate across a continuous spectrum between vibe coding and disciplined agentic engineering:
> “And there are going to be times when we're vibe coding, there's going to be times when we're agentic engineering. And the more diligence you have to apply to the process, the closer you are getting to agentic engineering.”
- Solo founders on greenfield stacks can afford pure vibe coding, but production systems demand strict quality gates.
- While building a simple habit tracker requires minimal overhead, shipping software at the scale of VS Code requires meticulous guarding against subtle regressions.
[42:13](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2533s) Verification: In-Editor Headless and Browser Testing
With around ten minutes remaining, the presenter skips ahead to demonstrate the `verify` skill within the Agent Skills package. This step ensures that the partial or complete implementation actually runs and functions properly in a browser environment. The agent executes `npm run build` followed by `npm run dev`, opening a live browser window directly inside Visual Studio Code. The running app renders a mock GitHub profile, prompting a laugh when an unfamiliar profile photo appears on screen. The hosts joke about the model pulling random profile pictures from the internet, musing that the mystery person might just be someone somewhere in the world who likes to bake cookies and happens to be on GitHub.
Behind the scenes, the verify step triggers automated checks using the `browser-testing-with-devtools` skill to see if UI interactions break runtime behavior:
- Tooling choices for automated browser validation are broad, encompassing Puppeteer, Playwright, and Vercel’s agent-browser tooling.
- The speakers note that web developers have a distinct tooling advantage here, as spinning up and automating end-to-end browser tests in editor environments remains much easier than running tests for native applications.
[44:23](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2663s) Code Review Gates and Simplification Loops
The conversation moves to the `review` phase. Drawing from discussions with Google engineers, the presenter points out that while manual code review remains standard practice, automated local multi-pass reviews provide immense value before human inspection. Some teams push this to adversarial multi-model reviews (for example, pairing Gemini for implementation with Codex or Claude Opus for review). In the demo run, the review skill outputs an evaluation spanning multiple categories:
- **Correctness:** Verifying that all unit tests pass.
- **Security:** Scanning vanilla JavaScript code for issues like cross-site scripting (XSS).
- **Readability & Architecture:** Checking structural clarity.
- **Recommendations:** Suggesting accessible enhancements, such as adding keyboard navigation to the contribution graph grid.
The presenter also highlights code simplification loops. After an agent implements a feature and tests pass, developers can run a simplification pass to clean up bloated logic while relying on the existing test suite to ensure no functionality breaks.
[46:43](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2803s) Non-Deterministic Review vs. Real-World Security Concerns
The co-host raises a pointed question about trust: how much can an engineer truly rely on non-deterministic LLM-based security reviews when critical issues like exposed secret keys or SQL injection vulnerabilities are on the line?
The presenter acknowledges that while specialized commercial and open-source tools go deeper, encoding security checklists into agent skills provides a vital baseline safety net. Vibe coding often leads to hazardous patterns, such as hardcoding API keys, introducing data leakage, or creating rate-limit vulnerabilities where external endpoints can be hammered repeatedly. Whether a developer is inexperienced or simply working fast, the model acts as an effective automated sanity check:
> “Like if you ever ask a model to review a code base, it will almost always find something.”
[48:19](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2899s) Deterministic Gates and the Limits of Fixing Non-Determinism with Markdown
The conversation turns to the critical vulnerability of releasing agentic code directly into production without rigorous validation. The hosts reflect on how meticulous automated review agents can be when pointing out necessary input changes, but express genuine apprehension about shipping AI-generated projects without strict boundaries. One host admits being terrified of posting a freshly built app online only for someone to instantly extract sensitive credentials, emphasizing: “I don’t trust—I’m scared to death to push anything live on Twitter and be like, ‘Here’s what I built,’ and then have somebody be like, ‘Oh, and here’s your Gemini key.’”
```mermaid
flowchart TD
accTitle: Quality Gate Pipeline for Agentic Code
accDescr: Flowchart showing agent code passing through skills and deterministic verification before shipping.
subgraph Dev ["Agentic Development"]
A["Agent Generates Code"] --> B["Agent Skills (Best Practices)"]
end
subgraph Gates ["Enforcement Gates"]
B --> C{"Deterministic Gate"}
C -->|"Fails Security/Perf"| A
C -->|"Passes Sign-Off"| D["Verified Build"]
end
subgraph Deploy ["Release"]
D --> E["Ship to Production"]
end
```
In response, Addy Osmani argues that development teams must clearly define what “good” and “done” mean across dimensions like security, performance, accessibility, and overall code quality. While skills provide a way to encode these best practices directly into prompt instructions, both agree that prompt-level guardrails alone are insufficient. The host wryly observes the industry's circular habit of relying on text files to regulate generative output: “Right now, Markdown is the solution to everything. It’s like, well, we have non-determinism. How are we going to fix that? With more non-determinism—with text files.”
- Teams must define deterministic quality gates for security, quality, performance, and accessibility before merging.
- Agent skills serve as an encoding mechanism for developer standards, but need enforceable tooling behind them.
- Relying purely on markdown prompt instructions leaves projects vulnerable to leaked API keys and regression risks.
---
[49:40](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2980s) Code Simplification Labs and the Offline Streak Demo
Wrapping up the workflow demonstration before opening for audience questions, Addy highlights the value of studying code simplification patterns published by open-source labs and engineering blogs. By tracking what external teams write about refactoring and code cleanup, developers can fold those patterns into their custom agent skills. Even on the modest demo application on stage, the simplification pass identified three or four distinct areas to streamline and clean up the implementation.
To respect the remaining session time, Addy skips the live deployment step and presents a pre-built reference app called Streak. The app features a GitHub-style contribution heatmap grid at the bottom that tracks daily personal productivity habits, such as spending mornings screen-free, planning workday goals, and conducting end-of-day reflections. Opening the browser’s developer tools, Addy navigates to the Application panel's local storage view to demonstrate that all habit entries live directly on the client. The application is completely local, operates fully offline without an external database, and fulfills its tracking objectives cleanly.
---
[51:31](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3091s) Q&A: Model Tiering (Flash vs. Pro) and MCP vs. CLIs
An audience member named Anthony asks how developers should split their workload between lighter, faster models and larger flagship models—specifically asking whether it makes sense to handle planning in Flash and implementation in Pro. Addy validates that pattern immediately: “That’s exactly my workflow. I tend to, and especially if you're trying to optimize for cost, that's a great way to go about it.” A lower-cost model can digest prompts and outline project architecture efficiently, preserving token expenditure until a high-capacity model like Gemini Pro, Claude Opus, or OpenAI Codex is needed to write the actual implementation.
```mermaid
flowchart LR
accTitle: Tiered Model Workflows
accDescr: Pipeline showing cost-effective planning in fast models moving to heavy implementation in pro models.
subgraph Plan ["Planning Phase"]
P["Gemini Flash / Low-Cost Model"]
end
subgraph Code ["Implementation Phase"]
I["Gemini Pro / Opus / Codex"]
end
P -->|"Optimized Plan & Context"| I
```
The hosts then pivot into a debate on the Model Context Protocol (MCP) versus traditional Command Line Interfaces (CLIs). Addy observes that workflows are rapidly evolving because agents often achieve goals far more efficiently by running existing terminal CLIs directly rather than exposing sprawling collections of MCP tools. While the host notes that some engineers, such as Rhys from OpenCode, argue MCP excels at solving authentication and granular permission boundaries that CLIs struggle with, both agree that skills and CLIs handle most daily tasks with less friction. For MCP usage, the host singles out Work IQ for Microsoft environments and the Context7 MCP server, urging developers to install it so their agents always look up current documentation before writing code.
- **Model Tiering Strategy**: Use low-cost, fast models (e.g., Flash) for planning and architecture; use high-capacity models (e.g., Pro, Opus, Codex) for code generation and refactoring.
- **MCP vs. CLI Friction**: CLIs are often simpler and more token-efficient for agents to invoke than configuring massive MCP tool definitions, though MCP holds an advantage for managing strict authentication and permissions.
- **Recommended MCP Tools**: Work IQ (enterprise Microsoft context) and Context7 (pulling live, accurate documentation into agent workflows to bypass model knowledge cutoffs).
---
[55:08](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3308s) Context Window Economics and Progressive Skill Loading
The discussion concludes with an analysis of token economics and the architectural limitations of context windows. The host critiques the current state of LLM interaction: “Honestly I feel like we need a better solution than a context window in general at some point in the future. This is like a really rudimentary primitive for interacting with LLMs.”
Because MCP requires advertising all tool specifications upfront, it consumes precious context tokens before any work begins. Agent skills, by contrast, load progressively:
1. The model initially receives only the skill name and a one-sentence summary.
2. If the user query requires that capability, the model loads the full skill instructions on demand.
3. If the skill references auxiliary files or documentation, those resources are fetched only when referenced.
This lightweight footprint makes skills significantly more economical as development shifts from unlimited AI usage to metered, quota-constrained token budgets. Following this point, the host thanks Addy Osmani for demonstrating his skills repository and wraps up the interview segment.
---
[56:43](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3403s) GitHub Copilot SDK Preview and Copilot CLI Enhancements
The broadcast transitions to technical updates regarding the GitHub Copilot ecosystem, led by the announcement of the GitHub Copilot SDK in technical preview across Node.js, TypeScript, Python, Go, and.NET. The SDK gives developers programmable control over multi-turn conversations, automated tool execution, and complete conversation life cycle management.
Alongside the SDK, significant feature updates have landed in the GitHub Copilot CLI over recent weeks. The CLI now supports expanded model selection and introduces built-in specialized agents for exploring codebases, planning tasks, and conducting automated code reviews, all of which interoperate directly with agent skills.
Installation and distribution workflows have also been streamlined. Developers can now install the CLI via package managers such as Homebrew and WinGet, or through automated installation scripts that support background auto-updates, while traditional standalone binary releases remain supported. Recent releases also include new command-line script flags, visual improvements to git diff views, and updated web-based controls for reviewing agent output.
[57:52](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3472s) Welcome to Microsoft Build 2026 and the Copilot SDK GA Announcement
The session kicks off from a vast open broadcast stage at Microsoft Build 2026. While actively inviting the audience to leave feedback, wishlist items, issues, and repository discussions, the host welcomes Patrick Nicoletich from the unified Copilot CLI and SDK team. The two banter about the cavernous venue, wondering aloud whether it was originally an airplane hangar or a boat yard, before turning directly to developer tooling.
Patrick shares the major milestone for the team: the Copilot SDK reached General Availability (GA) 1.0 stable just the day before. Having been developed since January, the SDK spent months in technical preview and internal use across Microsoft properties—including LinkedIn, Xbox, GitHub, Office Excel, and Copilot Studio. The goal of this internal phase was driving up quality and creating a unified agent harness across products before opening it to everyone. With the 1.0 release, developers can deploy the SDK straight into production environments.
Patrick sums up the milestone and the support readiness:
> “And so that journey internally is now going outward where GA, which means customers can use the SDK stable, they should stick it in production, we can support it in a wide array of environments and and we're here to help you put our harness and whatever solution makes sense for you.”
[01:00:26](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3626s) Architecture, Multi-Language Support, and the Rust Runtime Rewrite
Patrick clarifies what the Copilot SDK actually is: rather than a standalone agent, it is a thin client layer sitting on top of the established Copilot CLI server. This architecture allows developers to bring the exact same agent harness powering the GitHub Copilot CLI into any custom application. The SDK launched its technical preview with four programming languages and has since expanded to six with the addition of Rust and Java.
```mermaid
flowchart TD
accTitle: Copilot SDK Architecture and Runtime Evolution
accDescr: Diagram showing client languages interfacing through the Copilot SDK thin client down to the Copilot CLI server, contrasting the legacy Node runtime with the new Rust runtime.
subgraph Languages["Supported Languages (6 Total)"]
L1["Node / TypeScript / Python /.NET"]
L2["Rust"]
L3["Java"]
end
subgraph ClientLayer["Copilot SDK"]
SDK["Thin Client Layer"]
end
subgraph Runtimes["CLI Server Engine"]
direction TB
NodeRT["Legacy Node Runtime<br/>(100–120 MB RAM)"]
RustRT["New Rust Runtime<br/>(Sub-10 MB RAM, Native Support)"]
end
Patrick and the host discuss the aggressive performance re-engineering happening under the hood. While the CLI server was originally built on Node.js—consuming roughly 100 to 120 MB of RAM—the team is actively rewriting the runtime in Rust. Early proofs of concept have proven that the Rust implementation improves performance across every dimension while cutting the memory footprint down to sub-10 MB, unlocking native support across constrained and mobile environments.
- **Language coverage**: 6 languages supported (initial 4 from preview plus Rust and Java).
- **Memory footprint reduction**: Dropping from ~100–120 MB (Node.js) to sub-10 MB (Rust).
- **Architecture**: A lightweight client calling into the shared, local Copilot CLI server.
[01:02:15](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3735s) The Paradigm Shift: Token Abundance and Cross-Language Development
The conversation pivots to how agentic tooling changes software engineering habits. The host recalls an instance where he asked the Copilot CLI how to approach a project, and the agent suggested Go. Even though he did not know Go, he realized that personal language mastery is no longer a blocker when the agent possesses that knowledge. Patrick affirms that teams across GitHub have experienced a similar mindset shift over the past six to seven months.
The speakers reflect on the impact of token constraints. Having unmetered token access removes friction and dramatically accelerates testing from conception to implementation. The host jokes that generous AI token allowances should become a standard compensation benefit alongside corporate healthcare, observing how rationing tokens on expensive models like Claude Opus stalls developer flow.
Patrick describes the cultural luxury and operational benefit of testing model limits:
> “It's one of the luxuries that we have at GitHub to just burn a ton of tokens to really see what the limits are of the models themselves.”
Patrick highlights the core advantage of unconstrained exploration:
> “Exactly, mainly because you can just shorten the gap between an idea to outcome, yeah.”
[01:03:43](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3823s) Live Demo: The "Whim" Canvas-Based PM Workspace
Patrick transitions to a live demonstration of how he uses the Copilot SDK for product management work. Acknowledging that technical preview users initially built personal assistants, Patrick explains that as a PM, his daily output is centered around clear writing, syntheses, and specifications rather than code alone. To supercharge his workflow, he developed an application called "Whim," designed to be summoned on a whim via hotkeys from anywhere in the operating system.
Whim flips the traditional chat interface upside down. Rather than a chat drawer that occasionally outputs formatted text, Whim uses an inverted writing-first canvas. A PM can organize spaces, invoke task-specific skills, and summon various autonomous agent personas directly into the canvas. Patrick highlights two primary agent capabilities within the app:
1. **Ephemeral Cloud Agents**: Workers that spin up in the cloud, unconstrained by the local machine's memory or battery. Users can close their laptop, walk away, and have the agents persist, resume, and deliver deliverables.
2. **Development Agents**: Automated coding agents that check out repositories, inspect files, and apply pull requests or modifications on demand.
[01:06:10](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3970s) Multi-Agent Canvas Workflows and Sandboxed Execution
Patrick demonstrates the operational structure of his custom agent setup inside his workspace canvas. His setup relies on distinct agents: an "editor" agent, which serves as his primary driver because he is constantly drafting and modifying content, and a dedicated sandbox environment. If he needs to run code locally that could potentially be dangerous, he can dispatch the agent into a secure sandbox environment that shipped just the day before.
```mermaid
flowchart TD
accTitle: Multi-Agent Canvas Architecture
accDescr: Diagram illustrating how tasks written on a canvas dispatch into sandbox sessions and update the canvas directly.
subgraph Input ["User Canvas"]
A["List of Tasks / Issues"] --> B["Tag Agent: @editor deploy"]
end
subgraph Execution ["Agent Orchestration"]
B --> C{"Execution Target"}
C -->|"Dangerous Code"| D["Secure Sandbox"]
C -->|"Normal Task"| E["Active Session"]
E --> F["Automated PR / Fixes"]
end
subgraph Output ["Synchronized Results"]
D --> G["Live Canvas Updates"]
F --> G
G --> H["Rollup / Living Record"]
end
```
He explains that the ultimate goal is closing the distance between user intent and actual outcome, minimizing the repetitive intermediate steps that developers typically endure. To illustrate, Patrick writes a simple item on his task list to triage GitHub Copilot SDK issues and types `@editor deploy`. By tagging the agent directly inside the text, the task is immediately set in motion without requiring separate prompts.
The triggered session surfaces instantly within the document interface:
- Users can inspect active chats directly within the canvas or observe activities running across multiple canvases simultaneously.
- Every session features management controls, including remote-connection links to control the session from a mobile phone and a toggle for "Yolo mode" to let agents run unconstrained without manual approval prompts.
- Patrick explains the broader philosophy: “So this idea of multi agent orchestration, not really feeling like multi agent orchestration is where I think in general we're all headed because there's a lot of cognitive load.”
- The host agrees with this shift away from isolated silos, pointing out that having dedicated coding agents versus email agents makes little sense; a unified assistant should accompany the user everywhere, whether writing code or triaging email in Outlook.
[01:08:19](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4099s) Grounding AI Beyond the "Snake Oil"
Patrick highlights that his canvas interface removes the need to constantly monitor chat transcripts. The generated output, pull requests, and bug fixes feed directly back onto the canvas itself. Working on his custom workspace system (Wim) from within Wim, Patrick simply feeds it bugs, lets it open PRs, and watches the canvas update automatically. This turns the canvas into a dynamic record of execution that encapsulates the entire arc of work as it happens.
The host expresses frustration with the broader AI narrative online, remarking on how frequently people claim tools like "open claw" run their entire life, yet they cannot articulate what tasks are actually being automated. This lack of specificity makes much of the current agent discourse feel like snake oil. Grounding agent interactions in clear, tangible workflows that people can easily comprehend is critical:
- The GitHub Copilot SDK allows developers to rapidly assemble custom workflows in a weekend of agent-assisted coding, replacing what used to require weeks or months of engineering.
- Ephemeral cloud sandboxes (now available in preview alongside local sandboxes) give agents freedom to execute commands safely without host system restrictions.
- The host strongly advises developers against waiting for out-of-the-box magical solutions: “Build your own that works the way that you want to work and start just trying to automate the simple things that you're doing throughout the day.”
- Patrick emphasizes that building tailored agents is the best way to cultivate intuition for model nuances: “...we all have to build intuition how the models perform and they behave similar to when you engage with a person, you're trying to understand the nuances of other person themselves and trust is built over time.”
- The host jokes about having an ongoing personal rapport with his own CLI agent named Max, quipping about whether that relationship borders on psychosis, which Patrick lightheartedly confirms is necessary to fully leverage the technology.
[01:12:15](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4335s) Model Preferences: GPT-5.5 vs. Claude Opus and Copilot SDK Sessions
When asked about his go-to foundation model, Patrick reveals that he has shifted primarily to GPT-5.5 over recent weeks, having previously relied heavily on Anthropic's Claude Sonnet and Claude Opus models for nearly two years. He advocates for pairing models with complementary strengths across the development lifecycle:
- Patrick utilizes multi-model verification: “And so I'm a big, you know, let one model do the implementation, let one model verify.”
- GPT-5.5 excels specifically at code reviews and verification, whereas Opus remains exceptionally strong for generating initial code implementations and design thinking.
- The host echoes this exact sentiment, noting he uses Claude Opus for architecture, design, and creative ideation, but defaults to GPT-5.5 for direct code execution and implementation.
- Patrick highlights his upcoming breakout session dedicated to the Copilot SDK and related work across Microsoft, scheduled for 4:00 PM to 5:00 PM in Breakout 1.
[01:13:53](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4433s) GitHub Remote Control Sessions
A transition segment features GitHub team members highlighting remote control capabilities across developer environments. Ori, a Senior Product Designer at GitHub, explains how remote control sessions integrate GitHub Copilot CLI and VS Code directly with GitHub Mobile. This architecture lets developers initiate workflows on their primary machines and seamlessly continue monitoring or managing them while away from their desks.
Josh, an engineer at GitHub, describes the underlying engineering effort to bring all GitHub agentic offerings into a shared protocol. This shared language enables developers to initiate tasks locally via Copilot CLI, view all running agent sessions inside a single unified dashboard, and respond to prompts remotely to maintain continuous development velocity.
[01:14:39](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4479s) Freeing Developers from the Desk
The session transitions from a recorded segment discussing how mobile interaction lets developers step away from their workstations while maintaining control of their workflows. Being able to review and direct ongoing tasks remotely frees developers to move around rather than remain tethered to their hardware. As the screen returns to the live broadcast stage, hosts discover they are already on air, cutting off a humorous remark about an app that acts like a purring cat until an unexpected "goblin mode" is triggered.
[01:15:22](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4522s) Introducing Evan Boyle and GitHub's Core Agent Infrastructure
The stage host welcomes the audience and introduces Evan Boyle, an engineering manager on the Copilot CLI team at GitHub and an enthusiast of "all things fish." When asked what his role entails, Evan explains that he insists on staying deeply technical. He notes that leading developer tool teams requires firsthand engagement with code:
> “I don't think that you can build dev tools and be a manager in that space without writing a lot of code yourself and like living the life of your users.”
Evan details his team's responsibilities: they build the core agent loop powering the Copilot CLI and the Copilot SDK, which is increasingly adopted across agents throughout Microsoft and GitHub. His team also maintains the runtime, the CLI itself, and the newly revealed GitHub app showcased during the keynote. The host jokes about Evan working late with them at GitHub headquarters until midnight just three nights prior to rewrite their entire presentation demo from scratch.
[01:16:35](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4595s) Live Demo: From Community Issues to an Electron App
Addressing a common question encountered earlier on the commuter bus—"what is the Copilot CLI?"—Evan walks through a practical demonstration. During a previous presentation with Cassidy, they created a public repository and invited the audience to submit issues defining what software to construct on stage. Sixty issues were filed in short order. Inside the Copilot CLI, hitting Tab exposes open issues and pull requests directly within the interface. Evan instructed the agent to analyze the submissions, select the top candidate, and plan out implementation.
subgraph INTAKE ["Audience Input"]
A["60 Community Issues Submitted"]
end
subgraph CLI ["Copilot CLI Loop"]
B["Tab view lists Issues & PRs"]
C["Agent analyzes & selects top issue"]
D["Plan Mode generates architecture"]
E["Autopilot implements Electron App"]
end
subgraph ACTION ["Execution & Release"]
F["Voice command: Hold spacebar"]
G["Direct push / force push to main"]
H["Agent auto-closes issues with thank-you note"]
end
A --> B
B --> C
C --> D
D --> E
E --> F
F --> G
G --> H
```
The community voted for an Octocat widget that showers the screen with emojis whenever a Git operation runs. Using plan mode, the agent synthesized an implementation blueprint and proceeded autonomously on autopilot. Audience members then submitted follow-up feature requests, including:
- "Goblin mode" (a nod to a prior period when GPT models frequently fixated on goblins before OpenAI addressed it in a research paper)
- Sound effects, causing the cat icon to purr periodically and meow on commit
- Slower animation speeds
- Interactive cursor tracking (which was omitted from the final cut)
Evan triggers voice mode by holding down the spacebar, speaking into the visual audio meter to instruct the CLI: "instead of opening a PR, I want you to just push directly to main." He admits this is fine for an ad-hoc conference repository, joking that he would never force-push to official team repos. In total, the full Electron desktop app was generated in roughly 20 to 25 minutes, consuming about 1,300 credits—estimated at only $5 to $10 in model usage rather than an exorbitant token bill.
[01:19:56](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4796s) The Appeal of Terminal Interfaces and Parallel Sessions
The host asks why developers would choose a CLI over a traditional graphical IDE when orchestrating AI agents. Evan explains that the terminal removes UI clutter and helps him maintain intense focus while keeping log outputs visible. Furthermore, terminal workflows excel at parallel execution through tools like tmux or multiple windows arranged across large displays. To make multitasking seamless on constrained screens or single laptops, the Copilot CLI includes a dedicated session management system:
- Typing `/new` spins up a completely separate conversation thread while running existing tasks in the background.
- Typing `/sessions` displays an interactive dashboard of all active and suspended jobs, allowing rapid toggling between distinct agent loops.
Evan demonstrates this capability by initiating a background session with `/new` to crawl open issues and read the project's `README.md`. Meanwhile, he returns to the original thread and orders the agent to close out the completed community issues with a grateful reply to contributors.
The hosts marvel at the modern terminal experience. Many developers view the CLI as an intimidating environment filled with obscure keybindings, but modern Text User Interfaces (TUIs) have evolved to look and behave like responsive applications. Evan points out that the terminal interface actually exposes fewer keyboard shortcuts than Visual Studio Code, and the team even added mouse-click support to terminal tabs when users instinctively clicked on them.
[01:23:52](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5032s) Team Velocity and Massive Pull Request Scale
Evan shares the staggering pace of development within GitHub's AI-focused engineering groups. Because the team building Copilot tools relies extensively on AI agents for their own day-to-day work, code changes occur at unprecedented speed. Evan recounts tracking team throughput:
> “I think back in March I did a survey and with 10 people we were shipping 2000 PRs a month across app, CLI, SDK, agent loop.”
Due to this velocity, returning to an area of the codebase after even a few weeks feels like encountering brand-new software. Evan notes that if he wrote a feature a month prior, he must reconstruct his entire mental model of the project because the repository changes drastically week over week. When asking Moss, the engineer behind the new CLI interface, how heavily he relies on agentic coding, Moss affirmed that his day-to-day workflow is driven largely by AI agents.
[01:24:54](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5094s) Slash Commands and Multi-Model Code Review in the Copilot CLI
The conversation turns to favorite features tucked away behind slash commands inside the GitHub Copilot CLI. Evan starts a fresh session, bumps up the font size, and launches Copilot in "Yolo mode"—a setting he admits he does not always run, but uses here so the entire process streams live without constant approval interruptions.
```mermaid
flowchart TD
accTitle: Multi-Model Review Pipeline
accDescr: Three separate LLMs review code changes and pass their findings to an agreement filter before outputting issues.
subgraph Models["Parallel Evaluators"]
M1["Claude 3.7 Sonnet"]
M2["GPT-5.5"]
M3["Gemini"]
end
subgraph Consensus["Consensus Stage"]
Filter["Agreement Filter\n(Intersection of Findings)"]
Result["Final Consolidated Issues"]
end
M1 --> Filter
M2 --> Filter
M3 --> Filter
Filter --> Result
```
Evan demonstrates `/review` for multi-model code verification. Speaking into his microphone, he commands: “I'd like for you to do multi model code review with Sonnet, GPT 5.5 and Gemini. I'd like you to then compare the results and only feed me issues where the models are in agreeance.” The co-host, Burke, playfully pauses on whether "agreeance" is even a valid word, but Evan reassures him: “That's the beauty of, you know, LLMs is they, they, they pick up what you're putting down.” Burke observes that users often waste time backspacing minor typos when typing to agents, even though you can simply “bash the keyboard” and send it directly because the models easily resolve human errors.
- The `/review` command runs multiple frontier models in parallel to eliminate single-model hallucinations and isolate verified bugs.
- LLMs naturally parse slang, coined words, and keyboard typos without requiring manual prompt edits.
[01:26:25](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5185s) CLI Elicitation and Customizing Model Brevity
When Evan triggers the review on a clean `main` branch with no staged diffs or ahead-commits, the CLI handles the ambiguity through a mechanism called *elicitation*. Rather than failing silently, the interface asks whether to inspect the entire repository, a specific branch, or the latest commit merge. Prompted for an exact commit hash, Evan casually instructs it to “figure it out” using best judgment. The speed of the agent prompts a discussion on how AI tools naturally default to excessive output. Evan notes: “I have time to read everything, the agent says. They write walls of text, man.” Burke vents about receiving bloated, AI-generated emails: “People send me an e-mail that looks like they spent an hour on it that was written by an agent. You're wasting my time. Boil it down to one sentence.”
Evan explains that the team has experimented with configurable verbosity levels inside the GitHub mobile app and plans to bring those granular controls to the CLI. In the meantime, users can customize behavior on the fly using skills, hooks, or custom instructions. Evan demonstrates this by holding down the space bar and dictating a direct prompt: “I'd like for you to add to my custom instructions that I want you to be as brief and concise and to the point as possible. Unless I ask otherwise like one sentence. Make it easy for me to read and scan.” Hitting enter automatically modifies the repository's `.copilot/instructions` file. Evan points out that repository-level files make sense for solo projects, while shared team repositories are better paired with user-specific preferences stored in the personal global configuration directory.
[01:29:09](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5349s) Local Voice Mode and Speech-Driven Workflows
Burke asks how Evan operates the voice-dictation interface. Evan enters `/voice` to show the underlying configuration, revealing support for dedicated offline models such as NVIDIA's Nemotron family in English and Spanish. Evan highlights that these 700 MB voice models run completely on the local device, meaning developer speech is streamed and processed on-device without incurring latency or sending audio streams to external cloud inference providers.
When Burke surveys the live audience, only a tiny fraction of developers admit to speaking to their machines; most still prefer typing. Burke admits that speaking feels socially unnatural, remarking that talking aloud makes you look like you are talking to yourself: “I don't know about y'all, but when I speak I don't think obviously well.” Evan credits Scott Hanselman with converting him to voice workflows. During early weekly dogfooding sessions for the GitHub app, Evan observed Hanselman using Handy and dictating everything. Although initially skeptical, Evan found that speech unlocked a completely different way to program:
> “I realized I'm kind of a person who likes to pace around my office while I think, if that makes sense. So I have like a little USB remote that's hooked up to that shortcut. And I pace around my office with my headphones on and I ramble at my agent.”
Burke jokes that the industry has circled straight back to 1965 executive dictaphones. As Evan wraps up his appearance, he promotes his upcoming 4:00 PM session with Mario covering GitHub app agent workflows, following up on his earlier deep dive with Cassidy on the Copilot CLI. Burke issues a final challenge to the audience to push past their initial skepticism and test the Copilot CLI tools directly in everyday work.
[01:32:45](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5565s) The Vibe Coding Contest: Rust and Custom Agents
The broadcast shifts straight into a live "vibe coding" challenge where a participant outlines their rapid-prototyping setup using Rust. Starting from an image of the contest instructions snapped on a phone and AirDropped directly onto the dev machine, the participant used an agent to assess their environment and build out the workspace:
```mermaid
flowchart LR
accTitle: Vibe Coding Setup Flow
accDescr: Image prompt is expanded into an execution plan and handed to a custom agent that runs sub-agents.
A["Phone Photo of Challenge\n(AirDropped to Mac)"] --> B["Prompt Boosting &\nLocal Tool Verification"]
B --> C["Plan Mode\n(P0: Local Execution)"]
C --> D["Custom Beast Mode Agent\n(Sub-Agents & Rules)"]
```
The participant prompted the agent to “boost the prompt”—enriching the initial raw instructions into a high-fidelity specification—and draft a step-by-step execution plan prioritizing local execution as P0 over cloud deployment. The model immediately pushed back on the tech stack, asking: “Hey, you don't need Rust to do this you just need HTML and JavaScript. Do you still want to use Rust?” Determined to use Rust anyway, the developer overrides the suggestion and triggers a custom configuration dubbed "beast mode rebooted." This custom agent configuration enforces context-window limits, executes tasks across isolated sub-agents, and enforces non-negotiable coding rules.
[01:34:09](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5649s) Autopilot Mode, Skills, and Closing the Loop
The demonstration continues as the presenter switches into Copilot CLI's experimental autopilot mode, describing it as the command-line equivalent of a continuous execution loop. The agent works autonomously: testing the application, launching `localhost`, and capturing successive screenshots to validate the interface. Reaching the end of the run, the tool signals completion, leaving the presenter to quickly figure out how to launch and view the generated app.
```mermaid
flowchart TD
accTitle: Agent Testing and Loop Closure
accDescr: Diagram showing how agent skills interact to generate frontend code and verify it in the browser.
subgraph Design ["Code Generation"]
A["Autopilot Mode / Ralph Loop"] --> B["Front-End Design Skill (skills.sh)"]
B --> C["Opus Model"]
end
subgraph Verification ["Browser Validation"]
C --> D["Agent Browser Skill"]
D --> E["Open localhost & Capture Screenshots"]
E --> F["Verify Application & Close the Loop"]
end
```
The presenter explains the specific skill composition powering the demo. Following advice from Theo, they combined an Opus model with a front-end design skill sourced from `skills.sh`, calling it the most effective stack for interface work. To prevent the classic agent pitfall—declaring a job finished when the code does not actually execute—they integrated an agent browser skill. As the presenter emphasizes, agents often fail to self-verify:
> “Because one of the problems with with agents is that they can't close the loop.”
Using the browser skill allows the agent to visually inspect and confirm runtime behavior. The presenter tests the generated web app live by clicking a button to send kudos to a colleague named John, admiring the animated reaction and confirming that the data successfully persists across a hard browser reload.
- The experimental autopilot mode in Copilot CLI acts as a continuous autonomous loop similar to a Ralph loop.
- The workflow utilizes the front-end design skill from `skills.sh` alongside an Opus model.
- Browser automation tools take screenshots of `localhost` so the agent can inspect its own output rather than assuming the code runs.
---
[01:35:41](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5741s) Introducing Julia and the Reality of Model Integration in VS Code
The broadcast shifts back to the Microsoft Build stage, where the host welcomes Julia, a product manager on the Visual Studio Code team who joined in August. The host frames the discussion around the modern agentic coding era, noting that while end users casually swap between models in the UI and assume they all function identically, the engineering required behind the scenes to support each model is immense.
Julia validates this perspective, recalling that before joining the team she also assumed adding an LLM was as trivial as hooking up a new API endpoint. She quickly learned that making models behave reliably in the VS Code chat extension requires deep, ongoing collaboration with model providers:
> “It's not as easy as, oh, we get a new model endpoint and boom, here it is.”
Rather than treating models as interchangeable drop-in replacements, the team treats each one as an individual entity with unique traits. Even consecutive releases within the same vendor family—such as moving from GPT-5.4 to GPT-5.5—introduce distinct behavioral shifts that require custom tuning. The differences widen further when comparing models from competing providers like Anthropic and OpenAI. Because models function as black boxes to client-side developers, the VS Code team relies on direct research insights from the model creators to adapt their implementation.
---
[01:38:20](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=5900s) Anatomy of the VS Code AI Coding Harness
Julia pulls up a flowchart on screen to walk through the lifecycle of integrating a new model. The process begins informally when provider research teams notify the Copilot API and client teams about an upcoming checkpoint that shows promise. The team then focuses on tailoring what they call the "harness"—the runtime environment that wraps around the model. While an LLM inherently only consumes and emits tokens, the harness is the machinery that actually inspects workspaces, generates file edits, and executes terminal commands.
```mermaid
flowchart LR
accTitle: Four Pillars of the VS Code AI Harness
accDescr: The four core components configured for every AI model integrated into VS Code.
subgraph Harness ["VS Code AI Harness"]
direction TB
P["System Prompt<br/>Customized per model family"]
T["Built-in Tools<br/>e.g., apply_patch vs. insert"]
C["Context Management<br/>Tailored token window strategies"]
L["Agent Loop<br/>Execution & tool-invocation cycle"]
end
```
Whenever a new model is introduced, the team configures four core pillars within the harness:
1. **System Prompt:** Every individual model, even within the same provider family, receives a bespoke system prompt tuned to its specific instructions and quirks.
2. **Built-in Tools:** Different models have natural affinities for different editing mechanisms; the harness must expose the specific toolset each model is trained to call.
3. **Context Management:** Token window limits, context compaction, and relevance filtering are adjusted per model.
4. **Agent Loop:** The orchestration mechanism for how tool calls are dispatched, executed, and fed back into subsequent completion rounds.
Because of this tailored architecture, end users experience smooth, consistent functionality across models without realizing that each model runs within a customized operational envelope.
---
[01:40:40](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=6040s) Inspecting Chat Debug Logs and Nondeterministic Model Differences
To demonstrate how the harness works under the hood, Julia opens a stable build of VS Code and enlarges the interface to display a project called "DJ Julia." She submits a prompt asking Copilot to generate a new music pattern for her DJ app. She directs attention to the internal Chat Debug Logs, which expose the raw machinery of the harness: the precise system messages applied, search queries executed, styling commands dispatched, and step-by-step tool invocations.
In the debug view, Julia highlights how the system prompt is strictly separated from the user request message. Running the prompt against GPT-5.5 injects one customized system prompt, whereas running the identical prompt against an Anthropic model such as Opus 4.6 or 4.7 applies an entirely different system prompt. Tooling preferences also diverge sharply between architectures:
> “And the GPD model family always uses apply patch versus the anthropic models that you'd insert and insert files.”
Julia switches the active model from GPT-5.5 to Opus to run the exact same prompt live. She reminds the audience of the inherent nondeterminism in AI development: even when sending the same prompt to the same model family, sampling variance ensures the output will never be completely identical. The demo underscores how different system prompts and tool-calling behaviors shape the final code produced by each engine inside the editor.
[01:43:32](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=6212s) Tuning System Prompts Across Model Generations
The conversation opens with some humor around the nature of LLMs, with the host joking about dropping the phrase “stochastic process” because “it's like a $5 word, makes you sound really smart.” Even when testing the exact same model, the output remains inherently probabilistic and fluctuates across runs. Moving into the editor, the presenter demonstrates how system prompts must evolve between model versions. Comparing the earlier prompt against the newer one used with GPT-5.5 reveals concrete differences in instruction framing. With GPT-5.5, the engineering team noticed that the model was overly eager to immediately jump into executing terminal commands. To counter this, they crafted instructions explicitly requiring the agent to pause, inspect, and explore the workspace before attempting to run code in the terminal.
The host shares his own frustration from building an AI harness in Rust, remarking that simply switching models broke his tool because each model handles tool calls and instructions in completely different ways. The presenter notes that their team collaborates directly with OpenAI to tune system prompts for every new checkpoint:
> “And we are truly, truly optimizing for all of the new checkpoints. And we are optimizing in a way that it will hopefully work for most folks out-of-the-box.”
When asked whether the prompt harness is specialized for programming or generic text, the presenter confirms it is strictly tailored for code: the instruction file begins by grounding the agent with “you're an expert AI programming assistant,” steering it away from treating the session as a generic conversational interface and directing it toward tool usage such as running terminal commands inside VS Code.
---
[01:47:05](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=6425s) Offline Evals, SWE-bench, and VSCbench
When asked whether prompt changes are just improvised trial-and-error, the presenter explains that model providers usually advise testing existing prompts first to establish an unoptimized baseline. From there, the team runs two parallel testing tracks: extensive internal dogfooding and comprehensive offline evaluation suites. If performance drops or anomalies appear, prompts are occasionally rewritten entirely from scratch ahead of launch.
Addressing developer evaluation strategies, the presenter contrasts public benchmarks with proprietary internal suites. While public benchmarks like SWE-bench remain standard industry reference points, model providers themselves suggest viewing them primarily as safety checks:
> “The models are so smart now that you're always kind of going to get this. So use this as a regression test rather than really evaluating the quality of it.”
To evaluate realistic quality beyond SWE-bench, the team relies on **VSCbench**, an internal benchmark consisting of over 100 dedicated tasks and instances. They execute these benchmarks across varying reasoning effort levels to track how prompt adjustments impact real-world developer workflows.
```mermaid
flowchart TD
accTitle: Model Evaluation Pipeline
accDescr: Flow diagram showing how models are tested from baseline prompts through offline evals and benchmarks to the final model report card.
subgraph S1["1. Initialization & Baselines"]
A["New Model Checkpoint"] --> B["Baseline Prompt Test<br/>(Prior Generation Prompt)"]
end
subgraph S2["2. Dual-Track Testing"]
B --> C["Public SWE-bench<br/>(Regression Gate)"]
B --> D["Internal VSCbench<br/>(100+ Real Tasks)"]
B --> E["Internal Dogfooding"]
end
subgraph S3["3. Measurement & Scoring"]
C --> F["Azure Container Apps<br/>(VS Code Insiders)"]
D --> F
F --> G["Assertion Checklists<br/>(Run 5x for Non-Determinism)"]
G --> H["Model Report Card<br/>(Resolution Rate)"]
end
```
---
[01:49:46](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=6586s) Model Report Cards in Azure Container Apps
The presenter walks through a live offline eval run executed that morning. The test environment automatically spins up an Azure Container Apps (ACA) instance running the latest VS Code Insiders build to ensure tests run against current bits. Because LLM outputs are non-deterministic, test suites are ideally run five times per model to calculate an accurate distribution; this morning's run compared GPT-5.4 (run twice) against GPT-5.5.
Performance is summarized in an internal “model report card,” where the primary metric is the **resolution rate**. The test suite executes automated assertions against each scenario—treating the verification like a strict checklist—to verify whether the model met every requirement. In the demonstrated run, the model successfully resolved 153 out of 248 test cases.
Wrapping up the segment, the presenter offers guidance for engineers building AI coding systems:
> “And also then evals, offline evals are a huge topic and anybody building their own coding harness, it has been one of the tools that we as Microsoft have been super successful and the model providers are loving.”
Developers are encouraged not to judge a model on a single failed attempt, but to revisit checkpoints periodically as harnesses and system prompts are continually iterated and refined.
---
[01:52:12](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=6732s) Automatic Model Selection in Copilot
The broadcast transitions to a new segment focused on model fragmentation and selector fatigue. While users enjoy having access to a variety of models in GitHub Copilot, manually figuring out which model fits a specific task—and keeping up as new versions arrive overnight—adds unwanted cognitive overhead. To address this, the team introduced **Copilot Auto model selection** (“Auto” for short).
Auto eliminates manual toggling and policy settings by dynamically analyzing the current task in real time and routing it to the most capable model available:
> “We wanted to be able to abstract that complexity away from you and actually automatically give you the best available model for your task in real time so that developers can focus on what they want to do.”
- **Zero configuration:** Auto requires no custom setup, configuration files, or policy toggles.
- **Universal availability:** Users simply choose “Auto” directly from the model picker within their IDE, the GitHub Copilot CLI, or the Copilot Cloud Agent.
- **Ecosystem updates:** The host returns to discuss the rapid expansion of AI models showcased at Microsoft Build, noting that seven new MAI models were released during the event.
[01:53:45](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=6825s) Purpose-Built for GitHub Copilot: Introducing MAI-Code-Flash
The host welcomes researchers behind MAI-Code-Flash, eager to dig into technical details with the people who built it. Peng, the pre-training research lead, and Hunter, who leads post-training and reinforcement learning, explain that MAI-Code-Flash was not designed as an off-the-shelf LLM with post-hoc fine-tuning slapped on top. Instead, the model was engineered from its inception around the GitHub Copilot user journey. As Peng emphasizes:
> “Yeah, to me the purpose building model means that we didn't take a general purpose model and adapt it for code at the end.”
He adds that developing specifically for Copilot means the model does not merely spit out isolated code snippets. It actively understands workspace context, invokes native developer tools, tracks user intent, makes pinpoint edits, and streams back responses with minimal latency. Rather than gaming public synthetic benchmarks, the model is tuned specifically for real-world developer workflows inside VS Code. It knows when to run a unit test at the proper moment, when to avoid lengthy chain-of-thought delays on straightforward questions, and how to interact smoothly within the daily engineering loop.
```mermaid
flowchart LR
accTitle: MAI-Code-Flash Design Philosophy
accDescr: Diagram contrasting general purpose models adapted for code with MAI-Code-Flash purpose-built for VS Code and Copilot.
subgraph Traditional["Traditional Approach"]
direction TB
G["General LLM"] -->|"Late Adaptation"| C["Generic Code Snippets"]
end
subgraph PurposeBuilt["MAI-Code-Flash Approach"]
direction TB
D["Copilot Native Harness"] -->|"End-to-End RL & Tools"| V["Targeted Edits & Tests in VS Code"]
end
```
---
[01:56:09](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=6969s) The Multi-Stage Training Pipeline: From Cold Start to Reinforcement Learning
When the host asks how the team pushes data through the model to achieve this tailored developer behavior, Hunter explains that reinforcement learning (RL) serves as the crucial final stage. To ensure the model feels native to VS Code, the training harness integrates the exact toolsets, prompt templates, and software engineering environments that engineers encounter daily. A custom reward model evaluates task completion across realistic daily engineering scenarios so the model internalizes the right interactive behavior.
Peng clarifies that the full pipeline spans multiple progressive stages:
1. **Cold Start & Supervised Fine-Tuning (SFT):** Beginning from raw next-token prediction, the model undergoes a "cold start" to learn basic instruction following, output formatting, and simple reasoning tasks.
2. **Curriculum Data Scaling:** The model progresses stage-by-stage through increasingly diverse and complex coding tasks, akin to educating a child from kindergarten through postgraduate work.
3. **Product-Aligned Reinforcement Learning:** The final training stage places the model directly in the product environment alongside real product partners. By evaluating the model on the identical harness and evaluation pipeline used in production, the model is fully aligned to GitHub Copilot's practical needs.
> “The universe learning is the last stage to make the model perfectly aligned with user experience in GitHub compiler.”
---
[01:58:39](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7119s) Architecture and Efficiency: 137B MoE with 5B Active Parameters
Addressing why developer agents do not always require massive monolithic models, the researchers reveal the underlying architecture of MAI-Code 1 Flash. The model is structured as a Mixture of Experts (MoE) containing roughly 137 billion total parameters, while activating only 5 billion parameters per token inference.
```mermaid
flowchart TD
accTitle: MoE Token Routing Architecture
accDescr: Diagram illustrating token routing to sparse experts within the 137B parameter MoE structure.
T["Input Token"] --> R["Gating / Routing Layer"]
subgraph Experts["137B Total Parameter MoE Capacity"]
E1["Active Expert Cluster (5B Active Parameters)"]
E2["Inactive Expert Cluster"]
E3["Inactive Expert Cluster"]
end
R -->|"Select & Activate"| E1
E1 --> O["High-Speed, Low-Latency Stream"]
```
This sparse activation strikes an ideal balance across intelligence, inference speed, and token cost. By routing incoming tokens strictly to the relevant expert subnetworks, the engine achieves rapid time-to-first-token and low overall latency while preserving deep coding capability. Peng and Hunter highlight that combining fast model inference with token-efficient generation makes the model perform exceptionally well without burning unnecessary tokens on routine software engineering tasks.
- **Total Parameter Count:** 137 billion parameters across the entire MoE network.
- **Active Parameter Count:** 5 billion active parameters per token pass.
- **Optimization Focus:** Striking a sweet spot between raw coding intelligence, token efficiency, operational cost, and wall-clock latency.
---
[02:02:40](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7360s) Live Demonstration: Front-End App Generation in VS Code
Hunter and Peng transition to a live video demonstration showing MAI-Code-Flash running inside VS Code. In the demo, a developer issues a prompt to construct a front-end web application from scratch. The model immediately begins tool execution and code generation without lingering on prolonged reasoning loops.
The demonstration highlights how rapidly tokens stream on the fly as MAI-Code-Flash invokes VS Code's native tooling, creates the application structure, and assembles a polished, functional user interface in an exceptionally short timeframe. Hunter then prepares a second demo highlighting an AI agent workflow.
[02:03:30](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7410s) Fine-Tuning Frontier Models for Developer Workflows
The team explains that significant engineering effort went into improving the model's front-end capabilities alongside native training partners, resulting in highly efficient code generation for UI tasks like retro-style games. Rather than relying solely on standard industry benchmarks, the biggest takeaway from building this specialized frontier model was the necessity of grounding it in actual developer workflows. As one of the engineers points out, without using capacity grounding tasks directly tied to real compiler and development environments, the model could never have achieved its current level of execution. The model is being rolled out gradually into Visual Studio Code, where eligible users will find it populated in their model picker.
When discussing the testing and validation pipeline, the team emphasizes that high benchmark scores across broad suites are only the starting point. To guarantee that the model genuinely satisfies user needs in real-time coding sessions, extensive online A/B testing is conducted to monitor direct user engagement and track metrics such as prompt cancellation rates. In day-to-day use, the smaller, specialized model feels snappier and faster, holding its own against massive general-purpose frontier models for routine software engineering tasks across both front-end and back-end stacks in diverse programming languages. However, the engineers acknowledge the inherent trade-offs of compact architectures, noting that complex, high-difficulty edge cases still reveal scaling limits, and users looking for creative writing or poetry should choose broader models designed for those tasks. Reflecting on the real-world performance of the model, one engineer notes: “To me, I think for most of the task, it's comparable to today's frontier big models.”
- **Availability:** Rolling out progressively in Visual Studio Code via the integrated model picker.
- **Optimization focus:** Realistic developer workflows and grounding tasks rather than synthetic benchmarks alone.
- **Validation strategy:** Continuous online A/B testing measuring completion rates, interaction latency, and prompt cancellations.
- **Best-fit use cases:** Daily full-stack software development across multiple programming languages, with less emphasis on open-ended creative prose.
---
[02:06:26](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7586s) Multi-Agent Orchestration Demo with MAI Models
The demonstration shifts to a weekend project showcasing voice-driven, autonomous multi-agent software engineering. In the live audio demo, a user issues spoken commands to an orchestration layer that coordinates specialized coding agents. When instructed to run a web application locally, the assistant immediately prompts the user for explicit approval before spinning up the development server on localhost. Once approved, the user asks the agents to adjust the user interface styling by applying a blue and green gradient background, which the agent executes autonomously. The user then directs the agent to inspect the project's test suite, identify gaps in coverage, and execute the missing unit tests. After another explicit security confirmation to run the NPM script, the agent successfully writes and executes new unit tests for the profile header component.
```mermaid
flowchart TD
accTitle: Multi-Agent Orchestration Architecture
accDescr: Flow showing speech input routed through transcription, OS orchestration, and code generation models to execute local tasks with user approval gates.
subgraph Orchestration ["Agent Coordination"]
B --> C["MAI OS Orchestrator"]
C --> D{"Approval Gate"}
D -->|"Approved"| E["MAI Code Flash Model"]
end
subgraph Execution ["Local Workspace Actions"]
E --> F["Start Dev Server (localhost)"]
E --> G["Update UI (CSS Gradient)"]
E --> H["Generate & Run Unit Tests"]
end
```
When asked for a status update on what the agents are doing behind the scenes, the system reports that three autonomous agents are simultaneously analyzing the application architecture, with one actively formulating a plan to refactor components. The engineers reveal that this workflow was built by harmonizing three distinct specialized models: the state-of-the-art MAI Transcribe speech model, the MAI OS orchestration model, and the lightweight MAI Code Flash coding model. Combining purpose-built models allows developers to envision a future where multi-agent fleets collaborate seamlessly on software engineering tasks under natural human supervision. Delivering their final advice for developers, one of the creators states: “My take away is we are seriously treating the best model for our product, the best code and other product and this is just the beginning of our journey.”
- **Component models:** MAI Transcribe (audio/speech processing), MAI OS (agent reasoning and system-level orchestration), and MAI Code Flash (fast code synthesis and test generation).
- **Human-in-the-loop controls:** Deterministic approval checkpoints required before starting local processes or executing terminal commands like NPM test runners.
- **Autonomous parallel activity:** Simultaneous structural repository analysis, UI updates, test synthesis, and planned component refactoring.
---
[02:09:40](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7780s) Anders Hejlsberg on Porting the TypeScript Compiler to Go
In a recorded segment, Anders Hejlsberg details the profound technical motivations and architectural trade-offs behind the monumental project to port the TypeScript compiler from TypeScript/JavaScript to Go. Since its inception, TypeScript was proudly self-hosted—written entirely in TypeScript and running as a standard JavaScript application. This design delivered immense early dividends, enabling developers to run the complete compiler, language services, and IDE tooling natively inside web browsers. However, as TypeScript achieved widespread global adoption and enterprise codebases exploded in scale, compiler performance and scalability became the team's overriding bottleneck.
JavaScript’s execution model presents hard architectural ceilings for high-throughput compilation. It is fundamentally single-threaded by design, preventing the engine from leveraging multi-core processors through shared-memory concurrency and effectively leaving more than 90 percent of modern computing power untouched. Furthermore, JavaScript's permissive object model—where arbitrary properties can be attached dynamically to objects at runtime—necessitates complex shape transitions, hidden classes, and hash-table lookups rather than direct memory offsets in native machine code. Anders underscores that despite the team's deep, world-class mastery of V8 engine internals and micro-optimizations, JavaScript had simply reached its absolute limits. Capturing the frustration of exhausting language-level tweaks, Anders Hejlsberg explains: “And I mean, we had all of the tricks we had found out and we had applied and whatever and it was still just, we're not, we're never going to get any faster.”
---
[02:11:44](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7904s) Daniel Rosenwasser on the Evolution of TypeScript
Returning to the live stage, the host welcomes Daniel Rosenwasser, Product Manager for the TypeScript team. Daniel shares his personal trajectory at Microsoft, having started out as an engineer before transitioning into the product management role because, as he jokes, he simply talked too much and the team decided that was where he belonged. He outlines the expansive scope of the team's responsibilities: maintaining the TypeScript language syntax, evolving the core compiler, and engineering the foundational editor tooling and language server experiences that power both Visual Studio and Visual Studio Code. Importantly, this same infrastructure underpins modern JavaScript editing across those environments, meaning every JavaScript developer typing in VS Code relies on the exact same underlying machinery.
The conversation turns toward the historical roots of the project and the fundamental problem space TypeScript was invented to address. Long before TypeScript existed, software engineering teams worldwide struggled with the fragility of maintaining massive, business-critical applications in vanilla JavaScript. Because web browsers and web platforms universally mandate client-side code, building expansive websites required teams to scale up dynamic JavaScript codebases without type safety, automated refactoring, or compile-time verification—creating the exact friction that TypeScript set out to solve over a decade ago.
[02:12:52](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=7972s) The Origins of TypeScript and Growing into the Standard for JavaScript
The speakers reflect on the fundamental motivations behind creating TypeScript. As developers began building complex, cross-platform applications targeting the browser, they realized the browser ecosystem lacked the safeguards available in statically typed environments like C#. In C#, errors are flagged immediately with red squiggly underlines right as code is typed, functioning like spell check in Microsoft Word. TypeScript was conceived to fill this gap: providing safety, modern developer tooling, and modern language features that were not yet universally supported across web browsers and runtimes.
```mermaid
flowchart TD
accTitle: TypeScript Architecture and Porting Pipeline
accDescr: Pipeline showing TypeScript components processing code and the porting effort from JavaScript runtime to Go.
The speaker recalls joining Microsoft in the early days of TypeScript when attempting an open-source project in the JavaScript space seemed wildly ambitious and met with skepticism from the community and internally. Over the years, that perception changed dramatically. The team fostered an open, community-driven culture focused on developer tooling, culminating in TypeScript becoming the number one language on GitHub according to the GitHub Octoverse report, surpassing JavaScript itself. Today, writing TypeScript is simply the default standard for building JavaScript applications rather than an opposing alternative.
- TypeScript initially brought modern syntax and type safety to the web before runtimes supported upcoming ECMAScript standards.
- In GitHub Octoverse reports, TypeScript surpassed JavaScript as the top language on GitHub.
- “It's not like TypeScript versus JavaScript. It's I'm doing JavaScript development and I'm using TypeScript and that's just kind of the core experience.”
---
[02:15:23](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=8123s) Bottlenecks of Self-Hosting and Porting the Compiler to Go
For over a decade, TypeScript was strictly self-hosting—written entirely in TypeScript and compiling down to JavaScript. While this "dogfooding" provided firsthand insight into developer pain points, it introduced severe performance bottlenecks as enterprise codebases expanded. Despite monumental optimizations in JavaScript engines like V8 and SpiderMonkey, running on top of a dynamic runtime imposes inherent overhead. More critically, JavaScript is fundamentally single-threaded; while worker threads exist, they cannot perform efficient shared-memory concurrency without costly serialization or transfer overhead.
To break through this wall, the TypeScript team spent roughly a year and a half executing a complete port of the TypeScript compiler codebase to Go. Rewriting the core compiler components—including the scanner, parser, checker, and emitter—into Go unlocked native CPU execution and multi-core, shared-memory parallelism. The rewrite delivered massive performance gains across large projects while preserving exact functional parity with the original TypeScript compiler, maintaining identical diagnostics and error messages.
- TypeScript had been self-hosted and running on JavaScript engines for over 10 years.
- The single-threaded memory model of JavaScript prevented efficient parallel type checking across multi-core CPUs.
- “And in that port, what we found is a typical sort of speed up that you'll find on many code bases around 10X, right, which is pretty mind blowing.”
- The rewrite changes the compiler implementation language to Go, but the compiler continues to ingest TypeScript/JavaScript and output standard JavaScript code.
---
[02:18:27](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=8307s) The Methodical Porting Strategy: ts-morph and Deterministic Automation
The porting effort was tackled systematically rather than through an unguided, top-to-bottom rewrite or naive reliance on early AI agents. The team first hand-crafted (“artisanally”) the scanner and parser to establish core idiomatic Go data structures and patterns. Once the baseline architecture was in place, the team leveraged AI agents to write automation scripts, using tools like `ts-morph` to parse TypeScript abstract syntax trees (ASTs) and mechanically generate idiomatic Go code.
This hybrid approach combined deterministic script transformation with manual verification. Functions were mapped piece by piece, allowing engineers to review output, resolve intentional compilation errors, and tune the script generators whenever edge cases arose. By porting their existing test suite of unit and input-output integration tests, the team eliminated regressions and ported over 95% of existing tests, excluding only obsolete legacy features. Feedback from internal Microsoft teams and external community users over preceding months ensured that the Go-based compiler achieved near-complete behavioral parity.
- Initial parts of the codebase, like the scanner, were hand-ported to establish idioms and memory layouts.
- Automation scripts utilized `ts-morph` to inspect TypeScript ASTs and deterministically output equivalent Go code.
- Over 95% of original unit and integration tests were ported directly to validate behavioral compatibility between the Go implementation and the JavaScript-based predecessor.
- “And we were able to reuse all of our unit tests and our input output tests like our integration tests. And so over the last year and a half, we've been basically minimizing the differences in all the tests, right?”
[02:21:50](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=8510s) The Go Underlayer and Seamless Developer Transition
The speaker reflects on the stability of porting the TypeScript compiler to Go over the preceding months. For typical end users, replacing the JavaScript-based compiler engine with the Go underlayer will feel almost completely invisible—apart from a dramatic boost in responsiveness. The speaker notes that pitching compiler performance can be a subtle affair: “So that’s the funny thing. It’s very interesting to sort of demo these sorts of things, like, hey, I have a compiler, I have a faster compiler. Isn't that cool?”
While API integrations—such as specialized tooling for Vue.js or Angular—still have a few minor parity gaps, the team is collaborating closely with dev tool authors to close them shortly. Even with those ongoing efforts, the compiler is already mature enough for general adoption across the majority of projects.
- Most standard TypeScript projects can adopt the new Go underlayer immediately without disruption.
- Gaps in framework-specific tooling APIs (like Vue and Angular editing environments) are being closed in cooperation with their respective authors.
---
[02:22:56](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=8576s) Benchmarking the Existing JavaScript TypeScript Compiler on VS Code
To demonstrate real-world impact, the presenter opens the official Visual Studio Code repository—a massive code base spanning roughly 1.5 million lines of code. Compiling VS Code using the traditional Node.js/JavaScript TypeScript compiler (`tsc`) requires appending verbose memory flags to Node, because without explicit instructions to grant Node more RAM, the build crashes with out-of-memory errors.
```mermaid
flowchart TD
accTitle: Legacy TypeScript Compilation Bottleneck
accDescr: Visual diagram showing the single-threaded parsing bottleneck in the legacy Node.js-based TypeScript compiler.
subgraph SingleThread["Single-Threaded Execution (Node.js)"]
P1["Parse File 1"] -->|"Wait on Import"| P2["Parse File 2"]
P2 -->|"Wait on Import"| P3["Parse File 3"]
P3 --> Check["Type Checking (Single Core)"]
end
SingleThread -.->|"Saturates only one CPU core"| C15
```
Running `htop` illustrates the underlying bottleneck: Core 15 stays entirely pegged while the rest of the multi-core CPU remains idle. The presenter jokes that it resembles “the meme of a bunch of people like construction site and everyone’s just like watching one guy dig a hole and they’re like... looks good to me.” Because Node executes on a single event loop thread without native parallelism across CPU cores, file parsing must occur strictly one file at a time, blocking whenever an import dependency must be parsed before moving forward. Under this legacy setup, compiling VS Code took 97 seconds (1 minute and 38 seconds of wall-clock real time).
- Compiling VS Code with standard `tsc` requires custom Node memory ceiling flags to avoid out-of-memory crashes.
- Legacy compilation is single-threaded during checking and parsing, pinning only a single core (e.g., Core 15, then handed off to Core 28).
- Full clean non-incremental build of VS Code using legacy `tsc`: 97 seconds.
---
[02:24:59](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=8699s) Introducing `tsgo` and Parallel Multi-Core Speedups
The presenter installs the preview package globally via `npm install -g @typescript/native-preview`, which provides the standalone `tsgo` binary. This preview binary will eventually become the official TypeScript 7.0 release. Running the identical VS Code build using `tsgo` with extended diagnostics demonstrates the difference on `htop`.
Immediately upon execution, all CPU cores become saturated as the Go implementation parses the entire 1.5-million-line code base concurrently. During the type-checking phase, the load drops down to four cores because type checking cannot be solved with pure, unconstrained parallelism. Once type checking completes and code emission begins, work bursts across every available core again. The non-incremental full build completes in roughly 9 to 10 seconds, cutting the total execution time from over a minute and a half down by nearly 90%.
```mermaid
flowchart TD
accTitle: TypeScript Go Architecture Phases
accDescr: Execution pipeline of the TypeScript Go compiler showing multi-core saturation across parse, check, and emit stages.
subgraph Phase1["1. Parse Stage"]
ParseAll["Saturate All Cores (Parallel Parsing)"]
end
subgraph Phase3["3. Emit Stage"]
EmitAll["Saturate All Cores (Full Parallel Code Generation)"]
end
Phase1 --> Phase2
Phase2 --> Phase3
```
- Target release: The Go rewrite will ship as TypeScript 7.0.
- Command-line tool: `tsgo` (via `@typescript/native-preview`).
- VS Code code base parsing: 1.5 million lines executed in parallel across all available system threads.
- Performance: Dropped non-incremental compilation from 97 seconds down to 9–10 seconds.
---
[02:26:40](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=8800s) Production Adoption, CI Savings, and Editor Language Server
The native build is not limited to CLI invocations; the team has released the **TypeScript Native Preview** extension on the Visual Studio Marketplace. Major organizations are already running this native compiler architecture in production. The presenter lists adopters including Visual Studio Code itself, Sentry, Slack, Notion, and Figma.
Beyond desktop responsiveness, the pipeline optimizations yield massive infrastructure savings. The speaker shares that “We worked with one team at Microsoft that told us that we were saving them 400 CI hours a month.” These performance gains are equally critical for modern developer workflows where autonomous coding agents repeatedly invoke type checkers to validate generated code.
To close out the session, the presenter contrasts the editor experiences:
- **Legacy TypeScript Language Server**: When opening large TypeScript files, the editor displays an “initializing tsconfig” spinner while it parses the project sequentially file by file.
- **Native Preview Language Server**: Boots instantly and allows repeated language server restarts without blocking developer interaction. Despite a momentary glitch during the live demo, the host and presenter celebrate the performance leap and urge the community to try the preview immediately.
## Notable quotes
> [39:52](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=2392s) And there are going to be times when we're vibe coding, there's going to be times when we're agentic engineering. And the more diligence you have to apply to the process, the closer you are getting to agentic engineering. — Addy Osmani
> [01:15:22](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=4522s) I don't think that you can build dev tools and be a manager in that space without writing a lot of code yourself and like living the life of your users. — Evan Boyle
> [55:08](https://www.youtube.com/watch?v=AtWmQTfHZ74&t=3308s) Honestly I feel like we need a better solution than a context window in general at some point in the future. This is like a really rudimentary primitive for interacting with LLMs.
## Mentioned
Microsoft Build, Visual Studio Code, VS Code Insiders, GitHub Copilot, GitHub Copilot CLI, GitHub Copilot SDK, GitHub Mobile, TypeScript, JavaScript, Go, Node.js, Rust, Electron, Anders Hejlsberg, Daniel Rosenwasser, Addy Osmani, Evan Boyle, Patrick Nicoletich, Burke Holland, Pierce, Julia, Peng, Hunter, Scott Hanselman, Kent C. Dodds, MAI-Code-1-Fast, MAI-Code-Flash, MAI Transcribe, MAI OS Orchestrator, GPT-5.5, Claude Opus, Claude Sonnet, Gemini 3.5 Flash, NVIDIA Nemotron, Model Context Protocol (MCP), Work IQ, Context7, Whim, VSCbench, SWE-bench, Streak, Playwright, Puppeteer, tsgo, @typescript/native-preview, Azure Container Apps
## Video description
Join us on stage for live demos, surprise guests, games, live coding, and more. The latest from GitHub, Copilot, VS Code, Visual Studio, and friends, straight from the teams building it.
𝗦𝗲𝘀𝘀𝗶𝗼𝗻 𝗜𝗻𝗳𝗼𝗿𝗺𝗮𝘁𝗶𝗼𝗻:
This is one of many sessions from the Microsoft Build 2026 event. View even more sessions on-demand and learn about Microsoft Build at https://build.microsoft.com
LIVE104 | English (US)
Broadcast Stage
#MSBuild
Chapters:
0:00 - Event announcements and livestream details
00:06:41 - Overview of GitHub Copilot SDK and agent integration
00:40:03 - Commentary on industry differences in agentic engineering best practices and quality focus
00:47:00 - Deep dive into AI code review security concerns and need for deterministic gates
00:57:00 - Introduction of next session featuring Copilot SDK, launch details, performance improvements, and live demo with agent-based workspace
01:12:38 - Discussion on multi-model verification and GPT 5.5 advantages
01:21:14 - Demonstration of managing multiple sessions and slash commands
01:29:12 - Explanation of voice mode and local models used for speech interaction
02:09:40 - Introduction of TypeScript compiler porting from JavaScript to Go for performance
Presenters at Microsoft Build argued that scaling AI-assisted development requires moving beyond unstructured 'vibe coding' toward disciplined software engineering practices, deterministic verification, and deep systems optimization.
Disciplined Agentic Engineering and Deterministic Verification (15:10)
Engineering practices are shifting from informal vibe coding toward structured agentic engineering, particularly in legacy enterprise codebases. Presenter Addy Osmani demonstrated structuring agent skills across the traditional Software Development Life Cycle—Define, Plan, Build, Verify, Review, and Ship. Rather than giving models unconstrained authority, disciplined workflows limit the blast radius by restricting the specific files an agent can touch, requiring test-driven development (TDD) verification gates, and utilizing cost-effective tiered model pipelines, such as handling architectural planning in lighter models like Gemini Flash before dispatching code generation to frontier models.
While LLM-driven reviews reliably flag general code issues, presenters warned against relying entirely on non-deterministic model outputs for security, credentials, or production readiness. Highlighting the danger of leaked API keys and rate-limit exploits, Osmani argued that development teams must establish deterministic quality gates. Code generated by agents must pass automated headless browser checks and rigorous automated testing checklists before shipping to production.
Copilot SDK 1.0 and Unified Developer Tooling (57:52)
GitHub announced the General Availability of the Copilot SDK 1.0, extracting the unified agent loop originally built into the GitHub Copilot CLI into a stable runtime for third-party production integration. The SDK had been hardened through internal enterprise deployments across LinkedIn, Xbox, GitHub, and Office Excel. Supported across Node.js, TypeScript, Python, Go, and .NET, the SDK enables programmable control over multi-turn interactions, custom agent personas, and ephemeral cloud sandboxes that persist tasks independent of a local machine's battery or memory.
The GitHub team reported that relying on internal agentic tools dramatically accelerated team velocity, reaching a pace where ten engineers shipped roughly 2,000 pull requests in a single month. CLI capabilities have expanded to include consensus-driven multi-model reviews—evaluating changes simultaneously across models like Claude 3.7 Sonnet, GPT-5.5, and Gemini—as well as on-device voice workflows powered by local 700 MB models such as NVIDIA Nemotron to eliminate cloud audio latency.
Model Tuning, Harness Evals, and Dynamic Routing (95:41)
Visual Studio Code product manager Julia noted that integrating frontier models into editor harnesses requires deep behavioral tuning rather than simply swapping API endpoints. Different models exhibit stark differences in tool invocation and behavior; for example, the GPT family relies on apply patch routines while Anthropic models insert whole files. Furthermore, newer checkpoints require custom system prompting, such as explicitly instructing GPT-5.5 to explore the workspace before prematurely executing terminal commands.
To ensure reliability, the VS Code team runs automated offline evaluations inside Azure Container Apps to benchmark models against real engineering scenarios, tracking resolution rates across multiple runs to account for stochastic sampling variance. To alleviate model fragmentation for developers, GitHub introduced Copilot Auto model selection, which analyzes task context in real time to route requests to optimal models without manual configuration, alongside newly trained domain models like MAI-Code-Flash that align to development environments via product-aligned reinforcement learning.
Porting the TypeScript Compiler to Go (129:40)
Anders Hejlsberg detailed a multi-year effort porting the core TypeScript compiler from its self-hosted JavaScript/TypeScript implementation to Go. While self-hosting originally enabled TypeScript to run everywhere, JavaScript's single-threaded execution model and permissive dynamic memory structures created hard ceilings for large enterprise codebases, leaving modern multi-core processor capacity underutilized during compilation.
The rewrite into Go unlocks native CPU execution and multi-core shared-memory parallelism across the scanner, parser, type checker, and emitter while maintaining strict diagnostic parity with the JavaScript codebase. In demonstrations on Visual Studio Code's 1.5-million-line repository, the native engine avoided Node.js memory exhaustion and distributed type checking across all available cores. Released via the TypeScript Native Preview extension, early production adopters include Figma, Slack, Notion, Sentry, and internal Microsoft teams, with one team reporting savings of 400 CI hours per month.
As software workflows adopt autonomous agent loops and native multi-threaded compilers, development speed increases dramatically; however, teams remain challenged to bridge remaining framework API gaps and ensure non-deterministic model outputs do not bypass deterministic security and quality gates.
Source quotes
[39:52] Addy Osmani: And there are going to be times when we're vibe coding, there's going to be times when we're agentic engineering. And the more diligence you have to apply to the process, the closer you are getting to agentic engineering.
[75:22] Evan Boyle: I don't think that you can build dev tools and be a manager in that space without writing a lot of code yourself and like living the life of your users.
[55:08] Honestly I feel like we need a better solution than a context window in general at some point in the future. This is like a really rudimentary primitive for interacting with LLMs.
Original transcript
[0:30] Welcome to day 2 of Microsoft build.
[0:33] How is everyone?
[0:35] Let's get the energy up.
[0:37] That was pretty good for first thing in the morning
[0:39] on day two.
[0:40] I'm not going to do the thing where I'm like,
[0:42] I'll do better anyway.
[0:43] I'm Burke.
[0:44] This is my colleague Pierce.
[0:46] We were just talking about like, what is your title?
[0:49] What do you do here?
[0:50] What is it that you say that you do here?
[0:53] So I work on the VS Code team, but I
[0:55] also work on all things GitHub copilot, GitHub copilot apps,
[0:58] CLI, etcetera etcetera.
[0:59] So things.
[1:00] Things and my name is Burke.
[1:02] I also work on Copilot, CLI, Visual Studio Code.
[1:06] I do my best to make some YouTube videos from
[1:09] time to time and otherwise just sort of hang out
[1:13] with peers.
[1:14] A couple of announces for you today.
[1:16] So this right here is streaming Windows stuff after 2:15.
[1:23] So if you want to hear Windows stuff, this is
[1:25] where you definitely want to be.
[1:27] Hi to everyone watching here at Builds.
[1:30] I guess that's y'all.
[1:32] Or if there's nobody.
[1:33] Yeah, it's just these people here.
[1:35] OK.
[1:35] And then I'll still have people watching online on our
[1:39] social handles, VS Code social handles on YouTube, on XI.
[1:42] Guess we're streaming on X as well.
[1:44] GitHub, YouTube, Microsoft Developer Terminal, live.
[1:48] The live coding from the Expo floor.
[1:49] Have you all seen the live coding booth?
[1:51] It's like a glass.
[1:52] It's like a fish tank with people inside.
[1:55] Can anyone go in there or do you have to
[1:57] be?
[1:58] Do you think you have to be sanctioned?
[1:59] Do you have?
[2:00] I'm going to be locked in there for two hours
[2:01] today.
[2:02] Are you really?
[2:02] Yeah.
[2:02] What are you going to build?
[2:03] No idea, huh?
[2:04] Interesting.
[2:05] We'll find out together.
[2:05] Anthony was in it last year.
[2:07] Didn't you build like a programming language or something?
[2:09] Yeah, Anthony built a programming language in 10 minutes because
[2:12] that's what you can do now.
[2:15] I'm sure it works too.
[2:16] Everybody's using it.
[2:17] Let's see we have from 10:15 to 11:00 VS Code
[2:22] teams AI adoption story.
[2:25] Let's break out two O 4, is that yours?
[2:28] That's mine, yeah.
[2:29] That's you and.
[2:30] Me and Josh from the VS Code team.
[2:31] So the basic concept is like, OK, you use AI,
[2:34] now what?
[2:35] There's a lot of things that break.
[2:37] I thought it was going to be about like our
[2:39] AI adoption story, like why we put AI in.
[2:41] No, no, no.
[2:42] It's like literally like our team uses AI now.
[2:44] OK, now what?
[2:46] Gotcha.
[2:47] Interesting.
[2:48] OK.
[2:48] And then we have the oh, then we have the
[2:50] Copilot game show that's going to be happening on the
[2:53] keynote stage.
[2:55] That's going to be a ton of fun.
[2:57] Kent Kent C Dodds is going to do.
[2:58] Do you have to say the C when you say
[3:00] Kent C Dodds?
[3:02] Where's Kent?
[3:02] Where's Kent?
[3:03] Yeah, anyway, he'll be there Co hosting.
[3:05] We're going to try to live.
[3:06] We're going to vibe code something in 45 minutes, see
[3:09] how far we can get.
[3:09] It's going to be a lot of fun.
[3:11] Join us for that and then we have monthly release
[3:14] live streams, y'all on the VS Code channel if you
[3:17] want to keep because look how many people have no
[3:20] ability to keep up with what's going on right now.
[3:23] Am I the only one?
[3:25] I work on the product and I don't know what's
[3:27] in it because they're shipping every 15 minutes.
[3:30] Those monthly release live streams are a really good way
[3:32] to keep up with everything that's going on.
[3:35] You still won't know, but at least you'll know more
[3:38] than you do now, right?
[3:39] It's not possible to keep up with everything.
[3:42] And then they want me to mention the blog.
[3:44] I'm not doing that.
[3:45] OK, opener, we just said all that.
[3:50] All right, Pierce, let's talk about the highlights from day
[3:53] one.
[3:53] Let's do it.
[3:54] So first, let's talk about the GitHub Copilot app.
[3:57] It's this new thing.
[3:59] It's an app.
[4:00] It's not an editor, it's not a CLI.
[4:02] It's.
[4:03] What is it exactly?
[4:05] It's an agent native desktop experience for building things.
[4:09] So like that is a very marketing answer, right?
[4:12] But I think we've all kind of used these session
[4:14] managers, right, where you, you know, you spin up some
[4:17] agents, you're running multiple things at once, they're isolated and
[4:20] get work trees.
[4:22] And so like the app can do that too.
[4:24] But really like when you think about your job, right,
[4:27] there's a lot of work that is around all that,
[4:29] right?
[4:29] Like deciding what issues to pick up, triaging your backlog.
[4:33] Once you actually do stuff with agents, OK, sure.
[4:36] You submit APR, you have to get it across the
[4:38] finish line.
[4:39] The app helps with all that stuff as well.
[4:41] So I've noticed this.
[4:43] I don't know, I'd just like to pull the audience,
[4:45] but how many of y'all are actually running multiple agents
[4:47] at one time?
[4:50] OK, so several folks, like more than two, more than
[4:55] three.
[4:57] Anybody out here running like 50 agents in your screen
[4:59] is just a bunch of terminal sessions or is that
[5:02] only on social media?
[5:03] Like no one is doing that.
[5:05] So I did have this question about like how many
[5:08] agents is feasible in your opinion inside of these apps
[5:12] to run?
[5:12] Like should we be running 50 agents at one time?
[5:15] What is expected of me now?
[5:17] It's what am I supposed to be able to do
[5:19] as a developer?
[5:20] I don't even know anymore.
[5:22] I I think there's a lot of performative productivity of
[5:24] like to your point, people running 50 things at once,
[5:27] but it's just like always like when I sit down
[5:29] and write my performance review, it's like, what impact did
[5:32] I deliver, right?
[5:33] And if I can get there by running a couple
[5:35] things at once, great.
[5:36] I personally don't think I can go beyond 3 or
[5:38] 4 because I really start like losing track of what's
[5:41] actually going on and I have to batch the task
[5:43] together.
[5:44] I can't like constantly just be jumping in and out
[5:45] of things all day.
[5:47] So for me, I think that the Max is 3
[5:48] or 4.
[5:49] But with the app, like even if you're only running
[5:51] one thing at a time, the interesting thing is like
[5:53] the code is intentionally backgrounded, right?
[5:55] We have an awesome editor VS Code.
[5:57] If you want to see the code, you go to
[5:58] that.
[5:59] But the whole point of the app is you can
[6:00] stay outcome focused.
[6:01] So there's like a browser, you can click one, click,
[6:03] run.
[6:04] And so your first like kind of outcome that you
[6:06] do after the agent is finished is you're actually checking
[6:09] to make sure did it do the thing right?
[6:11] And then secondarily, you know, how do I feel about
[6:13] this code?
[6:14] And so the app is more focused on that first
[6:15] problem in a second.
[6:17] Do you really ask yourself how you feel about the
[6:19] code or?
[6:20] Depends, you're supposed.
[6:21] To ask yourself that question, but then you're just like,
[6:24] it works.
[6:24] I'm just not going to look because here's what I've
[6:27] noticed y'all with these new tools, it's like out of
[6:29] sight, out of mind, you know what I mean?
[6:31] Like if you don't see the code, then you don't
[6:33] get nerd sniped into worrying about it and you just
[6:36] it just goes and you're like, does it work?
[6:39] I don't know.
[6:39] This has been my experience.
[6:40] Yep.
[6:41] So the other thing we had is we got a
[6:43] lot of stuff on here.
[6:44] The SDK, Yep.
[6:45] Which it's not obvious to me why I need a
[6:48] copilot SDK.
[6:49] What is?
[6:50] What is that for exactly?
[6:51] So there is in every single GitHub copilot product, there
[6:54] is this thing called the agent loop and that's the
[6:56] prompts tools context that actually is powering the experience you
[6:59] get in all these things.
[7:01] So traditionally those had all been many different things.
[7:05] And then we built GitHub copilot CLI.
[7:07] And so basically what the SDK is, is taking that
[7:09] thing that we do a ton of offline evaluation on,
[7:11] we work with the labs on to really make sure
[7:14] is an awesome experience for everyone using GitHub copilot.
[7:17] And we take the core of that and it's an
[7:19] SDK.
[7:19] And so you can basically package that agent runtime from
[7:21] GitHub Copilot into any application.
[7:23] Yeah, it's very cool.
[7:24] And if if you're into these like who's doing is
[7:28] anybody out here using open claw or Hermes agent?
[7:31] Any one of these personal only one hand.
[7:33] Wow.
[7:34] OK, so yeah, that's been my experience as well, but
[7:36] one back there.
[7:37] So if you have a Co piloted sub, you can
[7:39] use the SDK and you can basically one shot your
[7:42] own assistant just to play around with it and see
[7:45] how it works.
[7:46] I I encourage you to try that out there.
[7:48] All right, last question.
[7:50] You have 45 seconds new coding model Mai code 1
[7:54] fast.
[7:56] What is it?
[7:56] And is it true that it's better than mythos?
[8:00] It is, it is the first coding model we built
[8:02] from scratch here at Microsoft.
[8:04] So I'm very excited about that.
[8:07] In the blog we compare it to Haiku.
[8:09] So it is kind of, you know, in that class
[8:11] of models, it's built for like fast efficient work.
[8:15] So like a lot of the training philosophy for this
[8:17] model was how do we get this to be really
[8:18] good in GitHub copilot?
[8:19] So specifically trained on GitHub Coabilot's tools, agent trajectories.
[8:23] So that should give it really, really good results when
[8:25] using it specifically in our harness inside of GitHub copilot.
[8:28] And there's a whole bunch of other efficiency things that
[8:30] it's also trained on, like it's supposed to dynamically control,
[8:33] like it's response depending on the prompt you give it.
[8:35] So if you say how interesting, it's not going to
[8:37] go on some diatribe, right?
[8:38] So it has like adaptive solution link training.
[8:42] Yeah.
[8:42] So it has all that going on.
[8:45] So yeah, I'm super excited.
[8:46] I mean, it's the first model that we've shipped for
[8:48] coding here at Microsoft from scratch.
[8:50] Yeah, we've had some fine tunes we've shipped before.
[8:51] So is it really the first coding model?
[8:53] I think so.
[8:55] Maybe there was one before like in the early days
[8:58] but within recent memory we had like Raptor mini but
[9:01] that was a GBT 5 mini fine.
[9:02] Tune.
[9:02] That's right.
[9:03] That wasn't something that was built from scratch entirely.
[9:07] So yeah, we're excited.
[9:08] Like, give it a try.
[9:09] Want to see your feedback?
[9:10] It's definitely like our first attempt in this space.
[9:12] So like it's rolling out in model picker right now
[9:15] for VS Code developers.
[9:16] You can also get it in auto.
[9:17] So if you don't see it, like keep checking back.
[9:19] We're rolling it out slowly, but we'd love your feedback.
[9:21] Awesome.
[9:22] All right, well, thank you so much, Pierce.
[9:24] Round of applause for Pierce here.
[9:25] Please go see Pierce in his breakout session.
[9:29] It's over there on the floor.
[9:30] I don't know what the number is anymore.
[9:31] There it is.
[9:32] All right.
[9:32] See ya.
[9:33] Bye.
[9:33] See ya.
[9:54] Did you know you can use voice to text when
[9:57] prompting in VS Code?
[9:58] All you need is the VS Code speech extension and
[10:01] it's a game changer to be able to just directly
[10:04] speak to kick off your agents.
[10:06] I have this VS Code Community contributor website that lists
[10:09] some of the top contributors for the latest VS Code
[10:12] release.
[10:13] It has a feature where you can give kudos to
[10:16] contributors and I think that this could use some confetti.
[10:19] To prompt this with just my voice, click the microphone
[10:23] icon in the chat input.
[10:26] Add a confetti animation.
[10:27] When a user gets their first kudos, we'll send that
[10:32] prompt off.
[10:34] The agent will get to work and in just a
[10:37] few moments without typing a single key, we have that
[10:40] feature implemented.
[11:02] All right, all right.
[11:03] Am I, Tom, am I supposed to say we're back?
[11:05] We're back, We're back.
[11:07] All right, folks, As I was saying, director, Google Cloud,
[11:12] AI best selling author, Yes.
[11:14] Speaker on AI developer experience.
[11:17] User experience.
[11:18] Adi.
[11:19] Asmani.
[11:21] You're you're very kind.
[11:21] Thank you for being.
[11:22] Hi, folks.
[11:23] Nice to be here.
[11:23] So I was I was I told you I was
[11:25] watching a Bloomberg piece about AI and you were in
[11:28] it and I told my wife I was like, I
[11:30] know that guy she's like no you don't I was
[11:33] like I do.
[11:33] She didn't believe me.
[11:34] She.
[11:34] Didn't believe?
[11:35] You.
[11:35] No, it's.
[11:36] Probably for the best.
[11:38] Yeah, but Addie, Addie, I've known you for a long
[11:41] time.
[11:41] You've been writing and making videos and speaking since way
[11:47] back jQuery days.
[11:49] Oh my.
[11:49] Gosh, doing the web, yes, A lot of web performance
[11:53] stuff and now a lot of AI agents and skills,
[11:56] yes.
[11:57] Yes.
[11:57] A lot of agents.
[11:59] And today we talked about you coming out and talking
[12:01] about you have a skills repo which was on the
[12:04] front page of Hacker News multiple times.
[12:07] I try not to look at Hacker News too much
[12:09] these days.
[12:09] You never know if it's going to be good or
[12:10] bad.
[12:12] Well, tell us about it.
[12:13] All right, so today I wanted to talk to you
[12:16] about agent skills.
[12:17] And you know, a lot of people here probably know
[12:20] what they are, but very quickly I'm, I'm Addy, I
[12:22] work on a bunch of agent stuff at Google.
[12:24] So things like the agent development kit, agent platform, and
[12:27] we work on a number of skills related projects at
[12:30] Google.
[12:30] So we've been thinking about skills for a while.
[12:32] If you're not really sure what skills are, they're basically
[12:36] a standardized way to give AI agents new capabilities and
[12:39] expertise.
[12:41] And they've been a really hot topic over the last
[12:43] couple of months, especially as do you, do you remember
[12:46] back in the days when people would like share their
[12:49] secret like shell config and there'd be like a lot
[12:52] of files and they'd be like, hey, this is these
[12:54] are my magic tricks.
[12:55] Yes, my dot files.
[12:56] These are my dot files that work really well for
[12:59] me my.
[12:59] Magic.
[12:59] Tricks.
[13:00] These are my, this is my bag of tricks.
[13:01] And you'd be like, I want that bag of tricks.
[13:03] Yeah.
[13:04] But then you you think, well, my, my bag of
[13:06] tricks might be different than your bag of tricks.
[13:08] And so I think of skills in a very similar
[13:10] way, you know, sometimes you want to be inspired by
[13:13] other people's ones.
[13:14] Sometimes it's OK to just take, you know, copy their
[13:16] bag of tricks.
[13:17] And so there, I think there's going to be some
[13:19] stuff in your people can just copy and take themselves.
[13:21] And sometimes they're going to be things here that will
[13:24] be inspirational.
[13:25] Now, just in case skills are not something you have
[13:28] spent a lot of time on, I thought it'd be
[13:30] interesting to just give people a very quick primer.
[13:34] And so I I vibe coded this very quick, very
[13:38] silly Windows desktop and a browser.
[13:41] And I thought that I'd drop Clippy in here because
[13:44] I haven't seen Clippy.
[13:45] I haven't seen Clippy so much at.
[13:47] Build we're not we're not allowed to mention Clippy.
[13:50] It's a very.
[13:51] It's a very OK.
[13:52] Yeah, it's very well there.
[13:53] There actually is a Clippy suit.
[13:55] There is a wear, but if you wear it you're
[13:56] not allowed to speak.
[13:57] True story really.
[13:58] Yep.
[13:59] Those are the rules.
[14:00] You didn't mention it.
[14:01] I did.
[14:01] So so we're fine.
[14:02] We're totally.
[14:03] Fine broke us into jail.
[14:04] All right, I broke your.
[14:05] This is weird because we have the Mac OS dock
[14:07] down here, but then the windows above it.
[14:09] I like it.
[14:09] It's bringing the best of all.
[14:10] It's just this is just terrible.
[14:11] It's like worst of all, the worst of all worlds.
[14:14] OK, so if you're not really sure what skills are,
[14:16] basically there's a standardized way of packaging expertise and and
[14:20] context that you know, your agent may not necessarily have
[14:23] it impacted workflows.
[14:25] Your instructions for your agent and you know the structure
[14:28] of them is kind of this.
[14:30] You have a name, you have a description.
[14:32] There can be a lot more content in here, but
[14:35] basically the name and the description are the main pieces
[14:38] that every single skill file is going to have.
[14:40] And these are repeatable.
[14:42] Typically you're not going to have a skill for something
[14:44] that's one off because a one off thing is going
[14:46] to be more like a prompt.
[14:47] But a skill is something that you or your team
[14:50] might be using on a reusable basis, and it might
[14:52] be useful across a few different projects.
[14:55] Now, skills load on demand.
[14:58] So you know you have a task that arrives into
[15:00] your agent, it will match it with a skill, it
[15:02] will load its steps and then it will try to
[15:04] follow them.
[15:05] It doesn't necessarily need to know more than the name
[15:07] and the description.
[15:10] So agent skills, as you mentioned, this is a project
[15:14] that I created a while back and for some for
[15:17] some back story, I had been taking a little bit
[15:21] slower on open source for a few years.
[15:24] You and I have been doing a lot of open
[15:26] source for a long time, long time, a long time.
[15:28] As you know, you create an open source project, it's
[15:30] a lot like having a kid.
[15:32] You have to maintain it.
[15:34] And so you have to be explicit about when you
[15:36] want to get an open source project out into the
[15:38] world and in a while.
[15:39] So I thought, OK, I will get one of these
[15:40] things out there.
[15:41] I had seen that Gary Tan, I had put out
[15:44] G stack.
[15:45] That's right, right.
[15:47] And I thought, well, you know, if if Gary Tan
[15:49] is doing G stack and people are, are learning from
[15:52] it, there's nothing to stop me from putting my perspective
[15:55] on agent skills out into the world.
[15:57] And so I started this project and it got some
[16:00] traction and I thought, you know, it'd be interesting to
[16:03] maybe walk folks through it.
[16:05] My perspective on agent skills is, I would say just
[16:08] a little bit different to other folks.
[16:10] I ground my agent skills around the software development life
[16:14] cycle.
[16:15] I think that they're, you know, I, I lean very
[16:18] heavily into agentic engineering, AI assisted engineering, that side of
[16:22] things where you try to be very disciplined about, you
[16:25] know, I'm going to be explicit about what I want
[16:28] built.
[16:29] I'm going to verify what gets built.
[16:30] I'm going to keep quality in mind.
[16:32] And so for my agent skills, I have a lot
[16:35] of steps that kind of map to the SDLC in
[16:37] here.
[16:38] So define, plan, build, verify, review, and ship.
[16:41] And for each of these different phases of the SDLC,
[16:45] there are a number of different skills that are inside
[16:48] this project.
[16:49] Get the map in here.
[16:50] So for defining, we've got skills that can interview you
[16:54] to get clarity around an idea if you have a
[16:57] very vague idea.
[16:59] So maybe you've had an idea in the shower and
[17:00] you're like, well, I kind of know what the outcome
[17:03] is, but I actually don't know what the UI's gonna
[17:05] look like or I don't know what the details are
[17:07] gonna be.
[17:08] You've got a skill here that can help refine it
[17:10] into something more concrete.
[17:11] You can do spec driven development.
[17:13] We've got planning skills for turning things into tasks.
[17:16] We've got building skills that can help you sort of
[17:19] refine this to a place where you've got very good
[17:22] front end UI, your API and interface design is in
[17:24] a good place, and then sort of skills for a
[17:27] number of these other phases for verification.
[17:29] That's really important.
[17:31] You know, I think you and I have done a
[17:33] lot in the browser over time, right?
[17:34] I still build, I, I used to work on Chrome.
[17:36] I still build a lot of, you know, projects that
[17:38] will render in the browser.
[17:40] And so testing in the browser automatically with an agent
[17:43] these days is something that I care about.
[17:45] So I have a skill that will do that.
[17:47] Automatic debugging is also part of this workflow.
[17:50] I've got a code review loop in here too that
[17:53] includes things like code simplification, security, and then I've got
[17:57] ship skills as well that will help you with everything
[18:00] from your CI to your Git workflow, your documentation, shipping,
[18:04] and launch.
[18:05] Now, there are a lot of phases in here.
[18:06] I'm actually going to show you how these work, but
[18:09] there's basically a pattern that I use for all of
[18:11] my skills.
[18:12] There's an overview I capture, like when these things should
[18:15] be used.
[18:16] I have rationalizations like why should this be used and
[18:20] why should this not be used?
[18:22] So the agent has a little bit of a guardrail
[18:24] that it can, it can keep in mind and then
[18:26] red flags as well so that the agent knows if
[18:29] things are really going kind of off kilter and then
[18:32] verification.
[18:32] So how do we know that we actually did the
[18:34] steps right and in a good job?
[18:36] Now all that stuff just maps to commands as well.
[18:39] So you can explicitly just use like a slash command
[18:42] in copilot in any of your agentic coding tools and
[18:45] trigger any of these phases.
[18:47] Now you don't have to use, of course, my agent
[18:49] skills, as I was saying, they're a lot like your
[18:51] bag of magic tricks or dot files.
[18:53] If there's something in here that's useful to you, copy
[18:56] it, steal it, you know, help, you know, have it
[18:58] make you productive.
[19:00] If you find that the way that I do things
[19:02] is helpful, you know, you can go in and use
[19:03] it wholesale.
[19:05] But I wanted to just show you a little bit
[19:07] of this in action.
[19:08] So I've got a scaffolded out project here where I've
[19:11] installed the skills.
[19:13] We've got some agent files, we've got prompts or just
[19:16] mapping all of those skills.
[19:18] This is what happens when you just install it inside
[19:20] a project.
[19:20] And then we've got some of the skills themselves and
[19:24] I'm over here inside sort of the copilot sidebar.
[19:27] I've selected Gemini 3.5 Flash.
[19:30] I work at Google, I work on Gemini.
[19:32] So I'm going to be using Gemini 3.5 Flash here.
[19:34] It's a great model for agentic coding and for agent
[19:37] tasks.
[19:39] I believe that, you know, VS Code has also got
[19:42] like an agents view these days as well, right?
[19:44] So all of these commands, you know, you can just
[19:47] type in slash, you will see that there are additional
[19:49] commands populated here for using the agent skills and you
[19:52] can use any of these.
[19:53] So like things like refine things like the code simplification,
[19:57] etcetera.
[19:58] But just so that we can see what's actually getting
[20:00] built out, I'm going to stick with using this view.
[20:04] And we can start off with maybe just even doing.
[20:09] A quick look at our skills and I'm just going
[20:12] to show you test development.
[20:13] I was talking about, you know, maybe giving you a
[20:16] bit of an insight into what these files contain now
[20:20] you'll see a lot of content in here.
[20:23] I try to be very diligent about what ends up
[20:25] in context windows.
[20:26] So again, if you feel that there is too much
[20:29] that is in here for your needs, you can always
[20:32] trim it back.
[20:33] You can always optimize it further, but I try to
[20:35] be very explicit.
[20:36] So for test driven development, I capture the DDD cycle.
[20:40] I have a very explicit take on like what does
[20:42] it mean to have a failing test?
[20:43] What does it mean to have a passing test?
[20:45] How do you think about refactoring?
[20:48] What are other patterns that I personally think work really
[20:51] well?
[20:51] Many of these patterns were ones that have been inspired
[20:54] by how we build software at scale at Google.
[20:57] And so I thought they'd be interesting to kind of
[20:59] encode inside these skills.
[21:01] So you'll see things like the test pyramid in here,
[21:03] decision guides.
[21:05] There's a number of best practices that are encoded in
[21:07] here as well.
[21:08] Lower down you'll also see that I capture things like
[21:12] test anti patterns, things to explicitly avoid, how to approach
[21:16] browser testing.
[21:17] So this is that connective tissue that can then take
[21:20] us back to things like browser testing, browser debugging.
[21:24] I tell it what to check.
[21:25] I tell it how to think about security boundaries where
[21:28] testing is concerned.
[21:30] You can be as explicit or open-ended as you kind
[21:33] of want with these things.
[21:34] I try to be explicit just because I think that
[21:37] there's value to being intentional with how you approach these
[21:40] things.
[21:41] Now, what I thought would be interesting, I've been someone
[21:44] that for a lot of my life has been trying
[21:47] to optimize my productivity.
[21:48] I think I've probably failed at it a lot.
[21:52] And so habit tracking is something that I look at
[21:54] now and then, right?
[21:55] I think a lot of people try to improve their
[21:57] habits and their systems.
[21:59] So I thought it'd be interesting for us to try
[22:01] building out a habit tracker.
[22:03] And maybe, you know, if you take a look at
[22:05] everybody's kind of looked at their GitHub contributions graph, maybe
[22:09] we have a habit tracker that uses that kind of
[22:12] visual.
[22:12] So like how, how well are you doing and following
[22:15] your habits over time?
[22:17] So, so we're going to kind of start off with
[22:20] here is I'm just going to trigger the refine command.
[22:24] And the reason I'm going to do this is I
[22:25] have a raw idea.
[22:26] So I want to build out a habit tracker, right?
[22:30] So we're going to say just something like a habit
[22:36] tracker GitHub inspired.
[22:39] And I don't really want a database back in.
[22:43] Maybe let's just have it all work in the browser,
[22:47] so store everything in the browser?
[22:50] That's relatively vague.
[22:52] It's pretty vague.
[22:52] It's pretty high.
[22:53] Are you going to say contribution graph or are you
[22:54] going to leave that out?
[22:55] I'm just going.
[22:56] To keep it, I'm just going to keep it GitHub
[22:57] inspired.
[22:57] I could say contribution graph, but that's being explicit.
[23:00] That is right.
[23:01] So we're just going to go and trigger refine now.
[23:05] And so it's analyzing our prompt parameters, it's analyzing the
[23:08] core components of the request.
[23:10] It's triggering the skill.
[23:11] You can see that it's been reading that skill in
[23:13] place.
[23:14] Now the first thing that's popped up here is it's
[23:16] asking me clarifying questions.
[23:18] So who is the primary target user?
[23:20] Great question.
[23:21] It doesn't know if this is just for me or
[23:23] I'm building this for a team, or I'm building this
[23:25] for my family or my friends or what.
[23:27] So the audience is going to be, let's say a,
[23:31] a room of software engineers at Build right or YouTube,
[23:35] because people are going to be watching this later.
[23:39] Just anyone.
[23:40] Just anybody, what are the constraints and expectations for browser
[23:43] only storage?
[23:44] Great question.
[23:45] You could be using index DB, local storage, session storage,
[23:48] lots of different options.
[23:50] I'm going to keep it very simple.
[23:51] I'm OK with local storage for this beyond the standard
[23:55] contribution grid, so here it was able to figure that
[23:58] out itself.
[23:59] What GitHub analogies do you want to explore?
[24:01] Like committing a habit, branching variations, pull requests?
[24:05] I don't want to go too deep into these analogies,
[24:08] so I think let's keep it.
[24:09] Let's keep it simple with the contribution grid.
[24:13] Interesting that it's leaning into that.
[24:15] It's like it wants to make the whole thing about
[24:17] GitHub.
[24:18] And this is one of those things, if you take
[24:20] a look at how that skill is defined, it actually
[24:23] has language around trying to begin converging on the direction,
[24:27] the idea that you have leaning into like, OK, well,
[24:29] I kind of get what you're going for.
[24:31] Let's keep going down that path.
[24:34] So what does success look like for the MVP?
[24:36] Because very often if you define like just a high,
[24:39] a high level prompt, it doesn't know if you're trying
[24:41] to build out something for a startup or a side
[24:43] project at the weekend or what.
[24:45] So I feel like the performance requirements are good.
[24:50] Like all of this stuff seems good.
[24:51] I don't necessarily need it to be offline first with
[24:54] service workers just yet.
[24:55] So load fast, clean UI, that's about it.
[25:03] Load fast and have a clean UI be responsive.
[25:10] OK, so we've got all these answers now.
[25:13] Didn't take us very long just to start getting clarity
[25:15] around this, right?
[25:16] I didn't have to go writing a big spec myself
[25:19] or anything like that just yet.
[25:20] And so it's trying to read the rest of my
[25:23] project yet just now trying to see if there's any
[25:26] other context in there.
[25:27] This is a fresh project, so there's not a whole
[25:29] lot that it needs.
[25:30] And here's the output, right?
[25:31] It has a problem statement.
[25:33] It has statements around like how we differentiate directions, think
[25:37] about stress tests, and it gives us a few directions
[25:39] we can take.
[25:40] So direction A is a deaf, deaf profile Configurator.
[25:44] So a very minimalist high fidelity clone of a GitHub
[25:46] profile allows us to track habits the same way that
[25:49] you would GitHub repositories.
[25:51] There's a lot more detail in here.
[25:53] Direction B is more of a keyboard driven habit terminal.
[25:57] I'm more of a 2DS.
[25:58] Are so hot.
[26:00] Right now, yeah, 2DS are so hot.
[26:02] Direction C is a markdown SVG exporter, so a plain
[26:05] text back end tool, your habit configuration and grid data.
[26:09] So this is more data.
[26:11] This is more like you really want to enter in
[26:13] raw data and manage that, which is not really the
[26:16] vibe I was going for.
[26:18] I think that the first one's probably the closest to
[26:21] what I had in mind.
[26:22] So I'm going to go with direction A.
[26:24] And you can also see that it's got other assumptions
[26:27] in here and push backs.
[26:30] So it's asking me which direction resonates the most.
[26:32] Just going to say direction.
[26:33] A.
[26:33] Did it?
[26:33] Did it recommend one I couldn't see under the counter?
[26:35] There so we can scroll back.
[26:37] Once you pick AB or CI will produce 1.
[26:39] It didn't recommend 1.
[26:40] You can actually change your skill files to say, well,
[26:43] I want you to provide your own perspective on it.
[26:46] OK, in this case I'm trying to you know.
[26:49] Because I noticed models like to recommend and they almost
[26:52] always when they give you choice and be like, well
[26:54] this is what I do.
[26:55] And then I'm like, well, if that's what you do,
[26:57] then that's what I want to.
[26:58] Do and sometimes I will ask the model like give
[27:01] me 3 choices and tell me what you think I
[27:03] should do just in case.
[27:05] Yes.
[27:05] But then I feel like I'm not.
[27:06] I'm just always doing what the model wants, you know
[27:08] what I mean?
[27:09] I'm like, well if you think it's best and it's
[27:11] probably best, which is not true, I have.
[27:12] So much to say about cognitive debt and cognitive surrender.
[27:15] That's right, that your next article on.
[27:19] So what we have here, so the app is called
[27:21] Streak, we have a problem statement, we've got a direction
[27:25] here, we've got some assumptions to validate like what is
[27:28] the retention rate?
[27:30] What is like the frequency going to be around habits?
[27:33] What's the mobile usability going to be since engineer configure
[27:37] setups on desktops, but often complete habits on mobile.
[27:40] Is local storage enough?
[27:41] Well, probably, yeah, local storage is well supported everywhere at
[27:43] this point.
[27:44] So I think that pretty much all of these ideas
[27:46] are cool.
[27:47] So we have our MVP scope defined for us here
[27:51] and it also says what it's not doing.
[27:54] So it's not doing cloud sync it, it's not going
[27:56] to be doing advanced social features.
[27:58] And it's got some open questions for us if we
[28:00] want to provide clarity around that.
[28:02] And it also asks us do we want to just
[28:03] save it?
[28:04] So I'm going to say yes, save it.
[28:07] So we've now taken a very high level idea and
[28:10] we've refined it.
[28:11] Now the next phase of this is actually turning this
[28:14] high level idea.
[28:15] So we've gotten it into a better place.
[28:17] Let's turn this into a more detailed spec.
[28:20] Now for some folks, you might say, well, it feels
[28:22] like it's a few, it's a few steps.
[28:24] You don't have to follow my way of doing things.
[28:26] If you want to just have like one step for
[28:28] creating your spec, you can do that.
[28:31] I just happen to like specificity.
[28:33] So you like to plan.
[28:34] You want the plan, the spec, and then the implementation.
[28:38] I'm a plan implement guy myself.
[28:40] Just skip over.
[28:41] That's totally OK Everybody's got their own workflow.
[28:43] Here you can see the file that it's generated, and
[28:46] this is going to be a starting point entry for
[28:49] the next step.
[28:50] So the next step is going to be spec.
[28:52] So we've got our idea.
[28:54] We can go and just type in slash spec and
[28:56] it's then going to read in the output from that
[28:59] last phase, it's going to start being able to generate
[29:02] something that's much more concrete.
[29:05] So it's taking in those assumptions, it's defining success criteria,
[29:09] scaffolding out the tech stack a lot more in detail.
[29:13] And then it's got those open questions in a little
[29:14] bit more detail and it's.
[29:15] Really snappy.
[29:16] Yeah, it's pretty quick.
[29:18] Again, use Gemini 3.5 Flash or whatever model is out
[29:21] at the time you're watching this on YouTube.
[29:24] Yeah, 'cause.
[29:25] It move AI moves fast.
[29:27] 4 hours everything will be different.
[29:29] Everything will be different, so we've got some open questions.
[29:32] Grid generation techniques?
[29:33] Should we use a standard GitHub contribution grids?
[29:37] We prefer rendering grids as modular dynamic SVG group structure
[29:41] or CSS grid element layouts.
[29:44] I'm going to be out.
[29:45] I know CSS grid, I use SVG, but I'm not
[29:48] an expert so I'm just going to for one.
[29:51] I'm going to say choose what you think is best
[29:54] and then first visit boilerplate.
[29:57] What default habits should we set up for new visitors?
[30:01] That's a good question.
[30:05] Bad habits or are we tracking bad?
[30:07] Habits.
[30:08] We're we're tracking good habits.
[30:09] OK, good habits we should good habits.
[30:12] Exercise, hydration, reading.
[30:16] Reading meditation.
[30:18] Do people meditate?
[30:19] Do scrolling?
[30:20] Do scrolling OK.
[30:21] Meditation.
[30:22] Meditation, probably.
[30:24] Better than do meditation after you're do scrolling.
[30:26] OK, do you have a desired location for this file?
[30:29] I'm just going to let it do what you think
[30:33] is best.
[30:35] Cool.
[30:36] What I love about this is all of this UI,
[30:39] All these questions are directly inside of the Copilot sidebar.
[30:43] I don't have to use some other app for it.
[30:45] Whatever your agentic UI surface is, you can just have
[30:48] a similar experience there.
[30:49] So it's now created a spec for us.
[30:51] I'm going to keep this and the next phase you
[30:53] can go and you can take a look at the
[30:55] spec.
[30:55] By the way, it's over here.
[30:56] It's going to be a much more detailed version of
[30:59] that original file that we had.
[31:00] But you'll see that we have success criteria.
[31:03] We've got metrics in place here for performance, so things
[31:06] like largest content, full paint.
[31:08] We've got a text stack that's defined.
[31:10] This is going to be very simplistic, so vanilla JavaScript.
[31:13] I didn't say I wanted something in React.
[31:15] I'm OK.
[31:16] With it never asked you about the text stack.
[31:18] Did it.
[31:18] It didn't ask me about the text stack.
[31:19] And that's another thing that you can encode inside of
[31:22] your skills.
[31:22] So this is an assumption that you just let this
[31:24] is.
[31:24] An assumption that I'm OK with.
[31:26] You can assume like it's defined by test and playwright
[31:30] for our end to end testing, which I'm OK with
[31:32] our storage, and then project structure and code style.
[31:36] And if there's anything in here that you don't like,
[31:38] you can always go back and either manually tweak it
[31:40] or work with your agent.
[31:41] So you configure this more.
[31:43] So you've got your spec.
[31:44] Now the next thing that you might want to do
[31:46] is actually start breaking this down into individual pieces.
[31:50] Now, this is useful if you've got a large project
[31:52] that you're working on.
[31:53] If you're working on something small, you probably just want
[31:56] to go straight into implementation.
[31:58] But I'm going to show you this plan in action
[32:00] anyway.
[32:00] So we've got a plan step here.
[32:03] And what we can do with plan is I can
[32:06] go and I can run plan.
[32:08] I'm just going to show you this working.
[32:10] And what this can do is help breakdown.
[32:12] What we say in software engineering often is if you've
[32:15] got a big problem, break it down into smaller pieces,
[32:17] right?
[32:17] And plan kind of helps you with that.
[32:19] It breaks your big spec down into granular chunks that
[32:22] can then be implemented.
[32:24] And that's really good if you're working with TDD because
[32:27] you've got, you know, very individual pieces that can be
[32:30] tested, verified as you move on to other pieces.
[32:33] People have lots of different ways that they like doing
[32:35] TDD.
[32:35] So I'm not going to assume that my way is
[32:36] is the right way or anything like that.
[32:38] So it's going and it's scaffolding out the basic parts
[32:41] of the project right now because that's fine.
[32:43] Like it can set up our boilerplate before we actually
[32:45] go and start implementing the rest.
[32:47] I'm just going to do keep on that and you'll
[32:50] see that it's asking before beginning phase one, I should
[32:53] initialize our structured To Do List to provide full transparency
[32:56] and it's going to do that now.
[32:59] So what you're going to have here is the package
[33:02] dot Jason file, which we see has been created.
[33:07] The implementation plan is established and saved to plan.
[33:10] So we've got a task list.
[33:11] Here you see phases and tasks.
[33:14] So we've got phase one task one, which has its
[33:17] acceptance criteria.
[33:18] And this is like build the pure logic, TypeScript interfaces,
[33:22] your types, your functions to compute the streaks, your total
[33:25] contributions within the last like 365 days, the local storage
[33:28] pieces, basically stuff that doesn't necessarily touch your UI.
[33:32] And we've got some acceptance criteria here as well.
[33:35] So we've got a list of things that have to
[33:37] be true as well as verification.
[33:38] So the test suite has to be able to run
[33:41] on that piece of logic in order to move forward.
[33:44] It also suggests, like, what files are probably going to
[33:47] be touched as a part of this just to like
[33:49] really harm them down.
[33:50] If you've been doing any vibe coding or genetic engineering
[33:53] for a while, you know that one of the ways
[33:55] in which we try to limit our blast radius is
[33:57] by saying, well, I'm just going to target these specific
[34:00] files because especially if you're working on an existing project,
[34:03] you don't want it to start overwriting or touching files
[34:06] that maybe have nothing to do with the task at
[34:09] hand, right?
[34:10] So we have a few different phases and tasks here.
[34:12] Some of the other phases will include our data storage.
[34:15] So that layer for local storage, it'll include sort of
[34:19] the visualization of our habits as that GitHub style contribution
[34:23] graph as well as the rest of our UI.
[34:26] So we've got our plan here.
[34:28] What you can do, again, this supports being very granular
[34:31] or or not having to think about that in as
[34:33] much detail.
[34:34] So if I wanted, I could do something like slash
[34:36] build task one and just have it do the first
[34:39] task on that list.
[34:40] So let's see what it does.
[34:42] It's created for to do's as a part of that,
[34:45] it's evaluating it.
[34:47] It's reading our test driven development skill right now.
[34:49] You can see that it's installing those first things I'm
[34:52] also just going to do.
[34:53] Yeah, if you.
[34:53] Always allow.
[34:54] You can do it down at the very bottom and
[34:56] turn on bypass approvals if you want to.
[34:59] I'm learning something new.
[35:00] Where do I where?
[35:01] Oh wait, wait, wait.
[35:01] I see what you're talking about.
[35:02] Here, right there, yeah.
[35:03] And then we'll stop asking, are you a bypass approvals?
[35:06] You recommend a bypassing approval.
[35:09] I bypass approvals on all my personal projects.
[35:16] So now you're building one phase at a time.
[35:20] Is this generally how you work or are you more
[35:22] I?
[35:22] Think it really depends on what you're building.
[35:24] So if I am building a personal side project and
[35:27] I just want to build something like maybe it has
[35:30] one purpose, probably going to skip building out one phase
[35:34] at a time this way and just have it build
[35:36] the entire thing.
[35:38] If I'm building something relatively complex, I'm OK doing it
[35:41] a phase at a time.
[35:42] It might feel like it's slower, but the fact that
[35:45] you go through all of these steps and you know,
[35:47] well, this slice is tested as we build out the
[35:50] UII know nothing else is broken.
[35:52] I've now done our data layer.
[35:54] Nothing else is broken.
[35:55] This gives you a lot more confidence as you're working
[35:57] through.
[35:58] It's just a just a different way of approaching.
[36:00] It's such a trade off though because then you know
[36:02] the back in the loop, like once again, you're the
[36:05] problem.
[36:06] You're slowing the agent down.
[36:07] It's tough.
[36:08] I struggle with this myself.
[36:09] Yes, yes, absolutely.
[36:11] And I would say that, you know, you have to
[36:13] make a judgement call at the end of the day
[36:15] about how much time you want to invest in where
[36:18] in in the software development life cycle you want to
[36:21] invest it.
[36:21] Do you want to put a lot of time into
[36:24] your spec?
[36:25] Just have your agent build out a lot of code
[36:28] and then spend time reviewing it, defining your tests at
[36:32] that point figuring out like, how do I evaluate quality
[36:36] like that?
[36:36] That can be OK as well.
[36:39] Or do you want to approach it with TDD in
[36:41] mind?
[36:41] Do you want to have like an explicit strategy around
[36:44] how you're going to verify quality so that as you're
[36:46] going through, it's already addressing these things?
[36:49] And I don't think there's a right or wrong way.
[36:50] It's just different ways of working, you know what I
[36:52] mean?
[36:54] So what you can see with this first phase is
[36:56] that the implementation is test driven.
[36:58] We have our first target failing red test to to
[37:01] tell us that OK, we have a test, it's failing,
[37:04] We're going to implement the minimal clean logic in streaks.
[37:08] We're red green here, yes.
[37:10] And as I said, you don't have to do this
[37:13] with like individual tasks.
[37:14] We can just keep running, built to work through the
[37:17] rest of this and then we can get on to
[37:20] other phases.
[37:21] So you can see that it is thanks to to
[37:24] Burke's tip around bypassing approvals.
[37:27] We're just going to keep running through this.
[37:28] And I'm a person that people will often talk about,
[37:31] you know, how do you as an engineer understand what
[37:34] is getting generated if you're not reading all of the
[37:38] code?
[37:38] Yeah, very often as as your agent is generating stuff,
[37:42] what are you doing?
[37:43] Are you switching over to another agentic task?
[37:46] Like very often I guess I found myself doing that.
[37:48] And so I started to slow down a little bit.
[37:51] And these days I try to read the trajectory a
[37:53] little bit more like what did it say, what was
[37:56] it thinking?
[37:56] Why was it thinking what it was thinking?
[37:59] And is there any nugget of information in there that
[38:02] I can extract?
[38:03] It just lets me know.
[38:03] Oh, OK.
[38:04] Well, that was the way I I was expecting what
[38:07] you mean.
[38:08] You know what I mean?
[38:08] Yeah, so it, it's interesting because I, I don't know
[38:12] what everybody else does, but I, I watch the agent
[38:14] work.
[38:15] Now, I'm not reading everything that it's doing.
[38:17] I'm mostly just staring at it when I should be
[38:20] doing something else.
[38:21] But I'm just, I'm literally watching agents spin.
[38:24] But I'm curious, like, are you like, are you looking
[38:27] at the code?
[38:29] Great.
[38:29] Question.
[38:31] I try to I generally, unless it's a personal project,
[38:34] because a lot of personal projects end up being throwaway
[38:38] things.
[38:38] Oh, I have this, I have this need for an
[38:40] hour.
[38:41] I'm just going to build something that's thrown away.
[38:43] And in those situations, I'm very OK with saying if
[38:46] I don't understand the code, the agent is going to
[38:49] go fix something for me.
[38:51] However, if it's going to be something I'm going to
[38:54] maintain for at least a few weeks or a few
[38:56] months, I generally go back and at least understand the
[38:58] architecture.
[38:59] I try to read through some of the core files
[39:02] so that if I discover that the agent wasn't able
[39:05] to fix something or it goes off on a tangent,
[39:08] I am able to go back in and make fixes
[39:11] myself, you know, very often.
[39:13] So I'm going to keep working through build and we
[39:15] can keep chatting.
[39:16] It's so it's going to keep doing it's TDD phases.
[39:19] As each slice gets implemented, it's going to go from
[39:21] red to green, implement the phase and get everything done.
[39:24] This is so interesting.
[39:25] I didn't even know about red green testing until Spec
[39:28] Kit came out.
[39:29] It was my first.
[39:30] It was the first time I'd ever seen that.
[39:32] That's the idea of writing failing tests and then making
[39:35] them work.
[39:35] And it was like 40.
[39:37] Five years old.
[39:38] It's kind of sad.
[39:40] You do not what it is.
[39:41] You do not look over 45 at all.
[39:44] I'm not, yeah.
[39:45] 445 it was yesterday.
[39:48] Happy birthday.
[39:49] Thank you so much.
[39:50] So we're going to keep doing this.
[39:52] I also wanted to to have to share a comment
[39:54] on this.
[39:55] I think that very often in the entire sort of
[39:58] agentic engineering diaspora, the industry paints very broad strokes about
[40:03] best practices, like if you're, if you're on Twitter, if
[40:07] you're on any social network, very often the best practices
[40:11] for a solo founder or a small startup team.
[40:14] Will be shared and the best practices for that kind
[40:18] of team that maybe don't have users yet, maybe they're
[40:21] working on an Evergreen code base, those are very different
[40:25] to people who are working on something either a legacy
[40:28] brownfield with a team.
[40:31] And so I think that it's important just to understand
[40:33] that, you know, there are situations where it's OK to
[40:36] just give in the vibes and have the agent figure
[40:38] things out.
[40:39] And if something goes wrong, we'll just have the agent
[40:41] fix it.
[40:41] In other situations where actually understanding what's happening behind the
[40:44] scenes is still quite important.
[40:45] Yeah, 100% agreeing.
[40:46] I actually so what's super interesting about this is that
[40:50] I think myself and most people are looking for what
[40:52] it just tell me what to install that's going to
[40:55] make everything work right.
[40:57] Is it is it agent skills?
[40:59] Is it super powers?
[41:00] Is it G stack?
[41:00] Like what is it?
[41:01] And the answer is, it might be some of any
[41:04] of those things, but your job is less about writing
[41:07] code now and more about building these work flows that
[41:11] actually work.
[41:11] And it's not always the same, even for different projects
[41:15] within the same organization.
[41:17] So absolutely.
[41:18] It's a.
[41:18] It's a.
[41:19] Very.
[41:20] I don't know what to call it.
[41:21] It's not software engineering.
[41:22] It's it's not automation.
[41:25] It's how would you even describe this?
[41:28] Like we all work at McKenzie now.
[41:31] I I don't know.
[41:32] I, I acknowledge that agentic engineering, we're now working on
[41:37] a spectrum, right?
[41:38] And there are going to be times when we're vibe
[41:40] coding, there's going to be times when we're agentic engineering.
[41:42] And the more diligence you have to apply to the
[41:45] process, the closer you are getting to agentic engineering.
[41:48] And I think that at big companies, small companies, anytime
[41:51] that you're working on something that exists and has existing
[41:55] users, you owe it to them to care about quality,
[41:58] to care about not breaking things in ways sometimes agents
[42:02] subtly do.
[42:02] Yeah, 100% agree.
[42:04] If you're building a habit tracker, that's one thing.
[42:06] If you're shipping VS Code, you have to be pretty
[42:09] meticulous, Yeah, about not breaking that.
[42:12] Yeah.
[42:13] So we have about 10 minutes left here, so.
[42:16] What I'm going to do is I'm just going to
[42:18] skip ahead a little bit and just show you very
[42:20] quickly some of these other phases.
[42:22] So the agent skills package also includes a verify step.
[42:27] Now what Verify does is it's able to just check
[42:30] that the implementation as it that has been done so
[42:33] far, whether it's complete or whether we've partially implanted something
[42:37] that can actually be used just yet, can run in
[42:40] a browser and actually do what it needs to do.
[42:43] So it's going to go and check that it's actually
[42:46] able to spin something up.
[42:48] Let's see if enough of it has been built that
[42:50] it's able to do that.
[42:51] So it's doing it's NPM run build right now.
[42:53] It's going to spin up it's NPM run dev and
[42:55] we'll see if this actually works.
[42:56] So we have.
[42:57] We have something that looks like a GitHub profile where.
[43:00] Here, pull that picture from.
[43:02] I have no idea who is that the magic of
[43:04] LLM.
[43:06] That someone if that person's watching.
[43:08] Someone.
[43:09] Someone in the world.
[43:12] Hello whoever this is this could this could be like
[43:15] a non developer as well.
[43:17] This could be a person that just likes to like
[43:18] bake cookies somewhere in the world.
[43:20] It just happens.
[43:20] To be on GitHub.
[43:21] Yeah, it just happens to be on GitHub.
[43:23] But you know, it's it's opened up this browser directly
[43:26] inside of VS Code.
[43:28] It's able to run its checks behind the scenes to
[43:31] see OK, well, is anything actually broken?
[43:33] If I interact with UI, does anything actually begin to
[43:36] break?
[43:37] And I think that this is very interesting, like if
[43:40] you actually go to the prompts and you go to
[43:43] our verify skill, it will end up trying out the
[43:45] browser testing with dev tool skill.
[43:47] There are many ways to do sort of automated browser
[43:51] testing these days, whether you're using Puppeteer, Playwright, the Versal
[43:55] like agent browser, there's a lot of different options.
[43:59] I'm not going to say there's there's a right one,
[44:01] but choose what makes sense to you.
[44:02] I just think there's a lot of really great tools
[44:05] these days for just making sure that your thing basically
[44:08] works.
[44:08] Whether you decide to kind of encode all of your
[44:11] user journeys and test them out very, very holistically is
[44:14] kind of up to you at the end of the.
[44:16] Day web devs as usual have it better than anybody
[44:19] else.
[44:20] Right?
[44:20] Like it's so much harder to do with native apps.
[44:23] Now for for my skills, the review phase ends up
[44:26] being really important and people have a lot of opinions
[44:30] about review.
[44:32] I tend to find, and this is something that I
[44:34] found talking to engineers at Google as well, we still
[44:38] have a culture where we tend to have engineers manually
[44:41] review code.
[44:42] Even even with that, even with trying to bring AI
[44:45] into our code review process, having, you know, an agent
[44:49] do a local first pass or multi pass is really,
[44:53] really valuable.
[44:54] Now there are some people that will take this all
[44:57] the way to adversarial code review with very deep, you
[44:59] know, patterns.
[45:00] There are people who will have other models, you know,
[45:04] like, hey, I'm going to, I'm maybe I'm going to
[45:07] use Gemini for the implementation and Opus or codecs rather.
[45:11] So you have a lot of flexibility here.
[45:13] There is, again, no right or wrong way.
[45:15] People will often decide these things based on vibes.
[45:19] But what you'll see is we have a number of
[45:21] different kinds of review checks that were done here.
[45:24] So what's working well, what's technically considered correctness?
[45:27] So all of the unit tests that were implemented so
[45:29] far appear to be, you know, passing.
[45:32] We have no security issues.
[45:34] It seems, you know, we're working with vanilla JavaScript.
[45:38] We of course want to avoid XSS issues or cross
[45:40] site, you know, cross site scripting, anything like that.
[45:44] Readability and architecture seems to be in a good place.
[45:46] Performance seems to be a good place.
[45:48] This is a simple, you know, application and it also
[45:50] applies some recommendations.
[45:52] So keyboard navigations for that contribution graph makes sense, right?
[45:56] Like, that's not something that I had considered.
[45:59] And you might find that depending on the complexity of
[46:02] what you're working on, that you, you can have this
[46:04] be much more elaborate.
[46:07] Another part of code review that I find useful these
[46:10] days is code simplification.
[46:12] Now, very often what we'll do is we'll have an
[46:14] agent implement something.
[46:15] We'll verify, Maybe you'll read the code, maybe you'll verify
[46:18] that, hey, it at least runs in the browser.
[46:20] Or, you know, if you're building a native app, hey,
[46:23] it at least runs, but you don't necessarily go back
[46:25] and ask yourself, hey, could this actually be simpler?
[46:28] And one of the wonderful things about TDD or having
[46:31] tests or quality gates around is that you can now
[46:34] go and do that code simplification loop and have something
[46:38] that can verify the rest of your logic still works.
[46:41] Yeah.
[46:41] Right, very interesting.
[46:43] So I question on the security one, how much do
[46:46] you trust the non deterministic?
[46:48] Because to me, this is the, this is the, this
[46:51] is where the, the rubber meets the road, right?
[46:53] The rest of this stuff you can deal with.
[46:55] The security stuff cannot.
[46:58] How much do you trust that review that you're not
[47:00] pushing a key, you're not pushing some sequel injections, something
[47:03] that's just going to get you in a ton of
[47:05] trouble down the road?
[47:06] Oh yeah.
[47:07] I think that there are a lot of goods, open
[47:10] source and commercial offerings that go even deeper into security
[47:14] than than these skills do.
[47:17] And so there are a lot of things that like
[47:18] we've seen anybody that's been doing by coding and jetting
[47:21] engineering for a while has run into these issues, right?
[47:24] Very often people will be using API keys in horrendous
[47:28] ways, right?
[47:30] They will be quickly putting together something and not realize
[47:33] that oh hey wait, there are actually tons of data
[47:35] leakage issues or the way that Office implemented is leaving
[47:38] you exposed to all kinds of problems.
[47:40] Or somebody can just nail your API over and over
[47:42] again because it's not locked.
[47:43] Exactly.
[47:44] And if you're, if you're a senior engineer, if you're
[47:46] experienced, you know what to look out for.
[47:49] If you're intermediate, if you're not quite as technical, if
[47:52] you're coming at this without as much of that background,
[47:55] or maybe you are an experienced engineer and you're just
[47:58] lazy.
[47:58] Because sometimes we are lazy.
[48:00] It's useful to encode those things in skills just as
[48:03] like a sanity check for yourself.
[48:05] Oh.
[48:05] Actually list the.
[48:06] So I find that I cannot think of all the
[48:09] edge cases, but models are actually really really good at
[48:13] this.
[48:13] Like if you if you ever ask a model to
[48:16] review a code base, it will almost always find something
[48:19] that you need to change your.
[48:21] Input.
[48:22] Right.
[48:22] They're really, really meticulous and good at that.
[48:25] So yeah, again, I don't know.
[48:27] I I feel like we need maybe better products or
[48:29] whatever it is, but all of this to me, agents
[48:32] code should go through some deterministic gate.
[48:35] That's like, yes, security sign off has been achieved because
[48:38] I don't trust I'm scared to death to push anything
[48:41] live on Twitter and be like, here's what I built.
[48:44] And then have somebody be like, Oh, and here's your
[48:46] Gemini key.
[48:47] And I think that's that's where there is a lot
[48:50] of value in understanding what does good mean?
[48:53] What does done mean?
[48:54] Right.
[48:55] And if you care about security, you care about quality,
[48:58] you care about performance, you care about accessibility, you care
[49:01] about any of these things, that's a good reminder to
[49:03] try and coding those into how you approach quality gates
[49:06] in your projects.
[49:07] Yeah, skills are just one way of of encoding that
[49:10] into how you think about all this stuff.
[49:12] But I think that just having tooling in place to
[49:14] make sure that you're not merging in code that's going
[49:17] against those best practices is always is always a good
[49:19] idea.
[49:20] Yeah, I feel like we need more of these deterministic
[49:22] gates.
[49:22] I just don't know what they are, you know, like
[49:24] right now, Markdown is the solution to everything.
[49:27] It's like, well, we have nondeterminism.
[49:29] How are we going to fix that with more nondeterminism
[49:32] with text files, With text files.
[49:35] So we're at about 6 minutes here.
[49:36] I'd like to do some Q&A because you can continue.
[49:39] Yeah, yeah.
[49:40] Let me.
[49:40] So I'll wrap this up in a minute.
[49:42] So there are a lot of great teams, labs who've
[49:45] been thinking about code simplification.
[49:48] And very often what I love to do is anytime
[49:50] that they open source their work or they write up
[49:52] a blog post about their work, I go and study.
[49:54] Like, is there something that I can learn from what
[49:56] they're doing that I can then incorporate into my skills
[49:59] or how I work?
[50:00] You can do that for code simplification as well.
[50:03] Even for this simple app, it's already found like 3
[50:05] or 4 opportunities to simplify the code.
[50:08] So that's great.
[50:09] We've got a ship step in here as well.
[50:11] Just for time.
[50:11] I'm just going to skip ahead and show you like
[50:14] one that I built earlier.
[50:16] This is like this is called streak.
[50:18] It's got a GitHub contribution graph style, you know, graph
[50:21] at the very bottom.
[50:23] And it just shows us over time, like are you
[50:25] actually following some of these best practices?
[50:27] So for me, am I going screen free for some
[50:30] part portion of the morning?
[50:32] Am I, you know, getting my work day goals planned?
[50:36] Am I reflecting so you can you know, this is
[50:39] not a complicated app, but you can pop open, you
[50:42] know, your browser dev tools, you can go into the
[50:45] application panel, local storage and we can see that all
[50:49] of our data is kind of here.
[50:50] We've got all of our entries for each of these
[50:53] things.
[50:54] It's all local.
[50:55] It all works like all this will work offline, but
[50:57] it does the trick.
[50:59] Now you can do we can do Q&A you.
[51:03] Can.
[51:04] Do.
[51:04] Q&A.
[51:04] So let's spend a few minutes.
[51:06] Do we have a Tom?
[51:07] How do we do this?
[51:07] We have a mic or we're going to do Q&A.
[51:11] Oh, right there.
[51:12] Yeah.
[51:12] So if you do, we'll continue building here.
[51:14] If you want to ask questions, now is your chance
[51:18] to do so.
[51:18] In the meantime, we can continue building your chat about
[51:21] skills if you want to.
[51:22] It's up to you.
[51:23] Awesome.
[51:24] If folks have questions, I'm happy to take it.
[51:25] Otherwise, I'm happy to keep showing you stuff.
[51:31] All right, we got one.
[51:32] It's Anthony.
[51:33] Hey.
[51:34] Yeah.
[51:34] A question for you.
[51:35] How would you?
[51:36] Split up the work between like a flash type model
[51:39] and a Pro model.
[51:40] Would you plan in Flash and then implement in Pro?
[51:44] How would you split the work?
[51:46] Did you hear that?
[51:46] Yeah, interesting question.
[51:47] So how do you split up between models?
[51:48] Like do you plan in flash and then implement in
[51:51] pro?
[51:52] Like are you multi model between the Gemini family?
[51:54] Like what's your personal?
[51:57] Take care.
[51:58] Yeah, I that that's exactly my workflow.
[52:00] I tend to, and especially if you're trying to optimize
[52:03] for cost, that's a great way to go about it.
[52:06] You can use a lower cost model often for the
[52:09] planning phase, especially if you're trying to like optimize for
[52:12] tokens, use a lower cost model for the planning phase,
[52:16] Use a much more capable model for your implementation.
[52:19] And whether that's, you know, going to be one of
[52:21] the Gemini Pro models or Opus or codecs, any of
[52:23] those options can be good for your needs.
[52:26] But I found that that flexibility works well and Copilot
[52:28] and many other tools also let you just like easily
[52:30] switch right between different models for these needs and these
[52:33] different phases, which is which is great.
[52:35] Interesting.
[52:35] I have a question for you.
[52:38] Where does MCP factor into this workflow?
[52:43] That's a great question.
[52:44] I have too many strong opinions on MCP these days.
[52:46] I think that let's.
[52:46] Do it.
[52:47] Yeah, we actually have some extra time so we can
[52:49] go over a little.
[52:50] Bit right?
[52:51] I personally think that things are things have been moving
[52:55] so fast, right?
[52:56] And earlier on we would talk about the difference in,
[52:59] you know, hey, MCP was really good for, you know,
[53:02] the, the connectivity right between data and agents and so
[53:06] on.
[53:07] And I think the people found that very often there
[53:10] are cases where you actually want to CLI instead of
[53:13] an MCP.
[53:14] And we've started to see workflows begin to evolve a
[53:17] little bit more because maybe you don't need to have
[53:20] so many tools exposed.
[53:21] Maybe just directly using CLI is going to be a
[53:23] more efficient way to achieve your goals.
[53:26] So I personally feel like we are still in that
[53:29] kind of evolution phase of seeing like, OK, where is
[53:32] MCP going to be the best fit versus just having
[53:35] people directly use the CLIS for things?
[53:38] This is kind of a contentious area, I feel like
[53:41] because I at one point I did say your MCP
[53:44] server should probably be a skill and ACLI, I kind
[53:48] of still stand by that just because a is are
[53:51] so good at using CLI.
[53:53] The only rejoinder I have to that I was chatting
[53:56] with with Reese, who I think is over at Open
[53:59] Code now about this on on X and he pointed
[54:02] out that the authentication is a real problem and that
[54:06] MCP does this really, really well.
[54:08] And with CLI it's really hard to control permissions for
[54:13] your agent, so I agree with it.
[54:15] But generally speaking I almost use entirely skills.
[54:18] The only MCPS I use is of course work IQ
[54:21] right?
[54:22] Cuz if you're in the Microsoft ecosystem that's like the
[54:25] best thing ever.
[54:26] And then context 7 I I.
[54:28] Always talk about that.
[54:28] It's great.
[54:29] It's.
[54:29] Amazing.
[54:30] If y'all aren't using the Contact 7 MCP server, that's
[54:33] the one thing you should install today and just tell
[54:36] your agent always read the docs before you do anything
[54:39] and it will use Contact 7 to read the documentation.
[54:42] Brilliant.
[54:42] I feel I love that, you know, even in Google
[54:45] Cloud, I know in lots of other places people have
[54:47] started to release and maintain more skill packs just because
[54:51] you can get so far these days with making sure
[54:53] that regardless of what cut off date your agent's data
[54:56] is going to be, you've got up to date documentation,
[54:59] up to date instructions about where to go, where to
[55:01] find things exactly like what the developer workflow should be.
[55:05] So I'm a I'm a huge fan of skills people
[55:07] don't know that by.
[55:08] Yet yeah, they're they're amazing and I one of the
[55:10] things the other thing advantage you talked about this early
[55:13] on is like that they're loaded progressively right.
[55:15] So one of the things we're all struggling with right
[55:18] now is that the context window is a problem.
[55:20] And honestly I feel like we need a better solution
[55:22] than a context window in general at some point in
[55:25] the future.
[55:25] This is like a really rudimentary primitive for interacting with
[55:29] LLMS.
[55:29] But the problem is that the context window fills up
[55:33] really fast and MCP has to specify all the tools.
[55:36] So if you didn't know, skills are progressively loaded.
[55:39] So what the model gets is like the skill name
[55:42] and a brief description.
[55:43] And then from that it then will load in the
[55:46] skill and then the skill may point to other files
[55:49] and then it will load in those files.
[55:51] And so it's just really, really economical for token usage,
[55:56] which is more important now.
[55:58] It's becoming more important every day as we are moving
[56:02] from a world of use as much AI as you
[56:05] want to to this is how many tokens you have.
[56:08] Use them wisely.
[56:09] Absolutely.
[56:10] All right.
[56:10] Well, Addy, thank you so very, very much for being
[56:13] here.
[56:14] I learned a ton.
[56:15] I was looking forward to it just for the free
[56:18] training on your skills repo, so I got some out
[56:21] of it.
[56:22] Thank you for being here.
[56:23] Thanks for having me, appreciate.
[56:24] It great.
[56:25] All right.
[56:27] Thank you so much.
[56:28] So we will be right back with our next guest
[56:31] and I guess maybe folks can come chat with you
[56:34] if they want to.
[56:35] Sure thing.
[56:35] All right.
[56:36] Thanks, folks.
[56:36] There we go.
[56:43] We have an SDK, it's available in in Node, TypeScript,
[56:47] pythongoand.net and you can use this SDK for multi turn
[56:51] conversations, tool execution, and full life cycle control.
[56:55] Now this isn't technical preview, but already we're seeing people
[56:58] build some really fun stuff with it.
[57:00] Beyond that, there's some great new features that have been
[57:03] added to the Get Hepco Pilot CLI in the last
[57:05] few weeks, including access to more models built in custom
[57:08] agents for things like explore, task plan, code review, and
[57:11] then you can use those custom agents with agent skills
[57:14] skills to have a really robust workflow.
[57:16] You can also install the CLI now by a homebrew,
[57:19] A winget, and there are install scripts.
[57:21] And if you install on any of these methods, then
[57:23] Copilot CLI will automatically update, which is great.
[57:26] But don't worry, we still have standalone releases.
[57:28] If you're needing that because of your distribution or however
[57:31] you want to get things.
[57:33] There are a bunch of new flags for using the
[57:35] Copilot CLI in scripts.
[57:36] There are improvements to the diff view.
[57:38] There's some web controls, there's lots and lots of stuff.
[57:41] So if you have not had a chance to play
[57:43] around with the GitHub copilot CLI, give it a shot.
[57:46] Especially now with the SDK.
[57:47] It's getting really good and the team is working really
[57:50] hard on adding stuff to it all the time.
[57:52] We are also actively seeking out your feedback.
[57:54] So if you have any thoughts, any wishlist items, you
[57:57] know feedback, let us know.
[57:58] You can let us know the comments of this YouTube
[58:00] video.
[58:00] Let us know you know on social media, create an
[58:03] issue or or you know post a discussion in in
[58:05] the repo.
[58:08] Hey, what's up everybody, Welcome back.
[58:10] And we are here at Build 2026, live in a
[58:14] vast open area, and I'm here with Patrick Nicoletich from
[58:21] the Copilot SDK.
[58:23] Yep, CLISDK team.
[58:24] We're we're one big team because we ride on the
[58:27] CLI server.
[58:27] So Mike is really, I can hear you better than
[58:29] I can hear myself.
[58:30] That's amazing.
[58:31] Yeah.
[58:31] So for those of y'all watching at home, we're in
[58:34] this vast.
[58:35] What do you even do?
[58:36] You know what this was before.
[58:38] No idea honestly.
[58:39] It's a beautiful building, well set up, but no idea
[58:42] what it.
[58:42] Was Yeah, it looks like it's like an airplane hanger.
[58:44] No, it's not wide enough, is it?
[58:45] Maybe a boat.
[58:46] Yeah, we're not.
[58:47] A boat yard.
[58:48] All right, so we're going to be talking about the
[58:51] Copilot SDK today, and this is something I've used extensively
[58:55] and I love it.
[58:56] This is one of these things where it's specifically for
[58:59] developers.
[58:59] Like once you see this, you'll be like, oh man,
[59:02] that's amazing.
[59:03] I can do anything.
[59:04] So take us into it.
[59:06] Yeah, how wonderful.
[59:07] Well, we just Georgia the SDK yesterday, which is fantastic
[59:10] because we've been working on it since January.
[59:13] This year.
[59:14] Just yesterday, one point O stable And so we we've
[59:17] been quietly working internally mostly to drive up quality and
[59:21] and also make sure teams across Microsoft's LinkedIn, Xbox, GitHub,
[59:24] we can all adopt the SDK and we can use
[59:27] a shared agent to carness together as much as possible.
[59:30] And so that journey internally is now going outward where
[59:33] GA, which means customers can use the SDK stable, they
[59:36] should stick it in production, we can support it in
[59:38] a wide array of environments and and we're here to
[59:41] help you put our harness and whatever solution makes sense
[59:44] for you.
[59:44] Awesome.
[59:45] So do we have any demos or anything we can
[59:47] walk through in terms of looking at how this actually
[59:49] works?
[59:50] This is one of those things where when they first
[59:52] told me about it, I was like, I don't get
[59:54] it.
[59:54] And then I actually used it and I was like,
[59:56] well, that's incredible.
[59:58] I'll show probably my favorite use case because it's very
[60:01] personal.
[60:02] Even though it's the same SDK that's rolling out in
[60:04] things like Office Excel right now and Co work and
[60:07] copilot Studio as an example.
[60:08] Oh yes.
[60:09] I can actually bring up a quick slide to just
[60:11] highlight a lot of the collaboration that's been happening across
[60:14] literally dozens of teams, hundreds of people, all the people
[60:17] on the ground building these products across these different solutions
[60:21] we've engaged with, learned how they're trying to adopt agentic
[60:24] solutions and modernize using the Copilot SDK.
[60:26] So just to clarify, when we say Copilot SDK, what
[60:29] do we mean there exactly?
[60:31] Yeah.
[60:31] So the SDK, it actually comes in six languages, which
[60:36] are right here.
[60:37] So there's the four we released at Technical Preview in
[60:40] January.
[60:41] There's the two that we've added since, which is Rust
[60:43] and Java.
[60:44] What it really is, is a thin client that sits
[60:47] over the Copilot CLI server that we've all known to
[60:50] grow and love over time.
[60:51] Yeah, and it's the same exact CLI server that's used
[60:54] in things like our GitHub Copilot app.
[60:57] We use the CLI very hands on and the SDK
[60:59] is yet another way to consume the CLI but in
[61:01] a way where you can integrate it into custom applications.
[61:04] Yeah.
[61:05] So the way that I thought about it was it
[61:07] just basically allows you to add the agent that's in
[61:10] the Copilot CLI to any application.
[61:13] And we really mean any application too.
[61:15] I mean, there's a lot of things that we're doing
[61:17] beyond just what was shipped in Georgia.
[61:19] We are rewriting our runtime in Rust so it can
[61:21] actually go in more places, more performant.
[61:24] All of those things are on the horizon and.
[61:25] Like the SDK runtime.
[61:27] The SDK runtime, yes.
[61:29] So we're making it more performant and we're optimizing it.
[61:32] It's another piece that I showed off yesterday during the
[61:35] session.
[61:35] Were we just highlight?
[61:36] Was it node originally?
[61:37] It was all Node.
[61:38] So when you were talking about loading up the CLI
[61:41] server, it's probably a hundred 120 megabytes of RAM that
[61:44] it's using up.
[61:44] We're trying to shrink it to sub 10 megabytes for
[61:47] the most part and we'll we're destined to get there
[61:49] for the most part.
[61:51] We have run some some earlier tests and things like
[61:53] that which show the difference between Node and Rust.
[61:56] And even while this was Apoc, it was a fully
[61:58] working runtime.
[62:00] I can actually show you in a different demo as
[62:02] well.
[62:03] But the main core outcome of that is that we
[62:04] just saw benefits across every performance dimension that we cared
[62:07] about.
[62:08] And then there was also the need to actually get
[62:10] our SDKS more platforms that require a native runtime.
[62:13] Right.
[62:13] Got it.
[62:14] Interesting.
[62:15] That's phenomenal.
[62:17] I see Rust popping up in so many places.
[62:19] It is.
[62:19] It is.
[62:20] It's a very good language for for.
[62:21] Agents the other day I was building some of the
[62:24] Co pilots CLI and I said how should I build
[62:26] this and it was like I would do it and
[62:28] go and I was like but I don't know go
[62:30] and then I remembered that is irrelevant the agent knows
[62:34] go and so I just I built the thing in
[62:36] go because I could.
[62:37] Yeah, absolutely.
[62:38] And and I think that's what's really amazing is it's
[62:40] pushing the teams inside of GitHub to think beyond the
[62:43] boundaries of their experience.
[62:45] Fundamentally just the environment has changed and we recognize that
[62:48] probably six to seven months ago and we've been really
[62:50] internalizing what that means for ourselves.
[62:53] And so when you look at the the Copilot CLI
[62:55] and SDK team, we fully embraced very large experiments.
[62:59] We fully embrace seeing what the technology can do.
[63:03] It's one of the luxuries that we have at GitHub
[63:05] to just burn a ton of tokens to really see
[63:07] what the limits are of the models themselves.
[63:09] That's very good, the job.
[63:10] Honestly, it's should start as a line item in the
[63:12] modern era because it is the IT is the biggest
[63:14] cheat code in general.
[63:15] It doesn't really come like health insurance, right?
[63:17] Like you have How good is your health insurance?
[63:19] But more importantly.
[63:20] How many credits are you?
[63:20] Giving how many tokens do I get?
[63:22] Exactly.
[63:22] Exactly, mainly because you can just shorten the gap between
[63:26] an idea to outcome, yeah.
[63:27] And and when you don't feel as token constrained, you
[63:30] can go test those outcomes much.
[63:31] I hate it, man.
[63:32] It's, yeah, It's really, really bad when you have to.
[63:34] Every decision you make is based on like, well, should
[63:37] I spend these tokens in this way?
[63:39] Do I really want to use Opus here?
[63:40] And it really constrains your productivity.
[63:43] Oh, absolutely.
[63:43] Do you have any demos that we can see on?
[63:45] The I'll, I'll show you one of the when we
[63:47] ship the technical preview.
[63:49] It was a technical preview, so we didn't expect it
[63:51] to go into like production grade applications at the time.
[63:54] So we saw a lot of personal assistants and I
[63:57] use one of my own of course being that I
[63:59] work on the SDK.
[64:00] I made one that was one of the first things
[64:01] I.
[64:01] Did multiple flavors.
[64:02] But I actually have really just been loving my current
[64:05] form because I'm a PM and one of the things
[64:07] that's really essential to being a great PM is you
[64:10] have to write all the time.
[64:11] And so I think coding tools are awesome.
[64:14] They're wonderful.
[64:15] But really like this desire for a great way to
[64:17] write deploy agents to support me in that action is
[64:20] where I'm currently at in my trajectory and what I'm
[64:23] really obsessed about day.
[64:24] To day, can you show it?
[64:25] To us, yeah, absolutely.
[64:26] Because was this, did you use this?
[64:27] Because we were working on something together and you did
[64:30] a write up and you sent it over to me
[64:31] and I was like, I did a one page here.
[64:33] I read through.
[64:33] I was like, that's really good you used your agent
[64:35] for.
[64:36] That exactly.
[64:36] Yeah.
[64:37] I call it whim because I want everything to be
[64:39] at a whim, which is why, you know, for example,
[64:41] I can hotkey it anywhere I am in the system.
[64:44] You can leave it up, you can pin it, you
[64:46] can resize it.
[64:47] But really the core of it is that when you
[64:49] are thinking about being APM and you're putting things on
[64:51] schedules and you're trying to really encapsulate the job that
[64:54] you're doing and then transform it into something that's repeatable.
[64:58] I only really need kind of spaces to rights.
[65:01] I need little workers and I need skills.
[65:03] If I have those three things you can accomplish, sadly
[65:07] enough, a whole range of PM responsibilities.
[65:10] And I find that it's more fun than ever to
[65:13] Ida to think about the right thing to do it.
[65:15] And so it's all canvas based.
[65:16] What I mean by that, similar to the UC in
[65:19] the GitHub copilot app where we're exploring ways to render
[65:22] content to make it interactive.
[65:24] This is the inverted version of that, meaning you start
[65:26] with the writing canvas and you spawn all your agents
[65:29] out of it directly.
[65:31] And So what I love about this is wait.
[65:33] Wait, wait, what?
[65:34] You're going to write up the the file 1st and
[65:36] then.
[65:37] Oh, it's.
[65:38] It's simpler than that.
[65:39] You're not going.
[65:40] To write anything.
[65:41] I will.
[65:42] I'll actually just highlight first that there's a variety of
[65:45] agent personas in my Win app which allow me to
[65:48] one run my agents in ephemeral cloud environments.
[65:51] I can now spawn an infinite number of workers, not
[65:54] worry about my local machine.
[65:56] I can close my laptop, I can pick up and
[65:58] resume whatever I want.
[65:59] That's one version of an agent I can use.
[66:01] There's my development agents where just by calling into action,
[66:04] it will code for me.
[66:05] It will develop, it will check out repos, it will
[66:07] modify, it will do all of those things you would
[66:09] expect.
[66:10] And then I have my editor, which is my, my
[66:12] main agent that I use because we're always working on
[66:14] the contents.
[66:15] And then the last piece is my sandbox.
[66:17] If I want to run something local, it's a bit
[66:18] dangerous.
[66:19] I can deploy this agent, then I can let it
[66:21] run it in a nice secure MXC sandbox that we
[66:23] shipped yesterday as well.
[66:25] But when it comes down to the action, right?
[66:27] There's intent and then there's the outcome.
[66:29] And normally there's a whole lot of steps in between.
[66:32] And in reality, what I like to do is shorten
[66:34] that as much as possible.
[66:36] And so when I think of things I need to
[66:39] do today, I just start building a list like most
[66:42] humans do.
[66:43] And I'm like well I need to triage copilot actually
[66:47] spell correct, copilot SDK get up issues and then I
[66:52] can just say editor deploy.
[66:55] You're tagging the agent.
[66:56] There, I've actually put it already in motion and I
[66:58] don't have to think about it any more than that.
[67:00] And the way that it actually shows up is you'll
[67:02] see it present in the document, You'll see that the
[67:05] session has been fired open right here.
[67:08] I can actually drill down into any of the chat.
[67:11] So as I deploy the agents across my canvas, I
[67:13] can actually check on what they're doing.
[67:15] If I care about what's happening across canvases, I can
[67:18] get that view as well.
[67:20] You get similar controls.
[67:21] So if I want to turn these into sessions I
[67:23] can remote into, I can just click a little button,
[67:26] it'll give me the remote link.
[67:27] I can dive into it from my phone so I
[67:29] can pick up on the go.
[67:31] You have, of course, things like Yolo mode because, you
[67:33] know, I like to run unconstrained.
[67:34] Yeah.
[67:35] It's asking for approvals right there.
[67:36] And you can continue to drill down a little bit
[67:38] deeper.
[67:38] So this idea of multi agent orchestration, not really feeling
[67:42] like multi agent orchestration is where I think in general
[67:45] we're all headed because there's a lot of cognitive load.
[67:48] We're all.
[67:49] Feeling.
[67:49] Oh man.
[67:49] And so when I think about the way I engage
[67:52] with people, how do I weave agents in to feel
[67:54] as normal and as common as how I communicate?
[67:57] With people.
[67:58] So I had the same thought where it's like, I
[68:01] don't want a coding agent, I just want an agent
[68:04] that's everywhere that I am.
[68:06] So if I'm in Outlook, it's the same agent.
[68:08] It knows I've been.
[68:09] I was chatting with it about code, now I'm chatting
[68:11] with it about e-mail, right?
[68:12] We don't need an agent just for code exactly.
[68:15] But this is interesting, I've never seen a modality here
[68:18] that's not a chat.
[68:19] And the nice, well, the nice thing is you can
[68:21] close these things out.
[68:22] And I, I rarely go into the chats anymore in
[68:24] this thing to begin with because the output actually ends
[68:26] up back on the canvas.
[68:27] So I'm communicating through the canvas with the agents themselves
[68:30] or they're off doing work and they'll open APR, whatever
[68:33] it might be.
[68:34] And if I open up my Wim workspace, you'll actually
[68:36] see me working on Wim in Wim where I'm like
[68:38] giving it bug fixes, sending it off, it's opening PRS,
[68:41] and it's just updating my canvas.
[68:43] I think what's really beautiful about that is the record
[68:46] itself is being built as I'm doing the work.
[68:49] And so at any point in time I can reflect
[68:51] back on that, I can roll that up and I've
[68:53] already encapsulated what I've been doing and how it's been.
[68:56] Done.
[68:56] Man, that's super interesting.
[68:57] There's so much like thing that's so irritating right now
[69:01] with AI is there's so many people talking about they're
[69:03] like, I have I've got open claw and it's running
[69:06] everything in my life and you're like, well, what's it
[69:09] doing?
[69:09] And they're like everything You're like, yeah, but what they're
[69:13] like all of it, yeah.
[69:16] But then you're like, but I don't know what all
[69:18] of it is.
[69:19] But so this is, it's, it's interesting to see like
[69:22] how you're actually using it.
[69:23] You know what I mean?
[69:24] Like, it does feel like it's snake oil sometimes to
[69:26] me where people are like, you know, yeah, I use
[69:29] it to automate my job, but they can't tell you
[69:31] what exactly it is that.
[69:33] They're doing, Yep, Yep.
[69:34] There's so much of that going around, little assemblies to
[69:37] a few other industries in that light, you know, in
[69:39] terms of making sure that we ground it in real
[69:41] use cases.
[69:42] They're easy to communicate, people understand them.
[69:44] I think that's really important because the technology is very
[69:46] generalized.
[69:47] Yeah, 100% agree.
[69:48] But often you have to build your own in order
[69:50] to realize that, and that's what the SDK allows you
[69:53] to.
[69:53] Do.
[69:53] Oh, definitely.
[69:54] And, and to build something like this, it, you know,
[69:56] used to be a tremendous amount of work.
[69:58] Now you can spend a weekend talking to your agent
[70:00] and get a whole lot of distance.
[70:02] And we continue to pack more capabilities in the SDK
[70:04] that make all of this just tremendously much easier.
[70:07] So things like remote didn't exist.
[70:09] So a few weeks ago, the ephemeral sandboxes in the
[70:12] cloud is yet again another way for you to break
[70:14] free of some of the constraints that are available to.
[70:16] Use those now.
[70:17] Yes, yes, we we shipped them.
[70:19] The cloud sandboxes are in preview, but they are available.
[70:22] They're in there, and they can be used as well
[70:24] as local sandboxes or shipped in preview today or yesterday
[70:27] as well.
[70:27] Yeah.
[70:27] So again, you know, for those watching online, those are
[70:31] here of you who are here.
[70:32] Like if you're interested in like how can an agent
[70:35] actually, how can I use AI to do something actually
[70:39] useful?
[70:39] I would absolutely start here and use the Copilot SDK
[70:43] if you have a sub, to build your own personal
[70:47] assistant to do what you need to do.
[70:49] Forget about what everybody else is doing.
[70:52] Forget about installing something that's just going to magically work
[70:54] out-of-the-box.
[70:55] Build your own that works the way that you want
[70:57] to work and start just trying to automate the simple
[70:59] things that you're doing throughout the day.
[71:02] Like for me that was the most transformative thing was
[71:04] when I built my own that works how I work.
[71:07] And I think it's like you're you're getting out a
[71:10] really interested insight, probably one of the greatest, which is
[71:13] that we all have to build intuition how the models
[71:15] perform and they behave similar to when you engage with
[71:18] a person, you're trying to understand the nuances of other
[71:21] person themselves and trust is built over time.
[71:23] And then that intuition scales across models.
[71:26] You can understand how to deploy them and leverage them.
[71:28] But it's really important to dive deep.
[71:30] And that's why I love starting with Copilot CLI.
[71:32] It's like the simplest way to go straight into a
[71:34] model, get a very pure experience.
[71:36] You can scale that up to the GitHub copilot app,
[71:39] which will give you cross session multi agent orchestration.
[71:42] But I think diving in and really understanding the parts
[71:45] and the bits using the SDK will will help you
[71:47] go an extra distance that's truly valuable.
[71:50] And there's something that's so rewarding about building your own
[71:53] agent, even though it is the CLI harness.
[71:56] But like, because mine's called Max and Max is like,
[71:59] like I have like AI have a repertoire with Max.
[72:03] I know Max, Max knows me.
[72:05] That's probably some sort of psychosis, isn't it?
[72:08] A little bit indeed to see if you need.
[72:09] Help.
[72:09] I mean, you have to lean in a little bit
[72:11] to the psychosis to embrace the, the, the capabilities that
[72:14] are available there.
[72:15] Yeah, amazing.
[72:15] OK, Patrick, one question for you.
[72:18] What's your go to model?
[72:20] What's going on behind the scenes here?
[72:21] Which one are you?
[72:22] Going GPT 5.5 these days is honestly, you've been really
[72:25] great.
[72:26] And I was somebody that sat on Sauna and Opus
[72:28] models for almost a couple years at this point.
[72:31] Some, but I find there was a nice balance, especially
[72:34] between the two classes of models.
[72:36] And so I'm a big, you know, let one model
[72:38] do the implementation, let one model verify.
[72:41] You get a nice mixture of answers and responses to
[72:43] balance it out.
[72:44] They all have different training data and inputs that help
[72:46] them decide what the right thing is to do.
[72:48] And so I I find GPT 5.5 is really great,
[72:51] especially for reviews.
[72:53] Still love Opus for generating code as well, but I've
[72:55] started to use GPT 5.5 pretty much exclusively the last
[72:58] few weeks or so.
[72:59] So I've had the exact same experience.
[73:01] I just recently posted about this, but I'm mostly on
[73:03] 55 now.
[73:04] Opus to me still use that for design and creative
[73:07] thinking, but for implementation and actually doing executing, I'm a
[73:11] 5 five, yeah.
[73:12] Absolutely.
[73:13] Well, Patrick, thanks to you so much for being here.
[73:15] You've got any other sessions?
[73:16] I do.
[73:17] It's 4:00 to 5:00.
[73:18] We're giving a Copilot SDK session where we'll talk a
[73:20] little bit more about the great work we've been doing
[73:22] across Microsoft.
[73:23] What's going on right now?
[73:24] Where?
[73:25] Is that is that here in this?
[73:26] Should be in breakout 1.
[73:27] Breakout one all right.
[73:28] So if you want to know more about the SDK
[73:30] breakout, one at.
[73:31] 4:00.
[73:32] 4:00 today.
[73:33] All right.
[73:33] Thank you.
[73:34] Perfect.
[73:34] Thank you.
[73:35] We'll be right back.
[73:53] I'm Ori.
[73:54] I'm a senior product designer at GitHub and I help
[73:58] to design the remote control sessions with Copilot CLI or
[74:02] VS Code on GitHub mobile.
[74:04] You get to start work on your computer and then
[74:07] you can remotely connect to it on your phone and
[74:11] work on the go.
[74:12] I'm Josh.
[74:13] I'm an engineer at GitHub, but I've been getting all
[74:15] of GitHub agentic offerings speaking the same language.
[74:18] One of the things that that has enabled is remote
[74:21] control.
[74:22] I'm able to stay connected and kick off new tasks
[74:25] with copilot CLI, respond to it and have it continue
[74:28] my local work.
[74:29] You can have multiple agent sessions going on at once.
[74:33] You can actually see all your sessions in one view.
[74:36] Remote control enables me to check on the progress of
[74:39] my work and direct that work while I'm not at
[74:42] my laptop.
[74:42] That frees you up to do anything you want.
[74:45] You don't have to stay close to your desk to
[74:48] stay close to the code.
[74:55] Does things when you do get operations and it purrs.
[74:59] It's like it was a cat but then some unwanted
[75:01] goblin mode.
[75:22] Oh, are we on?
[75:24] Are we live?
[75:24] Oh, thank you, Tom.
[75:27] What's up, everyone?
[75:29] Welcome back.
[75:29] Welcome back folks, folks who are here and at home.
[75:33] I'm here with Evan Boyle from the Copilot CLI team
[75:37] and a huge fan of fish fishing.
[75:40] All things fish.
[75:41] All things fish, Evan, what is it that you say
[75:46] you do here at at GitHub?
[75:48] GitHub, yeah.
[75:49] At GitHub, Yeah, at GitHub.
[75:50] So I'm an engineering manager, but I'm a very hands
[75:56] on guy.
[75:56] I don't think that you can build dev tools and
[75:59] be a manager in that space without writing a lot
[76:01] of code yourself and like living the life of your
[76:03] users.
[76:04] So my team builds a bunch of things.
[76:06] We build the core agent loop that powers the Copilot
[76:09] CLI and the SDK and increasingly all of the agents
[76:13] across Microsoft and GitHub.
[76:14] So that agent loop and runtime the SDK, the copilot
[76:19] CLI and newly the GitHub app.
[76:22] Yes, which we saw on the keynote.
[76:23] Which we saw on the keynote.
[76:24] And by the way, Evan was with us two nights
[76:27] three nights ago when we rewrote the whole demo from
[76:29] scratch.
[76:30] At GitHub.
[76:31] HQ until midnight, until midnight.
[76:34] Interesting times.
[76:35] All right, so we're going to look at the copilot
[76:38] CLI today.
[76:38] I was actually on the bus coming over here this
[76:41] morning and I was talking with someone and the person
[76:43] behind me tapping on the shoulder and said what is
[76:46] the copilot CLI?
[76:47] I keep hearing about that, but what is it?
[76:49] So maybe we should just start there.
[76:51] Yeah, we can totally start there.
[76:54] So I am in the Copilot CLI right now.
[76:58] You know, I just did a task a talk with
[77:01] Cassidy and we created this repo here.
[77:07] And so this repo, basically we asked people to open
[77:11] issues to decide what we were going to build live
[77:15] during the session.
[77:18] Pretty fun stuff.
[77:20] Got 60 issues open.
[77:22] And then with the Copilot CLI, what we did was
[77:25] we asked it, hey, go read through all of these
[77:28] issues.
[77:29] If you shift tab here inside of the CLI or
[77:32] sorry tab, you can see all of the open issues.
[77:35] You can see all of your open PRS.
[77:37] And so we had the Copilot CLI go analyze all
[77:40] of those issues, pick out the best one, and then
[77:44] we went to go implement it.
[77:45] So what the audience asked for was this little octocad
[77:49] that was going to sit here and, you know, shower
[77:52] you with emojis when you did git operations.
[77:55] Just any git operation.
[77:56] Any git operation.
[77:58] So we fed that issue into plan mode, created a
[78:02] plan and then let it go on autopilot.
[78:05] And so it built this initial version.
[78:07] And then we asked, hey, audience, go open up a
[78:10] bunch of requests for ways that we could improve this.
[78:14] And somebody asked for goblin mode, which is what you
[78:17] could see here right now.
[78:20] Is that because GPT was talking about goblins a whole
[78:22] bunch for a while?
[78:24] We were.
[78:24] They fixed that.
[78:25] They did.
[78:25] They did fix it.
[78:26] They put out a really interesting paper about it, actually,
[78:28] yeah.
[78:29] I would like to know why that is.
[78:31] And so we, we actually just got to the point
[78:34] that we had.
[78:35] So we we picked out five features.
[78:37] Goblin mode sounds, eyes following the cursor.
[78:42] Visibility I found the cursor.
[78:45] Does that work?
[78:46] Oh, that, that one didn't make the cut, I guess.
[78:48] And then slower animations.
[78:50] OK, so I think what I'm going to ask right
[78:54] now is instead of opening a PR, I'll I'll do
[78:58] this with voice too.
[78:59] So I'll hold down the spacebar, see this little voice
[79:02] meter pop up.
[79:03] All right, instead of opening a PR, I want you
[79:06] to just push directly to mate.
[79:09] That sounds like like a great idea.
[79:12] This is my repo, it's not my personal repo, so
[79:15] you know.
[79:16] Force push to main.
[79:17] I don't do that on our team repos, but for
[79:20] this one it's OK and I'll turn up the volume
[79:24] here and see if we can see if we can
[79:26] hear.
[79:27] So the goblin here, the goblin tap purse every couple
[79:31] of seconds and then it meows when you commit shipped
[79:35] it.
[79:36] There we go.
[79:36] Nice.
[79:37] So this was an electron app that we created with
[79:40] the audience in the course of, I don't know, 2025
[79:43] minutes.
[79:43] Very nice $1600 worth of tokens.
[79:46] I don't think that many, no, only I don't know,
[79:48] this is like 5 or 10 bucks I think something
[79:51] like that.
[79:51] That was all.
[79:52] That was all it.
[79:52] Was I?
[79:53] I got to do the math, but 1300 credits, I
[79:55] don't know what that converts to.
[79:56] OK, so on the CLI I have a question.
[79:59] Yeah, Why?
[80:01] Why am I in a terminal and not in an
[80:03] editor of an?
[80:04] Why do I need this exactly?
[80:06] I think, I think the nice.
[80:07] Oh, there we go.
[80:08] It's making noises.
[80:11] I don't know if you heard that.
[80:12] No, I can't hear.
[80:14] So yeah, I mean, I I think the terminal for
[80:17] me brings an awful lot of focus.
[80:19] There's no distractions.
[80:21] I can see the logs.
[80:22] Also, it's kind of fun to feel like a little
[80:24] bit of a hacker, right?
[80:25] Yeah.
[80:25] So it's easy to Multiplex these things.
[80:28] I can use team UX and run multiple instances.
[80:31] If I have a big monitor, I can cascade multiple
[80:33] of these across my screen.
[80:35] And we've even turned the CLI into like, if you
[80:38] like working in parallel, but you're working on a small
[80:41] monitor like this, We now have this slash new command.
[80:44] So slash new lets me start a new conversation.
[80:48] But, and so I could say like, go do some
[80:51] research on open issues that we might want to address,
[80:56] right?
[80:58] But then I can go to slash sessions and I
[81:01] can see all of my other running sessions in this
[81:05] instance.
[81:06] So the Copilot CLI can now manage multiple sessions in
[81:09] parallel, have all of those tasks running and you can
[81:13] switch between them.
[81:14] So if I go back to this original session, all
[81:16] right, it's live now.
[81:17] I'm going to ask it, Hey, go close out those
[81:22] issues we implemented, add a friendly message and a thank
[81:28] you to our community members.
[81:32] Cool.
[81:33] So wait a minute.
[81:34] So when you do new slash new IT starts a
[81:36] new session, but it the other one it's sort of.
[81:39] Backgrounds it, yeah, it backgrounded that existing session and I
[81:42] can see everything that's going on in parallel and switch
[81:45] between.
[81:46] So hey, I trust it to go and close out
[81:48] these issues.
[81:49] I'm now going to go back to my research session,
[81:51] right?
[81:52] So research open issues, crawling through the issues reading my
[81:56] readme right?
[82:00] But it's it's so it's super interesting to me though,
[82:02] like why are we back in terminals?
[82:05] Why are we here?
[82:06] Why are we here?
[82:07] I agree with you.
[82:07] I don't know if you, how many people have used
[82:10] like Clis here recently to do coding and development.
[82:15] See, like look, there's not a lot of folks look
[82:17] like a couple hands here, but how many people are
[82:19] using editors and agents?
[82:21] So if you haven't tried using a terminal and a
[82:24] CLI cuz you're like, why would I do that?
[82:27] It's because it's a magical experience and I can't put
[82:30] my finger on why that is.
[82:32] My preferred method of working is with Visual Studio Code
[82:35] open and the terminal as an editor tab, because you
[82:39] can do that and I'll have multiple of them.
[82:41] That's how I work now, yeah.
[82:44] The terminal just feels so good and I don't know
[82:46] what it is.
[82:47] It it does.
[82:48] And I think that, you know, for a lot of
[82:50] people who haven't tried terminal coding agents, like for me,
[82:53] I've never been a Vim or an Emacs person.
[82:56] And I know that when you think terminal, you think,
[82:59] Oh my, Oh my gosh, I've got to learn a
[83:01] ton of hot keys.
[83:02] It's going to be complicated.
[83:03] I'm going to be stumbling over myself.
[83:04] That's.
[83:04] Exactly what I think, that's what people.
[83:06] Think right, But you know, TUI applications have gotten so
[83:10] rich that they almost feel like a web app, right?
[83:14] They, they, they feel much richer than they used to.
[83:17] Like this.
[83:18] You know this, this view that we have up here
[83:21] with issues with pull requests, with Gist back to my
[83:24] session, like it begins to feel more and more like
[83:27] a full application, almost like your ID.
[83:29] And I've noticed that there's actually less keyboard shortcuts in
[83:33] the terminal than there are in Visual Studio Code.
[83:35] That's true, which is weird.
[83:36] And also someone the other day was like, I want
[83:38] to click on the tabs at the top.
[83:40] And I thought, well, you can't do that, It's a
[83:42] terminal.
[83:43] And then somebody was like, no, we already let you
[83:45] click on things, we can make that work.
[83:47] We can't.
[83:47] Oh, and it does.
[83:48] Work.
[83:48] It does work.
[83:49] They ship it.
[83:49] We.
[83:49] Implemented it.
[83:51] Amazing.
[83:52] So this is, yeah, this is one of the one
[83:55] of the things that's really funny about working on a
[83:58] team that is so heavily bought into AI.
[84:01] You can imagine the team at GitHub that is building
[84:04] the AI tools is the most AI forward, utilizes AI
[84:07] the most.
[84:08] And so it's been a couple of months since I
[84:12] did a you know, I think back in March I
[84:15] did a survey and with 10 people we were shipping
[84:19] 2000 PRS a month across app, CLISDK, agent loop.
[84:23] And so every time I come in contact with the
[84:26] CLI code base, I have to assume it's completely different
[84:30] than a week ago.
[84:31] If I if I built the feature a month ago
[84:33] and I come back to add something to it, I
[84:35] have to rebuild my view of the world because the
[84:38] velocity of this code is so high.
[84:40] Yeah, moves so fast.
[84:41] Yeah.
[84:42] I was talking to Moss, who built the the new
[84:44] UI here, and I was asking, I was like, how
[84:46] much agentic coding do you actually do on the team?
[84:49] He's like, well, I don't know what everybody else does,
[84:51] but I do a lot.
[84:52] A lot?
[84:52] Yep.
[84:53] A lot.
[84:53] Yeah, Very.
[84:54] Interesting.
[84:54] All right, let's talk about some of the other features
[84:56] of the CLI here.
[84:57] What are some some of your favorites because a lot
[85:00] of them are behind slash commands.
[85:03] Yes, a lot of them are behind slash commands.
[85:05] So I'm going to go ahead and start a new
[85:09] session and we'll go ahead and do it in here.
[85:13] So bump up the font.
[85:17] I'm going to run Copilot.
[85:18] I'm going to do it in Yolo mode.
[85:19] I don't always do this, but for the sake of
[85:22] demos, I want all of this to kind of stream
[85:24] while I'm working.
[85:26] So one thing that I really like to do is
[85:29] slash review.
[85:31] So this is multi model code review.
[85:33] And So what I'm going to say I'm going to
[85:35] dictate here.
[85:38] I'd like for you to do multi model code review
[85:45] with Sonet 46, GPT 55 and Gemini.
[85:50] I'd like you to then compare the results and only
[85:54] feed me issues where the models are in agreeance.
[86:00] Cool.
[86:02] Is agreeance a word?
[86:03] I think I just made it up Agreement.
[86:06] I bet.
[86:06] I bet it understands it anyway.
[86:08] It'll understand it.
[86:09] That's the beauty of, you know, LLMS is they, they,
[86:11] they pick up what you're putting down.
[86:14] And they don't care about spelling at all.
[86:16] Like I frequently watch people type and they'll be like,
[86:18] oh, I misspelled that, like backspace you.
[86:20] It doesn't.
[86:21] Matter.
[86:21] I just bash the keyboard.
[86:22] Just for sure, just send it.
[86:23] It knows.
[86:25] And so let's see, oh, this is, this is really
[86:30] cool.
[86:31] So this is what's called elicitation when the CLI detects
[86:35] that there's some ambiguity, right?
[86:38] Like I ran this slash review command and it, it's
[86:41] expecting, hey, I have commits that are ahead of my
[86:45] base Rep, right?
[86:46] Or I have unstaged or uncommitted changes.
[86:49] In this case, I'm just on main.
[86:51] So it's asking me what I want to do.
[86:53] Like, hey, do you want to review the whole repo
[86:56] specific branch, just the latest commit, merge.
[86:59] I'm going to say let's just go ahead and do
[87:00] the latest commit and merge.
[87:03] Oh.
[87:04] Oh, then you get a free text.
[87:05] And then it says branch name or commit range if
[87:08] applicable.
[87:09] I'm going to just say you figure it out.
[87:13] Oh, and once you.
[87:13] Best judgment and once you just I don't have the
[87:15] commit number off the top of my head unfortunately.
[87:18] It gave you something there.
[87:19] It gave you like a look.
[87:21] I thought I saw a hash.
[87:23] Maybe it did, maybe it did, I just missed it.
[87:25] Yeah, we're moving.
[87:26] We're flowing here.
[87:27] I have time to read everything, the agent says.
[87:29] They write walls of text, man.
[87:31] Yeah.
[87:32] I'm too busy.
[87:33] I'm building things.
[87:34] Yeah, it's interesting that you say that because that's actually
[87:38] something that we've experimented with in the GitHub app and
[87:42] probably we will bring into the CLI is sort of
[87:44] different being able to configure different levels of brevity.
[87:48] Some people really love being able to see in complete
[87:51] detail the entire log.
[87:53] Some people just want the final output.
[87:54] Yeah, say it in as few words as possible.
[87:57] I don't know if you all notice if you have
[88:00] people who send you or you read things that a
[88:03] is have generated.
[88:04] They're like this long man.
[88:06] Like people send me an e-mail that looks like they
[88:08] spent an hour on it that was written by an
[88:10] agent.
[88:11] You're wasting my time.
[88:12] Boil it down to one sentence.
[88:13] This is this is the beauty though of these, you
[88:17] know, these coding agents, the CLI is there's lots of
[88:20] ways to configure them, skills, hooks, custom instructions.
[88:25] So I'm actually just going to go ahead and like
[88:27] implement Burke's suggestion here.
[88:28] I'll hold down the space bar.
[88:30] I'd like for you to add to my custom instructions
[88:34] that I want you to be as brief and concise
[88:36] and to the point as possible.
[88:38] Unless I ask otherwise like one sentence.
[88:41] Make it easy for me to read and scan.
[88:45] I'm just going to hit enter here.
[88:46] This is going to update my copilot instructions file and
[88:49] generate a new instruction where now the agent is going
[88:52] to be a little bit more tailored to me and
[88:55] how I want to work inside of this repo.
[88:57] I'm doing this in the repo because I own this,
[88:59] but if I was working in a team I would
[89:02] put this in my dock copilot folder where I could
[89:04] have personal instructions that you know are specific to how
[89:08] I want to work with the agent.
[89:09] Now I notice you keep using voice mode.
[89:12] Yeah.
[89:12] Is that a plug in?
[89:13] Is that something like?
[89:14] How does that work?
[89:15] Yeah, absolutely.
[89:17] So you can toggle voice mode on and off here
[89:20] with the slash voice command.
[89:23] And if you show models, we have an English and
[89:28] a Spanish Nematron model.
[89:30] There are other models that are available to download as
[89:33] well.
[89:33] These are local models.
[89:34] These are local models.
[89:35] I believe they're from NVIDIA, but you have Nematron.
[89:38] You have to check me on that.
[89:40] And the nice thing about these models is that they
[89:42] stream.
[89:42] So the first time you set this up, you have
[89:45] a, you know, a download.
[89:46] Let me see if I can, you know, look back.
[89:48] So it's a big model, it's 700 megs.
[89:51] But the nice thing about that, all of it running
[89:53] locally and then, you know, I don't have to, you
[89:55] know, hit the cloud or you know, hit an inference
[89:58] provider for this.
[90:00] Yeah, that's super nice.
[90:01] So a question, how many of y'all are using voice
[90:04] mode and talking to agents?
[90:06] Is anybody people OK have people type to your agents.
[90:11] So most people why are you are y'all uncomfortable talking
[90:15] to just like speaking out loud because then it sounds
[90:18] like you're just talking to yourself.
[90:20] Is that what it is?
[90:21] Because that's how I feel.
[90:22] I feel uncomfortable when I do it different part of
[90:27] your brain.
[90:28] So like when you when you're typing, you're thinking and
[90:32] when you're speaking.
[90:33] I don't know about y'all, but when I speak I
[90:36] don't think obviously well.
[90:38] I would say for for me, Scott Hanselman is the
[90:41] one who turned me on to voice mode.
[90:43] He does everything in voice mode.
[90:44] He does.
[90:44] Everything in voice mode.
[90:45] And when we were doing some early dogfooding of the
[90:48] GitHub app and we had these weekly one hour Scott
[90:51] dogfooding sessions where we would watch him use the app
[90:55] and he was using handy at the time.
[90:58] And at first when I saw it, I didn't really
[91:00] get it.
[91:00] And I, I tried it once or twice and, you
[91:03] know, it didn't really stick.
[91:06] But eventually I realized I'm kind of a person who
[91:09] likes to pace around my office while I think, if
[91:12] that makes sense.
[91:13] So I have like a little USB remote that's hooked
[91:16] up to that shortcut.
[91:18] And I pace around my office with my headphones on
[91:21] and I ramble at my agent.
[91:23] Like, it's like, like dictation, like dictation, like I'm walking
[91:27] around in circles, thinking, staring at the sky.
[91:30] It works for me.
[91:31] In 1965, we're all dictating to recorders again.
[91:34] Exactly.
[91:35] All right, Evan, thank you.
[91:37] Do you have any other sessions here?
[91:38] Any What should people do next?
[91:40] I do.
[91:41] I have AI.
[91:42] Have a 4:00 PM session with Mario about the GitHub
[91:45] app this afternoon.
[91:46] Already did a session with Cassidy this morning that was
[91:49] recorded about the Copilot CLI going in depth on the
[91:52] features.
[91:53] The session with Mario on the GitHub app is at
[91:56] 4:00 PM today.
[91:57] Encourage you all to attend that if you want to
[92:00] learn more about how the team and I use the
[92:03] GitHub app and use agent workflows to guard our focus
[92:06] and focus on, you know, output rather than activity with
[92:10] coding.
[92:10] Aids awesome and for those y'all who listen y'all and
[92:13] for those online, if you're skeptical about these clis, these
[92:17] two is then I I definitely want you more than
[92:20] anyone else to go install the copilot CLI and just
[92:23] try it just try it.
[92:24] OK, that's my challenge to you, because I understand the
[92:26] skepticism.
[92:27] I was too.
[92:29] Once you do it, you're like wow, that feels amazing.
[92:32] So just give it a try.
[92:33] See what you think.
[92:34] All you got to do is Google the copilot CLI.
[92:36] You'll find it.
[92:37] You can't miss it.
[92:38] Evan, thanks for being here.
[92:39] Thanks so much Bert.
[92:40] We'll be right back.
[92:40] Thanks y'all, see you in a minute.
[92:45] So I'm using Rust and so here let me walk
[92:47] you through what I did so.
[92:48] First of all, I took a photo of the challenge
[92:51] with my phone.
[92:52] I did the same and air dropped it to my
[92:53] machine.
[92:54] The same thing and then.
[92:56] So first I had to ask is my machine can
[92:58] you figure to run rust?
[92:59] Can I build a website with Rust?
[93:01] Because I don't know.
[93:03] So.
[93:04] So it's like.
[93:04] Rust isn't installed in your machine.
[93:06] Here's how you can build.
[93:07] I was like cool, so yes, install what I need
[93:09] and I'm going to get a prompt.
[93:10] Ready for you.
[93:11] And then I, this is my prompt.
[93:13] We're doing the Vibe coding contest.
[93:14] I'm going to give you the rules.
[93:15] And then I said boost the prompt.
[93:17] It's basically like give add more fidelity to the instructions
[93:21] that they provided us and then create a detailed plan.
[93:24] So then I went into plan mode and then I
[93:26] said it would be better if we could deploy it,
[93:27] but you have to get it running locally.
[93:29] That's P0.
[93:30] If you have any questions, let me know.
[93:31] So then it goes through and I don't know if
[93:34] it's going to show the questions here, but one of
[93:38] the first things it said to me was it goes,
[93:41] hey, you don't need.
[93:42] Rust.
[93:42] To do this you just need.
[93:43] HTML and JavaScript.
[93:44] That was like the first thing the agent said it.
[93:46] Was like do you?
[93:46] Still want to use Rust.
[93:48] So then I I switched to my custom agent which
[93:50] is the beast mode.
[93:51] Rebooted.
[93:52] So my custom agent here does a couple things that
[93:55] are important.
[93:56] So first of all, it forces the thing to use
[93:58] context 7.
[93:59] It forces it to run things in sub agents.
[94:01] I know I'm not supposed to use sub agents.
[94:03] I don't care.
[94:03] I'm doing it anyway.
[94:04] I told you I wasn't going to listen.
[94:06] And then it, it lays out some mandatory coding principles.
[94:09] Now then the next thing that it does is I
[94:12] switched into autopilot, which I think is experimental in the
[94:15] CLI, but it's essentially the copilot CLI version of a
[94:18] Ralph loop.
[94:20] And then it just keeps going.
[94:22] It keeps going, it keeps going.
[94:25] It's trying to test.
[94:26] It's opening the local host, taking screenshots, taking screenshots.
[94:31] And then we get down here to the bottom and
[94:33] it's like, OK, cool, I'm done.
[94:34] And I was like, well, how do I run it?
[94:36] And so that took me a minute to figure out
[94:38] how to run it, but it's up.
[94:41] Dude, there's.
[94:42] A couple of things.
[94:43] So first of all, I use the front end design
[94:45] skill from skills dot SH, which I'll just have installed
[94:48] like it's just there.
[94:49] And Theo told me that Opus 46 with the front
[94:52] end design skill is like, that's the most goaded way
[94:55] to do design.
[94:56] So that's what I did.
[94:57] And then the.
[94:58] Agent browser skill is what it used for to test,
[95:02] Yeah, to make sure that it actually ran.
[95:05] Because one of the problems with with agents is that
[95:08] they can't close the loop.
[95:10] You're the one who pointed this out to me.
[95:11] And So what they do is they're like, cool, I
[95:13] did it.
[95:13] And then you go and you're like, you didn't.
[95:15] Do it.
[95:15] It doesn't work at all, so it has to be
[95:17] able to close the loop.
[95:18] So the agent browser skill enables it to use a
[95:20] browser and do that.
[95:24] I don't know if this actually works though.
[95:25] Let's hold on.
[95:27] Can we give people kudos?
[95:28] Let's give John some kudos.
[95:30] Hey, look at that animation.
[95:31] Look at that animation.
[95:33] Wait, does it persist if I reload?
[95:35] The page it does.
[95:37] Am I done?
[95:37] Did that win?
[95:41] Hello my friends, welcome back.
[95:42] We're here at Microsoft Build and I'm super excited to
[95:44] be here with Julia.
[95:45] How you doing my friend?
[95:46] I'm doing fantastic.
[95:47] Thank you the.
[95:48] Thing I love about Build is we get to talk
[95:50] about the cool stuff behind the scenes and we get
[95:53] to ask all sorts of questions.
[95:54] We have a live audience here as well, so they
[95:56] might ask some questions.
[95:57] Why don't you tell us who you are and what
[95:59] you do?
[95:59] Cool.
[96:00] And my name is Julia.
[96:01] I'm a product manager on the VS Code team.
[96:04] I've joined the team last year in August and yeah,
[96:07] it's been a fun ride.
[96:09] But The thing is, is that you just don't work
[96:12] on VS Code.
[96:14] Let me paint the stage.
[96:15] We're in a gentic coating era.
[96:17] Like for me, it started last December.
[96:19] And we, we kind of go and we pick the
[96:21] model and we just happily just go along and pick
[96:23] a model.
[96:24] And it seems super easy.
[96:26] But that's what you work on.
[96:27] Yeah, it's not as easy as it sounds, right?
[96:29] Yeah, so I thought the same thing.
[96:31] Funny that you say this.
[96:32] I'm like, you know, I'm in VS Code, I'm using
[96:34] the chat extension.
[96:35] And from time to time like a new model shows
[96:37] up and I'm like, oh great, I'm just going to
[96:39] use the new one.
[96:41] And now since I joined the VS Code team, I'm
[96:43] in charge of or I'm helping the team.
[96:45] Not in charge of, but I'm helping the team.
[96:47] It's very collaborative and what all goes into releasing these
[96:51] new models in our VS Code extension.
[96:53] And it is a ride, I can tell you that
[96:56] much.
[96:57] It's not as easy as, oh, we get a new
[97:00] model endpoint and boom, here it is.
[97:03] And it just works.
[97:04] There are so many things and pieces that go into
[97:07] this from us working very closely with the model providers
[97:10] and like getting their insights and what the new model
[97:13] is all about.
[97:14] So yeah, there are a lot of things that go
[97:16] above and beyond here.
[97:17] So I'm just picking the thing and I'm like, these
[97:19] things are all compatible or are the same.
[97:21] Can you give us a sense for how not the
[97:23] same?
[97:23] These are all like all very special flowers, whether they're
[97:26] from a certain company or from a certain type.
[97:29] Tell us about that.
[97:30] Yeah.
[97:30] So I think the biggest aha moment for me was
[97:34] even within the same model family, let's pick the GBD
[97:39] 1 and GBD 5.4 to 5.5.
[97:41] Even within this model family, it kind of had its
[97:45] own new personality and we had to adjust our coding
[97:48] harness to it.
[97:49] And that's why we work very closely with them because
[97:52] for us the models are also a black box.
[97:54] And so we are getting their insights like what has
[97:56] their research team on worked on and making the new
[97:59] model endpoint better.
[98:00] So truly it's like every new time you're getting a
[98:03] new model, it has its own personality.
[98:06] Even more across different model providers like Anthropic and Open
[98:10] AI, these even differ even more.
[98:12] But even within the same model family or like model
[98:16] provider and section, it's all very very very different.
[98:20] So we talk about models even from 5.4 to 5,
[98:22] we talk about personality or characteristics, trait.
[98:24] And you, we talked about the harness a little bit
[98:26] too.
[98:27] Do you have to make changes to the harness for
[98:30] every model?
[98:31] And if people don't know what a harness is, can
[98:32] you help them understand what a harness is and how
[98:35] new models make it so you have to change?
[98:36] Them totally, yeah.
[98:37] So I brought a little bit of.
[98:39] A.
[98:39] Let's go to the screen.
[98:40] Here load chart diagram OK, and just to give a
[98:43] very high level of hey, these are all of the
[98:46] steps that we have to think about and as I
[98:49] mentioned we work very closely with the model providers so
[98:52] it's very I thought it was very processy Oh yes,
[98:55] of course we learn about this new process it's very
[98:59] casual the model providers reach out to us they're like,
[99:02] hey, our research team told us we have something that
[99:06] is working and then our entire team from the copilot
[99:09] API team that gets the new model on boarded to
[99:12] to also us and client side like VS code, we
[99:15] all go in and we start working and in particular
[99:18] we work on optimizing the harness.
[99:20] So what is this harness and the harness itself is
[99:24] the system, the environment, your AI or your LLM is
[99:27] running essentially, the LLM only provides or gives back tokens,
[99:32] but how do we make changes how do we edit
[99:35] files to how do we run in terminal?
[99:37] So this is what we consider our harness and these
[99:40] are all of the changes we have to make sure
[99:43] it works once a new model comes up.
[99:45] And there are four very important things we always look
[99:48] at whenever we have a new harness.
[99:50] It's our system prompt.
[99:52] So for every model, for every different model across even
[99:55] different model providers, we have our own system prompt.
[99:59] We have different built in tools that these models can
[100:03] call.
[100:04] And even across different model providers, they tend to call
[100:08] different tools as well.
[100:09] So we have to make sure the right tool sets
[100:11] are being called.
[100:12] And then also context management, different models have different or
[100:16] work with different model context.
[100:18] And then also the agent loop and how do we
[100:21] call it?
[100:21] How do we load?
[100:22] Tool.
[100:22] So we just have to make sure it works end
[100:25] to end.
[100:25] So hold up.
[100:25] So you effectively every time you get a new model,
[100:29] every model gets its own special flower kind of situation
[100:33] in the harness.
[100:35] So that for us as we're using this, we really
[100:38] can't tell.
[100:39] Yes, correct.
[100:40] Well, actually.
[100:41] So let me switch over to my stable.
[100:44] Can you hit control plus a couple of times of?
[100:46] Course, there you go.
[100:47] All right, so this is a fun little project.
[100:51] It's called DJ Julia.
[100:52] Control Plus one more time because I'm.
[100:55] I'm old.
[100:55] There you go.
[100:55] There you go.
[100:56] And it's just like a fun little project that I
[100:59] have.
[100:59] And so right before this session, I created this prompt,
[101:03] which is, hey, for my DJ app, I want you
[101:06] to create a new music pattern for me.
[101:09] And just to kind of show you, first of all,
[101:12] everyone can go in and explore what the harness does
[101:15] whenever we make these calls to the LLM.
[101:17] So you see a bunch of styling, it does a
[101:19] bunch of different things rise, it's searched.
[101:22] So whenever you switch to our chat debug logs, you
[101:25] can really see all of the different things that is
[101:28] happening behind the scene.
[101:30] So.
[101:31] Hold on, you went really fast.
[101:32] Let's pause for a second so there is a chat
[101:34] debug log where you can see effectively every step the
[101:37] harness has taken.
[101:38] Exactly every step, every tool call, what is the system
[101:41] prompt that is being applied.
[101:43] So here, for example, let me close this real quick
[101:46] and you can see the request message is being separated
[101:50] between the system prompt.
[101:52] And you will notice, so this one is with the
[101:55] GBD 5.5 model.
[101:56] If you would do the exact same prompt with let's
[102:00] say Opus 4.6, it will have a different system message
[102:04] as well.
[102:04] That's how we optimize and we make sure every new
[102:07] model gets the latest and best together with the model
[102:10] providers that we work on because they clearly know the
[102:13] model so much better.
[102:15] And it also helps us to really optimize and make
[102:18] sure it works truly end to end as well.
[102:21] And then also if you're interested in seeing all of
[102:24] the different tools that are being called as an example,
[102:28] and the GPD model family always uses apply patch versus
[102:31] the anthropic models that you'd insert and insert files.
[102:35] So just someone nuances how different all of these models
[102:38] truly are.
[102:39] So you talk about like how different?
[102:41] Is there a way you could show us like how
[102:43] they are different?
[102:45] Sure, let's I mean let's try this here.
[102:50] So use this let's go back and for everyone to
[102:54] see.
[102:54] I use this using the GBT 5.51.
[102:57] So let's just do this with the Opus 4.71.
[103:03] I'm doing the exact same prompt.
[103:05] So first of all, also another challenge we have in
[103:09] the AI space, even using the same exact same model,
[103:13] it is nondeterministic.
[103:14] So every time you run this even within the same
[103:17] model family, you would always get a different input or
[103:20] output.
[103:21] But just to show you how much they differ even
[103:24] across within the same harness and just having the different
[103:28] model here.
[103:29] Yeah, because I mean this thing is sampling, you know,
[103:32] according to distribution of the of the models output.
[103:35] I like to say stochastic process as much as I
[103:37] can.
[103:38] It's like a $5 word, makes you sound really smart.
[103:40] I don't know what it means all the way, but
[103:41] that's what they told me.
[103:42] But effectively, this is all probabilistic, and so it's going
[103:45] to be different every time, even when they're the same
[103:47] model.
[103:47] Exactly.
[103:48] OK, So what do you got?
[103:49] So all right, if you remember what we had before
[103:52] our system prompt, let's go here.
[104:00] There you go.
[104:00] So this was the system prompt we had previously.
[104:05] Just going to close this and it does like a
[104:08] bunch of pre classification checks and stuff like that.
[104:13] So now if we go to the latest one that
[104:16] we trusted with the cloud one, you can see the
[104:20] instruction files are different with the GPD 5.5.
[104:24] We had something to make sure it inserts before because
[104:27] the GPD 5.5 ones, they are more prone to immediately
[104:31] go and run in terminal.
[104:32] And but we wanted to hey first explore a little
[104:35] bit and then go and run it because we have
[104:38] seen so much more success with these kind of system
[104:41] prompts rather than using the ones that you previously had
[104:44] with 5.4.
[104:45] And all of this was worked together with the Open
[104:48] AI folks.
[104:48] And this is this is really impressive because like betwixt
[104:52] us friends, I built my own harness in Rust and
[104:55] I started changing models assuming that it was and things
[104:59] just stopped working.
[105:01] So you effectively have to, when a model comes out,
[105:03] it's like, all right, gang, let's get together.
[105:05] Let's figure out what the right prompt is.
[105:07] Let's figure out how they do tool calls, et cetera.
[105:09] You have to do all of that work.
[105:11] Exactly, Yeah, I mean, I love that you're building your
[105:14] own harness, but one of the benefits for using our
[105:17] built in one, and there's no harm in building your
[105:19] own and of course making your own changes is we
[105:22] do work very closely with them.
[105:24] And we are truly, truly optimizing for all of the
[105:27] new checkpoints.
[105:28] And we are optimizing in a way that it will
[105:31] hopefully work for most folks out-of-the-box.
[105:33] And you don't have to worry about it because we
[105:36] know how much work that actually goes into always updating
[105:38] and making these changes.
[105:40] And that's one of the benefits for using GitHub Copilot
[105:42] and.
[105:43] Music.
[105:44] My harness is nowhere near as skillful as theirs and
[105:47] it's literally for helping me write stuff.
[105:50] And so notice that even for a separate task.
[105:53] Does your harness has to be different or is this
[105:55] all optimized for writing code?
[105:57] That's optimized for writing code.
[105:59] That's why you see in the instruction file usually, usually
[106:02] the very first one is, hey, you're an expert AI
[106:05] programming assistant.
[106:06] We're.
[106:07] So we are putting the model in context of where
[106:10] it is.
[106:10] It's in a coding environment.
[106:12] It has to make sure it is not just like
[106:14] giving you text and it is going in, it is
[106:17] using our built in tools like run in Terminal, all
[106:20] of these things to make sure it works in the
[106:23] environment we are kind of grounding it into.
[106:26] So I'm assuming you're not like, oh, we got a
[106:28] new model, everybody, let's just use the last one and
[106:30] changed a couple of words, see what it does.
[106:32] I'm sure there's like a better process to do this
[106:35] kind of testing.
[106:36] Can you tell us?
[106:36] Yeah.
[106:38] So to be very honest, and a lot of the
[106:40] times whenever we get a new checkpoint, the very first
[106:43] thing, and this is something we get from the model
[106:45] providers, they tell us just use the system prompts from
[106:48] the prior model and let's just see how it goes.
[106:51] And we're like, OK, I guess because they're also.
[106:54] So the reason why they want us to do that
[106:57] is they want us to see how good it already
[107:01] is without doing all of the optimization.
[107:04] Yeah.
[107:05] And how do we measure this?
[107:07] So we measure this by running a bunch of offline
[107:10] eval benchmarks.
[107:11] We do this by internal dogfooding.
[107:13] So these are two in parallel things that are going
[107:16] on.
[107:16] And once we have these benchmarks run, we share the
[107:19] results with the Model Provo writer.
[107:22] And then based on that, they are like, oh, OK,
[107:24] I see it's already maybe better, but let's start tweaking
[107:27] and let's start going in.
[107:29] How about this?
[107:30] So then we sometimes have had like a model where
[107:33] we basically just removed everything and we're like, OK, we
[107:37] have to start from scratch here.
[107:39] And then we optimize it and optimize it to make
[107:43] sure whenever the model launch stay is and it truly
[107:47] is the the the perfect model to hit the crowd.
[107:50] So, but I, but there's got to like, for example,
[107:53] I'm, I'm a traditional dev, I did some machine learning.
[107:56] We we had things called evaluations.
[107:58] We do things as devs, as unit tests.
[108:00] Do you have something like that that you systematically run
[108:03] through to improve the prompt or see like, hey, it's
[108:06] this good, but now it's this good?
[108:08] What?
[108:08] What does that look like in your process?
[108:10] So our process here and this probably is going to
[108:13] touch a little bit of evals.
[108:15] And so yes, we have offline evals especially we run
[108:19] these before the launch, after launch.
[108:21] We also still optimize the harness, which is very important.
[108:24] It's not never a ship.
[108:26] And yeah, of course it's always like we go in
[108:28] and we iterate even after.
[108:30] So what are the benchmarks or how do we evaluated?
[108:34] There are a bunch of public benchmark out there, Sweebench,
[108:37] one of the very known ones.
[108:39] And we still run these even though there are a
[108:41] lot of conversations, controversy around these public benchmarks.
[108:45] One of them being, and the model providers have told
[108:48] us directly as well, the, the models are so smart
[108:52] now that you're always kind of going to get this.
[108:55] So use this as a regression test rather than really
[108:58] evaluating the quality of it.
[109:01] So we still run these, we share it.
[109:02] We see if the model has regressed.
[109:04] We use it for optimization of our harnesses as well.
[109:07] But then the more important one is we use or
[109:10] we have our own internal benchmark, which is called VSC
[109:15] bench and it currently has 100 and plus tasks or
[109:18] instances that we run every time.
[109:21] So before launch, we run across different reasoning efforts.
[109:26] We make some changes to the system prompt as an
[109:28] example.
[109:29] We see how it changes and we look at these,
[109:31] what we call model report cards and to see how
[109:34] it's performing against it.
[109:36] So you're not just like Yoloing a model out, you
[109:39] have a full bench benchmark of a bunch of stuff
[109:42] that you're testing.
[109:43] Can you show us a little bit that the card
[109:45] that helps us out show us?
[109:46] Awesome, So I did this this morning.
[109:50] Can you?
[109:51] Control Plus a couple of times.
[109:52] Sorry, yes, you're good.
[109:53] Thank you.
[109:54] So I ran this this morning and basically how it
[109:58] works is we have a BS code insiders instance in
[110:02] an ACA.
[110:03] So whenever we run our offline evals, we spin one
[110:06] up, it's always using the latest.
[110:07] So I ran it this morning.
[110:09] So that truly has the latest insider version in it.
[110:12] And how I did this, I ran it against GBD
[110:16] 5.42 times and GBD 5.5.
[110:18] Typically we try to run it five times because of
[110:21] the non deterministic problem that we have with AI.
[110:24] And so then a next step, this is where it
[110:27] gets really exciting is the performance metric.
[110:30] We look at the resolution rate.
[110:32] So resolution rate is, was this task successful or not?
[110:37] And we evaluate this by certain assertions we have and
[110:40] we think of it like a checklist and it goes
[110:43] through and says for this test case, was it resolved?
[110:46] Did it hit all of the check marks?
[110:48] Yes.
[110:48] And then it's resolved and it goes in and shows
[110:52] 153 out of the 248 test cases were resolved.
[110:56] And this is how we can compare across the different
[110:59] models.
[111:00] This is pretty amazing.
[111:01] So what should developers take away from this?
[111:03] We've got about 45 seconds.
[111:05] And first of all, always keep trying new models.
[111:10] Give us feedback if there's anything that doesn't work as
[111:14] expected.
[111:14] We are continuously operating and doing this and also I've
[111:19] done the same.
[111:20] I gave a model A1 try kind of thing.
[111:22] But because we are putting so much effort in these
[111:26] models, just make sure to maybe sometimes come back and
[111:29] give it another try.
[111:31] And also then evils, offline evils are a huge topic
[111:34] and anybody building their own coding harness, it has been
[111:38] one of the tools that we as Microsoft have been
[111:41] super successful and the model providers are loving.
[111:45] So definitely a space to dig more into.
[111:47] Awesome.
[111:48] Well, thank you so much for being with us and
[111:49] thank you so much for watching.
[111:50] We'll see you after this.
[111:51] Thank you.
[112:12] I'm so excited you're here because as many of us,
[112:15] we've been experimenting using the how copilot and assigning task
[112:18] switching using that model selector and other times I don't
[112:22] really know what model to add.
[112:24] And now, thanks to you and the work and your
[112:26] team, we have the auto selector mode.
[112:28] The model landscape is changing so quickly, and that's great
[112:31] for AI advancements in our industry.
[112:34] But what it does add is that extra layer of
[112:36] complexity.
[112:37] So questions around what model do you use, What scenarios
[112:40] you use them for, and how do you actually scale
[112:42] that when there might be a new model version or
[112:44] a new model that comes out the very next day?
[112:47] We wanted to be able to abstract that complexity away
[112:50] from you and actually automatically give you the best available
[112:53] model for your task in real time so that developers
[112:56] can focus on what they want to do.
[112:58] And so we do that through Copilot Auto model selection
[113:01] or Auto for short.
[113:03] The best part about Auto is that it's extremely simple.
[113:06] There's no configuration, no policy toggles that you have to
[113:10] do.
[113:10] All you have to do is go go into your
[113:12] favorite IDE but GitHub Copilot CLI or Copilot Cloud Agent
[113:16] and go into your model picker.
[113:35] All of my friends, we are back and I, I
[113:37] love AI and at Build Microsoft build this year, there
[113:40] was a ton of models released.
[113:42] I think there were seven Mai models released, but one
[113:45] of the ones that was super interesting to me because
[113:48] I'm a dev is Mai Code Flash and I have
[113:51] some of the researchers here with us.
[113:53] We're going to ask the hard hitting questions and I'm
[113:55] going to try not to sound dumb because these are
[113:57] the smart dudes.
[113:58] Why don't you tell us who you are and what
[113:59] you do?
[114:00] We'll start with you.
[114:01] Hi everyone, I'm young, I'm the research lead of poetry,
[114:04] part of this model.
[114:07] Hi, I'm Hunter, leader of the different Americoder YEAH model.
[114:12] So this particular model was specifically built with GitHub Copilot
[114:18] in mind, is that right?
[114:20] So Peng, why don't we start with you?
[114:21] What does that actually mean?
[114:23] Yeah, to me the purpose building model means that we
[114:27] didn't take a general purpose model and adapt it for
[114:31] code at the end.
[114:33] But actually we build from the very beginning for GitHub
[114:38] compiler scenario and we optimize it for GitHub compiler experience
[114:44] from very beginning.
[114:46] So for those that are like thinking like OK Mai
[114:49] Mai Code Flash was optimized with VS Code Copilot in
[114:53] mind, what does that mean for people?
[114:56] What you mean is it's not just a generic code
[114:59] snippets.
[115:00] It will understand the positive context that use tools, understand
[115:05] the users intent and make a targeted edits and they
[115:09] respond quickly.
[115:10] Especially for GitHub competitive users, we will optimize for those
[115:14] real user workflow.
[115:16] I see.
[115:17] So why don't you tell us more about how what
[115:19] that looks like in practice when you optimize for BS
[115:21] code?
[115:22] Yes.
[115:22] So I think it means it's not just good at
[115:24] some benchmarks, it's definitely good at many benchmark.
[115:27] But in the real code agent it also means like
[115:31] it interacts with you in your proper way.
[115:34] It asks your questions without long thinking.
[115:37] For some simple questions, do the unit test at the
[115:40] correct time.
[115:41] At the right time it do the proper testing.
[115:44] So it basically optimize for the best user experience in
[115:48] US code.
[115:49] So tell me more about that because like I, I
[115:52] was trained in machine learning and effectively what I understand
[115:55] models to be is it's this huge structure, you push
[115:58] a lot of data through it, some linear algebra and
[116:00] calculus to optimize it.
[116:02] Tell me about the data that you're specifically pushing through
[116:06] it to make this model super good for VS Code
[116:08] users.
[116:09] That's a very good question.
[116:10] So that's why reinforced learning plays a very important role
[116:13] here.
[116:14] So which means in this training with last model, interact
[116:17] with the real product partners, with the real web code,
[116:20] with the same 2 sets that developer use every day,
[116:23] with the same prompts you use every day.
[116:26] And you can interact with the web wall with the
[116:28] environment like your daily suite tasks.
[116:29] And we can create and reward model with a lot
[116:32] of like Easter tasks for field for you.
[116:35] And that's it.
[116:37] Like have a very proper product behavior and also many
[116:40] things that you think the model should do the best
[116:42] for your daily suite task.
[116:45] And it's all be done by very reinforcement with the
[116:48] real product partners.
[116:50] I see.
[116:50] So there's a base model and then you're doing reinforcement
[116:54] learning on top of it to adapt the behavior.
[116:57] Is that right?
[116:58] Tell us about.
[116:58] That multiple stages.
[117:00] The universe learning is the last stage to make the
[117:02] model perfectly aligned with user experience in GitHub compiler.
[117:06] So tell me more.
[117:06] So you're doing, you're actually when you're training the model,
[117:09] you're training it with the actual BS code stuff.
[117:11] Is that right?
[117:12] Tell me about the stage.
[117:13] When we train that model, we have multiple stages.
[117:15] We started with supervised fine tuning to make the model
[117:19] follow user instruction and response with aligned format.
[117:23] And then we gradually trained the model with a lot
[117:26] of diverse data in a criminal way, from simple task,
[117:30] simple coding task to complex coding task stage by stage.
[117:33] And the final stage is that we use reinforcing learning
[117:36] to make the model to learn in this real environment
[117:39] as a productive.
[117:41] So in the real scenario, real environment, they stay close
[117:45] to product so that we make the training evaluation and
[117:49] the production in the same same workflow.
[117:52] So that's the difference from our model, that's the difference
[117:56] from our model, from the purpose model.
[117:59] And this is so cool because I want to restate
[118:01] this because as I'm hearing this, it's kind of super
[118:03] cool.
[118:03] You basically have a base model and you do supervised
[118:06] fine tuning.
[118:07] Yes, Sir.
[118:07] First with simple tasks, yes.
[118:10] Then with more complex, it's almost like it's you're taking
[118:13] it to school on how to be a good programmer.
[118:15] Simple task, more complex tasks, and then you test benchmarks
[118:19] over the top of that.
[118:20] Is that right?
[118:21] Yes, and this benchmark is not another benchmark.
[118:24] So this benchmark is customized to use the same environment
[118:28] as a product use the same harness.
[118:30] So it's a purpose built model for people that use
[118:34] this harness.
[118:35] Yes, that's really cool.
[118:37] So this is cool.
[118:39] Tell me about the model size, because one of the
[118:41] things I tell people is, hey, if you're doing a
[118:43] generic task and you're not sure, you got to use
[118:46] a pretty big model and then learn and then make
[118:48] a fine-tuned small model.
[118:50] Tell me about the model size here and why that
[118:53] particular that particularly matters.
[118:56] Mai code 1 flash is a 5B model active parameter
[118:59] model, but it actually Moe model.
[119:02] So the total capacity is about 137 billion parameter.
[119:06] So which is fairly large.
[119:08] I think model size plays a very important role here
[119:11] is like it cannot have a trade off between the
[119:13] model intelligence, the speed and the cost.
[119:15] That's the three things that we care about.
[119:17] And with this large Moe model, but with small active
[119:21] parameters, it actually achieve a balance point or sweep point
[119:24] for your daily sweep tasks.
[119:26] You don't need to waste a lot of tokens, but
[119:29] you still have very powerful models for your daily work.
[119:34] Yeah.
[119:34] So tell me more about that, because you said a
[119:36] 5 billion parameter model, but you're using a mixture of
[119:39] experts.
[119:40] So tell me how that works.
[119:41] So for people that haven't heard that, what does that
[119:42] look like?
[119:44] So basically, imagine it's a neural model and you have
[119:47] a lot of parameters, but in each token inferred only
[119:51] need to activate a part of them, and that part
[119:54] of them will serve a task.
[119:56] The rest of them can be activated for other tasks,
[119:59] other tokens.
[120:01] So which means the each inference only part of networks
[120:04] activated and this saves a lot of costs.
[120:07] Yeah, so, so the model total model is only 5
[120:09] billion parameters or?
[120:11] No.
[120:11] Total model is 137 parameters.
[120:14] But only active at a single time is the 5
[120:16] billion because of the mixture of experts.
[120:19] And this is the this is the cool part that
[120:21] people that I love about this machine learning has gotten
[120:24] so good that what a mixture of extra effectively shuts
[120:27] off an entire part of the model using nice vector
[120:30] calculus and it pushes the exact query through one of
[120:33] the experts so that you're actually getting super fast inference
[120:37] speed.
[120:38] So effective model size is 5 billion parameters.
[120:40] Yeah, not just one expert inside mixed on many experts,
[120:44] yeah.
[120:44] That's amazing.
[120:45] So what does that mean for speed and accuracy?
[120:49] So if you look at our blog, our model, we
[120:51] can see the latency of model is super good comparing
[120:54] with the same quality and intelligence.
[120:57] And also I think when I mentioned we also optimize
[120:59] model for token efficiency.
[121:01] So if you combine model efficiency and token efficiency, you
[121:04] will find it just a beast at its size.
[121:07] It's just fat.
[121:08] Yeah.
[121:08] Because it's like, it's like a big model with little
[121:11] models that together make it a little bit faster.
[121:14] So what what has been the most exciting part for
[121:16] you and challenging parts of building of building this model?
[121:21] Actually there's a lot of change.
[121:22] As I said, from beginning the model just put in
[121:25] next token so it cannot follow user instruction or to
[121:28] use tools.
[121:29] So we need to do something called a code start
[121:32] to make the model can follow user instruction and do
[121:35] some simple reasoning tasks.
[121:37] So basically it can just like children, which start from
[121:41] a very young baby and make it graduate, go up
[121:45] to a post doctor.
[121:47] So that's a good deal.
[121:48] The first step is kindergarten, right?
[121:51] That's good start.
[121:53] And after that we can keep the model and make
[121:56] the model keep climbing and that's what a higher level.
[122:00] Yeah.
[122:00] So a lot of folks when I look and this
[122:02] I fall into this trap because maybe, I don't know,
[122:05] we like biggie sizing things.
[122:07] Sometimes we fall into this trap of thinking that a
[122:10] bigger model is going to be better at specific tasks
[122:12] and that's not always the case.
[122:14] Sometimes a smaller model can be even better.
[122:16] Why is that the case in in for Mai flash?
[122:20] So I think model size matters a lot of intelligent
[122:23] but not all of it.
[122:24] We take good careful training.
[122:26] We use the same harness of very proper designed data
[122:29] for this model to make sure the model can always
[122:31] climb on its reward in training.
[122:34] So I think the outcome is very good.
[122:36] It's not just limited to the size.
[122:39] Very powerful model.
[122:40] OK, so I think you've brought something to show us.
[122:43] Can you show us what we've got?
[122:45] So we're gonna go to the screen here and you
[122:47] brought a little bit of something to show us in
[122:49] action.
[122:50] Is that right?
[122:50] Yes, yes.
[122:52] So what I'm showing is a demo we just made.
[122:55] This is using our model in West code developing a
[122:59] front end web app.
[123:00] You can see the model is very fast, you give
[123:02] it a problem, it do a lot of things like
[123:05] the token on the fly is super quick and use
[123:07] the customized training with W code to it and use
[123:10] the native W code very efficiently.
[123:12] And in the end you can see it developed a
[123:16] very nice looking app in a very short time.
[123:20] And I think here's another demo, it's a bit different.
[123:26] The model is prompted to build AI would say a
[123:30] ritual style game and you can see the UI is
[123:35] pretty good.
[123:36] We do put a lot of efforts on improving the
[123:40] models front end ability and with the native partners in
[123:44] training, yeah, it seems to use it pretty efficiently, yeah.
[123:49] Yeah.
[123:49] And this is pretty cool because now this particular model
[123:52] is available in VS Cover people to use.
[123:55] Is that right?
[123:56] We are rolling out the model gradually and you can
[124:01] use.
[124:01] If you got the model, you will see your model
[124:04] picker.
[124:05] So what has been the funnest part about building this
[124:08] model?
[124:08] Because like in this case, it's, you're building a frontier
[124:12] model, but for a very specific use case.
[124:15] What are some learnings that you've had along the way
[124:17] I'm.
[124:18] Learning, I think users real workflow is important for us
[124:22] to optimize their model for users purpose.
[124:26] Yeah, instead of you know the benchmark.
[124:28] So without using the capacity 10 minute grounding task, we
[124:33] cannot even make them model work as perfect as today
[124:37] in compiler.
[124:39] And can you tell us a little bit about how
[124:41] you test these models so folks get a sense for
[124:42] because you're not just like doing weights, but you're testing
[124:45] these models as they go.
[124:46] Tell us about the testing process.
[124:48] So definitely we have very good benchmark results.
[124:50] If you look at the broad across many benchmarks, we
[124:54] I think it's frontier model at this size.
[124:58] As Pauline said, we also care a lot about your
[125:00] user experience.
[125:01] So that's why we do a lot of AB testing
[125:03] online to make sure the model fulfill your task.
[125:06] Your engagement with models is crap.
[125:08] You don't cancel too much for the model.
[125:10] So if you've used this model, tell us what the
[125:12] experience has been like using this model.
[125:14] You said it's a little snappier, it's a little faster.
[125:17] Tell us about the experience.
[125:19] To me, I think for most of the task, it's
[125:22] comparable to today's frontier big models.
[125:25] But just for some challenge tasks, we must recognize this
[125:30] deal that's getting low disability, that's getting a small model
[125:35] still has some limited for challenge.
[125:38] But for most of the daily work, I think this
[125:40] work works good for me.
[125:42] So generally when we go from a bigger model to
[125:44] a smaller model, it it gets very focused on the
[125:47] particular task that you fine-tuned it for.
[125:50] What are some things that like people should not expect
[125:53] it to be?
[125:53] You know what I'm saying?
[125:54] Because I don't I don't want people to use the
[125:56] model be like, wow, it's not writing poems for me
[125:58] or something.
[125:59] Tell us about what are some things that are super
[126:01] good at and what are some things that maybe you
[126:03] should use other models for?
[126:06] So I would say for most of our daily suite
[126:08] tasks, back end, front end, different programming language, it's a
[126:13] very good model.
[126:14] But the writing poem do a lot of like creative
[126:18] writing things.
[126:19] We are not optimized for that, but I can still
[126:22] do something.
[126:23] But maybe you choose other models?
[126:25] Yeah.
[126:26] Cool, so I hear you have one more demo.
[126:29] Can you show us that one and this one?
[126:31] Does this one have audio?
[126:32] This one has audio.
[126:35] OK, so are we equipped for audio Tom on this
[126:37] one?
[126:38] We are ready for.
[126:38] It OK, they're ready for it.
[126:40] By the way, Tom's.
[126:41] Awesome, this is for fun.
[126:45] Can you maximize it?
[126:48] I can start coding agents, give them tasks.
[126:51] That sounds cool.
[126:52] Can you run the web app locally so we can
[126:54] start testing?
[126:56] I got it.
[126:56] Let me work on that.
[126:57] I need your approval to start the deep development server.
[127:00] Sure, I approve.
[127:04] The development server started on local host app.
[127:07] Thanks.
[127:07] Can you now change the background color of the web
[127:10] app to a gradient of blue and green?
[127:14] I got it.
[127:14] Let me work on that.
[127:20] The agent updated the project.
[127:21] Thanks, Now can you check what kind of unit tests
[127:24] we have and see if there's any missing and cleared
[127:27] them?
[127:29] I got it.
[127:29] Let me work on that.
[127:31] The test suite is ready to run.
[127:32] I need your approval to execute the NPM.
[127:35] Sure, I approve.
[127:38] The agent added unit tests for the profile header.
[127:42] What's going on with the agents right now?
[127:45] One moment checking.
[127:47] All three agents are actively checking the current app structure.
[127:50] One is planning to refactor components into a RE.
[127:56] So why don't we just tell us what we just
[127:57] looked at?
[127:58] So this model is kind of especially our weekend apart.
[128:02] We combined our latest state of art Mai Transcribe model,
[128:06] Mai OS model and Mai Code Flash model for this
[128:10] fun project.
[128:12] And that's kind of cool because now that you now
[128:14] that you have specific purpose built bottles, you can start
[128:17] to put them together and do some some really, really
[128:20] cool things.
[128:20] Tell us about how you came up with this.
[128:23] I think it's very natural when MVI comes all these
[128:26] seven models together.
[128:27] So it's kind of natural to think how we can
[128:30] combine and how we can imagine the suite developers in
[128:34] the future.
[128:35] So that's very natural project.
[128:37] So what should developers take away from all of this?
[128:39] We'll start with you and then we'll come to you.
[128:41] Take away.
[128:41] I think imagination is important.
[128:45] We can imagine in the coming few months as we
[128:47] have even better sense come out.
[128:49] Fantastic.
[128:50] What about you?
[128:51] My take away is we are seriously treating the best
[128:54] model for our product, the best code and other product
[128:57] and this is just the beginning of our journey.
[128:59] Please stay tuned and looking forward to what will happen
[129:02] next.
[129:03] That's amazing.
[129:03] And so as developers try this out, make sure you
[129:06] give us feedback because I'm sure you want feedback on
[129:08] how the models.
[129:09] Work.
[129:09] Yes, yes, feedback is very.
[129:11] Welcome a lot of feedback.
[129:12] And you're going to continue to improve this model and
[129:15] do work.
[129:15] Is that right?
[129:16] Tell me about the future of this.
[129:19] So definitely I want to mention we need feedback so
[129:22] we can improve the model and this is a mini
[129:25] site model.
[129:27] Just stay tuned for some more powerful beasts.
[129:30] Awesome.
[129:31] Well, thank you so much for spending some time with
[129:33] us.
[129:33] It's been amazing.
[129:33] I'm a nerd.
[129:34] I love this stuff.
[129:35] So thank you so much.
[129:36] And we'll be back right after this.
[129:38] We'll see you.
[129:38] Thank you.
[129:40] Let's talk a little bit about the project to port
[129:43] the TypeScript compiler to.
[129:45] Go.
[129:46] Can you talk a little bit about the motivations, kind
[129:48] of the trade-offs, the challenges of making that transition?
[129:51] Yeah, TypeScript from day one was self hosted written in
[129:54] itself.
[129:55] And that meant that TypeScript, the compiler, the tools that
[129:59] everything runs simply as a JavaScript app.
[130:01] Now that has a bunch of advantages.
[130:03] Like, heck, you can run a compiler in a browser,
[130:05] you know, and then so you can have an IDE
[130:08] in the browser, you know, with the full compiler and
[130:10] language servers and everything just works.
[130:12] But it comes with some drawbacks too, precisely because of
[130:17] this massive adoption.
[130:18] And then like the ever growing size of projects that
[130:21] that our users build, scalability has just become the number
[130:25] one issue.
[130:26] And then there's some inherent limitations of JavaScript itself.
[130:29] It's it's the languages by the sign single threaded and
[130:33] does not give you access to shared memory concurrency.
[130:37] So you're leaving it's like 90 some percent of the
[130:40] compute power on the table, right.
[130:43] And then JavaScript itself is just it's an expensive language
[130:46] you know to to execute it has a very permissive
[130:49] object model.
[130:49] You can just like walk up to an object and
[130:51] start slapping properties on it.
[130:53] And that means it's hard to optimize because this ultimately,
[130:56] when it runs, ends up being some hash table look
[130:59] up and there's some caching and there's something and whatever.
[131:02] But it's not like native code.
[131:04] So we knew we were leaving money on the table
[131:07] left and right.
[131:08] And as much as it pained us to to give
[131:10] up on on self hosting, I can't say enough good
[131:13] things about our team and about how smart these people
[131:16] are and how well they understand how JavaScript works and
[131:20] how surprising it is what you need to do to
[131:23] optimize.
[131:24] Like sometimes you have to do exactly the opposite of
[131:26] what you think you have to do in order to
[131:28] make this code go a little bit faster because of
[131:31] the way V8 is architected.
[131:32] And I mean, we had all of the tricks we
[131:34] had found out and we had applied and whatever and
[131:37] it was still just, we're not, we're never going to
[131:40] get any faster.
[131:44] All my friends, welcome back to Microsoft Build.
[131:46] We are here on the live stage and I'm super
[131:47] excited to be here with Daniel.
[131:48] How you doing bud?
[131:49] Doing right?
[131:49] Have it to yourself.
[131:50] Fantastic.
[131:51] So why don't you tell us who you are and
[131:53] what you do?
[131:54] And this is going to be a fun one.
[131:56] Tell us about it.
[131:56] Oh, it's.
[131:56] Going to be great.
[131:57] So my name is Daniel Rosenwasser.
[131:59] I'm the product manager on the type trip team.
[132:01] I've been there for a long time.
[132:02] I started out as a developer, moved into the PM
[132:04] space because I just kept talking and they were like,
[132:06] all right, let's move them over here.
[132:09] Yeah.
[132:09] But yeah, so I've been working on TypeScript.
[132:12] So that is the language, the compiler, the editor experience
[132:15] that you know and love in VS and VS Code.
[132:18] And also if you're just doing JavaScript in these editors,
[132:21] like we're supporting all of that stuff, it's all powered
[132:24] by the same guts that we've been working on for
[132:26] a long time now.
[132:26] So I.
[132:27] Remember talking to Anders about, I don't know, it was
[132:31] a year ago or so about a major change.
[132:34] But before we get to that, I realize, why don't
[132:36] you tell people what TypeScript is the blurb?
[132:38] It's super cool, I use it all the time.
[132:39] Yeah.
[132:40] So basically, you know, many, many years ago we found
[132:43] that there was like just this continuous problem for a
[132:46] lot of developers building out big JavaScript apps, right?
[132:50] All of us need to have websites, all of us
[132:52] need to have apps that are cross-platform, that are accessible
[132:55] to the browser sometimes.
[132:56] And so as we sort of needed to do scalability
[132:59] and like actually create more complex apps, we found that
[133:02] there were a lot of other things that we were
[133:05] sort of missing from our other languages.
[133:07] Like in C#, like, you know, that certain things are
[133:10] just plain not going to work.
[133:12] Not just from like when you compile it, but like
[133:14] as you're typing it in the editor, you're getting a
[133:16] little red squiggle, right?
[133:18] The same way that in Word you get something that
[133:19] tells you, hey, you might have made a mistake here,
[133:21] a typo.
[133:22] And so we knew that there was a great opportunity
[133:25] for a great solid safety and developer tooling that we
[133:27] could give people.
[133:28] And also at the time, we were able to bring
[133:30] people modern JavaScript constructs that were still not really available
[133:34] in every browser and runtime.
[133:35] And so TypeScript was a way to take JavaScript and
[133:39] add syntax for types and add a type checker and
[133:42] a compiler so that you could bring all the modern
[133:45] features that you know and love today.
[133:48] And the cool part is when I first started on
[133:51] that effort, right?
[133:52] I mean, I wasn't there from the very beginning, but
[133:54] I was there from early days.
[133:55] And it was like, this is ambitious, right?
[133:58] This is Microsoft, this is, we're trying to do an
[134:01] open source project that is in the JS space, the
[134:04] JavaScript space.
[134:05] And a lot of people were like, I don't know,
[134:08] even before I joined Microsoft, I was like, very skeptical
[134:11] about that effort.
[134:12] And then I joined, and then I found out that
[134:14] really the team behind all of it is really passionate
[134:17] and had a very good sensibility of, like, working with
[134:19] the community and really trying to have a good sense
[134:22] of developer tooling.
[134:24] And so it's been nice.
[134:26] Yeah, it has been nice.
[134:27] So nowadays, I mean, I don't know if you caught
[134:30] this on the GitHub Octoverse last past year, but TypeScript
[134:33] actually became the number one programming language on GitHub.
[134:37] Yeah.
[134:38] And so, you know, if you like beyond JavaScript, right?
[134:42] And so that is kind of nuts.
[134:44] If you would ask me that back then, I would
[134:46] have just left you out of the room completely.
[134:49] But now The thing is that that's interesting is that
[134:53] Java TypeScript is now the number one way for people
[134:57] to build JavaScript apps.
[134:59] Right?
[134:59] Like we don't really think of these things as so
[135:02] distinct anymore.
[135:02] It's not like TypeScript versus JavaScript.
[135:04] It's I'm doing JavaScript development and I'm using TypeScript and
[135:07] that's just kind of the core experience.
[135:10] So yeah, I don't know.
[135:12] I.
[135:12] Mean I think it's awesome.
[135:13] Like I I use it all the time in in
[135:15] particularly large JavaScript projects.
[135:18] Yeah, having type safety at the compiler level is super
[135:21] transpilation layer.
[135:23] Yeah.
[135:23] So about a year ago, right, there was a major
[135:26] shift, yes, from because TypeScript was built on TypeScript, kind
[135:30] of TypeScript on type.
[135:32] But there was a.
[135:32] Major shift that happened.
[135:34] Tell us about that, right.
[135:35] So TypeScript has built been built in TypeScript for over
[135:39] 10 years now, right.
[135:40] It's kind of kind of this food process.
[135:42] It's.
[135:42] 10 years already.
[135:43] Over that yeah and it's like people call it dog
[135:45] fooding right like you eat your own dog food so
[135:48] you know how good or bad your dog food tastes
[135:50] or bootstrapping basically by us doing that that gave us
[135:53] a really solid sense of like what the experience is
[135:55] day-to-day.
[135:56] But the problem with TypeScript being built in TypeScript is
[136:00] it ultimately compiles down to JavaScript.
[136:02] Yeah.
[136:03] JavaScript has had a miraculous amount of development on all
[136:06] the engines like in V8 and Spider monkey and all
[136:09] that people have made it really fast for the dynamic,
[136:13] you know interpreted language that that people kind of knew
[136:16] from like 1995 and but, you know ultimately there's an
[136:20] overhead right There's always going to be a little bit
[136:23] more that your engine needs to do and so there
[136:26] was this sort of gap between what Java should be
[136:29] able to do and what you're able to do natively
[136:32] on your CPU.
[136:33] Right and JavaScript kind of fundamentally is single threaded.
[136:38] Yeah, ways to do multi threaded JavaScript.
[136:42] But then these threads can't really share the same memory
[136:45] without handing it off fully to each other, right?
[136:48] And we looked into all sorts of solutions to try
[136:51] to make things faster, but we as have just kind
[136:54] of been promising on its delivering on its promise to
[136:58] do complex JavaScript apps.
[137:00] That means that there's a scalability gap, not just in
[137:02] terms of scaling on your team, but scaling on a
[137:04] big code base.
[137:05] And so people complain like my build times are slow.
[137:08] I'm waiting for my editor to like actually load up
[137:10] all the code I've got on disk.
[137:12] And so all right, let's go to the chase now
[137:15] we, we actually chose to do a rewrite in the
[137:17] last year and a half or so, taking the existing
[137:20] types of code base and porting it over to go.
[137:24] And in that port, what we found is a typical
[137:27] sort of speed up that you'll find on many code
[137:30] bases around 10X, right, which is pretty mind blowing.
[137:33] Yeah.
[137:34] And so part of that speed up comes from that
[137:36] native speed, and part of it comes from shared memory
[137:39] parallelism.
[137:39] So we're able to actually go faster not just on
[137:42] one core, but across all the cores that you've got.
[137:46] And I've got a little bit of a demo if
[137:47] we want to.
[137:48] Yeah, let's do it.
[137:48] So just to just to recap effectively, you didn't, you
[137:52] didn't.
[137:52] TypeScript isn't Go, it's.
[137:54] Right.
[137:55] Go is being used to do the transplation, to do
[137:58] the type checking, to do all the other stuff.
[138:01] Right, in the sense like basically all of the code
[138:03] that we had in the TypeScript code base we took
[138:05] like, you know, we've got different components, we've got a
[138:08] parser, we've got a checker, we've got like an emitter
[138:11] to like do your code Gen.
[138:13] and each of those components was ported over to Go
[138:15] code.
[138:16] But at the end of the day, you're still using
[138:18] this to compile TypeScript code, right?
[138:20] It's still reading JavaScript.
[138:21] It's reading TypeScript and it's emitting JavaScript and it's giving
[138:24] you the same exact errors that you've had in the
[138:27] past.
[138:27] And before we go to demo, because I remember, if
[138:30] I remember, I remember talking to Andrews about this, it
[138:34] wasn't like you just like wrote it and go and
[138:37] wrote new unit tests.
[138:39] You basically took every part piece by piece.
[138:42] And tell us about that process, because it was not
[138:45] like a, it was a very methodical approach.
[138:48] Oh, totally.
[138:48] Yeah.
[138:49] So this was a very, this was also back when
[138:51] you like agents were a little bit more flaky in
[138:53] the reliability and stuff like that.
[138:55] So a lot of people say like, why didn't you
[138:57] just could you have just buy ported it over?
[138:59] Right.
[139:00] And so at the time, definitely at the time, no.
[139:02] But the cool thing that we did was we used
[139:06] agents to actually write infrastructure to automatically port parts of
[139:11] our code base, right?
[139:13] So like admit the initial stuff was methodically going through
[139:16] the scanner and the parser.
[139:18] So these.
[139:18] Artisanally artisanal.
[139:21] Artisanal code.
[139:22] And, and so once we did that, we've kind of
[139:24] had a sense of the data structures and, and how
[139:27] all of the code would look and how we wanted
[139:29] the code to look.
[139:30] And that allowed us to say like, OK, we have
[139:32] a sense of what we want.
[139:33] Can we partially automate this?
[139:36] So we actually had a script that would take a
[139:38] TypeScript file, right?
[139:40] All of our files are kind of uniformly written a
[139:42] specific style.
[139:43] And we would say like, oh, when you're going through
[139:46] the parser, rewrite parts of the parser in so and
[139:49] so way.
[139:49] And that way whenever you're missing something, right, like you
[139:52] have portions of your component, you say, oh, well, I'm
[139:55] missing that functionality.
[139:56] That's literally a function that you're copying and pasting between
[139:59] the Kuco bases.
[140:00] And then you evaluate is this new function how I
[140:04] want it?
[140:04] Like is it acting the way I want it to
[140:06] act?
[140:06] Sometimes you get a compiler error and it's intentional because
[140:09] it makes you sort of think about like, how should
[140:11] this act?
[140:12] So you're just sort of like ripping pieces of auto
[140:15] generated code into the into the into the new code
[140:18] base.
[140:19] But the cool part about that is that it's all
[140:21] very deterministic, right?
[140:22] So if something is not right, you can always say,
[140:24] well, this script needs to be amended.
[140:26] I'll actually adjust the output of this thing and I'll
[140:29] make sure that it's giving me something that is more
[140:32] predictable, ideal and and more more idiomatic Go code for
[140:35] the sort of things that we're writing in the new
[140:38] code base.
[140:38] So did you like use the AST to generate Go
[140:42] code or was it?
[140:44] Yeah, yeah, we we would use the existing types of
[140:47] APIs.
[140:47] I see.
[140:48] Well, we would use a really cool tool called Man
[140:51] TS Morph, which uses our Asts and everything like that
[140:55] under the hood.
[140:56] But it's a really nice API and we would generate
[140:59] Go code out of that and that really speed up
[141:02] the development process.
[141:03] And we were able to reuse all of our unit
[141:06] tests and our input output tests like our integration tests.
[141:10] And so over the last year and a half, we've
[141:12] been basically minimizing the differences in all the tests, right?
[141:16] So we've basically been able to get like, I won't
[141:18] say over 95% of the tests boarded over, some of
[141:21] them don't apply anymore.
[141:23] But like that's because there are certain features that we've
[141:26] removed as well.
[141:27] But it's kind of incredible how consistent and fast we've
[141:31] been able to move in that process.
[141:34] I don't think we would have been able to make
[141:36] some same strides five years ago, for example.
[141:39] But it's been incredible because we got a lot of
[141:42] great users internally and externally who are giving us great
[141:45] feedback telling us like, hey, you missed this spot, right?
[141:48] But that was months ago, right?
[141:50] So it's only gotten more and more and more solid
[141:52] in the last few.
[141:53] Months.
[141:53] So in general, if someone switches over to the Go
[141:57] underlayer, they're not even going to.
[142:00] They're not even going to know.
[142:01] Other than that, wow, this feels really snappy.
[142:03] Right.
[142:03] Yeah.
[142:03] So that's The funny thing.
[142:04] It's very interesting to sort of demo these sorts of
[142:06] things, like, hey, I have a compiler, I have a
[142:09] faster compiler.
[142:10] Isn't that cool?
[142:10] Right.
[142:11] And I can show you why there's a little bit
[142:12] more subtle to do that.
[142:13] But yeah, for the most part, most people will not
[142:16] really notice a difference.
[142:17] There is some stuff that we are trying to work
[142:20] on right now, which applies to the API's.
[142:22] So if you're relying on stuff like the Vue JS
[142:25] editing experience or Angular as well, like there are certain
[142:28] gaps, you will still be able to get some benefits
[142:30] from this new compiler architecture.
[142:33] And in fact, you'll get like a fully, you know,
[142:35] the story, like the gaps will close over time actually
[142:39] pretty soon, right?
[142:40] We're working with all those dev tool authors, but most
[142:43] projects I would say are just like ready to actually
[142:46] start adopting.
[142:47] This All right, now that we've done the preamble, people
[142:50] have confidence.
[142:51] Let's take a look at what you got.
[142:52] Can you?
[142:53] There you go.
[142:54] You exactly.
[142:54] I was an ass.
[142:55] Let's take a look.
[142:55] What are we looking at?
[142:56] Here.
[142:56] So OK, I'm actually, I'm actually editing the VS Code
[142:59] code base right now.
[143:01] Like this is actually, if you go to like the
[143:03] VS Code repo on GitHub, this is their source code,
[143:07] right?
[143:07] So I'm going to actually show you what it is
[143:09] to compile their existing code base today.
[143:12] There's a pretty gross command that I've pasted in here
[143:14] because if you try to compile the VS Code code
[143:17] base with the existing JavaScript compiler or types of compiler,
[143:20] it runs out of memory.
[143:22] So you have to actually modify, you have to pass
[143:25] like certain flags to Node JS just to say like,
[143:27] by the way, I need some more RAM or you
[143:29] can use more RAM.
[143:30] It's OK.
[143:32] So what I'll also show you right now is not
[143:35] that because we're on Unix.
[143:38] OK, so this is H top.
[143:39] This is sort of like a process monitor.
[143:43] What am I doing here?
[143:44] Let's just do this.
[143:45] Look at look at this going old school.
[143:47] I love it.
[143:48] Yes.
[143:48] So what you'll notice is that I'm currently on, like
[143:52] if you look at Core 15, it's kind of hard
[143:55] to see on this screen, but like if I zoom
[143:58] out, Core 15 up here is the only one.
[144:01] Yeah, it's pegged right.
[144:02] Like basically, you know, this is sort of like the
[144:04] meme of a bunch of people like construction site and
[144:06] everyone's just like watching one guy dig a hole and
[144:09] they're like.
[144:10] Everyone's supervising.
[144:11] Yeah, looks good to me.
[144:12] Right.
[144:12] And this is an artifact of basically node because it
[144:15] only it's on a spin lock until it has an
[144:18] IO request.
[144:19] Right.
[144:19] Like be, I mean, right now it's just checking actually.
[144:22] So, so it's just checking and it's only able to
[144:25] use one core at any given time because it's not
[144:27] able to like dispatch between different threats, right?
[144:31] You know, when you parse, we go one file by
[144:34] 1 file.
[144:35] And if you say I'm going to import this guy
[144:37] over here, this file over here, then it's like, well,
[144:40] I have to wait for you to finish parsing before
[144:42] I can do anything else, right?
[144:44] So right.
[144:45] And then it handed it off to Core 28 or
[144:46] something like that.
[144:49] So not efficient, not efficient.
[144:51] Took 97 seconds in our opinion, one minute and 38
[144:54] seconds real time according to the system.
[144:56] That's a little bit hard to see, so I'll zoom
[144:58] in.
[144:59] OK, So do we have TS go?
[145:02] OK, we don't have TS go.
[145:04] So TS go is the binary that we have in
[145:07] place of TSC, the types of compiler until we're out
[145:11] of like preview stage, right?
[145:14] So if we do NPM install at type strip slash
[145:17] native preview and.
[145:19] Artisanally typing stuff in?
[145:21] What are you, some kind of animal?
[145:22] What's going on here?
[145:23] Let's do this globally actually, because all right, cool.
[145:28] So now we have TS go.
[145:31] Great.
[145:31] So what you'll see is 7 O, right?
[145:33] The next types of compiler is actually going to be
[145:35] version 7 point O, right?
[145:36] That's that's what we're and and it's nicknamed TS go
[145:38] for a lot of people.
[145:40] So if we start to do, if we try to
[145:42] do the same command with TS go, right, we'll do
[145:45] TS go.
[145:46] We'll point it to the source directory extended diagnostics.
[145:50] And then we'll also time it too, right?
[145:53] And then we'll do this, we'll look back at H
[145:55] top.
[145:56] What you'll see is that everything is in use right
[145:59] now, right?
[146:00] All of a sudden, all the cores got saturated at
[146:02] the very beginning.
[146:04] And then they're awake for a very brief second.
[146:06] You might have seen like we went down to like
[146:08] 4 and then we went, we really saturated again.
[146:11] So what happened there was we parsed all of the
[146:13] files in the guest code code base.
[146:14] I think it's like a million and a half lines
[146:15] of code.
[146:16] We checked only across 4 cores because checking is not
[146:20] a perfectly parallel problem.
[146:22] And then when we did a MIT, that was like,
[146:23] all right, throw it all the cores that you've got,
[146:25] right?
[146:26] And that made it really fast.
[146:27] So you didn't see it in the background, but that
[146:30] was like 9 seconds or just 10 seconds.
[146:32] We went from like a minute and a half to
[146:34] 10 seconds.
[146:35] Non incremental build, full build.
[146:38] And what I also want to show you as well,
[146:40] and that is just the command line experience.
[146:43] So for a lot of people that's like, well, that's
[146:46] cool, you only have a command line compiler.
[146:48] What else do you got?
[146:50] Well, we have a native preview extension.
[146:52] If you literally go to the marketplace and you type
[146:55] in native preview, TypeScript, native preview is the first thing
[146:58] that comes up, right?
[146:59] And so it says, hey, this workspace, you know in
[147:02] VS Code they're already type checking with the new thing,
[147:05] right?
[147:05] So as you're doing that, why don't you know who's
[147:07] using this already?
[147:08] VS Code, Century, Slack, Notion, Figma, like we've actually been
[147:12] working with a lot of big companies internally and teams
[147:16] internally and then companies externally because they know that the
[147:20] quality bar has already been like, you know, it's there
[147:23] for them, right?
[147:24] And and they're using it in production and they've got
[147:27] like much faster turn around times for their CI.
[147:30] We worked with one team at Microsoft that told us
[147:33] that we were saving them 400CI hours a month.
[147:36] And so like maybe in cost that's like, hey, that's
[147:38] pretty good.
[147:39] But in terms of developer costs, yeah, that's great.
[147:42] In terms of like I'm waiting for an agent to
[147:44] do a thing for me.
[147:45] That's really sweet.
[147:46] Also all.
[147:47] Right.
[147:47] So we've got about a minute left.
[147:48] What do you want to show us in the last
[147:49] minute?
[147:49] You got, I want to show you that basically let's
[147:52] let's disable the native preview.
[147:54] If I open up ATS file, you will sometimes see
[147:56] this little spinner here, right?
[147:58] That initializing TS config, all that, right?
[148:02] This is the existing TypeScript experience.
[148:04] And basically what's happening is it's doing that thing where
[148:07] it's going file by file and not able to load
[148:09] at all.
[148:09] I want you to say goodbye to that.
[148:11] Let's say goodbye.
[148:11] Let's say goodbye.
[148:12] So this thing, it's automatically enabled when you install.
[148:15] As soon as I open this, I can restart the
[148:18] language server over and over again and it's just speedy,
[148:22] right?
[148:22] Uh oh, you didn't see that?
[148:24] Anyways, it's fast.
[148:25] Yeah, hey, we're still working on code.
[148:28] It's OK.
[148:28] Look, yeah, I mean, there are some rough edges, of
[148:31] course, during the demo, it had to happen that way.
[148:34] But basically as soon as you open this thing up,
[148:37] I mean, it's still running at this point.
[148:38] All right?
[148:39] Yeah.
[148:39] So.
[148:40] This is amazing, like all.
[148:41] All I know is that like this is really cool.
[148:44] I'm excited to use it.
[148:45] Thank you so much for spending smiles.
[148:46] Bud, thanks so much.
[148:47] Yeah, All right, go try it.
[148:48] Yeah, it's available for you all now.