Levi Strauss Cuts App Build Time 72% With 17 GitHub AI Agents

17 agents, one SDLC: How Levi Strauss reimagined development on GitHub — GitHub

GitHub

Artificial intelligence accelerates enterprise software delivery only when companies overhaul handoff-heavy workflows instead of bolting autonomous tools onto legacy processes.

AI-assisted summary of 17 agents, one SDLC: How Levi Strauss reimagined development on GitHub — original source. Check the original for full context.

Source published:

Key ideas

  • Levi Strauss & Co. redesigned its software development lifecycle around 17 specialized AI agents running on GitHub Copilot Cloud Agent and Model Context Protocol (MCP), cutting mobile application build time by 72% from 9 months to 2.5 months. (0:03)
  • The operational framework establishes strict human gates where AI executes tasks while humans retain decision authority, reducing human involvement to just two primary approval checkpoints: spec review and final production merge. (6:47)
  • Rather than causing an initial productivity dip, squad throughput rose 51% immediately alongside a 54% gain in test pass rates, while AI-assisted pull requests jumped from 26% to 69%. (25:36)
  • To prevent merge conflicts caused by agents writing concurrently to pull requests, Levi Strauss decoupled the pipeline so specialized agents leave comments in parallel while a single downstream helper agent synthesizes fixes into a unified commit. (27:30)

Detailed notes

## Background Published on YouTube by GitHub on November 8, 2026, this GitHub Universe 2026 session features Jenny Patchen, Senior Director of Shared Platforms & Services at Levi Strauss & Co. The presentation details how Levi Strauss built an enterprise multi-agent software development lifecycle on GitHub Copilot's custom agents platform, employing 17 specialized agents connected via Model Context Protocol (MCP) to streamline everything from requirements and spec generation to automated testing, security, and deployment.

## TL;DR To overhaul its retail 'Back Pocket' mobile application development, Levi Strauss & Co. re-architected its traditional four-phase, handoff-heavy SDLC into an outcome-oriented system powered by 17 specialized AI agents and GitHub Copilot. Guided by the principle that humans make decisions while agents execute, the architecture routes tasks like EARS spec drafting, regression testing, and security scanning through single-purpose agents linked via MCP, reducing human intervention down to two critical pull request approval gates. The initiative cut application build time by 72%—dropping delivery from nine months to just 2.5 months—while increasing squad throughput by 51% and test pass rates by 54%. Beyond the technical implementation, the transformation required intentional change management, shifting senior developers from hand-crafting code to spec-driven review and proving that adopting AI requires redesigning the factory floor rather than simply swapping in a new motor.

## Atmosphere & style Jenny Patchen delivers a polished, pragmatic enterprise keynote that balances sweeping economic analogies with concrete, battle-tested engineering blueprints. The tone is refreshingly grounded, trading speculative AI hype for candid discussions about sticky-note friction maps, merge conflict bottlenecks, and the psychological hurdles experienced by senior engineers unlearning legacy habits. Watching the presentation feels like sitting through an empowering masterclass in organizational change, marked by clear architectural diagrams, measured confidence, and an overarching respect for developer craftsmanship.

## Bulletin - [00:03](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=3s) Levi Strauss & Co. redesigned its software development lifecycle around 17 specialized AI agents running on GitHub Copilot Cloud Agent and Model Context Protocol (MCP), cutting mobile application build time by 72% from 9 months to 2.5 months. - [06:47](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=407s) The operational framework establishes strict human gates where AI executes tasks while humans retain decision authority, reducing human involvement to just two primary approval checkpoints: spec review and final production merge. - [25:36](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1536s) Rather than causing an initial productivity dip, squad throughput rose 51% immediately alongside a 54% gain in test pass rates, while AI-assisted pull requests jumped from 26% to 69%. - [27:30](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1650s) To prevent merge conflicts caused by agents writing concurrently to pull requests, Levi Strauss decoupled the pipeline so specialized agents leave comments in parallel while a single downstream helper agent synthesizes fixes into a unified commit.

## Key takeaways - [00:03](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=3s) Treating AI adoption like electrifying a factory floor requires redesigning underlying workflows rather than dropping a new motor into legacy, friction-heavy processes. - [04:22](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=262s) Analyzing the legacy Back Pocket application lifecycle revealed that lead time was dominated by human waiting queues and handoffs, notably 2 to 4 weeks drafting stories and 3 weeks of manual QA. - [06:47](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=407s) The team applied a modified McKinsey task framework that strictly delineates 'AI Does', 'Human Does', and 'Human Decides', ensuring accountability stays with human engineers. - [06:47](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=407s) Every agent definition requires five foundational parameters upfront: trigger, input, action, output, and the explicit human handoff gate. - [09:46](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=586s) The architecture keeps agents in existing developer interfaces (VS Code, GitHub.com, Copilot CLI) and decouples external tool integrations through Model Context Protocol (MCP). - [09:46](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=586s) Agent configurations are managed as plain Markdown files in .github/agents/ with YAML frontmatter, subjecting agent prompts and permissions to standard PR review and audit workflows. - [13:23](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=803s) Requirements engineering relies on EARS syntax, and agents are instructed to stop and ask clarifying questions early whenever ambiguities arise to prevent downstream rework. - [13:23](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=803s) Automated quality gates execute in a strict sequential chain—tests, accessibility, performance, code review, and security last—ensuring security scans all prior agent adjustments. - [15:33](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=933s) Agent actions on pull requests generate explicit labeled commits and aggregated executive summaries so developers retain full visibility without parsing fragmented diffs. - [16:27](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=987s) Enterprise standards are packaged into 10 starter kits across 9 languages, backed by Copilot Agentic Auto-Fix to automatically resolve continuous integration failures. - [19:46](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1186s) An internal developer marketplace centralizes 24 custom agents, 19 skills, 15 MCP servers, and prompt libraries with community ratings to maximize organizational reuse. - [20:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1231s) Organizational change management was addressed via dedicated AI Principal coaching, live training on real production code, and open office hours rather than toy sandboxes. - [23:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1411s) Senior, high-performing engineers often faced the greatest learning curves because shifting to spec-driven development required unlearning deeply ingrained manual coding habits. - [25:36](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1536s) Team standups shifted away from manual status updates toward high-bandwidth discussions centered on architecture, critical decisions, and unblocking exceptions. - [27:30](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1650s) Simultaneous parallel agent commits initially triggered severe branch merge conflicts, prompting an architectural pivot to parallel commenting resolved by a single helper agent. - [28:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1711s) Levi Strauss compressed its initial 3-day squad onboarding curriculum down to a 3-hour agile format, providing a scalable model to onboard future teams rapidly.

## Detailed walkthrough ### [00:03](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=3s) Reimagining the Factory Floor: From Steam Motors to Multi-Agent Workflows

Jenny Patchen, Senior Director of Shared Platforms & Services at Levi Strauss & Co., opens her talk with an economic history lesson drawn from economist Paul David. When electricity was first introduced to factories, factory owners simply removed the central steam engine and installed an electric motor in its place, leaving the assembly lines, workflows, and spatial floor plans entirely unchanged. As a result, factory productivity remained stagnant for decades. Real productivity surged only when manufacturers completely rethought factory architecture to exploit the distributed nature of electric power. Jenny draws a direct parallel to the contemporary adoption of artificial intelligence: “And I think that's where we are today with agents. It's the same old factory and a new motor.”

At Levi Strauss & Co., the goal was to avoid that trap by reimagining the entire software development lifecycle (SDLC) on GitHub. The test ground for this initiative was "Back Pocket," the proprietary mobile application used by store stylists on retail floors to assist shoppers—offering critical retail capabilities such as "Endless Aisle" to find inventory across regions. Jenny displays a current-state journey map covered in pink and yellow sticky notes, representing deep organizational friction and pain points across the legacy SDLC. The end result of the transformation was the release of the rewritten Back Pocket app to stylists and fans across Southeast Asia.

- The legacy Back Pocket application took nine full months to build. - The re-architected SDLC enabled the new application to launch in just 2.5 months—a 72% reduction in build time. - The project was not an academic exercise in process redesign; it was driven by delivering real utility to stylists and retail fans in months rather than years.

[02:59](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=179s) Defining Success: Working Backwards from Business Outcomes

To transform the process, the team avoided jumping straight into prompt engineering and instead worked backward from three foundational definitions of success:

1. **Faster Speed to Value:** Eliminating procedural drag and addressing the friction documented on the workshop sticky notes so cross-functional work could ship rapidly without burning out developers. 2. **Higher Quality through Spec-Driven Development:** Introducing automated testing earlier in the lifecycle and leveraging spec-driven development to push ambiguous architecture and design questions far to the left, which directly curtailed production regressions and expensive downstream rework. 3. **Standards Built-In, Not Bolted-On:** The engineering leadership had spent extensive effort codifying best-practice standards for observability, security, and accessibility. However, those standards were historically vulnerable to human omission. As Jenny notes, “We wanted to make the right path, the easy path, and to streamline that for our team.”

[04:22](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=262s) Mapping the Legacy SDLC and Identifying the Long Poles

Before writing a single line of agent configuration, the cross-functional group—including representatives from Product, Engineering, Developer Velocity, Quality Engineering, and the company's AI Principal—convened for an intensive three-day, in-person offsite. Laptops remained closed while the team mapped out workflows on paper. Day one was dedicated to laying out the warts-and-all current state; day two focused on architecting the future-state vision and target metrics; and day three isolated the technical and operational enablers required to execute the vision.

```mermaid flowchart LR accTitle: Legacy Back Pocket SDLC Flow accDescr: The four sequential phases of the legacy SDLC displaying handoffs and major delays across six roles.

subgraph P1["Plan Phase"] A["Write User Story"] end subgraph P2["Build Phase"] B["Feature Development"] end subgraph P3["Verify Phase"] C["Manual QA & Regression"] end subgraph P4["Release Phase"] D["Production Deployment"] end

A -->|"2 to 4 weeks (or months)"| B B -->|"Handoff delays"| C C -->|"3-week bottleneck"| D ```

When redrawn, the legacy delivery pipeline spanned four rigid phases—Plan, Build, Verify, and Release—encumbered by 12 distinct handoffs across six functional roles. The delivery pipeline suffered from two massive "long poles in the tent": - **Writing User Stories:** Drafting user stories regularly dragged out between 2 and 4 weeks, and sometimes stretched into months due to complex cross-functional coordination and misaligned organizational dependencies. - **QA and Regression:** The quality assurance phase required three full weeks at the very tail end of delivery, creating an acute bottleneck that triggered extensive late-stage rework loops.

Analyzing the handoffs revealed that most of the lead time consisted of human beings waiting on other human beings. Jenny highlights the shift in mindset that resulted from this finding: “And so this changed the question we asked ourselves, which then wasn't how do we make humans work faster? But instead, how do we stop the waiting?”

[06:47](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=407s) Redesigning the Operating Model: Humans Decide, Agents Execute

To structure the division of labor between generative models and people, the team adopted and altered a McKinsey task-mapping framework. Every legacy SDLC task was reframed as an outcome rather than a chore. For instance, rather than tracking "writes requirements," the target outcome was designated as "requirements are clear, cross-functional, and aligned."

The team separated work across four explicit operational columns: - **AI Does:** Tasks such as drafting, synthesizing, implementing code, and ongoing monitoring—high-toil activities that require speed and processing power rather than nuanced human judgment. - **Human Does:** Tasks requiring context injection, taste, cross-stakeholder alignment, and creative domain expertise. - **Human Decides:** Approving, rejecting, prioritizing, or escalating decisions.

Jenny stresses the architectural and cultural importance of separating human actions from human decisions: “And this was one of our most important decisions was splitting human decides from human does for the reason that it always kept the accountability with the human.”

| Lifecycle Domain | Desired Outcome | AI Does | Human Does | Human Decides | |:--- |:--- |:--- |:--- |:--- | | **Requirements** | Requirements are clear, cross-functional, and aligned | Drafts testable requirements; captures dependencies; flags ambiguities | Provides raw context and initial business ideas; audits agent drafts | Prioritizes feature backlog; formally approves final requirements |

To implement any specific agent, the team mandated that five operational parameters be defined up front: 1. **Trigger:** The event initiating the workflow (e.g., a feature request arrives). 2. **Input:** The source artifacts (e.g., the Jira ticket details and standardized Levi Strauss spec templates). 3. **Action:** The operational mandate (e.g., drafts testable technical requirements and explicitly interrogates ambiguous criteria). 4. **Output:** The artifact produced (e.g., clear, testable technical requirements ready for build execution). 5. **Handoff:** The human gate (e.g., an engineer or product owner merges the spec-approval pull request).

[09:46](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=586s) The 17-Agent Enterprise Platform Architecture

Jenny presents the technical architecture that underpins this SDLC. The strategy relied strictly on meeting developers in their existing daily environments—such as GitHub.com, VS Code, and the Copilot CLI—rather than introducing fragmented secondary portals.

```mermaid flowchart TD accTitle: Levi Strauss Multi-Agent SDLC Architecture accDescr: Architecture diagram showing developer entrypoints, the central Copilot orchestration engine, 17 specialist agents, and external systems via MCP.

subgraph UI["Developer Interfaces"] IDE["VS Code / GitHub.com / Copilot CLI"] end

subgraph CORE["Execution & Orchestration Engine"] ORCH["Orchestration Agent"] CCA["Copilot Cloud Agent"] STANDARDS["Shared Skills & Enterprise Standards"] ORCH --- CCA CCA --- STANDARDS end

subgraph AGENTS["17 Specialist Agents"] REQ["Requirements Agent"] OBS["Observability Agent"] TEST["Testing Agent"] OTHER["14 Security, Accessibility, & Other Agents"] end

subgraph TOOLS["MCP Server Integrations"] JIRA["Jira / Confluence"] NR["New Relic"] PW["Playwright"] MISC["Figma / GitHub / Context7"] end

IDE --> ORCH ORCH --> REQ & OBS & TEST & OTHER REQ <-->|"MCP"| JIRA OBS <-->|"MCP"| NR TEST <-->|"MCP"| PW OTHER <-->|"MCP"| MISC ```

The system comprises 17 specialized AI agents routed by a central orchestration agent. Rather than creating generic assistants, Levi Strauss created single-purpose agents: - The **Requirements Agent** extracts Jira context to generate comprehensive technical specs. - The **Observability Agent** verifies and injects application instrumentation, querying telemetry via New Relic. - The **Testing Agent** authors, runs, and updates browser-level regression suites using Playwright. - **14 Additional Specialized Agents** manage accessibility validations, automated security analysis, and domain-specific code hygiene.

All agents run on top of GitHub Copilot Cloud Agent as their execution engine, decoupling integrations via the Model Context Protocol (MCP). Jenny emphasizes the enterprise maintainability of this approach: “And importantly, when these systems change out, we change the connection. We don't change the agent.”

Under the hood, each agent is defined as a simple Markdown configuration file stored inside the root `.github/agents/` folder at the organizational level. This makes all 17 agents instantly available across every repository in the company with zero setup required by developers. Each file contains YAML frontmatter defining metadata and MCP tool declarations (such as Jira or Confluence), followed by clear plain-text natural language instructions detailing what the agent is authorized to do and what it is explicitly barred from doing.

Changes to agent definitions are treated identical to production software changes: they require a pull request, code review, automated testing, versioning, and an audit trail. Because the definitions are plain Markdown, any developer, QA engineer, or product manager can inspect the system prompts and submit PRs to improve agent behavior.

[13:23](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=803s) The Multi-Agent Workflow and Human Gates

In Jenny Patchen's visual diagram of the software development lifecycle, human touchpoints are highlighted in white, while incoming signals appear in orange. The process initiates with the requirements agent, which kicks off the requirements specification and test planning using EARS (Easy Approach to Requirements Syntax), a structured syntax designed to keep automated tests clean and unambiguous. Patchen notes that from day one, Levi Strauss & Co. actively instructed their agents to halt and raise questions whenever specifications were unclear: “And it's one thing I also want to call out here is we really encouraged our agents to ask when anything was ambiguous, and they needed further guidance because it stopped rework later in the process.”

Once requirements take shape, the specification is submitted as a pull request for human sign-off. As soon as the engineer approves and merges the spec PR, the GitHub Copilot cloud agent steps in to spin up the actual build and open a subsequent pull request. Downstream specialists are brought into the loop—though Levi Strauss does not execute all 17 specialists on every single run; they are summoned only when relevant. Following the specialists, the automated quality gates fire in a non-negotiable sequence: testing, accessibility, performance, code review, and lastly, security. Security intentionally runs at the very end of this chain so that it evaluates every single automated modification made by preceding agents.

Next, an observability agent inspects the pull request to verify that all necessary tracing is added for New Relic telemetry. Once ready for rollout, an optional deployment agent provides recommendations on the safest release strategy. Only then does the flow circle back to a person: the engineer inspects the final output and merges to production, backed by continuous New Relic telemetry in runtime. Patchen underscores how narrow the human surface area has become: “And what I really want to call out here is that, you see, there's only two aspects where the human is really involved. Everything else is running by the agent's end to end.”

```mermaid flowchart TD accTitle: Multi-Agent SDLC Workflow at Levi Strauss accDescr: An end-to-end SDLC showing automated agents executing between two strict human review checkpoints.

subgraph Phase1["Phase 1: Requirements & Spec Gate"] Req["Requirements Agent<br/>(EARS Syntax)"] -->|"Spec PR"| HGate1{"Human Gate 1:<br/>Approve Spec PR"} end

subgraph Phase2["Phase 2: Build & Specialist Work"] HGate1 -->|"Merged"| CopilotBuild["Copilot Cloud Agent<br/>(Build & Opens PR)"] CopilotBuild --> Specialists["Optional Specialists<br/>(Up to 17 Agents)"] end

subgraph Phase3["Phase 3: Sequential Quality Gates"] Specialists --> QG1["1. Tests"] QG1 --> QG2["2. Accessibility"] QG2 --> QG3["3. Performance"] QG3 --> QG4["4. Code Review"] QG4 --> QG5["5. Security (Always Last)"] end

subgraph Phase4["Phase 4: Telemetry & Production Gate"] QG5 --> ObsAgent["Observability Agent<br/>(New Relic)"] ObsAgent --> DeployAgent["Optional Deployment Agent<br/>(Recommendations)"] DeployAgent --> HGate2{"Human Gate 2:<br/>Review & Merge PR"} HGate2 -->|"Deploy"| Prod["Production Run<br/>(Continuous Telemetry)"] end ```

[15:33](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=933s) Pull Request Experience: Labeled Commits and Summaries

Reviewing agent-generated work on GitHub is designed to be frictionless for the developer. Walking through a real commit history from a developer portal migration to Next.js, Patchen highlights how the initial build and execution plan receive clear, labeled commit updates from each functional agent—including testing, code review, accessibility, and security fixes. Rather than forcing engineers to parse raw diffs across multiple commits, the workflow collates their findings into an executive overview:

- **Labeled Commits:** Each agent writes fixes straight back into the branch with an explicit label indicating what changed and which agent executed it. - **Consolidated Summaries:** The platform aggregates individual agent findings into a single unified summary inside the PR description so developers immediately grasp every automated adjustment. - **Driver's Seat Philosophy:** The human developer retains final authority to approve or reject the final bundle before anything is merged.

[16:27](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=987s) Enforcing Standards: Security, Accessibility, Performance, and Starter Kits

Levi Strauss translates organizational standards into programmatic agent rules across four key disciplines. The security agent audits dependencies and runs secret detection aligned with OWASP guidelines and CVE databases, resolving vulnerabilities based on calculated severity. Accessibility compliance targets WCAG 2.2 AA standards, validated directly inside real headless browsers via Playwright and remediated in-branch. The performance agent also leverages Playwright to evaluate Core Web Vitals, operating under a strict engineering rule: performance must always be measured directly rather than estimated before remediation occurs. Finally, the observability agent ensures applications never ship without essential runtime instrumentation.

To scale these disciplines across the enterprise, Levi Strauss runs an agent strategy side-by-side with a standardized repository strategy. They built 10 organization-wide repo starter kits—covering 9 of their most common engineering languages from TypeScript to Swift—embedding CI/CD best practices, runbooks, documentation, `agents.md` context files, and Copilot instruction files from day one. High-velocity development is backed by strict automated controls:

> “And you can only go as fast as you have good brakes. And that's why we've built robust gates in our CI.”

These continuous integration checks enforce dependency reviews, coverage metrics, and New Relic preflight validation on every pull request. If an error surfaces during CI, Copilot Agentic Auto-Fix investigates the failure, devises a resolution, and submits a fix back to the pull request for engineer approval. For legacy repositories, the same tooling operates retroactively: an agent analyzes the existing codebase, contrasts it against the official starter kit templates, pinpoints structural deficiencies, and generates remediation pull requests for engineering teams to inspect.

[19:46](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1186s) The Internal Marketplace: Custom Agents, Skills, and MCP Servers

To avoid duplicate engineering and democratize tooling, Levi Strauss provides an internal marketplace structured into four distinct categories: Custom Agents, Skills, Prompts, and Model Context Protocol (MCP) servers. Exposed directly through GitHub and VS Code, the marketplace operates as an internal catalog where teams can consume, rate, and review tools with single-click availability.

| Shelf Category | Total Available | Purpose and Highlights | |:--- |:--- |:--- | | **Custom Agents** | 24 agents | Specialized role-based bots covering functional domains across the SDLC. | | **Skills** | 19 skills | Modular abilities and actions leveraged by Copilot agents during task execution. | | **MCP Servers** | 15 servers | Standardized Model Context Protocol connections into enterprise systems like Jira and New Relic. | | **Prompts** | Curated catalog | Reusable prompt templates tested and validated by internal engineering teams. |

Engineers leave community reviews and star ratings on every asset, driving organizational reuse, standardizing delivery baselines, and shortening time to value for downstream teams.

[20:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1231s) Change Management and AI Adoption Strategies

Patchen emphasizes that engineering the technology was simple compared to the human side of the rollout: “The tooling was really the easy part and the change management was the heavy lift in terms of the work.” When roles undergo seismic shifts, team members inevitably experience anxiety about their future responsibilities. To address these concerns and cultivate trust, Levi Strauss designed a multi-tier enablement program:

- **Industry Research Pre-reads:** Prior to day one of technical workshops, developers received curated reading materials featuring enterprise blueprints from leading engineering organizations alongside research from Gartner and McKinsey, encouraging them to pursue systemic transformation over small incremental gains. - **Embedded Principal AI Engineer:** A senior engineer functioned as a dedicated player-coach, embedding into team standups, hanging out in daily chat channels, running coaching sessions, and troubleshooting blockers shoulder-to-shoulder with developers. - **Hands-on Learning with Production Code:** Engineers trained directly on their active production repositories rather than abstract toy exercises in synthetic sandboxes. - **Developer Velocity Office Hours:** Open weekly forums established a safe space for developers to ask questions, explore newly released features, and observe peer demonstrations. - **Dedicated AI Hackathons:** A company-wide competition mandated AI utilization for all participants, preceded by technical upskilling; several solutions built during the event successfully shipped to production.

[23:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1411s) The Four Operating Rules and Managing Senior Engineering Friction

To settle technical debates and guide day-to-day work, Levi Strauss established four memorable operating tenets:

1. **Humans as judgment, agents on execution:** Developers handle ambiguous architectural decisions, taste, and critical thinking, while delegating routine, repetitive labor to agents. 2. **Context lives where agents can see it:** Repositories and tickets are durable storage; conversational chat is ephemeral. While chat will eventually fold into long-term context, the current framework relies strictly on tracked artifacts. 3. **The same bar for agents and humans:** Automated agents and human contributors pass through the exact same quality gates, code reviews, and testing thresholds. 4. **Auditable always:** Every autonomous agent action, commit, and human override is captured in persistent audit logs.

Adopting this mental model proved especially challenging for seasoned contributors. Citing a Korn Ferry white paper, Patchen observed that senior, high-performing engineers often faced the steepest learning curve when transitioning to AI-assisted workflows:

> “Because if you've been successful your entire career with a set of skills, it becomes very hard to walk away from those skills which have continually given you that success and set you ahead in your organization.”

Unlearning traditional coding habits required teams to deliberately build new rituals around spec-driven development and train developers to critically evaluate agent output rather than hand-crafting every implementation detail from scratch.

[25:36](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1536s) Measuring Impact: Throughput, Quality, and Adoption Metrics

Jenny Patchen breaks down the cultural and quantitative transformations within the engineering squads following the rollout of the multi-agent system. The day-to-day rhythm of team collaboration shifted noticeably: standup meetings were fundamentally altered so that they no longer centered on manual status reports. Instead, standups turned into high-bandwidth discussions focusing strictly on open architectural debates, active decision-making, and addressing exceptions. This shift allowed engineers to spend less time relaying updates and more time solving structural blockers together.

The initial rollout defied conventional engineering expectations regarding productivity dips during major organizational changes. Leadership originally anticipated that squad throughput would decline early on, recognizing that adapting to an entirely new multi-agent paradigm was a massive transition requiring runway for teams to process. Instead, throughput jumped immediately by 51%. Concurrently, test pass rates improved by 54%, demonstrating that automated agents were elevating baseline software quality rather than trading off craftsmanship for sheer output. As Jenny Patchen explains: “And in particular, you can think of velocity and quality often as a paradox. Where velocity declines and quality improves or quality declines and velocity improves. It's really irregular that both could increase together.”

Patchen addresses the common skepticism that higher throughput metrics can simply reflect generative agents producing bloated code for the sake of activity rather than real business value. She highlights that Levi Strauss & Co. delivered the new minimum viable product (MVP) substantially faster than the legacy MVP benchmark, validating that productivity matched concrete delivery outcomes. Developer adoption surged in tandem: while squads started with 26% of pull requests assisted by AI—having already explored early generative tools—that number rapidly surged to 69% over a concise observation window, with the ongoing trajectory moving toward full 100% adoption across future initiatives.

- **51% throughput improvement**: Squad velocity increased immediately out of the gate despite initial expectations of an adaptation-related productivity dip. - **54% automated pass rate**: Test suite success rates rose simultaneously, breaking the traditional velocity-versus-quality paradox. - **69% AI-assisted PR adoption**: Increased from a 26% baseline at the start of the engagement, with ongoing progress aimed at 100%. - **Outcome validation**: Verified that the accelerated pace represented actual product delivery of the rebuilt Back Pocket app, far outpacing the legacy release cycle.

---

[27:30](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1650s) Architectural Evolution: Eliminating Merge Conflicts and Agent Runaway

Reflecting on technical lessons learned, Patchen reveals how the team continuously iterated on their pull request review pipeline. The initial architectural ambition was to execute all automated PR agents in parallel to maximize evaluation speed. However, multiple autonomous agents independently updating files simultaneously generated persistent merge conflicts across branches. The team initially responded by pulling back from concurrent writes to manage branch stability, but quickly devised a cleaner, decoupled architecture.

In the modernized setup, the specialized verification agents—handling domains like security, accessibility, and code standards—run concurrently but strictly leave review comments instead of committing direct modifications. A dedicated downstream "helper agent" then ingests all generated review comments at the end of the pipeline, reconciles the feedback holistically, and applies a unified set of code fixes back to the branch. This eliminated merge conflicts while retaining the operational speed of independent parallel evaluations.

Patchen also highlights the necessity of designing systems where ambiguity surfaces early. Because autonomous agents frequently rush forward with complete confidence and generate outputs misaligned with human developer intent, the framework was architected to force early clarity checks rather than permitting unsupervised completion of ambiguous tasks.

```mermaid flowchart TD accTitle: Multi-Agent PR Review and Resolution Flow accDescr: Independent review agents comment on pull requests in parallel, and a downstream helper agent reconciles comments into a single unified commit.

subgraph Evaluation ["Parallel Review Phase"] A["Pull Request Branch"] --> B["Security Agent"] A --> C["Accessibility Agent"] A --> D["Quality & Spec Agent"] end

subgraph Feedback ["Feedback Aggregation"] B -->|"posts inline comments"| E["PR Comment Stream"] C -->|"posts inline comments"| E D -->|"posts inline comments"| E end

subgraph Resolution ["Automated Remediation"] E --> F["Helper Agent"] F -->|"resolves feedback & creates single commit"| G["Clean PR Branch Updated"] end ```

---

[28:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1711s) Scaling Adoption and the Philosophy of Process Reimagination

Scaling the agentic framework required streamlining the intensive onboarding curriculum used to train engineering squads. The inaugural onboarding workshop represented an enormous upfront investment: three solid days in a dedicated room, backed by an equivalent number of hours in pre-workshop preparation and significant post-session follow-through. While this hands-on immersion produced strong team results, it was not sustainable to scale across the broader enterprise.

Levi Strauss & Co. re-engineered the onboarding process into an agile format. Subsequent squad onboardings condensed the curriculum progressively down to as few as three hours, while maintaining the same positive behavioral and technical outcomes. This compressed format established a scalable playbook, allowing upcoming squads to transition into the agentic SDLC with far lower friction and quicker time-to-value.

Jenny Patchen ties the technical architecture back to the original mission of rebuilding the Back Pocket mobile retail platform. The project began with a legacy value stream map overwhelmed by systemic handoffs and operational friction, and culminated in a completely rebuilt, modern application deploying directly into Southeast Asia markets, achieving an overall 72% end-to-end process improvement.

Patchen concludes with a core philosophical warning for any organization looking to implement AI development agents: “Reimagine your process first. Don't put the new motor in the old factory.” Pointing back to the company’s history, she reflects on the parallel between physical craftsmanship and digital engineering: “In 1873, we reimagined workwear. This year, we reimagined software with our amazing engineering team at LS&Co.”

- **Workshop compression**: Reduced squad onboarding duration from a 3-day intensive workshop down to a streamlined 3-hour session without degrading implementation outcomes. - **Back Pocket deployment**: The transformed software delivery process powered the successful release of the new retail app in Southeast Asia. - **72% end-to-end efficiency improvement**: Achieved across the entire software delivery lifecycle from concept to release. - **Core implementation principle**: Automated multi-agent platforms require redesigning underlying operational processes rather than superimposing AI tooling onto legacy development habits.

## Notable quotes > [28:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1711s) Reimagine your process first. Don't put the new motor in the old factory. — Jenny Patchen

> [20:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1231s) The tooling was really the easy part and the change management was the heavy lift in terms of the work. — Jenny Patchen

> [16:27](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=987s) And you can only go as fast as you have good brakes. And that's why we've built robust gates in our CI. — Jenny Patchen

> [23:31](https://www.youtube.com/watch?v=-GpxKKgoWWg&t=1411s) If you've been successful your entire career with a set of skills, it becomes very hard to walk away from those skills which have continually given you that success and set you ahead in your organization. — Jenny Patchen

## Mentioned Jenny Patchen, Paul David, Levi Strauss & Co., GitHub, GitHub Copilot, GitHub Copilot Cloud Agent, Copilot CLI, VS Code, Back Pocket, Model Context Protocol (MCP), Jira, Confluence, New Relic, Playwright, Figma, Next.js, McKinsey, Gartner, Korn Ferry, OWASP, EARS

## Video description Levi Strauss & Co. built an enterprise multi-agent software development lifecycle on GitHub Copilot's custom agents platform. In this GitHub Universe 2026 session, Jenny Patchen, Sr. Director at Levi Strauss & Co., reveals how seventeen specialized AI agents handle requirements, spec generation, automated testing, security scanning, accessibility validation, and deployment connected to Jira, Confluence, and New Relic through Model Context Protocol (MCP). Every pull request is reviewed by 5 to 11 independent agents that automatically fix issues and commit back to the branch, while retaining strict human approval gates. Explore the enterprise architecture, lessons learned, and practical blueprints for adopting a multi-agent SDLC across your organization.

Speaker: Jenny Patchen, Sr. Director, Shared Platforms & Services, Levi Strauss & Co.

Watch more videos from GitHub Universe 2026: https://www.youtube.com/watch?v=AArI948SDz8&list=PLENrlACZ7mnU

— CHAPTERS —

00:00 Introduction to Levi Strauss & Co. 01:14 Reimagining the back pocket app SDLC 02:59 Defining success and objectives 04:22 Rewiring the process 09:46 Enterprise multi-agent architecture 16:44 Enforcing standards and quality gates 19:47 Internal marketplace and reusability 20:32 Change management and team adoption 24:56 Measuring impact and lessons learned 29:11 Conclusion and future outlook

#GitHubUniverse #GitHubCopilot #AIAgents

Stay up-to-date on all things GitHub by connecting with us:

YouTube: https://gh.io/subgithub Blog: https://github.blog X: https://twitter.com/github LinkedIn: https://linkedin.com/company/github Insider newsletter: https://resources.github.com/newsletter/ Instagram: https://www.instagram.com/github TikTok: https://www.tiktok.com/@github

About GitHub It’s where over 180 million developers create, share, and ship the best code possible. It’s a place for anyone, from anywhere, to build anything—it’s where the world builds software. https://github.com

[Source](https://www.youtube.com/watch?v=-GpxKKgoWWg)

Reading summary

Jenny Patchen, Senior Director of Shared Platforms & Services at Levi Strauss & Co., argues that realizing productivity gains from artificial intelligence requires fundamentally redesigning the software development lifecycle rather than superimposing agents onto legacy processes. By restructuring delivery around 17 specialized agents governed by clear human decision gates, the organization reduced build times for its proprietary retail application by 72%. Re-architecting the SDLC Around Human Decision Gates (4:22) Levi Strauss tested its development overhaul on Back Pocket, an in-store mobile application whose legacy build required nine months of development. Value stream mapping revealed that delivery was encumbered by 12 distinct handoffs across six functional roles, creating substantial lag: drafting user stories took between two weeks and several months, while manual QA created a three-week bottleneck at the tail end. Rather than attempting to speed up individual tasks, the team focused on eliminating idle waiting time by adopting a structured framework that differentiated high-toil execution from governance. The resulting operating model divides all development activities into three explicit columns: tasks delegated to autonomous agents (drafting specifications, implementing code, continuous monitoring), human context injection (business domain context and architectural taste), and human decisions (approving requirements, prioritizing backlogs, and merging production builds). Human involvement was deliberately consolidated into two primary checkpoints: approving the spec pull request generated in structured EARS syntax, and reviewing the final production release. Modular Multi-Agent Architecture and MCP Decoupling (9:46) The technical platform consists of 17 single-purpose agents coordinated by a central orchestration engine running on GitHub Copilot Cloud Agent. Specialist bots handle discrete domains—such as extracting Jira context to author requirements, authoring Playwright regression suites, and instrumenting New Relic telemetry—while external tooling integrations are decoupled via the Model Context Protocol (MCP). Patchen emphasizes that this abstraction layer ensures changing external tools requires modifying connection parameters rather than rewriting agent logic. Each agent is declared as a plain Markdown configuration file within the repository's root directory, incorporating YAML frontmatter for metadata and tool access alongside plain-text constraints. Agent definitions are versioned and audited like production code, allowing engineers and product managers to propose behavior modifications via standard pull requests. During build execution, downstream quality verification follows a strict, non-negotiable sequence: automated unit testing, accessibility checks, performance profiling, code review, and final security audits. Eliminating Merge Conflicts and Enforcing Standards (16:27) Early iterations of the multi-agent pipeline attempted to run all review specialists in parallel with direct write access, which triggered persistent branch merge conflicts. The team resolved this by updating specialized review agents—including security, accessibility, and code quality—to execute concurrently but strictly submit review comments. A downstream helper agent aggregates these comments at the end of the pipeline, reconciling findings to push a single, clean remediation commit with consolidated change summaries in the pull request description. To institutionalize engineering standards across repositories, Levi Strauss deployed 10 organizational starter kits covering nine primary languages. These templates embed pre-configured CI/CD pipelines, runtime telemetry validation, and `agents.md` context files. Standardized pull request gates enforce OWASP security compliance, Core Web Vitals performance benchmarks, and WCAG 2.2 AA accessibility validations in real headless browsers, backed by automated agentic auto-fixes when preflight checks fail. Change Management and Measured Delivery Gains (20:31) Patchen observed that cultural change management posed a larger operational hurdle than technical tooling, with high-performing senior engineers often facing the steepest learning curve as they unlearned manual implementation habits. The company mitigated friction by embedding a dedicated Principal AI Engineer directly into daily squad standups, running hands-on training sessions directly in active production codebases, and establishing an internal marketplace of verified custom agents, skills, and prompt templates. Despite initial executive forecasts of a temporary productivity dip, the engineering squad experienced an immediate 51% throughput improvement and a 54% increase in automated test pass rates, while AI-assisted pull request adoption grew from 26% to 69%. The rewrite of the Back Pocket application was completed in 2.5 months for deployment in Southeast Asia, and the team subsequently compressed the squad onboarding curriculum from an intensive three-day offsite to an agile three-hour session without degrading implementation outcomes. Levi Strauss & Co.'s rollout demonstrates that multi-agent development can simultaneously improve delivery velocity and baseline code quality when underpinned by spec-driven development, modular MCP tooling, and persistent human auditability. However, Patchen cautions that these efficiencies depend on addressing organizational resistance and completely overhauling legacy handoffs, warning that installing autonomous agents into an unexamined process merely replicates the historic trap of placing a new motor into an unchanged factory.

Source quotes

[28:31] Jenny Patchen: Reimagine your process first. Don't put the new motor in the old factory. [20:31] Jenny Patchen: The tooling was really the easy part and the change management was the heavy lift in terms of the work. [16:27] Jenny Patchen: And you can only go as fast as you have good brakes. And that's why we've built robust gates in our CI. [23:31] Jenny Patchen: If you've been successful your entire career with a set of skills, it becomes very hard to walk away from those skills which have continually given you that success and set you ahead in your organization.

Original transcript

[0:03] Please welcome Jenny Patchen. [0:14] Hi everyone, I'm Jenny Patchen. [0:16] I'm so excited to see you here today at Universe. [0:19] Thanks for joining for my talk. [0:21] So I'd like to start with a little story that you may not have heard. [0:27] Economist Paul David identified that [0:30] when electricity first came to factories, nothing changed. [0:35] The factory floor stayed the same. [0:38] The workflows stayed the same, and productivity unfortunately also stayed the same. [0:45] They took the old steam motors and they replaced it with the electrical motors [0:51] and they called it a day. [0:53] It actually took decades for productivity to surge. [0:58] And the reason why that happened is [1:00] because they finally reimagined how the factories should work with electricity. [1:06] And I think that's where we are today with agents. [1:09] It's the same old factory and a new motor. [1:13] I'm from Levi Strauss and Company, [1:16] and I'm excited to talk to you about how we've reimagined development on GitHub. [1:24] Back pocket is our app for store stylists. [1:27] It's what they use on the floor to help support our fans who are shopping. [1:33] It provides capabilities like Endless Aisle. [1:36] So on the left hand side here, you can see the current state map. [1:43] This was the beginning of our SDLC before we got started. [1:47] And all of those post-it notes, you see, those are all pain points. [1:52] So we had a lot of opportunities. [1:53] We wanted to improve through the process. [1:57] What you see on the right hand side is the new back pocket app launching [2:02] in Southeast Asia to support our stylists and fans there. [2:07] This talk is about the journey from the first picture to the second, [2:12] which took us about two and a half months. [2:15] Now that sounds fast, but how fast is it actually? [2:23] 72%. The Legacy Back Pocket app took us nine months to build. [2:30] The new version, as I mentioned, took us two and a half, [2:33] which was a huge improvement. [2:35] But we didn't set out to redesign this SDLC for its own sake. [2:41] We wanted to make sure that a stylist working on the floor had a better tool [2:46] in months, not years, and a fan that stylist was supporting was having [2:51] a better experience in months, not years. [2:56] So how did we get started? [2:58] First and foremost, we started from our definition of success [3:02] and we worked backwards. [3:05] Faster speed to value was one of the first outcomes we knew we wanted to achieve. [3:10] You saw our post-it notes prior. [3:12] I'll show you a little bit more as we move forward. [3:15] But we knew that there were inefficiencies in the process. [3:18] Although the team was passionate and hard working, [3:21] we knew that there was a lot more we could do to improve, [3:24] and we thought agents could help us. [3:29] Higher quality. So of course, automated testing. [3:33] We knew we could drive improvements there. [3:36] We also knew that spec driven development could help us reduce [3:40] ambiguity by shifting some of those conversations to the left, [3:44] all of which would help reduce any impact on production [3:48] and also drive improvements in rework. [3:53] Last. We wanted standards built in, not bolted on. [3:57] Our team has done a ton of work [3:59] in terms of activating standards across the organization [4:03] and defining what good looks like for observability, accessibility, security, [4:07] and other key areas. But all of that only works if an engineer [4:11] always remembers to apply it. [4:13] We wanted to make the right path, the easy path, and to streamline that for our team. [4:21] So we're going to talk about this in three pieces today. [4:24] First rewiring the process, then about the platform and last, [4:29] how we bring it all together. [4:30] But let's start with process. [4:32] Since this is the part that most AI talks skip. [4:37] So we started in one room together for three days with our workshop. [4:43] And I know you see a few laptops there, but for the most part they really were down. [4:47] And we were focused in terms of ideating on paper. [4:51] And we started with all key members of the team product engineering, [4:55] developer velocity, quality engineering. [4:58] Our principle for AI. [5:01] And we sat and we worked. [5:03] Day one was really focused in terms of that current state map. [5:07] Day two was focused in terms of our future state vision [5:11] and the outcomes we wanted to achieve. [5:13] And day three was all of the enablers that we needed to put together to get [5:17] to the start, to the finish, and achieve what we knew we wanted to. [5:24] In a moment, I'm going to show you our current state map redrawn. [5:28] There's four sections. [5:34] First plan. Next build. [5:38] Then verify and release. [5:41] And you can see here there's quite a lot of steps. [5:44] There's 12 handoffs over six roles. [5:47] And this process often took months end to end for the team to do. [5:51] And there were really two long poles in the tent. [5:54] First writing the user story and second, QA and regression. [6:00] Writing the user story often took 2 to 4 weeks and again, sometimes months, [6:06] depending on all of the cross dependencies [6:08] and cross coordination that had to happen across the organization. [6:12] QE and regression. That usually took three weeks [6:16] and it always happened at the end of the process. [6:18] Introducing friction and rework. [6:21] And what you can see here is that a lot of this process is really focused on waiting. [6:27] And so this changed the question we asked ourselves, [6:30] which then wasn't how do we make humans work faster? [6:33] But instead, how do we stop the waiting? [6:37] And we knew agents could help us do that. [6:39] And once we could see all of this, we could truly start redesigning the process. [6:47] So one of our key tenants was always agents start on execution [6:52] and humans make the decision. [6:54] And what you see here is a revised McKinsey model. [6:58] Now, all of the steps that you just saw on the preceding slide, [7:02] those were all became mapped to outcomes. [7:05] Now a step as an example would be rights requirements. [7:10] An outcome would be the requirements are clear, cross-functional and aligned. [7:16] So you have your first box and you move to the left. [7:20] So AI does then becomes your next column and your next cell to fill out. [7:25] And that's things like synthesizes, drafts, implements, and monitors, [7:29] areas where we know agents are going to be the strongest [7:32] and that they can help take a lot of that lift off the team [7:36] for activities that really don't warrant the judgment of a human being. [7:41] Next, we have human does and this is areas like adds context. [7:45] Judges manages stakeholders all of the areas where critical thinking, taste, [7:51] judgment. They all come to the forefront as skills that humans really bring [7:55] to the table. And last, we had humans decide, approves, rejects, works with, [8:02] approves, rejects and escalates. [8:05] And this was one of our most important decisions was splitting [8:08] human decides from human does for the reason that it always kept the accountability [8:13] with the human. And that was very clear. [8:15] The human always remains in the driver's seat. [8:21] So next we're going to walk through one example of this, [8:24] of what requirements looks like. [8:28] So as we talked about, requirements are clear, aligned and cross-functional. [8:35] What AI does, if you look [8:36] at the example of requirements is writes testable [8:39] requirements, captures dependencies. [8:43] What the human does brings the ideas. [8:47] Audits what the agent wrote. [8:51] And what the human does is prioritizes. [8:53] Approves the requirements that AI does sell is what becomes our requirements agent. [9:00] So let's start breaking that apart in more detail. [9:06] There's really five things you have to be able to answer [9:09] before you start building agents. [9:11] So let's talk through those first. [9:14] And continuing with our example of the requirements agent. [9:18] That would be the trigger which here would be a feature requests arrives. [9:22] The input would be the request the ticket. [9:25] Our spec template the action writes testable requirements asks [9:31] when ambiguous the output clear testable requirements the whole [9:36] team can build towards and the hand off a human merges the spec approval PR. [9:45] Now let's get into the nuts and bolts of the platform. [9:51] It wouldn't be an engineering conversation if we didn't have one [9:54] architecture diagram. So here's what we built. [9:58] You can see on the left hand side where developers work. [10:01] And it was important to us to really start [10:04] in the tools our engineering team was already using. [10:07] So this was tools like GitHub, Copilot CLI, VS Code. [10:15] We had the orchestration agent routing work across the team, [10:21] and the team of agents was really the 17 specialist agents, [10:26] the requirements agent who, of course, draft specs from the ticket, [10:30] the observability agent who instruments and investigates, [10:34] the testing agent who writes and runs the tests. [10:38] But there were 14 more that we incorporated in areas like security and accessibility. [10:47] We also had our MC, and these were the tools that our team was already using. [10:52] So the requirements agent is of course using Jira as the record of work. [10:57] The observability agent is using New Relic for telemetry and alerts. [11:02] The testing agent is using playwright to drive the browser for their tests, [11:06] and we had more such as GitHub, context seven and Figma. [11:10] And importantly, when these systems change out, we change the connection. [11:15] We don't change the agent. [11:19] Copilot cloud agent acts as the engine for all of these agents [11:24] because they're fundamentally specialized Prompts that run on Copilot cloud agent [11:32] and underpinning everything. [11:33] We had skills in place to help drive reusability of our standards and best practices. [11:42] So this may already look very familiar to many of you. [11:47] But let's walk through it. [11:48] So we're all working off a common understanding. [11:51] And an agent of course is really just a markdown file. [11:54] And you can see an example here of the requirements agent we've been following [11:58] throughout our conversation. [12:00] And there's a short header saying what the agent is, plus plain text instructions. [12:05] And that plain text instructions is really important [12:08] because it means anyone on the team can read this, [12:11] anyone on the team can understand it, [12:13] and anyone can also provide recommendations on how to improve it. [12:19] And of course, everything that's here says very clearly what the agent is, [12:23] what it can do, and what it can't do. [12:28] All of our agents are in the agent's folder on the GitHub repo, [12:33] and that means it's automatically in every repo [12:36] because it's at the organizational level with zero setup. [12:42] We manage our agent changes like data, like code changes, [12:46] with all of them going through a PR. [12:48] They're all reviewed, versioned and auditable, [12:52] and it's the same profile on github.com. Copilot CLI, CLI and VS Code. [13:00] And you can also see here the MCP Servers are declared in the profile. [13:04] There's Jira and Confluence as tools with this agent can use. [13:09] Now an agent on its own is great, [13:11] but where they really start to become powerful is [13:14] when you chain them together in your workflow. [13:18] Now, if you're looking at this slide, agents are the purple line, [13:23] humans are the white box, and our signals are in orange. [13:27] So first, starting with the requirements agent, which of course, [13:30] as we've talked about is kicking off our requirements spec and test. [13:34] And we've done this and we've been using ears for our requirements, [13:39] which is easy approach to requirement syntax that helps make [13:42] tests very straightforward. [13:45] And it's one thing I also want to call out here is we really encouraged our agents [13:50] to ask when anything was ambiguous, [13:53] and they needed further guidance because it stopped rework later in the process. [13:58] Once this agent kicks this off, [14:00] it goes to the human to approve the spec PR it then goes once they approve it [14:06] and the spec is merged to the Copilot cloud agent [14:10] to kick off the build and open that PR. [14:14] After that, it goes to the specialists and if this is. [14:20] These are often optional. [14:21] We don't run all 17 specialists every single time. [14:26] Then it goes to the quality gates and they always run [14:29] and they always run in this order, which is Tess accessibility Performance, [14:34] code review and security. [14:36] And importantly, security always runs last [14:39] so that it sees all of the changes that every other agent has made in the process. [14:46] Then it goes to the observability agent who instruments [14:49] and makes sure that everything is in line [14:51] so that we have full visibility in New Relic. [14:54] Once this code is deployed, [14:56] we have a deployment agent, [14:58] which is optional and makes recommendations to the engineer on the best way [15:03] to deploy. [15:05] At this point, it goes back again to the human being to review [15:09] and merge the PR before it goes to production with, again, that New Relic telemetry, [15:14] which helps ensure that we have a continuous cycle of improvement. [15:18] And what I really want to call out here is that, you see, [15:21] there's only two aspects where the human is really involved. [15:25] Everything else is running by the agent's end to end. [15:33] Now let's go through what the reviewer actually sees. [15:36] And this is an example commit history for our Next.js migration [15:41] for our developer portal. [15:42] I'm going to show you in a moment. [15:44] And you can see here the initial build the plan [15:47] and you can see feedback from all of the agents, including security, accessibility, [15:52] code review and testing. [15:54] And each one of these agents has provided a fix with a labeled commit. [16:02] You can't see it on this page. [16:04] But one thing we also provide our engineers is a consolidated [16:07] summary from each of these agents. [16:09] So it's very straightforward for the engineer [16:12] to understand exactly what's been changed. [16:17] This is because, of course, the human being remains in the driver's seat, [16:20] as we've talked about, and they then approve and merge. [16:27] Now we've talked a little bit about the importance of standards [16:30] and how all the work that we've done to try [16:32] to activate these across the organization. [16:36] So let's dive into these a little bit and how we've incorporated it with our agents. [16:43] So first security. This agent ensures that best practices for Owasp [16:49] and common vulnerabilities and exposure are in place. [16:52] It audits and conducts secret scans which are fixed by severity. [16:59] Accessibility. This agent checks for wcaG 2.2 AA compliance. [17:06] They're checked in a real browser using playwright, and then it's fixed in the PR. [17:13] Performance. This agent uses playwright as well to measure according [17:18] to Core Web vitals, how our applications are performing, and importantly, [17:23] we always want to make sure this is measured, never estimated before it's fixed. [17:30] And again, we have observability. [17:32] This makes sure that any missing instrumentation is added. [17:40] And what we're trying to do is ensure that every time we have a new Repo, [17:44] the team is set up right from the start [17:46] to incorporate our standards in the right way. [17:49] So our agent strategy runs in parallel with our Repo strategy. [17:54] And our Repo templates are really just starter kits that are made [17:57] available across the organization, [18:00] so that we have one consistent degree of engineering baseline. [18:05] Now we have ten starter kits, [18:07] nine of which are associated [18:08] with some of the most common programing languages that Ellis [18:11] and co like TypeScript to Swift. [18:14] And these help enforce best practices for CI, [18:17] CD pipeline documentation and runbooks from day one. [18:23] We have agents built in with agents.md files and Copilot instructions [18:28] so agents know the Repo as soon as they arrive. [18:34] And you can only go as fast as you have good brakes. [18:37] And that's why we've built robust gates in our CI. [18:41] These include dependency reviews, tests, coverage, [18:44] and New Relic preflight on every PR. [18:49] Last, we're using Copilot Agentic auto fix. [18:53] And this means where there's an issue in the CI workflow, [18:56] the agent investigates that comes up with a solution [18:59] and then pushes that to the engineer within the PR for approval. [19:05] Now, noticeably, everything we're talking about here is focused on new repos. [19:09] But we also have this tooling available for existing repos. [19:13] Of course, we have a lot I'm sure you do to the agent in this scenario. [19:19] Assesses the existing repo, looks at the templates, [19:22] and identifies where there's gaps in terms of the approach. [19:26] It then drafts those PRS and sends it to the engineer for approval. [19:31] All of this work helps to ensure that we have a strong engineering practice [19:36] with consistency, maintainability, delivery, confidence, and, of course, [19:41] readiness for AI assisted development. [19:46] This is our internal marketplace and you're going to see four shelves here. [19:51] This is our Custom agents skills Prompts and MCP Servers. [19:56] And this is available to every engineer in the company. [20:00] There's 24 agents, 19 skills and 15 MCP Servers. [20:05] These are all rated by the engineers who use them. You can see that here. [20:09] The stars, the ability to leave comments and review. [20:12] And the goal of that is to ensure that we're driving to reusability [20:16] and accessibility across the organization to reduce time to value. [20:22] These are shared and VS Code and GitHub as well. [20:25] And everything here is available with one click. [20:31] So we've talked about process, we've talked about platform. [20:34] Let's talk about how we bring them together. [20:40] The tooling was really the easy part [20:42] and the change management was the heavy lift in terms of the work. [20:46] And I think that makes sense [20:48] when you're in a moment of time where the changes are seismic [20:52] in terms of how you're expecting a role to transform. [20:55] And of course, understandably, our team had concerns about that, [20:59] that we had to work through and support them [21:01] and build that trust in terms of what all of this meant. [21:05] And we tried to do this in a few key ways. [21:08] So first, before day one of the workshop, [21:10] we sent a Pre-read and we pulled documentation from across the industry [21:16] in terms of what leading organizations were doing as regards an Agentic SDLC. [21:21] And we made sure that that was visible to the team. [21:24] We also polled sources like Gartner and McKinsey research as part of this [21:28] and packaged it up. And the idea for this was to really empower them to think big, [21:34] bold thoughts and not feel that it had to be mostly the same, [21:38] but to instead transform the work that we were doing, not just drive small, [21:43] incremental improvements. [21:47] We had a principal engineer who's an expert in AI embedded with the team [21:52] and supporting them throughout the process. [21:54] So this meant things like sitting on their stand ups every day, [21:58] being available on their chat, doing coaching sessions, [22:02] generally acting like a player coach where he was available to partner with them [22:06] on anything that was available and anything that they were struggling with. [22:12] We also had hands on training on their own code, [22:15] so right from the start we were focusing them on things that actually mattered [22:19] to them versus abstract problems in a sandbox. [22:26] And last, and this was something we did broadly [22:29] to support AI adoption at the organization. [22:32] We've had developer velocity office hours, [22:34] where anyone on the team can come ask ask questions in a safe space. [22:39] The team also walks through current capabilities that have just been released [22:43] that we're launching, and walks through key examples that other [22:47] teams are doing as well. [22:49] In terms of broader AI adoption. [22:51] There's also been a few things more we've been doing that have been successful. [22:55] We had an hackathon earlier this year as a way to [22:59] which was just exclusively focused in terms of like AI use cases. [23:03] Every team who participated had to use AI, [23:07] and that was a great opportunity for the teams to learn. [23:09] We did pre skilling in advance, and then they got to compete. [23:13] And there was a lot of engagement in terms of being able to do that. [23:17] And the results were amazing. [23:19] And a lot of it has actually already moved [23:20] to production that what the team has developed. [23:24] We also are planning an AI focused conference next year as well. [23:31] These are the rules we run on, [23:33] which are short enough to remember and sharp enough to settle arguments. [23:37] So first, as we've talked about humans as judgment agents on execution [23:42] and really making sure that we're building this trust across our team, [23:46] that what we want them to focus on is the ambiguous work, [23:50] the areas where they really need to come to the table as human beings [23:54] and bring that judgment, bring that taste, bring that expertise, [23:58] and we'll pass off the route routine, manual tasks to the agents to take care of. [24:04] Next context lives where agents can see it. [24:07] The Repo and the ticket are durable. [24:10] Chat is not. Now, of course, [24:13] as our understanding of agent knowledge continues to evolve. [24:17] Chad is absolutely going to be part of context, and we're not that far away from it, [24:21] but we're not there personally yet. [24:23] And we weren't there when we started with Back Pocket. [24:26] So we wanted to be honest about this and set it as an expectation for the team. [24:32] Next, we have the same bar for agents and humans. [24:35] Everything clears the same gate for good. [24:38] This makes again, sure that we're living up to the standards [24:40] and what looks like good engineering at Ellis and Co. [24:44] And last auditable always every agent action and human override is logged. [24:50] So we always know what's happening within the system. [24:55] So what changed for our engineers and what we measure. [24:59] Korn ferry did a white paper about six months ago where they talked [25:02] about the fact that it was actually a lot of the highest performing people on a team [25:07] who struggled with the adoption for AI. [25:09] Because if you've been successful your entire career with a set of skills, [25:13] it becomes very hard to walk away from those skills [25:17] which have continually given you that success and set you ahead in your organization. [25:21] So there was a lot there for people to unlearn, [25:24] and it was a big transformation in that way. [25:27] And so there were things like spec driven development [25:30] and reviewing agent output critically that we really needed to reinforce rituals [25:36] as well. Changed stand up is no longer about status. [25:39] It's about open discussion and decisions and exceptions. [25:44] If you look at the numbers on the right hand side, [25:47] that 51% improvement in throughput we were really pleased with. [25:52] We expected initially that actually the teams throughput would decline because again, [25:57] this is a huge change. It's a big difference for people. [26:00] And it's fair to think that that's an adaptation that they would need [26:03] a little bit of time to process. [26:05] So we were really pleased with this number right up front. [26:09] And the 54% pass rate was also great [26:13] because that told us that quality was also improving. [26:16] And in particular, you can think of velocity and quality often as a paradox. [26:21] Where velocity in declines and quality improves or quality declines [26:25] and velocity improves. It's really irregular that both could increase together. [26:31] And so these were great results for us that we were really happy with. [26:34] The other thing I'd call out as well is, of course, [26:37] there's a conversation of that throughput increasing. [26:41] And what does that mean in terms of like, is that just generally, you know, [26:45] agents creating more for the sake of creating more productivity [26:48] does not necessarily correlate with the outcomes you want to achieve. [26:52] But we go back to the fact that, again, we released our product so much faster, [26:56] the MVP versus the legacy M, v p. [26:59] And we're also seeing this high increase in pass rate last that 26% [27:05] in terms of AI assisted PRS is where the team started. [27:09] So they already had some engagement and some exposure with AI. [27:13] And by the end of this very short window, they went up to 69%. [27:17] And this number has, of course, continued to grow as they've done the work. [27:21] But this was in line with the period of time that I'm talking about. [27:24] And ultimately, we want this, of course, to be 100%. [27:30] So what we learned and what's next? [27:32] So I showed you the commit history with all of the different agents operating. [27:37] And what we started out wanting to do was [27:40] to have that be parallel where each agent could go and execute their tasks. [27:45] But what we saw was that introduced merge conflicts. [27:49] So instead we reverted to this approach of having everything parallel to drive that. [27:54] Now, very recently, we've actually changed that approach again, [27:58] which is that each of these agents now leave comments [28:01] and you have a helper agent come at the bottom, [28:03] look at all of those comments and then fix it. [28:06] And that helps us get to more of that balance between having efficiency [28:10] and a streamlined process, but without those merge conflicts. [28:15] So we also talked about ambiguity surfacing early by design. [28:19] It's very common for agents to run ahead with total confidence [28:23] and then end up giving you something that didn't match your intent. [28:26] So that was something we really wanted to build into the system up front. [28:31] The first workshop was a big ask. [28:33] It was three days in the room, [28:36] and it was about as many hours in terms of preparation heading into it, [28:40] and then there was more time after that as well. [28:42] And it was extremely successful, but it was also a lot of time. [28:47] And so our next step is really thinking through how do we streamline this. [28:51] And we've continued to bring on new squads. [28:53] And I've done this workshop in as few as three hours and still had great results. [28:57] So we know it's possible to keep going in this direction. [29:03] And that's where we're expecting to see squads continue to onboard faster [29:07] in the future. [29:11] So back to where we started. [29:12] Back to back pocket. As we talked about our conversation today was about [29:16] the journey from the current state map covered in pain points [29:20] to our beautiful new app launching now in Southeast Asia. [29:25] And we were really pleased with the 72% improvement that we were able [29:29] to drive through all of this work. [29:35] So let me leave you with one ask before you start building agents. [29:39] Reimagine your process first. [29:43] Don't put the new motor in the old factory. [29:47] In 1873, we reimagined workwear. [29:50] This year, we reimagined software with our amazing engineering team at Ellis [29:55] and Co. Thank you so much for your attention today, and thank you all for joining me. [29:59] I'll be at the after party down the hall in a little bit.