Grepr, A16z Warn AI Code Surge Hits Wall in Production Operations
a16z
AI models rapidly generate software syntax within bounded text distributions but fail at system operations because distributed infrastructure suffers from exponential state spaces and computational irreducibility that demand empirical experiments.
Using an agentic stack centered on Claude Code with automated sub-agent reviews, Grepr completed an Apache Iceberg and Parquet data-lake shredding project in 10 days that previously would have taken roughly six months. (22:55)
Observability costs have risen to consume 5% to 20% of total infrastructure budgets—ranking second only to compute—because telemetry data volume expanded faster than delivered product functionality. (7:23)
Martin Casado posits that AI adoption follows a three-stage difficulty hierarchy: code synthesis is solvable first, production operations will take substantially longer due to heavy-tailed runtime distributions, and cybersecurity will take the longest due to active adversarial manipulation. (41:26)
Autonomous systems often resolve economically by redesigning environments around machines rather than forcing models to handle open-ended complexity, similar to building specialized automated warehouses instead of adapting robots to human spaces. (55:05)
Detailed notes
## Background
Published by a16z on YouTube on October 16, 2026, this episode features Grepr CEO Jad Naous in conversation with a16z General Partner Martin Casado. The discussion explores why artificial intelligence excels at generating production code in minutes but struggles to debug, operate, and secure complex distributed systems in real-world production environments.
## TL;DR
Jad Naous and Martin Casado examine the stark divergence between code synthesis and production operations, illustrating how coding agents compress months of software development into days while widening the productivity divide between elite and average engineers. They contrast the bounded, syntactic realm of code generation against the combinatorial explosion of runtime states, heavy-tailed failure distributions, and intelligent security adversaries encountered in live infrastructure. Naous details how Grepr tackles these operational hurdles by using deterministic log clustering upstream to catch leading incident indicators and drastically trim observability spend. Ultimately, Casado argues that operational automation faces an economic equilibrium challenge where restructuring real-world systems and overprovisioning resources often outcompete complex model reasoning.
## Atmosphere & style
The conversation carries the warm, intellectually rigorous rapport of long-time Stanford systems colleagues who share deep engineering scars and mutual respect. Rather than uncritical AI hype, the dialogue unfolds as a pragmatic, dialectical inquiry where Casado challenges optimistic assumptions with historical computing parallels and VC realities, while Naous defends agentic workflows with concrete implementation details. It feels like an authentic, high-level whiteboard debate inside an engineering lab, marked by thoughtful digressions into state-space math, dry humor about VC analysis, and candid assessments of tech economics.
## Bulletin
- [22:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=1375s) Using an agentic stack centered on Claude Code with automated sub-agent reviews, Grepr completed an Apache Iceberg and Parquet data-lake shredding project in 10 days that previously would have taken roughly six months.
- [07:23](https://www.youtube.com/watch?v=blkIGLlZePg&t=443s) Observability costs have risen to consume 5% to 20% of total infrastructure budgets—ranking second only to compute—because telemetry data volume expanded faster than delivered product functionality.
- [41:26](https://www.youtube.com/watch?v=blkIGLlZePg&t=2486s) Martin Casado posits that AI adoption follows a three-stage difficulty hierarchy: code synthesis is solvable first, production operations will take substantially longer due to heavy-tailed runtime distributions, and cybersecurity will take the longest due to active adversarial manipulation.
- [55:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=3305s) Autonomous systems often resolve economically by redesigning environments around machines rather than forcing models to handle open-ended complexity, similar to building specialized automated warehouses instead of adapting robots to human spaces.
## Key takeaways
- [10:15](https://www.youtube.com/watch?v=blkIGLlZePg&t=615s) Coding agents act as asymmetric force multipliers that widen productivity differences between top-tier and average engineers up to tenfold, as high leverage requires an accurate mental model of system architecture.
- [14:35](https://www.youtube.com/watch?v=blkIGLlZePg&t=875s) Language models excel at generating syntax within compressed low-dimensional token manifolds, but they struggle with runtime semantics influenced by distributed environment states and dynamic workloads.
- [18:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=1135s) High-velocity AI code generation shifts the engineering review bottleneck away from manual syntax line checks toward automated testing harnesses, regression suites, and golden expectations.
- [32:30](https://www.youtube.com/watch?v=blkIGLlZePg&t=1950s) Production operations hit a complexity wall because every additional bit doubles a program's operational state space, creating combinatorial variations that exceed the number of atoms in the universe.
- [30:10](https://www.youtube.com/watch?v=blkIGLlZePg&t=1810s) Traditional AI SRE tools fail by trying to backtrack from lagging metric alerts; effective diagnosis requires deterministic log clustering upstream to catch novel patterns before incidents escalate.
- [37:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=2275s) Historically, systems engineers abandon dynamic or adaptive software optimization algorithms in favor of brute-force overprovisioning of compute, memory, and bandwidth.
- [39:50](https://www.youtube.com/watch?v=blkIGLlZePg&t=2390s) While failure states in distributed systems are practically infinite, the human debugging process itself follows a bounded diagnostic methodology that agents can replicate via hypothesis testing.
- [41:26](https://www.youtube.com/watch?v=blkIGLlZePg&t=2486s) Cybersecurity presents a fundamentally harder problem than operations because intelligent adversaries intentionally hunt for rare, out-of-distribution edge cases.
- [47:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=2828s) Production operations suffer from computational irreducibility, meaning execution behavior and emergent failures cannot be reliably predicted without running the system or its empirical simulation.
- [52:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=3128s) Autonomous driving demonstrates that technical feasibility alone does not guarantee viability: after 20 years and $170 billion in investment, its unit economics remain inferior to human-driven rideshare services.
- [56:41](https://www.youtube.com/watch?v=blkIGLlZePg&t=3401s) Enterprise buyers frequently prioritize immediate hard-cost reductions—such as cutting Datadog bills by 90%—over speculative future benefits like automated incident prevention.
- [57:47](https://www.youtube.com/watch?v=blkIGLlZePg&t=3467s) Coding tools sell rapidly because their ROI connects directly to engineer payroll on enterprise balance sheets, whereas operations tools struggle to price the avoided cost of prospective downtime.
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=blkIGLlZePg&t=0s) Teaser: AI Code Generation vs. Real-World Operations
The conversation opens with a discussion on the fundamental tension between software synthesis and software execution. While AI lives entirely inside a digital world, real-world systems are far messier and rarely conform to clean digital abstractions. In the early days, AI coding began as simple autocomplete, yet it rapidly surpassed expectations as models learned to represent increasingly complex software architectures. However, operating those systems in production introduces entirely distinct obstacles.
```mermaid
flowchart LR
accTitle: Code Synthesis vs Production Operations
accDescr: Contrasting the constrained digital domain of code generation with the open-ended operational complexity of production systems.
subgraph Synth["Code Synthesis Domain"]
A["Autocomplete"] --> B["Complex Software Modeling"]
B --> C["Generated Code"]
end
subgraph Ops["Production Operations Domain"]
D["Distributed Clusters"] --> E["Unbounded Real-World State"]
E --> F["Operational Troubleshooting"]
end
C -.->|"Must Run In"| Ops
```
The core debate centers on whether operational management can ever be bounded in the same manner as code generation. When Martin Casado questions where the data would even come from to train an AI model capable of understanding end-to-end operations, Jad Naous pushes back, suggesting the data exists, but the real difficulty lies in encoding raw operational reality into an abstraction an AI can reason about. Casado points out that running large distributed clusters has historically relied on the dirty secret of brute-force human supervision.
- As Martin Casado puts it: “The world that the AI lives in is all digital and there's the actual world and like those things are not necessarily the same.”
- As Jad Naous counters regarding the feasibility of training operational models: “My assumption is that we have all the data. The question I think is maybe about complexity and like to what extent can you take the things that you've described that are operational and convert them into something that an AI can reason about.”
---
[00:54](https://www.youtube.com/watch?v=blkIGLlZePg&t=54s) Stanford Days, Nick McKeown's Lab, and Early Systems Work
Martin Casado introduces Jad Naous, founder and CEO of Grepr, recalling their shared history in the Stanford Computer Science Department under adviser Nick McKeown in the Gates Computer Science Building. Casado reminisces about meeting Jad for the first time: he walked into the lab to find a motorcycle helmet and a pack of Marlboro reds sitting on the desk, thinking, "This guy must be unbelievably cool." Casado recalls wondering if cigarettes were even legal anymore, while Jad shares a memory of their adviser Nick catching him smoking outside the Gates building and looking at him in utter disbelief. The two also joke about Professor Balaji Prabhakar, whom Casado remembers fondly as one of the few faculty members who smoked, though Jad quickly clarifies that he has long since quit.
Jad entered the Stanford PhD program four years after Casado. While Casado finished his doctorate and went on to co-found Nicira, Jad stayed behind to work on the OpenFlow protocol and hardware-level network security. One of his defining thesis achievements was designing and building the very first hardware implementations for SHA-3 candidates to benchmark their real-world viability. Despite his deep work in the field, Jad ultimately grew disillusioned with networking as a lifelong pursuit, deadpanning that moving bits from one location to another was just not conceptually satisfying enough for him.
- As Martin Casado jokes about Jad's early persona: “I walked into the office and there was a motorcycle helmet and a pack of Marlboros. I'm like, 'This guy must be unbelievably cool.' So my question is is what happened to all your coolness?”
- As Jad Naous reflects on his shift away from networking: “I decided after a while that I actually don't like networking because it's just about moving bits from one place and putting them in another place.”
---
[04:02](https://www.youtube.com/watch?v=blkIGLlZePg&t=242s) MIT Postdoc, AppDynamics, and the Move to Venture Capital
After Stanford, Jad pursued a postdoctoral fellowship at MIT working with application security researcher Nikolai Zeldovich. Alongside fellow PhD student Ramesh Chandra, Jad worked on automated system recovery: roll back the entire operational state of an application upon detecting an exploit, patch the vulnerability, and replay the world forward onto the secure version. They tried commercializing the research into a startup, but the technology was too far ahead of its time, as bounding application state in the real world proved brutally difficult—an insight directly applicable to modern AI operations. Casado notes that these types of long-tail systems problems often act as a "fly trap for smart people," where early rapid progress creates the illusion of a total solution before hitting asymptotic limits.
Jad moved into industry at AppDynamics, spending three and a half years watching the organization swell from roughly 200 to 2,000 employees before being acquired by Cisco. Jad famously stayed at Cisco for exactly one day before Casado recruited him into venture capital at Andreessen Horowitz (a16z). Shifting from systems architecture to venture investing proved to be an intellectual shock; Casado advised him that the less he knew about traditional venture going in, the better, and ordered him on his first day to deliberately slow his mind down. After a two-year tenure collaborating with partners including Jennifer Li, Marc Andreessen, Ben Horowitz, Satish Dharmaraj, and Matt Bornstein, Jad returned to operational leadership as Chief Product Officer at Imply. Over three years, he scaled the engineering, product, info-sec, and design teams from 15 to more than 90 people before feeling too disconnected from hands-on work.
- As Jad Naous describes his shift from Cisco to venture: “I was a Cisco employee for one day. And I said, and then you're like, Jad, come do this VC thing with me. I'm like, what is VC? What? Like I'm an architect. I do like engineering stuff.”
- As Martin Casado explains the mental shift required between operator and investor roles: “It's actually people that move from operating world to VC, it requires a lot of reworking the brain. Just like going from a PhD to operating by the way. Like you have to like undo the brain damage.”
---
[07:23](https://www.youtube.com/watch?v=blkIGLlZePg&t=443s) Why Observability Broke and the Genesis of Grepr
After stepping down from Imply in 2023, Jad spent months interviewing engineering executives to uncover their sharpest infrastructure pain points. The universal mandate was cost reduction: after foundational cloud compute, the single largest line item draining budgets was observability tooling. Organizations routinely allocated between 5% and 20% of their total infrastructure budgets exclusively to observability suites.
Historically, companies treated observability spending as a direct proxy for buying reliability. For years, pouring more money into monitoring tools yielded a proportional rise in system uptime. However, modern microservice architectures, granular containerization, and deep vertical integration caused telemetry volume to explode across every layer of the software stack. The cost of collecting, ingesting, and retaining this tidal wave of logs, metrics, and traces began scaling far faster than the actual units of business functionality being shipped, causing the marginal reliability return per dollar spent to flatten out completely.
```mermaid
flowchart TD
accTitle: Modern Microservices and Telemetry Cost Scaling
accDescr: Microservices cause telemetry volume to grow faster than business value, leading to diminishing returns on observability spend.
subgraph Architecture["Architectural Shift"]
A["Monolith Architecture"] --> B["Distributed Microservices"]
B --> C["Vertical Integration Across Layers"]
end
subgraph Telemetry["Data and Spend Escalation"]
C --> D["Exploding Log/Metric/Trace Volume"]
D --> E["Observability Consumes 5-20% of Infra Spend"]
end
subgraph Value["Diminishing Returns"]
E --> F["Data Grows Faster Than Functional Output"]
F --> G["Marginal Reliability Gains Flatten"]
end
```
Jad observed that engineering teams were still willing to pay that 5% to 20% tax for reliability, but the data itself had degraded into expensive insurance and low-value "fluff." This insight formed the foundation for Grepr: instead of routing every bit of operational noise downstream to central databases, software should filter, analyze, and extract true signal upstream in real time. Processing the stream upstream slashes downstream storage and ingestion bills while surfacing incidents before they impact production.
- As Jad Naous analyzes why customers felt observability pricing was out of control: “The volume of data that you're collecting and the cost of observability was actually starting to grow faster than the units of function that you're outputting. And the marginal increase in reliability for every additional dollar you were spending on observability was just flattening out.”
- As Jad Naous summarizes the founding hypothesis of Grepr: “If we can figure out what is the signal, what is the noise, can we actually turn the observability data that you're collecting that it's mostly fluff and mostly insurance at this point and do that kind of extraction of information upstream, do whatever you need upstream and be able to ultimately massively reduce the cost of observability downstream.”
[09:23](https://www.youtube.com/watch?v=blkIGLlZePg&t=563s) Grepr's Dual-Layer Architecture and the AI Shift
Jad Naous explains that intercepting telemetry data provides two immediate benefits: first, it drastically cuts cloud observability costs, and second, the extracted signal can be directed into downstream operational systems to drive system reliability. When founding Grepr, they anticipated that AI would inevitably play a central role in infrastructure management. By inserting Grepr directly into the live telemetry stream to distill signals, they positioned the company as the strategic feeder layer for future AI tooling. Today, Grepr functions as a plug-and-play intermediary that routes data and cuts telemetry bills by roughly 90%, while building out an autonomous AI layer that consumes those clean signals to diagnose production incidents and resolve them in real time.
```mermaid
flowchart LR
accTitle: Grepr Data Pipeline Architecture
accDescr: Telemetry data flows into Grepr for 90 percent cost reduction and feeds downstream AI troubleshooting.
subgraph Ingestion["Ingestion"]
Raw["Raw Telemetry Stream"]
end
subgraph GreprEngine["Grepr Platform"]
Extract["Signal Extraction & Filtering<br/>(90% Cost Reduction)"]
Clean["Clean Signal Pipeline"]
end
subgraph Consumers["Downstream Consumers"]
Storage["Observability Backends"]
AILayer["AI Troubleshooting Layer<br/>(Automated Incident Resolution)"]
end
- Grepr acts as a drop-in, plug-and-play proxy sitting directly in customer telemetry data streams.
- The platform reduces telemetry spend by up to 90% before routing clean operational data downstream.
- The company's engineering team consists of 10 people operating at high leverage.
[10:15](https://www.youtube.com/watch?v=blkIGLlZePg&t=615s) How AI Augments Top Engineers (and Exposes the Rest)
Martin Casado shifts the conversation to the reality of engineering productivity, pointing out that Naous has architectural experience across products and teams and is currently shipping complex systems with just 10 people. Naous traces his personal journey from early code autocomplete to modern agentic systems. When tools like GitHub Copilot first emerged inside IntelliJ, the reaction was excitement that it could generate boilerplate test cases, tempered by the realization that it was still only autocomplete. Over time, advanced harnesses emerged, and Naous noticed an interesting dynamic: the best engineers enter a bidirectional feedback loop where the AI actually trains the developer on how to communicate with it, formulate queries, and extract maximum leverage.
When Casado asks whether this learning curve is tied to specific models or represents a universal skill, Naous points out that both model nuances and the execution harness matter. Different harnesses—whether OpenCode, Claude Code, or Codex—handle context retention, decomposition, and sub-agent orchestration differently. For instance, Naous notes that without explicit guardrails, Claude Code will run wild:
> “With Claude Code, I find that if I don't tell it don't spin up, you know, 20 sub-agents to do basic research, it'll go and spin up 20 sub-agents to do research and burn all your tokens.”
When Casado probes whether AI is an equalizer that brings weaker developers up to speed or an amplifier that makes top engineers pull away, Naous is unequivocal. He rejects the equalizer theory outright: mediocre engineers do not suddenly become elite with AI tools. Instead, the performance gap between average developers and top-tier talent has widened tenfold. Top engineers succeed with AI because their mental models can accommodate massive system complexity, allowing them to break distributed problems down into logically coherent sub-tasks that an agent can execute. Because modern models cannot reason through the edge cases of complex distributed algorithms on their own, the engineer's cognitive capacity remains the ultimate bottleneck.
- The performance disparity between average and elite engineers has multiplied by roughly 10x with modern AI tooling.
- High-level AI leverage depends on an engineer's capacity to manage distributed system complexity mentally before handing tasks to the model.
[14:35](https://www.youtube.com/watch?v=blkIGLlZePg&t=875s) The Semantics vs. Syntax Divide in AI Code Generation
The discussion sharpens around why models struggle with core systems problems, focusing on the query translation and optimization layers inside Grepr. To make a model construct a reliable translation engine, the engineer must grasp the semantics of the query language completely and communicate them to the AI. When Casado asks him to unpack "semantics" for an audience outside deep academic computer science, Naous draws an analogy:
> “The semantics of the word dog is the animal that actually walks on four legs and barks, and the syntax is the fact that it's just a three-letter word and it has three different letters.”
Casado simplifies this further for software: syntax constitutes the characters typed into an editor, whereas semantics represents what the runtime machine actually executes. Casado jokes about their differing backgrounds:
> “So you did computer science, I did electrical engineering, you became a computer scientist and I became a VC.”
The distinction is critical because syntax is trivial for LLMs to generate, but runtime semantics cannot always be deduced simply by reading code. Casado notes that in SQL, execution speed and plan behavior depend entirely on the underlying schema and data distribution. Naous refines this, arguing that runtime performance is a separate operational metric, whereas true semantics involves fundamental behavioral definitions—such as whether a system's engine configuration evaluates `NULL` as `FALSE`. Because models cannot intuit these environmental semantics unprompted, human architects must explicitly establish the rules. The pair agree to define syntax as the written code and semantics as actual system behavior for the remainder of the discussion.
[18:01](https://www.youtube.com/watch?v=blkIGLlZePg&t=1081s) Jad Naous's Personal Development Stack
Casado pivots to an earlier off-mic comment where Naous mentioned completing projects in a single week that previously required a dedicated team working for months. Asking for the practical details of that workflow, Naous reveals a setup that catches Casado off guard: he works primarily inside Windows using WSL (Windows Subsystem for Linux). Casado laughs in disbelief at the mention of Microsoft Windows, but Naous explains that he runs pure headless Linux command-line environments without an X server or desktop UI, driving his terminal development workflow directly via Claude Code.
[18:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=1135s) Windows, WSL, and the Multi-Agent Coding Workflow
Martin Casado expresses disbelief that Jad Naous develops on Windows, joking about whether he is secretly alt-tabbing into *Counter-Strike* or stuck in the 1990s. Jad defends his choice by pointing directly to ergonomic friction on macOS: he refuses to learn Apple keyboard shortcuts, despises the one-button mouse paradigm, and dislikes pressing numerous modifier keys. His development environment pairs Windows with WSL (Windows Subsystem for Linux)—which spins up a real Linux kernel inside a hypervisor within Windows—alongside Cloud Code and Anthropic’s Claude 3.5 Sonnet / Opus models.
```mermaid
flowchart TD
accTitle: jadNaousAgentWorkflow
accDescr: Workflow diagram showing human-orchestrated Claude CLI agents collaborating on PRDs, design docs, code implementation, and tech lead reviews.
Jad explains that this AI tooling has radically altered his productivity. Over a two-week period, he produced a massive stack of 20 pull requests to dump onto his 10-person engineering team. When Martin teases him about an inverted hierarchy where junior engineers spend all day reviewing their CEO's pull requests, Jad points out that everyone on the team is now writing and pushing PRs at the same blistering velocity. This surge in automated code has forced their review process to pivot away from manual line-by-line syntax checks and toward automated test harnesses, regression tests, and “golden expectations” to detect unintended behavioral drift.
To maintain context efficiency without hitting agent memory limits, Jad leverages command-line sub-agents instead of monolithic agent loops:
- Sub-agents run directly via the command-line interface, allowing them to keep isolated, persistent sessions over time.
- Jad limits himself to three or four parallel session windows at once; managing more causes him to lose context as tasks finish faster than he can review them.
- Sub-agents interact directly via the terminal rather than through GitHub PR comments. Jad configures an implementation agent and a dedicated “tech lead reviewer” agent that challenge each other on design choices and code quality.
- Jad acts as the orchestrator: he interrogates the generated PRD and design doc, verifies the proposed algorithms and PR task boundaries, directs the agents to parallelize independent tracks, and ultimately collapses the branches into a single linear stack for human inspection.
> “And but like the thing that has changed a lot in the review cycle is that like we started much more focused on to focus much more on like the the tests and like doing things like golden expectations and like are they shifting in different ways and so on.” — *Jad Naous*
---
[22:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=1375s) A Six-Month Project Shipped in Ten Days
Martin asks Jad to benchmark the productivity difference between building infrastructure software today versus several years ago, specifically emphasizing that this applies to rock-solid production code rather than superficial demos or throwaway prototypes. Jad illustrates the contrast using a recent core infrastructure feature he implemented alone: automated promotion (or shredding) within a data lake.
In high-throughput logging architectures, incoming telemetry arrives as semi-structured data with arbitrary, evolving schemas. To maximize query performance and compress the data cleanly using columnar storage formats like Apache Iceberg and Apache Parquet, the engine must inspect incoming records and automatically promote nested semi-structured fields into top-level columns that can be filtered and indexed independently. Building this capability entails extreme distributed systems complexity:
- Dynamically altering and versioning metadata definitions across writes.
- Safely handling concurrent commits to the metadata catalog across independent writer nodes.
- Coordinating write barriers and failure-recovery protocols without stalling data ingestion.
Jad completed and shipped the entire production-grade auto-promotion subsystem by himself in approximately 10 days—a project that previously would have taken an experienced distributed systems engineer around six months. Martin observes that this represents a 10x to 20x multiplier on shipping non-trivial, production-level software.
> “In previous days that would have taken maybe about 6 months.” — *Jad Naous*
---
[24:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=1495s) The Wall Between Writing Code and Operating Distributed Systems
Martin pivots the conversation to the fundamental gap between generating code syntax and managing live production behavior. While large language models are trained on virtually all public code repositories—making them exceptionally fluent at producing algorithms, boilerplate, and distributed data store primitives—they operate completely blind to the runtime dynamics of physical infrastructure. In real-world distributed systems, performance, availability, and correctness are rarely visible in static code; they depend entirely on dynamic traffic patterns, access distributions, cache hit rates, lock contention, and network partition characteristics.
| Dimension | Static Code Generation (What LLMs Excel At) | Runtime Operations (Where LLMs Struggle) |
|:--- |:--- |:--- |
| **Correctness** | Syntax, typing, and standard algorithm patterns | Consistency tradeoffs (e.g., locking vs. eventual consistency under high contention) |
| **Performance** | Local computational complexity ($O(n)$ implementations) | Cache thrashing, memory paging, dynamic resource saturation, and access skew |
| **Availability** | Implementing retry policies and timeout semantics | Cascade failure modes, thundering herds, and partition tolerance |
| **Feedback Loop** | Unit tests and compiler checks | Real-world telemetry, production logs, dynamic profiling, and tracing |
Martin illustrates this operational wall with his own experience building a distributed Gaussian splat renderer, noting that models produce terrible operational outcomes because rendering millions of splats depends entirely on dynamic streaming, GPU memory paging, and hardware latency that the model cannot infer from code alone. He questions where the training data would even come from to teach models operational trade-offs, given that the industry currently depends on seasoned human Site Reliability Engineers (SREs) and database administrators to tune systems and manage clusters.
Jad acknowledges that while LLMs struggle to reason about production dynamics unassisted, teams can guide them by strictly codifying target performance benchmarks and operational constraints into their test pipelines. He recounts a recent effort around query optimization for Grepr's stream processor, where the agent generated functional pipeline code from SQL but produced abysmal execution speed until confronted directly with performance profiling data and ordered to fix its pipeline efficiency.
> “What I don't understand is how it will know to like make the trade-offs needed because it doesn't even know what the access patterns are. And it's not like these things can like like where's even the data to learn how to like build a system that understands like all of the operations right like today we rely on human beings to like manage clusters and to like you know optimize databases etc.” — *Martin Casado*
[28:39](https://www.youtube.com/watch?v=blkIGLlZePg&t=1719s) Grounding Optimization in Benchmarks vs. Operational Reality
Jad Naous recounts working with an AI coding model on query optimization. Initially, the model did not realize it had skipped optimizing the code at all, acting as if everything was fine until he intervened directly: "I said well look there's these uh these conjuncts over here you're doing and and like this particular term is repeated and it should have been taken away and then it's like ah okay let me figure this out and so on." Once prompted, the model branched into a thread generating multiple hypotheses about what it could optimize to enhance performance.
The process succeeded because it could be anchored against empirical performance benchmarks using an actual dataset of user queries. However, Jad draws a sharp distinction between code-level optimization and operational decision-making:
> “It has no idea and that has to come from actual experiments that run in production.”
While an AI can recognize its own uncertainty and suggest running experiments to gather data, production systems rarely behave predictably. That operational unpredictability is precisely why production incidents happen in the first place.
[30:10](https://www.youtube.com/watch?v=blkIGLlZePg&t=1810s) Flipping the AI SRE Playbook: Leading Indicators vs. Lagging Alerts
Martin Casado asks whether Jad has seen success using AI Site Reliability Engineering (AI SRE) setups where running logs are fed to an AI to help tune the system, noting that anecdotal feedback from others attempting this has been poor. Jad agrees that conventional AI SRE tools fail because of how they frame the problem: they trigger on lagging indicators, such as a spiked metric or a fired alert, and then attempt to backtrack through a massive, labyrinthine system to divine the root cause. Backtracking through that complexity after an incident has already manifested is fundamentally problematic.
Grepr takes the inverse approach by searching for novel changes and early deviations before an alert triggers. The platform runs large-scale clustering across logs to find a needle in a haystack—a pattern or error message that has never appeared before:
> “And so the the the thing that we're looking at is like when things go wrong, usually the system will tell you something went wrong somehow.”
Rather than waiting for a full outage, Grepr catches early warning signals such as an exhausted connection pool, an exceeded memory threshold, or an emptied buffer. Jad notes that this approach has proven effective internally and across customer environments:
> “We ran it uh we ran it in uh in a couple of our customers environments and in the cases where there was a couple of cases where we ignored it. And those turned into production incidents.”
```mermaid
flowchart TD
accTitle: Grepr early anomaly detection workflow
accDescr: Pipeline showing logs fed into clustering models before passing new patterns to an AI reasoning agent.
subgraph Ingestion["Telemetry Ingestion"]
Logs["Customer Log Stream<br>(Petabytes/day)"]
end
[32:30](https://www.youtube.com/watch?v=blkIGLlZePg&t=1950s) The State Space Explosion of Distributed Systems
Martin questions the limits of applying machine learning models to operations. While models can master programming syntax because they have ingested virtually every line of code ever written, the state space of running software expands exponentially:
> “It's like every time you add a bit to a program, the state space of like what it can do operationally increases by a factor of two.”
This combinatorial explosion means operational states rapidly outnumber the atoms in the universe, which explains why mechanical attempts like chaos engineering rarely succeeded at scale. Martin observes that the dirty secret of distributed computing has always been human operators relying on tribal knowledge. Jad concurs that humans rely on runbooks alongside invisible context—such as uncommitted operational details heard in casual conversation at the water fountain—that is never recorded in system state. Nonetheless, Jad argues that assuming a system was stable previously, the majority of operational breakdowns stem from changes, making the isolation of that delta the primary objective.
When Martin asks whether Jad is expecting generative models to do predictive anomaly detection, Jad clarifies their architecture. Generative LLMs are not used for time-series forecasting or statistical baselining. Instead, a dedicated pattern-clustering pipeline deterministically monitors streams running at the scale of a petabyte of logs per day:
> “And we scan all of them and we we like every time we see a new kind of pattern emerge in your logs, a new thing that we haven't seen before, we lift it up to the AI.”
Once an unfamiliar pattern is flagged, the AI agent is brought in to reason about it heuristically. The AI pulls up the source code, inspects where the log message was emitted, checks if the issue has ever occurred historically, and evaluates whether the signal represents an impending failure.
[36:36](https://www.youtube.com/watch?v=blkIGLlZePg&t=2196s) Query Optimization vs. Operating a Distributed Cluster
When comparing operational domains, Jad and Martin contrast query optimization with managing large-scale distributed infrastructure:
- **Query Optimization**: Jad believes AI is "very close" to replacing humans in optimizing complex database queries, as the problem can be bound against query plans and synthetic workloads.
- **Cluster Operations**: AI is not ready to take over the job of an MIT-trained distributed systems engineer keeping an entire company's cluster alive:
> “I don't know like are we at the point where you would say that an AI can do that? No, not yet.”
Jad highlights how expectations evolve, recalling that AI coding began as simple autocomplete, with skeptics insisting it could never write complex software, until models expanded their grasp of system complexity. Martin remains skeptical about the parallel, concluding:
> “I just feel like like I'm a very simple thinker on this stuff and it to me these models are distribution in distribution out and it turns out like you can actually train on all the code ever created and it's actually like that manifold is pretty manageable and I just don't know if operations h is constrained in the same way if that”
[37:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=2275s) Overprovisioning vs. AI in Systems and Observability
Martin Casado characterizes production environments as behaving like the physical universe—dominated by heavy-tailed distributions, rare exceptions, and power laws that are inherently difficult for neural network models to learn. Looking back across the entire history of computer systems engineering, Martin argues that researchers always start with grand ambitions of designing clever, adaptive algorithms for virtual memory management, congestion control, and routing. Yet, practitioners repeatedly abandon those algorithms in favor of brute-force overprovisioning:
> “And what do we do instead? Every single time... We overprovision. No, we overprovision every time.”
He points out historical examples: swapping memory to disk was notoriously disastrous (“Dude, if you did that, you were [screwed]. Like that's it. Your computer was dead.”), so engineers simply started overprovisioning RAM. Similarly, adaptive network routing algorithms inevitably triggered route oscillations, prompting teams to overprovision bandwidth rather than try to predict dynamic shifts. Martin questions whether AI can ever manage production operations the way human operators do, emphasizing that unlike code generation, operations suffers from a severe lack of structured training data and is dominated by unpredictable edge cases.
Jad Naous pushes back with a more optimistic perspective grounded in day-to-day observability work. At Grepr, he works under the operational assumption that telemetry data is fully streamed and captured; the system already knows when an exception triggers or when a workload shifts. Martin notes that models are still not grounded on this telemetry data during pre-training. Jad agrees, clarifying that observability AI should never function by simply dumping gigabytes of raw logs into an LLM prompt:
> “Right? But the but the story here isn't like, 'Hey, Mr. Claude, here's a bunch of logs. Is something going on?' Right? And it has to be much more able to run experiments and kind of dive in and see like what are the different shapes of possible causal chains that actually fit in the environment that exists.”
Martin realizes this approach parallels human debugging. While the universe of possible failures is heavy-tailed and vast, the debugging methodology itself consists of a structured, manageable sequence of diagnostic steps that an agent can execute to constrain the state space. Jad contrasts this with earlier chaos engineering initiatives, which struggled because attempting to perturb and map an entire production state space randomly exposes engineers to a combinatorial explosion comparable to the atoms in the universe.
```mermaid
flowchart TD
accTitle: AI-Driven Operational Investigation Workflow
accDescr: Diagram illustrating how telemetry anomalies are processed by an AI agent testing hypotheses rather than consuming raw logs.
subgraph Ingestion ["Telemetry & Signals"]
A["Streamed Telemetry & Metrics"]
B["Anomaly / Behavioral Shift Detected"]
A --> B
end
subgraph AgenticLoop ["Diagnostic Reasoning Loop"]
C["AI Formulates Possible Causal Chains"]
D["Execute Targeted Experiments & Verification"]
E{"Fits Observed Patterns?"}
B --> C
C --> D
D --> E
E -- "No" --> C
end
subgraph Resolution ["Outcome"]
F["Constrained Root Cause Identified"]
E -- "Yes" --> F
end
```
[41:26](https://www.youtube.com/watch?v=blkIGLlZePg&t=2486s) The Security Problem and State-Space Combinatorics
Jad explains that Grepr applies this same hypothesis-driven methodology to cybersecurity. When a statistical deviation or behavioral anomaly appears—such as unexpected entity counts or unusual metric cardinalities—it serves as an alert perturbation. The agent takes that signal, generates candidate causal chains, and systematically queries observability data to verify which sequence matches reality.
Martin points out that security is fundamentally harsher than IT operations. While production operations deals with the natural occurrence of rare, broken states, security involves an intelligent adversary actively probing to weaponize edge cases:
> “The security one seems even harder because this isn't just like the natural order of rare states. You actually have an adversary that is trying to like intentionally find rare states and intentionally find stuff that you you you can't do.”
He predicts a multi-tier timeline where software coding will be solved first, production operations will take considerably longer, and security will take the longest of all.
Jad suggests that reality and system architectures are ultimately discrete—joking about the Planck length—meaning execution paths and states are mathematically bounded. Martin illustrates the deception of large finite numbers by sharing a story from their Stanford graduate school days walking around the Stanford Dish with their peer Sundar:
- Sundar walked up looking deeply despondent after realizing that a 4-minute MP3 file is only about 4 megabytes of data, leading him to mourn the idea that music is mathematically finite because all possible 4 MB bit sequences can theoretically be enumerated.
- Martin immediately laughed at Sundar's despair: “Motherfucker, dude, you know... you know how quickly you've like overcome like the atoms in the universe. You haven't even gotten to a K, right? It's like two to the two to the two to the two. Like, what are you talking about here, man? Be happy.”
- Four megabytes allows for $2^{33,554,432}$ unique bit combinations, rendering the space effectively infinite for human experience.
[45:01](https://www.youtube.com/watch?v=blkIGLlZePg&t=2701s) Why Operations Startups Struggle Compared to Coding Agents
Martin observes from venture investing that startups attempting to automate infrastructure operations face a much steeper uphill climb than companies focused strictly on code generation:
> “I will tell you from the perspective of a VC that looks at these companies, the ones that are trying to tackle operations are having a tough go at it. It is a it seems much much tougher than those that are tackling the code because I think it's a much tougher problem.”
Jad agrees, explaining that systems break not only from unexpected load spikes or deadlocks caused by uncoordinated code deployments, but because production operations is intrinsically tied to human organizational dynamics. Troubleshooting depends on fragmented institutional knowledge distributed across multiple engineering teams.
Nevertheless, Jad maintains that code generation is a proving ground and a microcosm for higher-level systems reasoning. He argues that if models can master code generation, engineers can layer complexity onto those foundations to iteratively conquer operational environments. Martin counters that believing these complex environments will yield so easily is wildly optimistic rather than fatalistic, concluding: “For me, I don't think the universe gives up its secrets that easily, but like we'll see.”
[47:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=2828s) Computational Irreducibility and the Limits of Models
Jad Naous reflects on how early skeptics thought the world of what code can implement was too constrained to capture human cognitive complexity, yet modern AI continues to conquer it. He finds it amusing that he, an operator, is pitching an expansive future vision to Martin Casado, an a16z venture capitalist whose job is normally to look into the future. Casado counters with his foundational framework for machine learning: models are trained on data distributions, and the fact that language models work so well is a testament to humans, who took a wildly high-dimensional universe and compressed it into a low-dimensional manifold of language.
Casado argues that code represents a similar reduction: it is merely the set of instructions that direct execution, not the execution itself. A massive disconnect remains between abstract instructions and the actual physical or operational universe. For Casado, certain phenomena are computationally irreducible, meaning one cannot simply predict the final state; one must run the simulation step by step. He suggests AI models may hit an event horizon where they can solve anything except problems that inherently require full simulation. Naous pushes back, asking why an agent cannot be architected to plan its own execution, simulate missing parts, and write code on the fly to probe and acquire the precise information it needs. Casado concedes that if an agent can autonomously run experiments like an artificial scientist, that distinction might eventually blur.
```mermaid
flowchart TD
accTitle: Dimensionality Reduction vs. Physical Simulation
accDescr: High-dimensional reality is compressed by humans into code and language, while operational systems require irreducible execution.
A -.->|"Compressed into"| D
B -.->|"Compressed into"| C
C -->|"Trains"| E
D -->|"Trains"| E
E -->|"Fails on"| F
G -->|"Simulates & probes"| F
```
- Casado defines computational irreducibility as scenarios where “you kind of have to run it to get the answer. You can't just like you know predict it.”
- Casado on human language: “The fact that it's useful that to me says that humans are so remarkable because they've looked at the universe which is this very highdimensional system and they've reduced it to a manifold that's actually useful for a model, right?”
[49:56](https://www.youtube.com/watch?v=blkIGLlZePg&t=2996s) Venture Capital as a Relationship Business vs. Analysis
Naous pivots the discussion and asks Casado how much of venture capital work an AI could actually execute. Casado explains that venture capital is fundamentally driven by human relationships, deal access, and founder psychology rather than sheer market analysis. Because he views financial markets as highly efficient, VC success is rarely about finding an obscure startup nobody else knows about; top investors generally identify the same promising companies. The real differentiation lies in raising capital, winning competitive deals against other firms, and delivering value post-investment—such as recruiting senior executives—which early-stage startups cannot easily handle alone.
While Casado admits that modern AI models produce market and company analyses that match or exceed the quality of junior associates and analysts, he downplays the ultimate value of that work in venture investing. He characterizes much of the traditional analytical diligence process as decorative posturing rather than the engine of returns. Since venture investing relies on trust, interpersonal alignment, and long-term optimism about technology waves, Casado argues that the core job requires being a human being unless an AI could somehow be dropped into a simulated environment where it could reward-hack authentic trust.
- Casado on VC analysis work: “I have seen anal analysis done by AI that are as good or better than any you know gene analyst I've ever worked with like so for the the analysis piece um they're great but I think this is more an indictment on VC's ability to like analyze that than it is like how powerful you know the models are.”
- Casado on the venture model: “The way that I think about venture you kind of have to be a human being to do it's like you know um because because that's the actual job.”
[52:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=3128s) In-Distribution Digital Spaces vs. Real-World Economics
The conversation shifts back to the divide between software operations and physical domains, with Naous pressing Casado on which sectors will be transformed immediately after software engineering. Casado asserts that AI will continue spreading like wildfire across constrained domains where the solution space shares the exact same distribution as the training data—such as source code syntax or generated images. However, when the solution space is governed by the messy, unpredictable natural universe, progress slows down dramatically and faces punishing unit economics.
To illustrate this gap, Casado brings up autonomous driving. When Sebastian Thrun won the DARPA Grand Challenge during Casado's time at Stanford, the tech industry assumed fully autonomous vehicles were right around the corner. Instead, it required two decades and approximately $170 billion in capital investment to achieve unit economics that remain worse than human rideshare services like Uber. Because models act as general universal function approximators, technical solvability is rarely the bottleneck; rather, the ultimate boundary is whether a solution achieves economic equilibrium against the physical world.
- Casado on constrained vs. unconstrained domains: “AI is going to continue to take off in a wildfire for a constrained domain where the where the solution space is in a constrained domain right where the distribution of the solution space is the same as the distribution of the training set the distribution of the syntax for code is the same as a distribution of the training set the distribution of the generated images is the same as the distribution of the training set in those you know areas I think it's going to continue to take off as a wildfire the domains where like the distribution of the solution is the natural world.”
- Casado highlights the autonomous vehicle milestone: “I remember when Sebastian Thrun won the DARPA Grand Challenge when I was at Stanford. Remember that? And we're like, hooray. You know, self-driving cars is around the corner. Seriously. And it took 20 years and $170 billion of investment to get to worse unit economics than Uber, right?”
[55:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=3305s) Adapting the Real World to AI
Naous notes that when technological problems appear intractable, humanity's historical response has not always been forcing tools to master the complexity of the world as-is. Instead, humans frequently redesign environments to fit the limitations of their machines. He points to automated warehouses and modern robotics as prime examples: rather than designing hyper-complex robots that can navigate messy, legacy human facilities, the winning approach was to completely rebuild and standardize the warehouse specifically around robotic capabilities. Naous suggests that as AI expands beyond pure digital spaces, society will actively restructure physical environments, operational protocols, and systems into machine-friendly interfaces.
Casado agrees that reshaping the operating environment is a proven path to bypassing complexity. While software currently thrives because its problem spaces are entirely digital and in-distribution, solving physical problems will always come down to the balance between compute expenditure, capital investment, and economic viability. If altering the physical environment brings down the required compute and capital to hit that threshold, the economic equilibrium will naturally shift in favor of automation.
- Naous on structural adaptation: “In some situations we actually tend to change the world to fit to what we can actually do from”
- Casado confirms the historical parallel: “100%. Of course I think this is the story of robotics.”
- Casado on real-world problem solving: “When you're dealing with the real world like you can solve any problem you want. It's just a matter of like compute and money and that the equilibrium point is going to be an economic one.”
[56:41](https://www.youtube.com/watch?v=blkIGLlZePg&t=3401s) The Economic Flywheel and the Datadog Bill Reality Check
Martin Casado reflects on the macro trajectory of AI systems, explaining that the true optimization function lies in constraining the environment and the system itself. In his view, the industry has entered a massive structural shift that will play out over decades:
> “And so I I think that we're going to like we're in now in this meta economic equilibrium flywheel where we're kind of optimizing this super general but very inefficient process with a solution set that people value. And I think you listen we've got 30 years of this and a lot of value is going to be created.”
Jad Naous laughs and offers a grounded, humorous anecdote illustrating what happens when bleeding-edge AI meets day-to-day enterprise buyers. When reaching out to prospective clients, the Grepr team would enthusiastically pitch their AI-powered reliability engine, explaining how the platform optimizes telemetry data, reduces costs, and catches hidden bugs before they ever cause an outage. Inevitably, the prospective buyer would cut through the pitch:
> “Wait, wait, wait, wait. Roll back. You said you can save 90% of our Datadog bills. Like that's what I want to buy. But but but the but the AI thing. And they're like, 'No, no, no, no. That's what we want to buy right now.'”
[57:47](https://www.youtube.com/watch?v=blkIGLlZePg&t=3467s) Why Code Sells Faster Than Operations
The conversation turns to the fundamental difference in how customers perceive value across AI for coding versus AI for operations. In coding, the value proposition is instant and visceral. As Jad notes, developers see a coding agent and say, “oh shit it just works.” Martin emphasizes that software engineering salaries represent one of the single highest line items on any tech company’s balance sheet, making developer productivity gains an immediate, no-brainer return on investment.
Operations and site reliability engineering (SRE) operate on a different psychological and financial plane. While Jad points out that preventing downtime, saving business reputation, and boosting SRE productivity have massive real-world economic value, buyers struggle to attach an upfront dollar amount to a disaster that has not happened yet. Furthermore, Martin observes that coding felt like pure magic because it delivered high-quality output far sooner than anyone thought possible, whereas AI for operational troubleshooting is still in its earliest, unproven innings. With that contrast, the two wrap up the discussion with mutual appreciation.
```mermaid
flowchart LR
accTitle: Customer Value Perception: Coding AI vs Operations AI
accDescr: Flow diagram comparing how buyers react to AI code generation versus AI observability and operations.
subgraph CodeAI["AI for Code Generation"]
C1["Instant Magic: 'It just works'"] --> C2["Tied directly to engineer headcount cost"]
C2 --> C3["Immediate, unambiguous ROI"]
end
subgraph OpsAI["AI for Operations & Observability"]
O1["Complex reality: Hypothesis testing & state"] --> O2["Abstract ROI: Avoided downtime & SRE time"]
O2 --> O3["Buyers gravitate to immediate hard savings: 90% Datadog bill reduction"]
end
```
- **Customer reaction to Grepr's pitch**: Buyers bypassed the AI reliability narrative to focus immediately on cutting their observability vendor spend (specifically Datadog bills by 90%).
- **Engineering productivity line item**: Martin points out that developer payroll is universally recognized as a top expense, making developer tooling budgets far easier to unlock.
- **Maturity curve**: Coding agents achieved widespread adoption because the output worked surprisingly well early on; operations AI is navigating a more complex problem space and remains early in its evolution.
## Notable quotes
> [32:30](https://www.youtube.com/watch?v=blkIGLlZePg&t=1950s) It's like every time you add a bit to a program, the state space of like what it can do operationally increases by a factor of two. — Martin Casado
> [55:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=3305s) When you're dealing with the real world like you can solve any problem you want. It's just a matter of like compute and money and that the equilibrium point is going to be an economic one. — Martin Casado
> [07:23](https://www.youtube.com/watch?v=blkIGLlZePg&t=443s) The volume of data that you're collecting and the cost of observability was actually starting to grow faster than the units of function that you're outputting. And the marginal increase in reliability for every additional dollar you were spending on observability was just flattening out. — Jad Naous
## Mentioned
Jad Naous, Martin Casado, Grepr, Stanford University, Nick McKeown, Balaji Prabhakar, Nicira, OpenFlow, SHA-3, MIT, Nikolai Zeldovich, Ramesh Chandra, AppDynamics, Cisco, Andreessen Horowitz (a16z), Jennifer Li, Marc Andreessen, Ben Horowitz, Satish Dharmaraj, Matt Bornstein, Imply, GitHub Copilot, IntelliJ, OpenCode, Claude Code, Codex, Windows, WSL (Windows Subsystem for Linux), Claude 3.5 Sonnet, Claude 3.5 Opus, Apache Iceberg, Apache Parquet, Claude, Sundar, Sebastian Thrun, DARPA Grand Challenge, Uber, Datadog
## Video description
AI can write production code in minutes. But can it debug and operate that code when something goes wrong? Grepr CEO Jad Naous argues that running real-world systems is a fundamentally harder challenge, with unpredictable workloads, failures, and countless possible states.
Jad joins a16z General Partner Martin Casado to discuss how coding agents have transformed engineering inside his 10-person team, including projects that once took six months and now take roughly 10 days.
They also explore whether AI can troubleshoot production systems by detecting changes, forming hypotheses, and running experiments, and why the best engineers may be gaining the most leverage from AI.
Timestamps:
00:00 Intro
00:54 From Stanford to founding Grepr
07:23 Why observability got so expensive
10:15 How AI is changing engineering
18:25 Jad's AI stack and workflow
22:55 A six-month project in ten days
24:55 Can AI reason about operations?
30:10 How Grepr catches incidents early
42:16 Why security is the harder problem
53:33 What comes after code
Resources:
Follow Jad Naous on X: https://x.com/jadtnaous
Follow Martin Casado on X: https://x.com/martin_casado
Learn more about Grepr: https://www.grepr.ai/
Stay Updated:
If you enjoyed this episode, be sure to like, subscribe, and share with your friends!
Find a16z on X: https://twitter.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Follow our host: https://x.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.
While artificial intelligence excels at software synthesis within constrained digital distributions, running distributed systems in production faces combinatorial state explosion and unpredictable runtime dynamics that current models cannot easily navigate, according to Grepr founder Jad Naous and Andreessen Horowitz general partner Martin Casado.
The Productivity Divide in Code Generation (10:15)
AI code generation has dramatically accelerated developer output, but both Naous and Casado argue it acts as an amplifier rather than an equalizer. Naous estimates the performance gap between average developers and top-tier talent has widened tenfold because elite engineers possess the mental models required to decompose complex distributed systems into coherent sub-tasks. Using multi-agent CLI workflows with Anthropic's Claude models under Windows Subsystem for Linux, Naous shipped an automated data lake promotion subsystem in roughly 10 days—a project he estimates previously required six months for an experienced distributed systems engineer. However, this surge in automated code has forced review practices to pivot away from manual syntax checks toward automated test harnesses, regression tests, and golden expectations.
Syntax, Semantics, and Computational Irreducibility (14:35)
A critical boundary separates static code generation from runtime operations. Large language models readily master syntax and local algorithms because they are trained on code repositories, but they remain blind to runtime operational semantics such as access skew, lock contention, cache thrashing, and network partitions. Casado describes many operational problems as computationally irreducible: unlike code generation, where high-dimensional concepts are compressed into language manifolds, operational outcomes cannot be inferred in advance and must be simulated or executed directly.
Leading Indicators vs. Post-Facto SRE Alerts (30:10)
Conventional AI Site Reliability Engineering tools often fail because they rely on lagging indicators, attempting to backtrack through vast systems after an alert or metric spike occurs. Grepr approaches operations by applying large-scale clustering upstream across telemetry streams—handling petabytes of logs per day—to catch novel deviations, such as an exhausted connection pool or an emptied buffer, before an outage manifests. Once an unfamiliar pattern is flagged deterministically, an AI reasoning agent investigates the source code and operational history to formulate and test causal hypotheses. Casado notes that unlike natural operational failures, cybersecurity poses an even steeper hurdle because intelligent adversaries actively exploit rare, weaponizable edge cases.
Economic Equilibrium and Commercial Adoption (52:08)
Commercial traction reflects the gap between code generation and operations. While developer payroll is an obvious, universally recognized line item that makes coding agent ROI immediate, buyers struggle to price insurance against unmanifested downtime. Naous recounts that enterprise buyers frequently brushed past Grepr's automated AI reliability pitch to focus strictly on an immediate 90% reduction in observability spend. Casado compares the operational automation challenge to autonomous vehicles, which required two decades and approximately $170 billion in investment to reach unit economics that still struggle against human alternatives, concluding that deployment ultimately hinges on economic equilibrium rather than pure technical feasibility.
Whether AI can fully automate operations remains contested: Casado expects historical brute-force overprovisioning to persist due to power-law edge cases, whereas Naous contends that systems can be tamed by combining deterministic pattern detection, autonomous hypothesis testing, and restructuring environments to match machine capabilities.
## Background
该视频由 a16z 于 2026 年 10 月 16 日发布在 YouTube 频道上。视频简介指出，AI 能够在几分钟内编写出生产级代码，但 Grepr 首席执行官 Jad Naous 认为，在真实世界中运行系统要困难得多，面临着不可预测的工作负载、故障及无数可能的状态；他与 a16z 普通合伙人 Martin Casado 共同探讨了编程 Agent 如何重塑其 10 人工程团队、AI 能否对生产系统进行故障排查，以及顶尖工程师为何能从 AI 中获得最大杠杆等话题。
## TL;DR
对话深入探讨了 AI 能够高效写代码却极难自主接管系统运维的核心根源：代码语法属于受限的数字流形，而生产环境运维则面临重尾分布、状态空间指数级爆炸以及计算不可约性等现实物理挑战。Jad Naous 结合自身在 Grepr 的实践，分享了他使用 WSL、命令行与多 Agent 并行开发将 6 个月数据湖核心功能压缩至 10 天上线的极致工作流，并指出 AI 正在让优秀工程师产生 10 倍的马太效应。针对“AI SRE”的落地困境，他们剖析了传统因果回溯的失效原因，阐述了通过底层流式处理捕获前置异常信号并交由上层 Agent 进行推理验证的解法。最终双方达成共识，相比于在数字受限域的快速突破，现实世界的运维与物理落地必将走向重新重构环境以迎合 AI、在算力与成本之间寻找平衡的元经济学均衡。
## Atmosphere & style
整场对谈兼具老友重逢的轻松诙谐与硬核技术探讨的深度穿透力。作为昔日斯坦福同门与投资领域的旧友，两人在调侃学界往事、抽烟轶事以及在 Windows 环境下写代码等幽默段子中迅速拉近距离，现场笑声不断。对话没有浮于表面的 AI 概念吹捧，而是充满架构师层面的激烈交锋与理性交火：从语法与语义的哲学辩析、4MB 状态空间的宇宙学比喻，到现实中客户“别谈 AI、先帮我砍 90% 账单”的真实商业痛点，听众既能感受到技术飞跃带来的工程震撼，又能深切体会到资深系统工程师面对现实物理世界复杂度时的敬畏与审慎。
## Bulletin
- [22:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=1375s) Grepr CEO Jad Naous 依靠 WSL 终端与 Claude Code 建立的多 Agent 架构，在约 10 天内独立交付了传统资深工程师需要攻坚 6 个月的数据湖“自动字段提升”生产级功能，开发杠杆放大约 10 到 20 倍。
- [13:13](https://www.youtube.com/watch?v=blkIGLlZePg&t=793s) 大模型在软件工程中引发了严峻的马太效应：AI 并未抹平能力差距，反而赋予高水平工程师 10 倍杠杆，因为只有具备处理高复杂度宏观架构能力的人脑才能完成解耦与指令拆解。
- [32:30](https://www.youtube.com/watch?v=blkIGLlZePg&t=1950s) 与代码文本在有限流形上的训练不同，生产运维面临指数级状态爆炸（每增 1 比特理论运维状态翻倍）和极端重尾分布，导致目前 AI 接管分布式集群和复杂排障仍步履维艰。
- [57:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=3425s) 商业落地上，相较于将“提升可靠性”折算成抽象价值，企业客户对立竿见影的成本削减表现出压倒性购买意愿，Grepr 凭借在上游数据流削减 90% Datadog 账单迅速敲定采购。
## Key takeaways
- [06:32](https://www.youtube.com/watch?v=blkIGLlZePg&t=392s) 传统微服务架构导致遥测数据呈爆炸式增长，企业在可观测性上花费相当于基础设施 5% 到 20% 的预算，但每多花一美元所换来的边际可靠性已接近断崖式跌平。
- [09:23](https://www.youtube.com/watch?v=blkIGLlZePg&t=563s) Grepr 采用上游实时流式处理架构，在数据源头过滤噪声并提取关键信号，既帮助企业削减 90% 的存储与索引成本，又为上层 AI 运维引擎储备高价值数据底座。
- [13:13](https://www.youtube.com/watch?v=blkIGLlZePg&t=793s) AI 正在拉大顶尖与普通工程师的差距：AI 无法代劳宏观系统思考，只有能深度掌握系统全局的架构师才能有效将其解耦成模型可执行的微观任务。
- [14:54](https://www.youtube.com/watch?v=blkIGLlZePg&t=894s) 语法与运行时语义存在本质鸿沟：大模型极其擅长生成合规的代码语法，但系统运行时的实际状态转移、配置交互与极端边界行为无法仅凭静态代码推断。
- [20:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=1205s) 基于 WSL 极简命令行的多 Agent 并发工作流是极致提效的关键：直接在终端启动具备持久会话的独立 Sub-Agent 分别负责设计、互审与并行编码，最终收敛至单主干由人类复核。
- [28:39](https://www.youtube.com/watch?v=blkIGLlZePg&t=1719s) 在基准明确的离线场景下 AI 表现出色，但一旦涉及“该分配多少物理内存”等动态资源配置，缺乏先验认知的 AI 必须依赖真实生产实验甚至试错。
- [30:10](https://www.youtube.com/watch?v=blkIGLlZePg&t=1810s) 市面上依赖指标尖峰等滞后告警进行因果回溯的 AI SRE 工具往往失效；有效的排障应依赖源头聚类模型捕捉极其细微的早期异常日志模式（如连接或缓冲区耗尽）。
- [03:59](https://www.youtube.com/watch?v=blkIGLlZePg&t=239s) 系统工程史上的核心经验往往不是依靠智能算法精准预测，而是通过过度配置（如直接堆满物理内存与网络带宽）来规避动态管理的高风险。
- [03:59](https://www.youtube.com/watch?v=blkIGLlZePg&t=239s) 尽管生产故障的状态空间如宇宙原子般庞大且具重尾特征，但人类工程师定位问题的“排障减枝工作流”是可管理的，AI 接管运维的关键在于学习该工作流而非单步黑盒预测。
- [04:01](https://www.youtube.com/watch?v=blkIGLlZePg&t=241s) 安全攻防在状态空间上面临更高维度的对抗性挑战：运维异常多源于系统自然并发或人为失误，而安全威胁则来自怀有恶意的主动攻击者蓄意构造罕见脆弱路径。
- [04:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=245s) 生产运维的另一大核心壁垒在于碎片化的人类隐性知识，故障排查往往需要串联分散在多个团队大脑中的背景信息，无法单靠读取代码与遥测数据闭环。
- [55:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=3305s) 面对复杂现实世界中的计算不可约性，历史经验往往不是让 AI 强行适配混乱的现实，而是重塑外部物理或软件环境规则（如自动化立体仓库），使其适配自动化工具。
- [58:13](https://www.youtube.com/watch?v=blkIGLlZePg&t=3493s) 代码生成与生产运维在落地感知上存在代际差异：代码工具直接对应昂贵且直观的研发工时，而防患于未然的运维稳定性难以折算即时 ROI，导致商业化普及更漫长。
## Detailed walkthrough
### [00:00](https://www.youtube.com/watch?v=blkIGLlZePg&t=0s) 引言：AI 编码与现实系统运维的鸿沟
节目开门见山地指出，AI 目前所生存的世界完全是数字化的，而真实的物理系统与生产环境则复杂得多，两者之间存在着巨大的断层。回顾 AI 在编程领域的发展轨迹，它最初只作为代码自动补全（autocomplete）工具出现，当时几乎没有人能预料到它能够构建出复杂的软件系统。但随后，随着大模型能够容纳和处理越来越高的系统复杂度，AI 彻底打破了大家的传统预期。
然而，生产系统的“运维”（Operations）是否具有与编写代码相同的约束条件，目前仍是一个巨大的问号。Martin Casado 抛出疑问：业界究竟是否存在足够的数据，来训练出一个能够透彻理解全部底层运维操作的 AI 系统？Jad Naous 认为我们其实拥有足够的数据，但核心挑战在于系统的无尽复杂度，以及人类如何将那些微妙、散乱的运维经验转化为 AI 可以进行逻辑推理的形式。过去很长一段时间里，运维分布式集群的“行业公开潜规则”（dirty secret）其实就是靠人类工程师在幕后人肉维持运转。两人试图探讨：我们是否已经到了可以用 AI 去自主运行这些庞大复杂服务的临界点。
> “The world that the AI lives in is all digital and there's the actual world and like those things are not necessarily the same.” —— Jad Naous
---
### [00:54](https://www.youtube.com/watch?v=blkIGLlZePg&t=54s) 斯坦福往事与早期学术探索
Martin Casado 热情欢迎 Grepr 创始人兼 CEO Jad Naous。两人不仅在技术圈相交多年，早年在斯坦福大学攻读博士时更是同在一个实验室。Martin 幽默地回忆起刚见到新学弟 Jad 时的第一印象：当时他在办公室看到桌上放着一个酷炫的摩托车头盔和一包万宝路香烟，心里暗想“这家伙简直太酷了”，随后打趣追问 Jad 现在的酷劲儿都去哪儿了。Jad 笑着自嘲那正是自己人生的“酷炫巅峰”，并爆料当时甚至还因为在 Gates 计算机大楼外抽烟，被两人的共同导师 Nick McKeown 当场撞见并投以异样的目光。Martin 还顺带提到了当时斯坦福唯一抽烟的教授 Balaji Prabhakar。
在学术路径上，Martin 比 Jad 早入学约四年。当 Martin 毕业并创立网络虚拟化先驱公司 Nicira 时，Jad 选择留在斯坦福继续深造。Jad 在底层网络与硬件安全领域投入了大量精力，深入研究了 OpenFlow，并在博士论文期间用硬件构建了业内首批 SHA-3 密码学哈希算法实现方案，以验证其在实际硬件环境中的可行性。不过，在深入研究网络技术后，Jad 却得出了一个出人意料的结论：他发觉自己其实并不喜欢传统网络，因为说到底它“只是把比特从一个地方挪到另一个地方”，虽然很多人觉得其中大有学问，但他认为这缺乏吸引自己的终极深度。
> “I decided after a while that I actually don't like networking because it's just about moving bits from one place and putting them in another place.” —— Jad Naous
---
### [04:02](https://www.youtube.com/watch?v=blkIGLlZePg&t=242s) MIT 博士后、安全初创公司与加入 a16z
离开斯坦福后，Jad 前往 MIT 与计算机系统和安全领域的大牛 Nikolai Zeldovich 合作开展博士后研究。在那里，他聚焦于应用安全（Application Security），并与当时的博士生 Ramesh Chandra 共同将研究成果商业化，创立了一家初创公司。这项技术的核心构想极其超前：当一个系统遭受黑客利用漏洞（exploit）攻击时，系统能否自动追踪所有相关依赖与操作，将整个应用的世界状态“回滚”（rollback）到未受损状态，然后将这些干净的状态平滑重放到打好修复补丁的应用新版本之上。然而，当他们试图在混乱的真实生产环境中落地时，发现运维状态的边界几乎是无限发散的，在当时的技术条件下根本无法约束其状态空间。最终他们虽无限接近成功，但技术在那个时代过于超前，项目只能作罢。
随后，Jad 加入了应用性能监控巨头 AppDynamics。在他在职的三年半时间里，公司团队规模从 200 人激增至 2000 人，并最终被思科（Cisco）以巨资收购。戏剧性的是，Jad 仅仅做了“一天的思科员工”就选择了离开。此时 Martin Casado 向他抛出橄榄枝，力邀他加入 Andreessen Horowitz（a16z）担任投资合伙人。Jad 坦言自己当时对风险投资一无所知，自认是个纯粹的架构师和系统工程师，而 Martin 告诉他“懂的越少越好”，并提醒他跨界的第一要务是“学会慢下来”。在 a16z 的两年多时间里，Jad 与 Martin、Jennifer、Mark、Ben、Satish 以及 Matt 等行业顶尖投资人并肩作战，这段经历彻底重塑了他的思维模型，帮他褪去了学术界的条条框框，理解了商业世界的宏观运转方式。
> “The question I think is maybe about complexity and like to what extent can you take the things that you've described that are operational and convert them into something that an AI can reason about...” —— Jad Naous
- **学术与职业转折点**：
- 斯坦福大学：师从 Nick McKeown，研发 OpenFlow 硬件及首套 SHA-3 原型。
- MIT 博士后：师从 Nikolai Zeldovich，与 Ramesh Chandra 探索利用系统状态回滚与热补丁进行自动化安全防御。
- AppDynamics：见证公司从 200 人扩张至 2000 人直至思科并购，仅当了一天思科员工。
- a16z 投资人：与 Martin 紧密合作，完成从底层工程师到宏观商业思维的“大脑重构”。
---
### [06:32](https://www.youtube.com/watch?v=blkIGLlZePg&t=392s) 可观测性成本失控与 Grepr 的诞生背景
离开 a16z 后，Jad 加入 Imply 担任首席产品官（CPO），负责搭建并扩展产品、工程、信息安全和设计团队，带领团队从 15 人规模扩张到 90 多人。经过大约三年的组织搭建，各个业务模块都配备了成熟的主管，Jad 发现自己每天的时间都被无休止的跨部门协调和例会所占据，这促使他决定重新回到一线去解决底层系统难题。2023 年，在与众多工程研发高管（VP of Eng / CTO）深度交流时，Jad 捕捉到一个普遍现象：当时所有团队都在削减 IT 开销，而除了核心云基础设施之外，账单上最昂贵、增长最不可控的开销正是“可观测性”（Observability）。
深入调研后 Jad 发现，企业普遍要拿出相当于基础设施支出 5% 到 20% 的预算去购买各类可观测性工具。但工程主管们内心真正想购买的并不是单纯的数据存储或图表，而是系统的“高可用与可靠性”（Reliability），可观测性仅仅是他们为可靠性买单的代理品（Proxy）。然而随着架构向超小规模、垂直集成的微服务（Microservices）深度演进，每一层产生的数据呈爆炸式膨胀，数据的收集与存储开销增速已经远远超过了企业实际业务功能的产出增速。企业在可观测性上多花一美元所换来的边际系统可靠性正在断崖式跌平。基于此，Jad 创立了 Grepr：与其把海量无用数据（大部分是心理安慰和防意外的“保险冗余”）一股脑丢到下游并为此支付天价存储与索引账单，不如在上游网络流中直接进行实时信号与噪声提取，在数据源头榨取价值并极大压缩下游开支。
> “And the marginal rate of or like the marginal increase in reliability for every additional dollar you were spending on observability was just flattening out.” —— Jad Naous
```mermaid
flowchart TD
accTitle: 可观测性边际效应递减与 Grepr 上游流式处理架构
accDescr: 展示了传统微服务架构数据爆炸导致边际可靠性降低，以及 Grepr 采用上游实时信号提取降低下游成本并恢复高可靠性的机制。
subgraph Problem["传统架构瓶颈"]
A["微服务与基础设施"] -->|"产生海量全量数据 (多为噪声)"| B["下游中心化可观测性平台"]
B -->|"吞噬 5%-20% 云基础预算"| C["成本急剧攀升"]
B -->|"边际可靠性改善趋近于零"| D["可靠性边际收益递减"]
end
subgraph Solution["Grepr 上游流式处理方案"]
E["生产系统原始遥测数据"] --> F["Grepr 上游实时信号/噪声分离"]
F -->|"丢弃无用噪音/保险数据"| G["大幅削减下游存储与查询成本"]
F -->|"提取高质量关键指标"| H["高价值信号直接赋能系统可靠性"]
end
```
### [09:23](https://www.youtube.com/watch?v=blkIGLlZePg&t=563s) Grepr 的数据管道与双层战略架构
Grepr 创立的切入点是解决遥测数据成本爆炸的问题，通过将自身置于企业的数据流中，提供即插即用的管道服务。“你把我们放进数据流里，即插即用，数据穿流而过，就能直接将成本削减 90%。”但将数据精简只是第一步，团队在早期就敏锐地察觉到了 AI 的崛起，并确立了一个更具防御性的长线战略：处于数据流的关键位置并提取出核心信号后，就能在未来的技术演进中占据关键的战略高地，将这些高价值信号输入到任何下游系统，用于驱动系统的可靠性与日常运维。
这一逻辑构成了 Grepr 的双层架构体系。随着模型能力的发展，团队目前正在全力构建第二层业务：将流式处理中捕获的信号直接喂给上层 AI，由 AI 接管日常的故障排查、根因定位与线上修复。
```mermaid
flowchart TD
accTitle: Grepr 双层产品架构
accDescr: 展示从原始遥测数据流到成本削减及 AI 运维决策的双层系统流动。
subgraph L1["第一层：数据流处理 (Data Stream Layer)"]
Raw["原始遥测数据流 (Telemetry Data)"] --> Pipeline["Grepr 管道过滤与提取"]
Pipeline --> Red["削减 90% 存储与分析成本"]
Pipeline --> Signal["高价值核心信号 (High-Value Signals)"]
end
subgraph L2["第二层：AI 智能运维 (AI Operations Layer)"]
Signal --> AI["AI 排障与运维引擎"]
AI --> Debug["故障排障与异常诊断"]
AI --> Fix["线上自动修复 (Self-Healing)"]
end
```
### [10:15](https://www.youtube.com/watch?v=blkIGLlZePg&t=615s) AI 赋能开发：从智能补全到人机共生学习
Martin Casado 抛出了一个核心行业命题：AI 究竟在如何根本性地重塑软件工程？作为一个拥有深厚架构底子、曾领导过产品和工程部门、如今带领着 10 人精简团队快速交付的资深工程师，Jad Naous 见证了这场生产力范式的剧变。
Jad Naous 回顾道，团队刚起步时，AI 还停留在非常初级的状态——大家用着 IntelliJ IDEA 和 GitHub Copilot，“我们当时震惊于‘天啊，看看它，居然能自动补全这些测试用例’，但冷静下来一想，它本质上终究只是自动补全（Autocomplete）而已。”
然而技术迭代的速度远超预期。Jad Naous 观察到的一个核心质变是：真正顶尖的工程师不会将 AI 仅仅当做打字助手，而是将其内化进工作流中，以至于演变成一种双向训练的状态——“我注意到的是，真正优秀的工程师能够以某种方式将它融入自己的工作流程中，甚至让 AI 反过来训练他们。”这种反向训练让工程师学会如何向模型传达意图、如何精准提问并榨取模型的极限上限。
- **模型与 Agent 外壳的适配**：高效开发者不仅要适应不同模型本身的微调偏好，更要深度理解外壳环境（Harness）的特性。团队尝试过 OpenCode、Codex 以及 Claude Code 等不同方案。
- **防止 Token 浪费的防线机制**：外壳对上下文窗口管理、子代理（Sub-agent）分发策略差异巨大。例如使用 Claude Code 时，若不明确施加限制指令，它可能会一口气自动派生出 20 个子代理去做基础调研，瞬间烧光你的 Token 配额。工程师必须对外壳与底层模型的交互机制有高度敏锐的认知。
### [13:13](https://www.youtube.com/watch?v=blkIGLlZePg&t=793s) 程序员差距拉大：为什么 AI 让高水平工程师产生 10 倍杠杆
随着团队的扩充，Jad Naous 发现了一个残酷的现实：普通工程师与顶尖工程师之间的生产力差距，在 AI 的加持下直接被放大了 10 倍。针对 Martin Casado 提出的核心疑问——AI 究竟是让优秀的人变得更优秀，还是把普通人变成了优秀工程师？Jad Naous 给出了极其明确的否定回答：普通工程师绝不可能仅靠 AI 突然蜕变成高手。
这种马太效应的根本原因在于认知负荷与复杂系统的解构能力。“归根结底，你必须真正理解系统的全貌，因为你的大脑能够处理的复杂度越高，你就越能将复杂任务拆解并恰当地分配出去。”
Martin Casado 敏锐地指出，在行业内普遍鼓吹“AI 降低门槛使所有人扁平化”的背景下，这种“优秀工程师通过 AI 获得不成比例的巨额提升”的论断在很大程度上是一个“非共识观点”（Non-consensus opinion）。Jad Naous 则强调，这在构建分布式系统与核心算法时是绝对颠扑不破的真理：
- **复杂认知无法外包**：AI 无法代劳宏观逻辑，“针对我们解决的问题，没有哪个模型能真正独立思考我们正在构建的东西。”人类大脑必须牢牢掌控宏观架构与复杂性。
- **查询翻译层的现实案例**：例如构建数据查询翻译层（Query Translation Layer）时，系统涉及深度查询优化，工程师必须先亲自透彻理解查询语义（Semantics），设计出语义解析的路径，然后将其清晰传授给模型，再指挥模型编写具体的翻译转换层。
### [14:54](https://www.youtube.com/watch?v=blkIGLlZePg&t=894s) 语法（Syntax）与语义（Semantics）的博弈
关于开发复杂性的讨论引申出了一段经典的计算机科学讨论：为什么单纯写代码（语法）容易，而让机器正确运行（语义）极难？
Jad Naous 用通俗的生活常识做类比：“语义是事物的真正含义。比如‘狗’这个词的语义，是指那只长着四条腿、会汪汪叫的实际动物；而语法只是‘由三个字母组成的单词’这一表面形式。”不同的词语法外观迥异，但语义可能高度重合。
Martin Casado 随后将其拉回计算机科学语境，总结出二者的本质差别：“在计算机科学中，我最简单的描述方式是：语法是你敲下的字符，而语义则是机器在运行时实际执行的动作。”
- **语法（Syntax）**：代码的字面文本与编写规范，这是当下大模型最擅长解决的层面。
- **语义（Semantics）**：机器在实际运行时表现出的状态转移与行为。有些系统的运行时语义甚至根本无法通过阅读静态语法代码直接推导出来，必须通过动态测试和环境交互才能获知。
两人在讨论中还就性能与语义的边界进行了风趣的交锋。Martin Casado 举了 SQL 的例子，指出一段看似完全合法的查询代码，其真实耗时与底层数据分布、表结构紧密相连；Jad Naous 则严谨地澄清，这部分更多属于“性能特征”，而在数据库语义层面，系统配置会直接改变语义规则——比如某些配置下 `NULL` 会被直接当作 `FALSE` 处理，如果脱离系统的宏观配置与运行环境，静态代码的真实行为根本无从得知。Martin Casado 随即打趣两人出身背景的不同（“你学的是计算机科学，我学的是电气工程；结果你成了计算机科学家，我成了风险投资人，问题大概就出在这儿”），并将讨论迅速拉回主线：当前行业的核心分歧，正是“写出代码文本”与“保证系统按预期运行”之间的巨大鸿沟。
### [18:01](https://www.youtube.com/watch?v=blkIGLlZePg&t=1081s) Jad Naous 的开发工作流：WSL 与极简命令行
谈到在工程实践中如何将过去数月的工作压缩到几天，Jad Naous 透露了自己的开发环境。出乎许多人的意料，作为一名构建顶尖硬核底层基础设施的工程师，他的主操作系统不是 macOS，而是 Windows。这一选择立刻引来了 Martin Casado 的大笑调侃：“什么？那是微软这家公司做出来的东西吗？”
Jad Naous 进一步详细拆解了自己的极简高效配置：
- **Claude Code 驱动**：核心利用 Claude Code 作为工程智能体外壳。
- **WSL（Windows Subsystem for Linux）作为绝对核心**：所有的开发、编译、运行都在 WSL 里的纯原生 Linux 环境下执行。
- **纯粹的无界面命令行（Headless CLI）**：开发环境中完全没有任何 X Window、图形界面（UI）或冗余桌面插件，完全靠极度精简的纯命令行终端配合 AI Agent 高速敲定指令并完成项目开发。通过这种极度专注底层语义与终端环境的流式操作，他能够以单兵作战的方式完成传统团队耗时数周乃至数月才能交付的复杂特性。
### [18:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=1135s) Jad 的极简工具链：Windows、WSL 与 Claude Code
两人的对话从一段略带调侃的技术习惯切入。Martin Casado 惊讶地问 Jad Naous 为什么至今还在用 Windows，甚至打趣他是不是平时要随手切屏去打《反恐精英》（Counter-Strike）。Jad 笑着解释，原因极其纯粹：他从来不想花时间去死记 Mac 上那些繁琐的键盘快捷键，更难以忍受单键鼠标和大量组合修饰键，“我只是觉得没必要浪费时间”。
在具体开发环境上，Jad 采用的是 Windows 结合 WSL（Windows Subsystem for Linux）的组合。他称赞微软在 WSL 上的工程实现非常出色，直接通过虚拟机监视器（Hypervisor）在 Windows 内运行真正的 Linux 内核，提供了顺畅的原生 Linux 体验。而在编辑器与终端之上，他最近依赖的核心编码工具是配合 Claude 3.5 Sonnet / Opus（对话中提及 Opus 5 / 5.5 等模型）使用的 Claude Code 命令行工具，这构成了他高吞吐量写代码的底层平台。
### [20:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=1205s) “多 Agent 协同”工作流：在命令行中拆分架构、实现与 Code Review
利用大语言模型写代码，Jad 摸索出了一套与众不同的工程范式。最近两周，他一口气向团队提了包含 20 多个 Pull Request（PR）的庞大改动，直接把代码“甩”给团队去审查。Martin 开玩笑说，现在的管理结构难道变成了“下属给 CEO 做 Code Review”吗？Jad 澄清，事实上整个团队都在用这种节奏疯狂提 PR，彼此互相审查，而团队内部的 Review 重心也因此发生了剧烈转变：大家不再纠缠于语法细节，而是高度聚焦于测试用例、黄金基线预期（Golden Expectations）以及系统行为是否发生未预期偏移。
Jad 透露了他最近发现的关键技巧：**不要依赖单模型内部的 Sub-Agent，而是让 Claude 在命令行中直接启动独立的 Sub-Agent**。因为命令行进程能长期保留持久会话（Persistent Session），不仅支持跨终端协同，还能极大节省模型的上下文窗口（Context Window）。在日常工作中，他通常同时保持三到四个会话窗口并行工作，因为一旦超过这个数量，人类工程师就会因为 Agent 完成速度过快而陷入上下文混乱与疲于切换。
整个自动化软件开发流程涵盖了从设计文档到多 Agent 审查与并发执行的全周期：
```mermaid
flowchart TD
accTitle: Jad Naous 的多 Agent 协同开发流水线
accDescr: 展示了从人类输入到 PRD 审查、并行编码及最终单堆栈人类复核的完整流程
subgraph Phase1 ["1. 方案设计与自动化审查"]
A["提出需求与架构目标"] --> B["生成 PRD 与 Design Doc"]
B --> C["启动 Tech Lead 审查 Agent"]
C -->|"直接通过命令行交互质询"| B
end
subgraph Phase2 ["2. 任务解耦与并发落地"]
B --> D["人类架构复核<br/>(确认算法/分工/PR依赖)"]
D --> E["主实现 Agent 拆解任务"]
E --> F1["命令行 Agent 1<br/>(持久会话)"]
E --> F2["命令行 Agent 2<br/>(持久会话)"]
E --> F3["命令行 Agent 3<br/>(持久会话)"]
end
subgraph Phase3 ["3. 堆叠整合与人类把关"]
F1 --> G["统一合并进单主干堆栈"]
F2 --> G
F3 --> G
G --> H["人类工程师逐个审查 PR 与测试"]
end
```
在这套结构中，各 Agent 之间不是通过 GitHub PR 交互，而是直接在命令行层面质询互审。例如主实现 Agent 会向扮演 Tech Lead 的 Reviewer Agent 提问“这样写是否合理”，双方围绕设计文档和算法拆分进行反复打磨。当设计确定后，主 Agent 负责识别可并行化的部分，分发给独立的命令行 Agent 去并发编写代码。Jad 强调，无论并发多么充分，最终必须把所有成果收敛回一个线性堆栈中由人类逐个过目，否则分支结构会变得异常臃肿杂乱。
### [22:55](https://www.youtube.com/watch?v=blkIGLlZePg&t=1375s) 生产级数据湖研发：单人 10 天干完 6 个月的工作量
Martin 明确确认，Jad 所说的不是玩具 Demo 或前端原型，而是真真正正要上线运行的核心基础设施生产代码。在被问及当下的效率与数年前相比有何变化时，Jad 拿出了他刚交付的重磅项目——数据湖中的“自动字段提升”（Auto-promotion）或称“数据粉碎”（Data Shredding）。
在现代数据湖（如 Parquet、Apache Iceberg 架构）中，日志数据通常是半结构化的，包含动态任意字段。为了实现极速分析，最理想的状态是将日志事件中的每一个键（Field）自动提拔为独立的物理列，以便在查询时可以独立于其他行进行扫描过滤。但要实现这一点在工程上极其复杂：涉及底层元数据系统的协调、元数据提交协议的并发控制，以及多写入节点之间的一致性保障。
- **历史研发周期**：单名资深工程师需要持续攻坚约 6 个月。
- **当前交付周期**：Jad 依靠大模型协作在 **10 天左右** 独立完成上线。
- **效率跃升幅度**：在全新产品的底层生产代码层面上，实现了 10 到 20 倍（Factor of 10 to 20）的杠杆提升。
### [24:47](https://www.youtube.com/watch?v=blkIGLlZePg&t=1487s) 运行态的困境：AI 能理解分布式系统的复杂运维与权衡吗？
面对这样惊人的提速，Martin 提出了一个根本性的工程疑问：模型之所以能写出语法正确的代码，是因为它吃透了海量开源代码；但核心基础设施和分布式算法的真正难点，往往不在于语法，而在于系统运行时的动态表现（Operational Properties）。一个分布式系统的性能、可用性与正确性，完全取决于实际的生产工作负载、流量模型和访问模式（Access Patterns）。
- **可用性与正确性权衡**：采用加锁会导致暂时不可用，采用最终一致性（Eventual Consistency）则意味着无锁读取下可能读到脏数据。
- **性能权衡**：是否增加缓存、如何排布内存，都取决于未知的访问频率。
Martin 直言，由于模型在写代码时根本看不到线上的动态流量，连训练数据中也几乎没有关于现实复杂运维状态的数据，今天的分布式集群仍然高度依赖人类运维和调优。他甚至分享了自己正在写的一个分布式高斯泼溅（Gaussian Splatting）渲染器的经历：由于几百万个泼溅图元需要在内存与显存之间根据运行时状态动态换页与流式调度，现有大模型在这一场景下的表现极为糟糕。他抛出核心问题：我们究竟能否将生产环境中的复杂运维状态，转化为 AI 能够进行逻辑推理的形式？
Jad 对此回应，Grepr 团队目前的做法是极其严密地在上下文中向模型定义好性能基准测试、系统运行规范，以及在流式处理器（Stream Processor）内部编码时的硬性约束。但这并不意味着模型能自主做对。他谈到自己手头正在推进的一个 SQL 管道代码生成与查询优化项目：当模型自动生成的代码性能极差、不符合生产标准时，工程师必须精准介入，直接告诉它：“你生成的代码完全是垃圾，必须重写优化。”通过严格的基准和反馈回路，逼迫模型在约束空间内逐步调整，逼近生产可用的性能标准。
### [28:39](https://www.youtube.com/watch?v=blkIGLlZePg&t=1719s) 代码优化与生产环境运维的本质区别
Jad Naous 详细讲述了此前让 AI 协助进行查询优化的过程。起初，AI 完全没有意识到代码需要做任何优化，表现得一副“这看起来挺好，能有什么问题”的样子。直到 Jad 明确指出问题：“看这里，你写的这些合取项（conjuncts），某个特定术语被重复计算了，本来应该直接去掉。”AI 这才恍然大悟，随后展开专门的排查线程，提出了一整套关于如何优化的假设。由于 Jad 为它提供了真实的用户查询数据集作为基准测试，AI 能够将优化扎根在具体的性能指标上，从而把这部分任务完成得相当漂亮。
```mermaid
flowchart TD
accTitle: 静态代码优化与动态生产运维对比
accDescr: 对比AI在面对代码优化与运维参数决策时的两种不同路径，突显生产环境的不确定性。
subgraph S1["代码基准优化"]
A["指出代码冗余项"] --> B["AI 建立优化假设"]
B --> C["基于查询数据集跑 Benchmark"]
C --> D["优化成功落地"]
end
subgraph S2["运维决策困境"]
E["询问资源配置: 该分配多少内存？"] --> F["AI 缺乏上下文与先验认知"]
F --> G["必须依赖生产环境真实实验"]
G --> H["生产环境不可预测: 引发 Incident"]
end
```
然而，一旦问题从纯静态代码转向生产环境的动态运维，AI 立刻暴露出根本短板。Jad 举例说，如果你去问 AI：“我应该给这个系统分配多少内存？”AI 根本毫无头绪。Jad Naous 强调：“It has no idea and that has to come from actual experiments that run in production.”（它完全不知道，这必须来自于在生产环境中运行的真实实验。）目前这类系统在运维端唯一做得不错的地方，仅仅是它能意识到自己答不上来，从而尝试去“运行某种实验”来获取信息。但在真实的生产环境中，系统的动态表现往往违背预期，而这正是线上故障（incidents）频繁发生的根源。
- AI 依靠具体的数据集基准（performance benchmarks）能够较好地完成离线算法与查询逻辑的迭代。
- 涉及系统物理资源分配（如内存、连接数）时，无法仅靠静态代码推理给出答案，必须依赖动态测试。
### [30:10](https://www.youtube.com/watch?v=blkIGLlZePg&t=1810s) 从滞后指标到前置异常捕获：Grepr 的运维检测逻辑
Martin Casado 抛出了一个核心疑问：业界目前常说的“AI SRE”（利用 AI 摄取系统运行日志并调优）到底靠不靠谱？Martin 直言，据他从其他从业者那里了解到的情况，反响普遍很差。Jad 对此解释道，Grepr 确实在运维和安全两个领域实践这一方向，但切入角度与常规的 AI SRE 截然相反。市面上多数产品都依赖滞后指标（lagging indicators）——比如某个监控指标突然飙升或触发了报警，再试图让 AI 逆向穿透极其复杂的分布式系统去追溯根因。这种反向推导因果（backtracking into a root cause）的做法在工程上极难成功。
Grepr 采取了完全逆向的思路：不从滞后警报倒推，而是从源头实时检测环境中是否出现了“前所未有的变化”。系统在底层执行大规模聚类，在海量日志中寻找那根“草堆里的针”。Jad 指出，当系统出故障时，它通常会在早期以某种微弱的信号发出呼救，例如抛出一条“连接耗尽”（error connections ran out）、“内存超限”（memory exceeded）或“资源缓冲区耗尽”（resource buffer is emptied）的底层报错。Grepr 捕捉的正是这种极早期的单一异常日志，并以此为线索展开调查。这套机制在 Grepr 内部及部分客户环境中运行良好，Jad 透露了一个极具说服力的细节：团队曾有几次忽略了该机制捕获的早期告警，结果那几次无一例外全部演变成了严重的生产故障。
- 传统 AI SRE 的死穴在于依赖指标尖峰等滞后警报做因果回溯，面对复杂拓扑往往失效。
- 真正关键的早期预警通常隐藏在细微的日志文本中（如连接数耗尽或缓冲区溢出错误）。
- 忽视早期的离群日志模式，直接导致了系统随后在生产环境中出现真实故障。
### [32:30](https://www.youtube.com/watch?v=blkIGLlZePg&t=1950s) 状态空间的指数爆炸与分布式集群的“脏秘密”
Martin Casado 从理论视角对 AI 接管运维提出了极大的质疑。他分析道，AI 擅长代码补全是因为它训练了全人类写过的所有开源代码，这些语料覆盖了足够广阔的有效场景；但软件的生产运维却完全没有边界。Martin 提出了一个尖锐的论点：“It’s like every time you add a bit to a program, the state space of like what it can do operationally increases by a factor of two.”（就像你每给程序增加一个比特，它在运维操作上的状态空间就会翻倍。）这种指数级增长会让系统的运行状态迅速超越全宇宙的原子总数，这也是为什么诸如混沌工程（chaos engineering）等产品化方案始终无法彻底解决运维难题。
Martin 更直言不讳地揭露了分布式系统的行业真相：长期以来维持分布式集群稳定运行的“肮脏秘密”（dirty secret），其实一直都是人类运维工程师在肉身硬扛。不仅如此，要训练一个能够接管集群的 AI，根本没有足够的全局数据和配置样本。Jad 承认运维状态空间近乎无限，很多上下文甚至不写在系统里，而是存在于现实人际交互中——比如某个工程师在茶水间偶遇同事时，才知道对方昨天偷偷上线了一个新变更。但 Jad 坚持认为，目标并非一开始就让 AI 接管全知全能的闭环，而是通过更完备的遥测（telemetry）将系统状态显性化。绝大多数在线上爆发的非预期事故，前提只要系统之前是好的，本质上都是某种“变化”引起的，只要能紧咬变更，问题就能大幅收敛。
- 程序每增加一个比特，其运维维度的理论状态空间就会呈指数级（2倍）暴增。
- 传统的混沌工程产品未能在工业界彻底攻克运维自动化难题，人工介入仍是底层支柱。
- 隐性知识（如未同步的口头部署变动）往往游离在遥测系统之外，成为排障盲区。
### [35:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=2108s) 预测模型与推理 Agent 的分工协同
针对 Jad 的阐述，Martin 提出了一个关于技术架构的本质困惑：生成式大语言模型（LLM）向来不擅长做数值预测和基线异常检测，这通常属于传统预测模型（predictive modeling）的领地，两者到底是如何串联的？Jad 澄清了误解：AI Agent 的任务绝不是去预测变化，而是以确定性的方式接收“变更已发生”的信号。
```mermaid
flowchart LR
accTitle: Grepr 运维双层协作架构
accDescr: 展示底层预测聚类模型与上层LLM推理智能体之间的协作流水线。
subgraph DataPlane["底层数据面"]
A["每日 PB 级原始日志"] --> B["大规模聚类与模式检测"]
B -->|"发现全新异常模式"| C["确定性信号触发"]
end
subgraph ReasoningPlane["上层推理面 (AI Agent)"]
C --> D["溯源: 检视源代码与函数逻辑"]
D --> E["比对: 历史出现频次与上下文"]
E --> F["验证: 并发跑假设与启发式规则"]
F --> G["决策: 判定严重等级与处置动作"]
end
```
Jad 剖析了 Grepr 的双层协同架构：底层系统每天摄取处理高达 PB 级别的日志数据，利用高效的预测模型进行扫描与聚类。一旦底层捕捉到此前从未见过的崭新日志模式，就会把这一确定性事实作为信号推送给上层的 AI Agent。此时，大模型的推理优势才真正介入：“它会去看代码，搞清楚这个报错信息来自代码的哪一行，理解那段代码具体在执行什么逻辑，去排查它过去是否发生过，并运行一系列启发式规则和假设来评估其影响，最后决定该采取什么行动。”
当 Martin 将话题拉回特定场景，询问 AI 替代人类专家（比如麻省理工学院博士 James Cowling 级别的高级工程师）进行数据库查询优化或直接掌管分布式集群还要多久时，Jad 给出了分化的判断：
- **查询优化**：Jad 坚定地认为“非常接近了”（Very close）。
- **掌控整个分布式集群**：Jad 坦承“还没到那个时候”（No, not yet）。
Jad 最后反思了软件工程对 AI 能力的低估：最初大家以为 AI 只能做代码自动补全（autocomplete），信誓旦旦地断言 AI 绝不可能构建复杂的企业软件，但如今 AI 已经彻底打破了这种认知边界。然而 Martin 依旧持保留态度，他认为代码的分布流形（manifold）是相对有限且已被人类全量代码覆盖的，但软件运行时所面对的真实物理世界运维挑战，其复杂边界恐怕远没有那么简单。
- Grepr 的架构模式是：底层聚类模型负责在 PB 级日志中寻找模式变动，上层 AI Agent 负责代码级因果推理与假设检验。
- 细分场景落地预测：AI 替代人类做 SQL 查询优化已近在咫尺，但独立接管并运行大规模分布式集群仍需时日。
### [03:59](https://www.youtube.com/watch?v=blkIGLlZePg&t=239s) 系统的历史：靠过度配置而非算法预测
Martin Casado 认为真实系统的运行状态极其复杂，呈现出严重的重尾分布（heavy tailed）、异常频发和幂律分布特征，这些都是目前大模型最难学习的领域。Martin 回顾了系统工程的发展史，指出工程师们总是一开始就雄心勃勃地想去设计精巧聪明的算法，比如智能的虚拟内存管理、智能拥塞控制或智能自适应路由。然而每次现实都会给所有人上一课，正如 Martin 所指出的：系统工程从不靠精准预测，而是靠过度配置（overprovisioning）。
Martin 列举了经典的工程教训：早期设计的 swap 机制试图把内存数据动态交换到磁盘上，可一旦系统真的触发 swap，机器基本上就卡死了，彻底失去响应；因此业界最终的解决办法就是简单粗暴地堆满物理内存。在网络路由领域也是如此，人们发明了无数声称能自适应流量波动的算法，结果却引发了剧烈的网络震荡，最终大家还是选择过度配置网络带宽。Martin 由此推论，在系统运维端，要么我们继续沿用这种前期过度配置的思路来避开复杂的运维控制，要么寄希望于 AI 像人类那样介入排障，但运维绝不同于编写代码，它要面对海量的偶发异常，且缺乏充足的训练数据。
> “And what do we do instead? Every single time... We overprovision. No, we overprovision every time.” —— Martin Casado
---
### [03:59:19](https://www.youtube.com/watch?v=blkIGLlZePg&t=14359s) 运维调试的核心是工作流而非单步预测
Jad Naous 对 AI 解决系统运维问题持有更为乐观的态度，因为 Grepr 每天打交道的正是真实的可观测世界。Jad 强调，可观测性平台的立足前提是系统已经收集并流式传输了所有数据——每当新事件发生、异常抛出或负载突变时，这些信号早已处于监控系统的视野之内。然而，使用 AI 的逻辑绝不是把几百万行未经加工的日志一股脑塞给模型并问它系统出了什么事，而是要让 AI 像人类工程师一样去驱动排错工作流，提出假设、设计实验、验证因果链。
Martin 敏锐地抓住了 Jad 的论点本质：尽管宇宙中的系统故障呈现无限发散的重尾分布，可能出现的罕见坏状态数不胜数，但人类工程师用来定位和调试这些故障的**排障步骤集合**却是完全可穷举、可管理的。当年“混沌工程”（Chaos Engineering）浪潮之所以陷入瓶颈，正是因为系统可能陷入的状态空间庞大如宇宙中的原子总数，自动化系统很难在浩瀚的状态空间中直接找到答案，过去只能靠人类按部就班地缩小排查范围。Jad 认为，现在可以教给 AI 的恰恰是这套严谨的减枝排障工作流。
```mermaid
flowchart LR
accTitle: AI排障工作流逻辑图
accDescr: 展示异常信号被检测后，AI生成因果链假设并通过可观测数据进行验证的排查流程。
subgraph Signals["数据监控层"]
A["系统遥测与指标流"] --> B["扰动与异常行为检测"]
end
subgraph Reasoning["AI 推理工作流"]
B --> C["提出多种因果链假设"]
C --> D["设计验证实验并匹配数据"]
end
subgraph Resolution["定位收敛"]
D --> E["排除伪因果，锁定根因"]
end
```
> “The universe is heavy tailed. The number of bad things that can happen is is probably an exception thing, but like the set of steps used to debug it is a manageable thing. That's what you can teach one of these models...” —— Martin Casado
---
### [04:01:26](https://www.youtube.com/watch?v=blkIGLlZePg&t=14486s) 安全领域的对抗性与极端状态空间
Jad 补充说，Grepr 也正在将这种“因果链验证”的工作流应用到安全领域。安全检测的核心是捕获统计指标中的离群扰动，例如某些实体的行为突变、某些维度的基数（cardinality）异常。系统一旦捕获到这种扰动信号，就会交由 AI 生成多种潜在的因果链假设，再去真实数据中逐一验证哪条假设契合攻击模式。但 Martin 提出反驳，认为安全领域的排障比运维还要艰难得多。运维中的极端罕见状态多是系统自然演变或巧合并发出来的；而在安全攻防中，存在着怀有恶意的主动攻击者（adversary），黑客会处心积虑、有针对性地去构造和触发那些未被发现的极其罕见的系统脆弱状态。
两人的探讨由此延伸到了物理离散与状态空间的哲学层面。Jad 提出从某种物理视角看，宇宙可能在普朗克尺度上是量化离散的，系统的所有可能状态及其演化路径在理论上也是离散有限的。Martin 则分享了当年在斯坦福研究生院时和同学 Sundar 的一段趣事：Sundar 曾神情落寞地跑来对 Martin 说，一首时长 4 分钟的 MP3 音频大约只有 4 MB 大小，由于每首歌都是由有限的二进制位（bits）构成的，这意味宇宙中所能存在的音乐总数是有限的，音乐的有限性让他无比伤感。Martin 当时直接打破了他的悲观：4 MB 所包含的可能状态数已经达到了 $2^{33,554,432}$ 级别，根本无需到 4 MB，仅仅到几个 KB 所能排列出的状态数就已经远远超过全宇宙中的原子总数了。对人类而言，这种所谓的“有限状态”在现实中就是货真价实的无限。
- **4 MB 的状态空间**：虽然在数学意义上是离散且有限的，但其组合爆炸呈现出的状态数远超宇宙原子总量，在工程实践中等同于无限。
- **攻防对抗的本质**：不同于运维中的自然偶发错误，安全攻防是高智商对手主动寻找盲区，因而模型更难提前通过常规数据分布去学习。
> “Motherfucker, dude, you know how quickly you've like overcome like the atoms in the universe. You haven't even gotten to a K, right? It's like two to the two to the two to the two. Like, what are you talking about here, man? Be happy. FOUR MEGA YOU LIKE THIS IS so much state.” —— Martin Casado
---
### [04:05:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=14708s) 运维的真正壁垒：随机负载与分布式人类协作
基于在风险投资领域接触众多创业项目的视角，Martin 坦言，目前市场上试图用 AI 解决运维难题的公司都走得异常艰难，这比单纯解决代码生成困难得多。Jad 对此表示百分之百认同，但他指出，运维之所以极端复杂，不仅仅在于系统本身会遭遇完全随机的外部负载突变、有人私自发布了未通知全员的新版本，或者系统在偶发并发下陷入死锁；更关键的阻碍在于**人**。
代码编写通常只是单个程序员面对编译器的局域活动，而生产运维则深度卷入了大量不同的人类工程师。整个组织的知识池（pools of knowledge）是高度碎片化且分布式存放在各个团队大脑中的。定位线上故障往往需要跨团队交互、把各个分散的信息片段拼凑起来，这种人际协作与隐性知识构成了极高的壁垒。不过 Jad 依然保持乐观，他认为代码是整个数字世界的一个微观缩影；如果 AI 能够在代码层面上被彻底攻克，那么随着复杂性一层层叠加，AI 未来也有希望逐步承接由大量人群协作与复杂系统交织而成的运维难题。Martin 则感叹，宇宙往往不会轻易交出它的终极秘密，AI 究竟能否真正攻克运维这一关，目前依然是一个悬而未决的未知数。
> “It is a much tougher problem and and it's not just because like the systems themselves are you know there's like external loads that are random that kind of happened to you... It's also because it involves humans a lot more and not just like one human who's the coder. It actually also involves the pools of knowledge are distributed too.” —— Jad Naous & Martin Casado
### [47:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=2828s) 语言降维、计算不可约性与模拟
Jad Naous 谈到人们过去对 AI 编程的悲观态度，最初大家总觉得用代码实现的真实业务逻辑过于错综复杂，机器根本无法达到人类大脑的完整理解深度，但事实证明这一障碍已被逐步瓦解。他打趣道：“真是有趣，我一个搞技术的在给你描绘未来，而你作为放眼未来的 VC 反倒在这里说‘不一定’。”
Martin Casado 随后分享了自己对大模型的底层理解。在他看来，大模型是对数据进行训练以捕捉某种分布。人类之所以了不起，是因为人类观察了宇宙这个超高维系统，并将其降维成了语言这种可以让模型有效理解的流形（manifold）：
> “人类观察了宇宙这个超高维系统，并把它降维成了一个对模型来说切实有用的流形。”——Martin Casado
然而代码本质上是把计算机所有的运行状态降维成了指令序列，这与物理宇宙或真实系统的实际运转之间存在根本性的割裂。Martin Casado 提出，物理世界存在“计算不可约性”（computational irreducibility），很多复杂系统必须真实跑一遍才能拿到结果，不可能纯靠推测直接得出答案。对此，Jad Naous 提出了反驳：AI 不一定非要在当下立即预测出终态，它完全可以自己规划执行路径、进行主动模拟，并编写辅助代码来收集未知信息，甚至模拟出像科学家一样的探索行为。
```mermaid
flowchart TD
accTitle: 高维现实向低维抽象与模拟的映射
accDescr: 展示高维真实世界如何通过语言和代码降维，以及 AI 遇到计算不可约问题时如何转向模拟。
subgraph HighDim["超高维真实世界 (自然系统 / 生产环境状态)"]
W["复杂系统状态与物理规律"]
end
subgraph Reduction["人类降维抽象 (模型训练数据域)"]
L["自然语言流形"]
C["代码与执行指令"]
end
subgraph ModelAction["模型处理机制"]
M["模型预测与生成"]
S["规划、编码与沙盒模拟实验"]
end
W -->|"人类观察归纳"| L
W -->|"逻辑与状态约束"| C
L --> M
C --> M
M -.->|"遭遇计算不可约性"| S
S -->|"运行并探查边界"| W
```
### [49:56](https://www.youtube.com/watch?v=blkIGLlZePg&t=2996s) AI 能取代风险投资吗？
Jad Naous 顺势将话题引向风险投资行业自身，询问 Martin Casado 认为 VC 的工作中有多少能被 AI 接管。Martin Casado 坦言，在他对风险投资的认知模型里，VC 是极度依赖人际关系的行业。他坚信市场在绝大多数时候是高效的，投资的胜负手从来不是“发现别人找不到的隐秘公司”，因为好项目大家都能看得到。
对于顶级 VC 而言，真正的护城河在于募资能力、拿到热门项目份额的“准入能力”（access）、赢下交易的谈判力，以及投后赋能——例如帮助初创团队物色顶级高管，这些是初创团队自身难以独立搞定的现实难题。虽然 AI 在商业分析层面表现惊艳，甚至超越了他合作过的初级分析师，但 Martin Casado 尖锐地指出，这种分析在投资决策中很多时候只是一场“歌舞伎表演”（kabuki show）：
> “我认为这更多是对 VC 分析能力的讽刺，而不是证明模型有多强大；我只是觉得那整套东西都是一场歌舞伎表演和胡扯。”——Martin Casado
只要技术的长期发展方向是正向的，优秀投资人的核心动作就是找到最出色的公司并想方设法投进去，而这种与人建立深层信任、利益结盟的工作，无法被 AI 的奖励机制简单攻克。
- 市场有效性假说：顶级项目往往处于全行业的共同视野中，发现信息差并不是核心优势。
- VC 核心价值链：人际关系、准入资格（Access）、交易赢取率（Win rate）以及高管招聘等投后增值支持。
- AI 分析的局限性：AI 能够高效出具完美的商业研报，但这并不能解决交易撮合与信任建立的本质问题。
### [52:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=3128s) 封闭域扩散与现实世界落地的经济学断层
对话重新回到 AI 从代码扩展到物理世界的主题。Jad Naous 提出，自动驾驶技术过去也被普遍认为不可能实现，但如今随着时间推移与技术迭代，成功落地已成定局；既然如此，在代码生成之后，下一个被 AI 彻底攻破的领域会是什么？
Martin Casado 提出了一个明确的划分标准：只要解空间属于“受限域”（constrained domain），且解的分布与训练集分布一致（如代码语法、图片生成），AI 就会如野火般迅速爆发普及；但当解的分布扩展到“真实物理宇宙”时，复杂度就会呈指数级攀升。他以斯坦福求学时的亲身经历佐证：
> “我记得在斯坦福时 Sebastian Thrun 赢下了 DARPA 大挑战赛，我们当时欢呼雀跃，觉得自动驾驶就在眼前了；结果整整花了 20 年时间、砸了 1700 亿美元的投资，最终换来的单位经济效益却比 Uber 还要糟糕。”——Martin Casado
AI 本质上是一个极其通用的“通用函数逼近器”（universal function approximator）。在纯数字世界之外，面对混乱的现实世界，技术能否落地最终并非取决于单纯的计算能力，而是由最终的经济平衡点（economic equilibrium）所决定。
### [55:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=3305s) 是让 AI 适应现实，还是重构现实迎合 AI？
面对物理现实带来的摩擦，Jad Naous 提出了另一个视角：历史上人类遇到棘手难题时，往往不是强行让工具去完全适应杂乱的现实，而是主动改造世界，使现实环境去适应自动化工具的能力范畴。
Martin Casado 对此表示完全赞同，并指出这正是现代机器人学与自动化立体仓库演进的核心逻辑。当年业界尝试过各种复杂的仓储方案，最后唯一行之有效的做法，正是彻底改造仓库本身的物理结构，让它全面适配机器人的运作逻辑。
> “在面对现实世界时，你想解决任何问题都能解决，这无非是算力和资金的问题，而最终的平衡点必将是一个经济学平衡点。”——Martin Casado
两人总结认为，目前 AI 之所以势如破竹，是因为它主要运行在纯数字世界中，解空间与训练集在同一分布内。未来当 AI 逐步渗透到复杂的运维、物流与现实场景时，人类很可能会主动重塑工作流、软件架构乃至现实世界的规则，让系统变得对 AI 更加友好，从而在算力、成本与经济可行性之间达成全新的平衡。
### [56:41](https://www.youtube.com/watch?v=blkIGLlZePg&t=3401s) 元经济均衡飞轮与系统约束的演进
Martin Casado 深入阐释了当前技术范式的底层逻辑。他指出，进行均衡计算时所依据的函数，实际上是在不断约束整个运行环境和系统本身。当前行业已经切入一个“元经济均衡飞轮”（meta economic equilibrium flywheel）中：业界正在用一种极具市场价值的解决方案，来持续优化这种虽然极其通用、但眼下极其低效的计算与工程过程。Casado 对这种范式的商业生命力展现出极大的信心，他总结道：“Listen, we've got 30 years of this and a lot of value is going to be created.” 在未来的二三十年里，沿着这一经济与技术飞轮，将源源不断地诞生出庞大的商业价值。
### [57:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=3425s) “别聊 AI 了，先帮我把账单砍掉 90%”
接续对商业价值的探讨，Jad Naous 分享了一个 Grepr 在面向企业客户销售时遇到的戏剧性故事。团队刚把融合 AI 能力的产品做出来，满怀热情地去联系潜在客户，开场白通常是：“我们做了这个 AI 系统，可以帮助大家提升系统可靠性，以下是我们的工作原理。”客户很感兴趣，约了会议。在正式的推介演示中，Naous 详细拆解道：“我们打造了这套系统，它能帮你优化数据、削减成本，还能让你变得更稳定可靠，因为我们能在故障演变成严重事故前就把问题排查解决掉。”
不料客户当场打断，直接抓住最实际的痛点：“等等，倒回去一下。你说你们能帮我们砍掉 90% 的 Datadog 账单？”Naous 回忆当时自己还试图把客户拉回主线：“对啊……但我们这个 AI 核心功能……”客户却根本不在乎概念包装，斩钉截铁地回答：“不不不，我们现在就想买这个能省钱的功能！”
```mermaid
flowchart LR
accTitle: Grepr's Value Proposition vs Customer Reaction
accDescr: Diagram contrasting Grepr's AI narrative with the customer's actual purchasing driver.
subgraph GreprPitch ["Grepr 团队演示的价值重心"]
A["引入全新 AI 架构"] --> B["提升系统稳定性 & 提前拦截 Incident"]
B --> C["优化数据流并大幅削减监控成本"]
end
subgraph CustomerReaction ["企业客户的实际购买决策"]
D["拦截 Datadog 巨额开销 (省下 90% 账单)"] --> E["立即掏钱采购"]
end
C -.->|"直击最敏感预算线"| D
```
- **销售落差**：创业团队极力兜售前沿 AI 技术与可靠性提升，但企业采购方对最直接、立竿见影的财务减负（削减 90% 监控账单）表现出压倒性的购买意愿。
- **价值度量的难易度**：代码生成类工具之所以能迅速席卷市场，是因为其价值极其直观，“就是工程师生产力，而那是账本上最昂贵的开销之一”。
- **运维维度的归因困境**：涉及运维和可靠性时，要将“避免了一次潜在宕机事故”精准折算成具体的美元金额并让客户当场掏钱，链路远比写代码复杂。
### [58:13](https://www.youtube.com/watch?v=blkIGLlZePg&t=3493s) 研发提效与运维智能化的代际差异
Martin Casado 与 Jad Naous 进一步剖析了为什么 AI 在“编写代码”和“保障运行”两个方向的采纳速度大相径庭。Casado 强调，在代码生成端，其对应的是最为直观的工程师工时与人效，ROI 算账一清二楚。虽然 Naous 指出在理论上你也可以将系统健康折算为 SRE 人效或避免业务停机的挽回损失，但客户的直观感知完全不在同一个量级上。
最后，Casado 总结了大众认知的心理差异：“Like this code was so magical because it was so good and nobody thought that could happen. And I think the operations stuff now it's still pretty early.” 代码补全和生成的惊艳程度在于它彻底颠覆了人们对模型能力的预期，一击即中；而利用 AI 去自动推理运维、处置故障并驾驭动态复杂的生产系统，目前整个行业依然处于非常早期的探索阶段。在轻快的笑声与相互致谢中，两人结束了这次深入对谈。
## Notable quotes
> [55:05](https://www.youtube.com/watch?v=blkIGLlZePg&t=3305s) 在面对现实世界时，你想解决任何问题都能解决，这无非是算力和资金的问题，而最终的平衡点必将是一个经济学平衡点。 — Martin Casado
> [47:08](https://www.youtube.com/watch?v=blkIGLlZePg&t=2828s) 人类观察了宇宙这个超高维系统，并把它降维成了一个对模型来说切实有用的流形。 — Martin Casado
## Mentioned
Jad Naous, Martin Casado, Grepr, a16z, Stanford University, Nick McKeown, Balaji Prabhakar, OpenFlow, SHA-3, MIT, Nikolai Zeldovich, Ramesh Chandra, AppDynamics, Cisco, Imply, IntelliJ IDEA, GitHub Copilot, OpenCode, Codex, Claude Code, Claude 3.5 Sonnet, Claude Opus, Windows, macOS, WSL, Apache Iceberg, Parquet, James Cowling, Datadog, Sebastian Thrun, DARPA, Uber
## Video description
AI can write production code in minutes. But can it debug and operate that code when something goes wrong? Grepr CEO Jad Naous argues that running real-world systems is a fundamentally harder challenge, with unpredictable workloads, failures, and countless possible states.
Jad joins a16z General Partner Martin Casado to discuss how coding agents have transformed engineering inside his 10-person team, including projects that once took six months and now take roughly 10 days.
They also explore whether AI can troubleshoot production systems by detecting changes, forming hypotheses, and running experiments, and why the best engineers may be gaining the most leverage from AI.
Timestamps:
00:00 Intro
00:54 From Stanford to founding Grepr
07:23 Why observability got so expensive
10:15 How AI is changing engineering
18:25 Jad's AI stack and workflow
22:55 A six-month project in ten days
24:55 Can AI reason about operations?
30:10 How Grepr catches incidents early
42:16 Why security is the harder problem
53:33 What comes after code
Resources:
Follow Jad Naous on X: https://x.com/jadtnaous
Follow Martin Casado on X: https://x.com/martin_casado
Learn more about Grepr: https://www.grepr.ai/
Stay Updated:
If you enjoyed this episode, be sure to like, subscribe, and share with your friends!
Find a16z on X: https://twitter.com/a16z
Find a16z on LinkedIn: https://www.linkedin.com/company/a16z
Listen to the a16z Show on Spotify: https://open.spotify.com/show/5bC65RDvs3oxnLyqqvkUYX
Listen to the a16z Show on Apple Podcasts: https://podcasts.apple.com/us/podcast/a16z-podcast/id842818711
Follow our host: https://x.com/eriktorenberg
Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see http://a16z.com/disclosures.
[Source](https://www.youtube.com/watch?v=blkIGLlZePg)
Source quotes
[55:05] Martin Casado: 在面对现实世界时，你想解决任何问题都能解决，这无非是算力和资金的问题，而最终的平衡点必将是一个经济学平衡点。
[47:08] Martin Casado: 人类观察了宇宙这个超高维系统，并把它降维成了一个对模型来说切实有用的流形。
Source quotes
[32:30] Martin Casado: It's like every time you add a bit to a program, the state space of like what it can do operationally increases by a factor of two.
[55:05] Martin Casado: When you're dealing with the real world like you can solve any problem you want. It's just a matter of like compute and money and that the equilibrium point is going to be an economic one.
[7:23] Jad Naous: The volume of data that you're collecting and the cost of observability was actually starting to grow faster than the units of function that you're outputting. And the marginal increase in reliability for every additional dollar you were spending on observability was just flattening out.
Original transcript
[0:00] The world that the AI lives in is all
[0:01] digital and there's the actual world and
[0:03] like those things are not necessarily
[0:04] the same.
[0:04] >> When we started doing AI for coding, it
[0:06] was autocomplete. Nobody really knew to
[0:08] what extent you can actually build
[0:10] complex software using AI and then it
[0:12] basically broke through the expectations
[0:14] because we were able to model more and
[0:16] more complexity in the systems.
[0:18] >> I just don't know if operations is
[0:20] constrained in the [music] same way.
[0:21] Where's even the data to learn how to
[0:23] like build a system that understands
[0:26] like all of the operations? My
[0:28] assumption is that we have all the data.
[0:30] The question I think is maybe about
[0:32] complexity and like to what extent can
[0:34] you take the things that you've
[0:36] described that are operational and
[0:38] convert them into something that an AI
[0:40] can reason about the dirty secret for
[0:42] the longest time of running like a
[0:44] distributed cluster with just human
[0:45] beings managing it. Do you think we're
[0:47] at a spot where you could like use AI to
[0:49] run one of these services? I think that
[0:51] there's enough to be able to
[0:56] All right, Jad. thanks for joining me on
[0:58] the podcast. For those of you who don't
[1:00] know, Jaden is the CEO of Greer, but
[1:02] also we uh went to grad school in the
[1:05] same group. And when I first met you, I
[1:08] we we had a new student. [laughter]
[1:11] >> I I walked into the office and there was
[1:14] a motorcycle helmet and a pack of Maros.
[1:17] [laughter]
[1:18] I'm like, "This guy must be unbelievably
[1:20] cool." So my question is is what
[1:23] happened to all your coolness?
[1:24] [laughter]
[1:27] Question number one, [laughter]
[1:30] >> maybe maybe maybe just for a context,
[1:32] maybe give give um a quick um uh
[1:36] overview of your background. So just
[1:38] just for those that are listening, you
[1:39] know, dad is one of the best systems
[1:40] thinkers I know. I love talking to him.
[1:41] We're going to go kind of to a deep dive
[1:43] of like engineering from a systems
[1:45] perspective and how AI is impacting us.
[1:47] We're going to kind of especially when
[1:49] it comes to things like operations, but
[1:51] just to ground it maybe a bit about your
[1:53] background.
[1:53] >> Sure. [snorts]
[1:55] So, uh, you met me at, uh, one of the
[1:58] peaks in my coolness.
[1:59] >> Yeah, you're very cool. [laughter]
[2:01] >> Motorcycle and Marorrow cigarettes.
[2:05] >> I remember thinking I'm like, "Wow, are
[2:07] cigarettes even legal?" [laughter] It
[2:09] was like so discordant.
[2:12] >> I remember Nick seeing me one day
[2:13] smoking outside of uh outside of Gates
[2:16] and looking at me and going like,
[2:18] [laughter]
[2:19] "G Nick Bun is our adviser. Gates was
[2:23] the the building." Yeah,
[2:24] >> actually one of my favorite thing I
[2:25] don't do you remember Balaji Prabakar
[2:27] the professor of course
[2:28] >> my favorite thing about him is that he
[2:29] smoked I was like the [laughter] only
[2:31] professor I just thought it was like
[2:32] great
[2:33] >> just to be clear I don't smoke anymore
[2:36] >> no too old too middle-aged for that now
[2:39] [laughter]
[2:39] um but let's see so you know I went to
[2:42] Stanford with uh with you you were
[2:43] always the guy I had to look into to
[2:46] look to look up to and be like hey
[2:49] >> this is the guy I want to be when I grow
[2:50] up
[2:51] >> I was I think it was I it was a I
[2:53] started started four years before you
[2:54] did. So I was ahead
[2:56] >> and so you were four years ahead of me.
[2:57] You graduated a while ahead of me. You
[2:59] went and started Nicer. I stayed and
[3:02] >> stuck around worked more on OpenFlow.
[3:04] Helped uh build a bunch of stuff on the
[3:06] hardware side from a security
[3:08] perspective. Actually,
[3:09] >> one of the one of the, you know, most
[3:12] fun projects that I worked on there was
[3:14] uh I built the first set of uh Shaw 3
[3:17] implementations that people were
[3:20] considering using and um I built them in
[3:23] hardware [snorts] to kind of show
[3:24] whether they work or not and whatever
[3:26] for one of my for uh one of my projects
[3:28] for my thesis. That was fun.
[3:30] >> Yeah. Yeah. Cool. [clears throat]
[3:31] And then
[3:34] I decided after a while that I actually
[3:35] don't like networking because it's just
[3:37] about moving bits from one place and
[3:39] putting them in another place.
[3:40] >> How hard can that be? [laughter]
[3:44] >> I think that would have to either be
[3:46] philosophically aligned that it's deep
[3:47] or not. Most people assume it's not. I
[3:49] think it's very deep. [laughter]
[3:52] >> I mean there's something interesting
[3:54] about it, but ultimately to me that's
[3:56] like it's not
[3:58] >> Yeah. It doesn't do it for you.
[4:00] >> Yeah. Um
[4:02] and then uh I actually went and did a
[4:04] posttock at MIT with Nikolai Zelda who's
[4:07] also amazingly deep and super smart.
[4:10] >> Um
[4:11] >> he's phenomenal.
[4:11] >> I did some application security work at
[4:13] the time. Started a company um based off
[4:16] some of the research I had worked on
[4:17] with Ramsh Chandra over there who was a
[4:20] PhD student at the time and ultimately
[4:22] technology was too early.
[4:23] >> What was it?
[4:25] So what we did was um well you know
[4:28] ultimately a lot of the work was
[4:29] actually really Romeshes and his PhD was
[4:32] on well hey if you get attacked and you
[4:35] get exploited can you recover from that
[4:38] by rolling back the state of the whole
[4:40] world that's related to that application
[4:42] and then roll it over on top of a
[4:44] patched version of the application to
[4:46] remove the exploit.
[4:47] >> Yeah. And so we started doing that in
[4:50] the real world and we're like ah that
[4:52] sounds really hard. [laughter]
[4:54] It's actually kind of relevant to what
[4:56] we're talking [snorts] about which is
[4:57] like to what extent can you actually
[4:59] bound the state for like operational
[5:01] systems which I would argue is very hard
[5:04] but
[5:04] >> yeah I
[5:06] we almost got it working [laughter]
[5:10] >> almost
[5:11] I feel like for all longtail disciplines
[5:14] this is why people get in like the fly
[5:15] track for smart people is like you see a
[5:18] bit of progress but you're as toing but
[5:20] anyways I digress. Let's keep going.
[5:22] Let's keep going with your uh
[5:23] >> So then I joined App Dynamics. I was
[5:25] there for about three and a half years.
[5:26] They went from about 200 people to 2,000
[5:28] people. Got acquired by Cisco. I was a
[5:31] Cisco employee for one day. And I said,
[5:34] and then you're like, Jad,
[5:35] >> I know. I [laughter]
[5:36] love you. Come work with me.
[5:38] >> Come do this VC thing with me.
[5:40] >> I'm like, what the
[5:41] >> Yeah.
[5:42] >> What What is VC? What? Like I'm an
[5:44] architect. I do like engineering stuff.
[5:47] What do you He's like, great.
[5:48] >> Yeah.
[5:49] >> The less you know, the better.
[5:50] >> Yeah. Literally what you said.
[5:51] >> It's true. And then the first day I
[5:53] started here, you said
[5:56] slow down. [laughter]
[5:58] >> It's actually people that move from
[5:59] operate like from like actually an
[6:01] operating world to VC. It requires a lot
[6:03] of reworking the brain.
[6:04] >> Yeah.
[6:05] >> Yeah. Just like going from a PhD to
[6:06] operating by the way. Like you have to
[6:07] like undo the brain damage. [laughter]
[6:11] >> This is my experience.
[6:12] >> This is what real world is like.
[6:13] >> I know.
[6:15] >> And then um I stayed here for a couple
[6:17] of years. Did a bunch of work with you.
[6:19] It was a really fun time. Worked with a
[6:21] lot of great people like Jennifer, Mark,
[6:23] and Ben.
[6:24] >> Satish.
[6:25] >> Satish. Um Matt was uh here when I first
[6:29] uh what almost towards the end when I
[6:31] was leaving.
[6:31] >> Yeah.
[6:32] >> And then um and then joined Apply as
[6:35] their chief product officer. Um helped
[6:38] them build uh their engineering product
[6:40] teams. Kind of scaled from about
[6:42] engineer like 15 people to over 90
[6:45] across product engineering infosac and
[6:47] design. And after about 3 years, um, I
[6:51] kind of had a head for each one of these
[6:53] pieces that I owned and I kind of was
[6:55] sitting around just coordinating and
[6:57] doing meetings all day and wanted to go
[6:59] do something else. So left and then um
[7:03] started talking to a lot of engineering
[7:05] leaders about the problems that they're
[7:06] facing and what they're trying to do.
[7:08] Everybody at the time, this was in 23,
[7:10] everybody was saying, "Hey, we're trying
[7:11] to reduce spend." And ask them, "What
[7:13] are you spending money on?" Everybody
[7:14] would say observability. And you know
[7:17] well first actually it was
[7:18] infrastructure and then observability.
[7:20] And so I started looking at that problem
[7:22] trying to understand like what is why do
[7:25] people think it's too expensive?
[7:27] [clears throat]
[7:28] Turns out there's a few things. First
[7:30] people were spending somewhere between 5
[7:31] and 20% of what uh they spend on
[7:34] infrastructure on observability. But
[7:36] ultimately what they were really trying
[7:38] to spend it on was reliability. They
[7:41] wanted to buy reliability and um
[7:44] >> of spend like predictability of spend
[7:46] >> reliability of their infrastructure and
[7:48] applications.
[7:49] >> I see. I see. I see.
[7:50] >> Right. And so they were putting money
[7:52] into observability as a proxy for
[7:53] reliability. And for the longest time,
[7:55] the more money you spend,
[7:56] >> the more reliability you got out. But as
[7:58] architectures changed and turn into
[8:01] microservices that are much smaller,
[8:03] that are much more vertically
[8:04] integrated. The volume of data that's
[8:05] being collected at every layer expanded.
[8:08] there was much deeper market penetration
[8:10] for observability and kind of the uh
[8:13] volume of data that you're collecting
[8:15] and the cost of observability was
[8:17] actually starting to grow faster than
[8:19] the units of function that you're
[8:21] outputting. And um and the marginal rate
[8:26] of or like the marginal increase in
[8:27] reliability for every additional dollar
[8:29] you were spending on observability was
[8:30] just flattening out.
[8:32] [gasps]
[8:33] Um, and so that's why people thought it
[8:35] was expensive, but people were still
[8:37] willing to pay 5 to 20% on reliability.
[8:39] And so the question was like, can we
[8:41] actually solve that problem and bring
[8:43] back the um that increase in
[8:47] reliability, that marginal increase in
[8:48] reliability,
[8:50] >> not necessarily by spending more money
[8:51] on observability, but some other way.
[8:54] And so the thinking was like, okay,
[8:56] great. If we can figure out what is the
[8:58] signal, what is the noise, can we
[8:59] actually turn the observability data
[9:01] that you're collecting that it's mostly
[9:02] fluff and mostly insurance at this point
[9:05] and do that kind of extraction of
[9:09] information upstream, do whatever you
[9:11] need upstream and be able to ultimately
[9:13] massively reduce the cost of
[9:15] observability downstream. And if you can
[9:17] extract that signal in real time
[9:20] upstream then you can one reduce the
[9:23] cost but two you can now feed it into
[9:26] other systems and use it for driving
[9:28] reliability operations afterwards
[9:31] [clears throat]
[9:32] >> and the distance at the time there was
[9:34] something that was called AI coming
[9:36] coming into the picture and we're like
[9:37] well great if we are in the data stream
[9:40] and we're extracting the signal then we
[9:42] are in a strategic position to feed that
[9:44] to whatever is going to come in the
[9:45] future because that's going to be
[9:46] necessary.
[9:47] >> Yeah.
[9:48] >> And so that's what we built. And um so
[9:50] Greer helps companies, you know, we
[9:52] started with helping them reduce their
[9:54] spend on telemetry massively. And so you
[9:57] put us in the data stream, plug and
[9:59] play, pass the data through, reduce your
[10:02] cost by 90%.
[10:04] And um and now we're building out that
[10:06] second layer where we're feeding those
[10:08] signals into the AI layer that's now
[10:10] going and troubleshooting and figuring
[10:12] out when things are going going wrong
[10:14] and fixing them. I'd love to just talk
[10:16] through kind of your experience on how
[10:20] AI is changing dev generally which is
[10:23] like a lot of people get asked this
[10:24] question a lot of people answer but I
[10:26] just feel like kind of like the
[10:27] engineers engineer like you've been an
[10:29] architect you've run product you run
[10:30] engineering teams like you know like
[10:33] actually how big is your company now how
[10:34] many people
[10:35] >> we're t
[10:36] >> yeah so you know and you've got a I mean
[10:38] you're building a lot of software with a
[10:40] relatively efficient team so
[10:42] >> maybe just a highle sketch on like you
[10:45] Well, if you're a senior engineer like
[10:46] you, you know, that has architectural
[10:49] experience, like like what sort of gains
[10:53] do you think people are seeing? Maybe
[10:55] talk through like how you use AI, what
[10:57] gains you're actually seeing, what is
[10:58] the stack, what is your process, how you
[11:00] think about it.
[11:01] >> So maybe I can give you like a
[11:03] historical view of like what's been
[11:04] happening in terms of like running a
[11:06] team and so on.
[11:07] >> And so what you know when we first
[11:09] started AR was still kind of
[11:11] autocomplete,
[11:12] >> right? I remember like at the time we
[11:14] were using uh co-pilot and uh intelligj
[11:18] and we're like oh my god look it
[11:19] autocomp completed these tests
[11:21] >> uh we're like yeah but it's just
[11:23] autocomplete and very rapidly things
[11:26] started evolving and what what I've
[11:29] noticed is that really good engineers
[11:32] are able to somehow put it into their
[11:35] workflows in a way where they the AI
[11:38] starts training them.
[11:41] Oh, interesting.
[11:41] >> In terms of like how to talk to it, how
[11:44] to get the best out of it.
[11:45] >> And what I do, as an aside, do you think
[11:48] that's model specific? Like is it do you
[11:50] learn how to use AI, you know, writ
[11:52] large or do you get efficient at a given
[11:55] model?
[11:56] >> I think there's like some nuances that
[11:59] different models bring. Um, I'm sure
[12:02] looking at your tweets you've [laughter]
[12:07] um there's like nuances, but there is a
[12:10] a way to work with AI and like
[12:14] I think that there's also a way to work
[12:16] with different harnesses and I think
[12:17] like you know we tried for example using
[12:19] open code or like different harnesses or
[12:21] like using codecs versus claw like claw
[12:25] code like different ones have different
[12:27] ways of managing context or thinking
[12:31] through problems or when to spin up the
[12:33] right sub agents and what to tell it
[12:35] like the kind of prompts like for
[12:36] example
[12:38] >> uh with cloud code I find that if I
[12:40] don't tell it don't spin up
[12:42] >> you know 20 fable agents to do basic
[12:44] research it'll go and spin up 20 fable
[12:46] agents to do research and burn all your
[12:48] tokens and so like a lot of these things
[12:50] kind of you start learning about the
[12:52] harness and the and the models and like
[12:54] how they interact with each other and
[12:56] then how they can interact with the rest
[12:57] of the world but I digress like I'm just
[13:00] going
[13:01] in my mind like this is kind of what I
[13:02] started noticing is that like some
[13:05] people were adapting and learning and
[13:07] kind of doing it that way um and others
[13:10] weren't as adaptive.
[13:13] >> Yeah.
[13:13] >> And then as we started hiring more
[13:15] people into the team that disparity
[13:17] between you know okay engineers and
[13:21] really good engineers just multiply 10x.
[13:25] >> Was it can you predict who will who
[13:27] could you predict who would be good? Was
[13:29] it like good engineers got better and
[13:31] bad engineers didn't or was it it
[13:33] really? Okay. Yeah. So, like if if
[13:35] you're a good engineer, you're just a
[13:36] better good engineer. It's not like
[13:37] there's some like bad engineers all of a
[13:39] sudden became good engineers.
[13:41] >> No.
[13:41] >> Okay.
[13:41] >> No. No. It it always ends up being like
[13:44] you have to really understand ultimately
[13:46] the the setup because the more
[13:48] complexity that your brain can handle,
[13:50] the more you can break down the tasks
[13:53] and kind of assign them. I think this
[13:55] might actually be uh like non- consensus
[13:58] opinion that like uh I mean I I do hear
[14:02] it sometimes but the idea that like the
[14:05] better engineers like are much better
[14:07] [laughter] those are are less and you
[14:09] you found this to be pretty universally
[14:11] true [sighs and gasps]
[14:13] I mean look my team is still small we
[14:15] haven't done it across 100 people
[14:17] >> yeah [clears throat]
[14:18] >> in my experience for the things that we
[14:20] do that is the case because we build
[14:21] distributed systems and distributed
[14:23] algorithms and like your brain has to
[14:24] handle the complexity because like
[14:26] >> AI doesn't do it,
[14:27] >> right? Yeah. Yeah. Yeah.
[14:29] >> And so for the problems that we solve,
[14:31] [clears throat and cough]
[14:31] none of the models can actually think
[14:33] through the things that we do.
[14:35] >> Like there's a lot of for example, we
[14:37] have a query translation layer. We were
[14:38] just talking about query optimization.
[14:40] >> Yeah.
[14:40] >> And so you have to really understand
[14:42] the semantics of the query and how to
[14:45] get to the semantics of the query and
[14:47] then explain it to the model
[14:49] >> and then have the model go and build the
[14:52] translation layer, let's say. Yeah. So
[14:54] for for for the people that are
[14:55] listening that that may not be so deep
[14:57] in computer science, what do you mean by
[14:58] semantics? [laughter]
[15:03] >> Uh so semantics are really the meanings
[15:06] of things. Like the semantic of a dog of
[15:10] the word dog is the animal that actually
[15:13] walks on four legs and barks.
[15:15] >> And the syntax is the fact that it's
[15:17] just three a threeletter word
[15:19] >> and it has three different letters. the
[15:21] shape of the word is different and that
[15:24] shape can be different between different
[15:26] words. [clears throat]
[15:27] Um, but different words can actually
[15:29] also mean the same thing. And so they
[15:32] might have the same semantics, but they
[15:34] might have a different syntax. So a dog
[15:37] and a puppy might be slightly different
[15:40] semantics.
[15:42] [clears throat] They refer to the same,
[15:43] >> but they're still called a dog. So the
[15:45] way that and just for for like computer
[15:47] science, the way that I describe it very
[15:48] simply is syntax, you know, is like the
[15:51] the characters you type and semantics is
[15:54] what the machine actually does, right?
[15:56] And so it's like like the actual runtime
[15:59] properties of the the system.
[16:01] >> So you did computer science, I did
[16:03] electrical engineering, [laughter]
[16:06] >> you became a computer scientist and I
[16:08] became a VC. So [laughter] there we go.
[16:12] >> That's the problem. I see.
[16:18] Yeah.
[16:18] >> Anyways, I just wanted for people that
[16:19] are that are listening, I just want to
[16:20] understand like there's the actual
[16:22] syntax which is writing the code, you
[16:24] know. Um, and then when you're talking
[16:25] about syntax, you're just talking about
[16:26] the code itself. Then when you talk
[16:27] about semantics, you're literally
[16:28] talking about you have a machine that
[16:30] interprets that code whether it's
[16:31] compiled or not. And that machine does
[16:33] something. The semantics is what the
[16:34] machine actually does. And these things
[16:36] are, you know, clearly very different,
[16:38] right? Like so for example, some
[16:39] semantics you can't even derive by
[16:41] looking at the syntax, right? you don't
[16:42] know what the machine is going to do
[16:43] because whatever you kind of have to
[16:45] test it you have to test it you know
[16:47] like and it can go all sorts of
[16:48] different ways when you test it and like
[16:50] a very classic version of this is is um
[16:54] uh is SQL for example like you can write
[16:56] you know some SQL query but like how it
[16:59] actually performs how long it performs
[17:01] depends entirely on like say what the
[17:03] schema looks like or what the data looks
[17:04] like in the database etc
[17:07] but that's I would not call that
[17:09] semantics though I would call that
[17:13] performance characteristics.
[17:15] [clears throat]
[17:15] >> Oh, sure. Yeah. I think that would be
[17:16] like
[17:17] >> like I like I would say like a the
[17:19] semantics part of that would be like
[17:22] um depending on like what the shape of
[17:23] data is or like what the configuration
[17:25] you might end up with different semantic
[17:26] because like you said you want null to
[17:28] be treated as false. Um and you won't
[17:31] know that unless like you also know the
[17:32] configuration of the system and and and
[17:34] so on. Okay. Okay. Okay. So, for the
[17:36] purpose of this conversation, I think we
[17:37] should just say that there's like the
[17:40] actual code you write and there's how
[17:41] the system behaves. And we'll call the
[17:43] second one semantics just so we can have
[17:45] a conversation. [laughter]
[17:48] >> There's a
[17:48] >> Well, what do you want to call them?
[17:49] >> We need to dive into the semantics of
[17:51] this and make sure they're clear.
[17:52] >> That's [laughter] right.
[17:55] >> Because there's a lot of
[17:56] >> I know. Okay. Okay. We'll keep it high
[17:58] level. Okay. So, so sorry. So, let's go
[18:01] back. So, um, you know, actually before
[18:05] we started the podcast, you and I were
[18:06] just shooting the [ __ ] and you were
[18:09] talking about like how you can do
[18:10] products, projects now that take a week
[18:13] where in the past would take a team and
[18:14] months. Can you maybe walk us through a
[18:16] little bit about like how you personally
[18:19] >> kind of well do projects and then kind
[18:22] of the gains that you see? [snorts]
[18:23] >> So like a little bit about like your
[18:25] stack and your process. So, I use um
[18:28] cloud code. Um
[18:32] I'm going to say something that's
[18:33] actually pretty funny, but I use uh
[18:35] Windows.
[18:36] >> What [laughter]
[18:40] was that created by this company
[18:41] Microsoft [laughter]
[18:45] and I use WSL? So, I use Linux for
[18:48] everything like development and so on.
[18:50] There's no like X or UI for it. I just
[18:52] use the command prompt.
[18:54] Um,
[18:55] >> how do you use Windows?
[18:56] >> Because I just never really wanted to
[18:57] learn the keyboard shortcuts.
[18:59] >> Are you a gamer? [laughter] It's like
[19:01] you tab tab over to Counter Strike or
[19:03] whatever.
[19:05] [laughter] Like why else? I'm not hating
[19:08] on Windows. I mean, you know, like I
[19:10] used it in the '9s.
[19:11] >> That's really like the only reason is I
[19:14] never wanted to learn the the shortcuts.
[19:16] The I I hate the onebutton mouse. I hate
[19:20] the one button stuff. I hate the number
[19:21] of modifiers you have to type for
[19:23] shortcuts and stuff like I I just I just
[19:25] never I'm like why why why
[19:28] >> why waste time. [laughter]
[19:30] >> Okay.
[19:32] >> So like yeah use Windows. Yeah.
[19:34] >> Um and so WSL I use cloud code.
[19:37] >> What is WSL?
[19:38] >> That's Windows Subsystem for Linux.
[19:41] >> So it's uh they actually run the Linux
[19:45] uh kernel within Windows in a hypervisor
[19:47] and provide you access to it. It's
[19:50] great. They've done a great job.
[19:51] >> Great. Windows WSL
[19:53] >> and Cloud Code.
[19:54] >> Cloud Code
[19:55] >> and uh like my latest the latest thing
[19:59] that I do is I use Opus 5.5. I tell it
[20:02] >> which is a really good model.
[20:03] >> It's an amazing model.
[20:04] >> Really good. Yeah.
[20:05] >> And it's I've shipped in the last two
[20:07] weeks these massive 20 PR things that I
[20:11] just dump on my team to go and and
[20:13] review and they're like [laughter]
[20:16] I'm like don't worry about it.
[20:17] >> Horrible. It's like is like the new
[20:19] reporting structure like the people
[20:21] below review the PRs for the CEO. Is
[20:24] that is that where we become [laughter]
[20:26] like
[20:27] >> No, everybody's like writing the same
[20:31] PRs too.
[20:31] >> Yeah. I I review my people PRs and like
[20:33] everybody's like just just PRs flying
[20:36] around in PR.
[20:38] >> Yes. And but like the thing that has
[20:40] changed a lot in the review cycle is
[20:42] that like we started much more focused
[20:43] on to focus much more on like the the
[20:47] tests and like doing things like golden
[20:49] expectations and like are they shifting
[20:51] in different ways and so on.
[20:53] But um Opus 5 and like the the trick
[20:56] that I've recently learned about is that
[20:59] you instead of using sub agents, you
[21:01] basically tell Claude um to spin up
[21:04] command line sub aents because those
[21:06] maintain their persistent session over
[21:08] time.
[21:09] >> And so you can have it spin up these
[21:11] agents separately. You can actually run
[21:12] them from other sessions and and so on.
[21:15] And uh that allows you to be much more
[21:17] efficient from a context perspective.
[21:19] >> How many agents do you run? How many
[21:20] >> how many
[21:21] >> how many what agents
[21:22] >> agents um typically I work on like three
[21:26] or four session windows at the same time
[21:29] because I found that more than that I
[21:32] just start losing context
[21:34] um between the different things because
[21:36] like they they start finishing faster
[21:38] and then I I then they wait.
[21:40] >> And do they interact via the do they
[21:42] interact via the PR? Is that how it
[21:44] works?
[21:45] >> The sub agents like together? No, they
[21:48] interact directly through the command
[21:49] line.
[21:50] >> Oh, wow. Okay.
[21:50] >> Yeah. So, like I asked like the the main
[21:53] like the main thing that does the
[21:54] implementation to have a tech lead
[21:57] reviewer and like they basically ask it
[21:59] questions. Does this make sense? Does
[22:01] this not make sense? Review my PR, etc.
[22:04] And so they iterate on these PRs
[22:06] together. Um, [snorts] and then they
[22:08] have a PR doc that basically they work
[22:10] on with a design doc and everything in
[22:12] it. Like the first thing that I do is I
[22:14] ask that agent to do it. Ask the tech
[22:15] leads to review the PR doc. I go through
[22:18] the the PRD. I go through the design
[22:20] doc. Make sure it makes sense. Go
[22:22] through each piece in like detail. Make
[22:24] sure that all the algorithms work and
[22:25] like all the pieces work, the division
[22:27] of labor works, the division between PRs
[22:30] and their relationships work. And then I
[22:32] tell it just go start start doing it.
[22:35] Figure out if there's like paralization
[22:37] that you can do and then spin up other
[22:39] agents to actually go do them. M
[22:42] >> um but ultimately you have to kind of
[22:44] put it all back in one stack and be able
[22:46] to go through it as a human one by one
[22:49] because otherwise it just gets messy and
[22:52] bushy and you can't really tackle it,
[22:55] you know?
[22:55] >> So how much time so can you give a sense
[22:57] of like a project you do now versus I
[23:01] don't know six years ago like time
[23:03] differences.
[23:05] So the latest project that I shipped was
[23:07] something
[23:08] >> by the way this is like to be clear this
[23:10] is production code. This is demo mockups
[23:13] whatever. So like the statement you're
[23:14] about to make is the gains that you get
[23:16] for like shipping production.
[23:17] >> Yeah. So I'll give you the example of
[23:18] the last there there's the last PR that
[23:22] I worked on or like the last set of PRs
[23:24] was what we call auto auto promotion in
[23:26] a data lake.
[23:27] >> Yeah. So in a data lake you store all
[23:28] the data and um typically the data is um
[23:32] column oriented parquet files and
[23:34] iceberg and so on.
[23:37] Um and when you get data from um logs
[23:40] they're semistructured they're arbitrary
[23:42] columns and so on. In order for it to to
[23:45] have the best layout of the data and to
[23:47] be able to query it efficiently and
[23:49] fast, you kind of the ideal situation is
[23:52] that every field in the log event
[23:54] becomes a column that you can filter on
[23:57] independently of the rest of the rows.
[24:00] >> And so we call that autopromotion or
[24:02] shredding.
[24:03] >> Um, and doing something like that is
[24:06] incredibly complicated, right? like you
[24:09] have to go and figure out how does the
[24:12] the metadata system work, how do the
[24:14] commits of the metadata system work, how
[24:16] do you coordinate across different uh
[24:18] writers and so on and um that took maybe
[24:23] about 10 days.
[24:25] In previous days that would have taken
[24:28] maybe about 6 months.
[24:30] >> Wow.
[24:30] >> For a single person.
[24:31] >> Wow.
[24:32] >> And was this just you or
[24:34] >> that was just me
[24:36] insane.
[24:39] Yeah. Sort of like factor of 10, factor
[24:42] of 20 improvement.
[24:44] >> Yeah.
[24:44] >> For production code for new products.
[24:46] >> Yes.
[24:47] >> You know, one, you know, one one um
[24:49] question I always have is like I can
[24:51] very much understand,
[24:55] you know, the these models are trained
[24:57] on all the code ever written, let's call
[24:59] it, right? So, I can understand why
[25:01] they're able to generate code, which is
[25:02] the syntax piece. Um
[25:06] but then like for you know you're doing
[25:08] core infrastructure let's say you're
[25:09] doing distributed algorithms like like
[25:11] like the properties like how it handles
[25:15] itself when it's running is very very
[25:16] different like often you just can't even
[25:18] look at the the syntax to know what's
[25:21] going to happen right like a lot let's
[25:23] say let's say you're I don't know you're
[25:24] you're let's say you're writing some
[25:25] distributed system and then like the way
[25:28] that it like it will perform the the
[25:31] availability
[25:33] um the correctness even is is going to
[25:35] be dependent on what the workload looks
[25:38] like you know like like what are the
[25:40] access patterns what is the traffic
[25:42] >> but are you talking about like the
[25:43] performance aspects of it though or are
[25:45] you talking about like this the
[25:46] >> well I'm think like availability like
[25:49] you may have to lock things so it's not
[25:50] available for a period of time I'm
[25:52] talking about correctness like maybe
[25:53] you've decided to do eventual
[25:55] consistency and so like if you do a read
[25:57] it's not going to be consistent because
[25:58] there isn't a lock right
[25:59] >> um and then performance like you know
[26:01] maybe you cache maybe you don't c these
[26:03] are all kind of trade-offs in the design
[26:04] space, right?
[26:06] >> And like unless you know like the access
[26:09] patterns, you won't know how it's going
[26:11] to perform in these things, right? And
[26:13] so I just think to myself, okay, I've
[26:15] got an AI, I can understand how it will
[26:16] write, you know, like like code for a
[26:19] generic distributed data store, for
[26:21] example. What I don't understand is how
[26:24] it will know to like
[26:26] make the trade-offs needed because it
[26:28] doesn't even know what the access
[26:29] patterns are. And it's not like these
[26:31] things can like like where's even the
[26:32] data to learn how to like build a system
[26:36] that understands like all of the
[26:38] operations right like today we rely on
[26:41] human beings to like manage clusters and
[26:43] to like you know optimize databases etc.
[26:46] And so can you maybe talk about how you
[26:48] think of like I get the writing the code
[26:49] but like you have these deep operational
[26:51] systems like how do you apply AI to that
[26:53] problem and like a have we made any
[26:54] progress at all or is this still an
[26:56] outstanding problem? The question I
[26:57] think is maybe about complexity and like
[27:00] to what extent can you take the things
[27:02] that you've described that are
[27:04] operational and like that happen in
[27:07] production and convert them into
[27:09] something that an AI can reason about
[27:12] >> and uh exactly like how can an AI reason
[27:15] about this system
[27:17] >> well a first off like to to what extent
[27:19] do you see that it writing the code that
[27:21] it can actually predict what's going to
[27:22] happen like how good of a job does it do
[27:24] at that and the second one is like Okay.
[27:26] Now, how do we make it reason about
[27:28] that? So, let's ask the first one like
[27:29] like if I if I use your method, the the
[27:32] JAD method of writing code and I built
[27:34] some big distributed system like how
[27:37] often does it actually get it from
[27:38] performance correctness?
[27:40] >> How how often is it like actually
[27:41] operationally like pretty sensible? And
[27:43] by the way, I I work on this this
[27:44] distributed Gaussian splat render and I
[27:46] would say it's [ __ ] Like it's horrible.
[27:47] like because like you know you've got
[27:48] millions of Gaussian splats that you've
[27:50] got to page in and out and stream and
[27:51] like it just totally depends on like the
[27:54] running system and so like none of the
[27:56] models do a good job.
[27:57] >> So in our world what I found is that
[28:00] we've defined basically really well what
[28:03] performance benchmarks look like and how
[28:05] the system should run
[28:07] >> and uh what the constraints of actually
[28:10] building within like our stream
[28:13] processor look like to get the right
[28:16] performance. But every once in a while
[28:18] it'll do like my la latest project has
[28:21] been all about query optimization and
[28:23] query performance. And so I was like hey
[28:25] >> the classic example of something that's
[28:26] >> I was like we are basically generating
[28:29] code that corresponds to SQL and we want
[28:32] to run it in a pipeline.
[28:33] >> Yeah.
[28:33] >> And we're like okay well you're
[28:35] generating [ __ ] code
[28:38] fix it.
[28:39] >> Yeah. And uh it started basically going
[28:42] through and what it didn't realize was
[28:44] that it wasn't actually doing initially
[28:46] any optimization on the code. It's like
[28:48] oh it's fine. It looks like what's the
[28:49] problem? And I said well look there's
[28:52] these uh these conjuncts over here
[28:55] you're doing and and like this
[28:56] particular term is repeated and it
[28:58] should have been taken away and then
[29:00] it's like ah okay let me figure this out
[29:02] and so on. And then like it basically
[29:04] went on a thread to try to figure out
[29:06] what are all like it built a bunch of
[29:07] hypotheses around like what are all the
[29:09] things that it can go and optimize to
[29:11] make this better.
[29:12] >> Yeah.
[29:12] >> Um
[29:14] but ultimately it was able because it
[29:16] was able to ground that in performance
[29:18] benchmarks and I was like look here's
[29:20] here's a data set of the queries and
[29:22] like what the users are asking and so
[29:24] on. Go work on it.
[29:26] >> Um and so it can basically do a
[29:29] relatively good job that way. But when
[29:32] it comes to saying something around like
[29:37] how much memory should I give this
[29:38] system?
[29:39] >> Yeah.
[29:40] >> It has no idea and that has to come from
[29:43] actual experiments that run in
[29:46] production.
[29:46] >> Yeah.
[29:47] >> And so what I think actually these
[29:49] systems seem to be doing well at this
[29:52] point in terms of operational stuff is
[29:55] that it knows that hey I probably won't
[29:58] be able to answer this question. Let me
[29:59] run an experiment somehow.
[30:02] and come up with that information. But
[30:03] of course when it comes to operations
[30:05] like production systems don't behave the
[30:07] way that you expect and ultimately
[30:09] that's why incidents happen.
[30:10] >> So have you found or are you doing any
[30:13] of like the AIS stuff where you actually
[30:15] have a running system that has logs and
[30:17] you feed it to an AI and that kind of
[30:18] helps you like tweak it? Yes. And how
[30:21] how well does that work?
[30:22] >> So we're just
[30:23] >> because anecdotally uh my experience
[30:26] with other people doing this has not
[30:27] been that great. Like the reports I've
[30:29] been getting anecdotally have not been
[30:30] great. So, a couple of things. The first
[30:32] one is we're doing that for both
[30:34] operational pieces and uh and for
[30:36] security, but the way that we're
[30:37] tackling it is not the way that other
[30:39] people are doing it from an AISR
[30:41] perspective. A lot of those systems
[30:43] basically start from a lagging indicator
[30:46] like, hey, this metric spiked or like
[30:48] this alert happened. Go back through a
[30:51] very complex system and try to determine
[30:54] why that happened,
[30:55] >> right?
[30:56] >> Like,
[30:56] >> yeah, sure. It's, you know, that's why
[31:00] those don't work because like
[31:01] backtracking into a root cause is
[31:03] actually very problematic. And so
[31:05] instead, what we're doing is doing it
[31:06] the other way around, which is can we
[31:09] actually start by figuring out if
[31:12] there's something different in your
[31:13] environment that has changed? Like can
[31:15] we do a large scale clustering, a large
[31:17] scale kind of figure out a needle in a
[31:21] haststack, something that is different?
[31:23] And then so you're assuming that it was
[31:25] working fine before and now it's not. So
[31:28] something happened, but it could be just
[31:31] like a change in like normal traffic
[31:33] patterns, right?
[31:34] >> That's fine too. That's fine too. And so
[31:36] the the the thing that we're looking at
[31:38] is like when things go wrong, usually
[31:40] the system will tell you something went
[31:42] wrong somehow. Yeah. Right. Um if it's
[31:46] working fine and scaling as normal,
[31:48] nothing will happen. But sometimes
[31:49] you'll see some like message which says
[31:51] error connections ran out.
[31:54] >> Yeah.
[31:55] >> Or memory exceeded or resource buffer is
[31:57] emptied or like some kind of indication
[32:00] like some kind of really early warning
[32:02] like something happened and that's the
[32:04] one message that we want to capture and
[32:06] then go from there and see
[32:08] >> is that an actual problem that we need
[32:10] to investigate and deal with. So how
[32:12] does it do? So far it's doing pretty
[32:14] well. I mean we're running it internally
[32:16] in our systems. We run it uh we ran it
[32:18] in uh in a couple of our customers
[32:20] environments and in the cases where
[32:23] there was a couple of cases where we
[32:25] ignored it.
[32:26] >> Yeah.
[32:26] >> And those turned into production
[32:27] incidents.
[32:29] >> Really? Oh wow.
[32:30] >> Yeah. It's kind of just just from an
[32:32] intuitive perspective like I can
[32:34] understand how these models understand
[32:36] syntax because again like I said they're
[32:37] trained on all of the co code ever
[32:39] written and it just turns out that
[32:40] covers a lot of useful cases. It's hard
[32:43] for me to actually even think of a bound
[32:45] on the actual operations of software,
[32:48] right? I mean, it feel like, you know,
[32:50] this is a ridic it's a ridiculous
[32:51] statement, but it's actually kind of
[32:53] true. It's like every time you add a bit
[32:55] to a program, the state space of like
[32:58] what it can do operationally increases
[33:00] by a factor of two. So, you very quickly
[33:02] get to like, [laughter] you know, like
[33:04] more more, you know, states than atoms
[33:07] in the universe. And arguably, this is
[33:09] why
[33:12] This is why like product approaches to
[33:15] operations have never really worked,
[33:17] right? Like the chaos engineering stuff
[33:19] never really worked and we've always
[33:21] kind of relied, you know, actually the
[33:22] dirty secret for the longest time of
[33:24] running like a distributed cluster was
[33:26] just human beings managing it. And so
[33:28] I'm just wondering and then you know and
[33:29] then in order to like even train an AI
[33:32] to do it, you have to have like enough
[33:34] data and enough configurations. And so
[33:37] like I'm just kind of curious like do
[33:38] you think we're at a at a a spot where
[33:41] you could like use AI to run a
[33:44] distributed cluster or run one of these
[33:45] services or do you think that's still on
[33:47] the come? I think that [snorts] there's
[33:49] enough to be able to
[33:52] have it start that loop, right? Like I
[33:55] mean ultimately like I think that I
[33:58] agree with you like there's infinite
[34:00] space there's a lot of different things
[34:01] but ultimately humans also operate on
[34:03] runbooks and like sure there's a lot of
[34:06] things that happen in the um in
[34:09] production that you ultimately need some
[34:11] human to go and like hey this is too
[34:14] complex there's like some state that's
[34:15] hidden that I know of because I talked
[34:18] to this other guy like at the water
[34:20] fountain the other day and he told me
[34:22] about this thing he deployed like
[34:24] there's a lot of stuff that potentially
[34:26] isn't visible in the state of the
[34:29] system. Yeah,
[34:30] >> but I'll argue that like the goal of
[34:33] what we need to be able to do is to like
[34:35] get to have enough telemetry and enough
[34:38] understanding to like make everything
[34:40] really visible from a hey here's what
[34:42] the system looks like, here's how it's
[34:44] working, here's like whatever. And then
[34:46] I think ultimately most of the issues
[34:49] that end up really showing up in
[34:51] operations
[34:53] assuming you have the system previously
[34:54] running okay is due to some kind of
[34:56] change. And if you're able to just focus
[35:01] on the goal of figuring out what the
[35:03] change was and like what and kind of
[35:07] sorry can I just ask a stupid question
[35:08] on this which is I I I never think of
[35:10] these generative models that good at
[35:11] predicting stuff and it sounds like
[35:13] you're talking a lot about predictive
[35:14] modeling like you predict a change or
[35:16] you baseline and you detect an anomaly.
[35:19] Like am I thinking about this wrong?
[35:20] >> I no you're not. Okay. Because the goal
[35:24] is not actually having the uh agent try
[35:27] to predict but have it uh get a signal
[35:32] that something changed.
[35:33] >> Oh deterministically that's something I
[35:35] see. I see.
[35:36] >> Yeah.
[35:36] >> Wait. So so there there sorry. So so
[35:39] there is a separate model that does the
[35:40] predictive stuff and then that will
[35:42] alert the AI.
[35:43] >> Yes.
[35:43] >> Okay. Yes.
[35:44] >> So yeah. So like what we do like what we
[35:47] do is we basically you give us your
[35:48] pabyte of logs per day.
[35:50] >> Yeah. and we scan all of them and we we
[35:53] like every time we see a new kind of
[35:54] pattern emerge in your logs, a new thing
[35:56] that we haven't seen before, we lift it
[35:58] up to the AI.
[35:59] >> I see. So that's that's a predictive
[36:01] model that actually does like does what
[36:02] predictor models are good at which is
[36:04] determining out of norm things or
[36:05] whatever whatever and it's like the that
[36:08] goes to the AI and the AI kind of uses
[36:10] that to figure out what to do.
[36:11] >> Exactly. So the AI basically can then
[36:13] reason and drive that
[36:15] >> I see
[36:16] >> that down. So it can say, "Oh, this
[36:18] message, let me actually look at the
[36:20] code and figure out where it comes from
[36:21] and understand what the code does. Let
[36:23] me see if it's actually happened before.
[36:24] Let me see." So it basically runs all
[36:26] these hypotheses and heristics to try to
[36:28] understand does it make sense? Does it
[36:30] need meaningful in some kind of way and
[36:32] then it can determine what to do with
[36:34] it. So how far away do you think we are
[36:36] for let's just go back to query
[36:37] optimization, not like logs or something
[36:38] like so. How far away do you think we
[36:40] are from having an AI basically do the
[36:42] job that a human used to do when it
[36:44] comes to query optimization? I think
[36:46] very close.
[36:47] >> Really?
[36:48] >> Yes. Wow.
[36:49] >> I think uh
[36:50] >> running a distributed cluster.
[36:52] >> I just think of like okay you've got
[36:54] like James Cowing PhD from MIT who runs
[36:57] a distributed cluster and like if the
[36:59] cluster's down like the company doesn't
[37:01] work and then if it's up it's running. I
[37:03] don't know like are we at the point
[37:04] where you would say that an AI can do
[37:06] that?
[37:07] >> No, not yet.
[37:08] >> Okay. I think it's I think I mean I
[37:11] think look look very very simply
[37:15] when we started doing like AI for coding
[37:17] it was autocomplete.
[37:19] >> Yeah.
[37:19] >> And nobody really knew to what extent
[37:21] you can actually build complex software
[37:24] using AI.
[37:25] >> Yeah.
[37:25] >> And like when we were looking at it
[37:27] we're like ah like AI is never going to
[37:28] be able to build
[37:30] >> X or Y or Z or whatever. And then it
[37:32] basically broke through the expectations
[37:34] because we were able to model like more
[37:36] and more complexity in the systems. I
[37:38] just I just feel like like I'm a very
[37:40] simple thinker on this stuff and it to
[37:42] me these models are distribution in
[37:43] distribution out and it turns out like
[37:45] you can actually train on all the code
[37:47] ever created and it's actually like that
[37:49] manifold is pretty manageable and I just
[37:51] don't know if operations h is
[37:53] constrained in the same way if that
[37:55] feels to me more like kind of like the
[37:58] universe [laughter] like heavy tailed
[38:00] and exceptionbased and power laws and
[38:02] all the tough stuff for these models to
[38:04] learn. But I don't I don't know. I mean
[38:05] you're the one that's in the field so
[38:07] maybe that's not the case. Maybe they're
[38:08] manageable, but
[38:09] >> but I I I will tell you the history of
[38:11] systems as you know is is you always
[38:15] want to build a smart algorithm that
[38:17] does the efficient thing. The smart
[38:18] virtual memory management, the smart
[38:21] congestion control, the smart routing.
[38:24] And what do we do instead? Every single
[38:25] time [laughter]
[38:26] >> we build the
[38:27] >> We overprovision. No, we overprovision
[38:29] every time. No, literally we every
[38:31] single time. Like like remember like
[38:33] swap memory where you like swap to disk?
[38:35] Dude, if you did that, you were [ __ ]
[38:36] Like that's it. Your computer was dead.
[38:38] So like we literally overprovision RAM
[38:40] and like routing you know how many like
[38:42] adaptive routing algorithms there are
[38:43] like we will respond to traffic but you
[38:45] have oscillations like [ __ ] it we
[38:46] overprovision we just use you know so I
[38:49] just feel like the history of systems is
[38:51] you don't predict stuff you
[38:51] overprovision because you can't and I
[38:54] feel like I mean you know so I feel like
[38:58] in the operation side so either we build
[39:00] this you know we we'll build the systems
[39:02] up front that are overprovisioned so you
[39:03] don't have to like do kind of complex
[39:04] operational stuff or maybe AI can do the
[39:08] way that a human being, but I think it's
[39:09] very different than code because you're
[39:11] dealing with like exceptions, there's
[39:13] not a lot of data, etc. I just feel like
[39:15] a different domain, but I don't know.
[39:16] I'm a I'm a VC. I'm just like,
[39:19] >> you know, I I I I kind of like the
[39:20] reason that I'm more uh optimistic here
[39:24] is because I live in the world where
[39:26] like what we do is actually deal with
[39:28] the observable world. Like
[39:30] >> I was going to say you're the literal
[39:31] expert on this. So, it's silly for me to
[39:32] even like ask you these questions.
[39:33] >> No, not at all. Because like in in my
[39:36] world, we We h like my assumption is
[39:39] that we have all the data is that like
[39:42] we know exactly when like a new thing
[39:45] happens or like an exception emerges or
[39:48] like a workload changes or like
[39:50] whatever. Like in my world, all of that
[39:52] has already been collected and streamed
[39:55] through and like
[39:56] >> but but the model that is not trained on
[39:59] a grounded source for that stuff like
[40:01] that.
[40:01] >> No, you're not.
[40:02] >> That would be really hard,
[40:03] >> right? But the but the story here isn't
[40:05] like, "Hey, Mr. Claude, here's a bunch
[40:10] of logs. Is something going on?" Right?
[40:13] And it has to be much more uh able to
[40:16] run experiments and kind of dive in and
[40:19] see like what are the different shapes
[40:22] of possible causal chains that actually
[40:24] fit in the environment that exists.
[40:26] >> I see. So yeah, this is what you said
[40:27] before now. Now I see what you're
[40:28] saying. You're saying that like
[40:30] >> the universe is heavy tailed. The the
[40:33] number of bad things that can happen is
[40:34] is probably ex an exception thing, but
[40:36] like the set of steps used to debug it
[40:39] is a manageable thing. That's what you
[40:41] can you can teach one of these models
[40:43] and that's what you have and therefore
[40:44] if you give it the right signals it can
[40:45] go through a number of steps to actually
[40:47] constrain this problem.
[40:48] >> Exactly.
[40:49] >> Interesting.
[40:50] >> Exactly. So it really is taking the role
[40:52] of the human because remember remember
[40:54] the whole remember the whole chaos
[40:55] engineering wave where like
[40:57] >> and and the reason it didn't go anywhere
[40:58] is like you're very so quickly like
[41:00] trying to like you know reduce like a
[41:04] state space that was atoms in the
[41:05] universe like so like super quickly like
[41:07] and actually make that manageable and it
[41:09] was like the human being that would go
[41:10] through a steps that would do that on
[41:11] the actual debugging and [snorts] this
[41:13] is what you're saying the AI would do.
[41:15] >> Yes. Right. So the goal isn't to like
[41:18] have the model just be like this is what
[41:20] happens. It's hey run through this
[41:22] workflow to determine what happened. And
[41:26] like we're trying that also on the
[41:27] security side for example where like hey
[41:29] in security you have a something
[41:33] different changed in your like
[41:35] statistics like some behavioral
[41:37] difference in terms of what entities
[41:39] exist the cardalities of various things
[41:41] that exist and so that becomes a
[41:43] perturbation in the
[41:45] >> uh whatever the universe and so we take
[41:47] that signal and then we feed it into an
[41:49] AI and we're like hey AI
[41:52] >> come up with a bunch of possible causal
[41:54] chains that exist and then go through
[41:58] and verify if any of those fit the
[42:01] pattern of the data that we actually
[42:03] see. And so that becomes a much more
[42:06] like hey here's a here's like a uh I
[42:10] don't know like fit this square into
[42:13] find the peg or like find the hole that
[42:15] fits this peg. And that seems even
[42:17] harder to me cuz the operations one the
[42:20] argument is
[42:21] >> the system can get in states in into
[42:24] broken [clears throat] states that are
[42:25] kind of like very rare and there's a lot
[42:27] of those rare states
[42:30] >> and so it's just not something that you
[42:31] can you know train a model on just
[42:33] because there's so many rare states it
[42:35] can get into that are that are bad and
[42:38] so maybe you can reduce that by doing
[42:39] the steps. The security one seems even
[42:42] harder because this isn't just like the
[42:45] natural order of rare states. You
[42:46] actually have an adversary that is
[42:48] >> trying to like intentionally find rare
[42:51] states and intentionally find stuff that
[42:53] you you you can't do. And so it may be
[42:56] the case that like code becomes solved.
[42:58] It takes a little while, you know,
[42:59] longer to do operations and it take a
[43:00] lot longer.
[43:01] >> Yeah. But like also like think about I
[43:03] mean I don't know to me like the whole
[43:04] world is sort of quantized to a certain
[43:06] extent. the plank length. [laughter] Is
[43:09] that what the physicists say? Yeah, I
[43:11] don't know. I'm not a physicist.
[43:13] Constant.
[43:14] >> I [laughter] think there's like actually
[43:15] I I literally don't know what I'm
[43:16] talking about right now, but I literally
[43:18] actually think that like like like
[43:19] spacetime is discretized into a blank
[43:21] length. But but yes I I mean I agree
[43:23] like to a certain extent what you just
[43:25] said but for operations and systems in
[43:28] general like there is a discretization
[43:31] of like
[43:33] the possible states and the path through
[43:36] them that are possible.
[43:38] >> Yeah.
[43:38] >> Right.
[43:38] >> I just think when you're dealing with
[43:40] >> there might be there might be a ton of
[43:43] them.
[43:44] >> There might be a ton of them. I'm going
[43:45] to tell you remember Sundar who um was
[43:47] in our in our grad school group.
[43:49] >> Yeah. in their ear.
[43:52] >> Yeah. Yeah. Yeah. Yeah. So, I remember
[43:53] he came to me once and he and he he was
[43:55] he was he was looking very sad. He's
[43:57] like, "Martin, I just had this really
[43:58] sad realization
[44:00] >> that, you know, if you take any MP3,
[44:03] remember MP3 is like the [laughter]
[44:04] music for us."
[44:04] >> Yes. He said he said MP3s have a finite,
[44:08] let's say, you know, four minutes. It's
[44:09] like four megabytes, you know, and that
[44:12] means there's a finite number of songs
[44:14] in the universe
[44:15] >> that can no because you know because
[44:17] it's it's all made of it's all made of
[44:20] bits, right? And it's four megabytes and
[44:21] that means you can you could just
[44:22] enumerate every one of those concepts.
[44:24] So so I'm so sad in this notion of like
[44:28] the the finitness of music, right?
[44:30] Literally, this is like we were walking
[44:31] on the dish when he had this
[44:33] conversation and I'm like,
[44:34] "Motherfucker, dude, you know,
[44:36] [laughter]
[44:37] you know how quickly you've like
[44:38] overcome like the atoms in the universe.
[44:40] You haven't even gotten to a K, right?
[44:42] [laughter] It's like it's like two to
[44:43] the two to the two to the two." Like,
[44:45] what are you talking about here, man? Be
[44:47] happy.
[44:47] >> Two to the two to the two four
[44:49] megabytes.
[44:50] >> FOUR MEGA YOU LIKE THIS IS so much
[44:52] state. Like, be happy. That's that's
[44:54] effectively infinite, right? Like what
[44:56] are we talking about here? [laughter]
[44:59] So I I do think at some level like we do
[45:01] underestimate like the number of states
[45:03] that like you know systems can get into
[45:06] and like maybe I will tell you from the
[45:09] perspective of a VC that looks at these
[45:10] companies the ones that are trying to
[45:12] tackle operations are having a tough go
[45:14] at it. It is a it seems much much
[45:16] tougher than those that are tackling the
[45:18] code because I think it's a much tougher
[45:20] problem. I 100% I agree. It is a much
[45:23] tougher problem and and it's not just
[45:25] because like the systems themselves are
[45:28] you know there's like external loads
[45:29] that are random that kind of h happened
[45:31] to you or that uh you know somebody just
[45:34] shipped a new thing and rolled it out
[45:36] into production didn't tell anybody or
[45:37] like
[45:38] >> uh there's some state that actually
[45:40] happens when like this thing happens and
[45:42] this thing happens and then you end up
[45:43] in a deadlock. Um it's also because it
[45:47] involves humans a lot more and not just
[45:50] like one human who's the coder. It
[45:52] actually also involves
[45:53] >> the pools of knowledge are distributed
[45:55] too.
[45:55] >> Exactly. and the interactions between
[45:57] like what it takes to like
[45:59] >> connect the pieces and put them together
[46:01] like
[46:02] >> but ultimately I mean I think about like
[46:05] where AI is and like what are the
[46:06] possible futures and like the the depth
[46:09] and the forwardlooking capacity of all
[46:11] the things like I think code is just to
[46:15] me it's a microcosm of like may maybe
[46:18] I'm just too fatalistic maybe I'm just
[46:20] like too pessimistic about how the world
[46:23] works and and I think of everything is
[46:26] like following some kind of
[46:27] probabilistic path. Um, and so to me
[46:30] like if we can make it work in code, we
[46:33] can continue expanding that level of
[46:35] complexity bit by bit and kind of adding
[46:38] layers and layers and layers until we
[46:39] can handle those problems of
[46:41] >> a lot of people. It's not fatalistic.
[46:43] It's the opposite. That's very
[46:44] optimistic is what people would say. For
[46:47] for me, I don't think the universe gives
[46:48] up its secrets that easily, but like
[46:50] we'll see. I mean like this is kind of
[46:51] the big question and and I would say to
[46:55] as yet it's not an answered question.
[46:57] >> No. Yeah. No. But
[46:59] >> I don't know a lot of things that um as
[47:02] humans we think of as being
[47:06] random or
[47:08] >> impossible to tackle. Yes. And that's
[47:11] what we saw with with coding like you
[47:13] know initially like people thought like
[47:15] the the possible world of all these
[47:18] things that you can implement in code
[47:19] are kind of limited and you can't really
[47:22] get a system to potentially understand
[47:25] the full complexity of a human brain
[47:26] that's required. But I think over time
[47:30] it'll all It's so funny that I'm
[47:32] actually telling you this and you're
[47:34] like the VC who's looking into the
[47:35] future and being like no well I mean I
[47:38] I'll tell you my view. My view is like
[47:39] the that this the study of these models
[47:41] are you train on data and um either that
[47:48] I that you know the
[47:52] let's try to say so you you train on
[47:53] data and that represents a distribution
[47:55] that distribution is useful or not and
[47:56] so like when you [clears throat] train
[47:57] on language the fact that it's useful
[47:59] that to me says that humans are so
[48:01] remarkable because they've looked at the
[48:03] universe which is this very
[48:04] highdimensional system and they've
[48:06] reduced it to a manifold that's actually
[48:08] useful
[48:09] for a model, right?
[48:10] >> Yes.
[48:10] >> Right. But that's different than the
[48:11] universe. Like that's there's a
[48:13] reduction from this high dimensionality
[48:14] to the language is much less
[48:16] dimensional.
[48:16] >> And I think that code is a reduction of
[48:19] all the operational states of a of a
[48:21] machine into like the instructions that
[48:22] make it run. And so to me, there's still
[48:24] a disconnect between like the
[48:25] instructions or the the description and
[48:28] the actual like the actual running of
[48:30] the universe and actually running the
[48:31] system. I think we can make gains toward
[48:34] it. But I what I will say is like you're
[48:37] kind of converging on simulation. Like
[48:39] there are things that are
[48:39] computationally irreducible. Like you
[48:41] kind of have to run it to get the
[48:42] answer. You can't just like
[48:44] >> you know predict it. And so like maybe
[48:46] we just are at an event horizon where
[48:48] like
[48:49] >> like models will solve anything that you
[48:52] don't have to actually simulate to get
[48:53] the answer from. I but why why why do
[48:57] you have to like just think about it in
[48:58] terms of hey it'll have to answer the
[49:00] question like now as opposed to being
[49:03] able to simulate and and sorry being
[49:06] able to plan its own uh execution to
[49:11] gain the information that it needs in
[49:13] order to be able to
[49:14] >> No maybe that's where we're going. Yeah.
[49:16] Like
[49:16] >> that's kind of what I'm fair enough.
[49:18] >> Yeah. And and I think like there's a
[49:20] lower dimensional there's lower
[49:21] dimensional space which is like these
[49:23] steps that you would take that we can
[49:25] reduce a whole bunch with AI and then
[49:27] whatever steps that you need that that
[49:29] can't do it can create the code to do or
[49:31] can simulate.
[49:32] >> And the point that you're making though
[49:33] is that like hey look the model can't
[49:35] actually go and figure out the
[49:37] additional dimensions of the world that
[49:39] we need to go and find to like model the
[49:42] universe into its world. Like that's
[49:44] probably I mean that's probably true.
[49:48] But can it actually simulate a scientist
[49:52] sufficiently to be able to get to that
[49:54] point? Like maybe,
[49:56] >> maybe. Cool. Can I ask you a question?
[50:00] How much of a of uh the VC work do you
[50:04] think is something that
[50:07] an AI can drive? So my model for VC is
[50:13] it's very relationship driven. It's I
[50:16] think so. Okay. I just I just have a
[50:18] belief that markets are actually pretty
[50:19] efficient, you know, and so it's not
[50:21] about like finding the company that
[50:23] nobody else finds. I think people all
[50:24] find like the right company. I think
[50:26] they're efficient. And then I also have
[50:27] the model of VC that [gasps]
[50:30] um, you know,
[50:33] tech tech is positive and I'm kind of
[50:36] optimistic about all tech and I don't
[50:37] worry about TAM. And so
[50:40] I I I think the key is access. It's like
[50:43] raising money. It's access. It's ability
[50:46] to win deals and there's ability to add
[50:48] value after that happens. And a lot of
[50:50] that is like
[50:52] stuff that that startups just can't
[50:55] naturally do like you know recruit you
[50:59] know big execs or help with that. And so
[51:02] the way that I think about venture you
[51:04] kind of have to be a human being to do
[51:06] it's like you know um
[51:09] because because that's the actual job.
[51:11] So for that I don't think like an AI
[51:14] if you gave an AI simulated universe
[51:16] where it could like you know like
[51:17] whatever reward hack like building trust
[51:20] relationships fine um but I think that
[51:22] that would have to be to be [laughter]
[51:25] that said I will say I have seen anal
[51:27] analysis done by AI that are as good or
[51:31] better than any you know gene analyst
[51:32] I've ever worked with like so for the
[51:34] the the analysis piece um they're great
[51:39] but I think this is more an indictment
[51:41] on VC's ability to like analyze that
[51:43] than it is like how powerful you know
[51:46] the models are. I just think it's all a
[51:47] kabuki show and [ __ ] I just think
[51:49] the right VCs are like you know you know
[51:52] tech is is is net positive. You know
[51:55] tech waves always add value like let's
[51:58] find the best company and invest in it.
[51:59] The markets are very very efficient and
[52:01] so I I I just feel like the answer to
[52:02] this question is almost kind of a
[52:04] technical like my answer is almost a
[52:05] technicality. So how like you know one
[52:08] of the things you know I came here I
[52:10] wanted to ask you this real this
[52:11] question which is like
[52:13] >> you know we started talking specifically
[52:15] about ops.
[52:16] >> Yeah.
[52:17] >> But to me like the there's this world of
[52:20] AI writing code which is kind of like
[52:22] this abstract simulation world that it
[52:24] lives in.
[52:25] >> Yeah.
[52:25] >> And then there's the rest of the world.
[52:27] >> Yeah. Exactly. I the entire discussion
[52:29] is is this. It's like there's like this
[52:31] >> there's like the AI the world that the
[52:33] AI lives in which is all digital and
[52:35] like whatever it is the training data
[52:37] the indistribution world and there's the
[52:38] actual world and like those things are
[52:39] not necessarily the same. They're quite
[52:41] often not. And so the question is like
[52:44] to what extent do you need to train an
[52:46] AI to be able to survive in a world and
[52:51] like what are the things that it
[52:52] actually I I don't want to answer that
[52:54] question like what do you need to to
[52:55] train but like I'm curious about the
[52:57] open spaces.
[52:58] >> So we talked a little bit about
[52:59] operations. You said I don't know if
[53:01] it's going to be solved or not. I'm like
[53:03] yeah Martina it's going to be solved.
[53:05] And uh in my mind like there's so many
[53:07] other things like for example
[53:09] self-driving cars like for a long time
[53:11] we thought that that's not going to
[53:13] happen but it's clear that it's actually
[53:16] going to be done. Um and it's going to
[53:18] happen. It's just taking time because of
[53:20] a lot of different things. And so what
[53:22] are like what are the other next stages
[53:27] the next areas that you think are kind
[53:29] of like lined up right after code? I
[53:34] mean my my sense is is you know AI is
[53:37] going to continue to take off in a
[53:38] wildfire for a constrained domain where
[53:40] the where the solution space is in a
[53:41] constrained domain right where the
[53:42] distribution of the solution space is
[53:44] the same as the distribution of the
[53:45] training set the distribution of the
[53:47] syntax for code is the same as a
[53:49] distribution of the training set the
[53:50] distribution of the generated images is
[53:53] the same as the distribution of the
[53:54] training set in those you know areas I
[53:57] think it's going to continue to take off
[53:58] as a wildfire the domains where like the
[54:01] distribution of the solution is the
[54:03] natural world. I think it's going to,
[54:05] you know, have, you know, it's going to
[54:06] be much harder, take longer, and I think
[54:08] the economic equivalency is going to be
[54:10] much more difficult. Self-driving is a
[54:12] great example. I remember when Sebastian
[54:13] Thrun won the DARPA Grand Challenge when
[54:16] I was at Stanford. Remember that? And
[54:18] we're like, hooray. You [laughter] know,
[54:20] self-driving cars is around the corner.
[54:22] Seriously. And it took 20 years and $170
[54:26] billion of investment to get to worse
[54:29] unit economics than Uber, right? So
[54:30] that's, you know, so I think that like
[54:33] solving like
[54:36] like solving real problems where you
[54:38] have to deal with the the distribution
[54:40] of the real universe is just much much
[54:42] harder. And so I view as AI as this
[54:44] general tool. It's a very effective
[54:46] tool. It's one where like you have you
[54:48] know a distribution of data and then
[54:51] hopefully you can apply that to like you
[54:53] know some problem and we're going to
[54:55] apply it to a bunch of stuff and I think
[54:56] economic equilibrium is going to
[54:57] determine what works and what doesn't
[54:59] work because I think it's a very very
[55:00] general tool it's like a universal
[55:02] function approximator
[55:03] >> and wherever that nets out I mean again
[55:05] >> you know one of the things that I said
[55:07] earlier was like how AI was training
[55:09] humans to like use it.
[55:10] >> Yeah. And I very often think about like
[55:14] you know some of the problems that seem
[55:15] intractable was is because the world is
[55:18] kind of that way and then we're trying
[55:20] to fit AI or the tools that we're
[55:22] building to manage the world as it is.
[55:25] But in some situations we actually tend
[55:27] to change the world to fit to what we
[55:29] can actually do from
[55:30] >> 100%. Of course I think this is the
[55:31] story of robotics.
[55:34] Yeah.
[55:34] >> The the automated data warehouse. And so
[55:36] the the question like that always comes
[55:38] to my mind like when we were looking at
[55:40] these data warehouses and whatever and
[55:42] like the only one that worked was the
[55:43] one where it was like you know what just
[55:45] automate the whole [ __ ] thing. Yeah.
[55:47] >> Um
[55:48] >> and um and I wonder to like what extent
[55:51] as we kind of improve or change how we
[55:57] uh or kind of expand the domain of what
[55:58] we want to solve with AI tools and so on
[56:01] that we actually start modeling the
[56:03] world or changing the the world and the
[56:06] things that can actually happen to fit
[56:09] >> what AI needs it to look like.
[56:11] >> Yeah. I mean I listen I I think that
[56:14] right now a lot of AI just operates in
[56:15] the digital world. Yeah.
[56:16] >> And for that is like literally the
[56:18] solution set is the same distribution as
[56:19] a training set in a sense. And so that's
[56:22] fine. And I I think that again like in
[56:24] the case of equilibrium like I think
[56:26] like when you're dealing with the real
[56:28] world like you can solve any problem you
[56:31] want. It's just a matter of like compute
[56:33] and money and that the equilibrium point
[56:37] is going to be an economic one. And I
[56:39] think one way you can impact that
[56:41] function that you're doing equilibrium
[56:43] over is to constrain
[56:45] >> the environment is constrain the system.
[56:46] And so I I think that we're going to
[56:48] like we're in now in this meta economic
[56:50] equilibrium flywheel where we're kind of
[56:54] optimizing this super general but very
[56:57] inefficient process with a solution set
[56:59] that people value. And I think you
[57:02] listen we've got 30 years of this and a
[57:03] lot of value is going to be created.
[57:05] >> You know one you want to hear a funny
[57:07] story? Yeah.
[57:08] >> So, you know, we built this AI thing and
[57:11] we're like talking to people. We're like
[57:13] reaching out to customers. We're like,
[57:14] "Hey, we built this AI thing. You know,
[57:16] we're helping customers become more
[57:18] reliable. Here's like what we're doing."
[57:20] They're like, "Great. We want to talk to
[57:21] you." And then we go on the call and
[57:23] we're like, "Hey, here's what we do. You
[57:25] know, we built this system. Helps you
[57:26] optimize your data. It helps you reduce
[57:28] your costs. And then it helps you become
[57:30] more reliable because now we can find
[57:31] these problems and solve them before
[57:33] they hit your incident." They're like,
[57:34] "Wait, wait, wait, wait. Roll back. You
[57:36] said you can save 90% of our data dog
[57:39] bills.
[57:39] >> Yeah.
[57:40] >> Like that's what I want to buy. But but
[57:42] but the but the AI thing. And they're
[57:44] like, "No, no, no, no. That's what we
[57:46] want to buy right now."
[57:46] >> Yeah.
[57:47] >> And it's uh it's interesting like a lot
[57:50] of these uh like for code, you know, I
[57:53] always think about it. It's like, "Oh,
[57:54] for code, people just get it. They're
[57:55] like, oh [ __ ] it just works."
[57:57] >> Yeah. Yeah. Yeah.
[57:58] >> And on the op side or like anything
[58:00] that's beyond that,
[58:01] >> Yeah. it's very to understand the actual
[58:03] value and put a dollar value to it and
[58:05] be a pay out the door. 100%. So, and I
[58:07] also think from from a coding
[58:08] standpoint, literally it's engineering
[58:09] productivity which is one of the highest
[58:11] line items on these things. So, it's
[58:12] just very straightforward.
[58:13] >> Exactly. But you can also think about
[58:14] it, it's S productivity or it's like
[58:17] your business going down. Maybe you can
[58:19] save an incident and what the cost is.
[58:22] >> I don't know. It's it's weird. I think
[58:24] like how people think about it outside
[58:26] of the
[58:27] >> Yeah. I I also I also just think like
[58:29] like this code was so magical because it
[58:31] was so good and nobody thought that
[58:32] could happen. And I think the operations
[58:34] stuff now it's still like
[58:36] >> pretty early.
[58:36] >> It's still pretty early. So,
[58:37] >> yeah. [snorts]
[58:38] >> All right. Well, listen, it's great to
[58:40] chat. Thanks, J. Thanks,
