LangChain Unveils Self-Repairing Agent Cloud to Break Model Lock-In

Why Own Your Intelligence? Harrison Chase Keynote | Interrupt London — LangChain

LangChain

Enterprise utility depends on proprietary context and execution harnesses rather than foundation models, requiring businesses to operate specialized cloud architectures to prevent runaway API costs.

AI-assisted summary of Why Own Your Intelligence? Harrison Chase Keynote | Interrupt London — original source. Check the original for full context.

Source published:

Key ideas

  • Enterprise utility in generative AI is shifting from raw foundation models to the surrounding harness—integrating skills, context retrieval, execution sandboxes, and governance guardrails. (0:06)
  • LangChain introduced Managed Deep Agents, unifying its open-source Deep Agents harness with cloud infrastructure to run multi-tenant, cloud-based background agents instead of local desktop apps. (12:04)
  • LangSmith Engine v2 introduces autonomous fix verification via preview deployments and active red teaming, having already scanned over 70 million traces and detected more than 21,000 issues. (28:53)
  • LangSmith now incorporates decision models like Jev for high-speed, low-cost ingest-time evaluations and real-time semantic trace exploration via AI Search. (19:18)

Detailed notes

## Background Published by LangChain on YouTube on October 30, 2026, this keynote features Harrison Chase, Co-Founder and CEO of LangChain, speaking at the Interrupt London conference. According to the page description, Chase explains what it means for companies to own their intelligence across the model, harness, and context, walking through the LangSmith product suite including Managed Deep Agents, SmithDB observability, and LangSmith Engine v2.

## TL;DR Harrison Chase outlines the architectural evolution of generative AI from early prototypes to specialized, production-ready vertical agents, arguing that real defensibility lies in the system built around the LLM rather than the raw model alone. To help enterprises 'own their intelligence,' he articulates three pillars—openness, compounding capability, and governance—backed by an operational flywheel aligning platform teams, engineers, and domain experts. Chase details the LangChain stack designed to support this paradigm, introducing Managed Deep Agents for cloud runtime execution, SmithDB and LangSmith Trajectories for trace observability and semantic AI search, and LangSmith Engine v2 for autonomous bug diagnosis, proactive fix verification, and red teaming.

## Atmosphere & style The presentation carries the polished, high-energy tone of an enterprise tech keynote, blending strategic visionary framing with rapid-fire architectural deep dives. Chase speaks with pragmatic confidence and technical fluency, steering clear of empty hype to focus on concrete operational hurdles like runaway API costs, messy trace telemetry, and the 'whack-a-mole' cycle of prompt debugging. The atmosphere feels purposeful and product-driven, highlighted by live UI demonstrations, architectural flowcharts, and tangible internal case studies that ground LangChain's ambitious enterprise vision in everyday developer realities.

## Bulletin - [00:06](https://www.youtube.com/watch?v=XOUSuxY074U&t=6s) Enterprise utility in generative AI is shifting from raw foundation models to the surrounding harness—integrating skills, context retrieval, execution sandboxes, and governance guardrails. - [12:04](https://www.youtube.com/watch?v=XOUSuxY074U&t=724s) LangChain introduced Managed Deep Agents, unifying its open-source Deep Agents harness with cloud infrastructure to run multi-tenant, cloud-based background agents instead of local desktop apps. - [28:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=1733s) LangSmith Engine v2 introduces autonomous fix verification via preview deployments and active red teaming, having already scanned over 70 million traces and detected more than 21,000 issues. - [19:18](https://www.youtube.com/watch?v=XOUSuxY074U&t=1158s) LangSmith now incorporates decision models like Jev for high-speed, low-cost ingest-time evaluations and real-time semantic trace exploration via AI Search.

## Key takeaways - [00:06](https://www.youtube.com/watch?v=XOUSuxY074U&t=6s) Production agent success depends on the architecture wrapping the LLM rather than the foundation model alone. - [03:12](https://www.youtube.com/watch?v=XOUSuxY074U&t=192s) Owning your intelligence requires specialization across the model, the harness, and proprietary context, driven by open architectures, compounding capability, and governance. - [06:59](https://www.youtube.com/watch?v=XOUSuxY074U&t=419s) Scaling enterprise agents demands cross-functional collaboration among platform teams, business-unit engineers, and non-technical domain experts spinning a rapid build-test-deploy-monitor flywheel. - [09:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=593s) Deep Agents provides an open-source execution harness featuring sandboxes, context offloading, prompt caching optimization, and subagent orchestration. - [12:04](https://www.youtube.com/watch?v=XOUSuxY074U&t=724s) Managed Deep Agents executes agents in cloud environments with built-in channels, secrets proxies, and Model Context Protocol (MCP) governance rather than running locally on laptops. - [18:23](https://www.youtube.com/watch?v=XOUSuxY074U&t=1103s) Agent traces constitute an entirely new class of telemetry data requiring SmithDB, a specialized data store optimized for complex execution graphs and natural language reasoning paths. - [19:18](https://www.youtube.com/watch?v=XOUSuxY074U&t=1158s) Decision models like Jev replace expensive text generation with structured classification, enabling cheaper and faster online evaluations during trace ingestion. - [21:27](https://www.youtube.com/watch?v=XOUSuxY074U&t=1287s) LangSmith introduced granular input/output RBAC, upcoming ABAC, and BYOC hosting to protect sensitive enterprise data while keeping execution graphs visible to engineers. - [23:55](https://www.youtube.com/watch?v=XOUSuxY074U&t=1435s) AI Search uses streaming decision models to discover subtle semantic failures and user confusion across production traces without relying on rigid SQL queries or error codes. - [26:01](https://www.youtube.com/watch?v=XOUSuxY074U&t=1561s) Custom Apps allow teams to vibe-code domain-specific dashboard visualizations directly inside LangSmith using natural language and an open-sourced design system. - [27:38](https://www.youtube.com/watch?v=XOUSuxY074U&t=1658s) LangSmith Engine automates the agent engineering lifecycle by diagnosing root causes, clustering failure modes, and opening automated pull requests with proposed fixes. - [28:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=1733s) Engine v2 verifies its own proposed code fixes in isolated preview deployments and uses proactive red teaming to stress-test agent boundaries before production failures occur. - [30:30](https://www.youtube.com/watch?v=XOUSuxY074U&t=1830s) Agents can self-report in-flight failures via dedicated tool calls, attaching runtime annotations and reasoning directly to trace metadata for batch analysis. - [32:10](https://www.youtube.com/watch?v=XOUSuxY074U&t=1930s) The enterprise future will be powered by thousands of vertical, domain-specific agents rather than a single monolithic model.

## Detailed walkthrough ### [00:06](https://www.youtube.com/watch?v=XOUSuxY074U&t=6s) The Evolution of Agents and the System Around the LLM

Harrison Chase opens Interrupt London by welcoming the crowd, noting how the event series has expanded. While the inaugural Interrupt conference was held exclusively in San Francisco, the conference series now spans San Francisco, New York, and London. To set the stage for LangChain's future direction, Chase reflects on the historical trajectory of generative AI over the preceding years. In 2023, when LangChain was first gaining traction, the ecosystem centered on building exploratory prototypes because the domain was brand-new. By 2024, production applications began appearing, though they remained relatively simple: single large language model (LLM) calls, basic chains, and conversational chatbots. In 2025, teams began venturing into agents for the first time.

```mermaid flowchart LR accTitle: Evolution of GenAI Applications accDescr: Timeline from 2023 exploratory prototypes to specialized production agents in 2026. A["2023: Prototypes<br/>Exploratory concepts"] --> B["2024: Simple Apps<br/>Chains & chatbots"] B --> C["2025: Early Agents<br/>First agent explorations"] C --> D["2026: Production Agents<br/>Vertical domain harnesses"] ```

Chase highlights that agents have finally come alive, profoundly restructuring the application layer across various vertical use cases. A critical realization has emerged across the industry: the model itself is not enough. While an LLM sits at the core driving reasoning, real enterprise utility comes from the architecture wrapping it. As Chase explains, “The thing that makes all of these application layer agents really take off and actually work is everything around the LLM, the system around it.” He illustrates this paradigm through real-world enterprise implementations featured at the conference:

* **monday.com**: Architected a closed, continual feedback loop using automated evaluations to iterate and systematically improve agent performance. * **Harvey**: Rather than relying strictly on off-the-shelf foundation models, they engineered specialized systems designed to fine-tune models specifically for legal and professional tasks. * **Orange**: Constructed an enterprise-grade agent harness anchored heavily around governance and policy guardrails to make execution safe and reliable.

Across all these deployments, production success hinges on the harness—the ecosystem providing skills, tool integrations, context retrieval, execution controls, and governance policies.

---

[03:12](https://www.youtube.com/watch?v=XOUSuxY074U&t=192s) Why Own Your Intelligence: The Three Pillars

Because generic models require deep domain customization to be useful, organizations face a strategic imperative: they must own their intelligence. Specialization across the model layer, the harness layer, and the context layer defines what it means to retain ownership. Chase outlines three macroeconomic and technical tailwinds accelerating this demand:

1. **Surging Compute and API Costs**: Broad enterprise adoption drives usage volumes rapidly upward; without architectural control over the underlying agentic systems, costs inevitably balloon out of hand. 2. **Rise of Open Models**: High-performing open-source and open-weight models from providers like NVIDIA, Kimi, and DeepSeek offer competitive unit economics, but more importantly, give teams the freedom to post-train and optimize weights for domain-specific tasks. 3. **The Differentiating Power of Data**: Generic models like ChatGPT or Claude cannot match domain-specific tools without proprietary context. The internal data, operational APIs, and domain-specific knowledge embedded into the harness are what create actual defensibility.

```mermaid flowchart TD accTitle: Three Pillars of Owning Your Intelligence accDescr: Openness, compounding capability, and governance form the foundation of intelligence ownership. subgraph Pillars["Core Pillars of Intelligence Ownership"] P1["<b>Open</b><br/>• Model optionality<br/>• Controllable harness<br/>• Flexible context sources"] P2["<b>Compounding</b><br/>• Manual evals & traces<br/>• Observability feedback<br/>• Automatic memory"] P3["<b>Governed</b><br/>• Identity & authorization<br/>• Traceable acting authority<br/>• Cost & behavioral guardrails"] end ```

To support this shift, Chase articulates LangChain's guiding objective: “Our mission at LangChain is to enable every company to own their intelligence.” He breaks this down into three foundational pillars:

* **Open**: Spans the entire apparatus. Rather than just referring to open-weight models, Chase clarifies that open primarily means total model optionality—retaining the flexibility to swap in the fastest, cheapest, or highest-performing engine for a specific job. It also means utilizing an open, controllable harness to direct agent behavior, as well as maintaining open context pipelines that pull data from internal data stores, APIs, and long-term memory. * **Compounding**: Initial prototypes never ship directly to millions of users; agents must compound capability over time. As Chase observes, “You want the intelligence of that agent to compound over time, whether it's manually with observability and then noticing things and then fixing those things, or whether it's automatically in the realm of kind of like memory.” * **Governed**: Critical for internal enterprise utility. Organizations must know exactly who an agent is acting on behalf of, track operational expenses, audit its reasoning paths, and maintain runtime guardrails. Chase notes, “We think agents will absolutely transform the operating model of companies out there, but you have to have governance over them.”

---

[06:59](https://www.youtube.com/watch?v=XOUSuxY074U&t=419s) Organizing People, Process, and Technology

Transforming a company to own its intelligence requires aligning organizational structure, operational methodology, and technical infrastructure. Chase organizes these elements across three categories: People, Process, and Technology.

```mermaid flowchart TD accTitle: Operating Model for Agent Development accDescr: The interaction between People, Process, and the Technology Stack. subgraph People["People"] direction TB P_Plat["Core GenAI Platform Team"] P_Eng["BU Agent Engineers"] P_SME["Domain Experts / Non-Developers"] P_Plat --- P_Eng --- P_SME end

subgraph Process["Process Flywheel"] direction TB PR_B["Build Agent"] --> PR_T["Test & Deploy"] PR_T --> PR_M["Monitor Trajectories"] PR_M --> PR_B end

subgraph Tech["LangChain Technology Stack"] direction TB T_Int["Intelligence Layer<br/>(Automated SDLC & Red Teaming)"] T_Obs["Observability & Evals<br/>(Compounding Feedback Loop)"] T_Run["Agent Runtime<br/>(Open-source Harness & Cloud Deployments)"] T_Int --- T_Obs --- T_Run end ```

For **People**, successful organizations organize agent development across three concentric personas: 1. *Core GenAI Platform Teams*: Dedicated agent experts who stand up foundational infrastructure, standard tooling, and scalable workflows for the broader business. 2. *Business Unit Agent Engineers*: Builders who sit close to operational problems, using platform tools to construct specialized, production-ready agents for distinct business lines. 3. *Non-Technical Builders and Domain Experts*: Non-developer subject-matter experts who understand operational edge cases and serve as the arbiters of what constitutes high-quality outputs.

On the **Process** side, Chase emphasizes that agent creation is non-linear and demands a rapid build-test-deploy-monitor loop. He notes:

> “The best teams that we see shipping agents, they're the ones who spin this flywheel as quickly as possible.”

Finally, Chase maps these operational demands to the **Technology** stack LangChain is constructing, which consists of three distinct layers: - **Agent Runtime**: Comprising LangChain's open-source libraries and managed deployment environments designed to execute agent harnesses reliably. - **Observability and Evaluations**: The feedback backbone that ingests execution traces to drive compounding performance gains. - **Intelligence Layer**: A new tier deployed on top of the stack. Just as traditional software development lifecycles (SDLC) are being automated, the agent engineering workflow itself must be automated by systems that detect runtime defects, run self-evaluations, and refine agent architectures autonomously.

[09:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=593s) The Three Layers of Agent Architecture and the Deep Agents Harness

Harrison Chase shifts his focus from broad strategy to the runtime environment, reiterating that building agents requires looking at the system in three distinct layers: business logic, the harness, and infrastructure. Business logic encompasses the domain-specific instructions, custom tools, tailored skills, and hooks used to modify the agent. He highlights that this layer is inherently proprietary to each organization: “This is stuff that we can never give you. This is domain-specific stuff that you all or your teammates are experts in and are going to codify into, again, all of these different things to really drive the agent in a particular domain or particular use case.”

```mermaid flowchart TD accTitle: Three Layers of Agent Architecture accDescr: Diagram showing Business Logic running inside the Deep Agents Harness, supported by Cloud Infrastructure.

subgraph Logic["Business Logic (Customer-Owned)"] BL["Instructions, Tools, Skills & Custom Hooks"] end

subgraph Harness["Deep Agents Harness (Execution Layer)"] direction TB EXEC["Execution Environment (Filesystem / Sandbox / Code Interpreter)"] CTX["Context Management (Summarization, Offloading, Prompt Caching)"] CTRL["Steering & Control (Human-in-the-Loop, Planning, Subagents)"] end

subgraph Infra["Managed Infrastructure"] INF["LangSmith Deployments, Tool Servers, Gateways & Sandboxes"] end

BL --> Harness Harness --> Infra ```

The harness layer encompasses everything wrapped around the model to connect it directly to external data sources, compute, context, and business rules. To streamline this layer, LangChain released **Deep Agents**, an open-source base harness engineered with foundational execution primitives. It provides execution environments ranging from simple file systems and lightweight code interpreters to full sandboxes. Deep Agents also incorporates production-grade context engineering patterns: conversational summarization for extended threads, context offloading for massive tool call outputs, architectural design tailored for prompt caching, human-in-the-loop steering mechanisms, and built-in planning workflows with subagent orchestration. While developers can still assemble granular custom harnesses from scratch using LangChain or LangGraph, Deep Agents provides an opinionated, production-ready starting point out of the box.

- **Business Logic Layer**: Instructions, domain-specific tools, skills, and specialized lifecycle hooks. - **Harness Primitives**: Memory and skill management, prompt caching optimization, human-in-the-loop steering, and dynamic subagents. - **Execution Environments**: File systems, lightweight code interpreters, and secure sandboxes.

---

[12:04](https://www.youtube.com/watch?v=XOUSuxY074U&t=724s) Managed Deep Agents: Running Agents in the Cloud

Moving to the infrastructure layer, Chase pushes back against the trend of desktop-bound local agents: “I don't think the future is just agents running on your laptop just for you. I think it's agents running in the cloud, working in the background, proactive, always on, multi-tenant, serving many users.” Running enterprise agents in the cloud demands extensive infrastructure: a robust agent runtime, authenticated and governed tool integrations, sandboxes for writing and executing code safely, and an LLM gateway to handle reliability, fallbacks, and cost controls.

To eliminate the friction of stitching these disparate systems together, LangChain combined the harness layer and managed cloud infrastructure into **Managed Deep Agents**. Rather than assembling five separate backend components, developers write their business logic and deploy it immediately to a scalable multi-tenant environment using a single CLI command (`mda deploy`). Chase emphasizes three core advantages of this setup:

1. **Rapid Connectivity**: Pre-built channels (Slack, Microsoft Teams, GitHub, incoming email) with out-of-the-box identity and authorization management. 2. **Decoupled Sandbox Execution**: Unlike designs that run the entire agent inside an isolated environment, Managed Deep Agents runs the agent runtime externally and treats the execution sandbox as a callable tool, backed by a secrets proxy to prevent model exposure to sensitive credentials. 3. **Enterprise Governance**: Built-in guardrails managing model permissions, spending caps, data protection, audit logging via Model Context Protocol (MCP) tool servers, and automated tracing via LangSmith Deployments (complete with checkpointing and task queues).

```mermaid flowchart LR accTitle: Managed Deep Agents Architecture accDescr: Multi-tenant channels trigger the agent runtime, which interfaces with external tools, models, and sandboxes under strict governance.

subgraph Triggers["Input Channels"] SL["Slack / Teams"] GH["GitHub Events"] EM["Email / Webhook"] end

subgraph Core["Managed Agent Runtime"] DEP["LangSmith Deployment"] GOV["Governance & RBAC"] end

subgraph Services["Infrastructure Services"] SB["Sandbox Tool (via Secrets Proxy)"] GW["LLM Gateway (Fallbacks & Budgets)"] MCP["Tool Server (MCP Protocol)"] OBS["LangSmith Tracing & Memory"] end

Triggers --> DEP DEP --- GOV DEP --> SB DEP --> GW DEP --> MCP DEP --> OBS ```

---

[16:49](https://www.youtube.com/watch?v=XOUSuxY074U&t=1009s) Internal Agents at Work: Open SWE and GTM Agent

To illustrate how deep agent harnesses function within daily operations, Chase highlights two internal agents built and deployed at LangChain: **Open SWE** and **GTM Agent**. Open SWE is an open-source coding agent whose responsibilities extend far beyond writing code snippets; it reviews pull requests, triages operational incidents, and will soon execute end-to-end QA workflows.

In parallel, LangChain relies internally on the proprietary GTM Agent to coordinate marketing, sales, and customer outreach. Designed as an always-on, autonomous assistant rather than a passive chatbot, it continuously reacts to incoming events and automations: > “It's a proactive agent, so it's always on. It's pinging you with updates. Pinged me to tell me that I had a big conference today. So thank you very much for that.”

The GTM Agent carries out background research, sends cross-domain communications using governed connectors, and spins up sandboxes to compile interactive artifacts that team members can access directly through generated preview URLs. Chase argues that deeply integrated, context-rich agents like these offer a genuine glimpse into the operational structure of future enterprises.

- **Open SWE**: Open-source engineering agent handling software authoring, PR reviews, incident triage, and automated QA. - **GTM Agent**: Always-on proactive assistant powering go-to-market operations, prospect research, outbound campaigns, and live sandbox-rendered visual artifacts.

---

[18:23](https://www.youtube.com/watch?v=XOUSuxY074U&t=1103s) Agent Traces as a New Data Primitive and SmithDB

Turning to the observability and evaluation feedback loop, Chase argues that monitoring autonomous systems requires treating agent traces as a completely new class of telemetry data. Unlike traditional application logs or shallow request-response metrics, agent traces feature deeply nested execution trees, massive token payloads, multimodal inputs, and unpredictable query access patterns. Crucially, they record unscripted human and machine reasoning: > “We think these traces contain natural language, and that natural language contains things like user intent and logic failures and glimpses into how people are using the agent, whether it's properly or improperly.”

Because standard databases struggle under the weight and structure of these execution graphs, LangSmith was engineered entirely around trace observability and evaluation. Sitting at the absolute core of LangSmith is **SmithDB**, an internally developed proprietary storage engine optimized explicitly for the high-throughput write requirements, variable payload sizes, and complex traversal paths inherent to multi-step agent traces.

[19:18](https://www.youtube.com/watch?v=XOUSuxY074U&t=1158s) Reimagining Ingest: Decision Models and Ingest-Time Online Evals

Harrison Chase explains that the query patterns required for dealing with agent traces are fundamentally different from traditional observability. SmithDB, the underlying database of LangSmith, is designed from the ground up to serve as the premier agent platform and data lake for these rich trace structures. Rather than treating traces as conventional logs or time-series metrics, LangSmith evaluates traces along four distinct axes: how they are ingested, stored, queried, and visualized. Harrison emphasizes: “And when we think about SmithDB and therefore LangSmith as a platform, we want to make it the best agent platform, the best data lake for these agent traces.”

The first axis of innovation is ingest. Agent traces arrive packed with natural language clues revealing how the system behaves, what end users are trying to accomplish, and how effectively the agent responds. Historically, teams captured these clues by running online evaluations powered by expensive frontier models or custom code judges. That paradigm is shifting rapidly with the emergence of decision models like Jev, which LangSmith now integrates directly as an online judge. As Harrison describes it: “What does that mean? It basically means that instead of generating text, it makes decisions.”

When using Jev as a judge, users pass in the agent state alongside a curated list of questions and candidate answers. Instead of expending compute to generate open-ended natural language explanations, Jev simply selects the best-fitting choice from standard, intuitive answer types and scores across multiple evaluation dimensions. Because it bypasses token generation, Jev operates dramatically faster and at a fraction of the cost of frontier LLMs. Harrison underscores the broader architectural lesson: “We think this is a great example of how the ingestion of agent data is different from kind of like traditional logs.”

- **Four Operational Axes of SmithDB**: Ingesting, storing, querying, and visualizing agent trace data. - **Jev as a Judge**: A purpose-built decision model that chooses among predefined categories or scores instead of generating text, enabling low-cost, high-speed automated evaluation at ingestion time. - **Natural Language Signals**: Ingest-time evals extract semantic signals directly from conversational turns, tool calls, and model outputs rather than relying on status codes or error logs.

```mermaid flowchart TD accTitle: SmithDB Agent Trace Platform Architecture accDescr: Diagram showing the four axes of trace management in SmithDB alongside the proactive intelligence layer in LangSmith Engine.

subgraph INGEST ["1. Ingestion Layer"] A["Agent Execution Traces"] --> B["Online Evals"] B --> C["Jev Decision Model Judge"] end

subgraph STORAGE ["2. Storage Layer (SmithDB)"] C --> D["SmithDB Cloud / On-Prem / BYOC"] D --> E["Granular Security (RBAC / ABAC)"] D --> F["LangSmith Trajectories (Message Lists)"] end

subgraph QUERY ["3. Query & Search Layer"] F --> G["Full-Text & Metadata Filters"] F --> H["AI Search / AI Query (Real-time Decision Eval)"] end

subgraph VIZ ["4. Visualization Layer"] H --> I["Standard Trajectory UI"] H --> J["Vibe-Coded Custom Apps"] end

subgraph ENGINE ["Intelligence Layer"] D -.-> K["LangSmith Engine (Proactive Agent Engineer)"] K -.-> L["Diagnosis, Clustering & Automated PR Fixes"] end ```

---

[21:27](https://www.youtube.com/watch?v=XOUSuxY074U&t=1287s) Storing Sensitive Traces: BYOC, Granular Permissions, and Trajectories

Turning to storage, Harrison highlights a massive operational divide between standard software logs and agent execution data: agent traces routinely capture deeply sensitive user inputs, proprietary context, and confidential enterprise data. To handle these compliance realities, LangSmith provides multiple hosting configurations. Beyond the multi-tenant cloud and full on-premise deployments, LangChain introduced Bring Your Own Cloud (BYOC), where all customer trace data remains safely inside the customer's cloud boundary while LangChain manages the hosted control plane.

Within the data storage model, LangSmith recently shipped Role-Based Access Control (RBAC) specifically scoped to trace inputs and outputs. Engineering teams frequently need visibility into the structural skeleton of an execution—such as tool invocations, token utilization, and latency—to monitor overall system health, but company policy often forbids them from seeing the actual raw inputs or generated outputs where personal or sensitive information lives. Looking ahead, Harrison previews attribute-based access control (ABAC), which will allow teams to tag traces with contextual metadata and apply granular authorization policies governing exactly which teams can view which traces. Acknowledging the unglamorous nature of enterprise security, Harrison remarks: “This type of, like, RBAC and ABAC is not usually something that I see people getting excited about, but it is so key for dealing with this sensitive information that's in all these traces.”

Beyond privacy controls, storage requires the right canonical schema. LangSmith launched LangSmith Trajectories to provide a standardized, message-list-based representation of agent interactions. While operators can still drill down into raw JSON payloads, tool invocations, and nested execution steps, presenting traces as clean conversational trajectories simplifies debugging and opens the platform to non-engineering subject matter experts for annotation. Furthermore, this message-first format mirrors the exact dataset schemas required for post-training and fine-tuning downstream models, making enterprise traces immediately training-ready.

- **Hosting Flexibility**: Multi-tenant cloud, fully isolated on-premise, and Bring Your Own Cloud (BYOC) with an external control plane and local data retention. - **Trace Input/Output RBAC**: Lets engineers inspect latency, token counts, and tool invocation trees without exposing confidential prompts or outputs. - **Upcoming ABAC Policies**: Dynamic metadata tagging to restrict trace access based on organizational attributes and data classification. - **LangSmith Trajectories**: A standardized message-list format that streamlines technical debugging, non-technical expert annotation, and offline post-training export.

---

[23:55](https://www.youtube.com/watch?v=XOUSuxY074U&t=1435s) Semantic Querying: Finding Needles in a Haystack with AI Search

Querying agent traces presents unique difficulties because the underlying data is heavily unstructured, natural language-heavy, and multimodal. Unlike deterministic software bugs that produce stack traces or HTTP error codes, agent failures rarely manifest as clean crash reports. Instead, failures emerge within subtle conversational semantics—such as an agent misunderstanding a subtle user requirement or a user becoming visibly confused by an ambiguous answer. Harrison notes that finding these subtle failures is the true bottleneck: “It's going to show up in the semantics of how people are using it, where the errors that happen aren't going to show up in logs.”

While LangSmith has long supported full-text indexing, metadata filtering, and filtering over large payloads, Harrison showcases a major leap forward called AI Search (or AI Query). Live on screen, Harrison demonstrates entering a plain natural language description into the trace search bar: *"user got confused by agent response."*

Under the hood, AI Search does not merely translate natural language into a rigid SQL or JSON query. Instead, LangSmith streams matching candidate traces through a lightweight real-time decision model, scoring each trace's semantic confidence against the prompt on the fly. In the interface, the engine highlights the exact textual evidence within the trace that matches the query, allowing engineers to isolate semantic bugs across millions of production logs without writing manual regex rules or hard-coded filters.

- **Semantic Failure Discovery**: Targets edge-case user interactions and soft hallucinations that bypass conventional error-handling and metric thresholds. - **Real-Time Decision Model Pipeline**: Rather than performing one-time text-to-SQL transpilation, candidate traces are streamed and evaluated dynamically against the query prompt. - **Inline Evidence Highlighting**: Displays specific conversational turns and tool responses that justify the decision model's confidence rating.

---

[26:01](https://www.youtube.com/watch?v=XOUSuxY074U&t=1561s) Visualizing Domain Data with Vibe-Coded Custom Apps

Visualizing trace data requires flexible front ends because agents operate across vastly divergent verticals. A legal research agent, an automated coding assistant, and a financial workflow agent each require completely different analytical lenses to review inputs, outputs, and intermediate scratchpads. While LangSmith Trajectories offers a clean baseline view, users repeatedly expressed a desire to tailor trace dashboards to their specific industry workflows.

To solve this, LangChain launched LangSmith Custom Apps. Demonstrating the feature via video, Harrison reveals an entirely "vibe-coded," custom HTML front end running directly inside the LangSmith workspace. Users can modify their interface in real time simply by typing conversational instructions into a prompt box. In the background, an embedded agent analyzes the workspace files, plans the structural changes, writes the underlying UI code, and immediately renders new dashboard widgets and visualization panes side-by-side with the chat.

Once generated, these vibe-coded views are saved as native, governed workspace pages accessible to the entire organization. Harrison observes that while teams were previously building one-off internal review tools outside LangSmith, centralizing that workflow provides critical guardrails: “But we think by bringing it into the UI, we can let people do this in a more governed way, where it knows all of the different APIs.” To ensure these bespoke apps feel polished and native, LangChain open-sourced its internal design system, allowing generated UIs to match the aesthetic and component styling of the core platform seamlessly.

- **Vibe-Coded Frontend Generation**: In-browser agentic development that generates custom HTML and JavaScript dashboard views from natural language prompts. - **Domain-Specific Workflows**: Allows legal, financial, or healthcare teams to build review interfaces suited to their specific document types and verification criteria. - **Open-Sourced Design System**: Guarantees visual and functional consistency between bespoke community apps and core LangSmith navigation. - **Enterprise Governance**: Custom apps remain securely authenticated against internal LangSmith APIs and persist as shared pages across workspace members.

---

[27:38](https://www.youtube.com/watch?v=XOUSuxY074U&t=1658s) Automated Intelligence: Diagnosing and Fixing Issues with LangSmith Engine

In the final section of his review of the platform, Harrison turns from observability infrastructure to actionable intelligence. Merely capturing, storing, and visualizing traces does not solve an engineering team's underlying operational burden. Harrison points out: “The visibility over the traces at this point has become table stakes.” The real pain lies in identifying systemic regressions across massive trace volumes, understanding whether a bug stems from the prompt, the retrieval context, or the base model, and implementing fixes without triggering regressions elsewhere.

To break teams out of the manual, reactionary "whack-a-mole" cycle of prompt engineering, LangChain introduced LangSmith Engine. Harrison describes the system as an automated, proactive agent engineer that continuously operates over production trace streams in the background. Instead of waiting for a human engineer to spot an issue, LangSmith Engine autonomously monitors incoming traces, clusters related failure modes, diagnoses root causes across model and retrieval layers, and prepares code-level remediation by opening pull requests against the agent's repository.

- **The Whack-a-Mole Problem**: Manual prompt tweaks often resolve one specific failure mode while inadvertently reintroducing regressions two months later. - **Automated Root-Cause Attribution**: Isolates whether errors originate in system instructions, context window truncation, retrieval failure, or base model capability. - **Proactive Agent Engineer**: Continuously scans, clusters, and diagnoses production traces to open automated pull requests with proposed fixes.

[28:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=1733s) LangSmith Engine v2: Proactive Fix Verification and Red Teaming

Harrison Chase walks through the continuous evolution of LangSmith Engine, highlighting the launch of Engine v2, which introduces major automated testing capabilities. When Engine analyzes agent workflows, it can automatically generate new data points and evaluation suites. If an organization connects Engine directly into LangSmith Deployments, it can even trigger deployments automatically. With Engine v2, LangChain added the ability for Engine to proactively test the code fixes it generates. Working within LangSmith Deployments, Engine takes the proposed patch, spins up an isolated preview deployment, and runs test cases against it to ensure the previously identified defects no longer occur before presenting concrete evidence to the developer that the fix works.

In addition to evaluating passive historical traces, Engine v2 introduces an active red teaming feature designed to proactively stress-test agents. Rather than waiting for runtime failures to emerge in production logs, the red teaming engine generates synthetic probes based on multiple probing strategies, fires them into the system, and evaluates the agent's behavior. Chase demonstrates this via a video clip showing a red teaming run in progress. The tool surfaces discovered vulnerabilities with comprehensive evidence: the exact prompt sent in, the observed output, the expected behavior, the judge's reasoning for flagging the failure, and recommendations on remediation steps. Developers can click directly into the underlying execution trace to inspect the failure step-by-step. As Chase describes it, “This is proactively kind of like stress testing your agent for places where it may mess up.”

```mermaid flowchart TD accTitle: LangSmith Engine v2 Automated Testing and Fix Verification accDescr: Diagram showing how LangSmith Engine v2 performs proactive red teaming and tests fixes via preview deployments.

subgraph RedTeaming["Proactive Red Teaming"] A["Generate Probing Strategies"] --> B["Inject Probes into Agent"] B --> C["Judge Observed vs Expected Output"] C --> D["Surface Issues, Traces & Remediation"] end

subgraph FixVerification["Proactive Fix Verification"] E["Engine Detects Trace Issues"] --> F["Generate Proposed Code Fix"] F --> G["Spin Up Preview Deployment"] G --> H["Run Verification Tests"] H --> I["Confirm Fix & Trigger Deployment"] end ```

[30:30](https://www.youtube.com/watch?v=XOUSuxY074U&t=1830s) Agent Self-Reporting and Engine at Scale

Observing agent execution in the wild revealed that agents often detect when they are faltering or hitting roadblocks mid-run. To prevent the system from having to rediscover those failure modes retroactively through downstream analysis, LangChain introduced an in-flight self-report tool. As an agent executes, it can invoke this dedicated tool or API endpoint whenever it encounters an issue, passing along structured annotations and chain-of-thought reasoning explaining what went wrong. The runtime immediately binds these self-reported diagnostics to the trace metadata in the background. When LangSmith Engine runs its scheduled batch analysis later that night, it ingests that agent-supplied reasoning directly, substantially accelerating root-cause analysis. Chase highlights the value of this direct feedback loop: “We think this is really really useful because agents can oftentimes recognize as they're running where things are going wrong.”

Chase details ongoing engineering work to make Engine faster, cheaper, and more effective at remediating bugs, benchmarked continuously against standard coding benchmarks. For enterprise and privacy-focused teams operating on-premises, LangChain released Bring Your Own Key (BYOK) support inside the self-hosted v7 release of LangSmith Engine. Highlighting Engine's rapid adoption across fast-scaling tech companies, Chase shares production milestones to date: Engine has scanned more than 70 million traces and automatically flagged over 21,000 discrete issues. Chase emphasizes that developer velocity in the agent era depends entirely on the speed of iteration: “I think a lot of the speed at which you can build agents is really rate limited by how quickly you can spin that flywheel.”

- **Trace Scale:** Over 70,000,000 traces scanned across LangSmith Engine deployments. - **Issues Uncovered:** More than 21,000 distinct agent failure modes and defects detected. - **Self-Hosted Features:** LangSmith self-hosted release v7 incorporates Bring Your Own Key (BYOK) capabilities for Engine. - **Self-Report Mechanism:** Runtime tool invocation attaches annotations and failure explanations to traces for nightly Engine processing.

[32:10](https://www.youtube.com/watch?v=XOUSuxY074U&t=1930s) The LangChain Stack and the North Star: Owning Your Intelligence

Chase synthesizes the complete LangChain product architecture into three foundational pillars: the runtime, the data/observability platform, and applied intelligence. On the runtime side, Managed Deep Agents reflects LangChain's firm belief that production agents must run in the cloud rather than locally on laptops, operating in a shared, multiplayer, and multi-tenant environment equipped with integrated identity, authentication, and service connectors. For observability and evaluations, LangSmith is architected to serve as the definitive data platform for agent traces, custom-built to handle the unique ingestion, storage, querying, and visualization demands that agentic workflows impose compared to traditional application logs. Layered over both is Engine's intelligence layer, which automates triage, evaluation, red teaming, and code patching.

Concluding his keynote, Chase articulates the core philosophy guiding LangChain's roadmap: helping organizations truly own their intelligence. He dismisses the notion of a single monolithic model running the global economy, arguing instead that enterprises must specialize systems by injecting their own proprietary context, harness logic, and domain knowledge. “The thing that drives us is really to help people own their intelligence, as I 100% think the future is thousands of vertical, domain-specific agents both externally and inside one's companies that are really driving change and improved kind of like processes and new value,” Chase states. He notes that building these systems reliably requires embedding domain expertise into model weights, prompts, harnesses, and evals—themes echoed throughout the technical talks at the conference. Chase thanks the audience for attending the inaugural London Interrupt conference and urges attendees to continue sharing feedback on what remains difficult as they push agents into production.

## Notable quotes > [12:04](https://www.youtube.com/watch?v=XOUSuxY074U&t=724s) I don't think the future is just agents running on your laptop just for you. I think it's agents running in the cloud, working in the background, proactive, always on, multi-tenant, serving many users. — Harrison Chase

> [00:06](https://www.youtube.com/watch?v=XOUSuxY074U&t=6s) The thing that makes all of these application layer agents really take off and actually work is everything around the LLM, the system around it. — Harrison Chase

> [03:12](https://www.youtube.com/watch?v=XOUSuxY074U&t=192s) We think agents will absolutely transform the operating model of companies out there, but you have to have governance over them. — Harrison Chase

> [27:38](https://www.youtube.com/watch?v=XOUSuxY074U&t=1658s) The visibility over the traces at this point has become table stakes. — Harrison Chase

## Mentioned Harrison Chase, LangChain, LangSmith, LangSmith Engine, Managed Deep Agents, Deep Agents, SmithDB, Jev, LangSmith Trajectories, AI Search, LangSmith Custom Apps, LangSmith Deployment, LangGraph, Open SWE, GTM Agent, monday.com, Harvey, Orange, NVIDIA, Kimi, DeepSeek, ChatGPT, Claude, Model Context Protocol (MCP), Slack, Microsoft Teams, GitHub, Interrupt London

## Video description Harrison Chase, Co-Founder and CEO of LangChain, explains what it means to own your intelligence, the specialization a team builds into the model, the harness, and the context, why he thinks agents need to be open, compounding, and governed to get there, and points to monday.com, Harvey, and Orange as examples of companies building the system around the model. He then walks through the LangSmith stack: Managed Deep Agents, which combines the Deep Agents harness and managed infrastructure into one, plus two agents LangChain built internally, Open SWE and its GTM Agent; observability on SmithDB, with Jev as a judge, RBAC on trace inputs and outputs, LangSmith Trajectories, the new AI Search, and LangSmith Custom Apps; and LangSmith Engine, where Engine v2 tests its own fixes on LangSmith Deployment and red teams your agent. He also introduces a self-report tool that agents can call to flag what went wrong while they run, and shares that Engine has scanned over 70 million traces and detected over 21,000 issues.

This session was captured at Interrupt London, the agent conference by LangChain.

Chapters: 0:00 Why the system around the LLM is what makes agents work 3:12 Why own your intelligence, and what does it take? 6:59 How to organize people, process, and technology for agents 9:56 What goes into a harness like Deep Agents? 12:04 How Managed Deep Agents runs your agent in the cloud 16:49 Inside LangChain's own coding and GTM agents 18:23 Why agent traces are a new type of data 19:57 How decision models like Jev make online evals cheaper 21:27 Storing sensitive traces, from BYOC to trajectories 23:55 Finding needles in a haystack with AI Search 26:01 Building your own trace views with Custom Apps 27:38 How LangSmith Engine finds issues and tests its own fixes 30:30 Letting agents self-report what went wrong 32:58 Why the future is thousands of vertical agents

Resources: → LangSmith Engine: https://www.langchain.com/langsmith/engine → Managed Deep Agents: https://docs.langchain.com/langsmith/python/managed-deep-agents-overview → AI Search in LangSmith: https://www.langchain.com/blog/agent-traces-ai-search-langsmith → LangSmith: https://www.langchain.com/langsmith → Interrupt, the agent conference by LangChain: https://interrupt.langchain.com

[Source](https://www.youtube.com/watch?v=XOUSuxY074U)

Reading summary

LangChain CEO Harrison Chase argues that enterprise generative AI utility relies on the architectural harness wrapping language models rather than raw models alone. To build defensible systems, organizations must own their intelligence by combining model optionality, governed cloud runtimes, and trace-driven evaluation platforms that autonomously diagnose and repair agent failures. Specializing Systems Beyond Foundation Models (3:12) Chase observes that generative AI has progressed from exploratory 2023 prototypes and simple 2024 chatbots to specialized production agents. Citing enterprise implementations at monday.com, Harvey, and Orange, he stresses that real-world utility depends on the harness rather than off-the-shelf foundation models. To navigate rising compute costs, the availability of open-weight models, and the need for proprietary differentiation, Chase contends that enterprises must own their intelligence across three pillars: openness (retaining model optionality and controllable context pipelines), compounding capability (improving performance iteratively via traces and memory), and governance (auditing identity, permissions, and runtime spend). Achieving this ownership requires cleanly separating proprietary business logic—the instructions, domain tools, and expert rules that organizations must provide themselves—from execution harnesses and infrastructure. To establish standard execution primitives, LangChain introduced Deep Agents, an open-source harness providing execution sandboxes, conversational summarization, prompt-caching context management, human-in-the-loop steering, and subagent orchestration. Multi-Tenant Cloud Infrastructure for Enterprise Agents (12:04) Rejecting desktop-bound local assistants as inadequate for enterprise operations, Chase forecasts that production agents must run in the cloud as always-on, proactive, multi-tenant background services. To remove the operational burden of stitching together disparate runtimes, gateways, and execution sandboxes, LangChain released Managed Deep Agents, enabling developers to deploy proprietary business logic to cloud environments via a single command (mda deploy). This managed architecture connects directly to corporate channels like Slack, Microsoft Teams, and GitHub while decoupling the core agent runtime from isolated execution sandboxes. The runtime treats code-execution sandboxes as callable tools behind a secrets proxy, preventing language models from directly accessing sensitive enterprise credentials. Within LangChain, this infrastructure powers Open SWE, an engineering agent that triages incidents and automates pull request reviews, alongside a proactive go-to-market agent that conducts prospect research and renders interactive sandbox visual artifacts. Trace-Driven Observability and Storage in SmithDB (18:23) Chase asserts that autonomous agent traces represent a fundamentally new telemetry primitive: complex execution trees filled with natural language signals, user intent, and subtle logic failures that traditional time-series logs cannot capture. To manage this data, LangChain built SmithDB as a dedicated storage engine within LangSmith, structured around four operational axes: ingestion, storage, querying, and visualization. At ingestion, LangSmith integrates decision models like Jev as online judges to score conversational turns against structured categories, evaluating performance faster and cheaper than generative frontier LLMs. For storage, LangSmith offers Bring Your Own Cloud (BYOC) deployment, role-based access control that hides sensitive raw prompts from engineering telemetry, and message-based "Trajectories" formatted for downstream fine-tuning. For retrieval, an "AI Search" pipeline streams candidate traces through real-time decision models to surface conversational errors without hardcoded regex rules. Users can also generate "vibe-coded" custom dashboard front ends using natural language prompts inside the governed workspace. Automated Agent Engineering with LangSmith Engine (27:38) Because manual prompt modifications frequently trigger unexpected regressions elsewhere, Chase characterizes simple trace visibility as table stakes; sustainable velocity requires automated agent engineering. LangSmith Engine operates over production trace streams to cluster failure patterns, diagnose root causes across prompts or retrieval pipelines, and submit automated pull requests with proposed code fixes. With Engine v2, the platform proactively verifies these fixes by spinning up isolated preview deployments within LangSmith Deployments to run regression tests before presenting patches to developers. It also conducts proactive red teaming, generating synthetic probing strategies to stress-test agent behavior and surface vulnerabilities before release. To shorten triage loops, agents can invoke in-flight self-reporting tools mid-execution to tag traces with diagnostic reasoning when encountering roadblocks. Chase reports that LangSmith Engine has scanned more than 70 million production traces and flagged over 21,000 distinct defects. Chase concludes that the future global economy will not rely on a single monolithic foundation model, but on thousands of vertical, domain-specific agents driving internal and external operations. Building these systems sustainably requires organizations to accelerate their build-test-deploy-monitor flywheels, continually embedding human expertise into model weights, runtime harnesses, and automated evaluations.

Source quotes

[12:04] Harrison Chase: I don't think the future is just agents running on your laptop just for you. I think it's agents running in the cloud, working in the background, proactive, always on, multi-tenant, serving many users. [0:06] Harrison Chase: The thing that makes all of these application layer agents really take off and actually work is everything around the LLM, the system around it. [3:12] Harrison Chase: We think agents will absolutely transform the operating model of companies out there, but you have to have governance over them. [27:38] Harrison Chase: The visibility over the traces at this point has become table stakes.

Original transcript

[0:06] Welcome to Interrupt, everyone. [0:08] It's great to be here in London. [0:14] Last year, we just did Interrupt in San Francisco. [0:17] This year we're doing it in San Francisco, New York, and London. [0:20] So it's really great to be here in person with all of you talking about agents [0:25] and evals and all that fun stuff. [0:28] We have a great lineup for you today. [0:30] We've got a ton of fantastic speakers. [0:32] Some fireside chats, some deep dives on a lot of the cool technology [0:36] that we're building. [0:37] And I wanted to kick off the day with just kind of like an overview of how [0:41] we think of the state of agents and AI and some of the directions that we're [0:46] pushing in. Before talking about the future, though, I want to talk a little [0:49] bit about how we got here. [0:51] And so I want to... [0:59] I want to figure out how the clicker works. [1:00] There we go. And so I want to start in the year, kind of, 2023. [1:04] And so this is when LangChain was first getting started. [1:07] A lot of LangChain was around helping people build prototypes of agents because [1:11] that's what everyone was thinking about at the time. [1:13] It was super new, super exploratory. [1:15] And then in 2024, we started to see applications come online, but they [1:18] were simpler types of applications. [1:20] They were a single LLM call, a chain of LLM calls, chatbots. [1:24] And in 2025, we started to see that people were exploring agents for the first [1:28] time. But then really this past year, I feel like this has been the year that [1:32] agents have come alive and in particular started to transform the application [1:37] layer. So we've seen a ton of agents being built for vertical use cases [1:43] for different points in the application layer. [1:46] And the key part about all of these agents is they're not just an LLM. [1:50] They're driven by an LLM, of course, that's at the center, but the thing that [1:53] makes all of these application layer agents really take off and actually work [1:58] is everything around the LLM, the system around it. [2:01] And so we've seen a number of companies basically build these systems, [2:05] these harnesses around these LLMs. [2:07] And that's what's causing agents to work. [2:08] And so for some examples of this, you'll hear later today from Monday about [2:12] how they built their agent, and they built kind of this continual feedback [2:16] loop with evals and started improving in that direction. [2:19] Harvey is a great example of a customer who's done this, where they are fine-tuning [2:24] models for specialized tasks. [2:25] So again, it's not just the model, it's the system around it, including this [2:29] system for fine-tuning models. [2:32] And then Orange, you'll hear from them as well later today about how they built [2:37] this specialized harness with governance in it. [2:40] And governance is a key part of what makes this agent work. [2:43] So we're really seeing that it's not just the model, it's everything around [2:46] the model. It's the harness with skills, tools, context, controls, [2:50] governance. And when we see these agents start to come into production [2:55] and really transform what's happening, it's really everything around the [3:00] model. And because of this, like, specialization that is happening, you're taking [3:04] this model and then you're building the system around it that makes it really [3:08] really work for a particular domain that you're an expert in. [3:12] This is why we think it's so important to own your intelligence, because [3:16] that specialization is what really makes these agents work. [3:20] And when we think of that specialization at the harness layer, [3:24] at the model layer, at the context layer, when we think of that specialization, [3:28] to us, that's what it really means to own your intelligence. [3:30] And we're seeing like a few trends that are driving this desire to own [3:35] one's intelligence and the importance of owning one's intelligence. [3:39] One is just cost. So we're seeing AI tools get adopted more than [3:43] ever now. Cost therefore is increasing a lot. [3:46] And so if you're not in control of this whole agentic system in some [3:49] form, then the costs will get out of control. [3:52] We're seeing open models start to become more and more prevalent. [3:55] And so NVIDIA, Kimi, DeepSeek, all of these have released really performant [3:59] open-source and open-weight models that you can use for cheaper, but also [4:04] that you can post-train and really make specialized. [4:07] And then the third thing that we're seeing people realize, and this [4:10] is, I think, the biggest thing that's driving these, like, vertical agents [4:13] that are really having impact, is the impact that data can have. [4:17] So we're seeing that when you're creating a kind of like vertical, differentiated [4:22] experience, the data that you bring, the context that you bring, the knowledge [4:26] that you bring to shape the harness, that's what really differentiates it from [4:31] someone just using ChatGPT or using Claude. [4:33] And so our mission at LangChain is to enable every company to own their intelligence. [4:38] We think this is what's needed to give birth to thousands of [4:42] vertical or company-specific harnesses. [4:45] And we really want to help make this happen. [4:48] So what are the core pillars of that? [4:50] In our mind, there's three. [4:52] The first is open. And so this refers to the whole agent apparatus itself. [4:57] And so again, the model, the harness, and the context. [5:00] We think all of these need to be open. [5:02] When we talk about a model being open, yes, there are open-weight models, but [5:06] more than that, we really just mean model optionality. [5:09] Having the option to choose which model to use, whichever one's best, whichever [5:12] one's fastest, cheapest, best for your domain, you should have model optionality. [5:16] The harness should be open and controllable. [5:19] So when you think about shaping these general-purpose LLMs for your domain, [5:23] a big part of that is making them do things in a particular way, and how [5:27] you enforce that is in the harness. [5:28] And so if the harness isn't controllable, and having it be open source [5:32] is a great way to have it be controllable, but if it isn't controllable, you're [5:35] not going to be able to pull those levers efficiently. [5:37] And then context as well. Context can come from anywhere. [5:40] It can come from pre-existing data sources. [5:43] It can come from APIs. It can come from memory that you build up. [5:46] And having a harness that works with any sources of context is really important [5:50] for being able to own that agent. [5:54] We think these agents, in order to own your intelligence, they also need to [5:57] be compounding. So what I mean by that is it's pretty easy to build an initial [6:01] version of an agent, but almost always that initial version is not what's production [6:05] ready enough to ship to millions of users. [6:08] And so how do you improve that agent over time? [6:10] This is where compounding comes in. [6:11] You want the intelligence of that agent to compound over time, whether it's [6:15] manually with observability and then noticing things and then fixing [6:19] those things, or whether it's automatically in the realm of kind [6:23] of like memory. [6:25] And then lastly, these agents need to be governed. [6:27] This is incredibly important for internal company agents when they're running. [6:32] Who do they act on behalf of? [6:33] How much is it costing to actually run these? [6:36] How do you know what's going on? [6:38] Having governance over these agents, and by the way, we think agents will [6:42] absolutely transform the operating model of companies out there, but you have [6:47] to have governance over them. [6:48] You have to know who they're acting on behalf of. [6:52] And so we think these three pillars, open, compounding, and governed, are really [6:57] the pillars of owning your intelligence. [6:59] And when we talk about, like, impacting this at a company, there's not [7:03] one thing that you can do to really make this change to owning your intelligence. [7:07] It's a combination of things. [7:07] And in particular, we think there's three different things. [7:10] People, process, technology. [7:11] When we talk about people and how you set a company up for success in terms [7:15] of organizing around the idea of owning your intelligence, we think there's [7:19] three different rings of personas that are involved in this process. [7:23] One is the core GenAI platform team. [7:26] These are agent experts. They're standing up the technology and oftentimes [7:30] processes for others in the company to benefit from and to use. [7:35] Second is all the agent engineers in different business units, and they're [7:40] the ones taking these technologies and really making them work for a [7:44] particular use case. And then lastly, there's non-technical [7:48] builders, non-developers who are absolutely involved in this agent building [7:53] process. They're oftentimes the best folks to know what good looks like for [7:57] a particular domain or for a particular problem. [7:59] And so when you think about owning your intelligence, it really [8:02] involves all three of these people. [8:04] You need people to set up the core infrastructure. [8:06] You need people to build the agents. [8:07] And then you need domain experts to really help codify what good or bad [8:12] looks like. We think there's certain processes that you're going to want to [8:16] implement as well. In particular, we think that building agents is very iterative. [8:20] So, as I mentioned, the first version of an agent that you build isn't going [8:23] to be the one that's going to serve millions of users successfully. [8:26] So you're going to want to spin up this life cycle and really get [8:30] this life cycle turning. You're going to want to set up a process where teams [8:33] build a version, but then they quickly test it, they quickly deploy it to a [8:37] small set of users, they monitor that, and then they take those learnings, [8:40] bring them back into build, and really spin this flywheel. [8:43] I think the best teams that we see shipping agents, they're the ones who [8:47] spin this flywheel as quickly as possible. [8:50] And then, of course, there's the technology that powers all of this. [8:53] We think there's kind of like two key parts to this technology that [8:57] we help people with. [8:59] One is the runtime. And so this is how you build and deploy agents. [9:03] This includes our open source libraries as well as our deployments. [9:07] And then there's observability and eval. [9:08] And we think this is core for that kind of like compounding loop. [9:12] And so these are pillars of the technology of what we build. [9:16] And recently we've been adding a new layer on top as well, which is intelligence. [9:19] So everything in the SDLC is getting automated. [9:22] And so should agent building. [9:24] And so yes, you want to have this flywheel turning, and we want to build a bunch [9:27] of tools to make it easy for you as agent engineers to turn that flywheel, [9:32] but we also want to build intelligence that helps do parts of this for you [9:36] as well. And so when we think of the stack that we're building, we [9:39] think of building a runtime. [9:41] We think of building this observability and evals and then intelligence on [9:44] top to really tie it all together. [9:46] And so for the rest of this keynote, I want to talk about these three buckets [9:49] and things that we're doing to push the frontier in all of them. [9:53] And so I want to start by talking about runtime. [9:56] So I said this before, but when we think about building agents, we think [10:00] about it in layers. And one layered way to view it is that when you build [10:04] agents, there's business logic that goes into a harness that runs on infrastructure. [10:09] So business logic includes things like instructions, tools, skills, maybe hooks [10:14] to modify some of the harness. [10:15] This is stuff that we can never give you. [10:17] This is domain-specific stuff that you all or your teammates are experts in [10:22] and are going to codify into, again, all of these different things to really [10:27] drive the agent in a particular domain or particular use case. [10:31] When we talk about the harness, we talk about all of the things around the [10:36] model that connect it to external sources of data and computation and context [10:42] and business logic. And so we spend a lot of time thinking about this. [10:45] What are the right primitives to run the model in and let it really action [10:50] on everything that's out there. [10:52] And so this is where our open source frameworks come into play. [10:54] And in particular, Deep Agents is one that we've released somewhat recently. [10:58] It's our newest one. And it's really focused on what this idea of a really [11:03] solid base harness looks like. [11:04] So it's intended for you to easily modify and easily build on top of, but it [11:09] has a lot of primitives that we think are common to how this harness or how [11:13] this execution gets done. So in particular, we give it an execution environment. [11:18] This can be a file system. [11:19] This can be a full-on sandbox. [11:21] This can be a lightweight code interpreter. [11:23] We do a lot of context management that we've seen just be best practice for [11:26] how to work with these models. [11:28] So we support skills, memory. [11:30] We do summarization when the conversation gets long. [11:33] We do context offloading for really large tool calls. [11:36] We architect it all in a way that takes advantage of prompt caching. [11:40] We add in kind of like ways to steer what the harness is doing. [11:44] So we add in human in the loop. [11:46] And then we also bake in things like planning and subagents so that you can [11:49] spin up small agents to do things. [11:51] And so all of these are aspects of a harness that we think are general, and [11:56] we provide this off the shelf in Deep Agents, although you can of course [11:59] build your own in a more granular way with LangGraph and LangChain. [12:04] And then lastly, you've got the infrastructure, and so I think, like, [12:08] this is maybe calling out something here. [12:10] I don't think the future is just agents running on your laptop just for you. [12:14] I think it's agents running in the cloud, working in the background, proactive, [12:18] always on, multi-tenant, serving many users. [12:21] And so when you think about going from agents that just run locally to [12:24] agents that run in the cloud, [12:25] there's a whole set of infrastructure that you need in order to make that happen. [12:28] You need a runtime for the agent, you need ways that it can connect to tools [12:33] in kind of like a properly auth'd and governed way. [12:36] You need sandboxes for it to write and execute code in. [12:39] And you need an LLM gateway to control how it can access models. [12:42] And so you've got all these pieces of infrastructure that are needed. [12:47] And we're building a lot of those into LangSmith. [12:49] So we have LangSmith Deployments. [12:50] This takes your agent, it bundles it up. [12:52] It deploys it into a server with streaming and secret storage and the checkpointing [12:56] and a task queue. We have a tool server which uses MCP as a protocol and [13:01] handles auth and has an audit log on it. [13:03] We've got sandboxes where you can write and execute code. [13:06] It's got a secrets proxy, so credentials never get in the sandbox, so the agent [13:10] can't kind of like see them. [13:11] And then we have an LLM gateway for reliability, with fallbacks [13:15] and cost controls on it. And so we have all these individual pieces in LangSmith, [13:20] and one of the things that we're doing to really make this building experience [13:24] faster for folks is bundling it all up into what we call Managed Deep Agents. [13:29] So Managed Deep Agents basically combines two of those layers into one. [13:33] It combines the harness layer and the managed infrastructure layer into one. [13:37] Means instead of... previously you might define your business logic, choose [13:40] your harness, and then choose your four or five different pieces of infrastructure, [13:44] now you just define your business logic and use Managed Deep Agents, and it's [13:47] all bundled in one. We think this makes it really really easy to go from your [13:50] business logic to something that's running in production on the web, multi-tenant, [13:56] in a scalable way. And so I want to highlight three different [14:00] kind of like aspects of Managed Deep Agents that we think are really impactful [14:04] for getting time to value. [14:06] First, we think that it has a lot of these things built in that help you build [14:09] fast. So when you're building fast for a multi-tenant agent, a lot of work, [14:13] yes, goes into the context engineering side of things, but a lot of work also [14:17] goes into how you connect to things safely. [14:20] And so tools, channels, and then the auth associated with those are [14:26] really important and really painstaking to set up. [14:28] And so by channels, I mean how the agent is triggered. [14:30] Is it triggered in Slack? Is it triggered in Teams? [14:32] Is it listening to new emails? [14:34] Is it listening to new GitHub events? [14:35] And so we take all of those in Managed Deep Agents. [14:39] We provide off-the-shelf integrations that also handle auth for those. [14:43] It's also very easy to deploy this outside of your computer. [14:46] So again, it runs in the cloud. [14:49] We've separated the agent runtime from the sandbox. [14:53] And so there's two different patterns of kind of like running agents that require [14:56] kind of like sandboxing in production. [14:58] One is you run them in the sandbox. [15:00] We think a better pattern is actually to run the agent outside the sandbox [15:03] and connect to the sandbox as a tool. [15:05] And so we do that, and we bundled it all up into a single kind of like CLI command, [15:10] mda deploy. [15:12] And then lastly, there's a lot of things that make it easy to govern these [15:14] agents at scale. We've seen these agents be really really useful for internal [15:18] company agents to transform what the companies look like internally. [15:21] When you do this, you need to have a lot of governance around those agents. [15:25] So who is this agent acting as? [15:27] Maybe it's acting as itself. [15:28] Maybe it's acting on behalf of whoever is using it. [15:31] What permissions does it have? [15:32] What tools can it call into? [15:34] What models can it use? How much can this agent spend? [15:37] Data that comes into it. [15:39] How is that being protected? [15:40] Tracing and eval is really important for knowing what's going on and then measuring [15:43] the impact of this agent. [15:45] And so all of these are bundled into Managed Deep Agents, and we think make [15:49] it pretty fast and uniquely differentiated in terms of what it takes [15:53] to get started building agents. [15:54] This is a little diagram of everything that's kind of like involved. [15:59] And so again, we think people are going to be building multi-tenant [16:02] applications that interact with the agent. [16:04] We think this agent is going to be exposed through a bunch of different interfaces, [16:07] UI, Slack, Teams, GitHub. It's going to be running on its own deployment, but [16:11] it's going to be connected to sandboxes. [16:13] Access to models is going to be running through a gateway. [16:15] It's going to be tracing to an observability platform that's going [16:20] to be powering evals. And then memory is going to need to be stored somewhere [16:23] as well. And so we try to bundle all of this up into Managed Deep Agents. [16:27] As I mentioned, we think this is really impactful for internal agents that [16:32] are transforming what companies look like. [16:34] We think that there'll be a lot of these. [16:36] We think that you'll have a lot of business users inside the company [16:40] being the best people to build these. [16:42] And so we want to make that as easy as possible. [16:44] And again, we think this needs to be really governed. [16:47] And so we think that we can provide that in this managed agent harness. [16:49] These are a few examples of things that we've built internally. [16:52] The two biggest ones that I'll call out are Open SWE, which is a coding agent. [16:56] So this isn't just helping write code. [16:59] It's also helping review code, triage incidents, and in the future [17:04] do things like QA. This actually is open source. [17:07] The second one that we use a lot internally is GTM Agent. [17:10] This is not open source yet, but this powers a lot of our go-to-market operations [17:14] from marketing and sales and outreach. [17:16] It's a proactive agent, so it's always on. [17:18] It's pinging you with updates. [17:21] Pinged me to tell me that I had a big conference today. [17:24] So thank you very much for that. [17:25] And it empowers a lot of what our go-to-market team uses and [17:29] does. And so, diving into that a little bit more. [17:32] Some of the things it can do are research. [17:34] It can do outreach across a bunch of different domains. [17:36] And again, this is why connectors are so important. [17:39] It can create artifacts. And so these are created in the sandbox. [17:42] And then in the sandbox, we expose URLs where you can view them. [17:45] And again, it's very much kind of like an autonomous [17:49] agent in that it's running not just when you message it, but on all of these [17:53] kind of like automations. [17:56] And so this is a little bit of like roughly what it looks like. [17:59] Again, lots of events coming in, the core agent runtime. [18:02] Governance is a key piece and then connections into external systems. [18:06] And so this type of internal company agent that's deeply integrated knows [18:11] a lot of your business context. [18:13] We think this is a glimpse into the future of what future companies [18:18] will look like when they're running, and we think Managed Deep Agents will [18:21] power a lot of that. [18:23] The second big thing I want to talk about is observability and eval. [18:26] So we think this is really core for spinning this loop. [18:29] We think this loop is really important to get running, and we think traces sit [18:32] at the center of this, and so we invest a lot in traces and thinking about [18:36] what they look like, and we think agent traces are a new type of data. [18:39] They're deeply nested. You've got really large payloads. [18:42] You've got different query patterns than before, and you've got a bunch of different [18:46] modalities. And we also think they're valuable in different ways. [18:50] We think these traces contain natural language, and that natural language contains [18:54] things like user intent and logic failures and glimpses into how people [18:58] are using the agent, whether it's properly or improperly. [19:01] And so LangSmith, of course, is our platform really built around this idea of [19:05] agent traces and agent observability and eval. [19:08] At the core of LangSmith is SmithDB. [19:10] SmithDB is a database that we built internally to be really good at handling [19:14] agent traces. It's really performant across a bunch of different dimensions. [19:18] And so all of these are different query patterns that we think are relevant [19:22] for dealing with traces. [19:24] And when we think about SmithDB and therefore LangSmith as a platform, [19:27] we want to make it the best agent platform, the best data lake for these [19:32] agent traces. And so we think about how agent traces are different and [19:36] we think about the different dimensions that you're going to want to ingest [19:39] traces on and store traces on and query traces on and visualize traces, like [19:43] how is doing all of these things different for agent traces compared to normal [19:48] observability or normal data. [19:49] We spend a ton of time thinking about that, and so I want to touch on these [19:52] four axes because this is a big part of where we're innovating in LangSmith. [19:57] And so the first is ingest. [19:58] When you ingest agent traces, they contain a bunch of these natural language [20:04] clues that you can use to tell how the system is behaving, what people are [20:08] using it for, how the agent's responding. [20:10] All of these can be encoded in kind of like online evals that you use LLM or [20:14] code to judge. And recently there's been a really big improvement to [20:19] how you can do this. So previously you would use frontier models for a lot [20:22] of these judges. But recently Jev and other decision models have kind of [20:26] like emerged as a way to do this more cheaply and more efficiently. [20:29] So what is Jev? Jev is the decision model. [20:31] What does that mean? It basically means that instead of generating text, it [20:35] makes decisions. So you pass in the state and then you pass in a list of questions [20:39] with different answers to those questions, and the model basically chooses [20:42] the answers, and this makes it really really fast and really really cheap. [20:47] And so when you use Jev as a judge, which is a new option that you can use [20:51] in LangSmith, you can build quickly. [20:54] So it has, like, these standard kind of like answer types that are pretty intuitive [20:58] to choose from. You can provide different choices and you can provide different [21:01] scores for it to kind of like judge. [21:03] So there's a variety of different dimensions that you can use Jev to score [21:06] on. And so this is available in LangSmith today. [21:09] We think this is a great example of how the ingestion of agent data is different [21:14] from kind of like traditional logs. [21:16] There are so many clues in the natural language of these agent traces that [21:20] you want to score, and things like decision models we think are a great step [21:23] forward in doing that reliably. [21:27] Next, I want to talk about storing the data. [21:29] So SmithDB is where we store the data. [21:31] One big difference between kind of like agent traces and traditional logs is [21:35] that there's a ton of sensitive data that is in these traces. [21:39] So we think a lot about how to best store these traces to deal with [21:43] the fact that there's so much sensitive data there. [21:45] So we have very flexible hosting of SmithDB and LangSmith. [21:48] So we run it in the cloud, obviously, but you can also run it on-prem or in a [21:52] new model we have called Bring Your Own Cloud, BYOC, where all the data remains [21:56] with you but the control plane lives with us. [21:58] One of the things that we've added recently is RBAC on inputs or outputs. [22:03] So what does this mean? You've got an agent trace. [22:05] You might want engineers to see the skeleton of the trace, the kind [22:10] of like tools that were called, the tokens, the latency, so they can see overall [22:14] system health, but you don't want them to see the inputs or outputs because [22:17] that's where sensitive data could be. [22:19] And so we added a new RBAC permission where you can control whether people [22:22] can see those inputs or outputs or not. [22:24] And then coming up soon as well is more granular ABAC permissions. [22:28] So what this will mean is that you can tag traces with different [22:32] metadata and then you can set kind of like policies where only certain people [22:36] or certain groups of people can see those traces. [22:38] This will basically give you much more fine-grained access to who can see which [22:43] types of traces. This type of, like, RBAC and ABAC is not usually something [22:48] that I see people getting excited about, but it is so key for dealing with this [22:51] sensitive information that's in all these traces. [22:55] We also think about the right format in which to store these traces. [22:58] And so one of the things that we launched recently and improving upon is [23:02] LangSmith Trajectories. What is LangSmith Trajectories? [23:05] So LangSmith Trajectories is a new format for storing traces. [23:09] We visualize it in this very kind of like user-friendly way. [23:13] You can still click into things and click around and actually click into things [23:17] even more and see all the raw details. [23:19] But by default we present it in this kind of like list of messages [23:24] way. So trajectories are basically a list of messages, and what that does is [23:29] it gives you a standardized format to view traces and also interact with traces. [23:33] So the standardized format makes it much faster to debug because you're [23:36] now viewing things in kind of like a very simple standardized way. [23:39] It makes it simpler to annotate. [23:40] So you can bring subject matter experts into the platform. [23:43] This is now a much more user-friendly way for them to view that, and they can [23:46] annotate there. And then this is also the same format, or it's a very similar [23:49] format to what's used in post-training of models. [23:51] So we think it makes it ready for that. [23:55] Next I want to talk about querying. [23:56] So this data is kind of like a new... or it's very text-based. [24:01] There's a lot of text in it. [24:02] There's a lot of multimodal data in it. [24:04] And when we think about kind of like searching for data within these traces, [24:09] it presents new challenges compared to traditional [24:12] data. So you're... [24:15] All of this data contains how people are using your agent, and most of them [24:20] are probably using it in standard ways, but there's probably a few that are [24:23] using it in, like, weird ways. [24:25] And so it's really about finding these needles in a haystack where the way [24:29] that they're using it, it's not going to show up in kind of like logs. [24:32] It's going to show up in the semantics of how people are using it, where the [24:35] errors that happen aren't going to show up in logs. [24:37] They're going to happen... [24:38] They're going to show up in the semantics. [24:40] And so we've added different ways to kind of like search [24:43] this data to date. So we have full text search, we have ways of basically filtering [24:47] on metadata and filtering on these large payloads, all of which are kind [24:51] of like unique problems for agent data. [24:54] But I'm really excited about this new way of filtering, which we're calling [24:59] kind of like AI Search or AI Query. [25:01] And so what this does, and let me flip to a video of this. [25:05] So what this does is you can enter in a natural language query right there. [25:09] So AI Search, you can type in something in natural language that you want to [25:13] see. So, user got confused by agent response, and then under the hood what we're [25:17] doing is we're streaming back results, and in real time we're running them through [25:21] a decision model. So I talked about these decision models, but we're running [25:24] it through a decision model and giving it a confidence of how it meets [25:28] that option. We're actually highlighting the places in here where there's evidence [25:31] for this. So again, this is not transforming natural language into a [25:37] structured query. It's actually taking that prompt and running it as a prompt [25:41] over all... You can pair this with other filters, but it's running it [25:45] very quickly and efficiently over all results that come back. [25:48] And so this is an example of how we're embedding more AI into the platform [25:53] to help with the analysis of this special type of data, this special type of [25:56] agent trace data. And then lastly, I want to talk about visualization of this data. [26:01] So again, this is a new type of data. [26:02] How do we think about visualizing it? [26:03] We've already seen a little bit of it with this trajectory [26:07] view. But one of the things that we've consistently heard is that because [26:11] agents operate in so many different domains and there might be different ways [26:15] of viewing or interacting with this data in different domains, you also [26:19] need different ways to review it in LangSmith. [26:21] And so that's why we launched LangSmith Custom Apps. [26:25] And I think we have a video here as well, but you can see this is a completely [26:28] vibe-coded, completely custom HTML front end. [26:32] You can edit it in the UI by going into this vibe coding app, giving [26:37] it a prompt. And so here, typing in how you want it to change in natural [26:41] language. And you can see that it's doing some work on the right. [26:44] We've got an agent in the back end. [26:46] It's exploring files. It's trying to figure out what to do. [26:49] And now it adds in this other part of the UI, which is visualized [26:53] on the right. This is then saved as a view that anyone in the organization [26:57] can come in and use. And so we've consistently heard that people [27:02] have specific ways that they want to view legal data or finance data, and they [27:07] might be vibe coding UIs outside of LangSmith, and that's fine. [27:11] But we think by bringing it into the UI, we can let people [27:16] do this in a more governed way, where it knows all of the different [27:19] APIs. We actually open sourced our design system, so it will look consistent [27:23] with the other parts of LangSmith. [27:25] And then it just exists in your workspace as a new page that you can [27:28] go in and look at. [27:30] And so these are all ways that we think about making LangSmith the best [27:34] kind of like data platform for trace data. [27:38] And then lastly, I want to talk about intelligence. [27:40] This is one of the key parts that we are working on in order to [27:44] help people get more value out of everything in LangSmith because, yes, great, [27:48] there's all this information that's in traces, but how do you action on it? [27:51] The visibility over the traces at this point has become table stakes. [27:55] Finding issues is really hard. [27:58] There's, again, it's finding needles in a haystack. [28:01] Understanding issues is hard. [28:02] You've got these large traces. [28:04] Yes, we're condensing it into trajectories. [28:05] And so you can kind of see how things kind of like come together here. [28:08] But it's still kind of like understanding issues is hard, and then fixing [28:12] issues is hard. And so great, there's some wrong output, but how do you know [28:15] whether it's going wrong in the prompt or the context or the model or the retrieval? [28:19] And then once you fix it, how do you avoid playing [28:23] whack-a-mole, where you kind of fix it once, but then it pops up again [28:26] in two months because you change the prompt in a different way. [28:29] And so when we think of trying to do this, you do this in a manual [28:34] way by spinning this flywheel. [28:36] But we launched LangSmith Engine a few months ago as a way to do this [28:40] in a more automated way. And so LangSmith Engine we think of as a proactive [28:44] agent engineer. It runs in the background on top of traces. [28:47] It monitors these issues. It clusters them and diagnoses them. [28:50] It proposes fixes via a PR. [28:53] It then creates other tests, so it can create data points and create evals, [28:58] and then if it's hooked into deployments, it can actually trigger a [29:00] deployment from there as well. [29:02] And so we're constantly working on this, and so Engine v2 we launched about [29:07] a week ago, and it has a bunch of different improvements and new capabilities. [29:12] So one of the new capabilities is that it will actually test its [29:17] fixes proactively. So this only works if you're running on LangSmith [29:20] Deployments. But one of the things that we do in LangSmith Deployments is we [29:23] take the changes that we make. [29:25] We then basically spin up a new preview deployment and then test that [29:30] preview deployment to see if the issues that we found previously now no longer [29:33] occur. So we can basically present evidence that these do in fact [29:37] fix the issues. Red teaming is another new capability that we launched. [29:42] Basically, here this is different. [29:44] So it's not just running on traces, it's actually coming up with new probes, [29:48] sending them into the system and seeing how it responds. [29:51] And so we have a little video here where you can see when you kick [29:55] off a red teaming thing, it starts running that in progress, and behind [29:59] the scenes, it has a few different strategies that it can use to basically [30:02] come up with issues. It will find a bunch of issues. [30:05] And then it provides different evidence for what happened. [30:08] So that's the prompt that's sent in. [30:10] This is the observed output, and then that's what we expected to happen there. [30:13] There's then basically why the judge decided this, what to do next. [30:17] You can click in and you can actually view the trace. [30:20] So you can view the trace where this happened, and really, and we'll [30:23] show you that these are in fact issues that can happen. [30:26] This is proactively kind of like stress testing your agent for places where it may mess up. [30:30] We've also realized that agents as they're running may [30:37] know when they run into issues. [30:39] And so rather than rediscovering it later, we want to give agents a way to [30:43] self-report what's going wrong. [30:45] And so this is a new thing that we're actually launching today, where basically [30:49] we have this self-report kind of like tool or endpoint that the agent can call [30:54] as it's running. It calls it with some annotation of, like, something it ran [30:59] into and some reasoning for what went wrong. That gets associated with the trace [31:04] in the background, and then when LangSmith Engine runs later that night, it will [31:08] pick up that logic, and so it picks up the logic that the agent realized as [31:12] it was running. We think this is really really useful because agents can [31:15] oftentimes recognize as they're running where things are going wrong. [31:19] And of course we're also working to make Engine cheaper, faster, better. [31:23] And so we've made it run a lot cheaper. [31:26] We've also made it better at finding fixes and better at fixing things as [31:30] well. And we benchmark this against other coding benchmarks. [31:33] And we know that we have a lot of self-hosted customers here in [31:38] the audience today. And so we also launched in the latest self-hosted [31:42] release, v7, bring your own key, which is now available for Engine self-hosted. [31:47] We are very very bullish on Engine. [31:50] We've seen it be used tremendously already so far, with over 70 million [31:55] traces scanned and detected over 21,000 issues. [31:58] And it's used by a lot of the fastest growing companies to really help them [32:01] spin that flywheel, because I think a lot of the speed at which you can build [32:05] agents is really rate limited by how quickly you can spin that flywheel. [32:08] And so we want to help folks do that. [32:10] And so that's roughly how we think about the stack of what we're building [32:15] and the improvements upon which we're making. [32:19] So we've got the runtime, and the big focus there is basically we [32:23] think this idea of agents running in the cloud, not on your laptop, in [32:28] a multi-tenant and multiplayer way is where the future's going, connecting [32:32] to different services, having some semblance of identity and auth, and so we want [32:36] to bake that all into Managed Deep Agents. [32:38] We really think about observability and eval as trying to be the best data [32:43] platform for agent traces. [32:44] And so we spend a ton of time thinking about how these agent traces have different [32:48] requirements on ingest, query, storing, and visualizing them, and building the [32:52] platform around that. And then we also want to build intelligence and have [32:56] that help drive a lot of things as well. [32:58] And again, I think it's important to recognize why we're building [33:02] these things and our guiding north star. [33:05] The thing that drives us is really to help people own their intelligence, as [33:09] I 100% think the future is thousands of vertical, domain-specific agents [33:16] both externally and inside one's companies that are really driving change [33:21] and improved kind of like processes and new value. [33:24] And so I think being able to own your intelligence is the only way to [33:29] really build these reliably. [33:30] You have to take what is your proprietary context or knowledge or skills and [33:35] embed that into the agent in some form, whether it's in the model weights or [33:39] the harness or the context, and so that's really a lot of what we think about. [33:44] And I think a lot of the speakers that you'll hear from today will also [33:48] echo this in some form, where they'll talk about evals and how they built those [33:52] to capture what good looks like, where they talk about the harness engineering [33:55] tips and tricks they do to make it really good for a domain, and so this [34:00] is the kind of like north star that guides us. [34:02] This is what we work on. We would love feedback on all of this. [34:06] And again, a big way that we get direction of where to [34:11] go is from all of you here sharing what's kind of like hard for you and where [34:14] you guys want to go and what's important for you. [34:15] So please keep the feedback going, keep the conversations with our team going, [34:19] and most importantly, thank you all for being here for our first ever [34:23] London Interrupt conference.