LangChain
Enterprise utility depends on proprietary context and execution harnesses rather than foundation models, requiring businesses to operate specialized cloud architectures to prevent runaway API costs.
AI-assisted summary of Why Own Your Intelligence? Harrison Chase Keynote | Interrupt London — original source. Check the original for full context.
Source published:
Key ideas
- Enterprise utility in generative AI is shifting from raw foundation models to the surrounding harness—integrating skills, context retrieval, execution sandboxes, and governance guardrails. (0:06)
- LangChain introduced Managed Deep Agents, unifying its open-source Deep Agents harness with cloud infrastructure to run multi-tenant, cloud-based background agents instead of local desktop apps. (12:04)
- LangSmith Engine v2 introduces autonomous fix verification via preview deployments and active red teaming, having already scanned over 70 million traces and detected more than 21,000 issues. (28:53)
- LangSmith now incorporates decision models like Jev for high-speed, low-cost ingest-time evaluations and real-time semantic trace exploration via AI Search. (19:18)
Detailed notes
## Background Published by LangChain on YouTube on October 30, 2026, this keynote features Harrison Chase, Co-Founder and CEO of LangChain, speaking at the Interrupt London conference. According to the page description, Chase explains what it means for companies to own their intelligence across the model, harness, and context, walking through the LangSmith product suite including Managed Deep Agents, SmithDB observability, and LangSmith Engine v2.
## TL;DR Harrison Chase outlines the architectural evolution of generative AI from early prototypes to specialized, production-ready vertical agents, arguing that real defensibility lies in the system built around the LLM rather than the raw model alone. To help enterprises 'own their intelligence,' he articulates three pillars—openness, compounding capability, and governance—backed by an operational flywheel aligning platform teams, engineers, and domain experts. Chase details the LangChain stack designed to support this paradigm, introducing Managed Deep Agents for cloud runtime execution, SmithDB and LangSmith Trajectories for trace observability and semantic AI search, and LangSmith Engine v2 for autonomous bug diagnosis, proactive fix verification, and red teaming.
## Atmosphere & style The presentation carries the polished, high-energy tone of an enterprise tech keynote, blending strategic visionary framing with rapid-fire architectural deep dives. Chase speaks with pragmatic confidence and technical fluency, steering clear of empty hype to focus on concrete operational hurdles like runaway API costs, messy trace telemetry, and the 'whack-a-mole' cycle of prompt debugging. The atmosphere feels purposeful and product-driven, highlighted by live UI demonstrations, architectural flowcharts, and tangible internal case studies that ground LangChain's ambitious enterprise vision in everyday developer realities.
## Bulletin - [00:06](https://www.youtube.com/watch?v=XOUSuxY074U&t=6s) Enterprise utility in generative AI is shifting from raw foundation models to the surrounding harness—integrating skills, context retrieval, execution sandboxes, and governance guardrails. - [12:04](https://www.youtube.com/watch?v=XOUSuxY074U&t=724s) LangChain introduced Managed Deep Agents, unifying its open-source Deep Agents harness with cloud infrastructure to run multi-tenant, cloud-based background agents instead of local desktop apps. - [28:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=1733s) LangSmith Engine v2 introduces autonomous fix verification via preview deployments and active red teaming, having already scanned over 70 million traces and detected more than 21,000 issues. - [19:18](https://www.youtube.com/watch?v=XOUSuxY074U&t=1158s) LangSmith now incorporates decision models like Jev for high-speed, low-cost ingest-time evaluations and real-time semantic trace exploration via AI Search.
## Key takeaways - [00:06](https://www.youtube.com/watch?v=XOUSuxY074U&t=6s) Production agent success depends on the architecture wrapping the LLM rather than the foundation model alone. - [03:12](https://www.youtube.com/watch?v=XOUSuxY074U&t=192s) Owning your intelligence requires specialization across the model, the harness, and proprietary context, driven by open architectures, compounding capability, and governance. - [06:59](https://www.youtube.com/watch?v=XOUSuxY074U&t=419s) Scaling enterprise agents demands cross-functional collaboration among platform teams, business-unit engineers, and non-technical domain experts spinning a rapid build-test-deploy-monitor flywheel. - [09:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=593s) Deep Agents provides an open-source execution harness featuring sandboxes, context offloading, prompt caching optimization, and subagent orchestration. - [12:04](https://www.youtube.com/watch?v=XOUSuxY074U&t=724s) Managed Deep Agents executes agents in cloud environments with built-in channels, secrets proxies, and Model Context Protocol (MCP) governance rather than running locally on laptops. - [18:23](https://www.youtube.com/watch?v=XOUSuxY074U&t=1103s) Agent traces constitute an entirely new class of telemetry data requiring SmithDB, a specialized data store optimized for complex execution graphs and natural language reasoning paths. - [19:18](https://www.youtube.com/watch?v=XOUSuxY074U&t=1158s) Decision models like Jev replace expensive text generation with structured classification, enabling cheaper and faster online evaluations during trace ingestion. - [21:27](https://www.youtube.com/watch?v=XOUSuxY074U&t=1287s) LangSmith introduced granular input/output RBAC, upcoming ABAC, and BYOC hosting to protect sensitive enterprise data while keeping execution graphs visible to engineers. - [23:55](https://www.youtube.com/watch?v=XOUSuxY074U&t=1435s) AI Search uses streaming decision models to discover subtle semantic failures and user confusion across production traces without relying on rigid SQL queries or error codes. - [26:01](https://www.youtube.com/watch?v=XOUSuxY074U&t=1561s) Custom Apps allow teams to vibe-code domain-specific dashboard visualizations directly inside LangSmith using natural language and an open-sourced design system. - [27:38](https://www.youtube.com/watch?v=XOUSuxY074U&t=1658s) LangSmith Engine automates the agent engineering lifecycle by diagnosing root causes, clustering failure modes, and opening automated pull requests with proposed fixes. - [28:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=1733s) Engine v2 verifies its own proposed code fixes in isolated preview deployments and uses proactive red teaming to stress-test agent boundaries before production failures occur. - [30:30](https://www.youtube.com/watch?v=XOUSuxY074U&t=1830s) Agents can self-report in-flight failures via dedicated tool calls, attaching runtime annotations and reasoning directly to trace metadata for batch analysis. - [32:10](https://www.youtube.com/watch?v=XOUSuxY074U&t=1930s) The enterprise future will be powered by thousands of vertical, domain-specific agents rather than a single monolithic model.
## Detailed walkthrough ### [00:06](https://www.youtube.com/watch?v=XOUSuxY074U&t=6s) The Evolution of Agents and the System Around the LLM
Harrison Chase opens Interrupt London by welcoming the crowd, noting how the event series has expanded. While the inaugural Interrupt conference was held exclusively in San Francisco, the conference series now spans San Francisco, New York, and London. To set the stage for LangChain's future direction, Chase reflects on the historical trajectory of generative AI over the preceding years. In 2023, when LangChain was first gaining traction, the ecosystem centered on building exploratory prototypes because the domain was brand-new. By 2024, production applications began appearing, though they remained relatively simple: single large language model (LLM) calls, basic chains, and conversational chatbots. In 2025, teams began venturing into agents for the first time.
```mermaid flowchart LR accTitle: Evolution of GenAI Applications accDescr: Timeline from 2023 exploratory prototypes to specialized production agents in 2026. A["2023: Prototypes<br/>Exploratory concepts"] --> B["2024: Simple Apps<br/>Chains & chatbots"] B --> C["2025: Early Agents<br/>First agent explorations"] C --> D["2026: Production Agents<br/>Vertical domain harnesses"] ```
Chase highlights that agents have finally come alive, profoundly restructuring the application layer across various vertical use cases. A critical realization has emerged across the industry: the model itself is not enough. While an LLM sits at the core driving reasoning, real enterprise utility comes from the architecture wrapping it. As Chase explains, “The thing that makes all of these application layer agents really take off and actually work is everything around the LLM, the system around it.” He illustrates this paradigm through real-world enterprise implementations featured at the conference:
* **monday.com**: Architected a closed, continual feedback loop using automated evaluations to iterate and systematically improve agent performance. * **Harvey**: Rather than relying strictly on off-the-shelf foundation models, they engineered specialized systems designed to fine-tune models specifically for legal and professional tasks. * **Orange**: Constructed an enterprise-grade agent harness anchored heavily around governance and policy guardrails to make execution safe and reliable.
Across all these deployments, production success hinges on the harness—the ecosystem providing skills, tool integrations, context retrieval, execution controls, and governance policies.
---
[03:12](https://www.youtube.com/watch?v=XOUSuxY074U&t=192s) Why Own Your Intelligence: The Three Pillars
Because generic models require deep domain customization to be useful, organizations face a strategic imperative: they must own their intelligence. Specialization across the model layer, the harness layer, and the context layer defines what it means to retain ownership. Chase outlines three macroeconomic and technical tailwinds accelerating this demand:
1. **Surging Compute and API Costs**: Broad enterprise adoption drives usage volumes rapidly upward; without architectural control over the underlying agentic systems, costs inevitably balloon out of hand. 2. **Rise of Open Models**: High-performing open-source and open-weight models from providers like NVIDIA, Kimi, and DeepSeek offer competitive unit economics, but more importantly, give teams the freedom to post-train and optimize weights for domain-specific tasks. 3. **The Differentiating Power of Data**: Generic models like ChatGPT or Claude cannot match domain-specific tools without proprietary context. The internal data, operational APIs, and domain-specific knowledge embedded into the harness are what create actual defensibility.
```mermaid flowchart TD accTitle: Three Pillars of Owning Your Intelligence accDescr: Openness, compounding capability, and governance form the foundation of intelligence ownership. subgraph Pillars["Core Pillars of Intelligence Ownership"] P1["<b>Open</b><br/>• Model optionality<br/>• Controllable harness<br/>• Flexible context sources"] P2["<b>Compounding</b><br/>• Manual evals & traces<br/>• Observability feedback<br/>• Automatic memory"] P3["<b>Governed</b><br/>• Identity & authorization<br/>• Traceable acting authority<br/>• Cost & behavioral guardrails"] end ```
To support this shift, Chase articulates LangChain's guiding objective: “Our mission at LangChain is to enable every company to own their intelligence.” He breaks this down into three foundational pillars:
* **Open**: Spans the entire apparatus. Rather than just referring to open-weight models, Chase clarifies that open primarily means total model optionality—retaining the flexibility to swap in the fastest, cheapest, or highest-performing engine for a specific job. It also means utilizing an open, controllable harness to direct agent behavior, as well as maintaining open context pipelines that pull data from internal data stores, APIs, and long-term memory. * **Compounding**: Initial prototypes never ship directly to millions of users; agents must compound capability over time. As Chase observes, “You want the intelligence of that agent to compound over time, whether it's manually with observability and then noticing things and then fixing those things, or whether it's automatically in the realm of kind of like memory.” * **Governed**: Critical for internal enterprise utility. Organizations must know exactly who an agent is acting on behalf of, track operational expenses, audit its reasoning paths, and maintain runtime guardrails. Chase notes, “We think agents will absolutely transform the operating model of companies out there, but you have to have governance over them.”
---
[06:59](https://www.youtube.com/watch?v=XOUSuxY074U&t=419s) Organizing People, Process, and Technology
Transforming a company to own its intelligence requires aligning organizational structure, operational methodology, and technical infrastructure. Chase organizes these elements across three categories: People, Process, and Technology.
```mermaid flowchart TD accTitle: Operating Model for Agent Development accDescr: The interaction between People, Process, and the Technology Stack. subgraph People["People"] direction TB P_Plat["Core GenAI Platform Team"] P_Eng["BU Agent Engineers"] P_SME["Domain Experts / Non-Developers"] P_Plat --- P_Eng --- P_SME end
subgraph Process["Process Flywheel"] direction TB PR_B["Build Agent"] --> PR_T["Test & Deploy"] PR_T --> PR_M["Monitor Trajectories"] PR_M --> PR_B end
subgraph Tech["LangChain Technology Stack"] direction TB T_Int["Intelligence Layer<br/>(Automated SDLC & Red Teaming)"] T_Obs["Observability & Evals<br/>(Compounding Feedback Loop)"] T_Run["Agent Runtime<br/>(Open-source Harness & Cloud Deployments)"] T_Int --- T_Obs --- T_Run end ```
For **People**, successful organizations organize agent development across three concentric personas: 1. *Core GenAI Platform Teams*: Dedicated agent experts who stand up foundational infrastructure, standard tooling, and scalable workflows for the broader business. 2. *Business Unit Agent Engineers*: Builders who sit close to operational problems, using platform tools to construct specialized, production-ready agents for distinct business lines. 3. *Non-Technical Builders and Domain Experts*: Non-developer subject-matter experts who understand operational edge cases and serve as the arbiters of what constitutes high-quality outputs.
On the **Process** side, Chase emphasizes that agent creation is non-linear and demands a rapid build-test-deploy-monitor loop. He notes:
> “The best teams that we see shipping agents, they're the ones who spin this flywheel as quickly as possible.”
Finally, Chase maps these operational demands to the **Technology** stack LangChain is constructing, which consists of three distinct layers: - **Agent Runtime**: Comprising LangChain's open-source libraries and managed deployment environments designed to execute agent harnesses reliably. - **Observability and Evaluations**: The feedback backbone that ingests execution traces to drive compounding performance gains. - **Intelligence Layer**: A new tier deployed on top of the stack. Just as traditional software development lifecycles (SDLC) are being automated, the agent engineering workflow itself must be automated by systems that detect runtime defects, run self-evaluations, and refine agent architectures autonomously.
[09:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=593s) The Three Layers of Agent Architecture and the Deep Agents Harness
Harrison Chase shifts his focus from broad strategy to the runtime environment, reiterating that building agents requires looking at the system in three distinct layers: business logic, the harness, and infrastructure. Business logic encompasses the domain-specific instructions, custom tools, tailored skills, and hooks used to modify the agent. He highlights that this layer is inherently proprietary to each organization: “This is stuff that we can never give you. This is domain-specific stuff that you all or your teammates are experts in and are going to codify into, again, all of these different things to really drive the agent in a particular domain or particular use case.”
```mermaid flowchart TD accTitle: Three Layers of Agent Architecture accDescr: Diagram showing Business Logic running inside the Deep Agents Harness, supported by Cloud Infrastructure.
subgraph Logic["Business Logic (Customer-Owned)"] BL["Instructions, Tools, Skills & Custom Hooks"] end
subgraph Harness["Deep Agents Harness (Execution Layer)"] direction TB EXEC["Execution Environment (Filesystem / Sandbox / Code Interpreter)"] CTX["Context Management (Summarization, Offloading, Prompt Caching)"] CTRL["Steering & Control (Human-in-the-Loop, Planning, Subagents)"] end
subgraph Infra["Managed Infrastructure"] INF["LangSmith Deployments, Tool Servers, Gateways & Sandboxes"] end
BL --> Harness Harness --> Infra ```
The harness layer encompasses everything wrapped around the model to connect it directly to external data sources, compute, context, and business rules. To streamline this layer, LangChain released **Deep Agents**, an open-source base harness engineered with foundational execution primitives. It provides execution environments ranging from simple file systems and lightweight code interpreters to full sandboxes. Deep Agents also incorporates production-grade context engineering patterns: conversational summarization for extended threads, context offloading for massive tool call outputs, architectural design tailored for prompt caching, human-in-the-loop steering mechanisms, and built-in planning workflows with subagent orchestration. While developers can still assemble granular custom harnesses from scratch using LangChain or LangGraph, Deep Agents provides an opinionated, production-ready starting point out of the box.
- **Business Logic Layer**: Instructions, domain-specific tools, skills, and specialized lifecycle hooks. - **Harness Primitives**: Memory and skill management, prompt caching optimization, human-in-the-loop steering, and dynamic subagents. - **Execution Environments**: File systems, lightweight code interpreters, and secure sandboxes.
---
[12:04](https://www.youtube.com/watch?v=XOUSuxY074U&t=724s) Managed Deep Agents: Running Agents in the Cloud
Moving to the infrastructure layer, Chase pushes back against the trend of desktop-bound local agents: “I don't think the future is just agents running on your laptop just for you. I think it's agents running in the cloud, working in the background, proactive, always on, multi-tenant, serving many users.” Running enterprise agents in the cloud demands extensive infrastructure: a robust agent runtime, authenticated and governed tool integrations, sandboxes for writing and executing code safely, and an LLM gateway to handle reliability, fallbacks, and cost controls.
To eliminate the friction of stitching these disparate systems together, LangChain combined the harness layer and managed cloud infrastructure into **Managed Deep Agents**. Rather than assembling five separate backend components, developers write their business logic and deploy it immediately to a scalable multi-tenant environment using a single CLI command (`mda deploy`). Chase emphasizes three core advantages of this setup:
1. **Rapid Connectivity**: Pre-built channels (Slack, Microsoft Teams, GitHub, incoming email) with out-of-the-box identity and authorization management. 2. **Decoupled Sandbox Execution**: Unlike designs that run the entire agent inside an isolated environment, Managed Deep Agents runs the agent runtime externally and treats the execution sandbox as a callable tool, backed by a secrets proxy to prevent model exposure to sensitive credentials. 3. **Enterprise Governance**: Built-in guardrails managing model permissions, spending caps, data protection, audit logging via Model Context Protocol (MCP) tool servers, and automated tracing via LangSmith Deployments (complete with checkpointing and task queues).
```mermaid flowchart LR accTitle: Managed Deep Agents Architecture accDescr: Multi-tenant channels trigger the agent runtime, which interfaces with external tools, models, and sandboxes under strict governance.
subgraph Triggers["Input Channels"] SL["Slack / Teams"] GH["GitHub Events"] EM["Email / Webhook"] end
subgraph Core["Managed Agent Runtime"] DEP["LangSmith Deployment"] GOV["Governance & RBAC"] end
subgraph Services["Infrastructure Services"] SB["Sandbox Tool (via Secrets Proxy)"] GW["LLM Gateway (Fallbacks & Budgets)"] MCP["Tool Server (MCP Protocol)"] OBS["LangSmith Tracing & Memory"] end
Triggers --> DEP DEP --- GOV DEP --> SB DEP --> GW DEP --> MCP DEP --> OBS ```
---
[16:49](https://www.youtube.com/watch?v=XOUSuxY074U&t=1009s) Internal Agents at Work: Open SWE and GTM Agent
To illustrate how deep agent harnesses function within daily operations, Chase highlights two internal agents built and deployed at LangChain: **Open SWE** and **GTM Agent**. Open SWE is an open-source coding agent whose responsibilities extend far beyond writing code snippets; it reviews pull requests, triages operational incidents, and will soon execute end-to-end QA workflows.
In parallel, LangChain relies internally on the proprietary GTM Agent to coordinate marketing, sales, and customer outreach. Designed as an always-on, autonomous assistant rather than a passive chatbot, it continuously reacts to incoming events and automations: > “It's a proactive agent, so it's always on. It's pinging you with updates. Pinged me to tell me that I had a big conference today. So thank you very much for that.”
The GTM Agent carries out background research, sends cross-domain communications using governed connectors, and spins up sandboxes to compile interactive artifacts that team members can access directly through generated preview URLs. Chase argues that deeply integrated, context-rich agents like these offer a genuine glimpse into the operational structure of future enterprises.
- **Open SWE**: Open-source engineering agent handling software authoring, PR reviews, incident triage, and automated QA. - **GTM Agent**: Always-on proactive assistant powering go-to-market operations, prospect research, outbound campaigns, and live sandbox-rendered visual artifacts.
---
[18:23](https://www.youtube.com/watch?v=XOUSuxY074U&t=1103s) Agent Traces as a New Data Primitive and SmithDB
Turning to the observability and evaluation feedback loop, Chase argues that monitoring autonomous systems requires treating agent traces as a completely new class of telemetry data. Unlike traditional application logs or shallow request-response metrics, agent traces feature deeply nested execution trees, massive token payloads, multimodal inputs, and unpredictable query access patterns. Crucially, they record unscripted human and machine reasoning: > “We think these traces contain natural language, and that natural language contains things like user intent and logic failures and glimpses into how people are using the agent, whether it's properly or improperly.”
Because standard databases struggle under the weight and structure of these execution graphs, LangSmith was engineered entirely around trace observability and evaluation. Sitting at the absolute core of LangSmith is **SmithDB**, an internally developed proprietary storage engine optimized explicitly for the high-throughput write requirements, variable payload sizes, and complex traversal paths inherent to multi-step agent traces.
[19:18](https://www.youtube.com/watch?v=XOUSuxY074U&t=1158s) Reimagining Ingest: Decision Models and Ingest-Time Online Evals
Harrison Chase explains that the query patterns required for dealing with agent traces are fundamentally different from traditional observability. SmithDB, the underlying database of LangSmith, is designed from the ground up to serve as the premier agent platform and data lake for these rich trace structures. Rather than treating traces as conventional logs or time-series metrics, LangSmith evaluates traces along four distinct axes: how they are ingested, stored, queried, and visualized. Harrison emphasizes: “And when we think about SmithDB and therefore LangSmith as a platform, we want to make it the best agent platform, the best data lake for these agent traces.”
The first axis of innovation is ingest. Agent traces arrive packed with natural language clues revealing how the system behaves, what end users are trying to accomplish, and how effectively the agent responds. Historically, teams captured these clues by running online evaluations powered by expensive frontier models or custom code judges. That paradigm is shifting rapidly with the emergence of decision models like Jev, which LangSmith now integrates directly as an online judge. As Harrison describes it: “What does that mean? It basically means that instead of generating text, it makes decisions.”
When using Jev as a judge, users pass in the agent state alongside a curated list of questions and candidate answers. Instead of expending compute to generate open-ended natural language explanations, Jev simply selects the best-fitting choice from standard, intuitive answer types and scores across multiple evaluation dimensions. Because it bypasses token generation, Jev operates dramatically faster and at a fraction of the cost of frontier LLMs. Harrison underscores the broader architectural lesson: “We think this is a great example of how the ingestion of agent data is different from kind of like traditional logs.”
- **Four Operational Axes of SmithDB**: Ingesting, storing, querying, and visualizing agent trace data. - **Jev as a Judge**: A purpose-built decision model that chooses among predefined categories or scores instead of generating text, enabling low-cost, high-speed automated evaluation at ingestion time. - **Natural Language Signals**: Ingest-time evals extract semantic signals directly from conversational turns, tool calls, and model outputs rather than relying on status codes or error logs.
```mermaid flowchart TD accTitle: SmithDB Agent Trace Platform Architecture accDescr: Diagram showing the four axes of trace management in SmithDB alongside the proactive intelligence layer in LangSmith Engine.
subgraph INGEST ["1. Ingestion Layer"] A["Agent Execution Traces"] --> B["Online Evals"] B --> C["Jev Decision Model Judge"] end
subgraph STORAGE ["2. Storage Layer (SmithDB)"] C --> D["SmithDB Cloud / On-Prem / BYOC"] D --> E["Granular Security (RBAC / ABAC)"] D --> F["LangSmith Trajectories (Message Lists)"] end
subgraph QUERY ["3. Query & Search Layer"] F --> G["Full-Text & Metadata Filters"] F --> H["AI Search / AI Query (Real-time Decision Eval)"] end
subgraph VIZ ["4. Visualization Layer"] H --> I["Standard Trajectory UI"] H --> J["Vibe-Coded Custom Apps"] end
subgraph ENGINE ["Intelligence Layer"] D -.-> K["LangSmith Engine (Proactive Agent Engineer)"] K -.-> L["Diagnosis, Clustering & Automated PR Fixes"] end ```
---
[21:27](https://www.youtube.com/watch?v=XOUSuxY074U&t=1287s) Storing Sensitive Traces: BYOC, Granular Permissions, and Trajectories
Turning to storage, Harrison highlights a massive operational divide between standard software logs and agent execution data: agent traces routinely capture deeply sensitive user inputs, proprietary context, and confidential enterprise data. To handle these compliance realities, LangSmith provides multiple hosting configurations. Beyond the multi-tenant cloud and full on-premise deployments, LangChain introduced Bring Your Own Cloud (BYOC), where all customer trace data remains safely inside the customer's cloud boundary while LangChain manages the hosted control plane.
Within the data storage model, LangSmith recently shipped Role-Based Access Control (RBAC) specifically scoped to trace inputs and outputs. Engineering teams frequently need visibility into the structural skeleton of an execution—such as tool invocations, token utilization, and latency—to monitor overall system health, but company policy often forbids them from seeing the actual raw inputs or generated outputs where personal or sensitive information lives. Looking ahead, Harrison previews attribute-based access control (ABAC), which will allow teams to tag traces with contextual metadata and apply granular authorization policies governing exactly which teams can view which traces. Acknowledging the unglamorous nature of enterprise security, Harrison remarks: “This type of, like, RBAC and ABAC is not usually something that I see people getting excited about, but it is so key for dealing with this sensitive information that's in all these traces.”
Beyond privacy controls, storage requires the right canonical schema. LangSmith launched LangSmith Trajectories to provide a standardized, message-list-based representation of agent interactions. While operators can still drill down into raw JSON payloads, tool invocations, and nested execution steps, presenting traces as clean conversational trajectories simplifies debugging and opens the platform to non-engineering subject matter experts for annotation. Furthermore, this message-first format mirrors the exact dataset schemas required for post-training and fine-tuning downstream models, making enterprise traces immediately training-ready.
- **Hosting Flexibility**: Multi-tenant cloud, fully isolated on-premise, and Bring Your Own Cloud (BYOC) with an external control plane and local data retention. - **Trace Input/Output RBAC**: Lets engineers inspect latency, token counts, and tool invocation trees without exposing confidential prompts or outputs. - **Upcoming ABAC Policies**: Dynamic metadata tagging to restrict trace access based on organizational attributes and data classification. - **LangSmith Trajectories**: A standardized message-list format that streamlines technical debugging, non-technical expert annotation, and offline post-training export.
---
[23:55](https://www.youtube.com/watch?v=XOUSuxY074U&t=1435s) Semantic Querying: Finding Needles in a Haystack with AI Search
Querying agent traces presents unique difficulties because the underlying data is heavily unstructured, natural language-heavy, and multimodal. Unlike deterministic software bugs that produce stack traces or HTTP error codes, agent failures rarely manifest as clean crash reports. Instead, failures emerge within subtle conversational semantics—such as an agent misunderstanding a subtle user requirement or a user becoming visibly confused by an ambiguous answer. Harrison notes that finding these subtle failures is the true bottleneck: “It's going to show up in the semantics of how people are using it, where the errors that happen aren't going to show up in logs.”
While LangSmith has long supported full-text indexing, metadata filtering, and filtering over large payloads, Harrison showcases a major leap forward called AI Search (or AI Query). Live on screen, Harrison demonstrates entering a plain natural language description into the trace search bar: *"user got confused by agent response."*
Under the hood, AI Search does not merely translate natural language into a rigid SQL or JSON query. Instead, LangSmith streams matching candidate traces through a lightweight real-time decision model, scoring each trace's semantic confidence against the prompt on the fly. In the interface, the engine highlights the exact textual evidence within the trace that matches the query, allowing engineers to isolate semantic bugs across millions of production logs without writing manual regex rules or hard-coded filters.
- **Semantic Failure Discovery**: Targets edge-case user interactions and soft hallucinations that bypass conventional error-handling and metric thresholds. - **Real-Time Decision Model Pipeline**: Rather than performing one-time text-to-SQL transpilation, candidate traces are streamed and evaluated dynamically against the query prompt. - **Inline Evidence Highlighting**: Displays specific conversational turns and tool responses that justify the decision model's confidence rating.
---
[26:01](https://www.youtube.com/watch?v=XOUSuxY074U&t=1561s) Visualizing Domain Data with Vibe-Coded Custom Apps
Visualizing trace data requires flexible front ends because agents operate across vastly divergent verticals. A legal research agent, an automated coding assistant, and a financial workflow agent each require completely different analytical lenses to review inputs, outputs, and intermediate scratchpads. While LangSmith Trajectories offers a clean baseline view, users repeatedly expressed a desire to tailor trace dashboards to their specific industry workflows.
To solve this, LangChain launched LangSmith Custom Apps. Demonstrating the feature via video, Harrison reveals an entirely "vibe-coded," custom HTML front end running directly inside the LangSmith workspace. Users can modify their interface in real time simply by typing conversational instructions into a prompt box. In the background, an embedded agent analyzes the workspace files, plans the structural changes, writes the underlying UI code, and immediately renders new dashboard widgets and visualization panes side-by-side with the chat.
Once generated, these vibe-coded views are saved as native, governed workspace pages accessible to the entire organization. Harrison observes that while teams were previously building one-off internal review tools outside LangSmith, centralizing that workflow provides critical guardrails: “But we think by bringing it into the UI, we can let people do this in a more governed way, where it knows all of the different APIs.” To ensure these bespoke apps feel polished and native, LangChain open-sourced its internal design system, allowing generated UIs to match the aesthetic and component styling of the core platform seamlessly.
- **Vibe-Coded Frontend Generation**: In-browser agentic development that generates custom HTML and JavaScript dashboard views from natural language prompts. - **Domain-Specific Workflows**: Allows legal, financial, or healthcare teams to build review interfaces suited to their specific document types and verification criteria. - **Open-Sourced Design System**: Guarantees visual and functional consistency between bespoke community apps and core LangSmith navigation. - **Enterprise Governance**: Custom apps remain securely authenticated against internal LangSmith APIs and persist as shared pages across workspace members.
---
[27:38](https://www.youtube.com/watch?v=XOUSuxY074U&t=1658s) Automated Intelligence: Diagnosing and Fixing Issues with LangSmith Engine
In the final section of his review of the platform, Harrison turns from observability infrastructure to actionable intelligence. Merely capturing, storing, and visualizing traces does not solve an engineering team's underlying operational burden. Harrison points out: “The visibility over the traces at this point has become table stakes.” The real pain lies in identifying systemic regressions across massive trace volumes, understanding whether a bug stems from the prompt, the retrieval context, or the base model, and implementing fixes without triggering regressions elsewhere.
To break teams out of the manual, reactionary "whack-a-mole" cycle of prompt engineering, LangChain introduced LangSmith Engine. Harrison describes the system as an automated, proactive agent engineer that continuously operates over production trace streams in the background. Instead of waiting for a human engineer to spot an issue, LangSmith Engine autonomously monitors incoming traces, clusters related failure modes, diagnoses root causes across model and retrieval layers, and prepares code-level remediation by opening pull requests against the agent's repository.
- **The Whack-a-Mole Problem**: Manual prompt tweaks often resolve one specific failure mode while inadvertently reintroducing regressions two months later. - **Automated Root-Cause Attribution**: Isolates whether errors originate in system instructions, context window truncation, retrieval failure, or base model capability. - **Proactive Agent Engineer**: Continuously scans, clusters, and diagnoses production traces to open automated pull requests with proposed fixes.
[28:53](https://www.youtube.com/watch?v=XOUSuxY074U&t=1733s) LangSmith Engine v2: Proactive Fix Verification and Red Teaming
Harrison Chase walks through the continuous evolution of LangSmith Engine, highlighting the launch of Engine v2, which introduces major automated testing capabilities. When Engine analyzes agent workflows, it can automatically generate new data points and evaluation suites. If an organization connects Engine directly into LangSmith Deployments, it can even trigger deployments automatically. With Engine v2, LangChain added the ability for Engine to proactively test the code fixes it generates. Working within LangSmith Deployments, Engine takes the proposed patch, spins up an isolated preview deployment, and runs test cases against it to ensure the previously identified defects no longer occur before presenting concrete evidence to the developer that the fix works.
In addition to evaluating passive historical traces, Engine v2 introduces an active red teaming feature designed to proactively stress-test agents. Rather than waiting for runtime failures to emerge in production logs, the red teaming engine generates synthetic probes based on multiple probing strategies, fires them into the system, and evaluates the agent's behavior. Chase demonstrates this via a video clip showing a red teaming run in progress. The tool surfaces discovered vulnerabilities with comprehensive evidence: the exact prompt sent in, the observed output, the expected behavior, the judge's reasoning for flagging the failure, and recommendations on remediation steps. Developers can click directly into the underlying execution trace to inspect the failure step-by-step. As Chase describes it, “This is proactively kind of like stress testing your agent for places where it may mess up.”
```mermaid flowchart TD accTitle: LangSmith Engine v2 Automated Testing and Fix Verification accDescr: Diagram showing how LangSmith Engine v2 performs proactive red teaming and tests fixes via preview deployments.
subgraph RedTeaming["Proactive Red Teaming"] A["Generate Probing Strategies"] --> B["Inject Probes into Agent"] B --> C["Judge Observed vs Expected Output"] C --> D["Surface Issues, Traces & Remediation"] end
subgraph FixVerification["Proactive Fix Verification"] E["Engine Detects Trace Issues"] --> F["Generate Proposed Code Fix"] F --> G["Spin Up Preview Deployment"] G --> H["Run Verification Tests"] H --> I["Confirm Fix & Trigger Deployment"] end ```
[30:30](https://www.youtube.com/watch?v=XOUSuxY074U&t=1830s) Agent Self-Reporting and Engine at Scale
Observing agent execution in the wild revealed that agents often detect when they are faltering or hitting roadblocks mid-run. To prevent the system from having to rediscover those failure modes retroactively through downstream analysis, LangChain introduced an in-flight self-report tool. As an agent executes, it can invoke this dedicated tool or API endpoint whenever it encounters an issue, passing along structured annotations and chain-of-thought reasoning explaining what went wrong. The runtime immediately binds these self-reported diagnostics to the trace metadata in the background. When LangSmith Engine runs its scheduled batch analysis later that night, it ingests that agent-supplied reasoning directly, substantially accelerating root-cause analysis. Chase highlights the value of this direct feedback loop: “We think this is really really useful because agents can oftentimes recognize as they're running where things are going wrong.”
Chase details ongoing engineering work to make Engine faster, cheaper, and more effective at remediating bugs, benchmarked continuously against standard coding benchmarks. For enterprise and privacy-focused teams operating on-premises, LangChain released Bring Your Own Key (BYOK) support inside the self-hosted v7 release of LangSmith Engine. Highlighting Engine's rapid adoption across fast-scaling tech companies, Chase shares production milestones to date: Engine has scanned more than 70 million traces and automatically flagged over 21,000 discrete issues. Chase emphasizes that developer velocity in the agent era depends entirely on the speed of iteration: “I think a lot of the speed at which you can build agents is really rate limited by how quickly you can spin that flywheel.”
- **Trace Scale:** Over 70,000,000 traces scanned across LangSmith Engine deployments. - **Issues Uncovered:** More than 21,000 distinct agent failure modes and defects detected. - **Self-Hosted Features:** LangSmith self-hosted release v7 incorporates Bring Your Own Key (BYOK) capabilities for Engine. - **Self-Report Mechanism:** Runtime tool invocation attaches annotations and failure explanations to traces for nightly Engine processing.
[32:10](https://www.youtube.com/watch?v=XOUSuxY074U&t=1930s) The LangChain Stack and the North Star: Owning Your Intelligence
Chase synthesizes the complete LangChain product architecture into three foundational pillars: the runtime, the data/observability platform, and applied intelligence. On the runtime side, Managed Deep Agents reflects LangChain's firm belief that production agents must run in the cloud rather than locally on laptops, operating in a shared, multiplayer, and multi-tenant environment equipped with integrated identity, authentication, and service connectors. For observability and evaluations, LangSmith is architected to serve as the definitive data platform for agent traces, custom-built to handle the unique ingestion, storage, querying, and visualization demands that agentic workflows impose compared to traditional application logs. Layered over both is Engine's intelligence layer, which automates triage, evaluation, red teaming, and code patching.
Concluding his keynote, Chase articulates the core philosophy guiding LangChain's roadmap: helping organizations truly own their intelligence. He dismisses the notion of a single monolithic model running the global economy, arguing instead that enterprises must specialize systems by injecting their own proprietary context, harness logic, and domain knowledge. “The thing that drives us is really to help people own their intelligence, as I 100% think the future is thousands of vertical, domain-specific agents both externally and inside one's companies that are really driving change and improved kind of like processes and new value,” Chase states. He notes that building these systems reliably requires embedding domain expertise into model weights, prompts, harnesses, and evals—themes echoed throughout the technical talks at the conference. Chase thanks the audience for attending the inaugural London Interrupt conference and urges attendees to continue sharing feedback on what remains difficult as they push agents into production.
## Notable quotes > [12:04](https://www.youtube.com/watch?v=XOUSuxY074U&t=724s) I don't think the future is just agents running on your laptop just for you. I think it's agents running in the cloud, working in the background, proactive, always on, multi-tenant, serving many users. — Harrison Chase
> [00:06](https://www.youtube.com/watch?v=XOUSuxY074U&t=6s) The thing that makes all of these application layer agents really take off and actually work is everything around the LLM, the system around it. — Harrison Chase
> [03:12](https://www.youtube.com/watch?v=XOUSuxY074U&t=192s) We think agents will absolutely transform the operating model of companies out there, but you have to have governance over them. — Harrison Chase
> [27:38](https://www.youtube.com/watch?v=XOUSuxY074U&t=1658s) The visibility over the traces at this point has become table stakes. — Harrison Chase
## Mentioned Harrison Chase, LangChain, LangSmith, LangSmith Engine, Managed Deep Agents, Deep Agents, SmithDB, Jev, LangSmith Trajectories, AI Search, LangSmith Custom Apps, LangSmith Deployment, LangGraph, Open SWE, GTM Agent, monday.com, Harvey, Orange, NVIDIA, Kimi, DeepSeek, ChatGPT, Claude, Model Context Protocol (MCP), Slack, Microsoft Teams, GitHub, Interrupt London
## Video description Harrison Chase, Co-Founder and CEO of LangChain, explains what it means to own your intelligence, the specialization a team builds into the model, the harness, and the context, why he thinks agents need to be open, compounding, and governed to get there, and points to monday.com, Harvey, and Orange as examples of companies building the system around the model. He then walks through the LangSmith stack: Managed Deep Agents, which combines the Deep Agents harness and managed infrastructure into one, plus two agents LangChain built internally, Open SWE and its GTM Agent; observability on SmithDB, with Jev as a judge, RBAC on trace inputs and outputs, LangSmith Trajectories, the new AI Search, and LangSmith Custom Apps; and LangSmith Engine, where Engine v2 tests its own fixes on LangSmith Deployment and red teams your agent. He also introduces a self-report tool that agents can call to flag what went wrong while they run, and shares that Engine has scanned over 70 million traces and detected over 21,000 issues.
This session was captured at Interrupt London, the agent conference by LangChain.
Chapters: 0:00 Why the system around the LLM is what makes agents work 3:12 Why own your intelligence, and what does it take? 6:59 How to organize people, process, and technology for agents 9:56 What goes into a harness like Deep Agents? 12:04 How Managed Deep Agents runs your agent in the cloud 16:49 Inside LangChain's own coding and GTM agents 18:23 Why agent traces are a new type of data 19:57 How decision models like Jev make online evals cheaper 21:27 Storing sensitive traces, from BYOC to trajectories 23:55 Finding needles in a haystack with AI Search 26:01 Building your own trace views with Custom Apps 27:38 How LangSmith Engine finds issues and tests its own fixes 30:30 Letting agents self-report what went wrong 32:58 Why the future is thousands of vertical agents
Resources: → LangSmith Engine: https://www.langchain.com/langsmith/engine → Managed Deep Agents: https://docs.langchain.com/langsmith/python/managed-deep-agents-overview → AI Search in LangSmith: https://www.langchain.com/blog/agent-traces-ai-search-langsmith → LangSmith: https://www.langchain.com/langsmith → Interrupt, the agent conference by LangChain: https://interrupt.langchain.com
[Source](https://www.youtube.com/watch?v=XOUSuxY074U)