LangChain
As foundation model capabilities commoditize, software differentiation shifts entirely to teams that build domain-specific harnesses and compound proprietary trace data through automated evaluation loops.
AI-assisted summary of Interrupt NYC: Opening Keynote — original source. Check the original for full context.
Source published:
Key ideas
- LangChain framed 'owning your intelligence' as the core competitive imperative for AI builders, shifting focus from commoditized foundation models to proprietary execution harnesses, compounding eval loops, and enterprise governance. (1:42)
- LangChain launched Managed Deep Agents featuring built-in enterprise authentication primitives (Connections, Identity, Channels), user-level memory, and native Parallel web search MCP integration. (12:42)
- The company introduced LangSmith Trajectories alongside the smithtune CLI, enabling teams to normalize complex multi-step agent traces and pipeline them directly into compute providers like Fireworks or Baseten for targeted fine-tuning. (19:50)
- LangSmith Engine v2 added proactive fix verification and automated pre-launch red teaming on preview deployment branches, having already scanned over 70 million production traces and flagged more than 21,000 issues. (26:03)
Detailed notes
## Background Published on YouTube by LangChain on October 5, 2026, this keynote captures the opening session of Interrupt NYC, LangChain's flagship agent conference. In the presentation, LangChain Co-Founder and CEO Harrison Chase outlines the concept of 'owning your intelligence' and walks through major updates across the LangSmith platform, including Managed Deep Agents, trajectory-based evals, fine-tuning tools, and LangSmith Engine v2.
## TL;DR Harrison Chase explains how the AI landscape transitioned from raw prototypes to domain-specific agents embedded directly in the application layer, driven by enterprise case studies like Rogo, Harvey, and JPMorgan Chase. He defines the three pillars required to own enterprise intelligence: an open and controllable harness, compounding evaluation loops, and rigorous governance. Walking through the three tiers of the LangSmith platform—runtime, observability/evals, and intelligence—Chase introduces Managed Deep Agents with enterprise auth and memory, decision model routing via the LLM Gateway, and LangSmith Trajectories for cleaner debugging. He concludes with announcements for fine-tuning via the smithtune CLI, LangSmith Custom Apps, and LangSmith Engine v2, which autonomously red teams agents and verifies its own code fixes on preview deployments.
## Atmosphere & style The presentation carries an energetic, forward-looking conference keynote vibe, with Chase speaking authoritatively and enthusiastically as he frames industry shifts across the past three years. The delivery blends high-level architectural philosophy with concrete product mechanics, moving methodically through framework diagrams and technical feature breakdowns. It feels like an ambitious practitioner-focused briefing, balancing enterprise infrastructure realities with palpable excitement about autonomous agent engineering.
## Bulletin - [01:42](https://www.youtube.com/watch?v=950byF7njfw&t=102s) LangChain framed 'owning your intelligence' as the core competitive imperative for AI builders, shifting focus from commoditized foundation models to proprietary execution harnesses, compounding eval loops, and enterprise governance. - [12:42](https://www.youtube.com/watch?v=950byF7njfw&t=762s) LangChain launched Managed Deep Agents featuring built-in enterprise authentication primitives (Connections, Identity, Channels), user-level memory, and native Parallel web search MCP integration. - [19:50](https://www.youtube.com/watch?v=950byF7njfw&t=1190s) The company introduced LangSmith Trajectories alongside the smithtune CLI, enabling teams to normalize complex multi-step agent traces and pipeline them directly into compute providers like Fireworks or Baseten for targeted fine-tuning. - [26:03](https://www.youtube.com/watch?v=950byF7njfw&t=1563s) LangSmith Engine v2 added proactive fix verification and automated pre-launch red teaming on preview deployment branches, having already scanned over 70 million production traces and flagged more than 21,000 issues.
## Key takeaways - [00:06](https://www.youtube.com/watch?v=950byF7njfw&t=6s) The AI industry has evolved rapidly from early 2023 prototypes into autonomous, domain-specific agents fundamentally reshaping the software application layer. - [01:42](https://www.youtube.com/watch?v=950byF7njfw&t=102s) Leading production implementations like Rogo, Harvey, and JPMorgan Chase's 'Jarvis' illustrate the shift toward building proprietary domain logic, routing, and scaffolding around base models. - [01:42](https://www.youtube.com/watch?v=950byF7njfw&t=102s) Escalating token costs, high-performing open-weight models, and commoditized base model intelligence are driving enterprise development teams to own their intelligence. - [04:25](https://www.youtube.com/watch?v=950byF7njfw&t=265s) Owning your intelligence rests on three core pillars: an open and model-neutral execution harness, a compounding evaluation loop, and robust enterprise governance. - [07:22](https://www.youtube.com/watch?v=950byF7njfw&t=442s) High-performing organizations structure agent efforts across three concentric rings: central platform engineers, hybrid agent engineers, and subject matter experts establishing domain ground truth. - [09:38](https://www.youtube.com/watch?v=950byF7njfw&t=578s) LangSmith's platform is organized into three distinct architectural tiers: cloud runtime infrastructure, agent-native observability and evals, and an overarching intelligence layer. - [12:42](https://www.youtube.com/watch?v=950byF7njfw&t=762s) Managed Deep Agents bundles the Deep Agents harness with managed cloud infrastructure, introducing Connections, Identity, and Channels to solve complex enterprise auth boundaries. - [15:51](https://www.youtube.com/watch?v=950byF7njfw&t=951s) Specialized decision models like Jev and SemIf offer rapid deterministic scoring for real-time guardrails, in-loop evaluation, and cognitive graph routing without generation latency. - [18:00](https://www.youtube.com/watch?v=950byF7njfw&t=1080s) The LangSmith LLM Gateway provides unified routing, message format normalization, enterprise rate limiting, cost auditing, and automated provider fallbacks. - [19:50](https://www.youtube.com/watch?v=950byF7njfw&t=1190s) LangSmith Trajectories abstracts multi-turn execution traces into standard conversational streams powered by SmithDB to simplify debugging and human annotation. - [22:57](https://www.youtube.com/watch?v=950byF7njfw&t=1377s) The smithtune CLI bridges production observability and post-training by exporting filtered trace datasets directly into model fine-tuning providers like Fireworks and Baseten. - [24:44](https://www.youtube.com/watch?v=950byF7njfw&t=1484s) LangSmith Custom Apps enables users to prompt internal coding agents to generate bespoke web interfaces on top of tracing data with inherited enterprise RBAC and ABAC controls. - [26:03](https://www.youtube.com/watch?v=950byF7njfw&t=1563s) LangSmith Engine v2 validates its own code patches by spinning up preview branches on LangSmith Deployment to verify bug fixes before presenting findings to engineers. - [26:03](https://www.youtube.com/watch?v=950byF7njfw&t=1563s) Engine v2 introduces automated agent red teaming to autonomously probe pre-launch deployments for prompt leaks and bad outputs while bootstrapping golden evaluation datasets. - [26:03](https://www.youtube.com/watch?v=950byF7njfw&t=1563s) LangSmith Engine has processed over 70 million production traces, detected more than 21,000 issues, and is expanding to self-hosted environments via BYOK support in v17.
## Detailed walkthrough ### [00:06](https://www.youtube.com/watch?v=950byF7njfw&t=6s) Welcome to Interrupt NYC: From Prototypes to the Application Layer
Opening the keynote with palpable energy, Harrison Chase welcomes the crowd to Interrupt NYC, remarking on the excitement of bringing LangChain's flagship agent conference to New York for the first time after holding it in San Francisco for the previous two years. Setting the stage for a jam-packed full-day schedule of announcements, fireside chats, and practitioner presentations, he pauses to contextualize how quickly the artificial intelligence landscape has transformed over the prior three years. While LangChain was founded just prior to the late-2022 generative AI boom, the industry has transitioned through clear generational phases at breakneck speed.
Recapping that progression, Chase explains that 2023 was primarily characterized by proof-of-concept prototypes. By 2024, development teams had begun shipping simple applications to production, though these early systems still largely relied on isolated, single LLM API calls. Exploration into multi-step agents expanded throughout 2025, culminating in the reality of the current landscape: autonomous agents reshaping the core application layer across entire verticals. Rather than serving as superficial "thin wrappers" around third-party models, production agents are now engineered with substantial software infrastructure to deliver defensible, differentiated experiences.
As Harrison Chase observed regarding the shift from generic wrappers to specialized production systems: “And really this year, we've started seeing that agents have transformed the application layer.”
- **Conference Background**: First New York City edition of Interrupt following two consecutive years in San Francisco. - **Industry Timeline**: - **2023**: Rapid prototyping and experimental proofs-of-concept. - **2024**: Production deployments of straightforward, single-LLM-call applications. - **2025**: Broad exploration into autonomous agents and multi-step architectures. - **2026**: Domain-specific agents fundamentally redefining the modern software application layer.
---
[01:42](https://www.youtube.com/watch?v=950byF7njfw&t=102s) Domain-Specific Agents and Owning Your Intelligence
To demonstrate how leading organizations construct value around foundation models, Chase highlights three prominent production case studies featured at the conference. First, fintech startup Rogo built a specialized model-routing harness to power an institutional-grade financial agent. Second, legal tech leader Harvey leveraged targeted model fine-tuning to maximize accuracy and procedural adherence across specialized legal workflows. Third, JPMorgan Chase engineered a deeply customized internal agent harness to power their proprietary "Jarvis" assembly line for enterprise operations.
```mermaid flowchart TD accTitle: Owning Your Intelligence Framework accDescr: Diagram illustrating how domain-specific context, routing, and fine-tuning wrap around foundation models. subgraph Scaffolding["Differentiated Domain Harness"] R["Intelligent Model Routing\n(e.g., Rogo)"] FT["Domain Fine-Tuning\n(e.g., Harvey)"] H["Custom Scaffolding & Context\n(e.g., JPMorgan Jarvis)"] end
subgraph Core["Base Intelligence Layer"] M["Open & Proprietary Foundation Models"] end
R --> M FT --> M H --> M ```
These architectures demonstrate what the industry now describes as "owning your intelligence"—the deliberate strategy of building domain-specific logic, data pipelines, and execution scaffolding directly in and around base models. Chase identifies three principal market pressures driving this movement: 1. **Escalating Token Expenditures**: As enterprise adoption scales into heavy production volumes, managing and auditing token consumption has become essential for preserving unit economics and proving business return on investment. 2. **Rise of Capable Open-Weight Models**: High-performing open models offer cost-efficient, self-hostable alternatives that allow teams to post-train and specialize weights for their exact domain tasks. 3. **Commoditization of Raw Model Intelligence**: Foundation model outputs are increasingly commoditized; real differentiation now stems from proprietary context injection, private data stores, and tailored business rules embedded within the application harness.
Reflecting on why differentiation has shifted away from foundation models alone, Harrison Chase stated: “And then the last thing that I think has become pretty clear over the past year is that the models themselves are becoming somewhat commoditized, and it's all of this stuff around the model, in particular the data, whether that is actual kind of like context you're providing or some of the domain-specific paths that you're baking into the harness.”
---
[04:25](https://www.youtube.com/watch?v=950byF7njfw&t=265s) The Three Core Pillars of Owning Intelligence
Connecting this philosophy back to LangChain’s founding, Chase recounts that while the project began a month prior to the release of ChatGPT, widespread adoption surged when engineers saw conversational AI and immediately wanted that same capability tailored to their specific industries and enterprise data. A core premise of LangChain has always been decoupling the application logic from the underlying model provider, giving builders complete model optionality.
Chase outlines the three fundamental pillars required to truly own intelligence in an agentic architecture: 1. **An Open and Controllable Harness**: The execution harness must remain strictly model-neutral, serving as both an offensive hedge (enabling instant migration to cutting-edge models from OpenAI, Anthropic, or specialized providers like TypeSafe’s Jev decision models) and a defensive safeguard against vendor lock-in or sudden API price increases. Furthermore, the harness must support granular control over context injection, whether expressed as explicit directed cyclic graphs via LangGraph or as extensible middleware within agentic frameworks like Deep Agents. 2. **A Compounding Evaluation Loop**: Launching an agent into production is merely the starting line. Builders must continually capture interaction data, trace edge-case failures, and compound the agent's capability over time through manual code and prompt adjustments or automated mechanisms like persistent memory and prompt optimization. 3. **Robust Governance**: Particularly critical for internal enterprise deployments, governance ensures organizations retain complete visibility into agent inventories, control compute spend, and enforce strict, role-based authentication and authorization across enterprise data sources.
Emphasizing the competitive advantage of domain-specific evaluation, Harrison Chase argued: “If you can define what better and what best looks like for your domain better than anyone else, you will be able to build the best agent for that domain.”
---
[07:22](https://www.youtube.com/watch?v=950byF7njfw&t=442s) Organizing People and Process for Enterprise Agents
Executing on these pillars requires restructuring how enterprise teams coordinate their workforce and design their software lifecycles. Chase details how successful organizations are structuring their talent into three distinct concentric rings: - **Ring 1: Core Platform Engineers**: Central infrastructure teams that deliver reusable tooling, operationalize governance, provide deployment rails, and maintain the compounding evaluation platform utilized across the business. - **Ring 2: Agent Engineers**: A newly emerged professional cohort that Chase describes as an interdisciplinary hybrid of data scientists, software engineers, and machine learning practitioners who consume platform tools to assemble, ship, and refine production agents. - **Ring 3: Subject Matter Experts (SMEs)**: Non-developer domain authorities situated at the business edge—such as legal analysts, traders, or clinical specialists—who must be equipped with intuitive interfaces to review agent behavior and establish what constitutes ground truth.
```mermaid flowchart LR accTitle: The Iterative Agent Lifecycle Loop accDescr: The continuous cycle of building, testing, deploying, and monitoring governed agents. subgraph Lifecycle["Iterative Agent Development Loop"] B["Build"] --> T["Test"] T --> D["Deploy"] D --> M["Monitor & Trace"] M -->|"Extract Learnings & Edge Cases"| B end ```
Alongside this organizational blueprint, organizations are embracing an inherently iterative development lifecycle. Because first-generation agent deployments inevitably fail on novel edge cases, high-performing engineering teams organize their workflow around a continuous feedback loop: build, test, deploy, monitor, and directly ingest production tracing learnings back into prompt and graph refinements—all executed within a compliant governance framework.
Describing this emerging engineering persona, Harrison Chase noted: “So, this is a great term to describe the weird hybrid mix of data scientist and engineer and machine learning engineer that makes up the people who are driving these agents.”
---
[09:20](https://www.youtube.com/watch?v=950byF7njfw&t=560s) Technology: Runtime Frameworks and Model Harnesses
Transitioning from organization and workflow into technical architecture, Chase introduces the foundational layers of the LangChain product suite designed to support this paradigm. The technological foundation begins with the runtime layer—the origin of LangChain’s open-source libraries, and the continuous focus of modern orchestration frameworks like LangGraph and Deep Agents.
These runtimes provide the programmatic scaffolding required to manage complex state, handle asynchronous tool execution, coordinate multi-agent loops, and enforce domain constraints before execution ever touches an evaluation or fine-tuning pipeline.
Summarizing LangChain's technical focus, Harrison Chase explained: “And when we think about the technology, and the technology that we help build and provide, there's really like two core parts to it: so one is the runtime.”
[09:38](https://www.youtube.com/watch?v=950byF7njfw&t=578s) LangSmith's Three Layers: Runtime, Observability/Evals, and Intelligence
Harrison Chase frames the architectural scope of LangSmith by breaking it down into two historical pillars and a newly added third layer. An agent cannot simply run locally on an engineer's laptop; it must run reliably in the cloud to serve an entire organization or external customers, which introduces a distinct deployment and runtime challenge. Alongside that runtime sits observability and evaluation, which Harrison describes as the bedrock of the compounding learning loop: being able to trace agent behavior and systematically evaluate whether actions were successful.
On top of runtime and observability, LangChain has recently layered an intelligence tier. This intelligence layer sits across the platform to help teams actively utilize trace data and speed up the agent development lifecycle. Harrison explains: “So now when we think about LangSmith, our commercial platform, we think about it serving the runtime aspect, serving the observability and eval aspect, and then building in intelligence across the platform to really help people utilize these components and speed up that process, that agent development life cycle that I talked about earlier.”
```mermaid flowchart TD accTitle: LangSmith Three-Tier Architecture accDescr: Diagram showing the three tiers of LangSmith: Intelligence Layer on top, supported by Runtime Infrastructure and Observability and Evals. subgraph L3["Intelligence Layer"] IL["Intelligence Across Platform (Engine v2, Red Teaming, Fix Proposals)"] end subgraph L2["Observability & Evals"] OE["LangSmith Tracing, Trajectories, Evals & Compounding Loops"] end subgraph L1["Runtime & Infrastructure"] RT["LangSmith Deployment, Sandboxes, LLM Gateway, Tool Servers"] end L1 --> OE OE --> IL ```
[10:38](https://www.youtube.com/watch?v=950byF7njfw&t=638s) Deconstructing the Agent Runtime: Logic, Harness, and Infrastructure
Harrison unpacks the runtime architecture of an agent into three distinct ingredients: business logic, the harness, and the underlying cloud infrastructure. Business logic encompasses instructions, tools, skills, and hooks—the specialized domain knowledge that developers supply. Skills required for an agent operating in finance differ drastically from those needed in a legal environment, making business logic the bespoke element contributed by each team.
The harness acts as the wiring layer, connecting the base model with this domain context so the model sees the right information at the exact right moment. Harrison points out that developers can either build custom, tightly controlled cognitive architectures using LangGraph or leverage general-purpose harnesses. To support the latter pattern, LangChain built Deep Agents, an open-source off-the-shelf harness launched roughly a year prior. Deep Agents borrows concepts from modern coding agents, featuring a dedicated execution environment, file system, sandbox, and context compaction mechanisms tailored for domain-specific applications. Finally, running in production requires cloud infrastructure: durable server execution, secure tool execution, code execution sandboxes, and reliable model access.
- **LangSmith Deployment:** Manages remote, durable agent runtime execution in the cloud. - **LangSmith Sandboxes (Beta):** Provides secure environments for running arbitrary agent-generated code. - **LangSmith LLM Gateway:** Handles unified access, routing, and governance across foundation models. - **Tool Servers:** Under active development to streamline secure, remote tool connectivity.
[12:42](https://www.youtube.com/watch?v=950byF7njfw&t=762s) Managed Deep Agents: Auth Primitives, User-Level Memory, and Parallel Search
To spare engineering teams from having to manually assemble disparate infrastructure components, LangChain introduced Managed Deep Agents, combining the Deep Agents harness and managed cloud infrastructure into a unified platform. Developers need only supply their business logic to launch a functional agent. Harrison introduces several major enhancements included in the latest Managed Deep Agents release, starting with enterprise authentication and permissions—an area he singles out as one of the hardest hurdles when launching internal corporate agents.
When an internal agent accesses tools like Notion or Jira, it must navigate complex identity boundaries: should it act under a fixed service identity or inherit the personal credentials of the calling user? Managed Deep Agents resolves this by introducing three core auth primitives: 1. **Connections:** Defines how end users authenticate to tool servers when invoking tools. 2. **Identity:** Distinguishes whether the agent acts as user-permissioned or service-permissioned as a whole. 3. **Channels:** Manages user context and credential attribution across communication surfaces such as shared Slack channels, ensuring runs are credited to the specific coworker sending the message.
Beyond authentication, the new release upgrades memory architecture and tool access: - **User-Level Memory:** Complements existing agent-level memory by tracking traits, context, and preferences specific to individual end users, enabling deep personalization across repeated interactions. - **Built-In Web Search via Parallel:** Agents can access real-time web search natively through Parallel's MCP without developers having to provision a separate third-party API key; the search integration runs directly against the user's LangSmith API key.
Harrison highlights how design partners use this setup: “We've been working with a number of early design partners on this, and we've seen really good results, especially for just getting an agent up and running and having it be like this company harness, and that's how we think about Managed Deep Agents, is this company harness that makes it really easy to get started building and sharing agents inside an org...”
```mermaid flowchart LR accTitle: Managed Deep Agents Workflow accDescr: Flow diagram illustrating how user business logic combines with Managed Deep Agents primitives into production runtime. BL["Business Logic (Instructions, Skills, Tools)"] --> MDA["Managed Deep Agents"] subgraph MDA_Feat["Managed Deep Agents Primitives"] AUTH["Auth Primitives (Connections, Identity, Channels)"] MEM["User-Level Memory"] PWS["Parallel Web Search MCP"] end MDA_Feat --> MDA MDA --> PROD["Production Agent Deployment"] ```
[15:51](https://www.youtube.com/watch?v=950byF7njfw&t=951s) The Rise of Decision Models: Jev and SemIf
Turning to model architectures, Harrison spotlights decision models, highlighting the launch of Jev over the preceding weekend. Unlike generative models, a decision model cannot generate arbitrary natural language text. Instead, given an input state and a discrete list of questions or options, it evaluates and outputs scores, boolean flags, or categorical choices. Harrison emphasizes that despite their narrower scope, decision models represent a critical shift in agent system design. While frontier models like Claude, Codex, and Deep Agents run primary multi-step loops, fast decision models can handle micro-decisions and deterministic routing.
Harrison details three primary use cases where decision models excel: - **In-Loop and Post-Hoc Evals:** Scoring an agent's intermediate actions in real time as tasks execute, allowing systems to flag or intercept bad outputs before they compound. - **Low-Latency Guardrails:** Filtering or vetting incoming user prompts almost instantaneously without incurring the multi-second latency penalty typical of standard frontier model calls. - **Model and Skill Routing:** Determining which specialized sub-agent, toolset, or cognitive graph branch to activate within LangGraph architectures.
Both Jev and SemIf—a prominent open-source decision model—are accessible directly through the LangSmith LLM Gateway, with Harrison predicting that specialized decision models will see rapid proliferation.
[18:00](https://www.youtube.com/watch?v=950byF7njfw&t=1080s) LangSmith LLM Gateway: Normalization and Traffic Governance
Harrison closes his walkthrough of the runtime tier by examining the LangSmith LLM Gateway, which entered beta roughly one to two months prior. The gateway serves as a unified routing, cost control, and governance hub across all internal and customer-facing LLM workloads. It standardizes model inputs and outputs across OpenAI and Anthropic message formats, while still offering a transparent pass-through mode so engineering teams can adopt vendor-specific features the moment APIs update.
The gateway also provides out-of-the-box integrations with developer-facing coding agents like Codex, Claude Code, and Cursor. Harrison argues that centralizing this traffic is vital for enterprise oversight: “Again, we think you should be routing all of your traffic, for governance and cost control reasons. We think you should be routing all of it through...”
[18:44](https://www.youtube.com/watch?v=950byF7njfw&t=1124s) Routing Every Model Call Through the LangSmith LLM Gateway
Harrison Chase transitions to the governance and infrastructure required when handling model providers across an enterprise. Because an LLM gateway must integrate with every AI application across an organization, LangChain focuses on three core operational pillars: cost controls, rate limiting, and fallbacks. Cost controls provide visibility into spending and allow teams to enforce granular budget limits across different teams, users, and API keys. Rate limiting ensures that a single runaway application cannot throttle shared quotas or dominate usage. Finally, provider fallbacks allow applications to gracefully switch to an alternative LLM provider when encountering upstream errors.
Looking ahead, Chase previews more stateful fallback behavior designed to eliminate wasted downtime:
> “So if you notice that requests are failing and Anthropic might be down, you don't even try them for the future. You just switch over to OpenAI for the next 10 minutes or 30 minutes, and make it really configurable in that way.”
He summarizes the ultimate objective for the gateway layer:
> “And so we want to make LangSmith LLM Gateway the most kind of like secure and reliable way to monitor and govern and fall back on different LLM providers when you're using them.”
[19:50](https://www.youtube.com/watch?v=950byF7njfw&t=1190s) Introducing LangSmith Trajectories
Moving from runtime systems to observability and evals, Chase emphasizes that traces serve as the fundamental system of record that powers the compounding intelligence loop. Historically, LangChain talked about observability using three main primitives: - **Run:** A single LLM invocation. - **Trace:** A single invocation of an entire agent. - **Thread:** A series of traces representing a multi-turn user interaction.
Over the past year, agent patterns have converged: whether builders are running coding agents or first-party assistants, the systems fundamentally run an LLM in a loop and compile an evolving sequence of messages. Because complex traces with thousands of tool calls become overwhelming to parse in traditional tree views, LangChain is introducing **LangSmith Trajectories**—a standardized format and UI representation for agent executions.
Trajectories offer three major benefits: - **Faster, intuitive debugging:** Instead of wading through massive raw traces, the UI condenses tool calls and iterations into a conversational stream that looks and feels like using a chat or coding agent. Users can scan sequences quickly and expand individual tool or LLM calls on demand. - **Simpler human annotation:** Because the full interaction history is cleanly rendered, reviewers and domain experts can leave feedback and evaluate steps far more easily. - **Standardized ingestion for post-training:** LangSmith parses messy trace formats from diverse SDKs and agent frameworks into a consistent trajectory schema, making production data immediately ready for fine-tuning.
```mermaid flowchart LR accTitle: Evolution from Traces to Trajectories accDescr: Diagram showing raw SDK agent traces condensed and normalized into LangSmith Trajectories.
subgraph Ingestion["Trace Collection"] A["Agent SDK Runs"] B["Coding Agent Runs"] end
subgraph Normalization["LangSmith Processing"] C["SmithDB Storage"] D["LangSmith Trajectories"] end
subgraph Applications["Downstream Uses"] E["Clean Debugging & Annotation"] F["Model Fine-Tuning"] end
A --> C B --> C C --> D D --> E D --> F ```
[22:57](https://www.youtube.com/watch?v=950byF7njfw&t=1377s) Turning Trajectories into Models with smithtune
Fine-tuning breaks down into two core ingredients: the training data and the training/inference compute infrastructure. To bridge the gap between production agent logs and specialized models, LangChain announces **LangSmith Fine-Tuning** alongside the new **`smithtune` CLI**.
Using `smithtune`, engineers can query and filter relevant traces directly out of LangSmith, pre-process them into fine-tune-ready trajectories, and send them directly into training platforms such as Fireworks or Baseten. Once training finishes, the models are evaluated and output as optimized, task-specific models. Chase underscores that this workflow unlocks fine-tuning across multiple use cases:
> “And so we think this is a really powerful way to get started fine-tuning, whether it's on agents or classification tasks or guardrails or anything like that.”
Underneath these trajectories is **SmithDB**, an in-house database introduced at a previous Interrupt conference. Built specifically to handle agent workloads, SmithDB dramatically accelerates ingesting, storing, and querying high-volume message histories, making rich trajectory visualizations possible without lag.
```mermaid flowchart LR accTitle: smithtune Fine-Tuning Workflow accDescr: End-to-end pipeline from LangSmith traces to an optimized model via smithtune.
A["LangSmith Traces<br/>(SmithDB)"] --> B["smithtune CLI<br/>(Filter & Preprocess)"] B --> C["Training Compute<br/>(Fireworks / Baseten)"] C --> D["LangSmith Evals"] D --> E["Optimized Task Model"] ```
[24:44](https://www.youtube.com/watch?v=950byF7njfw&t=1484s) Building Custom Interfaces with LangSmith Custom Apps
To solve the challenge of presenting internal data to non-technical stakeholders and domain annotators, LangChain announces **LangSmith Custom Apps**:
> “So, LangSmith Custom Apps you can basically think of as Lovable, but for your data that's already on LangSmith.”
Users can chat with an internal coding agent that builds bespoke web applications on the fly on top of their LangSmith data. LangChain open-sourced its own design system so the coding agent can construct polished, native-looking UIs out of the box. Crucially for enterprise governance, these generated apps automatically inherit the user's workspace boundaries, role-based access controls (RBAC), and attribute-based access controls (ABAC), providing a secure mechanism to bring human-in-the-loop reviewers directly into evaluation workflows.
[26:03](https://www.youtube.com/watch?v=950byF7njfw&t=1563s) Proactive Fixes and Red Teaming with LangSmith Engine v2
Shifting to the intelligence layer, Chase revisits **LangSmith Engine**, an autonomous system designed to act like a background AI engineer sitting directly on top of tracing projects. Rather than relying entirely on manual human triage, Engine continuously scans traces, clusters emerging failure modes on an automated issue board, writes candidate code patches, and generates regression datasets and evaluators.
Building on six months of deployment experience, LangChain is introducing **LangSmith Engine v2**, which adds two major automated capabilities:
1. **Proactive Fix Verification:** Deeply integrated with LangSmith Deployment, Engine no longer just guesses at code fixes. When it discovers a bug pattern in traces, it spins up a preview deployment branch to reproduce and confirm the failure. It then writes a fix, spins up a second preview branch, executes tests against it to verify the bug is resolved, and delivers an issue report complete with end-to-end evidence proving both the defect and the resolution. 2. **Automated Agent Red Teaming:** Previously, Engine could only catch errors that had already occurred in production traces. Through its deployment integration, Engine can now autonomously simulate adversarial inputs, generate failure hypotheses, and probe the agent deployment before launch: - Identifies prompt leakage or unauthorized learning. - Catches off-brand, hallucinated, or incorrect outputs. - Automatically bootstraps golden evaluation datasets, eliminating the historically painful manual effort required to generate test suites before shipping.
Chase highlights the real-world scale and efficiency milestones achieved by the Engine team:
> “Engine has already scanned over 70 million traces and detected over 21,000 issues.”
He stresses that speeding up the iteration loop is the single most critical factor in building reliable agents. To ensure enterprise availability, Chase announces that **Engine is now available for self-hosted environments using bring-your-own-key (BYOK)**, rolling out in self-hosted version v17 within a week or two.
- **Engine scale:** Over 70,000,000 traces scanned and 21,000+ distinct issues surfaced across fast-growing AI engineering organizations. - **Self-hosted availability:** Shipping in LangSmith self-hosted v17 with full BYOK support.
[30:52](https://www.youtube.com/watch?v=950byF7njfw&t=1852s) The Full Stack for Owning Your Intelligence and Looking Ahead
Harrison Chase wraps up the opening keynote by tying together the architectural layers LangChain has developed across the platform. At the foundation is the runtime layer: open-source harnesses paired with deployments, where managed agent runtimes are increasingly emerging as the most reliable path for teams pushing complex agent systems into production. Sitting directly above that is observability and evaluation, which LangChain has re-engineered to be distinctly agent-native. By focusing on multi-step trajectories rather than flat single-turn traces, allowing teams to construct tailor-made interfaces using Custom Apps, and enabling developers to feed production interaction data directly into fine-tuning workflows, teams can continuously adapt underlying models to their actual operational workloads.
```mermaid flowchart TD accTitle: LangChain Three-Layer Architecture accDescr: Architecture diagram showing the Intelligence Layer at top, Observability and Evals in the middle, and Runtime at the foundation.
subgraph Intelligence["Intelligence Layer"] Engine["LangSmith Engine<br/>• Trace scanning<br/>• Automated red teaming<br/>• Automated engineering tasks"] end
subgraph Observability["Observability & Evals"] Obs["LangSmith Platform<br/>• Agent trajectories<br/>• Custom Apps<br/>• Model tuning (smithtune)"] end
subgraph Runtime["Runtime Layer"] Run["Managed Agent Runtimes & Deployments<br/>• Open-source frameworks (LangGraph, Deep Agents)<br/>• Production managed execution"] end
Engine --> Observability Observability --> Runtime ```
At the top of the stack sits the intelligence layer driven by LangSmith Engine. Harrison emphasizes that automated trace scanning and adversarial red teaming represent merely the starting point for what LangChain envisions this layer handling on behalf of builders. As Harrison Chase puts it: “And then lastly, we have this intelligence layer that sits above, and we want to make Engine do a lot of the things that agent engineers and AI engineers are doing now.” The ultimate objective across all these layers is empowering organizations to take control of their systems and build deep, domain-specific capabilities—regardless of whether those agents power external user-facing products or automate and modernize complex internal workflows. Harrison concludes by thanking the crowd for attending Interrupt in New York City and invites the event's master of ceremonies, Jake, onto the stage to kick off the rest of the conference program.
- **Managed Agent Runtimes**: Bridging open source and deployment infrastructure to provide production-grade foundations for agent architectures. - **Agent-Native Observability and Evals**: Centered around structured trajectory analysis, bespoke visibility via Custom Apps, and continuous fine-tuning pipelines. - **The Intelligence Layer**: Positioning LangSmith Engine to automate tasks traditionally handled manually by AI and agent engineers. - **Owning Intelligence**: Giving organizations the harness, data loops, and tooling needed to build proprietary domain intelligence into both internal workflows and outward-facing products.
## Notable quotes > [04:25](https://www.youtube.com/watch?v=950byF7njfw&t=265s) If you can define what better and what best looks like for your domain better than anyone else, you will be able to build the best agent for that domain. — Harrison Chase
## Mentioned Harrison Chase, LangChain, LangSmith, LangGraph, Deep Agents, Managed Deep Agents, LangSmith Engine, LangSmith LLM Gateway, LangSmith Trajectories, LangSmith Fine-Tuning, LangSmith Custom Apps, SmithDB, smithtune, Rogo, Harvey, JPMorgan Chase, Parallel, Jev, SemIf, Fireworks, Baseten, OpenAI, Anthropic, Claude, Codex, Cursor, Lovable, Interrupt NYC
## Video description Harrison Chase, Co-Founder and CEO of LangChain, uses examples from Rogo, Harvey, and JPMorgan Chase to explain owning your intelligence, which means building domain-specific pieces in and around the model, and the three pillars it takes: an open, model-neutral harness such as LangGraph or Deep Agents, a loop that compounds what you learn from how people use your agents, and governance for internal agents.
He then walks through the LangSmith platform layer by layer: a new Managed Deep Agents release with auth primitives, user-level memory, and built-in web search via Parallel, decision models like Jev and SemIf through the LangSmith LLM Gateway, and three launches for observability and evals: LangSmith Trajectories, LangSmith Fine-Tuning with the smithtune CLI, and LangSmith Custom Apps. He closes with LangSmith Engine v2, which adds red teaming and tests its own fixes on LangSmith Deployment, and shares that Engine has scanned over 70 million traces and detected over 21,000 issues.
This session was captured at Interrupt New York City, the agent conference by LangChain.
LangChain products in this talk: LangSmith, LangSmith Engine, Managed Deep Agents, Deep Agents, LangGraph, LangSmith Deployment, LangSmith Sandboxes, LangSmith LLM Gateway, LangSmith Trajectories, LangSmith Fine-Tuning, LangSmith Custom Apps, SmithDB
Speakers: Harrison Chase, Co-Founder and CEO, LangChain
Chapters: 0:00 How agents went from prototypes to the application layer 2:43 Why teams want to own their intelligence now 4:25 What it takes to own your intelligence 7:22 How to organize your people and process around agents 9:20 What LangChain builds for runtime, evals, and intelligence 12:42 How Managed Deep Agents handles auth, memory, and web search 15:51 What a decision model like Jev is good for 18:00 Routing every model call through the LangSmith LLM Gateway 19:50 Why trajectories make agent debugging faster 22:57 Turning trajectories into optimized models with smithtune 24:44 Building custom UIs on your LangSmith data 26:03 How LangSmith Engine clusters issues and proposes fixes 28:04 How Engine v2 tests its fixes and red teams your agent 30:45 What LangChain wants Engine to take on next
Resources: → LangSmith Engine: https://www.langchain.com/langsmith/engine → Managed Deep Agents: https://docs.langchain.com/langsmith/python/managed-deep-agents-overview → LangSmith Fine-Tuning (smithtune): https://docs.langchain.com/langsmith/smithtune → LangSmith: https://www.langchain.com/langsmith → Interrupt, the agent conference by LangChain: https://interrupt.langchain.com
[Source](https://www.youtube.com/watch?v=950byF7njfw)