Cursor Lifts Model Accuracy 24% as Indexed Search Bests Raw Grep

RAG is dead, right?? — Kuba Rogut, Turbopuffer — AI Engineer

AI Engineer

Embeddings function as cached compute, allowing upfront codebase indexing to drastically undercut the recurring token and latency costs of unindexed agent grepping.

AI-assisted summary of RAG is dead, right?? — Kuba Rogut, Turbopuffer — original source. Check the original for full context.

Source published:

Key ideas

  • Despite widespread tech discourse claiming 'RAG is dead' in late 2025 and early 2026, Google search volume for RAG hit a major inflection point in mid-2025 and surged to all-time highs. (0:07)
  • Cursor reported a 23.8% increase in answer accuracy on its Composer model after adding semantic search, alongside a 2.6% increase in code retention and a 2.2% drop in dissatisfied user requests in production A/B tests. (4:43)
  • Embeddings function as cached compute, presenting a direct economic tradeoff between upfront indexing (Cursor) and repeated runtime discovery costs like unindexed filesystem grepping (Claude Code). (6:08)
  • Citing Google's Jeff Dean, expanding LLM context windows does not replace retrieval; architectures require staged retrieval to sift trillions of candidate tokens down into the right millions for processing. (9:44)

Detailed notes

## Background Published by the AI Engineer channel on YouTube on June 9, 2026, this presentation features Kuba Rogut, a deployed engineer at turbopuffer. The video addresses the popular discourse declaring retrieval-augmented generation obsolete, presenting real-world performance metrics from Cursor and architectural comparisons with Claude Code to demonstrate why iterative, multi-primitive retrieval remains vital.

## TL;DR Kuba Rogut challenges the social media consensus that 'RAG is dead,' pointing to record-breaking search volume and Cursor's double-digit accuracy gains from integrating semantic search into its composer workflows. He argues that while naive single-turn vector lookups are indeed outdated, modern RAG has evolved into iterative agentic retrieval that combines vector search, BM25, grep, and structured filters. Contrasting Claude Code's unindexed filesystem grepping with Cursor's pre-computed Merkle-tree indexing, Rogut frames embeddings as cached compute that amortizes costs over repeated queries. Ultimately, citing Google's Jeff Dean, he demonstrates that expanding LLM context windows still necessitates staged retrieval to filter down trillions of candidate tokens to the specific million needed for inference.

## Atmosphere & style The presentation is punchy, pragmatic, and delivered with the confident humor of an engineer confronting theoretical social media hype with cold production telemetry ('So take that Twitter'). Rogut speaks briskly to an engaged live audience, utilizing crisp benchmark charts, terminal traces, and architectural comparison tables to ground the discussion in practical trade-offs. The atmosphere feels like an insider systems post-mortem, replacing hand-waving debates with clear mental models around compute amortization and storage tiering.

## Bulletin - [00:07](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=7s) Despite widespread tech discourse claiming 'RAG is dead' in late 2025 and early 2026, Google search volume for RAG hit a major inflection point in mid-2025 and surged to all-time highs. - [04:43](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=283s) Cursor reported a 23.8% increase in answer accuracy on its Composer model after adding semantic search, alongside a 2.6% increase in code retention and a 2.2% drop in dissatisfied user requests in production A/B tests. - [06:08](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=368s) Embeddings function as cached compute, presenting a direct economic tradeoff between upfront indexing (Cursor) and repeated runtime discovery costs like unindexed filesystem grepping (Claude Code). - [09:44](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=584s) Citing Google's Jeff Dean, expanding LLM context windows does not replace retrieval; architectures require staged retrieval to sift trillions of candidate tokens down into the right millions for processing.

## Key takeaways - [00:07](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=7s) Social media claims that RAG is dead are contradicted by real-world developer demand, as evidenced by Google search volume surging in mid-2025. - [01:39](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=99s) The perceived failure of RAG stems from a narrow definition limited to naive, single-turn vector search rather than a broader retrieval ecosystem. - [01:39](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=99s) Comprehensive retrieval incorporates vector search, full-text algorithms like BM25, regex, glob, grep, and structured metadata filtering. - [02:08](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=128s) Agentic search is an iterative reasoning process where agents dynamically inspect context, issue targeted queries, and refine searches rather than executing single-step lookups. - [03:15](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=195s) Cursor optimizes team-wide codebase indexing by using cryptographic Merkle trees to detect structural similarities, re-indexing only modified files. - [04:43](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=283s) Semantic search integration in Cursor improved answer accuracy by 12.5% to 13.5% across evaluated models and nearly 24% on its Composer model. - [05:20](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=320s) Production A/B testing at Cursor revealed that modest single-digit metric improvements represent substantial qualitative gains because semantic search triggers selectively. - [06:08](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=368s) Claude Code's abandonment of vector search in favor of agentic grep represents an architectural tradeoff between zero upfront indexing and higher per-session compute costs. - [06:55](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=415s) Embeddings serve as cached compute: paying an upfront cost to chunk and index pays dividends when amortized across repeated team queries. - [07:30](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=450s) Unindexed grep discovery scales poorly across multi-developer environments where identical context queries consume thousands of tokens per run. - [08:38](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=518s) Single-turn top-k vector search is obsolete for advanced applications, giving way to multi-turn agentic retrieval that actively feeds agent reasoning loops. - [09:44](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=584s) Jeff Dean emphasized that massive context windows require staged retrieval to act as a lightweight sieve, filtering trillions of tokens into the right million. - [10:20](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=620s) Growing foundation model context windows complement rather than eliminate retrieval, providing a necessary filter to make ultra-large models economically viable.

## Detailed walkthrough ### [00:07](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=7s) Introduction and the "RAG is Dead" Narrative

Kuba Rogut introduces himself as a deployed engineer at turbopuffer, a full-text and vector search database built from first principles directly on top of object storage. Addressing a packed room, he dives into the prevalent social media sentiment that retrieval-augmented generation has run its course, pointing out that by late 2025 and early 2026, tech discourse on X (formerly Twitter) became saturated with claims that "RAG is dead" and that agentic file search is all developers need.

```mermaid flowchart LR accTitle: Public Perception vs Search Reality accDescr: Contrasting social media narratives claiming RAG is dead with the mid-2025 spike in Google search interest. subgraph Narrative["Social Media (X / Twitter)"] A["Claims: 'RAG is dead'"] --> B["Agentic file search replaces RAG"] end subgraph Reality["Google Search Trends"] C["2023: Initial AI surge"] --> D["2024: Plateau / cooldown"] D --> E["Mid-2025: Inflection point / massive spike"] end ```

Rogut counters the online narrative by presenting real-world search behavior. While Google search volume for RAG climbed during the initial generative AI boom in 2023 and plateaued across 2024, it reached a dramatic inflection point in mid-2025, surging to all-time highs. Kuba sharply dismisses the online consensus against the data, quipping: “So take that Twitter.”

[01:39](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=99s) Defining RAG vs. Agentic Search

Rogut clarifies the terminology, explaining that the apparent death of RAG is merely the death of an oversimplified definition. Many developers reduce RAG to naive vector search—simply generating embeddings for a text corpus, performing a single vector similarity query, and feeding the resulting chunks directly into an LLM. At turbopuffer, retrieval is treated as a far broader collection of mechanisms, encompassing vector search, full-text search algorithms like BM25, traditional filesystem utilities like grep and glob, regular expressions, and structured metadata filters.

```mermaid flowchart TD accTitle: Expanded Agentic Retrieval Architecture accDescr: How an iterative agent uses a wide array of search tools rather than a single vector search step. User["Agent Query"] --> Tools subgraph Tools["Retrieval Tooling Suite"] T1["Vector Search"] T2["Full-Text (BM25)"] T3["Grep / Glob / Regex"] T4["Metadata Filtering"] end Tools --> Loop{"Sufficient context found?"} Loop -- "No" --> Tools Loop -- "Yes" --> LLM["LLM Generation"] ```

At the same time, the industry’s understanding of "agentic search" has been colloquially reduced to unindexed filesystem exploration via grep, popularized by command-line coding tools like Claude Code. Rogut reframes agentic search not as mindless grepping, but as an iterative, tool-rich reasoning process. Describing this model, Kuba states: “what we actually believe it is and you know kind of the definition we want to we want to give it is it's really giving the agents a set of tools to kind of progressively and iteratively find and reason over context.” Instead of a single retrieval step, an agent reads files, inspects the current state, identifies missing context, issues new queries across its tooling suite, and iterates until it has accumulated the exact context needed to complete the task.

[03:15](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=195s) How Cursor Indexes Codebases with Merkle Trees and turbopuffer

To demonstrate effective agentic search in production, Rogut highlights Cursor, an early turbopuffer customer that published a detailed engineering breakdown in early 2026 on codebase indexing. When a developer opens a repository or checks out a new branch, Cursor automatically parses, chunks, and embeds the files to enable fast semantic search across the project.

Indexing large repositories from scratch is computationally prohibitive, especially within teams where dozens of engineers frequently touch the same codebase. To avoid re-chunking, re-embedding, and re-uploading identical files, Cursor uses Merkle trees—cryptographic hash trees—to compute structural similarities between codebases across a team. If the trees show substantial overlap, Cursor copies over the pre-existing index data and limits parsing and embedding exclusively to the modified files, relying on turbopuffer to ensure the operation remains fast and secure.

- Merkle trees compute file and directory differences across developer environments to prevent redundant index runs. - Only diffed and updated files are re-chunked and re-embedded, drastically reducing overhead. - turbopuffer manages and isolates multi-tenant codebase index storage securely in object storage.

[04:43](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=283s) The Measured Impact of Semantic Search in Cursor

Addressing why a development tool would take on the engineering overhead of continuous background indexing, Rogut cites Cursor’s published benchmarks and internal evaluations. On their internal Cursor context benchmark, adding semantic search produced a 12.5% to 13.5% average gain in answer accuracy across evaluated models. The impact was especially pronounced on Cursor's Composer model (prior to Composer 2), which saw a 23.8% surge in accuracy once backed by semantic retrieval tools.

```mermaid flowchart TD accTitle: Cursor Retrieval Benchmark Gains accDescr: Benchmark accuracy and production A/B testing improvements from adding semantic search. subgraph ContextBenchmark["Internal Context Benchmark"] M1["Average across models: +12.5% to 13.5%"] M2["Composer Model: ~24% increase in accuracy"] end subgraph ProductionAB["Online A/B Testing"] P1["Code retention in large codebases: +2.6%"] P2["Dissatisfied user requests: -2.2% drop"] end ```

In live production, Cursor validated these improvements through an online A/B test. The results demonstrated a 2.6% increase in code retention within large codebases and a 2.2% decrease in dissatisfied user requests. While single-digit percentage improvements might appear modest at first glance, Rogut emphasizes that the semantic search tool is not triggered on every single prompt. Because the tool only fires when relevant to the request, a 2% to 3% shift across all production traffic represents a substantial qualitative leap.

[06:08](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=368s) Embeddings as Cached Compute: Claude Code vs. Cursor

Rogut examines the alternative architectural approach championed by Boris Cherny, the creator of Claude Code. Cherny noted publicly on X that early iterations of Claude Code experimented with local vector databases and RAG, but the team ultimately abandoned them in favor of running agentic grep commands directly against the user's filesystem. Rather than treating this as proof that RAG has failed, Rogut frames the divergence through a core turbopuffer thesis: embeddings are essentially cached compute.

```mermaid flowchart TD accTitle: Per-Session Discovery vs Cached Index Traces accDescr: Comparing Claude Code iterative grepping against Cursor indexed retrieval. subgraph ClaudeCode["Claude Code: Per-Session Grep (No Index)"] direction TB C1["User Query: 'How metadata filtering works'"] --> C2["Grep codebase"] C2 --> C3["Read file contents"] C3 --> C4["Assess relevance & repeat loop"] C4 --> C5["Cost: ~6,000+ tokens consumed per session"] end subgraph Cursor["Cursor: Cached Compute (Indexed)"] direction TB K1["Upfront Index: Parse, embed, store in turbopuffer"] K1 -. "One-time cost".-> K2["Runtime Tool Call: 'How is metadata filtered?'"] K2 --> K3["Lightweight semantic search"] K3 --> K4["Immediate targeted context (low tokens, fast execution)"] end ```

To illustrate the tradeoff, Rogut compares the execution trace of a Claude Code session with a Cursor session answering the query: "How does metadata filtering work?":

| Characteristic | Claude Code (Agentic Grep) | Cursor (Indexed Semantic Search) | |:--- |:--- |:--- | | **Indexing Step** | None. Operates directly on raw local files. | Upfront parsing, chunking, and embedding into turbopuffer. | | **Context Discovery** | Iterative loop: agent greps, reads full files, assesses context, and re-greps. | Single lightweight tool call querying the pre-computed index. | | **Token & Time Cost** | High per session (e.g., 6,000+ tokens for a single sub-step); paid repeatedly on every query. | Negligible runtime token overhead; one-time indexing cost amortized over thousands of queries. | | **Multi-User Scaling** | 10 developers asking the same architectural question repeat the full grep loop 10 times. | All developers leverage the shared cache; answers return nearly instantaneously. |

Kuba summarizes the dynamic succinctly: “So obviously there's this like upfront cost of parsing a code base embedding it and making it available but this is like a one-time cost and then at runtime the agent can just query something like how is metadata filtered?” He notes that within turbopuffer's own engineering team, several developers who previously relied on Claude Code migrated to Cursor because the combination of pre-computed semantic search and Composer 2 drastically reduces latency and token consumption.

[08:38](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=518s) The Evolution Toward Agentic Retrieval

Rogut closes the architectural comparison by underscoring that simple, single-turn RAG—embedding chunks, querying top-k matches once, and blindly stuffing them into a context window—is indeed obsolete for advanced use cases. While that rudimentary pattern dominated the landscape in 2023 and early 2024, production systems have transitioned to multi-turn agentic retrieval. By providing autonomous agents with a rich suite of complementary retrieval primitives, modern systems achieve major performance breakthroughs over naive context stuffing.

[09:09](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=549s) The Shift to Iterative, Agentic Retrieval

Retrieval is no longer a naive, single-step operation where an application fires off a solitary query to an external vector database and blindly consumes whatever returns. Instead, modern AI agents run multi-step reasoning loops, dynamically executing semantic search, full-text search, or other filtering techniques as required at each step. These agents pull in strictly what is necessary for their immediate task, analyze the newly acquired information, and then issue follow-up searches based on their updated context. As Kuba Rogut explains: “It's becoming super iterative and these agents are really understanding what they're searching and searching to understand more in a sense.” This cyclical loop transforms retrieval from an isolated lookup into an active reasoning tool that unblocks entirely new software products.

```mermaid flowchart TD accTitle: Iterative Agentic Retrieval Loop accDescr: Diagram showing an agent iteratively reasoning, dispatching search queries, and filtering context down to relevant tokens.

subgraph AgentLoop ["Agent Reasoning Loop"] A["Agent Reasoning Step"] B{"Search Strategy"} C["Semantic / Vector Search"] D["Full-Text / Filter Search"] E["Fetch Targeted Context"] end

A --> B B -->|"Concept match"| C B -->|"Exact/Keyword match"| D C --> E D --> E E -->|"Analyze & refine"| A ```

- Modern architectures treat retrieval not as a single database call, but as an ongoing conversation between reasoning agents and storage layers. - Agents adapt their search mechanisms on the fly, alternating between vector indexes and full-text search depending on the nature of the missing context. - Information retrieval feeds directly back into the agent's reasoning loop, allowing it to formulate sharper, downstream queries.

[09:44](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=584s) Jeff Dean on Staged Retrieval and Context Windows

Even as foundation models expand their raw context windows to handle massive token counts, Kuba Rogut points out that sheer capacity does not eliminate the necessity of retrieval. Referencing an interview with Google's Jeff Dean regarding Gemini's massive context windows, Rogut emphasizes that expanding context sizes to astronomical levels still requires intermediate mechanisms to sift through raw data. Jeff Dean captured this core architectural principle by observing: “what you really need is stage retrieval, like a lightweight mechanism to narrow down these trillion tokens into essentially millions at a time.” In short, you do not need a trillion tokens dumped into a prompt at once; you need the right million.

At Turbopuffer, customers store and index trillions of tokens, making staged reduction critical in practice. Rather than saturating a context window with unfiltered corpus data, the retrieval system's role is to act as that lightweight sieve—filtering down trillions of candidate tokens to the specific 10,000, 100,000, or million tokens that directly resolve the user's intent. Rogut concludes the talk by underscoring that context window growth complements rather than replaces indexing, inviting the audience to discuss technical details and thanking attendees as the presentation wraps up.

- Jeff Dean highlighted that even with hypothetical trillion-token context windows, staged retrieval remains indispensable to filter massive corpuses into digestible subsets. - High-scale systems at Turbopuffer manage trillions of tokens by relying on lightweight pre-filtering down to targeted sets of 10,000, 100,000, or 1,000,000 tokens before populating context windows. - Retrieval acts as a precision filter that makes ultra-large context models economically and computationally feasible.

## Notable quotes > [09:44](https://www.youtube.com/watch?v=UM6sFg_jdlE&t=584s) what you really need is stage retrieval, like a lightweight mechanism to narrow down these trillion tokens into essentially millions at a time. — Jeff Dean

## Mentioned Boris Cherny, Claude Code, Cursor, Composer, Composer 2, Gemini, Google, Jeff Dean, Kuba Rogut, turbopuffer, X (formerly Twitter)

## Video description Cursor added semantic search and measured a 24% increase in answer accuracy on their composer model, a 2.6% gain in code retention in large codebases, and a 2.2% drop in dissatisfied user requests. Those numbers look small until you factor in that semantic search does not fire on every query. Meanwhile Google search volume for RAG hit a new inflection point in mid 2025 and went through the roof. The Twitter "RAG is dead" discourse and the actual usage curve are moving in opposite directions.

Kuba Rogut's argument is that the problem was never retrieval, it was the narrow definition of it. RAG is not just a vector search call. It is vector search, full text search, glob, regex, and filters used iteratively by an agent that keeps searching until it has what it needs. He contrasts Claude Code (grep per session, no index, repeat cost every run) with Cursor (one time upfront indexing, lightweight tool calls at runtime). Claude Code's approach is not wrong, it is a deliberate tradeoff. The frame that clarifies it: embeddings are cached compute, and whether to cache depends on query volume. Jeff Dean's version: you do not need a trillion tokens at once, you need the right million.

Speaker info: - https://www.linkedin.com/in/kubarogut/ - https://x.com/rogutkuba

Timestamps: 0:00 Introduction to the "RAG is dead" discourse 1:12 Google search volume trends for RAG 1:39 Defining RAG vs. Agentic Search 3:15 Cursor's indexing and semantic search approach 6:10 Contrasting Claude Code (grep) vs. Cursor (indexed) 6:40 The concept of embeddings as cached compute 8:38 The shift from simple RAG to Agentic Retrieval 9:44 Jeff Dean on context windows and stage retrieval

---

Watch along with the corrected transcript, summary, timestamps and resources on this talk's official AIE page: https://ai.engineer/talks/UM6sFg_jdlE-rag-is-dead-right-kuba-rogut-redacted-username

Join our upcoming AI Engineer conferences (see https://ai.engineer landing page):

- AIEi Shanghai, November 5–6 — the first for meeting top Chinese labs and engineers - AIE CODE in SF, November 10–12 — the top AI Coding conference returns - AIEi Sydney, December 7–8 — the first AIEi run alongside NeurIPS 2026 in Sydney

---

[Source](https://www.youtube.com/watch?v=UM6sFg_jdlE)

Reading summary

Kuba Rogut, an engineer at turbopuffer, argues that retrieval-augmented generation (RAG) is not obsolete, but has evolved from naive, single-turn vector lookups into iterative agentic retrieval where pre-computed indexes serve as cached compute. Countering the Narrative with Expanded Retrieval (0:07) By late 2025 and early 2026, social media discourse on X suggested that RAG was obsolete, replaced entirely by agentic file searching. Rogut counters this claim by pointing to Google Search trends, which plateaued in 2024 before experiencing an inflection point in mid-2025 that drove search interest to all-time highs. He asserts that what actually died is not retrieval itself, but the oversimplified pattern of naive vector search, where a system performs a single embedding lookup and dumps the resulting text chunks into a language model. At turbopuffer, retrieval is treated as a comprehensive suite of tools rather than an isolated vector call. Effective systems give reasoning agents access to vector search alongside full-text algorithms like BM25, regular expressions, structured metadata filters, and traditional filesystem tools like grep and glob. Instead of executing a single lookup, the agent progressively reads files, identifies missing context, and issues refined queries across these tools until it has collected the necessary information. Embeddings as Cached Compute Versus Agentic Grep (6:08) Rogut contrasts two divergent architectural strategies: the dynamic, unindexed filesystem exploration used by tools like Boris Cherny’s Claude Code, and the pre-computed semantic indexing implemented by Cursor. Cherny noted that Claude Code abandoned vector databases in favor of agentic grep commands run directly against the filesystem. Rogut frames this distinction around a central thesis: embeddings are essentially cached compute. Unindexed grep workflows require an agent to discover context iteratively during runtime—reading files, assessing relevance, and re-grepping—which consumes substantial time and tokens (often over 6,000 tokens for a single sub-step) on every user query. In contrast, Cursor pays an upfront parsing, chunking, and embedding cost once into turbopuffer, amortizing that expense across thousands of subsequent queries. To make this approach scalable for large developer teams, Cursor employs cryptographic Merkle trees to detect structural similarities across repositories, ensuring that only modified files are re-indexed rather than computing redundant embeddings. Empirical Impact of Semantic Search in Cursor (4:43) To justify continuous background indexing overhead, Rogut highlights benchmark data and live production evaluations published by Cursor in early 2026. On Cursor's internal context benchmark, introducing semantic search generated a 12.5% to 13.5% average improvement in answer accuracy across evaluated models. The performance gain was most pronounced on Cursor's Composer model prior to Composer 2, where accuracy rose by 23.8%. In live production, an online A/B test conducted by Cursor demonstrated a 2.6% increase in code retention within large codebases and a 2.2% reduction in dissatisfied user requests. Rogut emphasizes that because semantic retrieval tools trigger selectively rather than on every prompt, a single-digit percentage movement across total production traffic reflects a substantial qualitative improvement in output reliability. Staged Retrieval for Trillion-Token Contexts (9:44) Addressing the assumption that massive model context windows eliminate the need for external retrieval, Rogut references Google's Jeff Dean discussing Gemini. Dean noted that even with hypothetical trillion-token context windows, architectures still require staged retrieval as a lightweight mechanism to filter trillions of raw tokens down to the specific millions or tens of thousands relevant to an immediate task. Rogut explains that turbopuffer clients index and store trillions of tokens, making upfront data reduction practically and economically essential. Rather than saturating context windows with raw, unfiltered data, modern multi-turn retrieval functions as an active reasoning sieve that makes ultra-large context models computationally viable. Rogut concludes that while single-turn vector stuffing is dead for sophisticated production tasks, multi-turn agentic retrieval remains foundational, acting as a lightweight, cost-effective filter between massive enterprise data stores and expanding foundation model context windows.

Source quotes

[9:44] Jeff Dean: what you really need is stage retrieval, like a lightweight mechanism to narrow down these trillion tokens into essentially millions at a time.

Original transcript

[0:07] [music] [0:14] >> All right, welcome everyone. Thanks for [0:16] coming out. I see it's a full room, so I [0:18] appreciate everyone coming out. So [0:20] welcome to the talk about rag is dead, [0:22] right? So my name is Kuba. I'm deployed [0:24] engineer at Turbo puffer. So for those [0:26] that don't know what Turbo puffer is, we [0:28] are a full text search and vector search [0:30] database [0:31] built from first principles on top of [0:32] object storage. [0:34] If you would love to learn more, I'll [0:35] just come find me after the talk if you [0:37] have any questions. [0:38] So let's get started. So [0:40] this talk I get is up sorry about how [0:43] rag is dead, how hybrid tool tool rich [0:45] retrieval is becoming a default for [0:46] serious agentic search. [0:49] So [0:50] you guys have been on Twitter or other [0:51] social media platforms or I guess X they [0:53] call it now, [0:54] you might have seen a lot of tweets like [0:55] this about how rag is dead. You can see [0:57] there's lots of tweets especially in the [0:58] last you know in end of 2025 and in the [1:01] early of this year about how you know [1:03] rag is dead, agentic file search is all [1:05] we need and there's kind of a a lot of [1:07] tweet and a lot of kind of content about [1:09] this now. [1:11] But [1:12] you know interestingly if you're to look [1:13] at something like the Google Google [1:15] search volume over the last 2 years or [1:17] last couple years you can see that you [1:18] know in 2023 kind of like AI starts we [1:21] have the kind of this this [1:23] you know increase kind of caps out a [1:25] little bit in 2024 [1:27] settles down for about a year and then [1:28] about midway through 2025 we hit this [1:30] new inflection point where search volume [1:32] just goes through the roof. So take that [1:36] Twitter. [1:36] Um [1:38] So let's clarify first what is rag and [1:40] what is agentic search. These are kind [1:41] of the two terms a lot of people are [1:42] throwing out these days. [1:44] So rag what a lot of people think rag is [1:46] is just simple vector search. They just [1:48] think that this is just simply you know [1:51] embedding a bunch of you know a corpus [1:52] of contents [1:54] passing an embedding vector and getting [1:55] it back passing it through your LLM. And [1:58] what I Turbo puffer what we think this [1:59] actually means, you know if you break [2:00] down rag into retrieval augmented [2:02] generation, you know retrieval is not [2:05] just vector search. It's a lot of [2:07] different things. It could be vector [2:08] search full text search using stuff like [2:10] BM25, grepping, globbing, using regex, [2:13] using other just basic filters. [2:15] And the augmented generation is [2:17] obviously just passing it into your LLM [2:19] of choice. [2:20] And then agentic search. This is kind of [2:22] the terms people are throwing out a lot [2:24] these days. And generally when people [2:26] start talking about agentic search what [2:28] they usually talk about is essentially [2:29] just file system grep. So if you guys [2:31] are familiar with something like Cloud [2:32] Code [2:33] and kind of that's what Cloud Code Codex [2:37] you know a lot of people call this [2:38] agentic search and this just essentially [2:41] is grepping through your file system and [2:42] this is kind of why these terms are so [2:43] correlated. And what we actually believe [2:46] it is and you know kind of the [2:48] definition we want to we want to give it [2:49] is it's really giving the agents a set [2:51] of tools to kind of progressively and [2:53] iteratively find and reason over [2:54] context. So with Cloud Code you can you [2:57] know if you guys have are familiar with [2:59] it it can read your file you start [3:00] grepping through your file system read a [3:02] file decide that it hasn't found what it [3:04] needed what it what it needed to [3:05] actually complete the task and it will [3:08] you know find something again and then [3:10] keep doing this until it's happy you [3:12] know it's reached a happy state where it [3:13] can continue on with the task. [3:15] So we're going to take a step back [3:17] and talk about one of the companies that [3:19] use Turbo puffer [3:20] that we believe is doing an excellent [3:22] job with agentic search. This is a [3:23] company called Cursor you might have [3:25] heard of them. [3:26] Fun fact they're actually one of Turbo [3:27] puffer's very first customers. And they [3:30] have this excellent blog post that came [3:32] out in the beginning of 2026 about how [3:33] they index code bases. So for those [3:36] unaware when you open up a new code base [3:38] or new branch in Cursor what happens is [3:40] that Cursor will start embedding your [3:42] code base. So what they'll do is you [3:43] know chunk out you parse chunk and embed [3:45] your code base and make it available for [3:46] semantic search. And this blog post goes [3:48] into an excellent [3:50] kind of excellent technical detail of [3:51] how they do this. [3:53] Just to give you the gist essentially [3:54] the cool thing they do is that they [3:57] found that you know most people working [3:58] in a team let's say there's 100 [4:00] engineers [4:01] when they open up code bases they're [4:02] normally the same code base you know 99% [4:05] of the time because you have a team of [4:06] 100 people most of the time working on [4:08] one two maybe a few code bases, right? [4:11] And it's really expensive to have to [4:13] like re-chunk re-embed and re-upload [4:15] these code bases every single time. So [4:18] they essentially use like Merkle trees [4:19] which essentially is this crypto hash [4:20] tree to calculate similarities between [4:23] code bases people open on the same team. [4:25] And if they're similar enough [4:27] they will essentially copy over the data [4:29] and then only update the [4:31] and re-chunk and re-embed the files that [4:34] have changed and use Turbo puffer in [4:36] order to make sure this is done [4:36] securely. And [4:39] yeah it's just excellent blog post. They [4:41] they do some really cool stuff. [4:43] And you may think like this is a lot of [4:44] work. Why do they do this? Well the [4:47] reason they do this is also covered in a [4:48] in a different blog post about how they [4:50] use semantic search. Again they use [4:52] Turbo puffer Turbo puffer for this. [4:54] And what they find is on average across [4:56] models I think it's like a 12 and a half [4:57] or 13 and a half percent increase in [4:59] answer accuracy. This is in cross across [5:02] their internal Cursor context benchmark. [5:05] So you know not not a public benchmark [5:07] but [5:09] you can trust the numbers they give us. [5:10] And you can see on on the right side [5:12] their composer model so it's this this [5:14] is before composer two [5:16] they had a almost a 24% increase in [5:18] answer accuracy. So giving semantic [5:20] search to these tools and to these [5:22] models is really can drive real [5:24] performance gains. And you can see on [5:26] the on the bottom right [5:28] this is from an online AB test they did [5:30] which is also covered in their thing [5:32] in in their blog post [5:34] about how it's almost like a 2.6% [5:37] retention code retention in large code [5:39] bases and there's a 2.2% decrease in [5:42] dissatisfied user request. And you may [5:44] be thinking like oh well these numbers [5:45] aren't that big like 2.6% 2.2% not that [5:49] large but they also covered that [5:50] semantic search isn't used in every [5:53] single query. So in their online AB test [5:55] you know if you give it if you give this [5:57] tool to 100 query 100 random queries [6:00] not every 100 query will actually [6:02] benefit from the existence of a semantic [6:04] search tool so that's why these numbers [6:06] look kind of small. [6:07] Um [6:08] And now let's talk a little bit about [6:09] Cloud Code. So Cloud Code doesn't use [6:12] vector search as covered by the tweet [6:14] from Boris Cherney. [6:15] So those unfamiliar with Boris he's [6:17] essentially the founding father of Cloud [6:19] Code. And he says that in early [6:21] iterations of Cloud Code they actually [6:23] did use rag in their local vector DB but [6:25] they found that it just didn't really [6:27] work out for them. [6:30] But this is something that [6:32] is important to understand is something [6:33] we like we've kind of like taken on a [6:35] lot internally understanding um [6:39] here at Turbo puffer is this idea that [6:41] like embeddings and semantic search are [6:42] kind of cash compute. And you may be [6:45] thinking like cash compute like kind of [6:46] throwing out a lot of terms at me right [6:47] now. Like I don't know exactly what that [6:49] means. And I think it's like best to [6:51] walk through an example [6:53] of essentially almost like a Cloud Code [6:55] looking trace and a Cursor looking [6:58] trace. Of how how these agents will [7:01] understand your code base. So [7:04] on the left is kind of a per session [7:05] discovery of Cloud Code. So for example [7:07] if we're to ask the agent to understand [7:10] how metadata filtering works what it [7:12] have to do is grep read assess and [7:14] repeat and try to find the files it [7:17] needs in order to gain this [7:18] understanding on a per session basis. So [7:21] what this means is you could have you [7:22] know 10 agents on 10 different days [7:26] across 10 developers and you they can be [7:28] asking the same question multiple times [7:30] you know in day every day [7:32] every time the agent's going to have to [7:34] repeat these same exact steps [7:36] to gain the kind of like same [7:37] understanding of this code base. And [7:40] this could you know cost quite a few [7:41] tokens. You know 6,000 doesn't seem like [7:43] a lot here but just remember this is [7:44] like one sub step of an agent. [7:47] And then on the right is kind of like a [7:48] more Cursor looking trace where there's [7:50] this upfront cost of indexing but then [7:52] we're able to allow for this like [7:54] lightweight tool to help the agent kind [7:56] of retrieve this information at runtime. [7:59] So obviously there's this like upfront [8:00] cost of [8:01] parsing a code base embedding it and [8:03] making it available but this is like a [8:05] one-time cost and then at runtime the [8:07] agent can just query something like how [8:09] is metadata filtered? [8:10] It can get some simple results and it [8:12] would save a lot of tokens a lot of time [8:14] and just a lot of money. And this just [8:17] helps the agent to become a lot faster. [8:19] You know a lot of people on the team now [8:22] that maybe were big Cloud Code users [8:23] here at Turbo puffer [8:25] they've you know they've actually [8:26] started switching to Cursor just because [8:28] of how fast it's becoming especially [8:30] with their composer two models and also [8:32] this semantic understanding it's just [8:34] become [8:35] we're finding really really good. [8:38] So [8:40] from rag to agentic retrieval. Um [8:43] So what we're finding now is that a lot [8:45] of people are no longer doing the simple [8:47] rag you know the the Twitter quote [8:49] unquote rag of just [8:51] doing vector search once and throwing it [8:53] into the context windows. [8:55] What we're finding [8:56] is that [8:57] this worked you know back in 2023 early [8:59] 2024 at kind of the beginnings of AI but [9:02] a lot of the more sophisticated [9:03] customers are doing agentic search and [9:06] it's giving like real real big [9:07] performance gains [9:09] and kind of unlocking like new products. [9:11] Um [9:13] And what we're finding is you know [9:14] they're doing a ton of calls they're [9:16] reasoning these agents are reasoning [9:18] through several steps. They're searching [9:20] semantically or through full text etc. [9:22] as needed and they're only fetching [9:24] what's needed for that specific specific [9:27] use case but the important thing to know [9:29] is that you know retrieval is no longer [9:30] just this like simple one-time call to [9:33] vector DB. It's becoming super iterative [9:35] and these agents are really [9:37] understanding what they're searching and [9:40] searching to understand more in a sense. [9:42] It's kind of like interesting loop. You [9:44] know Google's Jeff Dean he went on a I [9:47] forget if it was a show or podcast or [9:48] whatever and he had this really good [9:50] quote that we like to use that we also [9:53] we thought was super interesting. He was [9:55] talking a little bit about I believe [9:56] about how Gemini's models were kind of [9:58] having these really big context windows. [10:01] Uh, I forgot the exact question the the [10:02] host asked them, [10:04] uh, but he was saying, you know, big [10:05] context windows, it doesn't matter if [10:06] you get to a trillion context window [10:09] size, [10:10] what you really need is stage retrieval, [10:12] like a lightweight mechanism to narrow [10:14] down these trillion tokens into [10:16] essentially millions at a time. [10:18] And like the exact quote is you don't [10:19] need a trillion at once, you need the [10:20] right million. [10:22] Um, this is something we think a lot [10:23] about Turbo here at Turbo Buffer. Um, [10:26] you know, we have customers that embed, [10:28] you know, have trillions of tokens [10:31] inside Turbo Buffer, and [10:33] as we see like the really important part [10:35] is just getting down to this right [10:37] 100,000, right 10,000, right million in [10:40] order to pass into these context [10:41] windows. [10:43] Uh, that's about it for the talk. Um, if [10:45] you have any questions about any [10:46] specifics, I'd love to, you know, either [10:49] have them ask now or you can find me [10:51] after the talk. [10:52] Uh, but appreciate you guys coming out. [10:55] >> [applause] [11:02] [music]