Meta Cuts Coded Dashboards as AI Agents Build 67% of Data Apps

@Scale

Upgrading AI reasoning models fails to deliver reliable enterprise data access on its own because smarter models merely return incorrect metrics with higher confidence unless paired with semantic ground truth and modular tools.

AI-assisted summary of Agentic Data at Scale: Transforming Data Experiences at Meta | Dinkar Pataballa from Meta — original source. Check the original for full context.

Source published: 2026-06-19

Key ideas

Meta is phasing out manually coded dashboards, with AI agent queries surging 30x over six months and roughly 67% of all new internal dashboards now created as agentic data apps. (0:05)
Data agents have reached broad adoption across Meta, with about 60% of weekly active users being non-data experts who query datasets and generate answers without writing code. (1:50)
Meta resolved agent reliability issues by moving away from brittle prompt scaffolding and raw model upgrades to an architecture uniting tools (reach) and semantic context (truth). (10:56)
The core engineering challenge with data agents remains silent failure: unlike broken code that crashes visibly, incorrect data quietly presents flawed metrics with high confidence. (1:50)

Detailed notes

## Background Published on YouTube by the @Scale channel on June 19, 2026, this talk features Dinkar Pataballa from Meta discussing how AI-native data experiences are reshaping enterprise decision-making. The presentation focuses on how Meta addresses the challenges of trust and accuracy across millions of datasets by enabling natural-language data consumption and letting non-technical users generate shareable data applications without writing queries.

## TL;DR Dinkar Pataballa outlines Meta's architectural transition from coded dashboards to autonomous data agents, noting that 67% of new dashboard creation and 60% of weekly non-data expert inquiries now run through agentic tools. He explains that while AI agents collapse the legacy analytics pipeline into rapid conversational interactions, they also eliminate the human validation layer that historically kept data trustworthy. To overcome the failure modes of fragile prompt scaffolding and overconfident model reasoning, Meta architected an AI data stack anchored in modular tools for reach and verified semantic context for truth. The framework evaluates not just raw accuracy but consistency across runs ('pass per key'), while confronting new challenges such as surging warehouse workloads and artifact sprawl.

## Atmosphere & style The presentation carries the measured, pragmatic authority of a senior engineering leader sharing hard-won battle scars from hyperscale infrastructure. Pataballa balances the genuine excitement of democratizing data access with an engineering sobriety, repeatedly returning to the unnerving reality that bad data fails silently whereas broken code crashes loudly. The narrative moves cleanly from the pain of legacy dashboard toil through honest admissions of early agent failures to a principled architectural blueprint, giving viewers an authentic inside look into enterprise-grade AI systems.

## Bulletin - [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) Meta is phasing out manually coded dashboards, with AI agent queries surging 30x over six months and roughly 67% of all new internal dashboards now created as agentic data apps. - [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) Data agents have reached broad adoption across Meta, with about 60% of weekly active users being non-data experts who query datasets and generate answers without writing code. - [10:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=656s) Meta resolved agent reliability issues by moving away from brittle prompt scaffolding and raw model upgrades to an architecture uniting tools (reach) and semantic context (truth). - [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) The core engineering challenge with data agents remains silent failure: unlike broken code that crashes visibly, incorrect data quietly presents flawed metrics with high confidence.

## Key takeaways - [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) A 30x surge in AI agent queries within six months is driving Meta to eliminate manually coded dashboards in favor of conversational data experiences. - [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) Friction in accessing data traditionally delayed critical decisions or forced leadership to make high-stakes choices without data backing. - [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) The legacy analytics workflow was slow but had free trust because human data scientists and engineers acted as expert intermediaries. - [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) Unlike code that fails loudly with syntax errors, flawed data silently corrupts downstream executive decisions when presented confidently by an agent. - [03:55](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=235s) Data consumption and data creation form a self-reinforcing flywheel, where answering a question leads to a shareable app, which in turn prompts new questions. - [05:34](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=334s) Meta's AI data stack is structured into three tiers: a context layer providing truth, composable tools providing reach, and an orchestrator coordinating reasoning. - [07:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=476s) The internal agent 'Data Mate' enables rapid consumption by disambiguating competing metrics, generating valid SQL, and providing traceable source citations. - [09:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=583s) Users can describe analytical needs in natural language to generate fully interactive, production-ready data applications without engineering handoffs. - [10:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=656s) Relying on thousands of lines of prompt instructions proved brittle and hit a rapid operational ceiling. - [10:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=656s) Upgrading foundation models alone failed to solve reliability because stronger reasoning over ambiguous or incorrect data simply produced wrong answers with higher confidence. - [10:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=656s) The architecture succeeded under the core mantra: models provide reasoning, tools provide reach, and institutional semantic context provides ground truth. - [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) Enterprises should treat data agents as an infrastructure play rather than an agent play, investing in reusable context and composable tools they can control. - [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) Context cannot be generated in a vacuum; it requires formalizing institutional domain knowledge, such as surface-specific definitions of revenue, into semantic models. - [14:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=883s) Consistency is as vital as accuracy for trust, requiring metrics like 'pass per key' to verify identical answers across repeated identical queries. - [16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) Machine-generated agent traffic causes exponential warehouse workload growth compared to human-paced querying, necessitating infrastructure designed for heavy automated loads. - [16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) The ultimate frontier is engineering infrastructure that makes bad data as visible and catchable as bad code across enterprise-scale systems.

## Detailed walkthrough ### [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) The Paradigm Shift: Wiping Away Coded Dashboards

In opening the final session of the conference day, Dinkar Pataballa points back to Barak's morning keynote, calling attention to a staggering metric: a 30x surge in AI agent queries over the past six months. Reflecting on a decades-long career spent constructing data infrastructure, platforms, and experiences, Pataballa notes that business inquiries have historically begun with the ubiquitous directive: "Look at my dashboard." He issues a bold claim that would have sounded preposterous merely a year prior: Meta is actively eliminating the need to manually code dashboards.

To illustrate why this matters, he paints a familiar high-stakes scenario. A product lead discovers an unexpected 3% drop in engagement and faces an urgent mandate from their VP to identify the cause by end of day. The answer inevitably resides somewhere within the data, but the lead faces immediate obstacles: they do not know where to start, which dataset or schema to inspect, which query to construct, or which analytics tool to open. At Meta's scale, data underpins every critical choice—from deciding what features to engineer and where to invest capital, to launching products and evaluating top-level executive strategy. When accessing data is slow, decision-making stalls; worse still, leaders are forced to make decisions with no data at all. Traditionally, product leads had only one recourse: submit an analytics ticket and wait.

- Barak's keynote revealed a 30x increase in agent-based queries across six months. - Meta is moving to phase out manually coded dashboards entirely. - Data access friction leads to slow corporate decisions or blind decision-making.

---

[01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) The Data Paradox and the Cost of Removing the Human Layer

The historical data paradox stems from a profound disconnect. Meta maintains millions of datasets, serves tens of thousands of decision-makers, and operates a massive computing infrastructure, yet the chasm between "the data exists" and "I have my answer" has remained massive. Bridging that gap traditionally required a disjointed sequence of stages—discovery, querying, visualization, and interpretation—each demanding specialized tools, distinct skills, and often entirely separate teams.

Despite that friction, the legacy system possessed an indispensable asset: "Trust was free." Decision-makers relied on data engineers and data scientists as trusted shepherds who inherently understood the nuances, table joins, business logic, and caveats. Even though answers were slow, they were dependable.

AI agents are collapsing that entire linear chain into an instantaneous natural-language conversation. Today, roughly 60% of non-data experts at Meta utilize data agents every single week, generating answers within seconds without ever writing code. However, dissolving the human intermediary reveals a far more dangerous dilemma: trust is no longer free. While broken code throws an explicit syntactic error, bad data silently manifests as an erroneous metric, which an agent might present with absolute confidence. If a flawed number enters an executive meeting, it leads directly to misguided decisions. Multiplied across hundreds of thousands of daily corporate inquiries, this dynamic creates a severe, compounding trust crisis.

- **Legacy data chain:** Discovery → Querying → Visualization → Interpretation (collapsing into a single conversational interface). - **Adoption:** Roughly 60% of non-data experts at Meta use internal data agents weekly. - **The Core Asymmetry:** Broken code fails loudly with an error, whereas incorrect data remains hidden and silently pollutes business decisions.

---

[03:55](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=235s) The Dual Dimensions: The Flywheel of Consumption and Creation

Dinkar Pataballa introduces his background as an Engineering Director in AI and Data Infrastructure at Meta, explaining that his teams spend their daily lives executing this agentic transformation across developer and business tooling. He frames his presentation around three core areas: engineering trusted agents at Meta's immense scale, sharing the genuine surprises and failures encountered along the way, and outlining future horizons.

```mermaid flowchart TD accTitle: The Data Consumption and Creation Flywheel accDescr: A circular flowchart showing how data consumption and data creation reinforce each other, underpinned by data trust.

subgraph Flywheel["Data Product Flywheel"] C["Consumption: Ask in plain language & get trusted answers"] A["Creation: Build shareable data apps without writing queries"] C -->|"Share insight via app"| A A -->|"Prompts follow-up questions"| C end

T["Underlying Prerequisite: Correctness & Trust"] -.-> C T -.-> A ```

Pataballa explains that user behavior breaks down into two core, interconnected dimensions: 1. **Consumption:** Enabling any employee to ask questions in plain English and instantly receive accurate, trusted answers. 2. **Creation:** Enabling any user to take that answer and build a shareable data asset or app without writing a single line of SQL.

These two dimensions operate as an interdependent flywheel: an employee asks a question, converts the resulting insight into a shareable application, and peers viewing that app formulate follow-up questions, spinning the cycle again. However, the entire mechanism depends on absolute correctness. A wrong initial answer produces an erroneous shared dashboard, which inevitably misleads downstream teams.

---

[05:34](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=334s) The AI Data Stack: Truth, Reach, and the Orchestrator

To support both consumption and creation without sacrificing integrity, Meta designed a foundational AI data stack built from the bottom up. At the base lies the raw data lake, described by Pataballa as the "data swamp." Directly above this raw storage rests the structural foundation: the context layer, or "truth." Truth is what gives the system business comprehension through semantic models, metric calculations, curated business rules, and analytical caveats. Without this semantic baseline, no downstream agent output can be trusted.

```mermaid flowchart TD accTitle: Meta's AI Data Stack Architecture accDescr: Layered architecture diagram showing raw data at the bottom, context and truth, composable skills for reach, the agent orchestrator, and the evaluation framework.

E["Evaluation Framework (Accuracy & Trust Validation)"]

subgraph Stack["AI Data Stack"] O["Orchestrator / Agent (Reasoning & Coordination)"] S["Composable Data Skills: Reach (Discovery, Schemas, Query Generation, Viz)"] T["Context Foundation: Truth (Semantic Models, Metrics, Rules, Caveats)"] D["Raw Data: Data Swamp"]

D --> T T --> S S --> O end

O -.-> E ```

Resting directly on the truth layer are composable data skills, which supply the agent with its "reach." Rather than baking static, rigid prompt instructions into the agent's core system persona, Meta structures reach as discrete, modular tools that the agent calls dynamically. These tools discover datasets across catalogs, interpret complex schemas, synthesize syntactically and semantically valid queries, and configure charts.

At the summit sits the agent orchestrator. While the agent manages conversational reasoning and step-by-step coordination, Pataballa emphasizes that an orchestrator is strictly limited by the truth and reach beneath it. An overarching evaluation framework continuously benchmarks the platform to ensure answers are consistently accurate.

- **Truth (Context):** Encodes semantic models (e.g., explicitly defining that a column named `revenue` or `engagement` represents net ad revenue on a specific surface, excludes defined edge surfaces, and aggregates on specific cadences), verified queries, and known caveats. - **Reach (Data Skills):** Discrete modular tools invoked to search millions of catalog entries, resolve schema ambiguities, and generate semantically sound queries. - **Orchestration:** The top-level agent reasoning layer coordinating tool use.

---

[07:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=476s) Consumption in Action: Data Mate and Traceable Trust

When observing the consumption experience on the surface, the interaction appears deceptively simple: a user asks a question in plain text and receives a formatted response. Under the hood, however, the agent performs heavy cognitive lifting. It searches millions of catalog entries to locate the single correct dataset, disambiguates between up to 15 different competing internal sources for terms like "revenue" or "engagement," tailors its metric interpretation to the user's operational scope, writes semantically valid SQL, and identifies its own limitations when data is missing or ambiguous.

Pataballa demonstrates this capability through **Data Mate**, an internal data analytics consumption agent deployed across Meta. When an employee queries Data Mate in plain language, the system supplies the calculation, analytical commentary, and immediate forward-looking recommendations alongside explicit data citations. These citations link directly back to the underlying source tables and queries, serving as a non-negotiable trust mechanism rather than a decorative UI element. Returning to the scenario of the product lead confronting a 3% engagement drop, Pataballa emphasizes the shift: instead of filing an analytics ticket and waiting days, the lead receives a fully traceable, cited diagnosis in seconds.

He then pivots toward the creation dimension of the platform: how this same workflow allows users to construct and distribute shareable analytics without ever having to write dashboard code by hand.

[09:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=583s) Replacing the Legacy Dashboard Workflow with Agentic Data Apps

In the traditional workflow, requesting a data tool meant handing off specifications to an engineer who would build a dashboard over days or even weeks. If the stakeholder needed even a minor tweak, the entire cycle would rinse and repeat, resulting in "genuine human toil in getting it out there for production consumption." Today, this friction is eliminated: users simply describe what they want in plain language, and AI agents generate a production-quality, interactive data application. This is not a throwaway prototype, but an actual production system running live across Meta.

On the interface, a user inputs a natural language description on the left-hand side, and the system renders an interactive, production-grade application on the right. The agent autonomously orchestrates data discovery, determines the exact queries, picks the visualizations, and arranges the layout so users can "vibe with it" until the data story is right. Despite launching less than four months ago, adoption has been rapid: roughly 67% of all new dashboard creation at Meta is now done through these agentic data apps, displacing a legacy platform and habit that persisted for more than a decade.

```mermaid flowchart LR accTitle: Legacy versus Agentic Dashboard Creation accDescr: Diagram contrasting the slow legacy cycle of dashboard creation with the real-time AI agentic approach.

subgraph Legacy["Legacy Dashboard Workflow"] direction TB L1["User Request"] --> L2["Manual Engineering<br/>(Days / Weeks)"] L2 --> L3["Dashboard Delivery"] L3 -->|"Minor Edits"| L1 end

subgraph Agentic["Agentic Data App Platform"] direction TB A1["Plain Language Prompt"] --> A2["AI Agent<br/>(Discovery, Queries, Viz, Layout)"] A2 --> A3["Interactive Data App<br/>(Production Ready)"] A3 -->|"Real-Time Iteration"| A1 end ```

[10:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=656s) The Three Phases of Agentic Trust: Scaffolding, Reasoning, and Context

The fundamental question that "genuinely kept us up at night was, can we trust what these agents are doing? Can we trust the answers? Can we give them for general grade usage?" Reaching an affirmative answer required progressing through three distinct evolutionary phases:

```mermaid flowchart TD accTitle: The Three Phases of Agent Architecture accDescr: Evolution of agent design from prompt scaffolding to model reasoning to tools and semantic context.

subgraph P1["Phase 1: Workflow Scaffolding"] S1["Thousands of Lines of Prompt Instructions"] --> S2["Brittle & Ceiling Reached Quickly"] end

subgraph P2["Phase 2: Upgraded Reasoning"] S3["Smarter Foundation Models"] --> S4["Plateaued: Wrong Data Answered Confidently"] end

subgraph P3["Phase 3: Tools and Truth"] S5["Tools (Reach) + Semantic Context (Truth)"] --> S6["High Accuracy & Consistency at Scale"] end

P1 --> P2 P2 --> P3 ```

- **Phase 1: Workflow Scaffolding.** The team started by writing thousands of lines of prompt instructions detailing how the agent should act as a data analyst—how to find tables, parse schemas, and navigate edge cases. It was like handing a manual to someone completely unfamiliar with Meta's data. While it worked initially, it was exceedingly brittle and quickly hit an operational ceiling. - **Phase 2: Upgraded Reasoning.** As newer, more capable foundation models arrived, the agents became smarter, and benchmark accuracy jumped temporarily before plateauing again. The core failure mode was clear: "better reasoning over wrong data just gives the wrong answers, delivering just more confidently." With 15 different revenue or engagement metrics available, the model could not determine the correct table or metric to pick, so it simply chose one arbitrarily and delivered flawed conclusions with high confidence. - **Phase 3: Tools and Truth.** The breakthrough occurred when the team stopped micro-managing the agent's step-by-step reasoning through prompt text and instead equipped it with capabilities (tools) and context (ground truth). This step-function change produced an internal mantra: *"Models give you the reasoning. Tools give you the reach. The context gives you the truth."*

[12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) Three Core Principles for Building Data Agents at Scale

Reflecting on these architectural learnings, three major principles guide building data agents across an enterprise:

1. **Invest in what you can control.** Teams do not control third-party model upgrades or emerging evaluation harnesses. What teams *do* control are the tool skills and institutional context supplied to the system. Unlike fragile prompt scaffolding, investments in capabilities and semantic models compound: every new skill or semantic definition added enhances every agent running across the platform. 2. **Treat it as an infrastructure play, not an agent play.** While entry points and agent varieties will proliferate over time, they must all sit on a unified underlying infrastructure platform. This shared platform must reliably supply trusted context, scale across millions of tables, reconcile ambiguous and shifting schemas, and handle massive query volumes. 3. **Never build context in a vacuum.** Context cannot be generated out of thin air; it requires directly encoding decades of institutional knowledge from data scientists and data engineers into formal semantic models. For example, revenue in Instagram has a distinct technical definition from revenue in the ads ecosystem. Embedding these rules ensures agents inherit human expertise rather than guessing or misinterpreting domain definitions.

[14:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=883s) Measuring Reliability: Benchmark Pass Rate and "Pass per Key"

To validate these system upgrades with hard empirical evidence rather than intuition, the team tracked agent evaluation metrics across the three architectural phases. Beyond the standard benchmark pass rate on internal company datasets, they introduced a second critical evaluation: the "pass per key" metric.

Because LLMs are non-deterministic, running identical questions multiple times often resulted in conflicting outputs. The pass per key metric evaluates consistency: if an agent is asked the exact same question five times, does it return the same answer five times? Pairing the models with the right retrieval tools and semantic context dramatically stabilized consistency alongside pure accuracy, establishing the foundation of trust required for production adoption: "For trust, consistency matters as much as accuracy."

- About 60% of weekly active users of the system are non-data experts. - Roughly 67% of Meta's dashboards are now created as agentic data apps. - The most transformative outcome is qualitative: employees who never previously considered themselves "data people" are now independently querying data and making data-informed decisions in their day-to-day work, proving that data-driven decision-making is no longer gated by technical expertise.

[16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) New Frontiers: Trust, Query Workload, and Compounding Artifacts

Achieving early success does not eliminate hard problems; it amplifies them across three primary dimensions:

- **Compounding Trust Requirements:** As more datasets, teams, and use cases onboard, expectations around data freshness, ownership, and reliability compound, magnifying the cost of any silent data error. - **Warehouse Workload Scale:** When humans manually wrote queries, the query volume to the warehouse grew at human speed. With autonomous agents acting on behalf of tens of thousands of employees, query volume multiplies exponentially—surging by 30x—demanding warehouse infrastructure engineered for massive machine-generated loads. - **Artifact Quality and Proliferation:** As agents generate apps, those apps prompt new questions, prompting agents to spin up even more downstream data apps. Managing the quality and lifecycle of these exponentially multiplying artifacts is critical to avoid system-wide sprawl.

Emphasizing the company motto that "the journey is only 1% finished," the true technical frontier is summarized by a core distinction: "Wrong code is visible. Wrong data is not." When code breaks, it crashes or throws a runtime error, but flawed data quietly produces believable, incorrect metrics. The ultimate challenge is engineering an infrastructure that makes bad data just as visible as bad code across the full scale and complexity of Meta's ecosystem.

## Notable quotes > [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) Look at my dashboard. — Dinkar Pataballa

> [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) Trust was free. — Dinkar Pataballa

> [09:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=583s) vibe with it — Dinkar Pataballa

> [10:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=656s) can we trust what these agents are doing? Can we trust the answers? Can we give them for general grade usage? — Dinkar Pataballa

> [10:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=656s) better reasoning over wrong data just gives the wrong answers, delivering just more confidently. — Dinkar Pataballa

> [10:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=656s) Models give you the reasoning. Tools give you the reach. The context gives you the truth. — Dinkar Pataballa

> [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) Invest in what you can control. — Dinkar Pataballa

> [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) Treat it as an infrastructure play, not an agent play. — Dinkar Pataballa

> [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) Never build context in a vacuum. — Dinkar Pataballa

> [14:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=883s) For trust, consistency matters as much as accuracy. — Dinkar Pataballa

> [16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) the journey is only 1% finished — Dinkar Pataballa

> [16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) Wrong code is visible. Wrong data is not. — Dinkar Pataballa

## Mentioned Dinkar Pataballa, Barak, Meta, Data Mate, Instagram, @Scale Conference

## Video description Every major business decision is ultimately built on data, yet getting to the right answers has long required specialized expertise, from knowing which tables to query to how to query them to building the right data applications. AI agents are fundamentally changing that equation, making data accessible to anyone who can ask a question in plain language.

At Meta's scale, with millions of datasets serving tens of thousands of decision-makers, this shift creates both massive opportunity and unique challenges around trust and accuracy. This presentation will detail how we built AI-native data experiences to address two key dimensions: enabling trusted answers through agentic data consumption, and letting users create shareable agentic data applications without writing a single query.

Want to learn more about the @Scale Conference? Visit our website here: atscaleconference.com

[Source](https://www.youtube.com/watch?v=BQf0TxsOMUg)

中文简报

Meta削减编写代码构建的仪表板，AI Agent构建了67%的数据应用单靠升级AI推理模型无法提供可靠的企业数据访问，因为更聪明的模型只会在缺乏语义真值和模块化工具配合的情况下，以更高的置信度返回错误的指标。 Dinkar Pataballa概述了Meta从编写代码构建仪表板向自主数据Agent的架构转型，指出目前67%的新仪表板创建以及60%的每周非数据专家查询均通过Agent工具运行。他解释称，尽管AI Agent将传统分析流程压缩为快速的对话交互，但也消除了历史上维持数据可信度的人工验证环节。为了克服脆弱的提示词脚手架和模型过度自信推理的失效模式，Meta构建了一个以用于拓展覆盖范围的模块化工具以及用于保障准确性的已验证语义上下文为基础的AI数据技术栈。该框架不仅评估原始准确性，还评估运行之间的一致性（“pass per key”），同时应对数据仓库工作负载激增和工件泛滥等新挑战。 Meta正在逐步淘汰手动编写代码构建的仪表板，AI Agent查询量在六个月内激增30倍，目前大约67%的新内部仪表板均作为Agent数据应用创建。 (0:05) 数据Agent在Meta已获得广泛采用，每周活跃用户中约有60%是非数据专家，他们无需编写代码即可查询数据集并生成答案。 (1:50) Meta摆脱了脆弱的提示词脚手架和单纯的模型升级，通过转向将工具（覆盖范围）与语义上下文（准确性）相结合的架构，解决了Agent的可靠性问题。 (10:56) 数据Agent的核心工程挑战仍然是静默失败：与会明显崩溃的损坏代码不同，不正确的数据会以极高的置信度悄无声息地呈现出有缺陷的指标。 (1:50)

Reading summary

Meta is shifting away from manually coded dashboards toward autonomous data agents, finding that reliable deployment at scale depends on grounding models in institutional context and tool capabilities rather than relying on prompt instructions alone. The Data Paradox and the Crisis of Silent Errors (1:50) Traditionally, accessing enterprise data required a multi-step pipeline—discovery, querying, visualization, and interpretation—handled by data engineers and data scientists. While this process created bottlenecks where decision-makers faced delays or made uninformed choices, it maintained high reliability because human experts verified business logic and schema nuances. As agents compress this pipeline into natural language interactions, roughly 60% of non-data experts at Meta now use internal data agents on a weekly basis. The Consumption and Creation Flywheel (3:55) Dinkar Pataballa frames enterprise data interaction around two interdependent behaviors: consumption (querying data in natural language to receive verified answers) and creation (turning those insights into shareable applications without writing SQL). These activities reinforce each other in a flywheel, where shared applications generate follow-up questions from other teams. Less than four months after introduction, roughly 67% of new dashboard creation at Meta shifted to these agentic applications, where users specify requirements in text and agents handle discovery, queries, visualizations, and layout. Evolution From Prompt Scaffolding to Context and Tools (10:56) Meta's agent architecture evolved across three phases to address early unreliability. The initial phase relied on extensive prompt instructions detailing analyst behavior, but this scaffolding proved brittle. Upgrading to more capable reasoning models in the second phase also hit a plateau; smarter models merely presented erroneous figures with greater confidence when selecting between competing internal metrics. The architecture stabilized in phase three around the principle: models supply reasoning, tools supply reach, and institutional context supplies truth. Grounding agents in formal semantic models and modular tools reduced arbitrary metric choices and improved benchmark reliability. Evaluating Consistency and Infrastructure-Led Scaling (12:43) To evaluate non-deterministic language models, Meta's engineering teams track both standard benchmark pass rates and a 'pass per key' metric, which tests whether an agent returns the identical answer when queried repeatedly. Pataballa outlines three implementation rules: invest in the controllable layers of tools and context, treat implementation as an enterprise infrastructure challenge rather than isolated agent development, and encode institutional knowledge directly from domain experts instead of generating context in isolation. Emerging Bottlenecks in Warehousing and Artifact Sprawl (16:00) Scaling agentic data systems introduces new infrastructure stresses. Machine-driven querying has driven a 30x increase in agent-based queries over a six-month period, requiring underlying data warehouses to support far heavier workloads than human-paced querying. Additionally, as agents autonomously generate data applications, managing the lifecycle and quality of these proliferating artifacts becomes essential to avoid organizational sprawl. Because flawed data produces plausible metrics rather than throwing runtime errors, system verification remains the critical operational focus. While agentic systems enable non-technical employees to make data-informed decisions, Pataballa cautions that the transition is early. Unlike software code that crashes noticeably upon failure, incorrect data manifests silently, making continuous verification of institutional context and data accuracy the primary challenge as agent query volumes grow.

Detailed notes — zh

## Background 本内容由 @Scale 频道于 2026 年 6 月 19 日在 YouTube 平台发布，记录了 Meta AI 与数据基础设施工程总监 Dinkar Pataballa 在 @Scale 大会上的主题演讲。演讲探讨了在 Meta 级别的数据规模与数万名决策者场景下，如何通过构建 AI 原生数据体验解决数据可信度难题，并实现免手写代码的 Agentic 数据消费与应用构建。 ## TL;DR Meta 内部在过去 6 个月内见证了 AI Agent 查询量激增 30 倍的范式转移，推动其逐步淘汰手工编写代码的传统仪表盘。为了解决“错误代码显而易见，但错误数据隐蔽无形”的信任悖论，Meta 团队历经了从编写 Prompt 脚手架、升级大模型推理，到最终依靠“工具与语义上下文”的三阶段架构演进。团队由此总结出“模型提供推理、工具提供触达、上下文提供真实”的核心法则，并推出了自然语言分析智能体 Data Mate 与交互式 Data Apps。目前该系统已支撑 Meta 约 67% 的新仪表盘创建，并赋能 60% 的非专业人员实现日常数据驱动决策，同时团队仍在攻克应对查询洪峰与让错误数据显形的前沿挑战。 ## Atmosphere & style 整场演讲呈现出典型硅谷顶尖工程架构大会的硬核、严谨与务实气质，节奏紧凑且干货密集。Dinkar 的表达沉稳自信、逻辑严密，既有直击痛点的工程洞察，又带着一线实战踩坑后的反思与坦诚。现场氛围在揭示震撼量化数据（如 30 倍查询暴增与 67% 看板替代率）时令人精神一振，而在探讨“错误数据隐形风险”时则显露出敬畏技术边界的清醒与深邃。 ## Bulletin - [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) Meta 内部在过去 6 个月内 AI 智能体查询量暴增 30 倍，正逐步淘汰纯手写代码构建仪表盘的传统模式。 - [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) 目前 Meta 约 60% 的非数据专业人员每周使用数据智能体，且新功能上线不到 4 个月便生成了全公司约 67% 的新仪表盘。 - [11:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=660s) 团队将 Agent 核心架构经验凝练为：模型提供推理能力（Reasoning），工具提供触达范围（Reach），而上下文语义模型赋予真实事实（Truth）。 - [14:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=883s) 针对大模型非确定性问题，Meta 引入 Pass per key（连续 5 次提问的一致性通过率）评测指标，强调一致性与准确性同等重要。 ## Key takeaways - [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) AI 智能体正在终结耗时繁重的传统看板开发流程，将跨工具、跨工种的工序链折叠进单次对话。 - [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) 传统模式下由数据团队人工把关而具备的隐形红利是“信任免费”，智能体时代系统必须能够自主证明答案的可信度。 - [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) 错误代码会直接运行报错，但错误数据往往表现为自信的幻觉，容易在规模化应用中直接演变为高管层的错误决策。 - [03:55](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=235s) 数据体验由“消费（自然语言提问）”与“创建（零代码生成应用）”构成闭环飞轮，其运转基础完全建立在底层数据的准确性上。 - [05:34](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=334s) AI 原生数据技术栈由底层的原始数据沼泽、语义模型真实层、可组合技能触达层以及顶层协调智能体构成。 - [05:34](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=334s) SQL 的语法正确并不代表业务语义有效，必须依靠语义模型封装业务规则与数据边界来杜绝模型幻觉。 - [07:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=476s) 数据溯源引用（Citations）在企业级分析智能体（如 Data Mate）中不仅是辅助体验，更是不可或缺的信任基石功能。 - [09:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=583s) 基于日常对话生成交互式 Data Apps 的能力颠覆了 Meta 十余年的遗留看板平台，发布不足 4 个月渗透率即达 67%。 - [11:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=660s) 在单纯 Prompt 指令脚手架或仅升级底层大模型推理的尝试中，缺乏真实上下文的模型只会更自信地给出错误答案。 - [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) 构建数据 Agent 必须聚焦于团队百分之百可控且具复利效应的要素：可组合技能与组织沉淀的语义上下文。 - [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) 数据 Agent 绝不能当作孤立玩具开发，而必须作为能承载数百万数据表与极端查询吞吐的基础设施平台来构建。 - [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) 上下文不能在真空中凭空建立，必须协同领域工程师将数十年沉淀的隐性业务口径编码进语义模型中。 - [14:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=883s) 数据 Agent 的最大价值在于打破了 SQL 与数据专长门槛，让以往从不看数的非专业人员能基于数据自主决策。 - [16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) 数据 Agent 的前沿核心挑战在于让隐蔽的错误数据能够像报错的代码一样清晰显形，并防范资产质量在自动化循环中劣变。 ## Detailed walkthrough ### [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) 数据的范式转移与仪表盘终结的宣言在会议的开场，演讲者 Dinkar Pataballa 引用了当天早些时候 Barak 的主题演讲数据：在过去短短 6 个月内，AI 智能体（Agent）的查询量暴增了 30 倍。他感叹道：“半年之内增长了 30 倍，大家细品一下（Let that sink in）。”这标志着人们使用数据的方式正在发生根本性的剧变。作为深耕数据基础设施、数据体验与平台数十年的从业者，Dinkar 回忆起以往几乎所有业务问题的起点都是同一句话：“看看我的仪表盘（Look at my dashboard）。” 基于这一背景，Dinkar 提出了一个一年前听起来极其荒谬的论断：在 Meta，他们正在逐步淘汰手写代码构建的仪表盘（wiping coding dashboards away）。为了说明这一变革的紧迫性，他举了一个每天都在 Meta 发生的经典场景：假设你是一名产品负责人（Product Lead），产品的用户参与度（Engagement）突然下降了 3%，VP 要求在今天下班前查明原因。你知道答案一定藏在海量数据的某个角落，但你根本不知道从何下手——不知道查哪个数据集、写什么查询语句，甚至不知道该打开哪款工具。在 Meta，无论是决定构建什么功能、向何处投资、产品是否上线，还是公司高管对外发布核心业务指标，所有决策都极度依赖数据驱动。当数据获取变慢，决策流程就会受阻；更糟糕的是，业务方可能会在毫无数据支撑的情况下盲目拍板。面对这一困境，过去人们唯一的出路就是：“提交一个工单，然后无尽地等待（you file a request, and then you wait）。” - **智能体查询量增长**：过去 6 个月内激增 30 倍（30x）。 - **行业痛点**：数据量虽大，但业务端获取答案极其缓慢，导致决策滞后甚至出现无数据决策。 --- ### [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) 数据悖论与信任成本的迁移 Dinkar 深入剖析了行业长期存在的数据悖论：Meta 拥有数百万个数据集、数万名决策者以及极其强大的基础设施，然而“数据存在”与“得到答案”之间的鸿沟依然巨大。过去，跨越这个鸿沟需要经历一套繁复的工序链条：从数据发现（Discovery）、编写查询（Querying）、制作可视化图表（Visualizing），到最后的数据解读（Interpreting）。链条中的每一个环节都需要不同的工具、不同的技能栈，甚至往往由完全不同的团队负责。这种摩擦真实且令人痛苦。然而，过去那套模式中有一个被所有人习以为常的隐形红利——“信任是免费的（Trust was free）”。数据团队是数据的守护者，他们深谙每个数据表背后的逻辑、局限性（Caveats）与业务上下文。虽然向他们要答案耗时长，但拿到的结果至少是可信的。而现在，智能体正在将这整条工序链折叠进单次对话中：用户直接用自然语言提问，即可在数秒内获得答案。目前在 Meta，约 60% 的非数据专业人员每周都会使用数据智能体。 ```mermaid flowchart TD accTitle: 传统数据流程与智能体交互模式对比 accDescr: 展示了传统繁琐的多阶段人工流程转变为单一对话式智能体交互的过程，强调了信任层由人工转移到系统的变化。 subgraph OLD["传统多阶段流程（耗时长、摩擦大，但团队提供人工信任）"] A["数据发现\nDiscovery"] --> B["编写查询\nQuerying"] B --> C["可视化展现\nVisualizing"] C --> D["业务解读\nInterpreting"] D --> E["数据团队人工校验\n（信任是免费的）"] end subgraph NEW["现代智能体模式（即时响应，单次对话）"] F["自然语言提问\nPlain Language"] --> G["数据智能体\nData Agent"] G --> H["即时答案 + 数据源引用\n（系统必须自证信任）"] end ``` 这种提效带来了更深层的隐患。当中间的人工环节被剥离后，信任不再免费。Dinkar 强调了一种令人担忧的对称性失衡：“错误的代码是显而易见的，但错误的数据往往隐蔽无形（wrong code is visible, wrong data is not）。”有 BUG 的代码会直接抛出运行异常，而一个被模型自信输出的错误数据，却会堂而皇之地直接进入高管会议并促成错误决策。当每天有成千上万个问题通过智能体流转时，这种错误会在规模化应用中产生滚雪球般的信任崩塌风险。 - **非专业人员渗透率**：Meta 内部约 60% 的非数据专业人员每周使用数据智能体。 - **核心风险**：代码错误直接报错，错误数据则会包装成“自信的幻觉”，直接演变为错误的业务决策。 --- ### [03:55](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=235s) 双轮飞轮：消费与创建 Dinkar 介绍了自己的背景：作为 Meta AI 与数据基础设施工程总监，他的团队负责构建支持全公司员工日常处理数据的平台与体验。在过去一年里，团队的核心精力都投入在这些系统的智能体化转型（Agentic Transformation）上。通过对全公司使用习惯的观察，团队将数据体验抽象为两大核心维度： 1. **消费流程（Consumption）**：任何普通员工能否直接提问，并即时获得可信、精准的答案？ 2. **创建流程（Creation）**：能否在不写一行查询语句的前提下，直接基于该答案构建出可共享的分析制品或轻应用？这两个维度构成了一个相辅相成的飞轮效应：用户提出问题并获得答案，将其打包为应用/仪表盘分享给团队；他人看到后产生新的衍生问题，从而激发新一轮消费。然而，整个飞轮运转的关键支点在于结果的正确性——如果底层答案错误，分享出去的就是错误的仪表盘，进而误导整条业务线的决策。 ```mermaid flowchart LR accTitle: 数据消费与创建的闭环飞轮 accDescr: 展示了用户从消费数据提问到创建分享应用的良性循环，核心依赖底层的真实与可信。 A["数据消费 (Consumption)\n自然语言提问并获取答案"] --> B["数据创建 (Creation)\n零代码生成可分享应用/看板"] B --> C["团队协同\n他人查看并产生新疑问"] C --> A D["底层基石：真实与信任 (Trust & Accuracy)"] -.->|支撑运转| A D -.->|支撑运转| B ``` - **数据飞轮模型**：消费（自然语言问答）与创建（零代码生成共享制品）相互驱动。 - **飞轮脆弱点**：高度依赖答案的准确性；错误的数据制品会导致决策失误成倍扩散。 --- ### [05:34](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=334s) AI 原生数据技术栈：真实（Truth）与触达（Reach）为了解决信任难题并支撑上述飞轮，Meta 构建了自底向上的 AI 数据技术栈（AI Data Stack）。Dinkar 将最底层未经治理的原始数据形象地称为“数据沼泽（Data Swamp）”。在数据沼泽之上，团队构建了关键的基础设施层，将整个技术栈分为上下文层、技能层与调度层： ```mermaid flowchart TD accTitle: Meta AI 原生数据技术栈架构 accDescr: 展示了自底向上的四层架构，包括数据沼泽、上下文真实层、可组合技能触达层以及顶层协调智能体。 Agent["顶层：智能体与编排器 (Orchestrator / Agent)\n负责逻辑推理与工具协调调度"] subgraph Reach["可组合数据技能 (Composable Data Skills) - 扩展触达能力"] Skills["跨数百万目录检索 / 模式消歧 / 语义有效 SQL 生成 / 可视化推荐"] end subgraph Truth["上下文与语义基础 (Context / Truth) - 确保业务真实"] Context["语义模型 (Semantic Models) / 指标定义 / 业务规则 / 验证过的查询 / 限制边界"] end Raw["底层：原始数据沼泽 (Data Swamp / Raw Data)"] Raw --> Context Context --> Skills Skills --> Agent ``` Dinkar 强调，顶层的智能体本身只是负责推理与任务协调，它表现的上限完全受制于底层提供的“触达（Reach）”与“真实（Truth）”能力： - **真实（Truth / 结构化上下文）**：核心是语义模型（Semantic Model）。它并不是魔法，而是一套严谨的结构化声明，例如明确定义某张表中的 `revenue` 或 `engagement` 列具体代表“扣除特定端外流量后的净广告互动/收入，统计周期为日频/周频”，同时封装已知的数据限制与经受检验的黄金查询。 - **触达（Reach / 可组合数据技能）**：赋予智能体在数百万数据资产目录中准确定位、解析并消除表模式（Schema）歧义的能力，确保生成的 SQL 不仅仅在语法上合规，更在业务语义上完全正确。这些能力被封装为可调用的独立工具（Tools），而非直接塞进智能体 Prompt 的静态人设指令。 - **评测框架（Evaluation Framework）**：在整个技术栈之上运行，持续监控并校验智能体输出结果的准确率。 - **核心概念区别**：语法正确（Syntactically correct）不等于语义正确（Semantically valid）；语义模型是规避幻觉的关键。 - **工具化设计**：数据技能以独立 Tool 形式挂载，确保智能体模块化解耦与可扩展性。 --- ### [07:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=476s) 消费落地实战：Data Mate 与引用机制在消费端落地时，表面上交互看似极简——用户输入自然语言，系统返回带引用的答案，但后台必须解决极具挑战性的技术壁垒：智能体需要在数百万目录中精准召回，在 15 种命名类似的收入或活跃度指标源中根据上下文区分业务含义，并生成符合语义的 SQL。更重要的是，“它必须知道自己不知道什么，并在不确定时主动说明（knows what it doesn't know, and says so）”。 Dinkar 重点指出了系统界面的“数据引用（Citations）”功能。在企业级智能体中，数据源引用绝不是一个锦上添花的功能（nice-to-have），而是一项不可或缺的“信任功能（Trust feature）”。以 Meta 内部的分析智能体产品 **Data Mate** 为例： - 用户提出自然语言问题后，系统不仅返回精准数值，还在其右侧附带可追溯的引用来源，支持钻取回源。 - 智能体同时提供衍生业务洞察（Insights）及后续行动建议（Recommendations）。 - 针对前文提到的产品负责人遭遇 3% 参与度下滑的场景，过去需要走流程提工单等待数天，现在几秒钟内便能获取带有完整追溯链路的严密答案。在此基础上，Dinkar 顺势引出了另一半重点——创建（Creation）。在旧模式下，产品负责人若想将这一分析结论沉淀并分享给团队，往往需要再次向数据团队提工单请求制作看板；而在全新的智能体架构下，整个仪表盘构建的逻辑正在被彻底重构。 - **核心产品**：Meta 内部数据分析智能体 Data Mate。 - **溯源机制**：每项统计数据直接绑定数据源表与逻辑链路，彻底消除“黑盒答案”。 ### [09:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=583s) 从数周繁琐等待到数分钟构建：AI 原生 Data Apps 的普及在过去的传统模式下，业务人员向数据工程团队提出一个看数或仪表盘需求后，工程师需要接单并着手搭建，整个流程往往要耗费数天甚至数周时间。一旦业务端提出微调需求，整个提单、排期、修改的流程又得全套重来一遍，其中充斥着大量重复且繁重的纯人力劳动（human toil）。而现在，用户只需要用日常的大白话描述自己的诉求，AI Agent 就能自动生成生产级别的交互式数据应用（Data Application）。这绝非概念验证或 Demo 原型，而是 Meta 内部正在日常真实运转的生产系统。演示界面的左侧是自然语言输入窗口，用户用平实语言表达意图；右侧则即时渲染出完整的交互式看板。Agent 会全流程自主搞定数据发现、确定取数查询逻辑、设计可视化图表，并排出清晰合理的信息布局。用户还可以与 Agent 进一步互动磨合（"vibe with it"），调整出最契合自身叙事需要的数据洞察体验。最令人震惊的数据在于，这项能力上线不到 4 个月，Meta 内部便有约 67% 的新仪表盘是通过这种 Data Apps 方式创建的，彻底颠覆并替代了公司沿用十余年的老一代遗留看板平台。数月之内，积攒了十年的用户习惯被迅速重塑。 - **上线周期与渗透率**：发布不足 4 个月，Meta 全公司约 67% 的仪表盘创建工作已转移至 AI 原生数据应用。 - **全自动流程闭环**：Agent 覆盖数据检索、SQL 查询编写、图表渲染与页面布局全流程，彻底终结了以往需数周等待的人力瓶颈。 --- ### [11:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=660s) 攻克信任难关的三阶段演进：从指令脚手架到语义事实面对这种爆发式增长，团队在夜深人静时真正焦虑的核心问题是：我们到底能不能信任这些 Agent 给出的答案？能否放心地将它们开放给全员日常使用？为了找到肯定的答案，Meta 团队历经了三个认知与架构阶段，并在这个过程中踩过了不少坑： - **第一阶段（工作流脚手架 / Workflow Scaffolding）**：团队最初编写了数千行详细的指令 Prompt，试图手把手教 Agent 如何扮演一个合格的数据分析师——从如何找表、如何理解 Schema 到如何处理各种边界情况。这就像给一个从未见过 Meta 数据的新人硬塞一本操作手册。系统虽然能跑起来，但极度脆弱（brittle），很快就触碰到了性能天花板。 - **第二阶段（推理能力升级 / Upgrading the Reasoning）**：随着行业底层大语言模型性能的演进，团队接入了推理能力更强的模型。Agent 确实变得更聪明了，准确率短期内有所上涨，但随即再度停滞不前。团队意识到一个严酷事实：**在错误的数据上施加更好的推理，只会用更自信的口吻给出错误的答案**。面对库里可能存在的 15 张不同的收入（revenue）或互动度（engagement）数据表，模型根本不知道该选哪一张、该如何解读口径，结果只能凭运气挑一张，然后自信满满地交付错误结论。 - **第三阶段（工具与上下文事实 / Tools & Context）**：这是带来质的飞跃（step function change）的转折点。团队不再试图在 Prompt 里事无巨细地指挥 Agent“如何做”，而是转而为它提供两样东西：具备充足覆盖范围的工具与能力（reach），以及通过上下文（context）所赋予的数据真实定义（truth）。团队将这一阶段沉淀的核心哲学凝练为一句话并在内部反复强调：**“模型赋予推理能力，工具赋予触达范围，而上下文赋予真实事实。”（Models give you the reasoning. Tools give you the reach. The context gives you the truth.）** 团队很早就拥有了模型的推理能力，之前真正缺失的正是触达工具与语义事实。 ```mermaid flowchart TD accTitle: Agentic Data Evolution Phases accDescr: 展现 Meta 团队探索 AI 数据 Agent 可信度的三个发展阶段及核心瓶颈与突破。 subgraph Phase1["第一阶段：指令脚手架 (Scaffolding)"] A["手写数千行详细 Prompt"] --> B["脆弱、迅速触达准确率上限"] end subgraph Phase2["第二阶段：升级模型推理 (Reasoning)"] C["换用更强底层大模型"] --> D["缺乏上下文导致'自信地给出错误答案'"] end subgraph Phase3["第三阶段：能力 + 上下文 (Tools & Context)"] E["模型提供 Reasoning (推理)"] F["工具提供 Reach (能力与触达)"] G["上下文提供 Truth (语义事实)"] E & F & G --> H["实现阶跃式准确率与高一致性"] end Phase1 --> Phase2 Phase2 --> Phase3 ``` --- ### [12:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=763s) 规模化构建数据 Agent 的三大核心法则基于上述惨痛教训与成功经验，演讲者为所有致力于规模化构建数据 Agent 的团队总结了三条关键建议： 1. **聚焦投资于自身可控的要素（Invest in what you can control）**：底层模型和测评框架会持续演进，但这并不受业务团队自身掌控。团队唯一能百分之百掌控的，是自身构建并沉淀的能力（Skills）与上下文（Context）。与脆弱的 Prompt 脚手架不同，对能力与上下文的投入是具有复利效应的（compounds）——团队每增加一项工具技能、每编码沉淀一个语义模型，平台上的每一个 Agent 都会随之变得更强。 2. **打造成基础设施工程，而非孤立的 Agent 玩具（Infrastructure play, not an agent play）**：今天涌现出一些数据 Agent，明天会有更多，未来还会出现各类主动式入口（proactive entry points）。所有这些 Agent 底层必须依托于一个统一、可信的基座平台，能够承载 Meta 级别极高的数据复杂性，并扛住恐怖的查询吞吐量。面对数百万张数据表、模棱两可的业务口径以及动态变化的 Schema，这本质上是一个高难度的底层系统扩展（scaling）问题。 3. **切忌脱离实际闭门造车（Do not build context in a vacuum）**：上下文无法在真空中凭空建立。企业内部的数据科学家与数据工程师积淀了数十年的组织级专业经验。例如在 Meta，“Instagram 的 Revenue”与“广告业务线（Ads）的 Revenue”底层定义截然不同。平台必须与这些领域专家紧密合作，将这些隐性行业认知固化编码进语义模型（semantic models）中，使 Agent 无需盲猜口径，而是直接继承专家经验。 --- ### [14:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=883s) 评测基准突破与非专业用户的全面赋能为了打消外界的疑虑，演讲者展示了跨越这三个演进阶段的量化评测数据。团队不仅关注标准企业数据基准测试上的通过率（pass rate），还专门引入了一个衡量稳定性的新指标——**Pass per key**（单键通过率/单题复现一致性）。由于大语言模型天然具备非确定性，该指标专门测试同一个问题被连续问 5 次时，系统是否能 5 次都给出完全相同的答案。结果显示，在引入正确的工具与语义上下文后，答案的一致性实现了质的飞跃。对于建立组织信任而言，**“回答的一致性与准确性同等重要”（consistency matters as much as accuracy）**。在业务影响层面，Meta 内部目前有 60% 的非数据专业人员每周都在使用这一系统，约 67% 的仪表盘都由该平台生成。然而演讲者强调，自己最为看重的一个指标并未写在 PPT 上：**那些以往从来不碰数据的人，现在开始主动提出数据问题，并在日常工作中基于数据做出明智决策**。数据驱动决策终于不再被技术专长或 SQL 门槛所阻拦。 --- ### [16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) 前沿挑战：让错误数据如同错误代码般显形现阶段的成功并不意味着难题已经被攻克，反而将挑战推向了更深水区： - **信任边界的复合扩散**：随着更多数据集接入、新用例落地，针对数据新鲜度（freshness）、数据所有权归属（ownership）以及系统稳定性的要求各不相同，信任风险会呈复合式放大。 - **查询规模的 30 倍暴增**：过去是人类手动提问看数，数据仓库承受的是人类节奏的负载；如今 Agent 全面接管人类工作，后台查询量直接飙升了 30 倍之多，数据仓库必须进化到足以承载这一量级的并发与吞吐。 - **产物质量的失控风险**：Agent 自动生成应用，应用引发更多深层问题，Agent 再去回答并生成更多用于讲述数据故事的应用。当这一链条以指数级规模自动运转时，必须时刻监控并防止生成资产的质量劣变。 Meta 内部常说“旅程才刚刚完成 1%”（The journey is only 1% finished），在这个前沿领域更是如此。演讲最后回扣了开篇的核心命题：**“错误的代码是显而易见的，但错误的数据往往隐蔽难察。”（Wrong code is visible. Wrong data is not.）** 数据 Agent 的真正前沿，绝不仅仅在于消除交互摩擦或提高回答速度，而是在于建立强大的机制，让隐蔽的错误数据能够像报错的代码一样清晰显形。在 Meta 级别的数据规模与复杂性下实现这一点，正是团队正在全力攻克的艰难而核心的事业。 ## Notable quotes > [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) 半年之内增长了 30 倍，大家细品一下（Let that sink in） — Dinkar Pataballa > [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) 看看我的仪表盘（Look at my dashboard） — Dinkar Pataballa > [00:05](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=5s) 提交一个工单，然后无尽地等待（you file a request, and then you wait） — Dinkar Pataballa > [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) 信任是免费的（Trust was free） — Dinkar Pataballa > [01:50](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=110s) 错误的代码是显而易见的，但错误的数据往往隐蔽无形（wrong code is visible, wrong data is not） — Dinkar Pataballa > [07:56](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=476s) 它必须知道自己不知道什么，并在不确定时主动说明（knows what it doesn't know, and says so） — Dinkar Pataballa > [11:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=660s) 在错误的数据上施加更好的推理，只会用更自信的口吻给出错误的答案 — Dinkar Pataballa > [11:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=660s) 模型赋予推理能力，工具赋予触达范围，而上下文赋予真实事实。（Models give you the reasoning. Tools give you the reach. The context gives you the truth.） — Dinkar Pataballa > [14:43](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=883s) 回答的一致性与准确性同等重要（consistency matters as much as accuracy） — Dinkar Pataballa > [16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) 旅程才刚刚完成 1%（The journey is only 1% finished） — Dinkar Pataballa > [16:00](https://www.youtube.com/watch?v=BQf0TxsOMUg&t=960s) Wrong code is visible. Wrong data is not. — Dinkar Pataballa ## Mentioned Dinkar Pataballa, Barak, Meta, Data Mate, Instagram, Ads, @Scale Conference ## Video description Every major business decision is ultimately built on data, yet getting to the right answers has long required specialized expertise, from knowing which tables to query to how to query them to building the right data applications. AI agents are fundamentally changing that equation, making data accessible to anyone who can ask a question in plain language. At Meta's scale, with millions of datasets serving tens of thousands of decision-makers, this shift creates both massive opportunity and unique challenges around trust and accuracy. This presentation will detail how we built AI-native data experiences to address two key dimensions: enabling trusted answers through agentic data consumption, and letting users create shareable agentic data applications without writing a single query. Want to learn more about the @Scale Conference? Visit our website here: atscaleconference.com [Source](https://www.youtube.com/watch?v=BQf0TxsOMUg)

Source quotes

[0:05] Dinkar Pataballa: 半年之内增长了 30 倍，大家细品一下（Let that sink in） [0:05] Dinkar Pataballa: 看看我的仪表盘（Look at my dashboard） [0:05] Dinkar Pataballa: 提交一个工单，然后无尽地等待（you file a request, and then you wait） [1:50] Dinkar Pataballa: 信任是免费的（Trust was free） [1:50] Dinkar Pataballa: 错误的代码是显而易见的，但错误的数据往往隐蔽无形（wrong code is visible, wrong data is not） [7:56] Dinkar Pataballa: 它必须知道自己不知道什么，并在不确定时主动说明（knows what it doesn't know, and says so） [11:00] Dinkar Pataballa: 在错误的数据上施加更好的推理，只会用更自信的口吻给出错误的答案 [11:00] Dinkar Pataballa: 模型赋予推理能力，工具赋予触达范围，而上下文赋予真实事实。（Models give you the reasoning. Tools give you the reach. The context gives you the truth.） [14:43] Dinkar Pataballa: 回答的一致性与准确性同等重要（consistency matters as much as accuracy） [16:00] Dinkar Pataballa: 旅程才刚刚完成 1%（The journey is only 1% finished） [16:00] Dinkar Pataballa: Wrong code is visible. Wrong data is not.

Source quotes

[0:05] Dinkar Pataballa: Look at my dashboard. [1:50] Dinkar Pataballa: Trust was free. [9:43] Dinkar Pataballa: vibe with it [10:56] Dinkar Pataballa: can we trust what these agents are doing? Can we trust the answers? Can we give them for general grade usage? [10:56] Dinkar Pataballa: better reasoning over wrong data just gives the wrong answers, delivering just more confidently. [10:56] Dinkar Pataballa: Models give you the reasoning. Tools give you the reach. The context gives you the truth. [12:43] Dinkar Pataballa: Invest in what you can control. [12:43] Dinkar Pataballa: Treat it as an infrastructure play, not an agent play. [12:43] Dinkar Pataballa: Never build context in a vacuum. [14:43] Dinkar Pataballa: For trust, consistency matters as much as accuracy. [16:00] Dinkar Pataballa: the journey is only 1% finished [16:00] Dinkar Pataballa: Wrong code is visible. Wrong data is not.

Original transcript

[0:05] Hello folks. I hope you've enjoyed all [0:07] the sessions today. [0:08] This is the last session, but for a [0:10] moment I want to go back to the start of [0:11] the day. [0:12] In Barack's keynote, he talked about a [0:14] 30x increase in agent queries [0:17] over the last 6 months. [0:19] 30 times in half a year. [0:21] Let that sink in. [0:23] Something significantly is changing on [0:25] how people use data. [0:27] I've spent my career building data [0:29] infrastructure, data experiences and [0:30] platforms. [0:31] And for decades, I've seen almost every [0:33] business question start the same way. [0:37] Look at my dashboard. [0:38] Today, I want to make a bold claim that [0:40] would sound absurd a year ago. [0:43] We're wiping coding dashboards away at [0:45] Meta. [0:46] Now, before I show you how, [0:48] let me ground it in a moment that plays [0:51] out many times a day. [0:53] Imagine you're a product lead. [0:55] Engagement dropped 3% and your VP wants [0:57] to know why by the end of the day. [0:59] You know that the answer exists [1:00] somewhere in data, [1:03] but you don't know where. [1:04] Where do you even start from? [1:06] You don't know which data set, which [1:07] query, which tool to open. [1:10] And this isn't about just for the dips. [1:12] At a place like Meta, every data [1:14] decision, [1:15] for every product decision, [1:17] we use data. [1:18] Whether it's what to build, where to [1:20] invest, whether to launch a product or [1:22] not, whether something's working or not. [1:24] But not only that, but even at the [1:26] company's leadership level, [1:28] some of the critical decisions around [1:30] where do we take the company forward, [1:32] how do we talk about our numbers [1:33] externally, it's all backed by data. [1:36] So, when the data access is slow, the [1:38] decision-making is also slow. [1:40] Or worse, they get made with no data at [1:42] all. [1:43] So, you can only do one thing you can, [1:44] which is [1:46] you file a request, and then you wait. [1:50] Now, this is the data paradox that the [1:52] industry has lived for many years. [1:54] Here at Meta, we have millions of data [1:55] sets, [1:56] tens and thousands of decision makers, [1:58] and a very powerful infrastructure. Yet, [2:01] the gap between the data exists [2:03] to I have my answer is enormous. [2:06] Getting there meant a chain of work, [2:08] starting from discovering the data, [2:10] querying it, visualizing it, [2:12] and then interpreting it. [2:14] Every link was a different tool, [2:16] a different skill set, sometimes and [2:18] many oftentimes actually as a different [2:20] team. [2:21] So, that friction is real, it was real. [2:24] It is painful. [2:25] But, here's what we took completely [2:28] for granted. [2:29] Trust was free. [2:31] You went to the data team. [2:32] They are the shepherds of the data. They [2:34] knew the tables. They knew the caveats. [2:36] They knew the context. [2:38] The answer took time, [2:39] but you could at least trust it. [2:41] Hold on to that for a second. [2:43] Because that's changing. [2:45] Agents are collapsing that entire chain [2:47] into a single conversation. [2:49] You ask a question in plain language, [2:51] and you get your answer. [2:52] No more waiting. [2:54] And the people are also voting their [2:56] behavior on this. Today, around 60% of [2:59] non-data experts [3:00] use our data agents on weekly basis. [3:03] People who never wrote a query in their [3:05] life are now getting answers for their [3:08] data questions in seconds. [3:10] This is a win, and it's a real one. [3:13] But, [3:14] solving access reveals a harder problem. [3:17] When you remove the human in between, [3:19] trust is no longer free. [3:21] The data team was the trust layer. [3:23] When an agent hands you an answer [3:25] totally confidently, [3:27] how do you know that it's actually doing [3:28] the right thing? [3:30] And the symmetry that we should all be [3:31] worrying about is that wrong code is [3:33] visible. [3:34] Wrong data is not. [3:37] A broken query will throw an error. [3:39] But, a confident wrong answer just [3:41] becomes a decision that's being made in [3:42] a meeting. [3:44] Now, multiply that by hundreds and [3:46] thousands of questions and queries that [3:47] are coming every day. [3:49] We now have a compounding [3:51] trust problem at scale. [3:55] I'm Dinkar Bharabola. I'm an engineering [3:56] director in AI and data infra. [3:59] My teams support and build platforms and [4:01] experiences that everyday people at Meta [4:04] use to act on their data. [4:06] For the past 1 year, our central focus [4:08] has been around agentic transformation [4:11] of these platforms and experiences. [4:14] So, everything that I'm about to show [4:15] you and talk about is something we've [4:17] been living. [4:20] So, the agenda for the next 15 minutes, [4:22] we're going to talk about three things. [4:24] At the very first, we're going to talk [4:25] about how we're building these data [4:26] agents, which are trusted at Meta scale. [4:29] Second, we're going to talk about what [4:30] has genuinely surprised us while [4:32] building along the way. [4:33] I'm also going to talk about a few [4:34] things that went wrong. [4:37] And finally, where this is all headed. [4:39] Because the most interesting problems [4:41] are still ahead of us. [4:43] Okay, when we looked at the people how [4:45] people are actually using data, it came [4:47] down to two dimensions. [4:49] Number one, consumption. Can anyone ask [4:51] a question and get direct trusted [4:53] answers right away? And then there's a [4:55] creation flow, which is can anyone build [4:57] something shareable from that answer [5:00] without writing a single query? [5:01] Now, these two feed into each other. [5:03] You ask a question, you get an answer, [5:05] share it via some app, and someone else [5:07] sees it, and then they have the next [5:08] question. [5:09] So, it's a flywheel from consumption to [5:12] creation and back. [5:13] But this whole with flywheel has one [5:15] strong dependency. [5:18] It only works if the answers are [5:20] correct. A wrong answer becomes a wrong [5:22] dashboard shared with the with the team, [5:24] which is again then driving wrong [5:26] decisions for themselves. [5:28] And this same underlying challenge [5:29] exists across, which is about trust. [5:33] So, [5:34] let me show you the platform that powers [5:36] both these both of these dimensions. [5:38] The AI data stack. [5:40] I want to build it in the way we [5:41] actually think about it from ground up. [5:44] At the very bottom, there is the raw [5:45] data. This is what I call as data swamp. [5:48] Now, but on the of this raw data sits [5:50] the foundation with context. [5:52] This is the layer that tells the system [5:54] what the data actually means. [5:56] The semantic models, the metric [5:58] definitions, the business rules, the [6:00] caveats, and nothing about this can be [6:03] trusted without it. So, the truth is the [6:05] first thing we lay down. [6:07] Sitting on top of the context is what I [6:09] call as our composable data skills. Now, [6:12] these give the agent the reach. [6:14] The ability to discover the right data [6:16] sets, [6:17] make sense of the schema, generate a [6:19] valid query, figure out how to visualize [6:21] it, [6:22] and many others data skills across the [6:24] ecosystem. [6:25] And then finally, on top of all of this, [6:27] at the very end sits the orchestrator or [6:29] the agent. Now, the agent reasons, it [6:32] coordinates, of course, but notice what [6:35] it's standing on. [6:36] The agent is only as good as the reach [6:38] and the truth that sits underneath it. [6:41] One platform built foundation up [6:43] covering both the consumption and [6:45] creation flows. [6:47] We'll go a little bit deeper into the [6:48] stack. [6:50] But first, I wanted to go and make what [6:52] reach and truth a bit more concrete. [6:55] Truth is the structured context, like I [6:57] said. At the very heart of it is a [6:58] semantic model. [7:00] A semantic model isn't magic. It's [7:01] basically a structured statement that [7:03] says, "This particular column called [7:05] revenue or engagement in this particular [7:06] table means net advertising engagement [7:09] or revenue for this particular surface [7:11] excluding these other surfaces computed [7:13] daily, weekly, and so on and so forth." [7:16] In addition to that, it also includes [7:17] the metric definitions, the verified [7:19] queries, the business rules, and all of [7:21] the other known caveats. [7:24] And then for the composable data skills, [7:25] which is the reach, is basically [7:28] discovering an ability to discover the [7:29] right data sets from across millions of [7:31] catalog entries, [7:33] understanding and disambiguating the [7:35] schemas, [7:36] generating queries that are semantically [7:38] valid, not just syntactically correct, [7:40] and each one is a tool that the agent [7:41] invokes. It's not an instruction that is [7:43] baked into the agent persona. [7:45] Now, on top of all this is an evaluation [7:47] framework that's looking at whether [7:49] we're doing the right whether we're [7:50] giving the right answers for the [7:52] questions that are coming in. [7:56] Now, [7:57] further along, like seeing this in [7:58] action. On the consumption side, let's [8:00] start with that. On the surface, it [8:01] looks almost very trivial, right? You [8:03] ask a question, you get an answer, of [8:05] course with citations. [8:07] But underneath that surface, the agent [8:09] is doing something genuinely hard. [8:12] It's navigating millions of catalog [8:13] entries to find the right data set. [8:15] It's distinguishing between 15 different [8:17] sources for revenue or engagement or [8:19] whatever you're looking at. [8:21] It's understanding that engagement here [8:23] means different things based on the [8:24] context that the user is asking for. [8:26] And then it's generating a sequel that [8:28] is semantically correct. [8:29] And more critically, it also knows what [8:31] it doesn't know, and it says so. [8:34] One detail I want you to notice here is [8:36] that every answer links back to the [8:38] source data. [8:40] The citation here is not a nice-to-have. [8:42] It's a trust feature. [8:44] It's how you check that the agents are [8:46] doing what they need to do. [8:49] Now, showing this in practice. [8:52] Data made is one of the data analytics [8:53] consumption agent in the company. [8:55] Here what you're seeing is a question [8:57] comes in in plain English, and here's [8:59] the answer coming back with citations [9:01] right alongside it. Now, with those [9:03] citations you can trace exactly where [9:05] every number came from. [9:06] In addition to that, it gives insights [9:08] and rough recommendations on what you [9:10] can do next. [9:11] Now, remember the product lead question [9:13] with that we started this with. [9:15] When a 3% engagement drop question comes [9:17] up, [9:18] the difference in the from the past to [9:19] now is instead of filing a request and [9:21] waiting for days, [9:23] you have a traceable answer in front of [9:24] you in seconds. [9:28] Let's go to the other half of creation. [9:30] Remember my claim at the top that we [9:32] wiped coding dashboards away? [9:34] This is what I meant. [9:35] In the old world, if the product lead [9:36] wanted to share something about their [9:39] engagement analysis with their team, [9:41] they'd file a request, our data team [9:43] would pick that request up, build a [9:44] dashboard, and that took days, sometimes [9:46] even weeks. [9:48] Now, if they wanted a small change to [9:49] it, the whole process rinses and [9:51] repeats. [9:52] Now, there was genuine human toil in [9:54] getting it out there for production [9:56] consumption. [9:58] But, now they describe what they want [9:59] again in plain language, [10:01] the agents build a real production [10:03] quality data application. This is not a [10:05] prototype. This is something that the [10:07] company's actually running on right now. [10:11] To see this a bit in action, on the left [10:13] is basically a natural language way of [10:15] describing what you want. [10:18] And on the right, you see the rendered [10:19] interactive application. [10:21] The agent handles it all. It does the [10:22] data discovery, it figures out what [10:24] queries to do, it figures out the viz, [10:26] it also figures out the layout that [10:28] makes sense, and you can vibe with it to [10:30] figure out the right experience of a [10:32] data story that you want to tell. [10:34] And the number that surprised me the [10:35] most here is that we launched this less [10:37] than 4 months ago. [10:39] And already, around 67% of dashboard [10:42] creation in Meta happened through the [10:44] data apps, [10:45] instead of the legacy platform that we [10:47] had for more than a decade. [10:49] So, a decade worth of habit [10:51] is being transformed in a matter of [10:53] months. [10:56] Now that you've seen both sides, [11:00] for us, the question that genuinely kept [11:02] us up at night was, can we trust what [11:05] these agents are doing? Can we trust the [11:06] answers? Can we give them for general [11:08] grade usage? [11:10] Now, getting to yes for that question [11:12] took us through three phases. And along [11:14] those three phases, [11:16] we had a bunch of surprises. I'm going [11:18] to walk you through some of those in [11:19] detail. [11:21] Phase one, we scaffolded the workflow. [11:23] We wrote thousands of lines of [11:24] instructions telling the agent exactly [11:26] how to be a data analyst. [11:28] Find the right tables, how to find the [11:30] right schemas, how to handle the edge [11:32] cases. It was like giving a manual for [11:34] someone who's never seen our data. It [11:36] worked, but it was brittle. [11:38] And we hit a ceiling pretty fast. [11:41] Then we went on to the next phase of [11:43] upgrading the reasoning. [11:45] Better models arrived. [11:46] The agent got smarter. [11:48] And the accuracy also improved for a [11:49] while. [11:50] Then it plateaued again. [11:52] And we realized why. [11:53] The better reasoning over wrong data [11:55] just gives the wrong answers, delivering [11:57] just more confidently. [11:59] The model didn't know which of the 15 [12:01] different revenue or engagement tables [12:03] or metrics are the right ones to pick [12:06] and how to even interpret them. [12:08] So, it just picked one and shared the [12:10] answer confidently. [12:12] That's when we went to phase three, [12:13] which is what really clicked for us. [12:16] We stopped telling the agent how to do [12:17] things. [12:18] And we gave it the tools and [12:20] capabilities with the reach, plus the [12:22] truth about the data through context. [12:24] This was a step function change for us. [12:26] And it taught us the line that we now [12:28] repeat internally. Constantly. [12:31] Models give you the reasoning. Tools [12:33] give you the reach. The context gives [12:35] you the truth. [12:36] We'd had reasoning since day one, [12:38] what we were missing was the reach and [12:40] the truth. [12:43] So, stepping back, [12:45] there are three things that I'd like you [12:47] all to take away from here, anyone who's [12:49] trying to build data agents and at [12:50] scale. I would talk about these three [12:52] things. [12:53] Number one, invest in what you can [12:55] control. [12:56] Better models will keep coming. Better [12:57] harnesses will keep coming and they will [12:59] help. [13:00] But you don't control the improvements [13:02] to models. [13:03] What you do control is the skills and [13:05] the context that you can build and [13:06] provide. [13:07] And unlike prompt scaffolding, [13:10] the in- this investment around [13:11] capabilities and context compounds. [13:14] Every skill that you add, every semantic [13:16] model that you encode, [13:18] makes every agent on the platform [13:20] better. [13:21] Number two, there is an infrastructure [13:23] play here. It's not an agent play. [13:25] Today we see some data agents. Tomorrow [13:27] there'll be more. In future, [13:28] [clears throat] there might be more [13:30] proactive entry points that are coming [13:32] in. [13:33] But, the platform [13:35] they all sit has to be this one platform [13:37] that provides [13:38] trust of capabilities and context at [13:43] scale for the data complexity that is at [13:46] Meta. [13:48] And note, it also has to handle for the [13:50] query volume that comes in. [13:53] As the scale there'll be millions of [13:54] tables, there'll be more ambiguous [13:56] semantics, there'll be even more [13:57] evolving schemas. [13:59] This is a hard scaling problem. [14:01] And the third thing that we learned was [14:04] that you cannot build this in context [14:05] without you cannot build the context in [14:07] vacuum. [14:08] Our data scientists and engineers have [14:10] decades of institutional knowledge. [14:13] They know revenue in Instagram means [14:14] this certain thing and it's very [14:16] different from the revenue that we talk [14:17] about in the ads world. [14:20] This is what we encode into the semantic [14:22] models. [14:24] So, the agent doesn't have to guess or [14:26] interpret. [14:27] It inherits the expertise from the [14:29] people [14:30] who know and understand this data well. [14:34] This is the partnership that we've [14:35] started building with our [14:38] platform builders, the agentic platform [14:39] builders, and the domain experts. [14:43] Now, I don't want you to take any of [14:45] this with pure faith. [14:46] So, I want to share some evidence. [14:49] This is what happened to our pass rate [14:50] on the data benchmark across three [14:51] phases. [14:53] On the left is the pass rate metric that [14:54] we measure the pass rate of our agents [14:57] on standard company data benchmark. [15:00] On the right is a different metric that [15:03] we started using to double down on [15:05] accuracy. [15:06] We call this the pass per key metric. [15:08] This metric on the right shares [15:10] the consistency of answers for the [15:12] questions. So, if you ask a question [15:14] five times [15:15] do you get the same answer five times? [15:18] Models are non-deterministic by nature. [15:20] So, with the right tools and context, [15:22] we started getting consistent answers. [15:25] For trust, consistency matters as much [15:27] as accuracy. [15:31] Now, stepping back and showing this [15:33] shift is real. As I said, 60% of [15:34] non-data experts are using this every [15:36] week. [15:37] About 67% of our dashboards are now [15:39] built through data apps. [15:41] But, the number that I care about the [15:42] most is actually not on this slide. [15:44] It's that the people who were never data [15:46] people are now asking data questions, [15:49] making data-informed decisions for their [15:51] day-to-day work. [15:52] Data-driven decision-making is no longer [15:55] gated by technical expertise. [16:00] Now, [16:00] success here doesn't mean that we've [16:02] retired all the hard problems. [16:04] If anything, it actually intensifies it. [16:06] Trust now is an even bigger scale [16:08] problem. There'll be more data sets [16:10] coming in, more use cases onboarded, [16:13] different requirements around freshness, [16:14] different requirements around ownership, [16:17] reliability, all the trust and errors [16:18] around it compound. [16:21] Scale is a different challenge in [16:22] itself. [16:23] Humans were the ones who were asking [16:24] questions, so there was a certain level [16:26] of workload scale coming into the [16:28] warehouse. [16:29] But now, agents taking over and doing [16:31] the work on behalf of humans, [16:34] that scale has gone manyfold, 30x, like [16:37] I said. [16:38] So, we need a warehouse that actually [16:39] can handle that volume of queries coming [16:42] in. [16:43] And finally, quality in itself is a [16:45] compounding problem as well. [16:47] Agents create apps that surface [16:49] questions, [16:50] and that the agent tries to answer, and [16:53] then it creates more [16:54] apps to tell the data story. [16:57] Now, this happening at [16:59] exponential scale will mean that the [17:00] quality of how these artifacts get [17:02] created also is something that we need [17:05] to keep in check. [17:06] We are saying at Meta, [17:08] which is the journey is only 1% [17:10] finished. [17:11] And that is absolutely true in this [17:13] space. [17:15] Which Which brings me back to where we [17:17] all started. [17:18] Wrong code is visible. Wrong data is [17:22] not. [17:23] So, the real frontier here is not about [17:25] building agents to remove friction or [17:27] make things faster in terms of answering [17:29] questions. [17:30] It's making wrong data as visible as [17:33] code wrong code is. [17:35] And doing that at the scale and the [17:37] complexity of Meta's data is the [17:40] challenge. That's the hard and essential [17:42] work. [17:43] And we at Meta that's the frontier we're [17:45] building towards. [17:47] Thank you.