Google Embeds Native AI Agents in BigQuery to Slash Analytics Costs
Ai4
Unifying multimodal foundation models and conversational agents directly inside the data platform eliminates data movement bottlenecks while cutting token consumption by up to twentyfold through on-the-fly query distillation.
## Background
Published on YouTube by the channel Ai4 on August 15, 2026, this presentation features Google Cloud Product Manager Vaibhav Sethi. The session covers strategies for unifying structured and unstructured data analytics directly within BigQuery using Gemini, demonstrating how to build and deploy governed conversational AI agents across an enterprise.
## TL;DR
Vaibhav Sethi details BigQuery's evolution into a multimodal, agentic data platform capable of processing unstructured assets natively through `Object Ref` types and built-in SQL AI functions. He explains key capabilities including serverless ScaNN vector search, token cost governance, zero-shot time series forecasting with TimesFM, and in-context tabular learning via TabFM. Through a hands-on demonstration diagnosing product returns for a mock retailer, Sethi shows how functions like `KEY_DRIVERS`, `AI.AGGREGATE`, and multimodal `AI.GENERATE` isolate business anomalies and audit catalog discrepancies. The presentation concludes by configuring and executing persistent, conversational data agents within BigQuery Studio that turn natural language prompts into secure SQL workflows.
## Atmosphere & style
The session has the energetic, practical, and highly technical atmosphere of an advanced developer conference demo. Sethi delivers a fast-paced walkthrough with engaging authority, blending architectural diagrams seamlessly with live query consoles. The tone remains grounded and empowering, maintaining audience interest during live SQL troubleshooting with lighthearted real-world examples of humorous catalog image mismatches.
## Key takeaways
- [00:04](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=4s) Moving data between disparate systems creates massive friction for enterprise AI initiatives; data platforms must natively unify structured data, unstructured content, and governance.
- [04:03](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=243s) BigQuery handles unstructured assets natively using the typed `Object Ref` pointer to Cloud Storage, enabling SQL and Python queries to combine tabular metrics with images, documents, and audio.
- [08:42](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=522s) In-database generative AI functions like `AI.generate` execute LLM prompts across millions of rows in parallel with a 50% discount via Gemini batch inference.
- [12:05](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=725s) Native document processing functions such as `AI.chunk_document` parse and chunk thousands of PDFs directly in SQL for vector indexing and RAG workflows.
- [12:05](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=725s) BigQuery optimizes semantic operations through query distillation, training a lightweight ML model on the fly to reduce token consumption by hundreds of times.
- [16:18](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=978s) Governance tools like `AI.count_tokens` calculate token costs prior to execution, allowing administrators to enforce project-level quotas and developer limits.
- [18:34](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1114s) Serverless vector indexing leverages Google's ScaNN technology and native Gemma embeddings to deliver hybrid keyword-and-vector retrieval across billions of records without dedicated vector databases.
- [20:05](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1205s) TimesFM provides zero-shot transformer-based forecasting and anomaly detection across thousands of time series simultaneously, accessible via SQL and Connected Sheets.
- [21:51](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1311s) TabFM enables in-context learning for structured classification and regression directly within BigQuery, surpassing custom-tuned XGBoost models on evaluation benchmarks.
- [26:10](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1570s) The built-in `KEY_DRIVERS` function automates multidimensional root-cause analysis by pruning low-impact dimensional combinations in flight to pinpoint performance anomalies.
- [34:35](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=2075s) Optimized classification benchmarks demonstrate a 20x token reduction and 5x faster execution time while maintaining a 99.992% parity match with pure LLM inference.
- [38:21](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=2301s) BigQuery Studio allows domain experts to configure conversational AI agents grounded in tables, verified SQL metrics, and Knowledge Catalog glossaries for enterprise-wide deployment.
## Detailed walkthrough
### [00:04](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=4s) The Enterprise AI Bottleneck: Why the Data Platform Matters
Vaibhav Sethi introduces the session as a deep dive into BigQuery and multimodal data processing, bridging from the preceding primer into architectural concepts followed by a live demonstration. While surveys show around 92% of enterprises are rushing to deploy AI—a figure Sethi jokes "should be probably 100% at this moment"—very few organizations allocate adequate attention to their underlying data platform. Overlooking the data tier introduces severe friction into enterprise AI initiatives. When platforms force developers to continuously move data between disparate systems to run AI analytics, they introduce heavy operational barriers that slow down both engineering teams and business stakeholders.
A modern enterprise data platform must be inherently multimodal. Storing structured tables apart from unstructured media creates artificial silos, and fragmenting governance controls across isolated systems makes enterprise-wide security impossible to manage. To truly accelerate deployment, the core data platform itself must unify structured data, unstructured content, and security policies under a single engine.
[01:46](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=106s) BigQuery's Evolution: From Serverless Warehouse to Agentic Platform
BigQuery originated in 2011 as Google's serverless, exabyte-scale data warehouse—a pioneer capability that Sethi points out many market competitors still fail to match fifteen years later. Over the last decade and a half, its scope broadened far beyond simple analytical querying. BigQuery added in-database machine learning capabilities in 2019, evolved into a multicloud architecture via BigQuery Omni to query data residing outside Google Cloud, and introduced BigLake to query open table formats like Apache Iceberg and Parquet. It then incorporated object tables for unstructured assets, followed by deep native Gemini integrations, Vertex AI vector search, and direct agentic execution.
Architecturally, BigQuery functions across several integrated tiers:
- **Storage and Lakehouse Layer:** Supports BigQuery's native managed format alongside open formats such as Apache Iceberg and Parquet.
- **Serverless Core Engine:** Provides the distributed, auto-scaling analytical processing backbone.
- **Advanced Capabilities Layer:** Exposes analytics through SQL and Python, vector search, knowledge graphs, stream processing, and multimodal AI functions.
- **Developer and Agent Layer:** Delivers conversational analytics agents and integration surfaces built directly into BigQuery.
```mermaid
flowchart TD
accTitle: BigQuery Platform Architecture
accDescription: Architectural breakdown of BigQuery across the Lakehouse, Core Engine, Capability, and Developer Agent layers.
[04:03](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=243s) Unstructured Data Foundation: The `Object Ref` Data Type
In traditional data warehouses, structured tables reside completely insulated from unstructured media. BigQuery handles multimodal data natively by leaving large unstructured assets—such as audio recordings, images, or documents—inside Google Cloud Storage (GCS), while BigQuery maintains a typed pointer called an `Object Ref`. Represented as an internal struct, an `Object Ref` holds the target object's GCS URI along with associated technical and business metadata.
Because `Object Ref` is a native first-class BigQuery data type, the analytical engine and custom Python User Defined Functions (UDFs) query both the content and the metadata directly. If an asset in Cloud Storage contains custom annotation tags, users can inspect, filter, and aggregate those attributes using standard SQL. Multiple `Object Ref` columns can reside within the same table alongside standard tabular types, persisting through transformations and ETL pipelines. This allows teams to execute multimodal queries that were previously intractable, such as querying a product catalog for electronic items with high return rates while simultaneously analyzing customer-submitted return photos to identify items showing "damage on arrival."
[06:45](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=405s) Full-Stack AI and ML Inside BigQuery
BigQuery has evolved into a complete, end-to-end machine learning platform managed directly through SQL. Without moving data outside the warehouse boundaries, data teams can perform large-scale feature engineering, train classification and clustering models, evaluate performance, and run distributed, serverless batch inference with full MLOps tracking through Gemini Enterprise.
For standard operational requirements, BigQuery provides pre-trained predictive transformer models for time-series forecasting, anomaly detection, classification, and regression. Users no longer need to design custom features or train architectures from scratch for everyday analytical workloads; pre-trained foundations can be invoked immediately against the data. Beyond predictive modeling, BigQuery integrates serverless vector indexing capable of generating and searching across billions of high-dimensional embeddings, as well as specialized developer tooling such as autonomous data engineering agents designed to draft and maintain Dataform pipelines.
[08:42](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=522s) Generative AI Inference with `AI.generate` and Structured Extraction
To perform generative AI inference at scale, BigQuery exposes specialized SQL functions, highlighted by `AI.generate`. By wrapping LLM invocations in basic SQL syntax, developers can run prompts across millions of table rows in parallel. For instance, summarizing an entire corpus like the BBC news dataset requires only invoking `AI.generate` against each row's article body column to produce a concise, single-sentence summary.
Beyond open-ended semantic tasks like text summarization, enterprises frequently need to transform messy unstructured text into strict tabular formats. By passing a defined output schema (such as a JSON schema) directly into `AI.generate`, the model is strictly constrained to output structured attributes—such as parsing unstructured clinical notes into distinct `name` and `age` columns. Helper functions like `AI.generate_table` automate this process further by directly expanding generated outputs into queryable relational columns without requiring intermediate JSON parsing steps.
- **Model Choice:** While Gemini serves as the optimized native default, users can route queries to Anthropic Claude or any custom Hugging Face model deployed onto a Vertex AI Model Garden endpoint.
- **50% Batch Discount:** Queries executed via `AI.generate` run through Gemini's batch inference pipeline, slashing model inference costs by 50% compared to standard synchronous API rates.
- **Real-World Production Uses:** Enterprises actively use BigQuery AI functions for entity extraction across text and imagery, customer review sentiment scoring, synthetic content generation, and automated IT log classification and root-cause summarization.
[12:05](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=725s) Document Processing and Managed AI Functions: Semantic Joins and Distillation
Preparing unstructured documents for Retrieval-Augmented Generation (RAG) typically requires brittle external pipelines. BigQuery streamlines this into SQL using `AI.chunk_document`. The function accepts an object table pointing to a Cloud Storage bucket containing thousands or tens of thousands of PDFs, parses the documents, chunks the text, and returns clean, embeddable segments directly into a BigQuery table ready for downstream vector indexing.
```mermaid
flowchart LR
accTitle: Document Chunking to RAG Pipeline
accDescription: Flow showing how PDFs in GCS are converted to chunks in BigQuery using AI.chunk_document and fed to downstream RAG workflows.
A["GCS Bucket\n(10,000+ PDFs)"] --> B["Object Table\n(Object Refs)"]
B --> C["AI.chunk_document\n(Native BigQuery SQL)"]
C --> D["BigQuery Chunks Table"]
D --> E["Vector Embeddings & RAG"]
```
Alongside raw inference, BigQuery provides specialized managed AI functions engineered for specific semantic operations:
- `AI.if`: Executes semantic filtering and complex joins. It allows cross-modal joins that are otherwise impossible, such as joining an image table with a catalog text description table based on whether the image advertisement actually depicts the described product.
- `AI.classify`: Performs classification across text or images based solely on user-supplied labels.
- `AI.score`: Computes normalized semantic relevance scores across multiple rows concurrently rather than evaluating individual rows in isolation.
- `AI.aggregate`: Synthesizes semantic takeaways across vast numbers of rows (similar to e-commerce review rollups) using an intelligent hierarchical pattern that prevents hitting LLM context window boundaries.
For `AI.if` and `AI.classify`, BigQuery introduces an innovative optimization: query distillation. Rather than routing every individual row through a costly LLM call, BigQuery dynamically trains a classical machine learning model on the fly using the data's vector embeddings as input features. The query engine then uses this compact distilled model to execute the evaluation across the remaining dataset. By selecting the `minimize_cost` optimization mode, users achieve a "multiple hundred times reduction in the amount of tokens you spend," accelerating query speeds to thousands of rows in seconds and scaling seamlessly across billions of records.
[16:18](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=978s) Productionizing Enterprise AI: Token Governance, Streaming, and Orchestration
Deploying AI workflows into production requires strict operational governance over compute and token consumption. BigQuery provides `AI.count_tokens`, enabling developers to calculate exact token requirements prior to query execution at zero cost. Administrators can enforce programmatic token quotas at the project and user levels—such as capping a specific department at 100 million tokens per day—while developers can assign hard limits on individual experimental queries during testing before deploying them into production.
For real-time operational workloads, BigQuery's continuous queries allow developers to embed generative functions like `AI.generate` directly into continuous streaming pipelines. Engineering workflows are maintained through Dataform, which provides Git versioning, testing, and modern software development life cycle (SDLC) practices for SQL-based ETL pipelines. Complex operational directed acyclic graphs (DAGs) can be orchestrated either through BigQuery's built-in scheduling capabilities or via Cloud Composer, Google's managed Apache Airflow service.
Turning to embedding generation, BigQuery supports scalable, cost-effective Google-managed embedding models out of the box, while also allowing teams to bring proprietary or open-source embeddings from Hugging Face by linking to Vertex AI Model Garden endpoints.
[18:34](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1114s) Serverless Vector Search and Hybrid Retrieval with ScaNN
BigQuery allows smaller machine learning models, such as the Gemma class of models used for embeddings, to run directly and natively inside the data warehouse. Once embeddings are generated across billions of rows, BigQuery leverages Google's proprietary ScaNN (Scalable Nearest Neighbors) indexing and search technology—the exact same infrastructure underpinning Google consumer products serving billions of users. This vector search capability is entirely serverless, eliminating the need to provision, configure, or manage dedicated third-party vector databases and their operational lifecycles.
Tables can be set up to maintain embeddings autonomously: whenever data is mutated, inserted, or updated, corresponding embeddings refresh automatically in the background. Beyond pure vector search, BigQuery supports hybrid search. In queries combining strict keyword constraints with broader semantic meaning, BigQuery executes both exact text matching and semantic vector retrieval simultaneously, applying smart reranking algorithms to return the highest-relevance output.
- **Native Gemma Embeddings:** Runs smaller embedding models directly in BigQuery without external API round-trips.
- **ScaNN Technology:** Scalable Nearest Neighbors index architecture handling billions of rows.
- **Fully Serverless:** No external vector database instances or lifecycle management required.
- **Autonomous Embeddings:** Automatic recalculation as underlying table data changes.
- **Hybrid Search & Reranking:** Merges exact keyword matching and semantic vector scoring into a single unified result.
---
[20:05](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1205s) Zero-Shot Forecasting and Anomaly Detection with TimesFM and Connected Sheets
Large language models represent only one facet of analytical intelligence in BigQuery; foundational models designed for numerical patterns are equally critical. Google integrates TimesFM, a zero-shot, transformer-based foundation model trained specifically to predict time series data rather than language tokens. Because it is pre-trained, TimesFM requires no bespoke customer training or fine-tuning: users supply historical records, specify a future prediction horizon, and can forecast hundreds or thousands of separate time series simultaneously in a single SQL query. For instance, a retailer managing numerous physical stores and thousands of individual SKUs can project demand 30 days into the future without spinning up specialized infrastructure.
TimesFM powers both historical anomaly detection and real-time outlier flagging on incoming data streams. Recognizing that many business users operate primarily in spreadsheets rather than SQL consoles, Google exposes these enterprise capabilities directly within Google Sheets through Connected Sheets. Formulas executed inside the spreadsheet push down computation to BigQuery, letting spreadsheet users run large-scale time series forecasting and anomaly detection on massive cloud data tables transparently.
[21:51](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1311s) Introducing TabFM for In-Context Tabular Classification and Regression
Building upon the transformer principles of TimesFM, Google introduced TabFM to handle structured classification and regression directly within BigQuery. Rather than requiring data science teams to manually train, tune, and evaluate traditional gradient-boosted decision trees (like XGBoost) on new labeled datasets, TabFM relies on in-context learning. Through a straightforward `AI.PREDICT` statement, users provide reference training data alongside unlabelled inference rows and target labels, executing the entire predictive pipeline natively inside BigQuery.
In published benchmark evaluations measuring model accuracy across tabular tasks, TabFM achieves superior Elo ratings compared to existing alternatives. It outscores traditional tree-based models custom-trained on target datasets as well as state-of-the-art pre-trained tabular foundation models out of the box, offering a frictionless path for fraud detection and risk scoring without dedicated machine learning pipelines.
- **In-Context Learning:** Eliminates iterative model training loops by passing training exemplars and inference rows into `AI.PREDICT`.
- **Classification and Regression:** Generalizes tabular prediction across discrete label and continuous target use cases.
- **State-of-the-Art Elo Scores:** Outperforms both custom-tuned decision tree models (e.g., XGBoost) and competing pre-trained tabular models on standard evaluation benchmarks.
---
[23:18](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1398s) Conversational Analytics, Agents in BigQuery, and the Data Agent Kit
Recognizing that "nobody likes to write code these days, especially when you're working with simple analytical queries," BigQuery includes conversational analytics natively within the BigQuery console. Users can define a targeted "bounding box"—restricting context to specific tables, schemas, and datasets—and engage in natural language chat. The agent translates user questions into secure SQL queries executed under the user's existing IAM credentials. Power users can configure domain instructions and schema context, then publish and share agents across the enterprise, including direct deployment into Gemini Enterprise.
Conversational agents in BigQuery are not limited to SQL `SELECT` aggregations; they are wired directly into underlying ML and multimodal primitives. A user can prompt the chat agent to categorize unstructured customer search queries, and the agent autonomously structures an `AI.CLASSIFY` invocation or applies TimesFM and TabFM models for predictive requests. For software engineers, Google offers the Data Agent Kit, featuring a VS Code extension with a data catalog browser, development tooling, and specialized agent skills that teach AI assistants how to orchestrate BigQuery, Apache Spark, and other GCP services across coding environments like Codex, Antigravity, and Cloud Code CLI.
```mermaid
flowchart LR
accTitle: Conversational Analytics and Data Agent Ecosystem
subgraph AgentDef["BigQuery Agent Studio"]
BB["Define Bounding Box\n(Tables, Schemas, Permissions)"] --> Agent["Conversational Agent"]
Agent --> ExtCap["AI Primitives\n(AI.CLASSIFY, TimesFM, TabFM)"]
end
[26:10](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1570s) Hands-on Demo: Diagnosing Cymbal Pets' Spike in Returns with `KEY_DRIVERS`
To demonstrate these multimodal and agentic capabilities, a real-world scenario centers on a business manager at "Cymbal Pets", an e-commerce retailer selling pet accessories. The business encounters a sudden spike in monthly product returns: monthly return volume jumps from a baseline of roughly 1,500 items up to 2,000 items. Standard operational practice requires tedious manual slicing across dozens of dimensional combinations (brands, categories, geographies) to identify the driver behind the delta.
BigQuery automates this multidimensional root-cause analysis through the built-in `KEY_DRIVERS` analytical function. Users supply the target metric (`return_count`) alongside an array of categorical dimensions (`brand`, `category`, `product_name`). Rather than experiencing the exponential combinatorial explosion typical of manual pivot tables, the underlying algorithm prunes low-impact permutations in flight. Executing `KEY_DRIVERS` isolates the primary culprit immediately: the "Cat Costume" brand within the "Apparel" category accounts for the overwhelming majority of excess returns, giving the team their initial "smoking gun."
- **Data Tables Used:**
- `catalog`: Product metadata, brand, category, product descriptions, and product image references.
- `returns`: Return timestamps, order/product IDs, and unstructured return comment text.
- `support`: Customer support cases with narrative descriptions and pre-computed embedding vectors.
- **`KEY_DRIVERS` Efficiency:** Replaces hours of manual dimensional slicing by evaluating cross-dimensional metrics and automatically pruning statistically negligible combinations.
---
[29:32](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1772s) Multimodal Root Cause Analysis with `AI.AGGREGATE`, `AI.CLASSIFY`, and `AI.GENERATE`
With the issue narrowed to the "Cat Costume" brand for March, an aggregation function (`AI.AGGREGATE`) processes all unstructured return comments to summarize the top reasons for customer dissatisfaction. Within seconds, the LLM-backed function identifies the three main complaints: the product failed to match its catalog description, items arrived damaged, or customers simply no longer needed the item.
```mermaid
flowchart TD
accTitle: Multimodal Root Cause Analysis Workflow
KD["Run KEY_DRIVERS Function"] --> FoundBrand["Isolate Spike:\nCat Costume Brand in Apparel"]
FoundBrand --> Agg["AI.AGGREGATE on Return Comments"]
Agg --> Reasons["Top Issue Identified:\nProduct Not Matching Description"]
Reasons --> Classify["AI.CLASSIFY Across Catalog Returns"]
Classify --> SelectItem["Inspect Product 215 via OBJECT_REF"]
SelectItem --> VisualCheck["Image Mismatch Found:\nLion Mane Title vs. Shark Costume Image"]
VisualCheck --> BatchGen["AI.GENERATE Catalog-Wide Audit:\nMultimodal Discrepancy Scoring"]
```
To quantify the distribution, `AI.CLASSIFY` tags all comments against these categories. Focusing on cases where the product did not match customer expectations, the investigation selects an exemplar (Product ID 215) and queries its metadata alongside its actual product image using BigQuery's `OBJECT_REF` construct. The query reveals a comical discrepancy: the product name reads "Lion Mane Costume," but the attached catalog image displays a cat wearing a shark costume.
To resolve catalog errors systematically rather than playing whack-a-mole, an `AI.GENERATE` query scales multimodal auditing across the entire catalog. The prompt instructs the model to inspect each product image against its corresponding title and textual description, returning a structured JSON response with a mismatch flag, a numerical severity score, and an explanation. The automated scan uncovers numerous humorous mismatches across the catalog—such as a cat shark costume illustrated by a cat dressed as a taco, several confused dog costume listings, and aquarium gravel cleaner listings with totally contradictory descriptions.
---
[34:35](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=2075s) Benchmarking Optimized Classification: 20x Token Savings at 99.992% Accuracy
The demonstration shifts to classifying 50,000 customer inquiries in the `support` table to route them into categories like shipping delays, returns, or billing disputes. Executing standard classification against massive row counts using brute-force foundation model prompting is slow and cost-prohibitive. BigQuery addresses this with an internal optimization mode: it trains a lightweight, domain-specialized model on the fly using few-shot embeddings and applies that trained model across the bulk of the dataset.
Running this optimized classification query over 50,000 rows executes in 24 seconds, consuming less than half a million tokens (under 500,000 tokens). By comparison, forcing BigQuery to evaluate all rows using the standard LLM ("maximize quality" mode) takes 2 minutes and 21 seconds—a 5x increase in runtime—and burns more than 10 million tokens. A parity check comparing the outputs reveals a 99.992% exact match between the optimized pipeline and raw LLM inference, proving that BigQuery can deliver a 20x reduction in token consumption and massive speedups with virtually zero degradation in analytical accuracy.
[37:45](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=2265s) Transition to BigQuery Studio: Creating and Configuring Agents
Having reviewed the SQL foundations—from `KEY_DRIVERS` and multimodal discrepancy audits to optimized text classification—the session moves back to the Google Cloud Platform console to showcase how these primitives transition from manual queries into autonomous, persistent business agents.
Within BigQuery Studio, the navigation interface exposes the new "Agents in BigQuery" workspace. Here, developers and data analysts configure specialized agents that package these underlying SQL functions, catalog boundaries, and classification logic into reusable, conversational assistants tailored for non-technical enterprise stakeholders.
[38:21](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=2301s) Configuring Knowledge Sources and Enterprise Context in the Agent
Vaibhav walks through the customization and configuration options available when building a conversational data agent, using a sample customer returns agent as an example. To ground the agent's reasoning, he connects several core knowledge sources, including the underlying raw tables used throughout the demonstration—specifically the catalog returns and customer support cases tables. Beyond simply attaching raw structured and unstructured data, the interface allows creators to inject detailed instructions, business glossaries, and semantic definitions. Providing this explicit business context drastically sharpens the agent’s accuracy and relevance when handling domain-specific inquiries.
To ensure consistency and accuracy across standard business metrics, creators can also provide verified queries directly to the agent. These are human-vetted SQL snippets that dictate exactly how complex or sensitive metrics, such as revenue calculations, should be computed. The agent also integrates directly with Knowledge Catalog, taking advantage of its metadata enrichment features to automatically ingest and generate domain glossaries from cataloged tables. This architecture allows a single data steward or domain expert who deeply understands the dataset to configure the agent with all necessary context, business logic, and guardrails, and then publish and share it securely with the rest of the enterprise.
subgraph AgentLayer["Agent Configuration & Execution"]
E["Conversational Returns Agent"]
F["BigQuery AI Functions\n(AI.GENERATE_TEXT, AI.TAG)"]
end
subgraph Consumption["Enterprise Deployment"]
G["Enterprise Users\n(Natural Language Chat Interface)"]
end
A --> E
B --> E
C --> E
D --> E
E <--> F
E --> G
```
- **Knowledge Sources:** Combines raw relational tables, semi-structured tables, and unstructured support case transcripts into unified agent knowledge.
- **Verified SQL Queries:** Prevents agent hallucination on key performance indicators by locking in predefined SQL formulations for critical metrics.
- **Knowledge Catalog Integration:** Automates metadata enrichment and semantic glossary creation directly from cataloged BigQuery tables.
- **Enterprise Sharing:** Enables a single subject matter expert to build, curate, and govern the agent before deploying it across the entire organization.
---
[39:38](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=2378s) Conversational Querying and Agent Execution with BigQuery AI
Vaibhav illustrates the agent in action by reviewing a previously executed natural language request: asking the agent to provide a summary of the top five support case issues within the apparel category. The agent performs its internal reasoning, interprets the underlying schema and contextual instructions, and automatically translates the natural language prompt into an optimized SQL query behind the scenes.
The generated query utilizes the native `AI.TAG` function directly within BigQuery to extract, categorize, and rank the top five issues in the apparel category without requiring manual text labeling or external pipeline orchestration. Vaibhav emphasizes that this completes the full journey of modern analytics: starting with base multimodal data in BigQuery, manipulating and analyzing it using scalable SQL and built-in AI functions, and finally abstracting the entire experience into a conversational AI agent that anyone in the organization can interact with.
> "So this is just a demonstration that you know we went from the base foundations of data in BigQuery to using SQL as the way to work with this data. But you don't need to stop there. You can actually move to the agent layer and just work with this data conversationally. Right behind the scenes it's just going to use the same functions for scalability."
Wrapping up the presentation, Vaibhav summarizes the end-to-end power of BigQuery AI and concludes the session by thanking the attendees for staying through to the end of the day.
## Notable quotes
> [00:04](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=4s) should be probably 100% at this moment — Vaibhav Sethi
> [12:05](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=725s) multiple hundred times reduction in the amount of tokens you spend — Vaibhav Sethi
> [23:18](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=1398s) nobody likes to write code these days, especially when you're working with simple analytical queries — Vaibhav Sethi
> [39:38](https://www.youtube.com/watch?v=CE8-ZMzlY3k&t=2378s) So this is just a demonstration that you know we went from the base foundations of data in BigQuery to using SQL as the way to work with this data. But you don't need to stop there. You can actually move to the agent layer and just work with this data conversationally. Right behind the scenes it's just going to use the same functions for scalability. — Vaibhav Sethi
## Mentioned
Vaibhav Sethi, Google Cloud, BigQuery, Gemini, Gemini Enterprise, BigQuery Omni, BigLake, Apache Iceberg, Apache Parquet, Vertex AI, Dataform, Google Cloud Storage, Object Ref, Python, BBC, Anthropic Claude, Hugging Face, Vertex AI Model Garden, Cloud Composer, Apache Airflow, Gemma, ScaNN, TimesFM, Connected Sheets, Google Sheets, TabFM, XGBoost, Data Agent Kit, VS Code, Apache Spark, Codex, Antigravity, Cloud Code CLI, Cymbal Pets, Knowledge Catalog
## Video description
Vaibhav Sethi - Product Manager, Google Cloud
A bulk of enterprise data is in documents, images, audio, and text, leaving a massive gap in business insights. This session explores how to bridge this gap by unifying both structured and unstructured data analytics directly within your data platform (BigQuery) with help of Gemini. We will demonstrate how to turn this unified data into actionable an actionable insights and into a conversational agent and deploy across your organization using Gemini Enterprise while maintaining robust governance and access control.
Join us to learn the blueprint for transforming your raw, multimodal data into an accessible, interactive, and governed AI experience for your entire workforce.
Takeaways:
Strategies for bringing structured and unstructured data together seamlessly in BigQuery.
How to create conversational AI agents natively within BigQuery to chat with your data.
Best practices for deploying these agents securely via Gemini Enterprise for broader organizational use.
Vaibhav Sethi argues that enterprise AI bottlenecks stem from fragmented data architectures, positioning BigQuery's unified multimodal data foundation, native SQL AI primitives, and internal agent orchestration as the solution for scalable enterprise deployment.
Unifying Multimodal Storage with Native SQL Primitives (0:04)
Sethi reports that although roughly 92% of enterprises are rushing to deploy AI, initiatives frequently stall because organizations treat structured data and unstructured media as separate silos, requiring continuous cross-system data movement. BigQuery addresses this by introducing the `Object Ref` data type. Unstructured assets like documents, audio, and images remain in Google Cloud Storage while BigQuery stores typed pointers containing URIs and metadata as first-class attributes alongside relational columns.
This unified model allows native SQL functions to operate across structured and unstructured content simultaneously. For unstructured document ingestion, `AI.chunk_document` parses and splits batches of PDFs directly into embeddable relational rows for vector indexing. Specialized functions like `AI.if`, `AI.classify`, `AI.score`, and `AI.aggregate` allow cross-modal joins and multi-row summarization. By default, `AI.generate` routes LLM requests through Gemini's batch pipeline—cutting inference costs by 50% relative to standard synchronous APIs—while also supporting Anthropic Claude or custom Hugging Face models via Vertex AI endpoints.
Built-In Embeddings, Vector Search, and Tabular Foundations (18:34)
Rather than relying on external vector databases, BigQuery embeds smaller models like Gemma natively and manages vector indexing through Google's ScaNN technology. Search scales serverlessly across billions of rows, automatically recalculating embeddings as source tables update and executing hybrid searches that combine exact keyword matching with semantic vector scoring.
Beyond language processing, BigQuery incorporates numerical foundation models. TimesFM delivers zero-shot time series forecasting and anomaly detection directly through SQL or Google Sheets formulas via Connected Sheets, allowing retail users to project demand across thousands of SKUs without custom training. For structured predictive tasks, TabFM provides in-context tabular classification and regression via `AI.PREDICT`. Sethi notes TabFM achieves higher Elo benchmark ratings than traditional gradient-boosted trees like XGBoost and competing pre-trained models without dedicated ML pipelines.
Operational Governance and Optimized Model Distillation (16:18)
Deploying generative SQL into production environments introduces cost and token management risks. BigQuery provides `AI.count_tokens` to let developers estimate token requirements before running queries, alongside administrative quotas to cap project-level consumption. For continuous workloads, streaming queries can run generative functions directly over live data feeds orchestrated through Dataform or Cloud Composer.
To balance latency and operational costs on massive datasets, BigQuery implements query distillation for classification tasks. Instead of sending every record to a full LLM, the engine trains a compact model on the fly using vector embeddings. In a demonstration classifying 50,000 customer inquiries, the optimized mode finished in 24 seconds and used under 500,000 tokens, compared to 2 minutes and 21 seconds and over 10 million tokens in pure LLM mode. A parity audit confirmed a 99.992% match between the distilled model and brute-force LLM inference.
End-to-End Multimodal Auditing and Agent Deployment (26:10)
A troubleshooting case study for retailer Cymbal Pets illustrated the end-to-end analytical pipeline. When monthly returns surged from 1,500 to 2,000 units, the automated `KEY_DRIVERS` function pruned dimensional combinations to isolate the spike within the "Cat Costume" brand. Next, `AI.AGGREGATE` and `AI.CLASSIFY` pinpointed customer complaints regarding catalog discrepancies. By referencing product images using `OBJECT_REF`, investigators spotted mismatches such as a "Lion Mane Costume" listing featuring a cat in a shark suit. An enterprise-wide audit using `AI.GENERATE` then systematically evaluated every catalog image against its title and description, returning structured JSON severity scores.
Finally, BigQuery Studio allows analysts to package these underlying SQL workflows into persistent, conversational agents. Creators define a bounded data scope, attach business glossaries and Knowledge Catalog metadata, and lock in human-verified SQL queries to prevent hallucinations on core business metrics. Non-technical stakeholders can then converse in natural language, which the agent translates into underlying BigQuery primitives like `AI.TAG` to rank and categorize issues automatically.
Sethi concludes that modern analytical workflows can transition seamlessly from raw multimodal storage to SQL-based AI inference, and ultimately into governed, conversational agents without requiring external database orchestration or data movement.
[0:04] Vaibhav Sethi: should be probably 100% at this moment
[12:05] Vaibhav Sethi: multiple hundred times reduction in the amount of tokens you spend
[23:18] Vaibhav Sethi: nobody likes to write code these days, especially when you're working with simple analytical queries
[39:38] Vaibhav Sethi: So this is just a demonstration that you know we went from the base foundations of data in BigQuery to using SQL as the way to work with this data. But you don't need to stop there. You can actually move to the agent layer and just work with this data conversationally. Right behind the scenes it's just going to use the same functions for scalability.
Original transcript
[0:04] Hey folks, so I'm going to be talking a lot
about Bitquery and multimodal data processing
[0:08] in Bitquery. So I think the last session was a
good uh segue uh to Bitquery and what all you
[0:14] can do there. Uh this session is structured a
little differently. So I have a bunch of slides
[0:20] uh where I lay the foundation uh of conceptual
ideas of what Bitquery can do for you
[0:25] and then I have a demo for you instead of a
workshop. So let me let me get started. Uh
[0:32] let's level set. Uh so everybody wants to put AI
in their enterprise uh today. So I have the number
[0:41] 92%. It should be probably 100% at this moment.
But a critical piece that very few enterprises
[0:48] give attention to is the data platform. And a data
platform is really really important if you want to
[0:54] do successful enterprise AI deployment. So there's
a few issues if you don't get the data platform uh
[1:00] part of the equation correct. One is that if your
data platform requires you to move data around
[1:06] and some of the capabilities which should be in
the data platform for AI analytics are not there,
[1:11] you still leave a large barrier uh for your
developers and business users to build on top
[1:16] of it. Secondly, a good data platform should be
multimodal. So you should not have a place where
[1:22] structured data lives separately from unstructured
data and you should not not have a situation where
[1:27] your uh governance controls uh for access
to the data are in many different systems
[1:33] right so good data platform needs to solve this
and that's the way to accelerate uh AI deployment
[1:39] so David gave you a very brief primer on uh on
BigQuery uh but uh I'll show you the evolution of
[1:46] Bitquery uh for this agentic era. So for folks who
are not familiar, BigQuery is Google's serverless
[1:53] exabyte scale uh data warehouse platform. It
started with data warehouse, we do a lot of
[1:57] things now. So we introduced serverless uh data
warehousing in 2011. It's been a decade and a half
[2:04] and still many of the competitors in the market
don't have this capability, right? So it's still
[2:08] truly differentiated. But over the past few years,
we have done a lot more. So we started introducing
[2:14] machine learning into BigQuery back in 2019. And
this is a pretty wish I could pause this diff but
[2:20] it's a pretty evolved set of features now. So
we are multicloud. You can have data outside
[2:25] of GCP and still use BigQuery as your data
organization uh layer and the query engine.
[2:31] Uh then we have uh big lake which is we support
data which is uh in iceberg format uh for you.
[2:38] Uh we have object tables which allow you to work
with unstructured data. And then finally over the
[2:44] last couple of years we have had deep integrations
with Gemini in bitquery. Uh we have added
[2:50] capabilities like vortex uh vector search and
more recently agentic capabilities into bigquery.
[2:57] So this is a snapshot of uh what bigquery looks
like at a conceptual level today. So at the bottom
[3:02] you have the lakehouse right. So you can store
data in the bigquery native format or you can
[3:08] have parket files or you can have iceberg it all
works. Then you have the serverless core engine uh
[3:14] which has you know the best uh scalable analytics
uh available and then you have the features and
[3:20] uh and capabilities at the the second I click
yeah uh at this layer. So you have analytics
[3:27] available via SQL and Python. Uh we have a lot of
AI capabilities and vector search capabilities.
[3:34] We have ability to represent your data as a graph
uh and do analytics graph analytics in BigQuery
[3:40] and also do stream processing in BigQuery. And
finally at the top layer uh your developers have a
[3:46] lot of choice when working with BigQuery data. We
just saw one example uh but there's a lot lot more
[3:51] that you can do. And then we have conversational
analytics agents built right into Bitquery.
[3:57] So the red boxes are the places uh which
I'm going to go deeper into in this session.
[4:03] So let's get started. So let's start with the
unstructured data foundation in BigQuery. So I I
[4:09] believe folks must be familiar how structured data
is you know typically stored in warehouses. So I
[4:14] won't cover too much of that. But let's look
at how Bitquery handles unstructured data.
[4:19] So the way we do it is that you have structured
data stored as normal tables in BigQuery.
[4:24] But for unstructured data, you can uh leave
this data in GCS which is you know the best
[4:29] typically the best choice for keeping unstructured
data. And what you keep in BigQuery is instead a
[4:34] pointer to this uh GCS location. So what we have
in Bitquery is what we call as object ref. So
[4:41] this has the URI of your uh audio file in this
case, right? As well as all kinds of metadata.
[4:48] Now the good thing is that this information
is directly queryable by the bigquery engine
[4:54] or if you define things like python udfs
they can directly query this information
[4:58] as well as any metadata which is
here is also queryable by our engine.
[5:03] So say you have done some annotation of metadata
and have added data in GCS and you want to filter
[5:09] on that particular tag or value you can do
that just with SQL queries in BigQuery. Right?
[5:15] Uh so this is the foundation of multimodel data
in BigQuery. So let's uh go a little bit deeper.
[5:21] So specifically how you handle multimodel data in
BigQuery is through this thing called object ref.
[5:27] So it's a new data type that we have added
to BigQuery. So it's really a strct. So as
[5:31] I was pointing out before, it has a URI for your
object in GCS plus all other kinds of metadata.
[5:39] So because this this is a a data type for a column
in BigQuery, uh you can have one or multiple
[5:44] columns of this type in a BigQuery table or you
can have uh some part of your data which is uh in
[5:51] a in a table with object refs, others are purely
structured data, right? And then what you can do
[5:56] with this is that this is highly composable.
So you can move this between tables. You
[5:59] can uh create these uh or transform these as part
of ETL pipelines. Uh and you can also persist them
[6:08] and it allows you to answer questions like this
which are traditionally very hard. Right? So for
[6:12] example, I can find electronic products. So
this is like basically looking at category of
[6:17] some product in a table with high return
rates. Again looking at structured data
[6:22] and then I'm doing something
interesting. I'm looking at
[6:25] whether the product has a customer photo and it's
showing damage on arrival. So in a so I'll show
[6:32] you how to do this in a in a demo. But you went
from combining structured and unstructured data
[6:37] as well as analytics on this unstructured data.
Uh you can do all this with objects in BigQuery.
[6:45] So that's the that's the data foundation
layer. I'm going to be spending a few moments
[6:49] to talk about what are the analytical
capabilities you have on this layer.
[6:55] So just a brief overview of all the AI and
ML stuff that you can do in BigQuery first.
[7:01] So as I was mentioning in that in that
GIF where we have been innovating right
[7:06] we have been working on the machine learning
parts in BigQuery for a while. So Bitquery is
[7:11] also actually a full-fledged ML platform. So if
you're familiar with SQL and you know you have
[7:17] basic needs like training a clustering model,
classification model, you can do all of that in
[7:21] BigQuery. You can do feature engineering with
SQL at scale. Uh you can do uh model training,
[7:27] eval uh serverless inference which is
distributed. So this scales really really well
[7:33] and you can do MLOps via the
Gemini enterprise platform.
[7:38] In addition, we have been adding pre-trained uh
predictive models to Bitquery. So uh I'll again
[7:44] talk about this later in my deck. Uh but uh very
quickly we have pre-trained models for forecasting
[7:50] and anomaly detection as well as classification
and regression. So most of the common enterprise
[7:55] tasks today don't even actually need you to
train any model or do any feature engineering.
[8:00] It's a pre-trained transformer. You can just
apply to the task and it's available in BigQuery.
[8:05] Then we have AI functions of all sorts
to work with this unstructured data. Uh
[8:10] we have search and vector search. Uh so you
can generate embeddings at scale in BigQuery
[8:15] and then uh use our serverless vector search
to index it and do searches at scales of
[8:20] billions of embeddings and then we have agents for
developers on BigQuery. So you we have specialized
[8:26] agents like a data engineering agent for you to
easily develop data form pipelines in BigQuery
[8:31] and then tools for uh agent builders to
be able to use BigQuery as a data source
[8:36] as well as a source for uh
differentiated capabilities.
[8:42] So I'll go a bit deeper into the Geni capabilities
in Bitquery first. Uh so let's start with this
[8:47] example. So Bitquery is very deeply integrated
with Gemini and also gives you the choice of
[8:52] other models. So we have a function uh we have
several functions but the one I'm illustrating
[8:57] here is one called AI.generate. With this
function you can basically do LLM inference
[9:02] on any kind of data in BigQuery. So in this
case I'm asking summarization of uh this BBC
[9:09] news data set. Right? So for each row
there's a body column and for each row
[9:13] I'm asking it to summarize I'm asking the LLM to
summarize the uh comp the u what's in this row
[9:21] in a single sentence right so this thing uh then
runs over whatever scale of data you have so it
[9:27] could be millions of rows right so this is how
simple it is to express LM inference in bigquery
[9:33] and and this is really powerful because in many
scenarios uh you're not you're doing sometimes
[9:38] semantic tasks like summarization but what
we also see a lot of our customers doing is
[9:43] starting from unstructured data. So it could be
things like even unstructured text and extracting
[9:48] some structure out of it which you want to use for
your business applications or analytics. Right?
[9:54] So you can do that easily by providing
a output schema in the same function
[9:59] which constrains the model to provide the output
exactly according to this. So in this case say I
[10:03] have a patient uh description. I can extract
the name and age and then persist this in a
[10:09] big query table. Right? So with very simple SQL
statements, you unlock a lot of new use cases.
[10:17] There's a lot of differentiators of by doing geni
in BigQuery. So the first is that you actually
[10:22] have a choice of LMS, right? The default
is Gemini. It works really well at scale,
[10:27] but you can also use cloud. You can also use
any hugging face model that you have deployed
[10:30] in Vert.xi with Bitquery. uh bitquery AI is built
for SQL users. So we have a lot of functions which
[10:37] simplify the life for like data workers and
analysts right so you we have for example a
[10:42] function called generate table which can take
that uh structured thing that I was talking
[10:47] about and just generate a table so that you
don't have to do post-processing of the result.
[10:52] Uh it's multimodel as we've been talking about
and it's very scalable and attractively priced.
[10:57] So on this part when you're using
a function like AI generate uh
[11:03] it's categorized as a batch inference call to
Gemini and because it's a batch inference you get
[11:08] like a 50% discount on the inference price right
so if you if you have use cases which are not
[11:13] extremely latency sensitive this is a great way to
save costs right and be able to run these at scale
[11:22] I'll quickly cover what we see our customers in
real life doing with uh with Bitquery AI. So we
[11:27] have a lot of customers doing entity extraction
both from text and unstructured data like images.
[11:32] Enrichment of the data with word knowledge of
LMS semantic synthesis of all kinds. So sentiment
[11:38] analysis is an example of that. Content generation
based on the data that you have in BigQuery
[11:43] and also a lot of log analytics use cases
right so classify my logs according to
[11:47] certain categories or summarize the next
action based on the logs that we are seeing.
[11:54] So that's the core uh Gemini inference core geni
functions in BigQuery. Uh but we have a lot more.
[11:59] So this data platform takes care of everything
that you would need to be like as a AI builder.
[12:05] So one set of functions uh we are quite excited
about is uh document processing functions.
[12:10] So you have say unstructured documents and PDFs
right and you want to build rag applications on
[12:16] it. uh a very important intermediate step is to
parse the documents and then chunk them right and
[12:21] then embed them create vector indexes on them
and all and all of that stuff right so you can
[12:26] do that very simply with SQL in bitquery with
a very simple statement like ai chunk document
[12:31] and this will read uh this object table which is
pointing to say a GCS bucket with thousands or
[12:37] like tens of thousands of PDFs and will give you
the chunks like this in a big query table right
[12:43] so these chunks are now embeddible
and uh you can to rag easily on this.
[12:50] Then we have another class of functions uh called
managed AI functions. Uh so these exist to like
[12:56] really simplify uh your use cases for special
needs and uh because they're more restricted,
[13:01] we are able to do a lot of optimizations
on these functions. I'll quickly cover the
[13:06] the range of functions we have here. We have a
function called AI. If and this is intended for
[13:11] semantic filtering and join condition uh and join
operations. So in this case I'm able to join a uh
[13:19] two tables. One has a dis an image and the
other table has a product description. Right? So
[13:26] normally you can't join tables like this. But in
this case I'm saying whether they are referencing
[13:30] to the same product or the advertisement uh
contains the product and join based on that
[13:34] condition. Then we have a classify function where
you can pass in uh structured or unstructured
[13:41] data. So text, images, whatever. And just specify
the categories. Uh, and that's really it. We have
[13:48] a score function. So you can score uh over a bunch
of rows semantically. And it does it intelligently
[13:55] so that the score is normalized across the rows,
right? So it's not individual uh row at a time.
[14:01] And then we have a semantic aggregation function.
So if you have been shopping recently, every
[14:05] website has uh you know a function where it says
that this is the aggregate in aggregate what the
[14:11] reviews are saying about this restaurant or this
product. So you can build something like that in a
[14:17] single line of SQL with this AI.ggregate function.
So what it does is that it will go through all the
[14:22] rows and aggregate the semantic meaning. So if you
do this naively, you know, you will very quickly
[14:28] hit into the context limits of LLMs. But we do it
in a smarter fashion. So we are able to not uh we
[14:34] not constrained by that limitation, right? So
it scales quite well to a lot of rows of data.
[14:41] So so those functions uh are interesting, right?
But you could also express what you're doing there
[14:46] with the base generate function, right? Like a
base LLM inference call. Uh but what we have taken
[14:51] these a step further and have introduced like
uh optimizations to them which are really really
[14:55] differentiated. So one thing that we do with our
AI and if uh AI if and classify function is that
[15:02] instead of passing everything through an LLM for
the binary or multi-class classification task here
[15:08] we actually train a classical machine learning
model on the fly using the embeddings of the data
[15:14] as the input feature and then we use that
[15:19] distilled model so to say for actual
query execution. So when this works,
[15:24] this results in like multiple hundred times
reduction in the amount of tokens you spend and
[15:30] queries execute really really fast, right?
So you can go like thousands of rows in a
[15:34] couple of seconds. Uh and you can scale
to billions of rows of queries processed
[15:41] and this is how it works uh in in terms of code,
right? So like we have the same if example. So
[15:46] I'm just trying to pro uh classify whether the
sentiment uh is positive or negative, right?
[15:52] And what I need to do is provide the
embeddings for this review column.
[15:56] Uh I can have the embeddings premputed in
which case I don't need to compute it like this
[16:01] or if I don't have it premputed
I can compute it in line
[16:04] and then choose the optimization mode minimize
cost. So if you if you do have the embeddings
[16:10] already available actually you don't need to
specify any of this and just works out of the box.
[16:18] Okay. So those are the core AI capabilities
in Bitquery. Uh as you think about actually
[16:23] putting this in enterprise settings, you need
controls and tools to productionize this.
[16:29] So Bitquery gives you a good amount of control
for your token spend. So you can count tokens
[16:34] before you execute a query with the count tokens
function. Right? So this doesn't invoke any token
[16:39] charges. Uh you can as admins put controls on
the project and user level. So you can say okay
[16:46] daily I don't want this project to be consuming
more than 100 million tokens or whatever right
[16:52] and you can as a developer
also do this at the query level
[16:55] so when you're testing something you can
set a lower limit and when you actually
[17:00] are ready for a production deployment you can
set a high limit for this uh token controls
[17:07] then you can also like build
much richer applications so the
[17:11] uh things that I was showing to you are
like static batch things But you can
[17:15] actually have the same code in a streaming
pipeline. So BigQuery has this feature called
[17:19] continuous queries and you can for example put
the generate function within a continuous query.
[17:25] And then for the developers in the room right
so you we have a product called data form which
[17:30] allows you to basically work with SQL for uh uh
with git sorry for building your ETL pipelines.
[17:36] So you can do like basically the standard software
development life cycle support is uh is there with
[17:42] data form and then we have multiple ways of
deploying your pipelines with bigquery right
[17:48] so you can use the bigquery's inbuilt pipeline
feature if you have very complicated DAGs you can
[17:53] use our managed airflow offering called
composer uh to also orchestrate stuff
[18:01] okay so that's uh AI functions and how
to deploy them and how to productionize
[18:05] them let's switch gears a bit and
talk about embeddings in Bitquery.
[18:10] So there's many ways to generate embeddings in
Bitquery. Basically everything is supported.
[18:14] So the first thing that you can do is
use Google provided models as a service
[18:19] to generate embeddings in BigQuery and they are
very cost effective and they scale quite well.
[18:24] If you do have specialized models, say you have
a specialized model that from hugging face that
[18:29] you want to use, you can deploy this in GCP uh
through a model garden endpoint and then use
[18:34] them in BigQuery or for smaller models, you can
actually natively run them in BigQuery directly.
[18:41] So the Gemma class of models for embeddings,
they run natively in BigQuery directly.
[18:47] Once you have those embeddings, so this
could be over billions of rows of data,
[18:50] we have inbuilt vector vector search. So we use
the same uh indexing and search technology which
[18:56] is used by a lot of Google products, a lot
of Google consumer products which have like
[19:00] billions of users. So they work at really large
scale. So that thing is called scan. Uh so with
[19:06] this scan index uh and search we are able to scale
to multiple billions uh rows of uh embedding data
[19:13] and this is completely serverless. So you don't
have to manage a separate uh vector database or a
[19:18] vector database instance or manage its life cycle.
It's all it's all in BigQuery. uh and also uh
[19:24] the embeddings are you can configure the bitquery
tables to generate embeddings autonomously.
[19:30] So as your data is mutated or new data comes in
the embeddings are automatically kept up to date.
[19:38] So in addition to vector search you can also do
hybrid search in BigQuery. So if I have a query
[19:42] like this you know which has a very specific
term which needs to have a exact text match.
[19:48] uh but there's also like parts of it which
need to have semantic match with other uh
[19:52] uh parts of the data. Uh we we support hybrid
search. So we are able to do get both the signals
[19:59] and smartly rerank uh the output so
that you get the best performance.
[20:05] Okay. So that was a very quick walk through of uh
you know uh the LLM capabilities in BigQuery. But
[20:13] as I was mentioning at the start of the session,
LLMs are not the only things that you can do. So
[20:18] one um other thing that you can do in BigQuery is
you can do forecasting and anomaly detection very
[20:24] easily and we do this through a model called times
FM. So times FM is a transformer-based model.
[20:30] Uh but this is trained for predicting a time
series instead of predicting a language token.
[20:36] Right? So this requires no training. So it's
zero short. you just provide some historical data
[20:41] and tell us how far in the future you want
to predict and you can predict on many time
[20:46] series in a single SQL statement right so if I'm a
retailer I have multiple stores multiple SKs right
[20:51] so I have many different time series and I want
a prediction of all of them for the next 30 days
[20:56] I can do it with a single statement like this so
no models to manage no infra to manage this just
[21:02] scales very well and gives you very accurate
predictions and similar to forecasting you can
[21:07] also do uh anomaly detection both on historical
data and the new data that you're seeing.
[21:15] So I I talked to I I talked to you about the SQL
interface, you know, for things like forecasting,
[21:19] but a lot of uh business users are also
in things like spreadsheets, right?
[21:25] So uh in Google Sheets, we have a feature called
connected sheets which lets you work with BigQuery
[21:30] data but from the spreadsheet interface. So
when you actually do any like formulas here,
[21:35] they get executed on BigQuery. So it's super
super scalable. So what we have also done is
[21:40] we have brought advanced stuff like time
series forecasting with times FM model
[21:44] directly to this connected sheet experience as
well as our anomaly detection experience to this.
[21:51] So next we have a new launch uh
there was a blog a few days out. I
[21:56] hope some people saw it. Uh but we're adding
this new model called tab FM into BigQuery.
[22:01] So tab FM uh is kind of builds on the times FM
idea but this is for classification and regression
[22:08] use cases. So say I'm uh I want to detect fraud in
my transactions right and I have some labeled data
[22:16] and I have some new uh data sets
that I want to detect fraud on.
[22:20] So what I can simply do is instead of like
training say XG boost model evaluating it
[22:24] optimizing it right I can just simply use the
AI predict statement provide the training data
[22:31] this is for in context learning provide the
new prediction data and just provide the label
[22:35] that you're trying to predict so again super
simple runs inside BigQuery and really accurate
[22:42] and on the accuracy part uh these are some numbers
uh we published recently so this compares uh Tab
[22:49] FM from Google versus other pre-trained
models. So as you can see on the ELO score
[22:55] for uh classification and also regression, we
beat everybody else in the market by quite a bit,
[23:02] right? So this this model is uh pretty good.
So it beats like a customtrained tree models
[23:09] on your data uh as well as like all of the
sophisticated pre-trained models out of the box.
[23:18] Okay, so that's the basic building blocks and
primitives of what you can do in Bitquery. Uh,
[23:24] now let's talk about agents in Bitquery. So,
you know, nobody likes to write code these days,
[23:31] especially when you're working with simple
analytical queries. So, for that scenario,
[23:35] we have conversational analytics in Bitquery.
So what this experience lets you do is in the
[23:40] Bitquery console uh you have a chat experience
where you can work with uh uh you can define
[23:47] the bounding box of what's important to you,
what tables and data sets are important to you
[23:51] and then you can chat with it. Uh so it will
write SQL queries to answer your questions.
[23:56] Uh because you're accessing the data with your
own credentials, it's all secure, right? And
[24:01] one user can define a agent, add a lot of context
and customizations to it, and then they can share
[24:07] it with the rest of the organization. And I'll
show you quickly how to do that in a demo.
[24:13] Uh then the nice thing is that all of
the AI capabilities that I talked about
[24:18] are integrated into the conversation analytics
as well. So you can ask it to do things like uh
[24:24] process unstructured data. So for example,
categorize search queries for me, right?
[24:29] uh with say a a classify statement. So you can ask
a question like this and it will figure out that
[24:34] it can use the a classify uh function for it. You
can also ask it to do predictive questions, right?
[24:40] Uh with a times FM or a tab FM kind of a model.
[24:44] So for an analyst, right, so you expand beyond
just simple aggregation or analytical queries
[24:51] with SQL to also the all of the AI stuff that
I showed to you just in this chat experience.
[24:59] And if you folks are using Gemini
Enterprise and I hope you are,
[25:03] you can publish the agents created
in Bitquery to Gemini Enterprise.
[25:07] So if your business users are in that
interface, they can get the same experience.
[25:13] And then for developers, uh I think you got a
sneak peek of this already. So we have uh recently
[25:19] launched a product called data agent kit. So it
has two things. So it has uh a VS code uh plug-in.
[25:27] So which brings you know the familiar catalog
browser and a lot of dev tooling uh to it
[25:32] as well as it has skills uh for uh telling
the agents on how to work with bitquery and
[25:38] the other uh GCP data services. So including
spark and orchestration services and so forth.
[25:44] So you can use this in any agentic coding tool
right so you can use this with codeexcloud uh
[25:49] anti-gravity right uh and you can also use it from
the CLI tools like uh anti-gravity CLI cloud code
[25:56] and so right okay so that's uh the uh slide part
of my talk let me switch gears and talk about uh
[26:10] okay talk about the demo a So uh I'm going
to be doing a SQL heavy demo right but
[26:20] you can see that it extends very well to
agentic interfaces and coding experiences too.
[26:25] So I have a hypothetical scenario where I'm a
store manager uh or not not a store manager.
[26:30] I'm a business manager of the symbol pets
company. It sells pet products, right? Uh and
[26:37] I have a anomaly in my returns. So my returns
have spiked in this month versus the last month.
[26:43] So my manager is asking me to evaluate what
happened like correct the issue. Right?
[26:49] So there's a few tables that I'll be working with
today. So let me just walk you through it. Right?
[26:52] So I have the catalog table. So this has you know
information about the product, brand, category
[26:58] as well as unstructured information like what is
the product image which is being shown to the user
[27:02] right? Then I have a returns table. So this is
showing which has a product ID, the return date
[27:09] and why was this product returned
and then I have a support table u
[27:15] which is for support cases. Right? So customers
are reaching out to to my support team.
[27:20] uh they're opening some support cases
and there's some details on what the
[27:24] customer is reaching out about and then
we also have an embedding column here.
[27:29] All right, so this embedding column exists so
that we can optimize that classify function
[27:33] in the future. Right? So that's just
showing you how the mechanics of it works.
[27:39] All right, so let's start with some code. So as
I was saying, we have a anomaly in our returns,
[27:44] right? So I can very quickly see
what is what is going on here.
[27:48] So normally we do like say 1,500 uh returns a
month. In this last month we had 2,000 returns,
[27:54] right? [snorts] So when you get a
situation like this, you know, the uh
[28:00] instinct is to start slicing and dicing your data,
right? And like finding out, okay, which brand,
[28:05] which category, which region actually contributed
to this delta in the in the metric. So the good
[28:10] news is that you can do this automatically in
BigQuery with a function called key drivers.
[28:15] And with this key drivers function, I can
specify the metric column which is of interest
[28:20] to me which is the return count and the different
dimensions on which I want to do this analysis.
[28:25] So I want to find out which specific brand,
category and product name combination actually
[28:29] contributed most to the change. Right? So this
is like t timeconuming analysis to do manually.
[28:36] Uh but we automate this with a key driver
function. The other innovation we have here
[28:40] is that this scales to very large data sets. So
if you're trying to do this manually, uh this
[28:46] combination of dimensions just explodes. So in our
execution of this function, we are able to prune
[28:51] out uh the the combinations which are contributing
less to the final metric. Right? So all of that
[28:57] complexity is taken care of for you and we can
uh run this to find out where the issue is.
[29:04] All right. Right. So key drivers ran and it tell
told us that the brand cat costume in the category
[29:10] apparel is actually contributing the most to
the difference and there are some other things
[29:15] you know which are like much smaller in magnitude
uh which are contributing to the difference right
[29:20] so we have a smoking gun like we can start
figuring out okay something is going in the
[29:24] cat costume brand uh and let's see if we can
sort out like what exactly went wrong there.
[29:32] So what we can do is use our AIA
function to look at our returns data set
[29:39] right. So I'm joining it with products and
filtering for cat costume. So I'm looking at
[29:43] basically all the returns for the cat costume
in the month of March here. So that's a that's
[29:48] the latest month I'm working with and I'm I can
use a aggregate to now tell me okay at aggregate
[29:54] level what happened with these returns right? So
what I'm asking the function to do is summarize
[29:59] the top three reasons for these returns. So I
can run it. Uh it takes a few seconds to run.
[30:06] Okay. So we have the result pack and can
people read this or maybe okay this is better.
[30:14] Okay. So I have a result pack telling me that the
top reasons for return were product not matching
[30:20] with description uh arriving damaged and
customer no longer needing the product. Right.
[30:27] Okay. So this is interesting. Uh so I got
aggregated information uh in this way.
[30:33] But now what I want to do is look at
all of my returns uh and classify each
[30:41] return according to what the aggregation
function told me. Right? So it told me that
[30:46] there was a scenario where the product was not
as expected and uh there was a scenario where it
[30:52] arrived damaged and some scenarios where the
customers didn't need it. Right? So I'm just
[30:56] trying to get a sense of what is the distribution
of these scenarios across my data set.
[31:00] So again I use the comments column and I use the
AI classify function. I provide these categories
[31:06] as well as description of the categories
to aid the LM to make this decision. Right?
[31:11] So this uh would take a while so I'm not running
it. Uh but I'll run uh analytics function on it
[31:18] to show you what does the distribution look like.
Okay. So we have a lot of scenarios where uh
[31:26] you know the product is not as expected and then
uh we also ask the system to just pick a random
[31:32] product for us which fits this scenario right
so there is some product 215 which was among the
[31:37] products which is not as expected so we can now
uh use this object ref thing that I was previously
[31:44] talking about and with SQL also like read this
image and see what's going on right so I'm looking
[31:50] I'm looking at product name description as well
as what actually is in this product image which
[31:55] was not matching with the user's idea of
what this product is. Right? So I run this.
[32:03] Okay. All right. So this is what
happened. Right. So the product
[32:06] is lion mane costume but it's like a
cute cat in a shark costume. Right. So
[32:12] there's a mismatch in the product image and
the product name. Right? So this seems like a
[32:17] promising direction of investigation, right?
So maybe we can do this at scale. Right? So uh
[32:24] so we found that one specific product had a issue
but maybe other products in my catalog also have a
[32:29] issue. Right? So what I can then do is do this at
scale. So I can use the generate function that I
[32:35] initially talked to you about. Uh give it a prompt
saying okay compare the product uh provided to you
[32:43] and compare its image description and analyze
whether they match or not. Right? and uh give
[32:50] me the output in a structured fashion. Right? So
score the out basically tell me whether there's a
[32:57] mismatch and then also score the mismatch. So
give me a number and an explanation for this
[33:04] and do this for my whole catalog. Right? So I
ran this query previously because it's it's again
[33:10] going to take a little bit of time. Uh so you can
see that I was able to run this analysis over my
[33:17] uh full uh um full catalog and now I
have the list of my products which are
[33:24] which have a mismatch right so as this as a
business manager for this line like I'm very
[33:29] quickly able to get to the issues in my catalog
right and then be able then I can resolve them.
[33:35] So, let's see what the what the query
sent us back, right? So, it sent us uh
[33:40] okay, this supposed to be a shark costume, but it
seems like it's in in a taco, right? So, a lot of
[33:46] cat cats have a lot of issues, right? But some
dogs also have issues in our product portfolio,
[33:52] right? And then we have maybe other products
like this aquarium gravel cleaner looks
[34:00] correct, but okay. So, the description
doesn't match in this case, right?
[34:04] So we had some scenarios where the image was not
matching the product uh product name and then we
[34:09] have some scenarios where description is not
matching the product image and the name right.
[34:13] So uh again like example of some analytics
that you can do and processing you can do
[34:19] right mixing unstructured and structured
data together and quickly get to insights.
[34:24] All right. So, this was my, you know,
investigation into the issues with my product
[34:29] catalog and I'm able to get to the root cause
here and I'm hopefully able to solve it. Right.
[34:35] Let's do some more analysis. So, we
also had the support table, right? Where
[34:40] uh we uh customers were reaching out
to us and uh you know complaining. So,
[34:46] I just want to do a very similar classification.
I want to find out you know whether I'm getting
[34:51] more issues with shipping more with returns
or more with billing issues and so forth right
[34:56] so I can run a query like this over 50,000
rows um on uh using the classify function right
[35:04] so I ran this live hopefully this works uh but
the idea here is that Bitquery is going to be
[35:11] able to use the optimized mode that I was talking
about to actually run this query right so instead
[35:16] of passing everything through an LLM. Hopefully,
we're very shortly going to see a result where
[35:21] BigQuery trained a model on the fly uh got
good quality results from the model and then
[35:29] applied that model to do the classification
task, right? So, this completed so I was able
[35:33] to process this 50,000 rows in 24 seconds, right?
And I was able to do it with this optimized mode
[35:41] and also take a note of the token count here,
right? So this is a half a million less than
[35:46] half a million tokens, right? So this is the
amount of tokens you spend for processing 50,000
[35:52] rows. So this is like a pretty low number
and I'll also contrast it uh in a moment.
[35:58] So this is like just uh so that you guys are
aware this is how the uh distribution of the
[36:06] classes look. So nothing very very interesting to
see here. But let me show you another query. Um,
[36:20] sorry, I'm just trying to Yeah. So the other
thing that you can do with the classify
[36:26] uh function is that you can choose
maximize quality as the optimization mode.
[36:32] So when you choose this, you're telling Bitquery
to always use a LLM for inference. Okay,
[36:38] so this would take several minutes to run. Uh,
it's the same query, but it's just in a it's
[36:43] turning essentially turning off the optimization
mode, right? So, but I just want to show you the
[36:47] contrast of uh what this looks like, right? So,
this query took 2 minutes and 21 seconds to run,
[36:52] like 5x more, right? And it consumed 10 million
plus tokens, right? So just in this very simple
[37:01] scenario, I was able to cut my token consumption
by 20x uh in the same classification scenario.
[37:09] So this is good but like did we actually
also achieve good quality on this right?
[37:14] So very one very simple and a bit naive
way to measure quality is just seeing
[37:19] whether our results matched between the
optimized mode and the nonoptimized mode. Right?
[37:23] So I'll just run this to see okay what
is our match percentage looking like.
[37:31] Okay. So you see we have 99.992% match, right? So
you get almost the same quality of results at 20x
[37:40] lower price in this example, right? So these are
the kind of powerful things that you can do in
[37:45] BigQuery. All right. So this is like from the SQL
interface. Uh let me switch to agents. Uh so if
[37:54] you are familiar with this interface I'm showing
you the GCP console and in uh GCP oops sorry
[38:01] in GCP console we have BigQuery studio which I was
showing to you and we have this new thing called
[38:08] um we have this new thing called agents in
BigQuery. So here you can create and configure
[38:14] agents uh in Bitquery. Uh so I have created
a agent uh previously. I'll just show you the
[38:21] kind of things and customizations you can do with
this agent. So I have the symbol returns uh agent
[38:27] and what I have here is uh a few knowledge sources
that I have added right. So I've added this raw
[38:34] tables that I was working with. So the catalog
returns and the support cases table uh to my
[38:39] agent. In addition, I can add instructions, right?
So you have like specific glossery or semantic
[38:46] meanings of of the data. You can add all that
here, right? So the agent performance uh really
[38:52] improves if you provide the agent that context.
You can oops uh you can also add verified queries.
[39:00] So you can have SQL queries which say compute
revenue or a certain metric in a certain fashion
[39:06] very specialized and you can add it here. Uh you
can add glossery from knowledge catalog. So all
[39:12] the tables that I was showing to you uh were in
knowledge catalog. uh that's a cataloging product
[39:18] and that has uh enrichment features so it
can autogenerate glossies for your data.
[39:22] So you can add that here and then there's a bunch
of settings additionally available to you. Right?
[39:27] So you can have this agent and then one
user who is expert on this data sets has
[39:33] all the context can create it and then share
it with the rest of your enterprise. Right?
[39:38] So what I wanted to just show you here
is I I ran this query previously uh on uh
[39:45] I I want summary of the support cases for the
apples category into like top five things, right?
[39:51] So this is very similar to the kind of analysis
we were doing previously uh in in the demo. And so
[39:57] the agent did its thinking and everything, right?
And finally it came up with a query like this.
[40:03] So it was actually able to use the AI tag function
uh to find the top five issues right uh in the
[40:11] apperal category. So this is just a demonstration
that you know we went from the base foundations of
[40:18] data in BigQuery to using SQL as the way to work
with this data. But you don't need to stop there.
[40:24] You can actually move to the agent layer and
just work with this data conversationally. Right
[40:29] behind the scenes it's just going to
use the same functions for scalability.
[40:34] All right. Uh yeah. So that's uh Bitquery AI in a
nutshell. Let me see if I have anything else. Uh
[40:41] yeah, so that's Bitquery AI in a nutshell. And
uh I think that's the end of my content here.
[40:47] Thank you so much folks for sitting
through this to the end of the day.
