Google Cloud Links BigQuery SQL to Gemini for Multimodal Data Search

Vibe-Code an App on Multimodal Analytics [HANDS ON; LAPTOP REQUIRED] With Google Cloud — Ai4

Ai4

While vibe-coded applications remain unfit for production, direct BigQuery object connections allow standard SQL queries to extract structured insights and vector embeddings from raw video and audio files.

AI-assisted summary of Vibe-Code an App on Multimodal Analytics [HANDS ON; LAPTOP REQUIRED] With Google Cloud — original source. Check the original for full context.

Source published: 2026-08-15

Detailed notes

## Background Published on YouTube by Ai4 on August 15, 2026, this video captures a hands-on workshop led by Google Cloud Developer Advocate Lucia Subatin. The session demonstrates how to query unstructured multimedia files using SQL in BigQuery and build a semantic search application deployed on Cloud Run.

## TL;DR Lucia Subatin guides attendees through configuring an ephemeral Google Cloud environment to build an end-to-end multimodal analytics pipeline. Participants link multimedia assets in Cloud Storage to BigQuery using object references and execute generative AI SQL functions powered by Vertex AI and Gemini to extract structured insights and vector embeddings from images, audio, and video. The session demonstrates semantic vector search over diverse media assets and introduces a containerized web application on Cloud Run, while providing catch-up SQL scripts for self-paced execution. Subatin concludes with a candid evaluation of the architectural boundaries and production viability of vibe-coded applications.

## Atmosphere & style The workshop carries a brisk, high-energy, and interactive tone as Subatin balances a condensed timeframe with live classroom troubleshooting. Her delivery is practical, humorous, and refreshingly candid, openly expressing her preference for robust SQL data pipelines over generative AI hype while reassuring attendees that breaking things in sandboxed environments is part of learning. The dynamic feels supportive and grounded, blending playful pet examples with serious enterprise cloud architecture.

## Key takeaways - [01:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=77s) BigQuery enables direct multimodal analytics over unstructured Google Cloud Storage assets—such as images, video, and multilingual audio—using standard SQL queries connected to Gemini. - [01:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=77s) The complete workshop architecture spans storage in Google Cloud Storage, analytical processing and embeddings in BigQuery, and frontend web hosting on Cloud Run. - [03:10](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=190s) Ephemeral sandbox environments allow risk-free experimentation but require clean Incognito browser sessions to avoid credential conflicts and accidental billing. - [10:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=617s) BigQuery leverages high-speed gRPC external connections to link directly with Vertex AI and Google Cloud Storage without requiring custom API glue code. - [10:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=617s) Multimodal assets including cross-lingual audio tracks and pet video clips are treated as queryable data sources directly inside SQL workflows. - [16:48](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1008s) A preconfigured catch-up script (bq_catchup.sql) allows fast-tracking of regional BigQuery multimodal pipelines by replacing environment variables. - [16:48](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1008s) Cloud Shell integrates Gemini Code Assist to provide in-line code explanations directly within the SQL development environment. - [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) BigQuery's OBJECT_REF constructors allow raw multimedia files stored in Cloud Storage to be treated as first-class database references in SQL queries. - [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) Native generative SQL functions like AI.GENERATE can query Google's Gemini models or custom open models hosted on Vertex AI Model Garden. - [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) Constraining generative AI outputs to structured JSON formats inside SQL queries ensures deterministic and stable downstream data processing. - [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) Generating multimodal embeddings allows semantic vector search across text and media, matching conceptual meanings rather than relying on exact keyword matching. - [22:09](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1329s) While vibe coding provides a fast way to assemble interactive application frontends, AI-generated application code typically requires rigorous review before being considered production-ready.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=4s) Workshop Introduction and Audience Check-In

With only 45 minutes allocated for a session that typically takes closer to an hour, the workshop kicks off at a rapid, high-energy pace. Lucia Subatin, a Developer Advocate at Google Cloud, introduces herself and invites participants to connect online or visit the Google Cloud booth afterward for deeper conversations. To calibrate the room, she conducts a quick show of hands to see who has previously used the Google Cloud Console or BigQuery, while offering immediate reassurance to non-technical attendees that they are fully supported.

A dedicated team of teaching assistants is stationed throughout the room to provide one-on-one troubleshooting whenever participants hit a roadblock. Attendees are encouraged to raise their hands at any point during the hands-on exercises so that a TA can quickly step in and resolve issues.

[01:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=77s) Workshop Architecture and Objectives

The core objective of the workshop is to build multimodal analytics workflows directly within BigQuery using standard SQL queries. Participants create database references to unstructured multimedia files stored in Google Cloud Storage—including photos, videos, and multilingual audio recordings featuring real pets, AI-generated animals, and the speaker's own cats. SQL statements are then used to extract AI-driven insights about these pets, such as their favorite foods, behaviors, and hobbies.

```mermaid flowchart LR accTitle: Multimodal Analytics and Application Architecture subgraph Storage["Google Cloud Storage"] Media["Pet Media Assets (Images, Audio, Video)"] end subgraph Processing["BigQuery & AI Services"] BQ["BigQuery Engine"] Conn["gRPC External Connection"] Gemini["Vertex AI / Gemini Platform"] BQ <--> Conn <--> Gemini end subgraph Hosting["Cloud Run"] App["Containerized Web Application (Vibe-Coded UI)"] end Media --> BQ BQ --> App ```

The data pipeline transitions from descriptive SQL analytics into generating vector embeddings, which are then integrated into a "vibe-coded" single-container web application hosted on Cloud Run. The speaker notes that vibe-coding an interactive app acts almost like a "choose your own adventure" game: its success relies heavily on how well users craft their prompts with Gemini.

- **Storage Layer**: Google Cloud Storage maintains master metadata and raw multimedia files (images, audio clips, and video). - **Processing Layer**: BigQuery serves as the analytical hub, orchestrating SQL queries, external connections, AI functions, and text generation. - **Application Layer**: Cloud Run hosts the resulting single-container web application to deliver interactive semantic search over multimodal assets.

[03:10](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=190s) Provisioning Lab Environments and Best Practices

Participants access the dedicated lab portal via `goo.gl/petverse-game` to launch their sandboxed Google Cloud environments. The speaker emphasizes keeping the link within the room due to limited seat capacity and advises everyone to use Google Chrome to ensure optimal compatibility. Attendees are encouraged to use Gemini or their preferred AI chatbot whenever they want deeper explanations of specific code snippets.

Clicking through the portal automatically provisions an ephemeral Google Cloud project and temporary student credentials starting with `student` and a project ID prefixed with `qwiklabs`. These sandboxed resources remain active for 90 minutes before automatic deletion, providing a risk-free playground to experiment without personal financial liability or company billing constraints.

The speaker highlights a critical step for avoiding session corruption: the Google Cloud Console must be opened in a clean Incognito/Private browser window with no other sessions active. This prevents accidental resource usage under personal accounts and guarantees that all assets deploy strictly within the lab-assigned region.

[07:45](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=465s) Cloud Shell Setup and Repository Initialization

Once inside the Google Cloud Console incognito session and after agreeing to the standard terms of service, participants open Cloud Shell in a persistent tab via `shell.cloud.google.com`. The speaker encourages attendees not to be intimidated by the command-line interface, reassuring them that "we can break stuff today, so don't worry about it."

To avoid character-garbling issues often caused by keyboard shortcut copying across operating systems, participants are instructed to use the lab interface's native copy buttons and paste into the terminal using `Ctrl+Shift+V` (or `Cmd+Shift+V` on macOS). After authorizing Cloud Shell, the terminal displays the student username alongside the highlighted project ID, confirming the environment is ready.

Attendees then clone the public workshop Git repository and execute an automated setup script. The underlying code remains publicly available so attendees can inspect or rerun the workflow on their own infrastructure long after the temporary lab environment expires.

[10:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=617s) BigQuery Connection and Multimodal Storage Setup

While the initial setup script runs, the speaker navigates to `console.cloud.google.com/bigquery` to unpack BigQuery's capabilities. BigQuery is highlighted as an enterprise-grade managed data warehouse engineered specifically for massive volume rather than transactional OLTP workloads, featuring native support for external object tables, Apache Iceberg, user-defined functions (UDFs), and graph analytics.

The setup script provisions an external cloud connection linking BigQuery to Google Cloud Storage and the Vertex AI / Gemini Enterprise Agent Platform. This backend link leverages high-speed gRPC communication, allowing SQL statements to seamlessly query external files and invoke foundation models behind the scenes without manual API glue code. The successful execution of the setup script is signaled by a terminal printout featuring a unicorn party emoji.

```mermaid flowchart TD accTitle: BigQuery External Connection and Data Ingestion Flow subgraph Terminal["Cloud Shell Execution"] GCloud["gcloud CLI Storage Command"] Repo["Public Workshop Repository"] end subgraph DataLake["Google Cloud Storage Bucket"] SampleBucket["Sample Source Bucket"] LabBucket["Lab Project Media Bucket"] SampleBucket -->|Copy media assets| LabBucket end subgraph BigQueryEngine["BigQuery Multimodal Pipeline"] Conn["gRPC Connection: Vertex AI / Gemini"] ObjectTable["BigQuery External Tables / Object References"] LabBucket --> ObjectTable Conn --> ObjectTable end Repo --> GCloud GCloud --> LabBucket ```

Using the `gcloud` CLI, participants copy sample pet media assets into their freshly created regional storage bucket. The speaker inspects the storage bucket to show the diverse dataset: - Image files (JPEGs and PNGs), including a sample dog named "Pixel". - Multilingual audio tracks designed to test cross-lingual audio processing. - Video clips, including one showcasing the speaker's own cat, "Sequel", whose favorite toy is a shoestring—setting up the upcoming exercise where BigQuery SQL statements will query the video directly to discover Sequel's favorite toy.

[14:43](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=883s) Self-Paced Walkthrough and Workshop Etiquette

Lucia Subatin acknowledges how distracting it can be to follow along with live instructions while trying to read and execute hands-on code: "If I were sitting there and I had somebody yapping at me while I'm trying to focus and do stuff, I would be losing my mind." For attendees who prefer following along at their own rhythm, she shares a pre-recorded walkthrough video. Because she added closed captions, attendees can follow along visually without sound, but she jokingly warns that anyone who turns up the volume needs to put on headphones: "If you're going to use audio because for some reason you like my voice, please wear headphones because that's going to be super annoying for everybody."

She checks in with the room to see how people are progressing through the lab, asking whether anyone is completely stuck or lost. Welcoming everyone into the "Petverse" hands-on environment, she jokes that she might regret her decisions before stepping away from the podium mic to circulate among the tables and assist attendees directly.

[16:48](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1008s) Catching Up with the Automated BigQuery SQL Script

Returning to the microphone to help attendees transition to the application-building phase before room time expires, Lucia introduces a catch-up script for anyone falling behind on manual configuration. She notes her personal engineering bias: "Personally, I'm not super excited about vibe coding in and of itself. I think we can do that at any time, and I prefer the BigQuery stuff." To fast-track the SQL setup, she points attendees to `bq_catchup.sql` inside the cloned `petverse-multimodal` repository.

```mermaid flowchart TD accTitle: BigQuery Catch-up Script Pipeline subgraph Prerequisites["Setup Prerequisites"] A["petverse_setup.sh"] --> C["GCS Storage Bucket"] B["Vertex AI Connection"] --> D["bq_catchup.sql"] end subgraph Execution["Catch-Up Execution"] D --> E["Replace REGION\n(e.g., europe-west4)"] E --> F["Replace PROJECT_ID\n(from GCP Picker)"] F --> G["Run in BigQuery Console"] end subgraph Features["IDE Assistance"] H["Cloud Shell / Editor"] --> I["Lightbulb Icon"] I --> J["Explain Code Feature"] end ```

Running the script requires two prior setup steps: executing the repo's `petverse_setup.sh` shell script in Google Cloud Shell and configuring the Google Cloud Storage (GCS) bucket. In the SQL editor, attendees use find-and-replace (`Ctrl+H`) to configure: - Replacing the placeholder `REGION` with their lab region (such as `europe-west4` or `us-central1`). - Replacing `PROJECT_ID` with their lab-assigned Google Cloud project identifier, copied from the GCP project picker or lab manual.

She also highlights a built-in productivity feature in Cloud Shell: highlighting any code block surfaces a small lightbulb icon that allows users to prompt Gemini Code Assist to explain the SQL code line by line.

[19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) Architectural Recap: Object References, AI SQL Functions, and Multimodal Embeddings

Lucia walks through the complete end-to-end architecture built during the session. The pipeline begins simply by loading a structured CSV into BigQuery, which is then extended into a multimodal analytics engine.

```mermaid flowchart LR accTitle: BigQuery Multimodal Analytics Flow subgraph Ingestion["Data Ingestion & Storage"] CSV["Pets CSV Table"] GCS["GCS Bucket\n(Video & Audio)"] end

subgraph BigQueryML["BigQuery SQL Processing"] OBJ["OBJECT_REF()\nConstructors"] AIFunc["AI.GENERATE / Vertex AI\n(Descriptions & Schema)"] EMB["Multimodal Embeddings\n(Text & Media Vectors)"] end

subgraph Discovery["Search Engine"] Search["Semantic Vector Search\n('Pets who like to chill')"] end

CSV --> OBJ GCS --> OBJ OBJ --> AIFunc AIFunc --> EMB EMB --> Search ```

- **Object References (`OBJECT_REF`)**: BigQuery tables link directly to raw multimedia assets stored in GCS buckets using BigQuery object reference functions, allowing queries to treat external video and audio files as first-class SQL references. - **Generative AI SQL Functions**: The workshop leverages native generative AI SQL functions such as `AI.GENERATE` (alongside related functions like `AI.GENERATE_TABLE`, `AI.IF`, `AI.AGGREGATION`, and `AI.SCORING`). - **Model Flexibility & Vertex AI Connections**: Through the connection ID initialized in `petverse_setup.sh`, `AI.GENERATE` can query Google's Gemini models or custom endpoints hosting open models (such as Hugging Face models hosted on Vertex AI Model Garden). - **Deterministic vs. Freeform Output**: While Gemini can generate unrestricted natural language, the queries explicitly constrain the output format (using structured JSON/schema generation) to ensure downstream SQL stability. - **Multimodal Embeddings & Semantic Search**: Numerical vector representations were created across both text and multimedia assets. This enables semantic search rather than brittle substring matching; for example, searching the concept *"Pets who like to chill"* accurately retrieves records where the pet's listed hobby is *"relaxing in the sun"*.

[22:09](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1329s) Vibe Coding Limitations and Session Wrap-Up

Lucia directs attendees to the recorded video for the frontend application creation, adding a frank disclaimer about AI-assisted code generation: "When it comes to the vibe coding, if you go into the video, you'll see a little bit of a rant me going, 'This is not going to be production-ready,' and an explanation of why."

Acknowledging that the reliability and safety of vibe coding remain contentious engineering topics, she invites anyone interested in debating the architectural nuances to chat with her outside the workshop room. She closes the session by thanking the room for their patience with venue Wi-Fi slowdowns and wraps up the presentation to audience applause.

## Notable quotes > [07:45](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=465s) we can break stuff today, so don't worry about it. — Lucia Subatin

> [14:43](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=883s) If I were sitting there and I had somebody yapping at me while I'm trying to focus and do stuff, I would be losing my mind. — Lucia Subatin

> [14:43](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=883s) If you're going to use audio because for some reason you like my voice, please wear headphones because that's going to be super annoying for everybody. — Lucia Subatin

> [16:48](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1008s) Personally, I'm not super excited about vibe coding in and of itself. I think we can do that at any time, and I prefer the BigQuery stuff. — Lucia Subatin

> [22:09](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1329s) When it comes to the vibe coding, if you go into the video, you'll see a little bit of a rant me going, 'This is not going to be production-ready,' and an explanation of why. — Lucia Subatin

## Mentioned Lucia Subatin, Google Cloud, Ai4, BigQuery, Google Cloud Storage, Cloud Run, Vertex AI, Gemini, Gemini Code Assist, Google Chrome, Cloud Shell, Apache Iceberg, Hugging Face, Vertex AI Model Garden

## Video description Lucia Subatin - Developer Advocate, Google Cloud

Important Note: This hands-on, interactive workshop is led by Google Cloud and is open to all Ai4 attendees at no additional cost. No RSVP or separate registration is required. To participate and get the most out of this session, please bring your own device (a laptop is recommended). Devices will not be provided on-site.

ABOUT THIS WORKSHOP: Learn how to search through video, audio and flat files from SQL statements to equip your applications and agents with multimodal analysis. Vibe-code a Cloud Run web application to see the power of semantic search over different types of files in action.

[Source](https://www.youtube.com/watch?v=jtwTWVmOA_Q)

中文简报

Google Cloud 将 BigQuery SQL 与 Gemini 关联以实现多模态数据搜索虽然基于 vibe 编码的应用尚不适用于生产环境，但 BigQuery 对象直接连接允许使用标准 SQL 查询从原始音频和视频文件中提取结构化洞察与向量嵌入。 Lucia Subatin 指导与会者配置临时 Google Cloud 环境，以构建端到端的多模态分析流水线。参与者利用对象引用将 Cloud Storage 中的多媒体资产与 BigQuery 关联，并执行由 Vertex AI 和 Gemini 驱动的生成式 AI SQL 函数，从而从图像、音频和视频中提取结构化洞察与向量嵌入。该环节演示了针对多种媒体资产的语义向量搜索，介绍了运行在 Cloud Run 上的容器化 Web 应用程序，并提供了便于自主把控进度的补充 SQL 脚本。最后，Subatin 对 vibe 编码应用的架构边界和生产可行性进行了坦率评估。工作坊展示了端到端的多模态分析架构：以 Google Cloud Storage 存储原始音视频资产，通过 BigQuery 结合 Gemini 执行 SQL 原生分析与向量嵌入，并将氛围编码（Vibe-coded）的前端应用部署至 Cloud Run。 (1:17) BigQuery 通过高速 gRPC 外部连接直接集成 Vertex AI 与 Cloud Storage，无需编写胶水代码即可通过标准 SQL 语句直接查询非结构化图像、多语言音频与宠物视频。 (10:17) 针对跟不上实操进度的参与者，工作坊提供了预置脚本 bq_catchup.sql，只需替换区域与 Project ID 即可一键恢复并快速搭建 BigQuery 多模态流水线。 (16:48) BigQuery 借助 OBJECT_REF 构造函数将云存储文件转为数据库一等引用，结合 AI.GENERATE 等原生 SQL 函数与多模态向量嵌入，实现了对非结构化数据的语义搜索（如检索“喜欢放松的宠物”）。 (19:26) 讲师 Lucia Subatin 提醒参与者，AI 氛围编码（Vibe coding）生成的代码并不具备生产环境就绪（Production-ready）的可靠性与安全性，适合作为快速原型而非工业级落地标准。 (22:09)

Reading summary

Google Cloud Developer Advocate Lucia Subatin demonstrates how BigQuery can operate as an enterprise multimodal analytics engine by using SQL to query unstructured media stored in cloud buckets alongside tabular data. However, she distinguishes in-database generative analytics from AI-assisted application generation, cautioning that "vibe-coding" front-end interfaces remains unready for production software. Multimodal Storage and SQL Object References (10:17) Traditional enterprise data warehouses are built for high-volume structured analytics rather than transactional OLTP workloads, but BigQuery extends this model to unstructured multimedia through external connections to Google Cloud Storage. Using gRPC-based connections to link the data warehouse with Vertex AI, the architecture allows standard SQL queries to interface directly with external object tables without requiring custom API glue code. Through object reference functions such as OBJECT_REF(), queries treat external assets—including image files (JPEGs and PNGs), multilingual audio tracks, and video clips—as first-class database references. By combining these object references with structured CSV tables, the database unifies disparate data types, such as querying video footage of Subatin's cat "Sequel" directly via SQL to determine the pet's behavior and favorite toy. Generative SQL Functions and Semantic Vector Search (19:26) The processing layer integrates native BigQuery generative functions—including AI.GENERATE, AI.GENERATE_TABLE, AI.IF, AI.AGGREGATION, and AI.SCORING—connected directly to Google's Gemini models or custom open models hosted on Vertex AI Model Garden. To preserve database consistency and prevent erratic natural language outputs from breaking downstream applications, the SQL queries enforce structured output schemas, such as formatted JSON. Beyond basic text descriptions, the pipeline generates numerical vector embeddings across both text and multimedia assets. These multimodal embeddings enable semantic vector search rather than rigid substring matching. For example, an abstract query such as "Pets who like to chill" successfully resolves to database records containing conceptually related attributes, such as an animal whose documented hobby is "relaxing in the sun." Production Limitations of AI-Prompted Application Development (22:09) While the workflow culminates in deploying a single-container web application to Cloud Run to deliver an interactive semantic search UI, Subatin expresses clear skepticism regarding prompt-driven "vibe-coding." She acknowledges her own engineering bias, noting that she prefers the rigor of BigQuery SQL pipelines over conversational front-end code generation. Subatin explicitly warns participants that code created purely through conversational prompting should not be considered production-ready. She frames vibe-coding as an interactive "choose your own adventure" that hinges heavily on prompt phrasing, pointing out that software reliability, safety, and architectural stability remain unresolved concerns in AI-generated application layers. Sandboxed Cloud Environments and Developer Tooling Workflows (3:10) Executing multimodal workflows safely requires isolated infrastructure to prevent session corruption and unexpected billing. The workshop provisions ephemeral Google Cloud projects through a dedicated lab portal, issuing temporary 90-minute student accounts prefixed with "qwiklabs" that automatically terminate. Participants are instructed to operate exclusively within Incognito browser windows to ensure all assets deploy strictly within the lab's assigned cloud region without interfering with personal accounts. To keep attendees on track within tight workshop timeframes, the workflow leverages Cloud Shell alongside automated setup and catch-up scripts like petverse_setup.sh and bq_catchup.sql. In addition, built-in IDE features—such as Gemini Code Assist lightbulb prompts in Cloud Shell—allow developers to generate line-by-line SQL explanations without leaving their terminal environment. The presentation demonstrates that BigQuery can effectively unify structured analytics, multimodal storage, and generative model inference under standard SQL. However, Subatin leaves the reliability of AI-generated application front-ends as an open limitation, maintaining that enterprise workflows require deterministic schemas and structured data management rather than unchecked vibe-coding.

精炼阅读摘要

Google Cloud开发者倡导者Lucia Subatin在Ai4工作坊中提出，企业可以直接利用BigQuery与Vertex AI及Gemini的原生集成，通过标准SQL对存储在Cloud Storage中的非结构化多模态数据进行分析与向量检索。同时，Subatin对当前技术界推崇的“氛围编程”（vibe coding）提出了明确的工程保留意见，指出完全由AI生成的原型代码在可靠性与安全性上难以达到生产级要求。 BigQuery多模态数据管道架构 (1:17) BigQuery在本次方案中被定位为核心分析中枢。作为面向海量分析而非传统事务型OLTP场景的托管数据仓库，BigQuery原生支持外部对象表、Apache Iceberg、用户自定义函数（UDF）及图分析能力。工作坊构建的数据流将Google Cloud Storage作为主数据与原始多媒体存储层，集中存放包含真实与AI生成的宠物图像、多语言音频以及视频文件等非结构化资产。通过配置脚本，系统在BigQuery与Vertex AI/Gemini企业代理平台之间建立了基于gRPC协议的高速外部云连接。借助BigQuery的对象引用函数OBJECT_REF()，分析人员能够将对象存储中的视频与音频等多媒体资源直接作为SQL表中的一等公民进行处理，无需编写繁杂的手工API胶水代码，即可通过SQL语句直接提取多媒体资产中的结构化信息。生成式SQL与跨模态向量语义检索 (19:26) 在数据处理层，该方案将基础的结构化CSV表格分析平滑扩展为多模态智能引擎。系统引入了一系列原生生成式AI SQL函数，涵盖AI.GENERATE、AI.GENERATE_TABLE、AI.IF、AI.AGGREGATION以及AI.SCORING。借助底层的外部连接，这些函数不仅能够调用Google原生的Gemini基础大模型，还允许连接部署在Vertex AI Model Garden上的开源模型（如Hugging Face模型）。针对生成式模型自由文本输出的不确定性，查询流程通过强制定义结构化JSON Schema规范，确保模型输出符合预期，维持下游SQL数据流的稳定性。此外，管道为文本及多媒体文件生成了数值型多模态向量嵌入，实现了高容错的跨模态语义搜索。例如，用户检索“喜欢放松的宠物”这一抽象语义概念时，系统无需依赖传统子字符串匹配，即可准确命中活动描述为“在阳光下放松”的相关记录。沙盒环境治理与自动化脚本加速 (3:10) 为了在45分钟的紧凑时间内保障实操安全，工作坊基于Qwiklabs平台为学员分配了90分钟自动销毁的沙盒项目，生成带有student前缀与qwiklabs标识的临时凭据，规避学员个人与企业的财务扣费风险。组织方特别强调，学员必须在Chrome浏览器的无痕窗口（Incognito）中操作，以彻底隔离个人账户会话，防止资产误部署或区域配置冲突。针对动手实操中的排障需求，部署流程依托Cloud Shell与公开Git仓库展开，通过petverse_setup.sh脚本自动完成存储桶创建、多媒体资产拷贝以及gRPC连接验证。为了协助因跨平台粘贴格式错误或环境问题掉队的学员，Subatin提供了bq_catchup.sql脚本，学员仅需批量替换区域代码（如europe-west4）和项目ID即可快速同步进度；同时，学员还可以利用Cloud Shell内嵌的Gemini Code Assist代码助手，点击代码旁的灯泡图标获取逐行SQL解释。氛围编程前端部署与其生产局限 (22:09) 在终端应用交付层面，工作坊展示了将多模态分析结果与向量检索能力接入单容器Web应用的方法，该应用通过Cloud Run进行托管。前端界面的搭建采用了自然语言提示词驱动的“氛围编程”（vibe coding）模式，由学员通过与Gemini交互生成交互式UI，Subatin将这种探索过程类比为“选择你自己的冒险游戏”，其成效高度依赖于提示词的设计水准。然而，Subatin在技术分享中明确表达了自己的工程偏好，坦言自己更重视BigQuery底层数据流建设，对氛围编程本身“并不特别兴奋”。她向学员提出明确的技术免责警示，强调通过氛围编程快速搭建的代码在现阶段“无法达到生产就绪状态”（not production-ready）。她指出，AI辅助生成的代码在工程安全性与体系稳定性方面仍存在巨大争议，开发者切不可将原型阶段的快速出码等同于企业级交付。本场工作坊验证了将BigQuery与Vertex AI深度整合以实现轻量化、SQL化多模态分析的技术可行性，极大降低了对音视频等非结构化资产提取智能特征的门槛。然而，主讲人对氛围编程生产可用性的直白警示也表明，在探索大模型辅助前台开发的敏捷性时，企业仍需将底层经过严格验证的数据库管道治理与实验性的快速代码生成清晰解耦。

Detailed notes — zh

## Background 本内容由频道 Ai4 于 2026 年 8 月 15 日在 YouTube 平台发布。视频记录了 Google Cloud 开发者布道师 Lucia Subatin 主持的动手实操研讨会，介绍如何通过 SQL 在 BigQuery 中分析音视频及多模态数据，并借助“氛围编码（Vibe-coding）”在 Cloud Run 上构建语义搜索 Web 应用。 ## TL;DR 研讨会围绕 Google Cloud 上的多模态数据分析与应用构建展开。参与者在临时的 Google Cloud 沙盒环境中，利用 BigQuery 的外部连接和内置 AI 函数（如 AI.GENERATE），直接通过 SQL 对存储在 Cloud Storage 中的宠物图片、音频和视频进行向量化与多模态分析。讲师还提供了快捷脚本与 Cloud Shell 的 Gemini AI 辅助功能，展示了从结构化与非结构化数据整合到跨模态语义检索的完整链路，并对“氛围编码”生成的原型距离生产环境就绪的差距给出了理性审视。 ## Atmosphere & style 现场节奏极其紧凑且充满活力，讲师在幽默风趣与专业严谨之间自如切换，时不时穿插关于自己养的猫“Sequel”以及调侃自己声音的自嘲段子来活跃气氛。随着 45 分钟时间限制的推进，讲师主动走下讲台提供点对点排错支持，展现出极具亲和力与务实的技术指导风格。整场研讨会既有动手实操的紧张感，又充满了互动探索的极客乐趣。 ## Key takeaways - [01:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=77s) 工作坊核心目标是通过 BigQuery SQL 对 Cloud Storage 中的多模态文件（视频、音频、图片）进行语义分析与向量生成，并借助 Vibe Coding 快速部署 Cloud Run Web 应用。 - [01:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=77s) 端到端系统架构由三层组成：Cloud Storage 负责媒体资产存储，BigQuery 与 Vertex AI/Gemini 负责 AI 处理与向量计算，Cloud Run 托管前端交互应用。 - [02:49](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=169s) 沙盒实验环境采用动态分配的临时 Google Cloud 账号，生命周期为 90 分钟，必须在无痕窗口（Incognito Window）中运行以防止会话冲突和个人账户意外扣费。 - [07:56](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=476s) BigQuery 通过底层 gRPC 协议与 Vertex AI / Gemini 建立高速安全连接，支持分析人员无需编写复杂管道代码即可直接通过 SQL 驱动多模态大模型。 - [12:24](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=744s) 多模态数据集中包含多语言音频和不同格式的视频素材，专门用于压力测试大模型对非结构化数据的理解与跨模态特征提取能力。 - [16:48](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1008s) 仓库提供了 bq_catchup.sql 快速同步脚本，学员只需全局替换区域和项目 ID 参数，即可一键自动构建所有 BigQuery 数据表、外部连接及多模态对象引用。 - [16:48](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1008s) Cloud Shell 编辑器内置了基于 Gemini 的 AI 辅助功能，选中文段代码即可通过侧边灯泡图标获取代码逻辑解析，降低复杂 SQL 的学习门槛。 - [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) BigQuery 通过 ObjectRef 机制将表格中的结构化元数据与存储桶中的非结构化音视频媒体无缝绑定，实现多模态数据的统一管理。 - [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) BigQuery 提供了丰富的内置 AI 函数矩阵（包括 AI.GENERATE、AI.GENERATE_TABLE、AI.IF、AI.AGGREGATE、AI.SCORE），支持 Schema 约束输出，并可桥接 Gemini 及 Hugging Face 等第三方模型。 - [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) 相比传统字符匹配，基于文本与媒体向量的多模态语义搜索（Semantic Search）能够跨越字面表达限制，实现自然语言查询与媒体语义的深层次精准召回。 - [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) 讲师指出纯粹的“氛围编码（Vibe Coding）”虽然开发速度极快，但生成的原型距离生产就绪（Production-ready）标准仍有巨大差距，核心价值仍在底层数据引擎的设计与构建。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=4s) 研讨会开场与背景介绍 Google Cloud 的资深开发者布道师（Staff Developer Advocate）Suvi Rajan 向现场观众致欢迎辞。她坦言今天的时间非常紧迫，原本需要大约 1 小时的实验内容被压缩在 45 分钟内，因此整体节奏会非常快。她笑称自己的名字非常独特，“只要在社交媒体上搜到这个名字，大概率就是我本人”，并欢迎大家在会后前往 Google Cloud 展位继续深入交流。在正式动手之前，讲师对现场听众的技术背景进行了快速举手调研，询问了大家平时使用 Google Cloud 控制台以及 BigQuery 的情况。针对现场部分自认为“非技术背景”的参会者，讲师贴心安抚大家不用担心，并介绍了现场的多位助教（TA），提醒大家在实验过程中随时举手获取支持。 - **主讲人**：Suvi Rajan（Google Cloud Staff Developer Advocate） - **时间限制**：课程通常需 1 小时，现场压缩至 45 分钟完成 - **现场支持**：配备多名助教随时协助排查操作问题 --- ### [01:17](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=77s) 核心实验目标与多模态系统架构本次研讨会的核心是在 BigQuery 中通过 SQL 语句对存储在 Google Cloud Storage 中的多模态文件（视频、图片、音频等）进行分析。这些多媒体资产涵盖了讲师自己养的猫、其他人的宠物以及由 AI 生成的虚拟宠物。参与者将利用 SQL 和生成式 AI 技术分析宠物的喜好、饮食习惯与行为特征，生成向量嵌入（Embeddings），并最终通过“氛围编码”（Vibe Coding）与 Gemini 交互，构建并部署一个运行在 Cloud Run 上的交互式 Web 应用。从系统架构来看，Cloud Storage 负责保存所有的多媒体原始文件和宠物主数据；BigQuery 作为中心枢纽，负责执行 AI 函数处理、文本生成与多模态数据分析；最终生成的容器化 Web 应用则托管在 Google Cloud Run 上，形成一个轻量但功能强大的端到端分析闭环。 ```mermaid flowchart LR accTitle: 多模态分析与应用部署架构图 subgraph StorageLayer["存储层"] A["Cloud Storage (图片/音频/视频/主数据)"] end subgraph AnalyticsLayer["分析与 AI 层"] B["BigQuery (SQL / 外部连接)"] C["Gemini / Vertex AI (多模态分析 / 向量生成)"] end subgraph ServingLayer["应用层"] D["Cloud Run (单容器 Web 应用)"] end A -->|"媒体引用与元数据"| B B <-->|"gRPC 高速调用"| C B -->|"检索与嵌入向量"| D ``` - **数据存储**：Google Cloud Storage（存放图片、音视频及主数据） - **数据处理与 AI 分析**：BigQuery（集成 Vertex AI / Gemini 平台能力，通过 SQL 进行向量计算与内容理解） - **应用托管**：Cloud Run（单容器 Web 服务，通过 Vibe Coding 方式快速生成） --- ### [02:49](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=169s) 实验环境准备与无痕模式配置讲师展示了本次实验的专属入口（`goo.gl/petverse-game`），并特别提醒现场观众不要将链接转发到场外，因为本次实验提供的沙盒席位有限。讲师开玩笑表示：“唯一会让我吐槽你的，就是你用的不是 Chrome 浏览器”，建议大家统一使用 Chrome 获得最佳兼容性。进入实验平台并通过人机验证后，系统会自动为每位学员动态配置一个临时的 Google Cloud 项目和专属账号。该沙盒环境生命周期为 1.5 小时，超时后会自动销毁释放资源。讲师重点强调：**必须通过右键在无痕窗口（Incognito Window）中打开 Google Cloud 控制台**。这样做一方面是为了防止已有登录会话干扰，另一方面也是为了确保学员不会误用自己的个人账单账户来扣费，“我们想为大家买单，绝不希望大家自己花钱”。 - **实验入口**：`goo.gl/petverse-game` - **沙盒生命周期**：90 分钟（超时后项目与临时账号自动删除，代码本身为公开开源） - **环境信息**：项目 ID 通常以 `qwiklabs-` 或 `quicklabs-` 开头，用户名为 `student-` 开头 - **关键规则**：必须使用全新的**无痕模式窗口**登录，杜绝个人项目产生意外扣费 --- ### [07:56](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=476s) 启动 Cloud Shell 与初始化 BigQuery 连接在无痕浏览器中登录 Google Cloud 控制台并同意服务条款后，讲师引导大家在新标签页中打开 Cloud Shell（`shell.cloud.google.com`），该终端标签页将在整个研讨会期间保持开启。为避免手动复制产生乱码，讲师推荐使用界面上的复制按钮，并在终端中使用 `Ctrl + Shift + V`（或 Mac 的 `Cmd + Shift + V`）进行粘贴。随后，学员克隆了公开的实验代码仓库并执行安装初始化脚本。在脚本运行期间，讲师向大家科普了 BigQuery 的核心优势：它是一个专为海量数据处理优化的全托管分析引擎，不仅可以直连 Cloud Storage 和 Apache Iceberg 等外部数据源，还支持图分析（Graph capabilities）与用户自定义函数（UDF）。初始化脚本在后台创建了一个外部连接，利用底层的 gRPC 协议高速打通 BigQuery、Cloud Storage 与 Gemini 企业智能体平台（原 Vertex AI），使分析人员无需编写底层管道代码即可直接通过 SQL 驱动大模型。 - **初始化终端**：在无痕标签页保持打开 `shell.cloud.google.com` - **执行命令**：克隆公开仓库并运行 `setup.sh` 初始化脚本 - **成功标识**：脚本运行成功后，终端会输出一只正在开派对的独角兽 Emoji（🦄🎉） - **底层打通**：通过 BigQuery Connection 借助 gRPC 直连 Vertex AI / Gemini 平台，支持极低延迟的多模态 SQL 计算 --- ### [12:24](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=744s) 复制多模态宠物数据与媒体内容探索环境配置完毕后，讲师演示了如何使用 `gcloud` 命令行工具从公共样本存储桶中将多模态数据复制到当前项目的 Cloud Storage 存储桶中。随后在控制台的存储桶页面中，大家可以看到已经成功同步了各类宠物的 JPEG、PNG 图片、多语言音频文件以及视频素材。讲师现场展示了其中的几份媒体文件，包括一只名为“Pixel”的帅气宠物照片，以及一段视频。视频中的主角正是讲师自己养的猫——名字恰好就叫“Sequel”（与 SQL 同音）。视频里展示了 Sequel 最喜欢的玩具是一根鞋带。讲师风趣地预告道：“接下来我们就要编写 SQL 语句，让大模型通过视频和音频告诉我们，Sequel 最喜欢的玩具到底是什么。” - **数据迁移**：通过 `gcloud` CLI 从公共 Bucket 同步图片、多语言音频与视频素材 - **数据多样性**：特意包含多种语言的音频与不同格式的视频，用于压力测试多模态大模型的理解泛化能力 - **实战案例**：分析讲师的猫“Sequel”玩鞋带的视频片段，通过 SQL 驱动多模态理解提取其爱好特征 ### [14:43](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=883s) 现场实操支持与视频跟练指引讲师 Lucia Subatin 引导大家查看之前配置的 Google Cloud Storage (GCS) 存储桶，并体贴地考虑到了现场不同学员的实操节奏。她幽默地调侃道，如果自己坐在台下全神贯注地写代码，旁边却总有人在耳边“喋喋不休（yapping）”，她肯定会抓狂。因此，为了满足希望按照自己节奏学习的学员，她准备了一段带有完整字幕的操作演示视频。她特别提醒大家：“如果你非要开着声音听我讲解——虽然不知道为什么会有人喜欢听我的声音——请务必戴上耳机，否则外放声音会对周围的人造成很大干扰。”她笑着补充说，在录音或视频里听到自己的声音感到别扭几乎是所有人的“普遍共识”。随后，Lucia 在场地内巡视，确认现场学员的实验进度，询问是否有人遇到阻碍或彻底在实验流程中“迷失”。她打趣地表示欢迎所有人来到“宠物元宇宙（Petverse）”，并笑称自己可能“很快就会为这个决定后悔”。为了不通过主讲台的音响持续轰炸全场，她暂时关闭了无线麦克风，走到学员中间进行点对点的个别答疑与排错支持。 - **视频跟练指引**：演示视频配备了完整英文字幕，学员无需开启音频即可静音跟做。 - **现场实操氛围**：讲师深入坐席区域协助排查实验环境配置、网络连接及权限问题。 --- ### [16:48](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1008s) BigQuery 快速同步脚本与 Cloud Shell 辅助功能由于研讨会时间临近尾声且场地即将清场，Lucia 登台分享了一个帮助大家迅速跟上进度的“逃生通道（Catch-up Script）”。她坦言，虽然本次工作坊的主题包含“氛围编码（Vibe Coding）”，但她个人对纯粹的氛围编码并没有那么狂热，因为氛围编码随时都可以尝试，而她更看重且偏爱的是底层 BigQuery 多模态数据引擎的搭建逻辑。为了让大家能迅速跳过繁琐的手动建表与函数配置，直接体验前端应用构建，她在配套代码仓库中提供了一个名为 `bq_catchup.sql` 的脚本。 ```mermaid flowchart LR accTitle: BigQuery快速配置流程图 subgraph Setup["环境初始化"] A["petverse_setup.sh"] --> B["配置 GCS Bucket"] end subgraph Config["脚本参数替换 (Ctrl+H)"] B --> C["替换 REGION (如 europe-west4)"] C --> D["替换 PROJECT_ID"] end subgraph Execute["BigQuery 执行"] D --> E["运行 bq_catchup.sql"] E --> F["自动构建多模态数据表与AI配置"] end ``` 该脚本的执行依赖两个前置条件：首先需在 Google Cloud Shell 中运行仓库自带的 `petverse_setup.sh` 初始化脚本，其次需完成 GCS 存储桶的创建与多媒体资产上传。在 BigQuery 控制台中粘贴 `bq_catchup.sql` 后，学员只需使用全局替换快捷键 `Ctrl + H` 进行两项参数修改： 1. 将 `REGION` 批量替换为实验所在区域（如 `europe-west4` 或 `us-central1`）。 2. 将 `PROJECT_ID` 替换为当前 Google Cloud 项目 ID（可从控制台项目选择器或实验手册中复制）。执行后，脚本将自动完成所有此前需要手动创建的 BigQuery 数据表、外部连接和多模态对象引用配置。此外，Lucia 还分享了 Cloud Shell 的内置 AI 辅助功能：在编辑器中选中文段代码时，侧边会出现一个“小灯泡”图标，点击即可调用 Gemini 解释代码逻辑，极大降低了理解复杂 SQL 与多模态管道的门槛。 --- ### [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) 多模态架构全流程回顾与语义搜索实战在工作坊的总结阶段，Lucia 系统梳理了在 BigQuery 中实现多模态分析与语义搜索的核心架构链路。首先，系统从基础 CSV 文件加载宠物基本属性，随后利用 BigQuery 特有的 `ObjectRef` 结构向数据表中扩展多媒体列，使表格行直接关联存储桶中的视频和音频文件。通过专用的 `ObjectRef` 内置函数，BigQuery 能够将结构化表格数据与非结构化多媒体资产无缝绑定。 ```mermaid flowchart TD accTitle: BigQuery多模态语义搜索架构图 subgraph DataIngest["数据层与对象引用"] CSV["CSV 宠物元数据"] --> PetsTable["BigQuery pets 核心表"] GCS["GCS 存储桶 (音视频文件)"] -->|ObjectRef 函数| PetsTable end subgraph AIEngine["AI 函数与向量生成"] PetsTable -->|AI.GENERATE 提示词工程| DescGen["生成结构化描述"] PetsTable -->|Vertex AI 文本模型| TextEmbed["文本向量 (Text Embeddings)"] GCS -->|Vertex AI 多模态模型| MediaEmbed["媒体向量 (Media Embeddings)"] end subgraph Search["语义检索层"] Query["自然语言查询: 'Pets who like to chill'"] --> VectorSearch["向量近似检索 (Semantic Search)"] TextEmbed -.-> VectorSearch MediaEmbed -.-> VectorSearch VectorSearch ==> Result["精准召回: 喜好为'在阳光下放松'的宠物"] end ``` 在 AI 能力调用上，Lucia 重点剖析了 BigQuery 内置的 AI 函数生态（包括 `AI.GENERATE`、`AI.GENERATE_TABLE`、`AI.IF`、`AI.AGGREGATE`、`AI.SCORE` 等）。其中，`AI.GENERATE` 是最具通用性的函数，支持在 Prompt 中自由插值数据表字段，并通过安全连接调用 Vertex AI 平台上的 Gemini 大模型，或挂载部署在平台上的 Hugging Face 等第三方开源模型；在生产实践中，通常会通过 Schema 约束其输出以保证确定性（Deterministic）。随后，系统分别为文本描述与音视频媒体生成高维数学向量表示（Text Embeddings 与 Media Embeddings），并在 BigQuery 内部实现多模态语义搜索（Semantic Search）。Lucia 强调了语义搜索与传统逐字字符串匹配（Character-by-character search）的本质区别：当用户检索“喜欢躺平放松的宠物（Pets who like to chill）”时，传统检索无法命中字面不符的记录，而语义向量检索能够精确召回爱好描述为“在阳光下放松（relaxing in the sun）”的宠物，实现了跨模态、跨语义维度的深度匹配。在收尾环节，Lucia 针对“氛围编码（Vibe Coding）”提出了理性的审视。她指出，自己在视频末尾有一段关于工程落地的深入讨论，核心观点在于“氛围编码生成的应用距离具备生产就绪性（not going to be production-ready）还有巨大差距”。她表示这是一个值得深入探讨的话题，欢迎大家在会后继续交流，并向克服现场网络延迟、坚持完成实验的学员们表达了诚挚的谢意。 - **BigQuery AI 函数矩阵**：涵盖 `AI.GENERATE`（文本/描述生成）、`AI.GENERATE_TABLE`（表格扩充）、`AI.IF`（条件逻辑判定）、`AI.AGGREGATE`（智能聚合）以及 `AI.SCORE`（智能评分）。 - **模型生态集成**：BigQuery 连接不仅原生支持 Gemini 模型端点，还支持通过 Model Endpoint 桥接 Hugging Face 等自定义托管模型。 - **跨模态向量检索**：支持对纯文本与音频/视频多模态向量运行统一检索空间下的近邻匹配。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=4s) 只要在社交媒体上搜到这个名字，大概率就是我本人 — Suvi Rajan > [02:49](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=169s) 唯一会让我吐槽你的，就是你用的不是 Chrome 浏览器 — Suvi Rajan > [02:49](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=169s) 我们想为大家买单，绝不希望大家自己花钱 — Suvi Rajan > [12:24](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=744s) 接下来我们就要编写 SQL 语句，让大模型通过视频和音频告诉我们，Sequel 最喜欢的玩具到底是什么。 — Suvi Rajan > [14:43](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=883s) 如果你非要开着声音听我讲解——虽然不知道为什么会有人喜欢听我的声音——请务必戴上耳机，否则外放声音会对周围的人造成很大干扰。 — Lucia Subatin > [14:43](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=883s) 宠物元宇宙（Petverse） — Lucia Subatin > [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) 氛围编码生成的应用距离具备生产就绪性（not going to be production-ready）还有巨大差距 — Lucia Subatin > [19:26](https://www.youtube.com/watch?v=jtwTWVmOA_Q&t=1166s) 喜欢躺平放松的宠物（Pets who like to chill） — Lucia Subatin ## Mentioned Ai4, Apache Iceberg, BigQuery, Chrome, Cloud Run, Cloud Shell, Gemini, Google Cloud, Google Cloud Storage, Hugging Face, Lucia Subatin, Pixel, Sequel, Suvi Rajan, Vertex AI ## Video description Lucia Subatin - Developer Advocate, Google Cloud Important Note: This hands-on, interactive workshop is led by Google Cloud and is open to all Ai4 attendees at no additional cost. No RSVP or separate registration is required. To participate and get the most out of this session, please bring your own device (a laptop is recommended). Devices will not be provided on-site. ABOUT THIS WORKSHOP: Learn how to search through video, audio and flat files from SQL statements to equip your applications and agents with multimodal analysis. Vibe-code a Cloud Run web application to see the power of semantic search over different types of files in action. [Source](https://www.youtube.com/watch?v=jtwTWVmOA_Q)

Source quotes

[0:04] Suvi Rajan: 只要在社交媒体上搜到这个名字，大概率就是我本人 [2:49] Suvi Rajan: 唯一会让我吐槽你的，就是你用的不是 Chrome 浏览器 [2:49] Suvi Rajan: 我们想为大家买单，绝不希望大家自己花钱 [12:24] Suvi Rajan: 接下来我们就要编写 SQL 语句，让大模型通过视频和音频告诉我们，Sequel 最喜欢的玩具到底是什么。 [14:43] Lucia Subatin: 如果你非要开着声音听我讲解——虽然不知道为什么会有人喜欢听我的声音——请务必戴上耳机，否则外放声音会对周围的人造成很大干扰。 [14:43] Lucia Subatin: 宠物元宇宙（Petverse） [19:26] Lucia Subatin: 氛围编码生成的应用距离具备生产就绪性（not going to be production-ready）还有巨大差距 [19:26] Lucia Subatin: 喜欢躺平放松的宠物（Pets who like to chill）

Source quotes

[7:45] Lucia Subatin: we can break stuff today, so don't worry about it. [14:43] Lucia Subatin: If I were sitting there and I had somebody yapping at me while I'm trying to focus and do stuff, I would be losing my mind. [14:43] Lucia Subatin: If you're going to use audio because for some reason you like my voice, please wear headphones because that's going to be super annoying for everybody. [16:48] Lucia Subatin: Personally, I'm not super excited about vibe coding in and of itself. I think we can do that at any time, and I prefer the BigQuery stuff. [22:09] Lucia Subatin: When it comes to the vibe coding, if you go into the video, you'll see a little bit of a rant me going, 'This is not going to be production-ready,' and an explanation of why.

Original transcript

[0:04] So, we only have 45 minutes and normally this thing takes closer to an hour. [0:09] So, I'm going to go fast. My name is Suvi Rajan. I'm a staff developer advocate at Google Cloud. [0:15] If you want to connect with me, I have some social media there. I am lucky to have a unique name. So, [0:23] if you find me, it's probably me. So, feel free to connect and otherwise I will be at the booth [0:29] after this. So, if you want to have further conversations about what we're doing today, [0:33] I'm happy to chat. Just by a little show of hands to know what we're doing, how many of you are [0:39] using or have used the Google Cloud console today? [0:46] All right. Nice. How many of you have used BigQuery? [0:51] Cool. All right. How many of you would consider yourselves not technical? [0:59] Cool. We got you. No worries. So, I have my wonderful TAs over there, my teaching assistants. [1:05] Please raise your raise your hands. They will be helping you out with any questions. So, at [1:10] any point if you get stuck, please raise your hand and somebody will come in and help you with that. [1:17] What we're doing today. We are going to be creating some artifacts within BigQuery. [1:24] We are going to be using SQL most of the time and we're going to be [1:28] creating references to multimedia files that we'll have in storage. Those are going to be [1:34] videos, pictures, audio files. Some of them are my cats and some of them are somebody else's pets [1:41] and some AI-generated pets and we're going to use SQL to analyze what these pets like, [1:47] what they like to eat, what their hobbies are and we're going to do some AI magic with that. [1:54] The we're going to create embeddings as well. That's going to be pretty fast, but we're going to [1:59] use those embeddings in a vibe coded application. The vibe coded application will be very similar [2:06] to you doing, basically, your own adventure. So, it's up to you whether it works or not, [2:12] or basically Gemini and and how you prompt it, but we'll help you with that as well. [2:19] A little bit of what our architecture marketecture looks like. We are going to have [2:24] uh the cloud storage holding all of our multimedia and some master data from our pets. We're going to [2:30] use BigQuery centrally to do all of the processing of our AI functions and some generation of [2:37] uh text and descriptions. And finally, Cloud Run is going to host our application, [2:43] which is going to be a web application in a single container. So, I'm going to show you [2:49] uh some slides that I'm going to ask you to please not share it outside. And the reason for that is [2:55] we have a limited amount of seats that we can offer today for you to complete this lab. And if [3:01] we start sharing this outside of people who may not be here today may be taking up those seats. [3:10] And that is where the lab is going to be for you folks. [3:14] So, we're going to head over to goo.gl/petverse-game. [3:24] When you are there, you should be met with something like this. [3:28] We're all here to learn. So, if you have questions about what a piece of code is doing, [3:33] feel free to ask Gemini or your uh AI chatbot of choice. [3:39] Uh the only thing that I will shame you about is if you're using a browser that isn't Chrome. [3:47] Uh and that is not just because I'm petty, but it's mostly because [3:52] it works better if you're using Chrome. So, please open your Chrome browsers. [3:58] You're going to be signing in. You can create an account if you do not have one. [4:03] Hopefully, I can sign in. Great. I'm going to get started. Don't worry if you're [4:08] lagging a little bit behind. I have a little trick for you folks to catch up. [4:15] The beauty of this, we just click start and it's going to, of course, [4:20] figure out that I'm not a robot. Ask me these things that I almost always fail the first time. [4:30] This time I didn't. [4:33] And this is going to provision a Google Cloud project for you and uh an account, a Google [4:39] account. This Cloud project is going to be deleted together with the account after an hour and a [4:46] half, which is a little bit more than the time that we have here. So, if you want to continue [4:52] playing outside, the lab will continue going. However, after the hour and 30 minutes mark, [4:58] it will just shut off and delete everything. The good news is this is publicly available, [5:04] so you can do this at home. The code is not going to go away. Uh but the the actual resources [5:10] are going to go away. And that actually gives us a little bit more freedom to do some things [5:15] that we normally wouldn't do in our companies' projects or uh when we are footing the bill. [5:24] This is my project ID. You are going to have your own project ID. It's going to be [5:31] slightly different, but the important thing is it's going to start with Quick Labs. My username [5:36] starts with student. Yours will start as with that as well. And then we have the password [5:41] and we have a region. This region is going to be important mostly when you're live coding [5:47] because it will be different for you and that's on purpose. We are going to create all of our assets, [5:53] all of our database stuff, and storage bucket in the region that is assigned to us by the labs, [5:59] and then ideally the application should be in the same region, you know, because of latency. [6:07] So, I'm going to [6:11] right click here, and this is super important. The success of this lab [6:18] depends on this very specific thing. First, I do not have any other incognito windows open. [6:25] Uh the reason for that is I do not want to pollute the sessions. I want to only use the session that [6:29] was created for this. And more the most important reason for that is I don't want you folks to give [6:37] us money for this lab. We want to cover this. And if you're signed in with your own users, there's [6:43] a chance that you accidentally use your own projects, and that is no good. So, we right click [6:50] and we open in the incognito window. It will hopefully automatically sign me in. [7:04] It's going to ask me a bunch of stuff as well. Yes, I do understand. Again, this [7:10] uh account is going to get deleted in a bit. [7:16] And we are in the Google Cloud console. Of course, I agree to the terms of service. [7:28] And moving forward, I will be interacting mostly with the incognito window, and my [7:37] um instructions are going to remain in this non-incognito window. [7:45] We have the objectives that we already explained, so I'm going to skip those. [7:51] What I just said is also specified here in case you missed it. [7:56] And we're going to start the Cloud Shell. The Cloud Shell, the first time you're touching it, [8:01] it may be a little bit daunting, but we can break stuff today, so don't worry about it. [8:08] So, I just copied shell.cloud.google.com and I'm going to paste it in a new tab in this [8:13] incognito window. This tab is going to remain open for the entirety of our session today. [8:25] We're noting the project ID and the region that we have before, [8:30] and we're going to start pasting stuff into the Cloud Shell. [8:36] I'm going to use this copy button. You could potentially select control C or command C and then [8:43] control shift V later. Uh sometimes it garbles the characters, so use the button whenever possible. [8:53] We're going to click authorize. [9:13] Okay, so hopefully that is a bit visible for everybody at the back. More? Yeah, I wear glasses, [9:20] so I feel you guys. Good. Thank you. And I'm going to paste here. To paste, control shift [9:28] V or command shift V if you're on a Mac. I have a Windows computer, and then we press enter, [9:35] and it's going to start executing commands. We know that we are doing the right thing because [9:40] my username starts with student. My Google Cloud project is here in yellow. [9:48] And now we are ready to make this work. Uh we're going to see this message. This is not [9:54] for you to copy. And we're going to clone a repository that as I said is going to remain [10:01] with you folks afterwards. So, do not worry about saving this. It's publicly available. [10:11] And we're going to run the setup script. [10:17] While this runs, I'm going to explain what we are doing with the script. The script is going to [10:22] get into BigQuery. For that, I'm going to get ahead of this, but just to show you what it [10:27] looks like. I'm going to open a new tab. And I'm going to go into console .cloud.google.com [10:38] /bigquery. You'll have this URL. You do not have to memorize it. [10:50] And for those of you who are seeing BigQuery for the first time, BigQuery is a managed database [10:55] that is optimized for volume. And it can do a lot of things. Uh some of the things that it can do is [11:01] tap into storage like we're going to do today. And it's not going to tap into storage just as [11:06] references for a multimedia. You can use it uh with Apache Iceberg as well if you want. It can [11:12] execute different external functions or UDFs as well if you want. It has graph capabilities, which [11:21] are pretty neat. And again, it's optimized for volume. You wouldn't use this for transactions, [11:26] even though I've seen some people do it. But just because you can doesn't mean you should. [11:32] Right now, um we are going to create a connection. Actually, my script just created the connection. [11:38] This connection is my connection to other Google Cloud resources. In our case, we're going to use, [11:44] uh, storage and, um, Gemini Enterprise Agent Platform. We changed the name from Vertex AI, [11:51] and I'm still getting used to it. Uh, but basically, this is what will allow us to [11:57] execute SQL and have BigQuery go through the back end, through GRPC, really fast, connect [12:04] into all of these services, and tap them, and you will not have to be bothered by any of that. [12:09] Which is pretty nice. My script worked, and I know that it worked because I see the emoji [12:16] of a unicorn having a party. Hopefully, you do, too, and we can join their party. [12:24] And now, we are going to create that storage bucket. That will be used saying for our [12:31] multimedia analytics. So, this is the G Cloud CLI. You probably have seen this before. It's just like [12:38] any other CLI. It allows us to interact from the terminal with all of our Google Cloud resources. [12:46] And as you can see here, it's copying from the sample data and media storage bucket that has [12:52] a bunch of, um, files for our pets. So, if I go into my storage buckets, now I should see [13:05] what we just created, the new bucket and the copy of our beautiful pets. [13:13] So, this is what we just did. We can see that we have all of these JPEGs and PNGs for [13:21] different pets. And, for example, I'm going to take this handsome guy called Pixel. [13:30] And we can see the preview here. If you're curious about it, you should have [13:35] your Cloud Shell as well. Then, if we want to look at the different additional media that we have, [13:46] we can see here that we have different types of of files. So, we have audio, [13:53] we have video. Some of these audios are in different languages as well, and this is on [13:58] purpose because we want to test everything cuz why not? So, if you were to open this [14:08] I'm going to try and preview this. [14:16] We can see the cat. [14:20] This is Sequel, and his favorite toy is a shoestring. [14:27] So, we're going to ask our Sequel statements to tell us what Sequel's favorite toy is. [14:35] That, by the way, is my cat, in case you were wondering. [14:43] And now, you get to see this same storage bucket that you were looking looking at before. [gasps] [14:50] If I were sitting there and I had somebody yapping at me while I'm trying to focus and do stuff, [14:55] I would be losing my mind. But, if that is your preference, I have something for you, [15:00] which is a video. So, if you want me to continue yapping at you and then showing how to do this, [15:06] I can share a video with you. [15:15] Here. And it's that one. My only ask is that if If going to So, I put captions on it. So, you don't [15:25] have to use audio. But, if you're going to use audio because for some reason you like my voice, [15:32] please wear headphones because that's going to be super annoying for everybody. And I'm going [15:38] to go around and see how you folks are doing with this lab. Do we have any questions so far? [15:52] No? [15:58] I knew that would happen. [16:01] Yeah. I I don't know about you folks, but in videos when I hear my own voice, I don't like it. [16:09] Uh so, yeah. I'll appreciate the headphones. I think it's a universal experience, right? [16:16] Cool. All right. Any questions? Comments? Is anybody super stuck, super lost? [16:25] I'm welcoming in everybody to the pet verse. I'm going to regret my decisions. [16:31] Cool. Okay. I'm going to go over there with you folks, and hopefully we can turn off this mic so [16:36] I'm not just blasting to everybody. I'll be back. Okay. I want to show you folks a little trick cuz [16:48] I know we're running out of time, and we're going to get eventually kicked out. I'd love to stay. [16:55] But, if you want to jump into the vibe coding side of things, personally, I'm not super excited [17:01] about vibe coding in and of itself. I think we can do that at any time, and I prefer the big [17:07] query stuff. Uh however, in the repository that you cloned today, I left um where is my, here. [17:20] A little catch up script. Pet verse multi model and I have this BQ catch up dot sequel over here. [17:31] This thing, if I copy it into BigQuery, it will depend on two things having run, [17:38] the pet verse setup.sh, which is also in the repository. And the setup of a GCS bucket. [17:52] So, basically these commands that you all copied as part of the lab in the cloud shell. [17:58] The next thing is going to be like control H and I'm going to replace region with whatever region [18:05] I was using in the lab. I'm just going to go with US Central 1 here. Or no, what was it? [18:14] Connection. Okay, Europe West 4 in my case. [18:24] And replace all. [18:30] And then the project ID [18:38] is going to be this project here. You can copy it from the picker [18:46] or from the lab itself. And if you're doing this at home, it will be in the picker as well. [18:53] And replace all. And this is going to try and set up all of the BigQuery things that we tried [19:01] to do manually. The other thing that I was telling folks is depending on a bunch of stuff [19:10] that are configured or not, you in the cloud shell, if you select code like this, [19:16] you will get a little light bulb on the side and you can ask it to explain the code [19:21] as you go. And that is pretty useful as you're doing things. [19:26] So, to recap, we loaded data from a CSV file that is not very impressive, but it happens. [19:34] We added tables to We added columns to our pets table using the object ref structure so that we [19:42] can tap into the videos and audios that we saw in the storage bucket. We are generating the content [19:51] of those references with these specialized object ref functions that BigQuery has. [20:01] There's better ways of doing that, but it's easier for you to copy and paste like this. And then, [20:07] a few questions that I got here. So, AI that generate is one of the multiple AI functions that [20:13] BigQuery has. There's AI generate, AI generate table, AI if, AI aggregation, AI scoring. There's [20:19] a lot of that. The beauty of AI generate is one of the most generic ones. You can do whatever [20:25] prompting you want and interleave the fields or columns over here. This connection ID was [20:31] created by the Petverse setup. And this endpoint can be any of the endpoints that you see in Gemini [20:39] agent platform. Or it could be an endpoint that you create yourself. So, you could host models [20:46] from Hugging Face or whatever models you want. And as long as they are available in agent platform, [20:54] you can connect them from here. In this case, we are uh specifying the output, but you can always [21:02] let Gemini do whatever it wants. Normally, we want to be deterministic and specify the output. [21:10] We're asking the AI to generate for the descriptions. [21:14] And then we created embeddings. Embeddings are this numerical mathematical representations of [21:20] the contents of what we did. We created text embeddings and we created media embeddings. And [21:31] we use those embeddings to do uh semantic search. The semantic search is different from [21:38] doing character by character searches as most of you know. It allows us to ask it like we did in [21:44] the example, "Pets who like to chill." And we're going to get uh pets whose hobbies is relaxing [21:51] in the sun. That is because we're searching semantically. And we did that through text [21:57] and through the multimedia. Technically, if you want to use the multimedia with text, it will [22:02] work just the same. It's just a big model for just text. So, I wanted to share the two other things. [22:09] When it comes to the vibe coding, if you go into the video, you'll see a little bit of a rant me [22:14] going, "This is not going to be production-ready." Uh and an explanation of why it's going to be not [22:20] a production-ready. But, yes, I know that this is a contentious topic. I'm happy to talk about it [22:27] outside. And thank you very much for joining. I'm sorry that some of you got delayed by the [22:33] internet being slow, but hopefully we can stay in touch. Thank you. [applause]