Voltage Park Cuts Monolithic LLMs to Scale Video AI for Middle East

The Fast Lane to AI: Build in Weeks With Voltage Park Inc — Ai4

Ai4

Deploying end-to-end foundation models creates runaway inference costs and unpredictable failures, requiring developers to decompose pipelines into specialized vision and reasoning microservices on tiered hardware.

AI-assisted summary of The Fast Lane to AI: Build in Weeks With Voltage Park Inc — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by the Ai4 channel on August 29, 2025, this presentation features Saurabh Giri and Ragha Prasad of Voltage Park discussing strategies for deploying enterprise AI systems in weeks rather than quarters. According to the description, the session provides a battle-tested playbook for tool selection, rapid architecture design, and overcoming procurement cycles to avoid pilot purgatory.

## TL;DR Drawing from operational experience running large-scale GPU infrastructure, Voltage Park CPTO Saurabh Giri details a four-part playbook for moving enterprise AI projects into production rapidly. Illustrated through an anonymized case study of converting thousands of passive CCTV surveillance feeds into an active natural-language search engine for a sovereign Middle Eastern government, the talk advocates decomposing complex tasks into modular 'See, Filter, Reason' stages. The strategy emphasizes rightsizing hardware to model computational profiles, generating synthetic video data to bypass legal compliance bottlenecks, and managing generative model non-determinism with standard microservice reliability patterns like retries and structured schema enforcement.

## Atmosphere & style The presentation carries a brisk, pragmatic, and practitioner-focused conference tone grounded in operational infrastructure reality rather than speculative hype. Saurabh speaks with confident engineering authority, puncturing the mystique of monolithic AI models by treating them as fallible software microservices that require traditional defensive hardening. The mood remains engaging and approachable throughout, enlivened by casual humor about the company's sponsored coffee lounge and an interactive guessing game revealing their sovereign client.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=msorJ_oEeKY&t=4s) Voltage Park operates as a specialized, AI-native cloud service provider that owns and scales its own GPU infrastructure across multiple data centers. - [03:26](https://www.youtube.com/watch?v=msorJ_oEeKY&t=206s) Enterprise computer vision deployments often require transforming passive data archives into active, natural-language-searchable intelligence at scale. - [05:15](https://www.youtube.com/watch?v=msorJ_oEeKY&t=315s) Using monolithic end-to-end foundation models for multi-step tasks creates fragile black-box failure modes, latency spikes, and unsustainable inference costs. - [05:15](https://www.youtube.com/watch?v=msorJ_oEeKY&t=315s) Decomposing pipelines into distinct micro-stages—See, Filter, and Reason—allows independent horizontal scaling and efficient batching for expensive reasoning LLMs. - [08:34](https://www.youtube.com/watch?v=msorJ_oEeKY&t=514s) Scaling past proof-of-concept requires rightsizing hardware to model workloads, such as packing classic vision models onto older accelerators and reserving flagship GPUs for reasoning. - [10:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=641s) When data compliance, curation, or governance stalls development, engineering teams should introduce pragmatic simplifiers to maintain iteration velocity. - [12:30](https://www.youtube.com/watch?v=msorJ_oEeKY&t=750s) Generating synthetic video bypasses lengthy legal and compliance hurdles while enabling proactive testing against visual degradations like fog, noise, and bad angles. - [12:30](https://www.youtube.com/watch?v=msorJ_oEeKY&t=750s) Simulating high-volume synthetic data through the pipeline identifies hardware bottlenecks and validates GPU memory allocation prior to live rollout. - [15:06](https://www.youtube.com/watch?v=msorJ_oEeKY&t=906s) Because generative AI is inherently non-deterministic, model endpoints must be engineered like standard microservices with retries, schema validations, and fallback mechanisms. - [15:06](https://www.youtube.com/watch?v=msorJ_oEeKY&t=906s) Leveraging frontier models capable of enforcing structured JSON output significantly simplifies schema validation and programmatic error handling. - [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) A functional AI factory tightly integrates model selection, application software, and underlying GPU compute to optimize token economics toward zero marginal cost. - [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) Architecting a minimal, use-case-driven footprint allows teams to ship production systems in weeks with a foundation that can scale from 50 to 500,000 streams.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=msorJ_oEeKY&t=4s) Introduction and Voltage Park's Infrastructure Footprint

Saurabh Giri, Chief Product and Technology Officer at Voltage Park, opens the session by introducing his team and laying out the talk's core theme: moving enterprise AI projects from ideation to production in weeks rather than quarters. Joined by colleagues including engineer Ragha Prasad, Jessica Francis from marketing, and Brandon and Joseph from the sales team, Saurabh quips that Voltage Park is sponsoring the conference coffee, making the coffee lounge the easiest place to catch the team for follow-up discussions.

Drawing on practical lessons from running workloads at Voltage Park and building Amazon's foundational AI platforms, Saurabh emphasizes that these insights come from guiding thousands of customers through high-scale deployments. Voltage Park is a cloud service provider (CSP) purpose-built for AI workloads. Their mission centers on serving two primary audiences: offering cost-effective, high-tier compute hardware to startup builders, and operating an "AI factory" consumption layer to help enterprises transition from ideation to production.

- Voltage Park operates as one of the few specialized CSPs in the United States that owns and operates its GPU fleets at scale. - The company maintains an infrastructure footprint spanning six data centers and two corporate offices, with ongoing rapid expansion. - Their customer base includes tier-one foundation model builders—companies prominent enough to occupy major billboard campaigns along Silicon Valley's Highway 101 corridor.

---

[03:26](https://www.youtube.com/watch?v=msorJ_oEeKY&t=206s) The Case Study: Transforming Passive Video Surveillance

To anchor the technical discussion, Saurabh and Ragha introduce an anonymized real-world enterprise customer deployment. The client possessed thousands of continuous CCTV surveillance cameras distributed across their premises. However, their system was entirely passive: human security operators had to manually review vast archives of recorded footage during post-incident postmortems, burning hundreds of hours searching for relevant activity.

The client's goal was to transform this legacy archive into an active, natural-language-searchable surveillance engine. They required the ability to evaluate specific temporal and conditional queries, such as: *"Show me snippets of surveillance footage that show a car, but not a truck, entering a gate between 2:00 a.m. and 4:00 a.m."* Manually scanning video at that scale was mathematically impossible, necessitating an automated, high-throughput AI solution.

---

[05:15](https://www.youtube.com/watch?v=msorJ_oEeKY&t=315s) Lesson 1: Decompose the End-to-End Problem (See, Filter, Reason)

Saurabh highlights the primary failure mode in modern AI architecture: attempting to solve complex, multi-stage requirements in a single shot using an all-encompassing frontier foundation model. While modern multimodal LLMs can natively interpret video, reason over context, and return answers, throwing raw video directly at an end-to-end model creates severe fragility, unpredictable latency, black-box failure modes, and runaway infrastructure costs.

Instead, the team decomposed the pipeline into three specialized, containerized micro-stages—**See**, **Filter**, and **Reason**. First, a lightweight visual model identifies objects and generates bounding boxes without performing reasoning. Second, a filtering layer strips away empty frames and irrelevant footage. Third, a reasoning model matches the customer's natural-language query against the structured scenario descriptions extracted from the remaining segments.

```mermaid flowchart LR accTitle: Modular Video Query Architecture subgraph S1["1. See"] A["CCTV Video Streams"] --> B["Object Detection Model<br/>(Bounding Boxes & Labels)"] end subgraph S2["2. Filter"] B --> C["Rule/Heuristic Filtering<br/>(Prune Irrelevant Frames)"] end subgraph S3["3. Reason"] C --> D["Scenario Descriptions"] E["Natural Language Query"] --> F["Reasoning LLM"] D --> F F --> G["Matching Video Snippets"] end ```

- **Independent Horizontal Scaling:** If the camera count doubles, the ingestion, object detection, and filtration tiers scale out independently without overloading the expensive reasoning cluster. - **Batch Optimization:** Queries hitting the reasoning stage are cleanly batched, isolating expensive LLM inference from high-volume frame rates. - **Architectural Flexibility:** Modular microservices eliminate reliance on an uncontrollable monolithic model where perception, filtering, and logic are hopelessly entangled.

---

[08:34](https://www.youtube.com/watch?v=msorJ_oEeKY&t=514s) Lesson 2: Match the Hardware to the Model

Saurabh warns that engineering teams often ignore operational unit economics until completing a proof of concept (POC), only to discover that their production inference costs are unsustainable. Scaling past the pilot phase requires rightsizing compute capacity to match each model's specific compute profile rather than defaulting to flagship accelerators across the entire pipeline.

For the initial "See" phase, Voltage Park avoided modern transformer or diffusion architectures entirely, deploying a battle-tested, classic object detection model. Because these models have tiny computational footprints, placing them individually on dedicated NVIDIA H100 GPUs would be an egregious waste of capital. Instead, the team optimized resource utilization by packing multiple model instances onto single GPUs or routing simpler workloads to previous-generation hardware like NVIDIA A100s, reserving premium H100 capacity exclusively for heavy reasoning LLMs.

| Pipeline Stage | Model Profile | Hardware Strategy | |:--- |:--- |:--- | | **See (Detection)** | Classic non-transformer vision model | High-density instance packing; A100s or older accelerators | | **Filter** | Lightweight heuristic/filtering algorithms | Minimal GPU/CPU capacity allocation | | **Reason** | Large language model (LLM) | Dedicated flagship GPU clusters (e.g., H100s) with optimized batching |

---

[10:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=641s) Lesson 3: Start with Domain-Adapted Models and Unblock with Synthetic Data

Complex AI systems inevitably rely on multiple domain-adapted open-source models tailored to specific subtasks. However, customization against actual enterprise production data frequently stalls engineering velocity. Data curation, governance, compliance, and audit trail requirements can easily add weeks or months of administrative friction.

Saurabh advises teams to identify bottlenecks immediately and deploy an unblocking "simplifier" rather than waiting for pristine data pipelines. In this customer deployment, the team utilized **synthetic data generation** as an accelerator to bootstrap model evaluation and tuning before touching sensitive, real-world CCTV streams.

> "As soon as you hit something that takes time, it's important to find a simplifier that lets you get started even if it's not perfect. So you get started, you iterate with something that works, and then you iterate until you get it to where you want it to be."

[12:30](https://www.youtube.com/watch?v=msorJ_oEeKY&t=750s) Unblocking Pipelines with Synthetic CCTV Data and Stress Testing

Accessing real-world customer video data presents an immediate compliance and legal headache. Organizations legitimately agonize over privacy policies, data handling safeguards, and secure storage requirements when exposing their end customers' data. Consequently, procuring the first tranche of production footage often turns into an agonizing, weeks- or months-long bureaucratic standoff before engineering can even begin. To prevent their developers from sitting idle, Voltage Park completely sidestepped this roadblock by generating synthetic video. With current open-source video generation models producing hyper-realistic CCTV footage, the team bypassed procurement entirely; in fact, the sample graphic displayed in their presentation showed an artificial setting populated with entirely generated people, yet it flawlessly validated their object identification and scenario evaluation stages.

Generating synthetic footage unblocked the team across three distinct operational layers. First, it decoupled developer velocity from customer legal timelines, allowing engineers to construct and iterate across all three pipeline stages without pause. Second, it opened up unprecedented resilience and edge-case testing. Rather than waiting to encounter poor real-world conditions by chance, the engineers programmatically simulated diverse CCTV impairments—altering graininess, skewing camera angles, inserting visual interruptions and foreground objects, degrading image resolution, and adding heavy fog. Third, synthetic pipelines enabled at-scale load simulations. Pushing artificial data through the pipeline allowed the team to accurately characterize their hardware footprint, tune resource allocations, and pinpoint systemic bottlenecks—such as whether latency spikes stemmed from GPU memory context limits or pipeline inefficiencies—effectively derisking the entire deployment before live customer data arrived.

```mermaid flowchart TD accTitle: Synthetic Data Testing and Validation Architecture subgraph DataGen["Data Simulation Layer"] SG["Synthetic Video Generator"] Var["Edge Degradations<br/>(Fog, Grain, Angles, Occlusion)"] SG --> Var end subgraph Pipeline["Processing Pipeline"] OD["Object Identification"] SD["Scenario Description & Evaluation"] Var --> OD OD --> SD end subgraph Validation["System Derisking"] Scale["At-Scale Stress Testing"] Bottleneck["Hardware Footprint &<br/>Memory Bottleneck Analysis"] SD --> Scale Scale --> Bottleneck end ```

- Synthetic generation sidesteps weeks or months of legal and compliance procurement delays. - Developers can artificially degrade video with graininess, adverse angles, visual noise, drops in resolution, and foggy atmospheric effects to test model resilience. - Pushing high-volume synthetic video identifies hardware bottlenecks and validates GPU memory allocation long before handling live streams.

---

[15:06](https://www.youtube.com/watch?v=msorJ_oEeKY&t=906s) Lesson Four: Managing Non-Determinism as Dependent Microservices

The fourth core operational lesson is coming to terms with the reality that any software system incorporating AI is inherently non-deterministic. While it is easy to become enamored with the polished demos of state-of-the-art frontier models and assume they will deliver consistent magic in production, relying on deterministic behavior is "a folly." Prompting a generative AI model five times with the exact same input will yield variations, and occasionally it will hallucinate a completely incorrect response. Robust production systems cannot assume perfection; they must be engineered to absorb fluctuations and fail gracefully when an inference call fails completely.

Voltage Park tackled this unpredictability by treating internal model dependencies exactly like third-party microservices. By viewing model endpoints through standard software engineering lenses, the team introduced traditional reliability patterns: explicit retries, schema validations, and fallback mechanisms. The growing ability of frontier and reasoning models to output strictly structured JSON substantially streamlines this pattern, allowing engineers to validate payload formats programmatically, catch schema mismatches, and route fallback routines without bringing down the surrounding application.

- GenAI models queried multiple times with identical inputs will produce divergent outputs or outright failures. - Machine learning components must be decoupled and treated as conventional microservices. - Traditional software hardening techniques—such as automated retries, schema validation, and defensive fallback logic—are mandatory for production AI. - Leveraging reasoning models capable of enforcing structured JSON output significantly eases schema enforcement.

---

[16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) The Voltage Park AI Factory and Scaling to Half a Million Cameras

Even the disciplined execution of these engineering lessons will not take an enterprise project from zero to production in four weeks without integrated full-stack hardware and software infrastructure. Voltage Park frames this capability around their "AI factory"—a term often abused in the industry, but which they strictly define as a system where "the input is enterprise data, the output is actionable business intelligence." In the context of computer vision, raw surveillance camera feeds enter the system, and operational, usable alerts exit. Powering that journey requires three tightly coupled components: selecting the right open-source or frontier models, writing the consuming application software, and provisioning the underlying infrastructure of accelerated GPU compute.

Unlike generic infrastructure approaches, Voltage Park distinguishes its platform by prioritizing a tightly bounded, use-case-driven methodology. Rather than trying to stand up an over-engineered platform to solve every hypothetical enterprise challenge, they determine the absolute minimal hardware and software footprint necessary to ship a specific use case into production. Because Voltage Park operates GPU clusters at scale, they integrate infrastructure management directly with application engineering. This tight coupling is critical for pushing the cost per token down "asymptotically approaching zero."

From the enterprise buyer's standpoint, complex distributed infrastructure is abstracted away into "one phone number that they call" where things simply work without operational headache. The underlying architecture remains identical whether an organization begins processing feeds from 50 cameras or scales up to 500,000 cameras. In a lighthearted closing reveal as time expired, the speaker prompted the audience for wild guesses regarding the confidential customer behind this multi-camera deployment. While unable to name the entity directly, the speaker revealed that the system was built for "one of the three governments in the Middle East" before inviting attendees to join them for follow-up conversations at the coffee station.

- The AI factory abstracts raw data intake (e.g., CCTV feeds) into actionable business alerts via an end-to-end integration of models, software, and GPUs. - Deployments focus on minimal, modular footprints tailored to a single use case, adding capacity incrementally as demands expand. - Operating hardware and software together allows teams to optimize token economics toward zero marginal cost. - The underlying architecture remains uniform whether orchestrating 50 cameras or scaling out to half a million streams. - The highlighted production rollout was developed for a major sovereign government in the Middle East.

## Notable quotes > [10:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=641s) As soon as you hit something that takes time, it's important to find a simplifier that lets you get started even if it's not perfect. So you get started, you iterate with something that works, and then you iterate until you get it to where you want it to be. — Saurabh Giri

> [03:26](https://www.youtube.com/watch?v=msorJ_oEeKY&t=206s) Show me snippets of surveillance footage that show a car, but not a truck, entering a gate between 2:00 a.m. and 4:00 a.m. — Saurabh Giri

> [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) the input is enterprise data, the output is actionable business intelligence. — Saurabh Giri

> [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) asymptotically approaching zero — Saurabh Giri

> [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) one phone number that they call — Saurabh Giri

> [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) one of the three governments in the Middle East — Saurabh Giri

## Mentioned Saurabh Giri, Ragha Prasad, Jessica Francis, Brandon, Joseph, Voltage Park Inc, Amazon, NVIDIA H100, NVIDIA A100, Silicon Valley's Highway 101

## Video description Saurabh Giri, CPTO, Voltage Park Ragha Prasad, Engineer, Voltage Park, Inc

Abstracts

Deploying AI doesn't require a years-long infrastructure project. In this talk, I'll share the exact strategies I've used to help teams break free from endless procurement cycles and ship production AI systems in weeks, not quarters. Through real-world case studies I'll demonstrate the infrastructure patterns that scale with your growth. I'll walk you through my battle-tested playbook for: Selecting tools Architecting for speed Avoiding the common pitfalls that trap teams in "pilot purgatory"

Leave with a concrete roadmap to transform your AI ambitions into shipping products.

[Source](https://www.youtube.com/watch?v=msorJ_oEeKY)

中文简报

Voltage Park拆解单体LLM，在中东规模化落地视频AI 部署端到端基础模型会导致推理成本失控和不可预测的故障，促使开发者将流水线拆解为在分级硬件上运行的专用视觉与推理微服务。 结合运营大规模GPU基础设施的实战经验，Voltage Park CPTO Saurabh Giri详细介绍了一套推动企业级AI项目快速投产的四步方案。演讲以一个匿名案例进行说明——为某中东主权政府将数千路被动CCTV监控视频流转化为支持自然语言搜索的主动引擎，主张将复杂任务解构为模块化的“看、滤、想”（See, Filter, Reason）阶段。该策略强调根据模型计算特征合理配置适配硬件、生成合成视频数据以绕过法律合规瓶颈，并利用重试机制和结构化模式（schema）约束等标准微服务可靠性模式，来应对生成式模型的不确定性。 Voltage Park 建议避免直接使用单体大模型处理复杂多阶段任务，而是将视频检索流水线解耦为“识别（See）”、“过滤（Filter）”和“推理（Reason）”三个微阶段，以实现独立水平扩展并削减推理成本。 (5:15) 为防止进入生产阶段后面临无法承受的算力成本，工程团队应合理匹配硬件规格，例如在 NVIDIA A100 等旧款加速器上高密度打包轻量视觉模型，并将顶级 H100 GPU 集群专门留给高负载推理 LLM。 (8:34) 针对真实数据审批耗时过长的痛点，团队通过生成逼真的合成监控视频绕开合规阻碍，并在正式上线前模拟大雾、噪点、异常角度等极端边缘场景以及高负载压力，有效验证了 GPU 显存分配与系统瓶颈。 (12:30) 生产系统需将不可预测的 AI 依赖项视作传统微服务，引入结构化 JSON 验证、自动重试与降级机制；该全栈架构已成功应用于中东某国政府项目，能够支持从 50 台监控扩展至 50 万路摄像头的超大规模分析。 (16:41)

Reading summary

According to Voltage Park Chief Product and Technology Officer Saurabh Giri, shipping enterprise AI in weeks rather than quarters requires decomposing monolithic models into specialized pipelines, rightsizing hardware to model workloads, and using synthetic data to bypass governance bottlenecks. Decomposing Video Workloads into Specialized Stages (5:15) Giri argues that routing raw, high-volume video directly into end-to-end multimodal foundation models introduces unpredictable latency, opaque failure modes, and unsustainable inference costs. In an enterprise surveillance deployment transforming passive CCTV footage into a natural-language search engine, Voltage Park addressed this by separating the architecture into three containerized micro-stages: See, Filter, and Reason. In this pipeline, a lightweight visual model first identifies objects and extracts bounding boxes without performing higher-level interpretation. Next, a heuristic layer filters out static frames and irrelevant footage. Only the resulting scenario descriptions are passed to a large language model, which evaluates complex natural-language queries such as identifying specific vehicles entering premises during specified hours. Giri notes that this modular structure allows ingestion and filtration to scale horizontally without overwhelming the reasoning tier. Matching Accelerator Hardware to Model Footprints (8:34) Engineering teams frequently overlook inference unit economics during the proof-of-concept phase, discovering unsustainably high operational expenses only when scaling into production. Giri asserts that teams must match compute capacity to the computational profile of each model rather than defaulting to flagship GPUs across the entire pipeline. For the initial detection stage, Voltage Park avoided modern transformer or diffusion architectures, deploying classic non-transformer vision models instead. To optimize efficiency, the team packed multiple model instances onto single GPUs or assigned workloads to previous-generation hardware like NVIDIA A100s, while reserving dedicated clusters of flagship NVIDIA H100 accelerators solely for batched queries in the reasoning stage. Bypassing Compliance Delays with Synthetic Video (10:41) Procuring real-world enterprise video footage routinely introduces compliance, privacy, and legal negotiations that can stall engineering work for weeks or months. To maintain velocity, Voltage Park generated hyper-realistic synthetic CCTV footage to bootstrap model evaluation and pipeline development prior to receiving client data. Synthetic generation also enabled proactive resilience and load testing. Developers programmatically injected edge-case degradations—including camera angle distortions, atmospheric fog, visual noise, foreground occlusions, and reduced resolution—while pushing artificial data at scale to expose GPU memory context limits and pipeline bottlenecks before handling live client streams. Hardening Non-Deterministic Models as Microservices (15:06) Giri emphasizes that software systems incorporating generative AI must be designed around the reality of non-determinism, as identical prompts can yield divergent outputs or hallucinations. Rather than assuming model infallibility, Voltage Park treats internal model components as traditional, external microservices requiring standard defensive software patterns. To absorb unexpected inference failures without application downtime, the architecture incorporates automated retries, fallback routines, and strict schema validation. Giri highlights that using reasoning models capable of enforcing structured JSON outputs streamlines this validation process, allowing programmatic inspection of payloads before they reach consuming services. Voltage Park reported applying this modular pattern to build a video surveillance platform intended to scale from 50 to 500,000 cameras for an unnamed Middle Eastern sovereign government. The company contends that rapid enterprise execution depends on combining modular application software with dedicated, tightly managed GPU infrastructure to lower marginal inference costs.

精炼阅读摘要

Voltage Park 首席产品与技术官 Saurabh Giri 指出，企业级人工智能项目若要在数周内完成从概念到生产的落地，必须打破对端到端前沿大模型的盲目依赖，转向模块化流水线解耦、算力精细适配，并通过合成数据与微服务工程化化解合规与非确定性风险。 架构解耦：见、滤、思三层微服务化 (5:15) 许多企业在构建复杂视觉系统时，倾向于使用单一的多模态大模型直接处理原始视频流并生成分析结论。Giri 强调，这种端到端单体架构会导致系统极端脆弱、推理延迟难以预测、调试陷入黑盒状态，并带来失控的算力开销。在实际的视频监控改造案例中，Voltage Park 将处理链路拆解为三个独立的容器化阶段：“见（See）”、“滤（Filter）”与“思（Reason）”。 在“见”阶段，轻量级视觉模型仅负责识别目标并生成边界框；在“滤”阶段，规则与启发式算法剔除空帧与无用片段；在“思”阶段，推理大语言模型将用户输入的自然语言查询（例如“检索凌晨2点至4点有小汽车但无卡车进入大门的画面”）与提取出的结构化场景描述进行匹配。这种模块化设计支持各层级的独立水平扩展，使昂贵的推理集群与海量高帧率视频流隔离开来，大幅提升了批量推理效率与系统弹性。 算力精细适配与推理单位经济学 (8:34) 工程团队常在概念验证阶段忽视推理的单位经济成本，导致方案在规模化生产时陷入财务不可持续的困境。Giri 警告称，走出测试阶段的关键是依据每个模型的实际计算负载精确匹配硬件规格，而非盲目将整条流水线全部部署在顶级加速芯片上。 在目标检测的“见”阶段，Voltage Park 放弃了计算密集的现代 Transformer 或扩散架构，转而使用成熟且算力占用极小的经典目标检测模型。为避免将低负载模型单独置于 NVIDIA H100 GPU 上造成资本浪费，团队采用高密度部署策略，在单张 GPU 上打包运行多个模型实例，或将轻量任务分流至 NVIDIA A100 等上一代硬件，而将稀缺且昂贵的 H100 集群专门预留给重度语言推理阶段，以此保障生产环境下的经济可行性。 合成数据加速研发与极限压力测试 (10:41) 获取客户真实视频数据往往受阻于繁琐的隐私合规、数据治理和法律审计程序，此类行政流程通常耗费数周乃至数月，导致研发进度停滞。Giri 建议团队在遇到阻塞时应立即寻找“简化器”先行破局。Voltage Park 利用开源视频生成工具制作高保真合成监控视频，在不接触真实敏感监控数据的情况下，直接启动了全套系统的评估与调优。 合成视频的应用不仅使开发流程与客户法务审计周期彻底解耦，还为系统提供了程序化的压力测试条件。工程师无需被动等待现实中的极端天气，即可算法化注入图像噪点、畸变偏角、前景遮挡、分辨率劣化与浓雾天气等干扰条件，全面检验视觉模型的鲁棒性。此外，大规模输入合成数据还能在接入实机前精准测定 GPU 显存上限与流水线瓶颈，提前排除系统性风险。 防御性工程：将非确定性模型视为微服务 (15:06) 生成式 AI 系统具有天然的非确定性，将实验室中的理想演示当作生产环境的稳定性基准是不可取的。面对完全相同的提示词输入，生成模型依然可能给出存在差异甚至发生幻觉的输出。因此，工业级 AI 系统不能建立在完美输出的假设之上，必须具备容忍波动和优雅降级的能力。 Voltage Park 将内部模型端点视作存在故障风险的第三方微服务，引入了传统软件工程的防御性手段，包括自动化重试策略、模式校验（Schema Validation）及后备回退机制。借助前沿推理模型输出严格结构化 JSON 的能力，工程团队可以在程序层面自动化验证响应负载格式、拦截模式异常并执行容错分支，防止单点模型失败导致整个应用崩溃。 Voltage Park 将这套工程方法论整合为其全栈“AI工厂”，其在全美拥有6座数据中心的专用 GPU 算力集群，并已在中东某主权政府项目中验证了该架构——将数十万路无源摄像头升级为自然语言检索系统，并支持从50路平滑扩容至50万路视频流。Giri 总结道，通过将软硬件深度垂直整合，企业能够以用例为导向裁剪最小生产足迹，并推动单 Token 推理成本渐进趋近于零；但这也同时意味着，周级交付速度高度依赖底层算力设施与应用软件架构的端到端直接掌控。

Detailed notes — zh

## Background 该内容由频道 Ai4 于 2025 年 8 月 29 日在 YouTube 平台发布。视频主要记录了 Voltage Park 的 CPTO Saurabh Giri 与工程师 Ragha Prasad 的主题演讲，分享他们帮助企业摆脱漫长采购周期、在数周内将 AI 系统推向生产环境的实战策略与基础设施架构方法。 ## TL;DR 演讲围绕如何高效构建生产级 AI 系统展开，提出了三大核心工程法则：一是解构端到端问题，将任务拆解为“识别、过滤、推理”三阶段流水线，拒绝单一大模型全包；二是实现硬件与模型特征的精准匹配，通过小模型聚合部署与高价值算力倾斜来压降生产成本；三是借助合成数据绕过漫长的数据合规审查，加速模型冷启动与极限工况压测。同时，演讲强调必须将 AI 组件视为非确定性的微服务进行治理，并介绍了 Voltage Park 软硬协同的“AI 工厂”架构，帮助企业以最小硬件足迹实现从数十路到数十万路视频流的无缝弹性扩展。 ## Atmosphere & style 整场演讲风格务实干练、直击痛点，充溢着资深基础设施工程师拆解复杂系统的从容与敏锐。讲者以幽默轻松的开场迅速拉近距离，随后通过硬核的真实案例层层推演，语调沉稳且富有洞见，毫不留情地打破了“端到端大模型万能”的行业幻象。整场分享节奏紧凑、干货密集，既有对生产环境高昂成本与非确定性风险的清醒警示，又带给听众一种条理清晰、可立即落地的工程实战底气。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=msorJ_oEeKY&t=4s) Voltage Park 是一家专注于 AI 工作负载的云服务提供商，在全美拥有 6 大自主运营的数据中心，团队具备构建亚马逊 AI 基础设施的经验。 - [03:26](https://www.youtube.com/watch?v=msorJ_oEeKY&t=206s) 传统 CCTV 监控系统极度依赖事后人工排查，效率低下且无法扩展，客户的核心诉求是通过自然语言进行精准的主动式时空多实体智能检索。 - [05:15](https://www.youtube.com/watch?v=msorJ_oEeKY&t=315s) 避免采用单一大模型端到端解决所有问题的“万能论”，盲目构建单体黑盒会导致系统过度设计、脆弱且推理成本失控。 - [05:15](https://www.youtube.com/watch?v=msorJ_oEeKY&t=315s) 合理的 AI 架构应将问题拆解为“识别（See）”、“过滤（Filter）”与“推理（Reason）”三大解耦阶段，分别独立容器化部署并按需水平扩展。 - [08:34](https://www.youtube.com/watch?v=msorJ_oEeKY&t=514s) 硬件必须与模型特征精准匹配，前端轻量化检测模型可采用单卡多实例聚合或上一代 GPU，将昂贵的旗舰 GPU 算力保留给高阶大语言模型。 - [10:38](https://www.youtube.com/watch?v=msorJ_oEeKY&t=638s) 在企业数据面临合规审查与清洗瓶颈时，引入合成数据作为“简化器”，能够快速跑通闭环迭代，避免项目陷入原地等待的死局。 - [12:30](https://www.youtube.com/watch?v=msorJ_oEeKY&t=750s) 利用开源视频生成模型生成高保真合成画面，不仅能绕过漫长的数据采购合规流程，还能模拟大雾、遮挡、噪点等极端工况进行全链路鲁棒性压测与显存调优。 - [15:06](https://www.youtube.com/watch?v=msorJ_oEeKY&t=906s) AI 系统本质上具有非确定性，盲目期待大模型在生产中给出完美结果是不切实际的，必须将其视为普通依赖微服务进行工程治理。 - [15:06](https://www.youtube.com/watch?v=msorJ_oEeKY&t=906s) 借助重试机制、模式校验（Schema Validation）、降级兜底及模型结构化输出，能够有效防范大模型偶发失效或非确定性异常。 - [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) “AI 工厂”架构以用例为核心驱动，追求最小软硬件足迹与深度软硬协同，旨在将 Token 成本压低至近乎为零，并支持从数十路平滑弹性扩展至数十万路并发流。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=msorJ_oEeKY&t=4s) 开场介绍与 Voltage Park 的算力底座 在本次以“驶向 AI 的快车道：数周内从构想到生产”为主题的分享中，Voltage Park 的首席产品与技术官（CPTO）Saurabh Giri 携团队登台。Saurabh Giri 介绍到场团队成员包括杰出工程师（Distinguished Engineer）、市场营销人员 Jessica Francis，以及来自销售团队的 Brandon 和 Joseph，并幽默地提到他们赞助了本届大会的咖啡，欢迎参会者随后去展位交流。Saurabh 指出，Voltage Park 团队曾深度参与构建亚马逊（Amazon）的 AI 平台，服务过数千家客户完成从概念验证到生产落地的全流程。本次演讲旨在将这些跨越超大规模系统的实战经验，浓缩为 20 分钟内最核心、最实用的工程落地法则。 Voltage Park 是一家专门服务于 AI 工作负载的云服务提供商（CSP），其核心使命是“让开发者轻松构建 AI 产品，让企业高效消费 AI”。在供给侧，Voltage Park 对初创团队极其友好，以极具竞争力的成本提供顶尖的硬件算力与服务；在需求侧，他们通过自研的“AI 工厂”（AI Factory）机制，帮助企业缩短产品化路径。作为全美少数几家自主拥有并大规模运营 GPU 的云厂商之一，Voltage Park 目前拥有 6 个数据中心与 2 处办公室，其算力版图正在快速扩张。Saurabh 强调，虽然许多普通大众可能未曾听闻 Voltage Park，但加州湾区 101 高速公路广告牌上的那些家喻户晓的顶尖前沿模型研发商，几乎全都是他们的核心客户。 - **团队背景**：核心工程团队具备主导构建顶级公有云（Amazon）AI 基础设施的深厚经验。 - **算力布局**：在全美拥有 6 大数据中心且正在加速扩建，专注于大规模 GPU 集群的自主运营。 - **客户画像**：涵盖正在推进前沿大模型边界的硅谷头部明星企业与高成长初创公司。 --- ### [03:26](https://www.youtube.com/watch?v=msorJ_oEeKY&t=206s) 真实案例剖析：被动监控系统的智能化改造挑战 为了让方法论更具实操性，团队以一个真实的客户案例展开讲解（为保护客户隐私已对部分细节做脱敏处理）。该客户在园区内安装了数千个 CCTV 监控摄像头，长期以来运行着一套极其被动的视频监控系统。在这种传统架构下，人工操作员只能在事件发生后进行“事后复盘”（postmortem），面对海量的非结构化视频素材，往往需要花费数百个小时进行地毯式的人工排查，效率极其低下且无法随监控点位增加而线性扩展。 客户的核心诉求是将这套被动、滞后的录像存储库，改造为支持主动式、自然语言驱动的智能检索系统。例如，操作员希望直接输入复杂的复合语义查询：“帮我调出凌晨 2 点到 4 点之间，有小汽车（Car）进入大门但没有卡车（Truck）进入的监控视频片段。”在传统人工回溯流程中，这类精准时空与多实体约束的排查需要耗费数天；该客户决定引入 AI 技术重塑整套监控管线，实现数周内从零到上线的目标。 --- ### [05:15](https://www.youtube.com/watch?v=msorJ_oEeKY&t=315s) 核心法则一：解构端到端问题，拒绝“单一大模型万能论” Saurabh 与工程师指出，团队在落地 AI 时最常犯的致命错误，就是试图“一枪打到底”（solve everything in one shot in one go），用一个全能模型完美解决所有链路。盲目追求端到端的大一统模型，必然导致系统过度设计、极度脆弱、成本失控，沦为一个难以控制和维护的“黑盒”。实际上，多模态前沿大模型虽然兼具图像理解、视频识别和逻辑推理能力，但将其直接作为单体流水线承载全量原始视频流是极其低效的。 合理的架构必须将端到端的问题拆解，精准识别哪些环节真正需要高阶 AI，哪些环节适合传统算法。针对该监控分析系统，Voltage Park 将其拆解为三大独立解耦的阶段：“识别（See）”、“过滤（Filter）”与“推理（Reason）”。 ```mermaid flowchart LR accTitle: CCTV 智能视频检索处理流水线 subgraph S1["识别阶段 (See)"] A["海量原始 CCTV 视频流"] --> B["轻量化目标检测模型<br/>(绘制边界框 & 提取类别)"] end subgraph S2["过滤阶段 (Filter)"] B --> C["时空与规则过滤引擎<br/>(剔除无关镜头与静止画面)"] end subgraph S3["推理阶段 (Reason)"] D["用户自然语言查询"] --> E["高阶推理大模型 (LLM)<br/>(语义匹配与场景描述判定)"] C --> E end E --> F["输出目标视频片段"] ``` - **识别（See）**：仅负责基础目标识别与绘制边界框（Bounding Boxes），提取类别标签，不参与任何高阶语义推理。 - **过滤（Filter）**：基于元数据和规则快速剔除大量无关或无变动的监控片段，大幅削减下游计算负载。 - **推理（Reason）**：最后引入高阶大语言模型，将用户的自然语言查询与前序生成的场景化结构描述进行精准匹配判定。 - **工程收益**：三大组件运行在各自独立的容器化环境中，可独立水平扩展。当接入摄像头数量翻倍时，只需横向扩容前端的识别与过滤节点，而后端昂贵的大模型推理批处理（Batch）负载可保持稳定受控。 --- ### [08:34](https://www.youtube.com/watch?v=msorJ_oEeKY&t=514s) 核心法则二：硬件算力与模型特征精准匹配 Saurabh 直言，许多团队在完成 POC（概念验证）之前，根本无法真正体会到 AI 在生产环境中运行到底有多昂贵。如果在系统设计之初没有将成本压降作为核心考量，一旦进入正式投产，高昂的推孟/训练开销会立刻成为制约业务扩展的最大瓶颈。因此，核心准则在于“使硬件与模型精准匹配”，探索运行整套流水线所需的“最小硬件足迹”（Minimum Hardware Footprint）。 在具体工程实现上，团队针对不同层级的模型制定了差异化的硬件分配策略： 1. **边缘/轻量化检测模型**：前端目标检测选用的是经过实战检验的经典小模型（甚至非 Transformer 或 Diffusion 架构）。为了不浪费昂贵且紧缺的 NVIDIA H100 算力，工程团队采用单卡多模型实例聚合（Packing N copies onto a single GPU）技术，或直接将其部署在上一代性价比较高的硬件（如 NVIDIA A100）上。 2. **高阶复杂推理模型**：将最先进的旗舰 GPU 算力保留给负责最终语义理解的大型语言模型（LLMs），确保计算密度与高价值任务对齐。 3. **资源精细化调优**：在过滤和数据中转层动态分配适量 GPU 资源，通过严密的基准测试和持续调优，确保计算资源无闲置、无溢出。 --- ### [10:38](https://www.youtube.com/watch?v=msorJ_oEeKY&t=638s) 核心法则三：基于领域模型启动，利用简化策略破局数据瓶颈 将架构拆解与硬件选型结合后，多数非平凡（Non-trivial）的企业级业务场景都会沉淀为由若干开源模型组合构成的技术方案。此时的落地路径是：首先选取最贴合该业务环节的领域适配开源模型（Domain-Adapted Open-Source Model），随后利用真实的生产数据进行微调与定制。 然而，项目在此处最容易陷入停滞——真实世界的数据极其杂乱脏污，企业内部还面临繁琐的合规审查、数据清洗、审计追踪（Audit Trails）等流程，若按传统瀑布流顺序推进，项目周期会瞬间被拉长数周乃至数月。Saurabh 强调，工程落地的核心元认知在于区分“能快速完成的事”与“需要耗费时间的事”。一旦遇到耗时卡点，必须立刻寻找“简化器”（Simplifier），先用可用方案跑通闭环并快速迭代，而不是原地等待完美数据。在本次监控案例中，团队引入的关键简化器便是“合成数据”（Synthetic Data），在保障敏感监控隐私合规的同时，大幅加速了模型的冷启动与定制进程。 ### [12:30](https://www.youtube.com/watch?v=msorJ_oEeKY&t=750s) 借助合成视频突破数据采购瓶颈与压力测试 在涉及企业客户或终端用户敏感数据的项目中，数据合规和存储权限往往是最大的绊脚石。客户会极度担忧“谁在接触这些数据”以及“数据该如何存储”，导致仅为了采购和获取第一批真实数据，团队就得耗费数周乃至数月的时间去走流程。为了彻底绕过这一漫长的等待期，团队果断转向了前沿的合成视频生成技术。当前的开源视频生成模型已经足够成熟，能够生成逼真度极高的合成 CCTV 监控画面。演讲者提到，演示文稿开篇展示的监控图像实际上完全是由 AI 生成的——画面中的人物并不存在，场景也非真实地点，但它却能完整跑通目标识别与场景描述评估的完整管线。 合成数据为整个研发流程带来了三大核心维度的解耦与加速：首先，它彻底疏通了开发管线，工程师无需干等真实数据到位，即可立即着手构建、测试和调优三阶段流水线；其次，它极大拓展了测试边界，团队可以随意合成各种极端工况下的监控视频，比如调高噪点、改变摄像头角度、模拟画面遮挡、降低分辨率与画质、添加大雾等恶劣天气效果，从而大幅增强系统在应对真实复杂环境时的鲁棒性；最后，合成数据能够用于大规模压力测试。通过向系统注入海量合成数据，团队可以清晰识别出系统的瓶颈所在——究竟是硬件资源受限、模型运行效率低下，还是上下文显存不足，进而精准规划硬件足迹（Hardware Footprint）并合理分配算力资源，从根本上降低了项目交付的风险。 ```mermaid flowchart TD accTitle: 合成视频数据驱动的研发与压测流程 subgraph DataGen ["合成数据生成阶段"] A["开源前沿视频生成模型"] --> B["高保真合成 CCTV 监控画面"] B --> C["极端工况模拟<br>(噪点/大雾/低分辨率/遮挡/多角度)"] end subgraph Pipeline ["AI 业务处理流水线"] C --> D["阶段一：目标识别与检测"] D --> E["阶段二：场景语义描述"] E --> F["阶段三：评估与告警触发"] end subgraph Optimization ["系统调优与压测"] F --> G["系统级瓶颈定位"] G --> H["显存/上下文占用调优"] G --> I["算力足迹与 GPU 资源精准分配"] end ``` - **摆脱采购泥潭**：利用开源生成模型即时产出高质量监控数据，跳过长达数月的数据合规审批流程。 - **全场景鲁棒性验证**：可控生成起雾、低分辨率、画面噪点与视角异常等边缘案例（Edge Cases）。 - **全链路规模化压测**：在真实数据进驻前完成全负载模拟，摸清 GPU 显存与计算资源分配的真实瓶颈。 --- ### [15:06](https://www.youtube.com/watch?v=msorJ_oEeKY&t=906s) 拥抱非确定性：将大模型视为微服务进行工程治理 在将大模型引入生产环境时，团队必须建立一个核心认知：“任何哪怕只在单处使用了 AI 的软件系统，本质上都是非确定性的（inherently non-deterministic）。”人们往往很容易被前沿大模型在特定场景下的惊艳表现所折服，并盲目寄希望于它每一次都能给出完美的表现，但在生产级软件工程中，“这是一种极其危险的幻想（in production it is a folly）”。生成式 AI 模型面对同一个问题，问两次或五次，极大概率不会得到一模一样的答案，有时甚至会输出完全荒谬的错误结果。 因此，生产系统必须具备应对非确定性输出以及模型彻底失效时的优雅降级能力。解决这一难题的关键在于工程化思维的转变——团队将所有 AI 子组件和模型依赖项完全视为普通的“依赖微服务（Dependent Microservices）”。这意味着成熟的传统软件工程手段可以无缝复用： - **重试机制（Retries）**：处理偶发的超时或异常响应。 - **模式校验（Schema Validations）**：严密约束并检验输出结构。 - **降级回退（Fallback Mechanisms）**：在模型完全不可用或解析失败时提供兜底策略。 - **结构化输出利用**：充分利用现代推理模型（Reasoning Models）输出结构化 JSON 的能力，将其紧密嵌入到微服务治理体系中。 --- ### [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) Voltage Park 的“AI 工厂”架构与规模化落地实践 即便团队完美掌握了上述所有开发原则，也并不意味着能在四周内凭空把一个项目从零推向生产环境。要实现飞速交付，背后必须有极其扎实的软硬件基础设施支撑。Voltage Park 将这套体系称为“AI 工厂（AI Factory）”。尽管这个词被行业过度滥用，但其本质定义非常纯粹：输入是企业级原始数据，输出是可落地的商业智能（Actionable Business Intelligence）。以智能安防为例，输入是成千上万路摄像头原始视频流，输出则是立即可用的精准告警。 Voltage Park 的 AI 工厂展现出与众不同的核心竞争力： 1. **用例驱动与极简模块化**：不追求构建能解决一切问题的虚幻通用系统，而是聚焦于单个具体用例，以最小化的软硬件足迹（Minimal Footprint）快速落地投产，随后根据新用例按需进行模块化增量扩展。 2. **软硬件深度协同降本**：凭借在大规模运营 GPU 算力集群方面的深厚积累，配合顶尖的软件工程能力，构建高度软硬一体化的紧耦合技术栈。这种深度优化旨在不断压低每个 Token 的推理成本，使其渐进逼近于零。 3. **极简交付与无缝弹性伸缩**：对于客户而言，他们没有多余的时间、耐心或专业知识去折腾底层复杂的软硬件栈，他们只需要“拨打一个电话”，系统就能正常运转。底层架构始终保持一致，能够无缝从几十颗 GPU 平滑扩展至任意规模算力，在业务端支撑从最初的 50 路摄像头输入直接弹性扩展到 50 万路并发视频流。 演讲尾声，讲者留下了一个悬念，邀请现场观众竞猜这一落地案例的幕后客户。虽然由于保密协议无法透露具体名称，讲者最终揭晓该客户是“中东地区的三大政府机构之一”，并热情邀请参会者在茶歇区进一步交流探讨。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=msorJ_oEeKY&t=4s) 让开发者轻松构建 AI 产品，让企业高效消费 AI。 — Saurabh Giri > [00:04](https://www.youtube.com/watch?v=msorJ_oEeKY&t=4s) 加州湾区 101 高速公路广告牌上的那些家喻户晓的顶尖前沿模型研发商，几乎全都是他们的核心客户。 — Saurabh Giri > [03:26](https://www.youtube.com/watch?v=msorJ_oEeKY&t=206s) 帮我调出凌晨 2 点到 4 点之间，有小汽车（Car）进入大门但没有卡车（Truck）进入的监控视频片段。 > [05:15](https://www.youtube.com/watch?v=msorJ_oEeKY&t=315s) 团队在落地 AI 时最常犯的致命错误，就是试图‘一枪打到底’（solve everything in one shot in one go），用一个全能模型完美解决所有链路。 — Saurabh Giri > [08:34](https://www.youtube.com/watch?v=msorJ_oEeKY&t=514s) 许多团队在完成 POC（概念验证）之前，根本无法真正体会到 AI 在生产环境中运行到底有多昂贵。 — Saurabh Giri > [08:34](https://www.youtube.com/watch?v=msorJ_oEeKY&t=514s) 使硬件与模型精准匹配，探索运行整套流水线所需的‘最小硬件足迹’（Minimum Hardware Footprint）。 — Saurabh Giri > [10:38](https://www.youtube.com/watch?v=msorJ_oEeKY&t=638s) 工程落地的核心元认知在于区分‘能快速完成的事’与‘需要耗费时间的事’。一旦遇到耗时卡点，必须立刻寻找‘简化器’（Simplifier）。 — Saurabh Giri > [15:06](https://www.youtube.com/watch?v=msorJ_oEeKY&t=906s) 任何哪怕只在单处使用了 AI 的软件系统，本质上都是非确定性的（inherently non-deterministic）。 > [15:06](https://www.youtube.com/watch?v=msorJ_oEeKY&t=906s) 人们往往很容易被前沿大模型在特定场景下的惊艳表现所折服，但在生产级软件工程中，‘这是一种极其危险的幻想（in production it is a folly）’。 > [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) 输入是企业级原始数据，输出是可落地的商业智能（Actionable Business Intelligence）。 — Saurabh Giri > [16:41](https://www.youtube.com/watch?v=msorJ_oEeKY&t=1001s) 对于客户而言，他们没有多余的时间、耐心或专业知识去折腾底层复杂的软硬件栈，他们只需要‘拨打一个电话’，系统就能正常运转。 — Saurabh Giri ## Mentioned Voltage Park, Saurabh Giri, Ragha Prasad, Jessica Francis, Brandon, Joseph, 亚马逊 (Amazon), Ai4, NVIDIA H100, NVIDIA A100, 中东政府机构 ## Video description Saurabh Giri, CPTO, Voltage Park Ragha Prasad, Engineer, Voltage Park, Inc Abstracts Deploying AI doesn't require a years-long infrastructure project. In this talk, I'll share the exact strategies I've used to help teams break free from endless procurement cycles and ship production AI systems in weeks, not quarters. Through real-world case studies I'll demonstrate the infrastructure patterns that scale with your growth. I'll walk you through my battle-tested playbook for: Selecting tools Architecting for speed Avoiding the common pitfalls that trap teams in "pilot purgatory" Leave with a concrete roadmap to transform your AI ambitions into shipping products. [Source](https://www.youtube.com/watch?v=msorJ_oEeKY)

Source quotes

[0:04] Saurabh Giri: 让开发者轻松构建 AI 产品，让企业高效消费 AI。 [0:04] Saurabh Giri: 加州湾区 101 高速公路广告牌上的那些家喻户晓的顶尖前沿模型研发商，几乎全都是他们的核心客户。 [3:26] 帮我调出凌晨 2 点到 4 点之间，有小汽车（Car）进入大门但没有卡车（Truck）进入的监控视频片段。 [5:15] Saurabh Giri: 团队在落地 AI 时最常犯的致命错误，就是试图‘一枪打到底’（solve everything in one shot in one go），用一个全能模型完美解决所有链路。 [8:34] Saurabh Giri: 许多团队在完成 POC（概念验证）之前，根本无法真正体会到 AI 在生产环境中运行到底有多昂贵。 [8:34] Saurabh Giri: 使硬件与模型精准匹配，探索运行整套流水线所需的‘最小硬件足迹’（Minimum Hardware Footprint）。 [10:38] Saurabh Giri: 工程落地的核心元认知在于区分‘能快速完成的事’与‘需要耗费时间的事’。一旦遇到耗时卡点，必须立刻寻找‘简化器’（Simplifier）。 [15:06] 任何哪怕只在单处使用了 AI 的软件系统，本质上都是非确定性的（inherently non-deterministic）。 [15:06] 人们往往很容易被前沿大模型在特定场景下的惊艳表现所折服，但在生产级软件工程中，‘这是一种极其危险的幻想（in production it is a folly）’。 [16:41] Saurabh Giri: 输入是企业级原始数据，输出是可落地的商业智能（Actionable Business Intelligence）。 [16:41] Saurabh Giri: 对于客户而言，他们没有多余的时间、耐心或专业知识去折腾底层复杂的软硬件栈，他们只需要‘拨打一个电话’，系统就能正常运转。

Source quotes

[10:41] Saurabh Giri: As soon as you hit something that takes time, it's important to find a simplifier that lets you get started even if it's not perfect. So you get started, you iterate with something that works, and then you iterate until you get it to where you want it to be. [3:26] Saurabh Giri: Show me snippets of surveillance footage that show a car, but not a truck, entering a gate between 2:00 a.m. and 4:00 a.m. [16:41] Saurabh Giri: the input is enterprise data, the output is actionable business intelligence. [16:41] Saurabh Giri: asymptotically approaching zero [16:41] Saurabh Giri: one phone number that they call [16:41] Saurabh Giri: one of the three governments in the Middle East

Original transcript

[0:04] Thank you for coming to the session. We [0:07] are going to talk about fast lane to AI. [0:11] How you go from ideiation to production [0:14] in weeks. I'm Sab Gri. I'm the chief [0:19] product and technology officer of [0:22] Voltage Park. With me here today in the [0:25] room and outside the room are a bunch of [0:27] people from Voltage Park. uh Yazraa, [0:31] distinguished engineer, Jessica Francis [0:33] from our marketing team, Brandon sales [0:36] from Brandon and Joseph from our sales [0:38] team. I think if I'm not mistaken, we [0:40] are sponsoring coffee this year. So [0:43] that's where you'll find us hanging out [0:45] if you want to stop by later and say [0:47] hello. [0:49] So in this session [0:52] we are going to distill [0:56] some practical lessons from having done [1:01] this several times both at Voltage Park [1:04] and also the engineering team at Voltage [1:07] Park was involved in building Amazon's [1:10] AI platform. So we've seen this at scale [1:13] with thousands of customers in their [1:16] journeys from ideiation to production [1:19] and we've distilled down just a few [1:22] lessons as much as we can fit in 20 [1:24] minutes. Obviously everything is more [1:27] complicated than it is but these I think [1:30] you will find useful. [1:33] Just to give context, um, [1:37] many of you may not have heard of [1:38] Voltage Park, so I'll just introduce who [1:40] we are. Voltage Park is a cloud service [1:44] provider that specializes in AI [1:47] workloads. [1:49] Our mission is to make it easy for [1:52] builders to build AI products and for [1:55] companies to consume consume them. On [1:59] the builder side, we work with startups. [2:01] Very startup friendly. We provide [2:04] startups world-class hardware service. [2:09] World thanks. Worldclass hardware [2:13] service at very reasonable uh cost. [2:18] And on the consumption side, we work [2:21] with companies uh that are trying to go [2:24] from ideation to production. The way we [2:27] do it is something uh that we call our [2:30] AI factory. We'll talk about it brief. [2:31] That's not the main point of this [2:33] discussion, but we'll talk about it [2:35] briefly later. [2:37] We are one of few CSPs in the US that [2:41] own and operate GPUs at scale. We are in [2:45] six data centers today, two offices and [2:48] our footprint is very quickly expanding. [2:56] some of the most sophesticated model [2:58] builders today, [3:02] household names, the sort of people if [3:04] you drive down 101 freeway in the Bay [3:07] Area, the sort of people you see on [3:09] billboards, they are our customers. So [3:13] we are, you may not have heard of us, [3:15] but the people who are pushing the edge [3:19] of what's possible in AI today, they [3:22] have heard of us and they are our [3:23] customers. [3:26] So uh let's get to the main content. [3:31] This is all based on a real customer [3:34] example. For privacy reasons, we [3:37] switched around the facts a little bit u [3:40] for anonymity, but this is a real [3:42] customer use case where we went from um [3:46] nothing to production uh within weeks. [3:49] Da would you like to frame? [3:50] >> Hey. Yeah. Um so this customer uh has [3:55] thousands of CCTV cameras installed on [3:59] their premises. Uh they have what we [4:02] would call a very passive uh video [4:05] surveillance system. Um they have human [4:08] operators uh who are sifting to this [4:10] data looking for uh as a as a postmortem [4:14] operation to kind of go back and look at [4:16] uh what what needs to be uh looked at [4:19] and and they have to sift through vast [4:22] quantities of data to get to where they [4:24] have to get to. And the requirement from [4:26] this customer was how can we shift this [4:28] passive system uh into something that is [4:31] a little bit more active uh and easier [4:34] for the human operators to use uh and [4:37] evaluate. Uh, for example, a very [4:40] specific type of uh query that the [4:43] customer wanted to get answered was, uh, [4:46] show me these snippets of surveillance [4:48] footage that shows a car but not a truck [4:52] entering a gate between 2 a.m. and 4:00 [4:54] a.m. Now, obviously to do this in their [4:58] existing system uh involves [5:01] hundreds of hours of sifting through [5:03] video footage um and that obviously did [5:06] not scale. And that's basically why they [5:09] decided to apply uh AI to this [5:11] particular use case. [5:15] And that brings us to lesson number one. [5:20] It's very important to decompose the end [5:23] to end problem. A mistake that a lot of [5:26] people make is in trying to solve [5:28] everything in one shot in one go [5:31] perfectly. And if you try to do that, [5:34] inevitably most of the time you'll [5:36] realize that you end up with a system, a [5:40] solution that's overengineered and over [5:44] complicated and overly expensive. So [5:47] it's very important to break it down and [5:50] uh the next thing is also quite [5:52] important. It's very important to be [5:54] thoughtful about in which of these steps [5:57] is the AI actually useful. Trying to use [6:01] AI in situations where nonAI is [6:04] perfectly fine results in a system where [6:08] for example you're trying to use one [6:10] grand model to solve everything end to [6:12] end and then you're stuck with [6:15] inevitable tradeoffs such as [6:18] >> Yeah. Um so so let's assume in this case [6:21] you you all might have been very [6:22] familiar uh the current state of [6:24] frontier model technologies that you can [6:28] they are very good at image [6:29] understanding video understanding the [6:31] same model can also reason about uh the [6:34] particular query that you have and we [6:36] could have easily built this using one [6:38] gigantic model pumped the data straight [6:41] to it gotten back our uh results. Uh but [6:44] as you will see through the lessons [6:46] learned that often leads to unexpected [6:49] outcomes uh fragility in system uh [6:52] exploding costs uh and somewhat of a [6:54] blackboxy system that then you're not [6:56] able to control and uh maintain uh in a [6:59] long enough duration of time. For this [7:02] particular use case uh we broke down the [7:05] problem into roughly three large [7:07] components. There's a C filter and [7:09] reason. Uh we wanted a model that would [7:12] be able to just do only object [7:14] identification, not reason about [7:16] anything, just draw bounding boxes [7:18] around uh objects and class labels that [7:20] we've defined. Uh we needed a filtering [7:23] system that would basically weed out [7:26] large chunks of the video that are no [7:28] longer relevant or have nothing to do [7:30] with the query at hand. And then at last [7:33] we actually needed a reasoning system [7:35] that would then take the uh customer [7:38] natural language query and then try to [7:40] match it to the scenario description [7:43] that has been seen and filtered by the [7:45] previous steps in the pipeline. Um and [7:48] and by breaking down this into [7:51] subcomponents and designing it as a [7:53] system, what we ended up accomplishing [7:56] was having three separate containerized [7:58] environments [8:00] that all can individually scale [8:02] independently. Uh so for example, if the [8:05] customer decided to double their CCTV [8:07] footage or the number of cameras [8:09] involved, um we could horizontally scale [8:12] the number of object identification and [8:14] filtration systems. we could control the [8:17] amount of batch queries that go into our [8:19] reasoning models as the last step. Um, [8:21] and it gives us a lot of flexibility on [8:24] how we can define the system as opposed [8:26] to trying to use one gigantic model uh [8:29] where all these three steps are kind of [8:30] subsumed within it. [8:32] [Music] [8:34] Lesson number two, match the hardware to [8:37] the model. What does this even mean? Why [8:39] are we talking about hardware all of a [8:41] sudden? The answer is [8:45] until you finish a P, you don't really [8:49] fully appreciate how expensive it is to [8:52] use AI in production. So if you don't [8:55] design the end-to-end system keeping in [8:58] mind from the beginning how how can I do [9:01] it as inexpensively as possible you will [9:04] find out after the P that it is quite [9:07] expensive and then suddenly cost becomes [9:10] a bottleneck to scaling out in [9:12] production [9:14] and uh this manifests in several ways [9:16] right and I think it also ties back to [9:18] one other point that we made at the [9:20] beginning of this talk which is you [9:22] first have to pick the right models [9:24] Um, for object identification, we use [9:26] the tried and tested model that's not [9:28] even transformer or diffusion based. [9:30] It's existed for a really long time. [9:32] It's it's a battle tested uh model that [9:35] can draw bounding boxes around CCTV [9:38] footage images. Um, but it's an [9:40] extremely small model and we didn't want [9:42] to waste uh you know precious H100 [9:44] capacity on hosting these models. Um, [9:47] and there we employ different techniques [9:49] like packing n number of model copies [9:51] onto a single GPU. Um you could also [9:55] explore the possibility of using uh you [9:58] know A100s or you know previous [10:00] generation hardware to host some of [10:02] these smaller and simpler models and [10:05] then you rightsize your available [10:07] capacity for the uh you know the [10:10] powerful LLMs um and and of course [10:13] obviously the filter requires some [10:14] amount of GPU capacity as well. In doing [10:17] so, uh what you're trying to accomplish [10:21] is to say what is your minimum hardware [10:23] footprint that is required to host this [10:26] pipeline. Um and as you will see in the [10:28] subsequent slides, this requires some [10:30] amount of tuning uh and a lot of [10:32] experimentation to get the resource [10:34] allocation right. [10:35] [Music] [10:38] Next one. [10:41] Lesson number three. So if you connect [10:45] the dots between the first two lessons, [10:48] inevitably for a lot of use cases, [10:52] non-trivial use cases, you'll find [10:56] that you end up using a few open-source [10:59] models. And then if you're doing that [11:02] the important thing is to begin with the [11:06] right domain adapted open-source model [11:11] for that step for that component and [11:14] then you have to customize it with [11:16] actual production data. And right here [11:20] is where um many projects get stuck [11:23] because data is messy. You have to [11:26] curate the data. You have to think about [11:29] things like in many enterprises [11:32] compliance audit trails all of that it's [11:34] messy. So if you do a project a [11:37] sequential way then immediately you're [11:40] adding weeks if not months. So the I [11:43] think the meta point here is every [11:45] project has things that can be done [11:48] quickly and then things that take time. [11:51] As soon as you hit something that takes [11:53] time, it's important to find a [11:56] simplifier that lets you get started [11:59] even if it's not perfect. So you get [12:01] started, you iterate with something that [12:04] works and then you iterate until you get [12:07] it to where you want it to be. How did [12:10] we uh what simplifier did we do for this [12:12] one? Yeah. In so in this case, uh, usage [12:15] of synthetic data helped us in numerous [12:18] ways and I'll go through them in in a in [12:20] a rough sequence. Um, [12:23] handling something as sensitive as CCTV [12:25] footage uh has to be done with extreme [12:28] care, especially if this is your [12:30] customer's customer's data. Um, they are [12:33] worried about who's handling this. We [12:35] have to be careful about how we are [12:36] storing this. Um so as sort of mentioned [12:39] this can be a weeks or even month-long [12:42] effort to just get started in [12:44] procurement of the first trunch of data. [12:47] Um we sidest stepped all of this and we [12:50] immediately started to do synthetic [12:53] video generation. Uh the [12:55] state-of-the-art in video generation [12:57] today is pretty advanced and you have [13:00] open source models that can actually [13:01] produce very very realistic synthetic [13:05] CCTV footage. Uh in fact at the [13:07] beginning of the slides uh if you had [13:09] seen the graphic um that's the that's an [13:12] actual synthetic uh generated CCTV [13:15] footage. None of the people on that [13:16] video are real. The place is not a real [13:18] place. Uh but the it has gone through [13:21] our pipeline which has done the uh [13:22] object identification and the scenario [13:24] description evaluations. Um [13:28] so very to to begin with it it unblocks [13:31] our development pipeline. So we are able [13:33] to just generate synthetic data. we are [13:35] able to build the three steps of our [13:37] pipeline. Uh our devs are not kind of [13:39] waiting for how to you know proceed and [13:42] what to test and tune and so on and so [13:44] forth. The second thing it unblocks is [13:46] our ability to test. Um with synthetic [13:49] data we are able to simulate different [13:52] scenarios of CCTV footage. We can [13:54] increase how grainy the footage is, the [13:56] various angles, uh interruptions, [13:58] objects, um loss of uh you know [14:01] resolution, quality, foggy images. All [14:04] of these now can be tested and we start [14:06] to get we start to make our system more [14:09] resilient uh to what will be eventually [14:11] the actual conditions of the CCTV [14:13] footage that we're going to get from our [14:15] end customers. Um [14:18] the last part is once you have synthetic [14:21] data and once you're feeding it through [14:23] your system you can actually simulate an [14:26] atscale test and that's very important [14:29] and it connects back to the slide just [14:31] before you need to know your hardware [14:33] footprint. Uh you need to be able to [14:35] tune and allocate the right amount of [14:36] resources to the various models in your [14:38] pipeline. So once you have synthetic [14:40] data and once you start pushing it [14:42] through your system you start to [14:43] identify what are the bottlenecks where [14:46] is the uh actual systemic bottlenecks is [14:49] it the hardware problem are the models [14:51] being efficient as the is the sufficient [14:53] memory available for the context so on [14:56] and so forth and we largely derisk the [14:57] project um in this way by just relying a [15:02] lot on our synthetic data set. [15:06] Lesson number four, [15:09] any software system that uses AI even in [15:13] a single place is inherently [15:17] non-deterministic. [15:19] You have to accept that and make peace [15:20] with it. Is very tempting um to be [15:24] impressed with the capabilities of the [15:28] frontier models that indeed look like [15:30] they're perfect for some use case and [15:32] then to be hopeful that that happens all [15:34] of the time. But in production it is a [15:37] folly. [15:39] Genai models you ask the same question [15:41] twice or five times. You won't get the [15:43] exact same answer five times. Sometimes [15:47] it gives a completely wrong answer. And [15:49] your software system needs to be uh [15:53] resilient to handle the non-determinism [15:57] and also when there's a complete failure [15:59] to deal with that gracefully. [16:02] And in in this case um we started to [16:06] treat all of our subcomponents and model [16:08] dependencies that we took just as [16:10] dependent microservices which means all [16:13] of the conventional software engineering [16:15] practices like retries, schema [16:17] validations [16:19] um and you know fallback mechanisms etc [16:21] were applicable here. Uh many of the [16:24] models these days be reasoning models or [16:27] um or such actually do produce [16:29] structured JSON. So that does help us do [16:31] a great deal of uh you know great deal [16:35] towards helping treat them as dependent [16:37] services. [16:41] So um these were some lessons they are [16:43] not all the lessons obviously [16:46] and even if you apply all of these [16:48] lessons flawlessly [16:51] it doesn't really mean you'll go from [16:53] nothing to something in production in [16:55] four weeks. That's not how it works. So [16:58] in order to do this you need a lot of [17:01] other infrastructure on the hardware [17:04] side on the software side and um in [17:08] different places you can do different [17:09] ways with voltage park we call it our AI [17:12] factory it's a much abused term u I'll [17:15] I'll simply define what we mean what we [17:18] mean is it is a place where the input is [17:23] enterprise data the output is actionable [17:27] business intelligence. So in that [17:30] example, the input was raw feeds from [17:32] video cameras. The output was [17:34] actionable, usable alerts. And to pull [17:37] this off, all you need is models, the [17:40] right models. You need software that [17:43] consumes the models. You need [17:45] infrastructure, hardware and software, [17:47] GPUs included. [17:49] Now, everybody has a factory. How's our [17:52] factory different? These are few [17:56] fundamental ways where I think we are [17:58] thinking about this in a different way. [18:01] The first way is it is very use [18:04] casedriven. It's not about let's stand [18:07] up capabilities that can solve anything. [18:10] Let's start with one use case that [18:11] you're trying to solve. Let's talk about [18:13] what is the minimal footprint needed to [18:18] get something working in production. So [18:20] our factory is minimal. It is very [18:23] modular. You add more use cases, you can [18:26] add the other incremental things needed [18:30] both in the hardware and the software to [18:32] take that use case to production. One of [18:35] the places where we are different is [18:37] because we run GPUs at scale. We've uh [18:42] become fairly good at it. So then by [18:44] adding uh world-class soft engineering [18:47] capability, we end up with a stack [18:49] that's very tightly coupled. And the [18:52] simple reason why this matters is to [18:54] drive down cost. If you don't do it, it [18:57] is impossible to take the dollar per [18:59] token down. It's asytoic [19:03] approaching zero. So that's how our [19:06] factory is different. From a customer's [19:08] perspective, many customers don't have [19:10] the time, patience or expertise to mess [19:13] around with all of this themselves. To [19:15] them, it's very simple. is one phone [19:18] number that they call. They don't care [19:22] how it's working. It just works. There's [19:24] one phone number they call and this [19:26] thing scales. It scales from a few GPUs [19:28] to any number of GPUs. The architecture [19:32] is the same. So in the case in this [19:35] example, you could start with feeds from [19:37] 50 cameras, scale that to half a million [19:40] cameras. The architectures exactly the [19:42] same. So we're coming up on time 10 [19:45] seconds. Any wild guesses as to who the [19:47] customer is? I can't name them, but [19:49] anybody want wants to guess. You've [19:51] heard of them. Since we are at time, [19:53] I'll just tell you. It is one of the [19:56] three governments in the Middle East. [19:58] You've definitely heard of them, but I [20:00] won't name them. Thank you all. We'll be [20:02] at the coffee station uh if you want to [20:05] hang out. [20:07] [Applause]