Oracle Deploys 131,000-GPU Nvidia Blackwell Clusters for Frontier AI

Unlock AI Performance with NVIDIA's Accelerated Computing With Oracel And NVIDIA — Ai4

Ai4

Bare-metal networking and direct chip interconnects give Oracle Cloud up to a 220% price-performance edge over hyperscaler rivals, winning nearly all premier frontier AI labs as training scales toward gigawatt-scale facilities.

AI-assisted summary of Unlock AI Performance with NVIDIA's Accelerated Computing With Oracel And NVIDIA — original source. Check the original for full context.

Source published:

Detailed notes

## Background Published on YouTube by the Ai4 channel on August 24, 2025, this session features Sanjay Basu from Oracle and Dave Salvator from NVIDIA presenting on accelerated computing infrastructure on Oracle Cloud Infrastructure (OCI). The talk covers how NVIDIA architectures such as Blackwell, Hopper, and Ada-Lovelace integrate with OCI to support demanding next-generation AI workloads and optimize deployment performance.

## TL;DR The speakers map out the computational shift from simple generative AI to reasoning-centric agentic models that require up to 100x more tokens, necessitating unprecedented scale in both inference and training. NVIDIA outlines architectural solutions to this challenge, including the Blackwell NVL72 unified domain, Grace CPU-to-GPU high-speed interconnects, native FP4 precision, and Dynamo's disaggregated serving architecture. Oracle then illustrates how OCI delivers this platform to enterprises through sovereign, edge, and cloud deployment models, backed by database vector search and embedded SaaS agents. The presentation culminates with OCI's AI Supercluster architecture, which leverages bare-metal instances, 3.2 Tbps non-blocking networking, and gigawatt-scale data center facilities to scale up to 131,000 GPUs for frontier model training.

## Atmosphere & style The session kicks off with relaxed, lighthearted humor about missing confidence monitors and recognizable speaker headshots before swiftly pivoting into a dense, highly technical architecture briefing. The handoff between Salvator's chip-and-software analysis and Basu's enterprise infrastructure deployment creates a brisk, complementary rhythm that balances deep engineering concepts with operational realities. It carries a calm, matter-of-fact authority that makes staggering technological leaps—like gigawatt facilities and 131,000-GPU clusters—feel like the tangible next steps of modern industrial computing.

## Key takeaways - [00:04](https://www.youtube.com/watch?v=0M674LDRdpA&t=4s) The industry is transitioning toward agentic and physical AI, driving up to a 100x increase in token demand due to multi-step reasoning and chain-of-thought processing. - [01:57](https://www.youtube.com/watch?v=0M674LDRdpA&t=117s) NVIDIA Blackwell NVL72 unifies 72 GPUs into a single coherent NVLink fabric domain to eliminate networking bottlenecks for Mixture of Experts (MoE) model architectures. - [01:57](https://www.youtube.com/watch?v=0M674LDRdpA&t=117s) The Grace Blackwell (GB200) superchip utilizes high-speed Chip-to-Chip (C2C) interconnects to bypass standard PCIe limits, enabling seamless shared memory between CPUs and GPUs. - [03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) Native FP4 precision on Blackwell shifts the Pareto frontier in AI serving, enabling peak cluster utilization while simultaneously delivering responsive, low-latency token generation. - [03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) Oracle Cloud Infrastructure is scaling physical data center power into gigawatt territory, including its flagship Abilene, Texas facility scaling from 200 MW toward 1.2 GW. - [05:46](https://www.youtube.com/watch?v=0M674LDRdpA&t=346s) NVIDIA's Dynamo inference serving framework decouples the compute-heavy prefill phase from the memory-bound decode phase, yielding a 30% to 40% performance improvement. - [08:50](https://www.youtube.com/watch?v=0M674LDRdpA&t=530s) NVIDIA maintains an annual release cadence toward upcoming Vera/Rubin platforms and the 2028 Feynman architecture, supported by OCP-standardized liquid-cooled rack blueprints. - [10:07](https://www.youtube.com/watch?v=0M674LDRdpA&t=607s) OCI supports sovereign and edge AI requirements by delivering its full accelerated stack through Dedicated Region Cloud@Customer (DRCC) and ruggedized Roving Edge appliances. - [12:25](https://www.youtube.com/watch?v=0M674LDRdpA&t=745s) Oracle brings AI to the data layer by embedding agents directly into Fusion Applications and providing native vector search inside Oracle Database 23ai and MySQL HeatWave. - [15:15](https://www.youtube.com/watch?v=0M674LDRdpA&t=915s) OCI Superclusters leverage bare-metal instances interconnected by a 3.2 Tbps non-blocking Clos fabric using RoCEv2 and InfiniBand to scale clusters up to 131,000 B200 GPUs. - [15:15](https://www.youtube.com/watch?v=0M674LDRdpA&t=915s) Published MLPerf benchmarks show OCI achieves up to 220% better price-performance economics over competing hyperscalers, driving broad adoption among frontier AI model providers.

## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=0M674LDRdpA&t=4s) The Shift from Generative AI to Agentic and Physical AI

Dave Salvator kicks off the presentation with a lighthearted comment about needing to sneak glances back at the presentation slide behind him because there is no confidence monitor on stage, jokingly remarking that the audience might recognize the speakers' headshots "at the post office." Laying out the agenda, he explains that he will first provide an architectural overview of the NVIDIA computing platform before handing over to Sanjay Basu to discuss how Oracle Cloud Infrastructure (OCI) delivers that platform to enterprise customers.

The industry is currently moving along a transformative trajectory: transitioning rapidly from initial generative AI to agentic AI, and ultimately heading toward physical AI in the form of robotics and autonomous edge devices. This evolution brings an unprecedented stair-step surge in required compute capacity—not just for training foundational models, but particularly for real-time inference workloads.

```mermaid flowchart LR accTitle: Evolution of AI Computing Workloads subgraph Progression["AI Evolution"] A["Generative AI<br/>(Input & Output Tokens)"] --> B["Agentic AI<br/>(Chain-of-Thought Reasoning)"] B --> C["Physical AI<br/>(Robotics & Autonomous Edge)"] end subgraph Demand["Compute Impact"] B -.-> D["Up to 100x Tokens<br/>(Massive Reasoning Phase)"] end ```

Unlike basic generative AI pipelines that simply take input tokens and generate output tokens in a single shot for standard Q&A, reasoning-centric agentic models require up to 100x more tokens. Agentic systems break down multifaceted challenges into smaller component problems, solve them sequentially, and make their decisions transparently through explicit chain-of-thought processing. The critical operational hurdle for enterprises is balancing responsive latency for end users while simultaneously maintaining peak overall system utilization across compute clusters so that services remain cost-effective.

- **Reasoning compute surge**: Agentic workflows demand up to 100x more tokens than conventional implementations due to expansive intermediate reasoning steps. - **Problem decomposition**: Next-generation agents move beyond simple retrieval to structured problem solving, decomposing tasks into manageable sub-goals with visible chains of thought. - **Operational trade-off**: Infrastructure operators must balance user responsiveness against high hardware utilization to sustain lower total cost of ownership (TCO).

---

[01:57](https://www.youtube.com/watch?v=0M674LDRdpA&t=117s) Blackwell Architecture, NVL72, and Grace CPU-to-GPU Interconnects

To meet these computational demands, NVIDIA developed the Blackwell architecture and the Blackwell Ultra platform. Blackwell functions essentially as "one giant GPU." Using NVLink fabric technology, NVIDIA connects 72 Blackwell GPUs into a single unified NVLink domain (the NVL72). This massive shared fabric directly addresses the dominant architecture of modern agentic applications: Mixture of Experts (MoE). In MoE models, distinct expert subnetworks are partitioned across separate GPUs and must communicate constantly with extreme throughput and ultra-low latency. Faster inter-GPU communication directly translates to higher throughput and significantly reduced TCO.

Beyond GPU-to-GPU interconnects, the architecture redefines how host CPUs and GPUs cooperate as peer processors. In the Grace Blackwell (GB200) superchip configuration, pairs of Blackwell GPUs connect directly to an Arm-based Grace CPU via high-bandwidth Chip-to-Chip (C2C) links. This proprietary pathway bypasses the traditional bottlenecks of standard PCIe buses, enabling high-speed bi-directional data transfers and allowing models running on the GPU to dynamically access large pools of system memory whenever workloads overflow GPU memory limits.

- **NVL72 unified domain**: Bridges 72 Blackwell GPUs into a coherent NVLink cluster acting as a single logical accelerator. - **MoE acceleration**: Optimizes inter-expert routing and communication across GPUs to prevent network stalls during inference routing. - **High-speed C2C interconnect**: Grace CPU and Blackwell GPUs communicate through direct chip-to-chip links, providing seamless, high-bandwidth access across GPU and host system memory.

---

[03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) Shifting the Pareto Curve and Gigawatt-Scale AI Factories

Deploying these architectures allows AI infrastructure operators to shift the Pareto efficiency curve up and to the right. In AI serving environments, responsiveness (delivering tokens rapidly to human end-users) and system throughput (maximizing overall token output across the cluster) naturally pull in opposite directions. Optimizing exclusively for low latency often causes GPU underutilization, while pushing hardware to full saturation can degrade interactive response times.

```mermaid flowchart TD accTitle: Pareto Frontier Optimization from Hopper to Blackwell subgraph Frontier["Inference Optimization"] H["Hopper Architecture (FP8/FP16)"] -->|"Blackwell Architecture + NVIDIA FP4"| B["Shifted Pareto Frontier<br/>(Higher Throughput + Lower Latency)"] end subgraph Scale["Physical AI Factories"] B --> OCI["Oracle Cloud Infrastructure<br/>(Abilene, TX Facility: 200 MW to 1.2 GW)"] end ```

Moving from the Hopper generation to the Blackwell architecture—bolstered by native NVIDIA FP4 precision support—enables modern "AI factories" to expand the boundary of what is possible. Data centers can simultaneously keep hardware engines fully saturated while delivering snappy, low-latency experiences to broader user bases.

To support this shift at massive physical scale, Oracle is deploying Blackwell B200 and GB200 systems across its data centers. As an example of this hyperscale expansion, Oracle is constructing massive Stargate AI data centers, including a flagship facility in Abilene, Texas, which begins at 200 megawatts and is expanding toward 1.2 gigawatts of total dedicated power capacity.

- **FP4 precision boost**: Native 4-bit floating-point support dramatically expands computational density and memory efficiency over previous generations. - **Pareto optimization**: Higher compute density shifts the operational envelope, allowing organizations to achieve peak system utilization without sacrificing user-facing latency. - **Abilene Stargate deployment**: Oracle Cloud Infrastructure's Abilene site represents a leap in physical scale, scaling from 200 MW toward 1.2 GW.

---

[05:46](https://www.youtube.com/watch?v=0M674LDRdpA&t=346s) Full-Stack Acceleration: CUDA-X, Dynamo Disaggregated Serving, and DGX Cloud

Silicon alone cannot solve enterprise AI challenges without a mature, full-stack software environment. NVIDIA backs its hardware with hundreds of domain-specific CUDA-X acceleration libraries built over decades for AI, scientific simulations, and high-performance computing (HPC). Continuous software optimizations applied to existing hardware yield ongoing performance gains, directly driving down ongoing operational TCO. On OCI, this software layer is integrated top-to-bottom across Oracle Cloud's native platform services.

```mermaid flowchart LR accTitle: Disaggregated Serving Architecture subgraph DisaggregatedServing["Dynamo Inference Framework"] Req["Incoming Request"] --> Prefill["Prefill Phase<br/>(Compute & Bandwidth Sensitive)"] Prefill --> Decode["Decode Phase<br/>(Memory Latency Sensitive)"] Decode --> Resp["Streamed Output Tokens"] end ```

To optimize generative and agentic inference workloads, NVIDIA introduced an inference serving framework called Dynamo. Dynamo incorporates disaggregated serving, which isolates the compute-heavy prefill phase from the memory-bandwidth-sensitive decode phase during token generation. Because each phase exhibits fundamentally different performance characteristics and bottlenecks, running them on specialized, decoupled compute pipelines yields a 30% to 40% performance improvement on the same underlying hardware platform.

For enterprises requiring specialized support and white-glove engineering help, Oracle Cloud hosts NVIDIA DGX Cloud. This turnkey offering combines OCI's high-performance cloud infrastructure with co-engineered, enterprise-grade direct support from NVIDIA's specialized engineering teams, streamlining model development and deployment while NVIDIA continues its aggressive one-year hardware roadmap cadence.

- **CUDA-X software ecosystem**: Hundreds of specialized libraries continually extract extra performance out of deployed silicon to lower ongoing TCO. - **Dynamo disaggregated serving**: Separating the prefill and decode stages during inference eliminates phase interference and delivers an estimated 30% to 40% performance gain. - **NVIDIA DGX Cloud on OCI**: Provides a co-supported enterprise environment pairing Oracle infrastructure with direct engineering support from NVIDIA. - **Roadmap cadence**: NVIDIA remains committed to delivering new computing architectures on an annual release cycle.

[08:50](https://www.youtube.com/watch?v=0M674LDRdpA&t=530s) NVIDIA's Relentless Hardware Roadmap and MGX Reference Designs

NVIDIA is executing on a strict yearly release cadence across its computing portfolio, systematically revamping both GPU and CPU architectures to match the exponential growth of artificial intelligence. Looking beyond current platforms, NVIDIA is preparing its next-generation Vera CPU, designed to integrate seamlessly into the upcoming Rubin platform targeted for next year, followed further down the roadmap by the Feynman architecture projected for 2028. To ensure the broader ecosystem can physically deploy these dense, high-power systems, NVIDIA develops modular MGX reference designs. A prominent example is the GB200 NVL rack architecture, which NVIDIA contributed directly to the Open Compute Project (OCP) as a standardized reference blueprint.

```mermaid flowchart LR accTitle: NVIDIA Architecture Evolution Roadmap subgraph Current["Current Generation"] direction TB B1["Blackwell Architecture<br/>(GB200 NVL / MGX)"] OCP["OCP Standardized<br/>Liquid-Cooled Racks"] B1 --- OCP end

subgraph Next["Upcoming Platforms"] direction TB R1["Rubin Platform<br/>(Vera CPU + Rubin GPU)"] F1["Feynman Architecture<br/>(Targeted 2028)"] R1 --> F1 end

Current --> Next ```

These standardized blueprints have proven instrumental for data center operators and cloud partners who are retrofitting facilities for liquid cooling, often implementing direct-to-chip liquid distribution for the very first time. By maintaining complete transparency around multi-year architecture roadmaps and providing turnkey rack-level designs, NVIDIA aims to give data center architects the lead time and engineering frameworks required to match hardware availability with facility readiness.

- **Upcoming Architectures**: Vera CPU paired with the Rubin GPU platform, pushing toward the future Feynman architecture in 2028. - **Open Compute Contribution**: GB200 rack specifications contributed directly to OCP to standardize liquid-cooled data center deployments. - **Ecosystem Acceleration**: MGX reference designs eliminate engineering bottlenecks, helping infrastructure providers deploy extreme-density platforms rapidly.

---

[10:07](https://www.youtube.com/watch?v=0M674LDRdpA&t=607s) The Rise of Agentic AI and OCI's Distributed Deployment Models

Taking over the presentation, Sanjay Basu reframes the purpose of modern AI model training and fine-tuning around a single historic shift: infusing business software with true agency. Unlike prior eras of computing where applications rigidly followed programmed heuristics, modern foundational models give software the ability to reason, self-determine, and formulate multi-step plans autonomously. Delivering this degree of agency demands compute clusters comprising hundreds of thousands—and ultimately millions—of accelerated processor cores working in concert, which Oracle Cloud Infrastructure (OCI) packages into an elastic, metered cloud environment.

```mermaid flowchart TD accTitle: OCI Distributed AI Deployment Topology subgraph Core["Centralized Cloud"] PR["OCI Public Cloud Regions"] GC["OCI GovCloud Regions"] end

subgraph Hybrid["Dedicated & Sovereign AI"] DRCC["Dedicated Region Cloud@Customer (DRCC)"] C3["Compute Cloud@Customer (C3)"] end

subgraph Edge["Edge Deployments"] RE["Roving Edge Infrastructure<br/>(NVIDIA L4, L40S, A10 GPUs)"] end

Core --- Hybrid Hybrid --- Edge ```

To accommodate regulatory constraints and strict data residency requirements, Oracle differentiates itself from other hyperscalers by making its full NVIDIA-accelerated AI infrastructure available across multiple deployment form factors rather than restricting it to a few centralized hyperscale regions. Organizations can consume these capabilities inside standard OCI Public Cloud regions, isolated GovCloud regions, or through dedicated on-premises footprints.

- **Sovereign & On-Premises Offerings**: Dedicated Region Cloud@Customer (DRCC) brings full OCI AI capabilities into customer-owned data centers for regulated industries, complemented by Compute Cloud@Customer (C3). - **Edge Inference**: OCI Roving Edge infrastructure packages ruggedized compute equipped with NVIDIA L4, L40S, and A10 GPUs to run localized inference directly in disconnected or tactical environments. - **The Core Shift**: Transitioning from passive applications to self-determining systems that execute reasoning, multi-step planning, and autonomous execution.

---

[12:25](https://www.youtube.com/watch?v=0M674LDRdpA&t=745s) Bringing AI to the Data: Fusion Apps, Database 23ai, and Managed Models

A fundamental philosophy of OCI's AI strategy is running inference and fine-tuning directly adjacent to where enterprise data already lives. For organizations running native Oracle SaaS, autonomous AI agents are embedded directly into Fusion Applications. For custom or third-party enterprise workloads, OCI provides a spectrum of managed database and AI platform services that integrate vector search capabilities directly into enterprise data stores, including Oracle Database 23ai with native AI Vector Search, MySQL HeatWave, Managed Redis, and Managed PostgreSQL.

```mermaid flowchart TD accTitle: Oracle Cloud Infrastructure Full AI Stack subgraph Apps["Applications & Agent Layer"] FA["Oracle Fusion Applications (Embedded Agents)"] AIS["OCI AI Services (Third-Party Integrations)"] DS["OCI Data Science Platform"] end

subgraph Platform["Foundation Models & Blueprints"] MM["Managed Models<br/>(xAI Grok, Cohere, Meta Llama 3.1 & 3.2)"] NVAIE["NVIDIA AI Enterprise Blueprints"] end

subgraph Data["Database & Vector Layer"] DB23["Oracle Database 23ai (Vector Search)"] MHW["MySQL HeatWave"] MDB["Managed Redis & PostgreSQL"] end

subgraph Infra["OCI Supercluster Infrastructure"] GPU["NVIDIA Accelerated Computing<br/>(A100, H100, H200, B200, GB200)"] end

Apps --> Platform Platform --> Data Data --> Infra ```

Beyond raw database capabilities, OCI's Generative AI service provides managed access to leading frontier and open-weight models, including xAI's Grok, Cohere command models, and Meta's Llama 3.1 and 3.2 suites. Developers can orchestrate agentic workflows by combining OCI Data Science tools, native cloud services, and pre-built NVIDIA AI Enterprise blueprints.

- **Embedded Application Agents**: Direct integration into Oracle Fusion Applications to automate enterprise business workflows without data egress. - **Native Vector Databases**: Native vector search support across Oracle Database 23ai, MySQL HeatWave, and open-source database engines. - **Managed LLM Ecosystem**: High-performance hosting for xAI Grok, Cohere, and Meta Llama 3.1/3.2 models running on top of NVIDIA accelerated compute.

---

[15:15](https://www.youtube.com/watch?v=0M674LDRdpA&t=915s) OCI AI Superclusters: Networking Architecture and Frontier Scale

The foundational differentiator underpinning Oracle's AI ecosystem is the OCI Supercluster architecture. Basu highlights that published MLPerf benchmarks demonstrate OCI delivers up to 220% better price-performance economics compared to competing hyperscaler instances across NVIDIA A100, H100, H200, and Blackwell B200/GB200 hardware. This efficiency is achieved by eliminating hypervisor overhead on bare-metal instances and connecting them through an ultra-low-latency, non-oversubscribed, non-blocking Clos network fabric delivering 3.2 Terabits per second (Tbps) per node utilizing both RoCEv2 (RDMA over Converged Ethernet) and NVIDIA InfiniBand.

```mermaid flowchart LR accTitle: OCI Supercluster Scaling and Interconnect Architecture subgraph Nodes["Compute Nodes"] BM["Bare Metal GPU Instances<br/>(H100 / H200 / B200 / GB200)"] end

subgraph Interconnect["Non-Blocking Fabric (3.2 Tbps / Node)"] direction TB RoCE["RoCEv2 (RDMA over Converged Ethernet)"] IB["NVIDIA Quantum InfiniBand"] end

subgraph Scale["Supercluster Capacity"] direction TB C1["Current: 65,000 H200 GPUs"] C2["Target: 131,000 B200 GPUs"] C1 --> C2 end

Nodes <--> Interconnect Interconnect <--> Scale ```

This networking backbone allows OCI Superclusters to scale seamlessly from small two-node clusters up to massive deployments of 131,000 GPU nodes within a single cluster. Because of this architectural efficiency, Basu notes that "except one, all of the frontier model providers are actually using OCI" for large-scale pre-training. Oracle currently operates active training clusters of 65,000 NVIDIA H200 GPUs and is expanding to 131,000 NVIDIA B200 GPUs in single clusters, supported by massive multi-megawatt and gigawatt data center power buildouts scaling from 200 kW racks up to dedicated 1.2 MW, 2.6 MW, and gigawatt-scale sites. The entire orchestration and scheduling environment is built on open-source standards to provide maximum workload flexibility.

- **Networking Throughput**: 3.2 Tbps non-blocking Clos fabric deployed using RoCEv2 and InfiniBand for ultra-low latency internode communication. - **Frontier Cluster Scale**: Scaling from small clusters to 65,000 H200 GPUs today, and up to 131,000 B200 GPUs in a single interconnected fabric. - **Data Center Power Evolution**: High-density engineering designed to accommodate power requirements scaling from 200 kW high-density compute pods to gigawatt-scale campus facilities. - **Industry Adoption**: Trusted infrastructure partner for nearly all premier frontier AI model developers, backed by open-source workload orchestration and job scheduling.

## Notable quotes > [00:04](https://www.youtube.com/watch?v=0M674LDRdpA&t=4s) at the post office. — Dave Salvator

> [01:57](https://www.youtube.com/watch?v=0M674LDRdpA&t=117s) one giant GPU. — Dave Salvator

> [03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) shift the Pareto efficiency curve up and to the right. — Dave Salvator

> [03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) AI factories — Dave Salvator

> [05:46](https://www.youtube.com/watch?v=0M674LDRdpA&t=346s) running them on specialized, decoupled compute pipelines yields a 30% to 40% performance improvement on the same underlying hardware platform. — Dave Salvator

> [10:07](https://www.youtube.com/watch?v=0M674LDRdpA&t=607s) infusing business software with true agency. — Sanjay Basu

> [10:07](https://www.youtube.com/watch?v=0M674LDRdpA&t=607s) giving software the ability to reason, self-determine, and formulate multi-step plans autonomously. — Sanjay Basu

> [12:25](https://www.youtube.com/watch?v=0M674LDRdpA&t=745s) running inference and fine-tuning directly adjacent to where enterprise data already lives. — Sanjay Basu

> [15:15](https://www.youtube.com/watch?v=0M674LDRdpA&t=915s) delivers up to 220% better price-performance economics compared to competing hyperscaler instances — Sanjay Basu

> [15:15](https://www.youtube.com/watch?v=0M674LDRdpA&t=915s) except one, all of the frontier model providers are actually using OCI — Sanjay Basu

## Mentioned Sanjay Basu, Dave Salvator, NVIDIA, Oracle, Oracle Cloud Infrastructure (OCI), Open Compute Project (OCP), xAI, Cohere, Meta, Blackwell, Blackwell Ultra, GB200, B200, NVL72, Grace CPU, Hopper, Ada-Lovelace, Rubin, Vera CPU, Feynman, H100, H200, A100, L4, L40S, A10, NVLink, MGX, InfiniBand, RoCEv2, CUDA-X, Dynamo, DGX Cloud, Dedicated Region Cloud@Customer (DRCC), Compute Cloud@Customer (C3), Roving Edge Infrastructure, Oracle Fusion Applications, Oracle Database 23ai, MySQL HeatWave, Managed Redis, Managed PostgreSQL, OCI Data Science, NVIDIA AI Enterprise, Grok, Llama 3.1, Llama 3.2, OCI Supercluster, MLPerf, Abilene, Texas

## Video description Sanjay Basu, Senior Director, AI/GPU - Cloud Engineering, Oracle Dave Salvator, Director of Accelerated Computing Products, NVIDIA

Abstracts

Model complexity and data sources are on the rise, creating new challenges and opportunities. This session will explore how the latest advancements in NVIDIA's cutting-edge GPU infrastructure platforms on Oracle Cloud Infrastructure (OCI)—including the Blackwell, Hopper, and Ada-Lovelace architectures—can effectively meet the evolving needs of next-generation models. Attendees will learn how to optimize their AI infrastructure deployment using NVIDIA's comprehensive end-to-end reference architecture, which integrates seamlessly with OCI to deliver exceptional performance tailored for demanding AI workloads.

[Source](https://www.youtube.com/watch?v=0M674LDRdpA)

中文简报

甲骨文为前沿 AI 部署 13.1 万块英伟达 Blackwell GPU 集群 裸金属网络与芯片直连技术让 Oracle Cloud 相比超大规模竞争对手拥有最高达 220% 的性价比优势；随着训练规模向吉瓦级设施扩展，其几乎赢得了所有顶尖前沿 AI 实验室的青睐。 演讲者阐述了从简单的生成式 AI 转向以推理为核心的智能体模型的算力变革，后者需要多达 100 倍的 token，因而在推理和训练中均需要前所未有的规模。NVIDIA 概述了应对这一挑战的架构解决方案，包括 Blackwell NVL72 统一域、Grace CPU 到 GPU 高速互连、原生 FP4 精度以及 Dynamo 的解耦服务架构。随后，Oracle 展示了 OCI 如何在数据库向量搜索和嵌入式 SaaS 智能体的支持下，通过主权、边缘和云部署模式向企业提供该平台。演讲最后介绍了 OCI 的 AI Supercluster 架构，该架构利用裸金属实例、3.2 Tbps 无阻塞网络以及吉瓦级数据中心设施，可扩展至多达 13.1 万块 GPU 用于前沿模型训练。 AI 行业正迅速从生成式 AI 转向代理式与物理 AI，因多步推理与思维链处理，推动 Token 需求剧增高达 100 倍。 (0:04) NVIDIA Blackwell NVL72 通过统一的 NVLink 架构将 72 块 GPU 互联为单一计算域，消除混合专家（MoE）模型的网络瓶颈，配合原生 FP4 精度与 C2C 互连大幅推移推理帕累托边界。 (1:57) NVIDIA 推出的 Dynamo 推理服务框架通过解耦计算密集型 Prefill 阶段与内存带宽受限的 Decode 阶段，实现底层硬件 30% 至 40% 的性能提升。 (5:46) Oracle 云基础设施（OCI）正在建设吉瓦级数据中心，包括从 200 MW 扩建至 1.2 GW 的德克萨斯州阿比林园区，并支持单集群部署高达 131,000 块 Blackwell B200 GPU。 (3:39) OCI 支持主权 AI 与边缘部署形态，通过 Dedicated Region Cloud@Customer 与加固型 Roving Edge 设备将全栈加速计算延伸至本地和严苛环境。 (10:07)

Reading summary

NVIDIA and Oracle Cloud Infrastructure argue that the transition from generative AI to agentic and physical AI requires a fundamental redesign of computing infrastructure, combining dense multi-GPU fabrics, disaggregated inference software, and gigawatt-scale bare-metal cloud networks. Reasoning Compute Surge and Disaggregated Inference (0:04) According to NVIDIA's Dave Salvator, the artificial intelligence landscape is shifting from basic generative pipelines toward agentic and physical systems. Unlike conventional implementations that generate single-shot responses, agentic systems use explicit chain-of-thought reasoning to break complex problems into sequential sub-tasks, generating up to 100 times more tokens. This surge creates a critical operational challenge for infrastructure operators, who must maintain low-latency responsiveness for end-users while keeping underlying compute engines fully utilized to control total cost of ownership. To manage this trade-off, NVIDIA complements its hardware with specialized software layers, including CUDA-X acceleration libraries and the Dynamo inference serving framework. Dynamo implements disaggregated serving, decoupling the compute-intensive prefill phase from the memory-bandwidth-sensitive decode phase during token generation. By isolating these distinct operational bottlenecks across specialized compute pipelines, disaggregated serving yields an estimated 30% to 40% performance gain on the same underlying hardware platform. Blackwell Architecture, Interconnects, and Hardware Roadmap (1:57) To handle large-scale reasoning and Mixture of Experts (MoE) models—where partitioned sub-networks require constant inter-GPU communication—NVIDIA developed the Blackwell architecture. The NVL72 platform links 72 Blackwell GPUs via NVLink into a unified logical accelerator, preventing inter-expert routing stalls. In the GB200 superchip configuration, Blackwell GPUs connect directly to an Arm-based Grace CPU using high-speed Chip-to-Chip (C2C) interconnects, bypassing traditional PCIe bus bottlenecks and allowing GPUs to access host system memory dynamically. Hardware improvements such as native FP4 precision shift the operational Pareto frontier, expanding computational density to allow simultaneous high throughput and low latency. To assist data centers transitioning to direct-to-chip liquid cooling, NVIDIA contributed its GB200 NVL rack designs to the Open Compute Project (OCP). Salvator notes that NVIDIA continues on an aggressive annual roadmap, moving toward the Rubin platform featuring the Vera CPU, followed by the Feynman architecture targeted for 2028. OCI Superclusters and Gigawatt-Scale Networking (15:15) On the infrastructure delivery side, Oracle Cloud Infrastructure's Sanjay Basu details the architecture of OCI Superclusters. OCI deploys NVIDIA GPUs—including H100, H200, and Blackwell B200 and GB200 systems—on bare-metal instances without hypervisor overhead. Compute nodes communicate over an ultra-low-latency, non-blocking 3.2 Tbps Clos network fabric supporting both RoCEv2 and NVIDIA Quantum InfiniBand. Basu asserts that published MLPerf benchmarks demonstrate OCI provides up to 220% better price-performance than competing hyperscalers, leading nearly all premier frontier AI model developers to train on the platform. OCI's physical footprint is expanding to support extreme cluster sizes, scaling from existing clusters of 65,000 H200 GPUs up to planned single-fabric deployments of 131,000 B200 GPUs. Supporting these clusters requires massive power engineering; Oracle is constructing dedicated facilities such as the Stargate data center in Abilene, Texas, which is designed to scale from an initial 200 megawatts toward 1.2 gigawatts of capacity. Distributed Deployment Models and Data-Adjacent AI (10:07) Rather than confining accelerated computing exclusively to centralized hyperscale regions, OCI provides distributed deployment options to address strict data residency and sovereign compliance requirements. Organizations can consume the same NVIDIA-accelerated infrastructure in public cloud regions, isolated GovCloud regions, or on-premises via Dedicated Region Cloud@Customer (DRCC) and Compute Cloud@Customer (C3). For tactical or disconnected environments, OCI deploys ruggedized Roving Edge devices equipped with NVIDIA L4, L40S, and A10 GPUs. Basu emphasizes a strategy of bringing AI directly to existing enterprise data stores to avoid costly egress. Autonomous agents are embedded directly into Oracle Fusion Applications, while vector search capabilities are integrated natively into managed database services, including Oracle Database 23ai AI Vector Search and MySQL HeatWave. OCI also hosts managed foundation models—such as xAI Grok, Cohere, and Meta's Llama 3.1 and 3.2—coupled with NVIDIA AI Enterprise blueprints for custom agent orchestration. The combined NVIDIA and Oracle strategy focuses on mitigating the exponential compute and memory bottlenecks of agentic AI through dense chip packaging, network disaggregation, and bare-metal cluster scale. However, realizing these gains at enterprise scale remains contingent on data centers successfully retrofitting for direct-to-chip liquid cooling and securing the gigawatt-level power infrastructure necessary to sustain massive hardware deployments.

精炼阅读摘要

NVIDIA与Oracle Cloud Infrastructure（OCI）阐述了向智能体AI（Agentic AI）演进所带来的算力与架构挑战，展示了涵盖Blackwell硬件、全栈软件以及OCI千兆瓦级（GW）超级集群的端到端技术堆栈。 智能体推理激增驱动算力架构升级 (0:04) Dave Salvator指出，人工智能正从初期的生成式AI向基于思维链推理的智能体AI演进，并最终迈向机器人与边缘自主设备的物理AI。由于智能体系统需要将复杂任务拆解为子目标并执行逐步推理，其推理阶段消耗的Token数量可达传统单次问答系统的100倍。企业在运营上面临着既要满足终端用户低延迟响应，又要维持算力集群高利用率以控制总体拥有成本（TCO）的双重挑战。 针对混合专家模型（MoE）中专家子网络跨GPU密集通信的瓶颈，NVIDIA推出了Blackwell架构与NVL72集群，利用NVLink将72颗GPU整合成统一域，从而提升跨节点吞吐量。此外，GB200通过高带宽片间互连（C2C）绕过标准PCIe总线，使Grace CPU与Blackwell GPU实现高速双向互联，支持GPU动态访问系统内存池以突破显存上限。 软硬件协同推移帕累托能效边界 (3:39) 在模型部署环节，系统整体吞吐量与单用户交互延迟往往相互掣肘。Blackwell架构结合原生FP4低精度计算，推移了推断效率的帕累托边界，使数据中心能够在高饱和运转的同时保持低延迟。此外，NVIDIA推出Dynamo推理服务框架，采用解耦部署架构将计算密集的Prefill（预填充）阶段与受内存带宽限制的Decode（解码）阶段分离，据称可在相同底层硬件上获得30%至40%的性能提升。 为确保基础设施与算力升级节奏同步，NVIDIA维持每年更新架构的规划，包括接下来的Vera CPU与Rubin平台，以及规划于2028年的Feynman架构。为了协助运营商应对直接液冷改造，NVIDIA将GB200 NVL机架设计提交至开放计算项目（OCP），作为高密度液冷部署的开放标准蓝图。 就地计算理念与全形态部署模式 (10:07) Sanjay Basu阐述了OCI的核心策略：将AI算力直接置于企业数据资产所在地，避免大规模数据外流。在应用层，Oracle将自主智能体嵌入Fusion SaaS应用中；在数据层，通过Oracle Database 23ai（集成原生AI向量搜索）、MySQL HeatWave、托管Redis及PostgreSQL提供就地向量检索支持，并提供Meta Llama 3.1/3.2、Cohere和xAI Grok等前沿与开源模型的全托管服务。 为满足不同地区严格的监管和数据驻留合规要求，OCI打破了集中式超大规模区域的局限，提供多形态交付模式。除公有云和政府云（GovCloud）外，OCI通过Dedicated Region Cloud@Customer（DRCC）和Compute Cloud@Customer（C3）将全套NVIDIA加速基础设施引入客户本地机房，并通过搭载NVIDIA L4、L40S和A10 GPU的Roving Edge基础设施为网络隔离场景提供边缘推理支持。 千兆瓦级超大规模互联集群架构 (15:15) 底层基础设施方面，OCI通过消除裸金属实例的虚拟化开销，并采用基于RoCEv2与NVIDIA InfiniBand的无超额配置、无阻塞Clos网络拓扑，提供每节点3.2 Tbps的互连带宽。Basu引用公开的MLPerf基准测试称，OCI实例在性价比上比竞品云厂商最高提升220%，并表示绝大多数前沿大模型开发商均在使用OCI进行大规模预训练。 目前OCI已上线由65,000颗NVIDIA H200组成的单一网络互联训练集群，并正向由131,000颗Blackwell B200构成的超级集群扩展。为支撑这一计算密度，物理数据中心基础设施正从200 kW高密度机架演进至兆瓦乃至千兆瓦（GW）级站点，例如位于德克萨斯州阿比林的Stargate数据中心，规划电力容量正从初期200 MW扩建至1.2 GW。 该方案展示了针对智能体工作负载的软硬件垂直整合蓝图，但其极高密度的计算演进高度依赖液冷设施落地、开放网络协同以及千兆瓦级电力供应的持续到位。

Detailed notes — zh

## Background 本视频由 Ai4 于 2025 年 8 月 24 日发布在 YouTube 平台，由甲骨文（Oracle）云工程高级总监 Sanjay Basu 与英伟达（NVIDIA）加速计算产品总监 Dave Salvator 联合主讲。视频内容主要探讨了部署在甲骨文云基础设施（OCI）上的 NVIDIA 最新 GPU 架构（涵盖 Blackwell、Hopper 与 Ada-Lovelace 等），以及如何利用端到端参考架构优化企业级严苛 AI 负载的运行性能。 ## TL;DR 演讲梳理了 AI 从生成式迈向智能体 AI 与物理 AI 的演进路径，指出推理 Token 激增 100 倍对计算系统带来的巨大挑战。NVIDIA 重点解析了针对 MoE 架构优化的 Blackwell NVL72、C2C 内存直连以及提升 30%–40% 吞吐的 Dynamo 解耦推理技术。随后 Oracle 展示了 OCI 覆盖公有云、主权云及边缘节点的分布式 AI 云布局，并深入介绍了其支持单集群 131,000 颗 Blackwell GPU 互联、供电达吉瓦（GW）级别的超大规模前沿大模型训练集群。 ## Atmosphere & style 整场演讲呈现出专业硬核且节奏干练的科技峰会氛围，开场 Dave 幽默自嘲没有返送屏并调侃两人的演讲照片，迅速打破了严肃的技术距离感。随后两位技术专家进行了流畅的软硬件双重视角交接，从微观的芯片总线带宽、解耦调度，一路铺陈到宏观的 1.2 吉瓦 Stargate 超级数据中心实景。演讲既具备深度的技术架构说服力，又传递出工业级 AI 算力底座在大规模扩展时的宏伟与务实感。 ## Key takeaways - [00:04](https://www.youtube.com/watch?v=0M674LDRdpA&t=4s) AI 发展正从生成式 AI 迈向智能体 AI 与物理 AI，智能体带来的复杂思维链拆解使得推理计算量激增达 100 倍，对系统利用率与低延迟响应提出了极高要求。 - [01:57](https://www.youtube.com/watch?v=0M674LDRdpA&t=117s) Blackwell 架构通过 NVLink Fabric 将 72 颗 GPU 构建为单一的 NVL72 统一域，专为混合专家模型（MoE）不同专家节点间的极速通信与超低延迟协同而设计。 - [01:57](https://www.youtube.com/watch?v=0M674LDRdpA&t=117s) GB200 单元通过芯片到芯片（C2C）高速互联直连 Grace CPU 与 Blackwell GPU，摆脱了传统 PCIe 总线带宽瓶颈，实现显存与系统内存的高速双向扩展。 - [03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) 结合 Blackwell 架构与 NVIDIA FP4 精度，AI 推理的帕累托曲线整体向右上推移，使 AI 工厂能够在维持极高硬件利用率的同时交付极佳的低延迟交互体验。 - [03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) Oracle 在得克萨斯州阿比林建设的 Stargate 超级数据中心现已具备 200MW 装机容量，未来规划将持续扩张至 1.2GW 乃至 2.6GW 级别的吉瓦级电力规模。 - [05:46](https://www.youtube.com/watch?v=0M674LDRdpA&t=346s) 软硬件协同是降本增效的核心，NVIDIA CUDA-X 软件栈与 OCI 深度集成，通过底层算法持续调优在不改动硬件的前提下摊薄客户的 TCO。 - [05:46](https://www.youtube.com/watch?v=0M674LDRdpA&t=346s) 开源推理服务框架 Dynamo 引入 Prefill 与 Decode 解耦调度机制，针对算力敏感与访存带宽敏感阶段分别优化，可直接带来 30% 至 40% 的推理性能飞跃。 - [08:50](https://www.youtube.com/watch?v=0M674LDRdpA&t=530s) NVIDIA 保持每年推出一代新架构的技术节奏，并已将 GB200 MGX 液冷机架参考设计正式捐赠给 OCP 联盟，协助生态伙伴加速推进液冷数据中心落地。 - [10:07](https://www.youtube.com/watch?v=0M674LDRdpA&t=607s) 现代企业应用正步入具备自主判断与规划的智能体时代，OCI 将搭载 NVIDIA 算力的基础设施无缝覆盖至公有云、政府云、主权区域及本地边缘节点。 - [12:33](https://www.youtube.com/watch?v=0M674LDRdpA&t=753s) OCI 践行“推理直接带到数据所在之处”策略，通过 Oracle Database 23ai 原生向量检索与托管主流大模型，支持企业快速编排端到端智能体工作流。 - [15:16](https://www.youtube.com/watch?v=0M674LDRdpA&t=916s) OCI AI 超级集群基于 3.2 Tbps 无阻塞 Clos 网络构建，单集群已稳定上线 65,000 颗 H200 GPU，并正在规划部署 131,000 颗 Blackwell GPU 的超大规模集群。 - [15:16](https://www.youtube.com/watch?v=0M674LDRdpA&t=916s) 全球绝大多数顶尖前沿大模型提供商均已选择在 OCI 上进行大规模分布式训练，在 MLPerf 基准测试中 OCI 的单位经济效益最高领先其他头部云厂商 220%。 ## Detailed walkthrough ### [00:04](https://www.youtube.com/watch?v=0M674LDRdpA&t=4s) 演进轨迹：从生成式 AI、智能体 AI 到物理 AI NVIDIA 加速计算产品总监 Dave Salvator 开场幽默地调侃现场没有返送监视器，两位演讲者只能不时回头看大屏幕，并指着幻灯片上两人的证件照片开玩笑说“可能在邮局通缉令上见过”。他随后切入正题，介绍了本次演讲的分工：先由他梳理 NVIDIA 的加速计算平台与最新架构，再由 Oracle 云工程高级总监 Sanjay Basu 介绍 Oracle 如何将该平台落地到云端并向企业客户开放。 ```mermaid flowchart LR accTitle: AI演进发展轨迹 A["生成式 AI (Generative AI)<br/>基础输入输出"] --> B["智能体 AI (Agentic AI)<br/>复杂推理与思维链"] B --> C["物理 AI (Physical AI)<br/>边缘自动设备与机器人"] ``` Dave 指出，AI 技术正经历从**生成式 AI（Generative AI）**迈向**智能体 AI（Agentic AI）**，并最终走向以自主边缘设备、机器人等为代表的**物理 AI（Physical AI）**的演进路径。这一转变带来了计算需求的剧烈阶跃（stair step），无论是在模型训练还是在推理侧，算力消耗都呈爆发式增长： - **推理 Token 激增**：在推理阶段，智能体 AI 所需的推理计算量预计将达到以往实现的 **100 倍（100x）**。 - **思维链与问题分解**：传统的生成式 AI 主要处理简单的输入（Prompt）和输出（Completion）Token；而推理型/智能体 AI 会在中间生成海量的“推理 Token（Reasoning Tokens）”。它不再停留在基础的知识库一问一答，而是将复杂问题拆解为多个子问题逐步求解，并以更透明的“思维链（Chain of Thought）”方式呈现。 - **系统核心挑战**：企业部署面临的核心难题，在于如何在保证优秀用户响应体验（低延迟）的同时，维持整个系统的高利用率（High System Utilization），从而实现高性价比的服务交付。 --- ### [01:57](https://www.youtube.com/watch?v=0M674LDRdpA&t=117s) Blackwell 架构创新：NVLink 72、C2C 互联与混合专家模型（MoE） 针对上述算力与协同挑战，Dave 重点介绍了 NVIDIA 推出的 Blackwell 架构以及即将推向市场的 Blackwell Ultra 架构。通过自研的 NVLink Fabric 技术，NVIDIA 成功将 72 颗 GPU 互联成一个单一的统一域（NVLink Domain）——即 **NVL72**，使其在逻辑上运作得宛如“一颗超大型 GPU”。 ```mermaid flowchart TD accTitle: NVL72 与 Grace Blackwell 互联架构 subgraph Compute_Node ["GB200 计算单元"] CPU["Grace CPU (ARM 架构)"] GPU1["Blackwell GPU 1"] GPU2["Blackwell GPU 2"] CPU <-->|"C2C 高速互联 (优于 PCIe)"| GPU1 CPU <-->|"C2C 高速互联 (优于 PCIe)"| GPU2 end GPU1 & GPU2 <-->|"NVLink Fabric 统一互联"| NVLDomain["72 颗 GPU 统一 NVLink 域<br/>(超低延迟 MoE 专家间通信)"] ``` 这一全新架构为当前主流的模型设计带来了巨大的架构红利： - **专为 MoE（Mixture of Experts）设计**：现代智能体应用普遍采用混合专家架构，将不同的专家模块部署在不同 GPU 上。NVLink 提供了极高的跨 GPU 通信吞吐与超低延迟，专家节点通信越快，系统性能越强，从而直接降低总拥有成本（TCO）。 - **Grace CPU 与 GPU 对等互联（C2C Link）**：在 GB200 系统中，每两颗 Blackwell GPU 搭配一颗 ARM 架构的 Grace CPU，二者通过专有的芯片到芯片（C2C, Chip-to-Chip）链路直连。这种设计彻底摆脱了传统 PCIe 总线的带宽瓶颈，实现了 GPU 显存与系统内存（System Memory）之间的极速双向按需迁移与大范围内存扩展。 --- ### [03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) 帕累托曲线优化与 Oracle 超级数据中心规模 在讨论推理服务性能时，Dave 引入了**帕累托曲线（Pareto Curve）**的概念。在 AI 推理中，系统响应速度（低延迟）与系统吞吐量/利用率之间存在天然的权衡博弈。优化一个指标往往会牺牲另一个，企业必须在二者之间寻找最佳平衡点。 ```mermaid flowchart LR accTitle: 从 Hopper 到 Blackwell 的帕累托曲线提升 A["Hopper 架构 (FP8/FP16)"] -->|"升级至 Blackwell + FP4"| B["帕累托曲线整体向右上移动<br/>(同等低延迟下实现更高系统利用率与吞吐)"] ``` 从 Hopper 架构跨越到 Blackwell 架构并引入 **NVIDIA FP4** 精度支持后，整个帕累托曲线被显著“向右上推移”： - 数据中心（现称为 **AI 工厂 / AI Factories**）的输出效率大幅提升。 - 允许运营商在维持超高硬件利用率（最大化硬件投资价值）的同时，为更多并发用户提供低延迟的极致交互体验。 Dave 随后展示了 Oracle 正在美国得克萨斯州阿比林（Abilene, Texas）建设的 **Stargate 超级数据中心**实景： - 该园区当前装机容量已达 **200 兆瓦（MW）**。 - 规划目标将扩展为高达 **1.2 吉瓦（GW / Gigawatt）** 级别的超大规模 AI 计算基地。 - Oracle 正全面部署 NVIDIA B200 与 GB200（Grace Blackwell 200），并在全美多地加速兴建类似规模的 AI 基础设施。 --- ### [05:46](https://www.youtube.com/watch?v=0M674LDRdpA&t=346s) 全栈软件生态：CUDA-X、Dynamo 解耦推理与 DGX Cloud Dave 强调，“硬件只是起点，没有生态系统支持和成熟的软件栈，硬件无法单独解决任何问题”。NVIDIA 平台的核心竞争力在于软硬件协同的全栈闭环： - **CUDA-X 软件库集群**：历经多年迭代沉淀了数百个加速库，横跨生成式 AI、科学模拟以及高性能计算（HPC）。通过底层算法持续优化，在不更换硬件的前提下不断压榨出额外性能，进一步摊薄客户的 TCO。 - **与 Oracle 云（OCI）的深度集成**：NVIDIA 顶层应用及加速库深度内嵌在 Oracle Cloud 服务的底层与上层架构中。 ```mermaid flowchart TD accTitle: Dynamo 解耦推理机制 subgraph Disaggregated_Serving ["Dynamo 解耦推理架构"] Request["推理请求"] --> Prefill["Prefill 计算阶段<br/>(计算敏感型算力节点)"] Prefill --> Decode["Decode 生成阶段<br/>(访存带宽敏感型算力节点)"] Decode --> Output["响应输出"] end ``` 针对 AI 推理服务架构，Dave 重点介绍了全新推出的开源推理服务框架 **Dynamo** 及其核心的**解耦服务（Disaggregated Serving）**机制： - **分离 Prefill 与 Decode 阶段**：传统推理将 Prompt 预填充（Prefill）与逐 Token 解码（Decode）混合在同一计算单元上，而两者对算力（Compute-bound）与显存带宽（Memory bandwidth-bound）的敏感度完全不同。 - **显著性能提升**：通过物理或逻辑上的解耦调度，在同等硬件平台上可直接带来 **30% 至 40%** 的性能飞跃，对大规模部署的经济效益产生决定性影响。 - **Oracle Cloud 上的 DGX Cloud 服务**：为缺乏全套自建 AI 运维经验的企业提供托管方案，客户不仅能获得 Oracle 官方的云基础设施支持，还能直接享有 NVIDIA 原厂的企业级专业技术支持与运维保障。 ### [08:50](https://www.youtube.com/watch?v=0M674LDRdpA&t=530s) NVIDIA 架构演进与 MGX 液冷参考设计 NVIDIA 正在以每年一次的强劲节奏向市场持续交付全新架构。除了备受瞩目的新一代 GPU 外，CPU 架构也在同步高速迭代：即将推出的 Vera CPU 将作为 Rubin 平台的核心组件于明年正式推向市场；放眼更长远的技术演进路径，路线图甚至已经延伸到了 2028 年的 Feynman（费曼）架构。为了让整个行业和生态伙伴能够跟上这股前所未有的创新步伐，NVIDIA 主动向业界全面公开了未来数年的产品规划细节，使数据中心建设者和云厂商能够提早做好基础设施层面的规划与改造。 为了解决极高算力密度带来的散热与部署难题，NVIDIA 推出了模块化的 MGX 参考架构设计。其中，专为 GB200 打造的机架参考设计已正式捐赠给开放计算项目（Open Compute Project, OCP），成为 OCP 官方标准参考设计之一。这一举措极大地降低了合作伙伴部署液冷数据中心的门槛，帮助众多首次涉足液冷技术的企业大幅缩短了系统上市时间（Time to Market）。面对 AI 模型能力呈指数级爆发的现状，唯有通过端到端系统架构的极致性能，才能兑现新一代 AI 技术的算力承诺。随后，演讲正式交接给甲骨文云基础设施（OCI）团队。 - **CPU 与平台演进**：Vera CPU 将搭载于 Rubin 平台并于次年发布，长线规划已延伸至 2028 年的 Feynman 架构。 - **开源机架标准**：GB200 MGX 液冷机架设计已贡献给 OCP 联盟，协助生态伙伴加速推进液冷数据中心的大规模落地。 --- ### [10:07](https://www.youtube.com/watch?v=0M674LDRdpA&t=607s) 智能体时代来临与 OCI 全场景 AI 云布局 AI 模型的微调、训练与爆发式增长，本质目的在于让企业级应用具备真正的智能化。信息技术发展史上首次迎来了能够赋予应用“自主性（Agency）”的转折点——现代应用程序不再只是按固定逻辑执行代码，而是具备了自主判断（Self-determination）、逻辑推理（Reasoning）以及多步规划（Planning）的能力。要支撑这种由数以亿计代码与复杂模型构建的智能体工作流，底层必须依赖数百万级计算核心的协同加速。OCI 将 NVIDIA 顶尖硬件、弹性伸缩机制、精准计量与极简易用性打包为统一的云服务体系。 与其他超大规模云厂商（Hyperscalers）截然不同的是，OCI 将搭载 NVIDIA 算力的 AI 基础设施无缝渗透到了客户所需的每一个物理边界： - **公共云区域（Public Regions）与政府云（GovCloud）**：面向通用公有云需求与严苛合规场景。 - **专属区域与主权 AI（Dedicated Region Cloud@Customer / Sovereign AI）**：为监管严苛或拥有既有数据中心的企业直接交付“本地化 OCI 云”。 - **计算云客户化（Compute Cloud@Customer / C3）**：在企业本地机房提供一致的 OCI 计算体验。 - **边缘算力（Roving Edge Infrastructure）**：联合 NVIDIA 部署 L4、L40S 和 A10 GPU，满足边缘端实时低延迟推理需求。 ```mermaid flowchart TD accTitle: OCI 全场景分布式 AI 部署架构 accDescr: 展示 OCI 如何将 NVIDIA 算力部署于公有云、政府云、主权区域及边缘节点 subgraph CloudCore ["核心公有云与主权云"] A["OCI Public Region 公有云"] B["GovCloud 政府合规云"] C["Dedicated Region 主权 AI"] end subgraph DistributedEdge ["客户本地与边缘计算"] D["Compute Cloud@Customer (C3)"] E["Roving Edge 边缘节点<br/>(NVIDIA L4 / L40S / A10)"] end CloudCore --> DistributedEdge ``` --- ### [12:33](https://www.youtube.com/watch?v=0M674LDRdpA&t=753s) 融合数据层与生成式 AI 智能体生态 在企业应用层面，OCI 的核心策略是“将推理与智能体直接带到数据所在之处”。无论是 Oracle 旗下的原生 SaaS 产品还是 Fusion 云应用，都已经全面内嵌了自主智能体能力；对于非 Oracle 的第三方业务系统，则可通过 OCI AI Services 实现灵活的智能接入。在底层数据库与存储矩阵中，OCI 提供了集成本地向量检索能力的 Oracle Database 23ai、全托管 MySQL HeatWave，以及托管 Redis 和托管 PostgreSQL，确保所有企业数据与智能体均运行在由 NVIDIA 算力加速的高效底座之上。 在模型构建与生成式 AI 平台方面，OCI 提供了丰富的全托管主流基础大模型支持，包括 xAI 的 Grok 系列、Cohere 模型以及 Meta 发布的 Llama 3.1 和 3.2 系列。开发者可以直接基于 OCI Data Science 平台与原生 AI 服务编排复杂的智能体工作流，亦可直接采用 NVIDIA AI Enterprise Blueprints（企业参考蓝图）快速落地特定场景解决方案。整套软硬件协同方案使用户无需在底层基础设施运维上耗费精力，专注于业务逻辑与智能体创新。 | 层次 | 核心产品与技术组件 | 关键能力与 NVIDIA 协同 | |:--- |:--- |:--- | | **SaaS / 业务应用层** | Fusion Applications、原生 AI Services | 内置智能化 Agent，就地完成业务推理与决策 | | **数据与存储层** | Oracle Database 23ai、MySQL HeatWave、托管 Redis/PostgreSQL | 原生向量搜索（Vector Search）与高并发数据检索引擎 | | **生成式 AI / 模型层** | OCI GenAI 平台、xAI Grok、Cohere、Meta Llama 3.1/3.2 | 提供全托管基础模型，支持微调、提示词工程与评估 | | **平台服务与蓝图** | OCI Data Science、NVIDIA AI Enterprise Blueprints | 快速构建多智能体编排工作流，开箱即用参考架构 | --- ### [15:16](https://www.youtube.com/watch?v=0M674LDRdpA&t=916s) OCI AI 超级集群架构与前沿大模型基础设施 OCI AI 基础设施的核心竞争力在于其顶尖的硬件性能与网络架构。在权威 MLPerf 基准测试中，OCI 展现出的单位经济效益与吞吐表现相较于其他头部超大规模云厂商最高领先 220%。这一性能飞跃归功于 OCI 打造的超大规模计算集群——“超级集群（Supercluster）”。该集群支持从 2 个裸金属节点平滑扩展至单个集群内的 131,000 个节点/GPU，底层采用了无超配、无阻塞的 Clos 网络架构（Non-blocking Clos Network），提供高达 3.2 Tbps 的横向互联带宽，并同时支持基于以太网的 RoCEv2 与 InfiniBand 两种高速网络协议。 在前沿大模型（Frontier Models）训练领域，除了一家特定机构外，全球绝大多数顶尖大模型提供商均已选择在 OCI 上进行大规模分布式训练。目前 OCI 已在线稳定运行包含 65,000 颗 H200 GPU 的单一超大训练集群，并正全力构建搭载 131,000 颗 Blackwell 系列 GPU（B200 / GB200）的下一代超级算力集群。为承载如此密集的计算负荷，OCI 的数据中心供电规划正从现有的数百兆瓦级稳步跨越至 1.2 吉瓦乃至 2.6 吉瓦规模，整套系统在上层通过开源的托管调度与编排服务实现高效管理，为全球 AI 创新提供工业级的算力底座。 - **前沿模型客户群**：“除了少数个别厂商外，全球几乎所有的前沿大模型提供商都运行在 OCI 之上。” - **超大集群规模**：单集群已上线 65,000 颗 H200 GPU，下一阶段将扩展至 131,000 颗 Blackwell GPU。 - **极致网络架构**：3.2 Tbps 双协议（RoCEv2 与 InfiniBand）无阻塞 Clos 网络，消除大规模并行训练的通信瓶颈。 - **能源基础设施**：支持单数据中心集群供电能力向吉瓦（GW）级别持续跃升。 ## Notable quotes > [00:04](https://www.youtube.com/watch?v=0M674LDRdpA&t=4s) 可能在邮局通缉令上见过 — Dave Salvator > [00:04](https://www.youtube.com/watch?v=0M674LDRdpA&t=4s) 在推理阶段，智能体 AI 所需的推理计算量预计将达到以往实现的 100 倍（100x）。 — Dave Salvator > [01:57](https://www.youtube.com/watch?v=0M674LDRdpA&t=117s) 一颗超大型 GPU — Dave Salvator > [03:39](https://www.youtube.com/watch?v=0M674LDRdpA&t=219s) AI 工厂 / AI Factories — Dave Salvator > [05:46](https://www.youtube.com/watch?v=0M674LDRdpA&t=346s) 硬件只是起点，没有生态系统支持和成熟的软件栈，硬件无法单独解决任何问题 — Dave Salvator > [12:33](https://www.youtube.com/watch?v=0M674LDRdpA&t=753s) 将推理与智能体直接带到数据所在之处 — Sanjay Basu > [15:16](https://www.youtube.com/watch?v=0M674LDRdpA&t=916s) 除了少数个别厂商外，全球几乎所有的前沿大模型提供商都运行在 OCI 之上。 — Sanjay Basu > [15:16](https://www.youtube.com/watch?v=0M674LDRdpA&t=916s) 超级集群（Supercluster） — Sanjay Basu ## Mentioned Sanjay Basu, Dave Salvator, NVIDIA, Oracle, Oracle Cloud Infrastructure (OCI), Blackwell, Blackwell Ultra, Hopper, Ada-Lovelace, GB200, B200, H200, L4, L40S, A10, NVLink, NVLink 72 (NVL72), Grace CPU, Vera CPU, Rubin, Feynman, CUDA-X, Dynamo, DGX Cloud, MGX, Open Compute Project (OCP), Stargate 超级数据中心, Oracle Database 23ai, MySQL HeatWave, Fusion Applications, OCI Data Science, NVIDIA AI Enterprise Blueprints, xAI Grok, Cohere, Meta Llama 3.1, Meta Llama 3.2, MLPerf, Clos 网络, RoCEv2, InfiniBand ## Video description Sanjay Basu, Senior Director, AI/GPU - Cloud Engineering, Oracle Dave Salvator, Director of Accelerated Computing Products, NVIDIA Abstracts Model complexity and data sources are on the rise, creating new challenges and opportunities. This session will explore how the latest advancements in NVIDIA's cutting-edge GPU infrastructure platforms on Oracle Cloud Infrastructure (OCI)—including the Blackwell, Hopper, and Ada-Lovelace architectures—can effectively meet the evolving needs of next-generation models. Attendees will learn how to optimize their AI infrastructure deployment using NVIDIA's comprehensive end-to-end reference architecture, which integrates seamlessly with OCI to deliver exceptional performance tailored for demanding AI workloads. [Source](https://www.youtube.com/watch?v=0M674LDRdpA)

Source quotes

[0:04] Dave Salvator: 可能在邮局通缉令上见过 [0:04] Dave Salvator: 在推理阶段，智能体 AI 所需的推理计算量预计将达到以往实现的 100 倍（100x）。 [1:57] Dave Salvator: 一颗超大型 GPU [3:39] Dave Salvator: AI 工厂 / AI Factories [5:46] Dave Salvator: 硬件只是起点，没有生态系统支持和成熟的软件栈，硬件无法单独解决任何问题 [12:33] Sanjay Basu: 将推理与智能体直接带到数据所在之处 [15:16] Sanjay Basu: 除了少数个别厂商外，全球几乎所有的前沿大模型提供商都运行在 OCI 之上。 [15:16] Sanjay Basu: 超级集群（Supercluster）

Source quotes

[0:04] Dave Salvator: at the post office. [1:57] Dave Salvator: one giant GPU. [3:39] Dave Salvator: shift the Pareto efficiency curve up and to the right. [3:39] Dave Salvator: AI factories [5:46] Dave Salvator: running them on specialized, decoupled compute pipelines yields a 30% to 40% performance improvement on the same underlying hardware platform. [10:07] Sanjay Basu: infusing business software with true agency. [10:07] Sanjay Basu: giving software the ability to reason, self-determine, and formulate multi-step plans autonomously. [12:25] Sanjay Basu: running inference and fine-tuning directly adjacent to where enterprise data already lives. [15:15] Sanjay Basu: delivers up to 220% better price-performance economics compared to competing hyperscaler instances [15:15] Sanjay Basu: except one, all of the frontier model providers are actually using OCI

Original transcript

[0:04] Uh, good afternoon. Hopefully everyone [0:05] can hear me. Um, we're going to have to [0:07] like be sneaking peaks at the screen [0:09] since we don't have a monitor up here. [0:11] So, uh, hope you'll humor us on that. [0:12] There are mug shots. Might have seen [0:14] those at the post office. Okay, keep [0:16] going. So, here's our agenda. Um I'm [0:19] going to sort of go first give sort of [0:21] an overview of the NVIDIA platform and [0:22] then Sanjay will follow on to talk a bit [0:24] more about um how Oracle takes that [0:26] platform into the cloud and makes it [0:28] available to its customers. So as we've [0:31] seen and you've been hearing other [0:32] speakers talk about you know we're on [0:33] this trajectory now moving from [0:35] generative AI to Agent AI and ultimately [0:37] that will make its way into physical AI [0:39] with things like autonomous edge [0:41] devices, robotics etc. Um the other [0:44] thing that of course this is driving is [0:45] an incredible uh increase in the [0:47] required compute to deliver these [0:49] services. Agentic in particular really [0:51] rep represents a major stair step and [0:53] how much compute we're going to need not [0:55] only for training but especially on the [0:56] inference side and we'll talk a bit more [0:58] about that as we move through the [0:59] slides. So as an example you know we're [1:02] saying that we believe reasoning is [1:03] going to require up to 100k or 100x [1:06] rather I should say more tokens than [1:09] previous implementations. And a lot of [1:10] that just has to do with the fact that [1:12] with generative AI, you basically have [1:13] input tokens and output tokens. But with [1:16] reasoning AI, you have obviously in the [1:17] middle a huge number of reasoning [1:19] tokens. And we sort of give a simple [1:22] example here of kind of contrasting [1:23] between, you know, knowledge bas you [1:25] know very basic Q&A versus actual [1:27] problem solving. And that's what Agentic [1:29] AI is really coming to the market to do. [1:31] you know taking complex problems [1:33] breaking them down into component [1:34] problems solving for those and then also [1:36] doing that in a more transparent way [1:38] where you can actually see it's chain of [1:39] thought thinking right so um and of [1:42] course the the challenge in delivering [1:44] all this is striking a balance between [1:46] great user experience while maintaining [1:48] um high overall system utilization and [1:50] really getting the most out of your [1:51] platform so that you can deliver these [1:53] services in a cost-effective way [1:57] so our main solution to that today is [1:59] our Blackwell architecture and very very [2:01] soon our Blackwell Ultra architecture [2:04] which is actually coming into market uh [2:05] this year. Uh and so it's basically one [2:08] giant GPU and what we've bu we've been [2:10] able to do with this platform uh through [2:12] our NVLink fabric technology is bring 72 [2:15] of these GPUs together in a single [2:17] unified NVLink domain. So it can [2:20] essentially act as one giant GPU. Now [2:22] this has some very obvious advantages. [2:24] Most agentic uh applications that are [2:26] being built today are using what's [2:28] called mixture of experts model [2:29] architecture ore for short. This [2:32] generally means that you're going to [2:33] have multiple experts parked on [2:34] different GPUs and they all need to talk [2:36] to each other and the faster they can [2:37] talk to each other the more performance [2:39] you're going to get and higher [2:40] performance translates generally [2:42] speaking into lower TCO right and so um [2:46] this is just an amazing new architecture [2:48] on a bunch of different levels. First of [2:49] all, there was the NVL72 [2:51] um domain where we're bringing 72 GPUs [2:53] together and then also sort of what [2:55] we've done to architect of how we bring [2:57] CPU and GPU together more peer [2:59] processors because uh we basically have [3:01] what are called the C toC links where um [3:03] for each pair of those uh Blackwell GPUs [3:06] you have one of our grace CPUs an [3:08] ARMbased grace CPU and they talk to each [3:11] other what we call our C to link and so [3:14] this uh very much is a is kind of a [3:16] somewhat novel architecture in the sense [3:18] that we are finding a way to um frankly [3:20] engineer a better pathway than say PCIe [3:23] as an example um and will allow for more [3:25] performance, more scalability and also [3:27] more access to system memory on an as [3:30] needed basis if the model um needs to [3:32] move from say GPU memory over to system [3:34] memory and vice versa. um that very fast [3:36] interconnect enables that [3:39] and of course Oracle is uh has deployed [3:42] uh Blackwell uh both in in the form of [3:44] RB200 and then our GB2000 Grace [3:46] Blackwell 200 um in their data centers [3:49] and are bringing this uh to market not [3:51] only bringing it to market but bringing [3:52] it to market at amazing scale and sort [3:54] of an example of that well actually [3:56] before I jump to that let me quickly [3:57] discuss this uh in terms of one of the [4:00] things that we're solving for with uh [4:02] with Agentic AI is this is what we call [4:05] a parto curve and right what it's [4:06] representing here is ideal sort of [4:09] points of performance at each of these [4:10] stops along the way and again I touched [4:13] on this a minute ago this notion that [4:14] responsiveness and overall system [4:16] performance kind of tug at each other [4:18] right and the more you solve for one [4:20] that can come at the expense of the [4:21] other so finding that right balance is [4:23] is a challenge that every organization [4:25] has to solve for and it's not a [4:27] one-sizefits-all answer um because again [4:30] you know it depends on you know where [4:31] you where you're kind of sitting your [4:32] goals but one of the key takeaways here [4:35] that we sort of have talked about for a [4:36] couple of months now is moving from our [4:38] Hopper architecture to our Blackwell [4:40] architecture with uh with NVIDIA FP4 is [4:43] allowing us to achieve a massive [4:45] increase in overall data center or AI [4:47] factory now as we like to call them um [4:50] output and that ability to take that [4:51] whole PTO curve and shift it up and to [4:53] the right which is sort of ideally where [4:55] you want it to keep going and the [4:56] further up into the right you go the [4:58] more you can solve for both keeping your [5:00] systems lit up and busy in other words [5:02] keep your utilization where you want it [5:03] so you're getting value from the [5:04] platform while at the same time [5:06] delivering great user experiences to [5:08] more people, right? So allowing us to [5:10] scale and allowing people to really [5:11] enjoy your service which hopefully will [5:13] in turn grow your service. [5:16] So this is what I meant to say about [5:18] scale. So this is a picture from one of [5:20] the Stargate data centers that Oracle is [5:21] building. This one's in Abalene, Texas. [5:23] Um already at 200 megawatts, it's on its [5:25] way to becoming a 1.2 gawatt um data [5:28] center installation. Uh and more of [5:30] these are being built around the US. So [5:32] uh Oracle is very much all in on [5:35] bringing AI at scale to its users um [5:38] basically uh to be able to build these [5:40] next generation applications and Sanjay [5:42] will touch more on that in just a few [5:43] minutes. [5:46] So we've talked about hardware um and [5:48] hardware is just the beginning because [5:49] at the end of the day hardware absent [5:52] ecosystem support and really a a strong [5:55] mature software stack doesn't solve any [5:57] problems. So you need both. you really [5:59] need that full stack endto-end solution [6:00] and that's exactly what our platform is [6:02] going to give you. So we have of course [6:04] hundreds of these CUDA X libraries now [6:06] uh each each meant to sort of solve for [6:08] different aspects different types of [6:10] applications by the way not only just [6:11] for AI but also for scientific [6:13] simulation and and high performance [6:14] computing and this is something of [6:16] course we've developed over many years [6:18] so this is a software stack that is [6:20] robust mature and is also constantly [6:22] being optimized and and and being [6:25] improved in such a way that we're able [6:27] to continue to deliver more performance [6:29] every time we bring more performance to [6:30] a platform on the same hardware [6:33] that essentially is think of that as [6:35] just another way to lower TCO right as [6:37] you think about going to deploy. [6:40] And so then if you put that in context [6:42] of the overall stack and now we sort of [6:44] show how our stack kind of you know [6:45] slots in with the overall Oracle cloud [6:47] stack um you see that again it's a very [6:50] complete solution top to bottom that [6:52] includes of course all the hardware here [6:54] um as well as sort of showing sort of [6:56] inputs and outputs if you will but again [6:58] it also shows sort of some of our [7:00] topofstack services uh almost all of [7:02] which are offered by Oracle right and so [7:05] uh in addition of course Oracle has its [7:06] own services many of which have our [7:08] technologies deeply integrated into them [7:10] so that as you're using those services, [7:12] you're also getting the benefit of the [7:13] NVIDIA acceleration from them. [7:16] And so I did want to touch briefly on a [7:18] newer one that we've brought um into [7:20] market called Dynamo. We introduced this [7:21] a little earlier this year. Um this is [7:24] essentially an inference serving [7:25] framework that we have delivered [7:26] primarily aimed at doing both uh [7:29] generative as well as agentic AI. Um and [7:32] one of its big features is of course the [7:33] ability to do disagregated serving. And [7:35] what disagregated serving does is it [7:37] basically targets and puts the uh the [7:41] pre uh the prefill process separated [7:44] from the decode process during inference [7:46] because each one has a different sort of [7:48] performance sensitivity right and so in [7:50] doing that we have seen that we are [7:52] getting pretty significant performance [7:54] bumps again even on that same platform [7:56] where you can get you know we now I [7:58] don't have exact numbers for you but the [7:59] range is on the order of 30 to 40% more [8:01] performance by going to disag. So that's [8:04] a pretty significant bump and if you [8:05] think about that in terms of the [8:06] economies of deployment that has some [8:08] very potentially positive uh material [8:10] impact [8:13] and so uh another thing that we offer on [8:14] the Oracle cloud platform is what's [8:16] called DGX cloud. And so this is yet [8:18] another way particularly for for those [8:20] organizations who may want a little bit [8:22] of extra support. Not only does this [8:23] come of course with support from Oracle, [8:25] but it also comes with uh enterprise [8:27] grade support from Nvidia. So, this is a [8:29] really great service for those looking [8:31] to sort of get started, who maybe don't [8:33] have all the in-house expertise that [8:34] they want just yet and are looking for [8:37] some additional support and help from [8:38] NVIDIA, from Oracle to get their [8:40] applications built and deployed. [8:45] There we go. And of course, the other [8:46] thing we're doing at NVIDIA is we're on [8:48] a one-year cadence with our roadmap in [8:50] terms of delivering new architectures to [8:51] the market on a yearly cadence, new GPU [8:53] architectures. We're also in the [8:55] process, of course, as you can see, of [8:56] revving our our CPU architecture. We [8:59] have our upcoming Vera CPU, which will [9:01] be part of the Reuben platform, um, [9:02] which we plan to bring into market next [9:04] year. And then we sort of see us pushing [9:06] out to the right here, uh, including our [9:07] Fineman architecture way out there in [9:09] 2028. And so, we're just on this this [9:12] sort of relentless pace. But the other [9:13] thing to notice there is you see the two [9:14] rack designs. And these are things where [9:16] we actually build reference designs of [9:18] these called MGX reference designs. uh [9:21] the one example for GB200 we've also [9:23] donated to the open compute project OCP [9:25] so it is now one of their reference [9:26] designs and this has facilitated uh [9:29] tremendously for some of our partners to [9:31] be able to deploy liquid cooling in data [9:33] centers many of whom are doing it for [9:34] the first time and so this is another [9:36] way that we help speed time to market [9:38] and the reason we're showing this much [9:39] of our roadmap to the industry and [9:41] basically to the public is to help them [9:43] understand what's coming and when it's [9:45] coming so that they can be prepared and [9:46] then things like our MGX reference [9:48] designs help them get to market faster [9:50] to be able to sort of match our pace. [9:52] And we're doing all this primarily [9:53] because the rate at which AI [9:54] capabilities are growing. This is the [9:56] performance we're going to need to [9:58] actually deliver on that promise. Right. [10:00] Okay. And with that, I'm now going to [10:02] hand it over to Sanjay, who will talk [10:03] about OCI and the great things they're [10:05] doing with NVIDIA technology. [10:07] >> Thanks, Dev. Hi, everyone. Um, so why [10:11] are we having all these AI models, [10:15] right? Why we are having finetuning [10:18] training and everything? Uh because we [10:22] want to make our applications [10:25] intelligent [10:27] and in the entire history of the [10:30] information technology [10:33] uh this is the first time we are [10:36] actually enabling application with [10:39] agency. [10:41] Now the applications are able to [10:43] selfdetermine. [10:45] Now the applications are able to reason [10:50] and now the applications are take the [10:54] next step as in planning and for that we [10:57] need [11:00] hundreds of codes hundred and thousands [11:02] of codes millions of codes and that is [11:05] exactly what he said that how Nvidia is [11:09] helping to accelerate that journey and [11:13] what we are doing is that we putting [11:15] everything together in a package and we [11:19] are delivering it to you as a cloud [11:25] where you have the elasticity where [11:27] where you have the metering and where it [11:30] is easy to use and unlike other [11:33] hyperscaler what we are doing is that we [11:36] are making this AI infrastructure [11:39] with Nvidia available in our public [11:44] region. [11:45] in our Gulf cloud region. If you already [11:49] have a uh data center and you are in a a [11:53] regulatory industry, we can bring our [11:57] cloud to you. So that is the DRCC [12:01] sovereign AI. Then C3 is uh cloud [12:06] compute at customer and then roving edge [12:10] is our edge solution. If you want to do [12:13] inference at the edge, we are work we [12:16] have worked with Nvidia to provide L4, [12:20] L40s, A10 at the edge. So what I will do [12:25] is that I have actually way too many [12:29] slides. So, so one of the things that we [12:33] are doing at Oracle is that we are [12:36] making your application intelligent [12:39] not only by giving you the ability to [12:42] fine-tune also to take inference at [12:46] where your data is. So if you are using [12:50] Oracle application, our SAS product, our [12:55] fusion apps, we are making this agents [12:58] available to fusion apps. If you are [13:03] wanting to use AI services with any [13:06] other application that is not a native [13:09] Oracle application, we have our AI [13:12] services. And then at our database [13:16] layer, we are giving you the option of [13:20] running Oracle and nonOracle databases. [13:24] We are giving you the option of using [13:26] our uh database [13:29] 23 AI which is our Oracle database with [13:33] vector. We are also giving you the [13:37] opportunity to run our MySQL [13:41] heatwave which is a managed MySQL [13:44] option. We also have managed radius and [13:48] managed hostess and all these [13:52] applications all these agents they are [13:54] running on our AI infrastructure which [13:57] is powered by Nvidia. [14:01] So what we are [14:04] so what we are doing is that we are also [14:07] giving you the ability to build your [14:11] agents on our genai platform and we are [14:16] also providing access to managed models [14:20] like X.AI's AI is rock three [14:26] rock four co here and then the metam [14:31] models both 3.1 and 3.2 two. So you can [14:35] actually use this models on OCI and they [14:39] are all powered by Nvidia uh [14:44] GPUs and then you can use our data [14:46] science platform and then you can use [14:49] our other [14:52] services and you can build out your [14:54] agentic workflow and some of the [14:58] services are native services on OCI and [15:02] for some of the services we are using [15:04] Nvidia [15:06] AI enterprise uh blue loop prints. So I [15:10] will go over that in a short bit. So [15:16] whatever we are doing with the [15:18] applications and with the agents the key [15:22] differentiator is our AI infrastructure. [15:25] So we have published our ML parf and you [15:29] can see that with our ML parfomics [15:34] for running workloads on our AI infra is [15:38] actually 220% [15:41] better than some of the instances of [15:44] other hyperspalar [15:48] when we are running A100 H100 H200 [15:53] B200 [15:55] GB maybe 200 and if we are running it in [15:58] a cluster. One of the key point that we [16:03] have introduced is how we are operating [16:05] our backbone which we are calling our [16:09] super cluster and our supercluster can [16:13] scale from two bare metal nodes to [16:19] 131,000 [16:20] nodes in a single cluster. [16:25] Uh so for that what we had to do is that [16:28] we have provided a non overs subscribed [16:32] non clothing network which is uh 3.2 two [16:38] terabits per second and we are having [16:42] this speed achieved by using both Rocky [16:47] V2 which is over Ethernet and also [16:51] infinite band. [16:56] So this is the portfolio of the GPUs [17:01] that we are running on OCI. Of course, [17:04] we have a future road map also, but [17:08] right now we are providing you a [17:10] platform which can go from inference to [17:14] finetuning to frontier model training. [17:18] So uh except one all of the frontier [17:23] model providers are actually using OCI [17:28] and and what we are doing for the large [17:32] scale out training right now we are [17:36] running 65,000 [17:40] H200 [17:42] GPUs [17:43] in a single cluster and and with the V [17:48] 200 we are actually building out a [17:51] cluster of 131,000 [17:54] GPUs and that is what he showed where we [17:59] are actually having uh 200 kilowatt [18:03] uh 200 kilowatt data centers and we are [18:09] scaling up to 1.2 and then to 2.6 six to [18:13] run 1 million uh kilow center. So what [18:18] we are doing is that we are [18:22] not only enable you to build agents on [18:26] OCI but we are also providing our [18:29] managed orchestration and scheduling [18:33] services and everything is built on [18:36] opensource.