NVIDIA Introduces ovrtx Library for In-App RTX Sensor Simulation

NVIDIA Introduces ovrtx Library for In-App RTX Sensor Simulation NVIDIA has released ovrtx as prerelease software on GitHub, providing a lightweight C and Python SDK that allows developers to embed RTX sensor simulation and OpenUSD rendering directly inside their existing application workflows. ovrtx enables host applications to generate camera, lidar, radar, and semantic segmentation outputs from OpenUSD scenes while retaining complete control of the application loop. A typical integration follows a five-step pattern: creating a renderer, loading USD scenes, configuring RenderProducts and RenderVars, stepping simulation from the host loop, and retrieving output tensors in CPU or GPU memory. The library integrates into modular Omniverse workflows alongside ovstage for shared scene data, ovphysx for physics, and ovstream for browser-based streaming. Early workflows and tests include a SimReady Blender sample blueprint and cloud-native CAD visualization explorations with PTC Onshape Render Studio. Architecture and Integration Steps Part of the NVIDIA Omniverse libraries included in the NVIDIA Agent Toolkit, ovrtx exposes real-time physically based RTX rendering and sensor modeling through C and Python APIs. Rather than forcing developers into a standalone platform, ovrtx runs as an on-demand service embedded within existing host software. Integrating ovrtx involves instantiating the renderer, ingesting OpenUSD data through files, strings, or references, and setting up USD-native RenderProducts and RenderVars for outputs like color, depth, semantic segmentation, and lidar point clouds. During execution, the host loop updates the scene and steps the simulation by supplying a delta time, receiving outputs mapped to CPU or GPU memory for model consumption or display. Modular Omniverse Stack ovrtx can operate as a standalone visualization engine or alongside other Omniverse libraries. Central to multi-library operations is ovstage, a shared runtime substrate across CPU and GPU memory that lets ovrtx read scene changes while libraries like ovphysx write simulation updates, eliminating duplicate scene graphs. Developers can combine ovrtx with ovphysx to simulate physical dynamics like collisions and friction, with ovstream to stream rendered frames to browser interfaces, or with ovstorage to manage USD assets across storage workflows. Sample Workflows and Enterprise Adoption NVIDIA outlined multiple implementations ranging from a minimal single-image rendering smoke test to interactive viewers and full application integrations. At SIGGRAPH 2026, NVIDIA demonstrated a SimReady Blender workflow blueprint capable of running locally on DGX Station hardware using NemoClaw. Additionally, the PTC Onshape Render Studio team is testing ovrtx to bring path-traced rendering into cloud-native CAD and product data management systems, using OpenUSD to connect product design geometry with downstream visualization and simulation without disrupting existing application architecture.

NVIDIA 推出 ovrtx 库：支持在现有应用中集成 Omniverse RTX 传感器仿真 NVIDIA 宣布作为 NVIDIA Agent Toolkit 组成部分的 Omniverse 库推出 ovrtx 预发布版本。该轻量级 C/Python SDK 允许开发者在完全掌控宿主应用架构和循环的前提下，从 OpenUSD 场景生成相机、激光雷达、毫米波雷达等 RTX 传感器及渲染输出。 ovrtx 是轻量级 C 与 Python SDK，目前已在 GitHub 发布预发布版，用于生成相机、激光雷达、毫米波雷达、语义分割等传感器输出。 宿主应用保留对主循环和架构的控制权，通过配置渲染器、加载 OpenUSD 资产、定义 RenderProducts/RenderVars、驱动步进并读取张量来完成集成。 ovstage 作为共享运行时阶段，可与 ovphysx（物理）、ovstream（串流）、ovstorage（存储）等模块协同工作，共享场景数据和仿真状态。 该库已在 Blender 工作流蓝图和 PTC Onshape 云原生 CAD/PDM 渲染设计中完成实际验证与应用探索。 架构与集成流程 ovrtx 强调由宿主应用掌控整体工作流。典型集成包含五个步骤：首先创建并配置 ovrtx 渲染器；随后从文件、URL 或内存字符串加载/引用 OpenUSD 场景；利用 RenderProducts 与 RenderVars 原生定义相机、激光雷达或语义分割等所需输出规范；接着在宿主应用主循环中调用步进（step 或异步 step_async）推进模拟；最后将生成的张量映射回 CPU 或 GPU 内存供下游处理、分析或通过 DLPack、PyTorch 直接调用。 多库协同与底层机制 ovrtx 依赖 Omniverse RTX 渲染器，结合 RTX GPU 上的 RT Core 与 Tensor Core 实现物理级光线输运仿真。在多库工作流中，ovstage 提供通用的 OpenUSD 场景数据基底，使 ovphysx 写入的物理状态能够直接被 ovrtx 读取而无需维护独立场景。此外，开发者还可按需组合负责交互式浏览器推流的 ovstream 及负责数据同步的 ovstorage。 实际应用与生态整合 NVIDIA 展示了多种落地实践，包括基础的 Hello World 渲染测试、基于 Vulkan/CUDA 或浏览器的实时查看器，以及联合 ovphysx 的双进程物理模拟。在工业场景中，SimReady Blender 样本工作流提供了向 3D 工具添加仿真代理的蓝图，支持在搭载 NemoClaw 的 DGX Station 本地运行；PTC Onshape 团队则探索了将其作为底层 API 嵌入云原生 CAD 和 PDM 工作流，在保障原有产品架构的同时实现路径追踪与物理仿真。

Developers building 3D, design, simulation, robotics, and industrial digital twin applications need ways to bring physical AI capabilities into the tools and services they already use. Many of these workflows already depend on OpenUSD scenes, simulation-ready (SimReady) assets, Blender -based workflows, CAD pipelines, or domain-specific app stacks. The challenge is how to provide applications and AI agents with callable tools to inspect scenes, generate sensor outputs, test changes, and return evidence developers can trust. NVIDIA Omniverse libraries , now part of NVIDIA Agent Toolkit , provide modular building blocks for adding these capabilities into existing applications. ovrtx is the RTX sensor simulation library in that stack, now available as prerelease software on GitHub. It provides developers with a lightweight C and Python SDK for generating camera, lidar, radar, and related sensor outputs from OpenUSD content while keeping control inside the host application. Within the Omniverse libraries stack, ovstage provides a shared runtime stage for OpenUSD scene data, enabling libraries such as ovphysx and ovrtx to exchange simulation state, scene changes, and prim identities through a common data substrate. This post walks through how ovrtx integrates into existing workflows as part of the Omniverse libraries model. Steps include loading OpenUSD content, configuring render and sensor outputs, stepping from an application loop, reading results back into the app, and applying the same model in a PTC Onshape Render Studio example and a SimReady Blender sample workflow. How do NVIDIA Omniverse libraries help application developers? Simulation and rendering are becoming part of the application layer for a wide range of workflows including design, robotics, autonomous vehicles (AVs) , industrial digital twins, and physical AI. Developers and teams need to visualize OpenUSD scenes, generate physically grounded data, test perception systems, validate behavior, and view results inside the tools they use. These include CAD systems, Blender-based workflows, PLM or PDM workflows, OpenUSD asset pipelines, internal simulation stacks, and domain-specific applications. A previous post, Integrate Physical AI Capabilities into Existing Apps with NVIDIA Omniverse Libraries , introduces a library-first approach for adding Omniverse capabilities to existing application stacks. That post covers how standalone libraries can expose sensors, physics, and storage capabilities through focused APIs, giving teams more control over where those capabilities run in their own tools and services. ovrtx applies that approach specifically to Omniverse RTX sensor simulation and visualization. As announced at SIGGRAPH 2026 , NVIDIA Agent Toolkit now includes Omniverse libraries, with ovrtx serving as the RTX sensor simulation layer. It helps applications and agents generate camera, lidar, radar, semantic segmentation, and visual preflight outputs from OpenUSD scenes. Used with other Omniverse libraries, it becomes part of a tool-driven workflow for preparing and validating simulation-ready 3D content. Figure 1. Existing application workflow using OpenUSD scene data with ovrtx to generate RTX-rendered camera and sensor outputs That application control matters whenever sensor simulation needs to fit inside an existing workflow. For physical AI teams, those outputs can support synthetic data generation (SDG), perception testing, robotics workflows, simulation-ready digital twins, and design review. With ovrtx now available as pre-release software on GitHub, you can inspect the C and Python APIs, run the examples, test OpenUSD scene loading, and evaluate how RTX sensor outputs fit into your own application loop. How to integrate ovrtx into existing workflows The core principle behind ovrtx is straightforward. As a developer, you can bring NVIDIA Omniverse RTX sensor simulation into your own applications while maintaining control of the application architecture, UI, data model, and workflow. The lightweight C and Python SDK is designed for projects that need a real-time 3D viewport, RTX rendering, or sensor simulation, including camera, lidar, radar, ultrasonic, and related outputs. A typical integration follows five steps: Create and configure an ovrtx renderer. Load or reference OpenUSD content from the application’s existing pipeline. Define RenderProducts and RenderVars for the camera, lidar, radar, semantic segmentation, or rendered outputs the workflow needs. Step rendering or sensor simulation from the application loop. Map the results back to CPU or GPU memory so the app can display, save, analyze, or pass them to another process. A pseudo-code example is shown below—the host application owns the workflow. renderer = ovrtx.Renderer() renderer.open_usd(app_scene_or_usd_layer) render_products = {"/Render/Camera"} # Add lidar, radar, or other outputs as needed. while app.is_running(): app.update_scene_state() products = renderer.step( render_products=render_products, delta_time=app.frame_delta_seconds, ) for product in products.values(): for frame in product.frames: color = frame.render_vars["LdrColor"].map(device=ovrtx.Device.CPU) app.use_render_output(color) Control remains in the host application. The app owns the loop; ovrtx produces the requested render or sensor outputs; and the results return to the workflow for display, storage, analysis, or downstream processing. Core components in the ovrtx integration flow The host application uses a few core pieces when it calls ovrtx: the Omniverse RTX renderer, an OpenUSD stage, and RenderProducts and RenderVars that define the outputs to generate. Omniverse RTX Renderer Omniverse RTX Renderer is the core technology exposed by the ovrtx library. It takes advantage of NVIDIA RTX, combining “on chip” RT Cores in NVIDIA RTX GPUs with AI acceleration enabled by Tensor Cores. NVIDIA Omniverse RTX enables real-time, physically-based rendering and simulation of light transport in 3D environments, using flexible physically-based lighting and materials to model cameras, lidar, radar and related sensor outputs. ovstage ovstage is the shared runtime stage for Omniverse libraries. It provides C and Python APIs for reading, writing, querying, and managing OpenUSD scene data such as transforms, materials, hierarchy, metadata, and simulation state across CPU and GPU memory. In a multilibrary workflow, ovstage allows ovphysx to write updated simulation states and ovrtx to read changed scene data for sensor simulation—without each library maintaining a separate scene representation. OpenUSD NVIDIA has adopted OpenUSD as the foundational data layer for NVIDIA Omniverse and simulation workflows. OpenUSD defines a standard data representation for 3D scenes and provides C++ and Python APIs to read and write those scenes. Data in OpenUSD is organized into a scene graph that ovrtx can render. USD stages can define and handle complex geometry, photometric lights, OpenPBR materials, cameras, and other sensors. RenderProducts and RenderVars RenderProducts and RenderVars provide a USD-native way to define the outputs to obtain from a renderer or sensor simulator.They provide a lot of flexibility to specify output types, resolutions, formats and other parameters. A RenderProduct is associated with one or more sensors, and ovrtx generates the outputs defined by the RenderVars associated with it. RenderProducts and RenderVars can be defined programmatically and stored in the USD scene or defined at runtime using the ovrtx C or Python APIs. Application flow Think of ovrtx as an on-demand service your app calls when it needs to visualize an OpenUSD scene or generate render and sensor outputs. The first steps, creating a renderer, loading OpenUSD content, and configuring RenderProducts and RenderVars, usually happen once or infrequently. The final steps, stepping rendering or sensor simulation and reading outputs, typically run inside the application loop. Create and configure an ovrtx renderer Your existing app, whether Python, C, or C++, starts by creating a renderer instance and setting basic options it needs. Initialize ovrtx and construct a renderer object, pointing it to the correct GPUs and any global configuration options. If you have multiple views or sensors, you still have a single renderer instance coordinating them. Multiple RenderProducts will be attached later. renderer = ovrtx.Renderer() Load or reference OpenUSD content Next, the renderer needs to know which USD stage to draw from. Point the renderer at an OpenUSD file or URL with open_usd , load USDA directly from a string with open_usd_from_string, or compose additional referenced content with add_usd_reference . If your app already has an Omniverse or USD pipeline, ovrtx can read that OpenUSD content directly instead of requiring the app to rebuild the scene in another format. renderer.open_usd("warehouse_scene.usd") # or compose additional content into the stage: # renderer.add_usd_reference("extra_props.usd", prefix="/World/Props") Configure render and sensor outputs With a renderer and stage in place, the app displays which outputs it needs. Define RenderProducts Example: An RGB camera on a robot, a top‑down surveillance camera, a spinning lidar sensor on a vehicle. Each RenderProduct is tied to a camera or sensor prim path in the USD stage and a resolution. Attach RenderVars. For each RenderProduct, specify which outputs ovrtx should generate. Examples include: Color, such as LdrColor or HdrColor Depth or distance Surface normals Semantic segmentation Lidar point clouds Radar detections Other sensor‑specific tensors Step rendering or sensor simulation from the application loop The app calls ovrtx from its main loop, which may already manage physics, animation, user input, networking, or other simulation state. After the app updates the OpenUSD stage, it advances rendering or sensor simulation by passing a time increment. ovrtx then generates the requested outputs for the configured RenderProducts and RenderVars. Returning the output to the application For each generated frame, the app reads the requested outputs as tensors, either on CPU or GPU memory. It consumes these in other parts of the application logic, as needed. Getting GPU and CPU views of the data: ovrtx exposes each RenderVar as a tensor handle. Wrap it using DLPack, NumPy, or PyTorch utilities. To stay on GPU (to feed a model, for example), keep the buffer on device and pass it directly to PyTorch or Warp without copies. To save images or point clouds, request these outputs on CPU memory, and use other logic to write PNGs, NumPy arrays, and so on. op = renderer.step_async( render_products={"/Render/Camera"}, delta_time=1.0 / 60, ) pending = op.wait() products = pending.fetch() for _name, product in products.items(): for frame in product.frames: var = frame.render_vars["LdrColor"].map( device=ovrtx.Device.CPU ) When to use ovrtx with other Omniverse libraries ovrtx can be used on its own or alongside other NVIDIA Omniverse libraries, depending on what the host application needs to do. Start with ovrtx when the application needs RTX sensor simulation, viewport output, or visualization. Add other libraries when the workflow also needs physics, streaming, storage, or another application service. Application need Use Function RTX sensor simulation, viewport output, or visualization ovrtx Generates rendered frames and sensor outputs from OpenUSD content Physics simulation with rendered or sensor output ovrtx + ovphysx Adds physics behavior that can be visualized or consumed in the app workflow Streaming rendered output to a remote or browser-based experience ovrtx + ovstream Adds a streaming path for interactive viewing or review Managing OpenUSD content across storage workflows ovrtx + ovstorage Adds a path for reading, writing, or syncing OpenUSD scene data Sharing OpenUSD scene state across rendering, physics, sensors, and application code ovstage + ovrtx / ovphysx Provides a shared runtime stage so libraries can exchange scene data, simulation updates, and changed prim state Table 1. Common ways to use ovrtx with other Omniverse libraries based on the application capability developers need to add Use this table as a starting point for deciding what the host application needs. Start with ovrtx when the workflow needs rendered frames, viewport output, or simulated sensor outputs from OpenUSD content. Add ovphysx, ovstream, or ovstorage only when the same workflow also needs physics behavior, interactive streaming, or OpenUSD content management. The following examples show how those choices map to practical integration patterns, from a minimal ovrtx render to interactive viewing and multi-library workflows. Examples of ovrtx integration in practice Start with the NVIDIA-Omniverse/ovrtx GitHub repo for direct SDK examples, setup guidance, and API-focused skills. For agent guidance, reference NVIDIA Agent Skills . To see how Omniverse libraries fit together in early workflows, explore the NVIDIA-Omniverse/omniverse-labs GitHub repo, which includes experimental samples, integration experiments, reference patterns, and proof-of-concept apps from the NVIDIA Omniverse team. Example: Minimal “hello world” Start with the minimal ovrtx example in the ovrtx repository. This “hello world” example is a simple smoke test for validating the direct SDK path before adding interaction, sensor outputs, or a larger application shell. V ideo 1. Learn how to run the minimal example, connect Python, C, and C++ applications to Omniverse RTX rendering processes, and generate sensor outputs for robotics and synthetic data workflows The example shows how to create a renderer, load an OpenUSD scene, render a single image, and copy the results back to the CPU for writing as a PNG. Pass the --png flag and the output is written to _output/render.png , which can be inspected with any image viewer. Without the flag, the example opens the rendered image directly in your default image viewer. Use this for: Adding RTX sensor simulation capabilities to an existing application without building a full Omniverse Kit app. Example: Build an interactive USD viewer Use the Omniverse Realtime Viewer skill when the goal is to build a viewer application around OpenUSD content, especially when the workflow needs browser-streamed access to a GPU-backed renderer. ovrtx remains the rendering layer, while the viewer adds the surrounding app structure: scene loading, camera controls, selection, render settings, UI, and local desktop or browser-streamed delivery. For lower-level C++ viewport integration, the vulkan-interop example shows how to display ovrtx output in a real-time viewport using Vulkan and CUDA. Video 2. Learn how a single natural language prompt scaffolds a working viewer app with a server, frontend, scripts, assets, USD scene files, RenderVar outputs, and an ovrtx / ovstream backend For browser-streamed workflows, a server owns the ovrtx renderer and OpenUSD stage access, then uses ovstream to send rendered frames to a browser UI. For a hands-on path, use the Brev launchable to deploy and test a real-time viewer application built with the ovrtx and ovstream libraries. To learn more, read the agent-assisted integration walkthrough on the NVIDIA-Omniverse/omniverse-labs GitHib repo. Use this for: Developing a streaming USD viewer with RTX rendering, browser-based camera controls, and a GPU-backed deployment path using NVIDIA Omniverse libraries and an AI coding agent. Example: Adding physics with ovphysx After scene visualization, the next step is often making it behave physically, with gravity, collisions, friction, and other simulation behavior. That is where ovphysx comes in. Like ovrtx, ovphysx can be called from an application workflow so the app can simulate behavior and visualize the result in a real-time viewport. Video 3. Learn how to add physical AI capabilities to an existing application using NVIDIA Omniverse libraries, including ovrtx, ovphysx, ovstorage, and MCP-based agent workflows The Omniverse-Labs ovrtx plus ovphysx sample shows this combined workflow in practice. The sample uses a two-process architecture: the viewer process is owned by ovrtx, while a separate ovphysx worker streams pose data back to the application loop. Use this for: Adding physics simulation to a rendered or sensor-aware OpenUSD workflow. Example: SimReady Blender sample workflow The SimReady Blender sample workflow, on display at SIGGRAPH 2026, shows how software makers can use Omniverse libraries, now part of NVIDIA Agent Toolkit, to add agent-ready simulation capabilities—including NVIDIA RTX sensor simulation, physics, and validation—into existing 3D applications while keeping creators in control. This is now openly available as a blueprint for integrating Omniverse libraries into Blender. Video 4. Learn more about bringing agent-ready simulation into existing 3D applications using Omniverse libraries Use this for: Understanding how Omniverse libraries can help software makers add agent-ready simulation capabilities into existing 3D applications while keeping creators and developers in control. Designers and engineers who want to run this kind of agent workflow locally can deploy it on NVIDIA DGX Station using NVIDIA NemoClaw . NemoClaw packages the Nemotron 3 Ultra open frontier model, a secure agent runtime, and Omniverse libraries into a complete local stack—a personal AI workstation with access to Blender and simulation tools. How to integrate ovrtx into cloud-native design workflows PTC Onshape Render Studio team is exploring how ovrtx can fit into cloud-native CAD and PDM workflows, where product data, collaboration tools, rendering needs, and established user experiences already come together. That makes it a useful validation point for showing how NVIDIA Omniverse RTX rendering can fit into an existing application workflow while preserving the architecture, data model, and product experience users already rely on. Figure 2. PTC TypeScript RTX Workbench renders Onshape design content with NVIDIA Omniverse RTX, showing that ovrtx can bring application-controlled, path-traced visualization into cloud-native design workflows This early work uses Onshape content to explore how ovrtx can support cloud-native rendering workflows with more complex product design geometry. ovrtx enables integration into existing systems, so teams can evaluate how it fits into the services and application layers they already use, instead of treating rendering as a separate environment. For the PTC Onshape Render Studio team, ovrtx helps address three practical integration needs: Application-controlled rendering: The host application controls when rendering happens, how outputs are configured, and how results return to the workflow. Cloud-native integration: Rendering capabilities can be exposed through the application or service layer teams already use, including cloud-native service workflows. OpenUSD-based scene exchange: ovrtx renders from OpenUSD scene data, giving teams a practical way to connect product design geometry, visualization, and downstream simulation workflows. For application teams, the takeaway is that ovrtx can make RTX sensor simulation and visualization part of the workflow they already own. The PTC work shows how a lower-level API can support production-oriented exploration inside cloud-native design systems, where rendering must stay connected to product data, collaboration workflows, and the application experience. According to  Brendan Hack, Senior Director of Software Development at PTC Onshape, “ovrtx gives us a lower-level API path for bringing NVIDIA RTX into systems cloud-native design tools while keeping the application architecture and product experience in our control.” Get started with NVIDIA ovrtx NVIDIA Omniverse libraries provide modular building blocks for integrating RTX sensor simulation, physics, storage, and related physical AI capabilities into existing application stacks. With ovrtx, you can add camera, lidar, radar, semantic segmentation, and visual preflight outputs to OpenUSD-based workflows. For teams developing design, simulation, robotics, industrial digital twin, synthetic data generation, or physical AI applications, ovrtx provides a practical path to evaluate RTX sensor simulation inside the tools, pipelines, and application loops they already own. Ready to get started? Explore Omniverse libraries: Visit the NVIDIA-Omniverse GitHub repo to find tools and skills that help agents and developers add RTX sensor simulation, physics, OpenUSD workflows, storage, and validation to existing applications. Run the examples: Review Omniverse Labs for early experimental samples and proof-of-concept projects that show how Omniverse libraries fit together. Use NVIDIA physical AI agent skills: Use agent skills to turn repeatable setup, simulation, and evaluation tasks into workflows that developers can inspect, adapt, and run with an AI coding agent. Run locally on DGX Station : Deploy NemoClaw on DGX Station in just three steps, in under 30 minutes.