NVIDIA Details 5-Step Workflow to Build SimReady Robotics Assets Using Frontier AI Models

NVIDIA Details 5-Step Workflow to Build SimReady Robotics Assets Using Frontier AI Models NVIDIA outlined a structured five-step workflow for preparing CAD models into simulation-ready OpenUSD assets for robotics using NVIDIA Omniverse libraries, SimReady Foundation standards, and frontier AI models like GPT-6 Astra. The process is demonstrated using an ABB Robotics YuMi dual-arm robot in NVIDIA Isaac Sim. SimReady specifications establish OpenUSD standards to ensure assets possess valid collision geometry, mass, joint mechanics, and friction properties for robotics simulation. The five-step pipeline comprises STEP-file conversion, visual appearance matching, physics configuration, SimReady validation, and simulation task execution. Frontier AI models running via Codex CLI interface with Isaac Sim over a remote Python connection to execute Omniverse agent skills and script asset modifications. In physics setup, missing robot properties were resolved by retrieving a public URDF, normalizing STEP-derived volumes to specified totals (38 kg robot, 0.28 kg grippers), and applying approximated friction values. Validation concluded with Isaac Sim pick-and-place tasks, where both YuMi arms completed four cycles manipulating 40 g cubes via physics-driven contact and handled a 10 g Sharpie marker modeled from an image. SimReady Framework and Workflow Setup Preparing CAD assets for robotics simulations requires more than converting geometry into OpenUSD; models must also have valid mass, friction, joints, and collision bounds to avoid simulation failures such as grasp instability. The SimReady Foundation defines profile-based criteria to inspect asset structure, materials, and physics. The demonstrated setup relies on Isaac Sim 6.1 (packaged with simready-foundation-tier-core), Omniverse AI agent skills, and GPT-6 Astra running in Codex CLI. Codex communicates directly with Isaac Sim through a live remote Python server to execute asset-processing skills. Conversion and Appearance Alignment The initial step imports ABB YuMi STEP geometry into Isaac Sim using skills such as omniverse-cad-to-simready and omniverse-cad-to-usd. Next, the agent evaluates manufacturer reference images and videos to adjust visual appearance. Using usd-pipeline and remote Isaac Sim tools, the AI visually estimates and tunes colors, metallic responses, and roughness to align the asset with reference materials while preserving underlying CAD geometry and mounting transforms without using AI-generated textures or measured material reconstruction. Physics Rigging and SimReady Validation To configure physics, the agent rigs rigid bodies, joints, limits, and collision geometry for both arms and grippers using manufacturer datasheets. When data was incomplete, the agent fetched a public URDF to determine zero configurations and joint axes. Per-link masses were derived from geometric volumes and scaled to manufacturer specifications—38 kg for the robot body and 0.28 kg per gripper—assuming uniform part density, 0.8 static friction, and 0.6 dynamic friction. Validation checks verified that all 21 rigid bodies possessed positive masses and admissible inertia tensors, followed by tests on finger mimic actions, drives, and articulation constraints. Simulation Pick-and-Place Demonstration The final stage evaluated the configured robot through dynamic manipulation tasks in Isaac Sim. In a test involving 45 mm, 40 g cubes, both YuMi arms completed four pick-and-place cycles across a 122.2-second simulation, holding cubes for two seconds before releasing them onto matching color targets and into a box using solely friction-based contact. A secondary test evaluated an agent-generated Sharpie marker modeled from a reference image with approximated cylindrical collisions and an estimated 10 g mass, which the YuMi gripper successfully lifted and placed into a tray.

NVIDIA展示利用前沿AI模型创建机器人SimReady资产的五步工作流 NVIDIA介绍了如何结合SimReady Foundation规范、Omniverse工具链及前沿AI模型（以GPT-6 Astra为例），将ABB YuMi机器人的CAD资产高效转化为可用于Isaac Sim物理仿真的SimReady资产，涵盖从几何转换到任务闭环的完整流程。 工作流依托Isaac Sim 6.1与Codex CLI远程Python连接，由AI智能体调用Omniverse技能自动化推进。 包含STEP导入、外观对齐、物理属性配置、SimReady标准验证及仿真任务测试五大步骤。 AI智能体在技术数据不完整时能自主检索公开URDF补全关节轴向，并按几何体积与总重推算质量分布。 在Isaac Sim中成功完成双臂色块抓取放置以及基于单张图像重建记号笔的拾取仿真验证。 CAD导入与外观对准 工作流启动需在Isaac Sim目录中运行Codex CLI并开启远程Python服务，使AI智能体能直接操作仿真应用。第一步通过CAD-to-SimReady技能将ABB YuMi机器人的STEP文件转化为OpenUSD格式并载入环境；第二步依据参考图片和视频调整材质颜色、金属性和粗糙度，在保留原始CAD几何与安装变换的同时完成外观对齐。 物理属性配置与参数推算 针对OpenUSD资产配置刚体、关节及碰撞几何。在厂商参数不全的情况下，AI智能体检索公开URDF确定关节轴与零位配置；各连杆质量基于几何体积估算并按整机38千克、夹爪各0.28千克进行归一化；转动惯量假定为均匀实体计算，接触属性设定静态摩擦0.8与动态摩擦0.6，同时采用凸包简化碰撞几何。 SimReady规范与结构校验 依据SimReady Foundation要求及Isaac Sim内置验证引擎进行规则校验。检查覆盖单位、几何结构、材质绑定、驱动和关节链等维度，确认21个刚体均具备正质量与物理合理的惯性张量，同时验证了底座固定安装与双指联动等运行表现，确保资产结构符合仿真要求。 双臂任务与图像衍生对象仿真 最终阶段进行了两项物理仿真任务。第一项是在122.2秒的连续物理仿真中，双臂夹爪依靠真实接触摩擦力完成了四轮立方体的拾取、平移与入盒操作，未采用任何运动学吸附；第二项是由AI基于参考图像估算尺寸并重建Sharpie记号笔的三维碰撞资产，由机器人夹爪成功将其抓起并放入托盘。

Preparing CAD assets for robotics simulation requires more than converting geometry to OpenUSD : developers must configure and validate materials, collision geometry, joints, and other physics properties before testing robot behavior. NVIDIA Omniverse libraries , guided by SimReady Foundation specifications and agentic NVIDIA skills , provide a structured workflow for converting and validating simulation ready ( SimReady ) assets. Frontier AI models can assist developers at each stage by interpreting reference materials and calling Omniverse tools on their behalf, reducing manual preparation. This post walks through how to prepare an ABB Robotics YuMi robot for simulation using the SimReady Foundation and NVIDIA Omniverse tools, with GPT-6 Astra as one example of a frontier AI model that can assist with the process. The workflow covers five steps: STEP-file conversion, appearance validation, physics configuration, SimReady validation, and a final pick-and-place task with both arms and grippers in NVIDIA Isaac Sim . What Is SimReady and Why Does It Matter for Simulation? SimReady defines requirements for preparing OpenUSD assets for specific simulation use cases. Developers select a SimReady profile for their intended use and validate the asset against its requirements. Profile-specific checks validate the asset’s structure, materials, and physics properties, then flag issues for developers to fix and recheck. A robot model can look correct but still fail in simulation if it is missing simulation properties. For example, missing collision geometry can allow objects to pass through its gripper and incorrect joints can prevent coordinated motion. Simulation properties like mass and friction directly affect whether a grasp remains stable. The following technologies and tools support this workflow: OpenUSD represents geometry, assembly structure, materials, and simulation properties in a common asset representation. SimReady Foundation provides specifications and validation guidance for preparing simulation-ready assets. NVIDIA Omniverse libraries provide capabilities that developers and agents can use to convert, inspect, and prepare OpenUSD content. NVIDIA Isaac Sim provides the environment for configuring physics and testing robot behavior. NVIDIA Omniverse AI Agent skills (such as CAD-to-SimReady skill ) guide agents through asset conversion, material and physics assignment, and validation. SimReady Robotics Workflow: An ABB YuMi Robot Example This walkthrough demonstrates the five-step SimReady workflow using an ABB Yumi robot . GPT-6 Astra is used here as one example of a frontier AI model that can help write Python code to call NVIDIA Omniverse libraries at each stage, but the workflow can be applied across many models and robot systems. The example uses both arms and grippers, configured during the workflow. Cubes and a Sharpie marker were used as demonstrations in step 5. Camera-based perception can be added in a separate workflow. Before you begin: Install Isaac Sim Create a folder inside the Isaac Sim directory and name it ‘reference’, we will use this reference folder to hold the relevant assets for the workflow. Add the robot specifications , STEP files, and reference images and videos in this folder (details below in the table) Install your frontier AI model In this workflow example, we used GPT-6 Astra . Launch Codex CLI in your Isaac Sim directory It is important to launch Codex inside the Isaac Sim directory so that the agent can find the necessary skills which will be needed in the workflow. In this workflow example, we used Isaac Sim 6.1 and Codex CLI and launched Isaac Sim in windowed mode. Use the following example prompt in your active Codex CLI to start Isaac Sim: Start Isaac Sim in windowed mode with the remote Python server enabled. Use a fresh terminal for this. The remote Python server in Isaac Sim enables your agent to ‘talk’ via Python to your live Isaac Sim application. Confirm that the connection works before continuing. At each step, adapt the sample prompt to your task in your frontier AI agent, then review simulation results and validation feedback to identify issues and refine the asset. 1. Import the robot from the STEP files As the first step in this workflow, we will import the robot from the STEP files in the reference folder. For this example, provide the following inputs in the reference folder within the Isaac Sim directory: Sample Input Purpose ABB Robotics YuMi robot STEP files . Source geometry for conversion to OpenUSD. Reference images and videos from ABB Robotics.  This sample workflow uses the linked ABB Robotics video and image . Guide the asset’s appearance and material assignments. SimReady Foundation specification repository Note: Isaac Sim 6.1 is also packaged with simready-foundation-tier-core Supply robot requirements, related skills, and validation guidance. CAD-to-SimReady skill Provide guidance for CAD conversion, material and physics assignment, and SimReady validation. Table 1. Sample inputs and their purposes for preparing the YuMi robot for simulation Note: Isaac Sim 6.1 is packaged with simready-foundation-tier-core. Ask the agent to to use the supplied inputs to guide conversion to OpenUSD and import into Isaac Sim. Sample prompt: Figure 1. YuMi robot and grippers imported into Isaac Sim after STEP-to-OpenUSD conversion using CAD- to-SimReady Skill NVIDIA Skills used: omniverse-cad-to-simready, omniverse-cad-to-usd, and isaac-sim-remote Check before continuing: The robot asset is imported and available for inspection. The task and supplied requirements are understood, and missing information is identified. 2. Match the visual appearance The next step is to inspect the imported robot to confirm that expected materials and textures are present and loaded correctly. Compare the asset with the supplied ABB images and videos to check scale, orientation, and part placement. Ask the agent to add or refine materials and textures using these references. Sample prompt: Use the ABB images and videos in the reference folder to match the imported robot’s appearance to the physical robot. Add or adjust materials and textures, and report any visual discrepancies or missing reference information. Figure 2. YuMi asset prepared with Astra (left) alongside the ABB Robotics reference image (right) Assumptions: The agent visually matches the robot’s materials by adjusting colors, metallic response, and roughness while preserving the CAD geometry and mounting transforms. The material settings were visual estimates, without measured material reconstruction or AI-generated textures. NVIDIA Skills used: isaac-sim-remote, usd-pipeline Check before continuing: The robot has the expected scale, orientation, parts, and appearance. Any missing materials, textures, or other visual discrepancies are documented. 3. Configure physics properties At this stage, the OpenUSD asset contains the robot’s visual geometry, but joints, rigid bodies, and collision geometry have not yet been configured. Ask the agent to configure and validate the physics properties for both arms and grippers. The datasheet we have in the reference folder will be used by the agent to make sure the joint ranges and speed limits match the simulated robot. If manufacturer data is incomplete, you can ask the agent to identify missing parameters and check technical manuals or available URDF robot descriptions for the exact model. Where mass properties remain unavailable, estimate them from geometry using documented density or mass-distribution assumptions, following NVIDIA’s mass-property guidance . For example, when not given enough context, we observed the agent pulling a public URDF for the robot and used that to identify the correct axes for joints and the zero configuration. Sample prompt: Rig the physics and validate the physics properties for both YuMi arms and grippers. Check joint definitions, axes and limits, mass and inertia, collision geometry, base mounting, and gripper motion. Figure 3. YuMi physics configuration with collision geometry and joint axes during step 3 The initial pose may vary. Figure 4. YuMi poses after physics configuration Simulation assumptions in this workflow: Joint configuration: The agent retrieved a public URDF for the robot and used it to identify joint axes and the zero configuration. Per-link masses: Estimated from STEP geometry volumes and normalized to manufacturer-specified totals: 38 kg for the robot and 0.28 kg for each gripper. Centers of mass and inertia: Calculated assuming solid parts of uniform density; internal motors, gearboxes, and wiring were not modeled individually. Contact properties: Assumed static friction of 0.8, dynamic friction of 0.6, and zero restitution. Collision geometry and appearance: Convex collision shapes, surface roughness, and material response were approximated. Calibration: These parameters were not calibrated against physical YuMi measurements. NVIDIA Skills used: usd-articulation, physics-simulation, isaac-sim-validator, isaac-sim-remote Check before continuing: Joints move around the expected axes and within their limits. Base mounting and gripper motion behave as configured. Estimated properties and any validation failures are documented. 4. Validate the SimReady properties The next step is to validate the robot’s OpenUSD asset against the SimReady requirements for the intended simulation task. Ask the agent to run the selected SimReady Foundation checks, review the findings, and revise the asset as needed. Sample prompt: Validation results in this workflow: SimReady Foundation: The YuMi asset’s units, asset dependencies, geometry, materials, rigid bodies, joints, drives, and articulation were checked against SimReady Foundation requirements. Isaac Sim asset checks: Verified scale, orientation, part placement, gripper mounting, material coverage, and resolved texture dependencies via validation engine inside Isaac Sim. Mass and inertia: All 21 rigid bodies had positive masses and physically admissible inertia tensors; runtime values matched authored values. Joints: Checked axes, frames, limits, drives, finger mimic behavior, and fixed-base mounting. Runtime behavior: Tested arm and gripper motion, contact, release, and collisions along the tested trajectories. These checks validated the simulation implementation. They did not establish agreement with real-world robot dynamics or collision safety across every possible pose. Figure 5. YuMi asset in Isaac Sim after visual, physics, and SimReady validation steps Check before continuing: The sequence meets the agreed acceptance criteria and the selected SimReady checks pass. Tuning changes, estimated properties, and any unresolved failures are documented. 5. Final task for robotic simulation Run a final validation in Isaac Sim, combining the visual, physics, and SimReady checks with a pick-and-place task. Example 1: Cube pick-and-place Ask the agent to add color targets and a box to the test scene and define the target locations and color metadata, matching color targets are selected using scene metadata. Sample prompt: Use both arms and grippers to pick and place cubes. Add color targets and a box to the test scene, and define their locations and the targets’ color metadata. Run the complete grasp, lift, hold, transfer, and release sequence. Use scene metadata to select the matching color target for each cube. Also test placement into the box. Check scale, orientation, part placement, materials, textures, joint axes and limits, collision geometry, and gripper behavior. Run the selected SimReady Foundation validation checks. Simulation assumptions: We assumed dimensionless contact-friction coefficients of 0.8 static and 0.6 for dynamic. The grasp tests passed with these settings, but the coefficients were not measured or calibrated against the physical finger–object contact. This example uses 45 mm, 40 g test cubes and the simulation assumptions described in Step 3. Demonstrated pick-and-place results: Completed cycles: Both arms and grippers completed four pick-and-place cycles in total during a 122.2-second physics simulation. Task sequence: Each arm grasped its cube, lifted it, held it for two seconds, transferred it, and released it onto its matching color target. Each arm then picked up its cube again and placed it inside the box. Contact behavior: Finger contact and friction carried the cubes, without attachment joints, kinematic holds, or direct cube-pose updates during simulation. Acceptance criteria: All cycles passed the defined grasp, hold, transfer, release, and placement criteria. Recording: The MP4 and GIF were generated from this simulation. Figure 6. YuMi pick-and-place simulations with colored cubes and target markers Example 2: Pick-and-place with an image-derived object A separate demonstration used a Sharpie marker created from a reference image to have the agent create a 3D asset which the robot will pick up. Simulation assumptions: Agent estimated the dimensions of the marker as 140mm and a 10 gram mass with the center of mass at the geometrical center of the Sharpie. Collision geometry was approximated with cylinder geometry for the cap, barrel and rear plug. Demonstrated results: YuMi’s gripper lifted the marker from the tabletop and released it into a blue tray in Isaac Sim. Figure 7. YuMi picking and placing a Sharpie marker modeled from a reference image in Isaac Sim Successful simulation tests demonstrate behavior under the tested conditions. Results may vary depending on the frontier AI model and prompt used. NVIDIA Skills used: manipulation-ik, physics-simulation, isaac-sim-validator, isaac-sim-remote, isaac-sim-rendering Try the Workflow SimReady Foundation and NVIDIA Omniverse tools help developers turn CAD files and reference materials into assets they can configure, validate, and test in simulation. Use OpenUSD, SimReady Foundation, and NVIDIA Omniverse tools and skills to prepare assets for your robotics simulation tasks. The YuMi walkthrough provides one example of the SimReady robotics workflow using GPT-6 Astra as the frontier AI model. To adapt the approach to another robot or frontier AI model, confirm access to the required tools and skills, supply the relevant reference materials, and repeat the asset and task validation. Results may vary with the model, reasoning effort, and prompts used. Try the ABB Robotics sample workflow Start with the SimReady Foundation repository and CAD-to-SimReady skill to guide asset preparation. The ABB Robotics YuMi robot STEP file, reference video and image are available to follow along with this walkthrough. Learn More Explore robotics simulations with Isaac Sim and Isaac Lab . Learn to assemble OpenUSD scenes through the free OpenUSD learning path . Learn robotics and digital twin skills through the free physical AI courses . Join us at NVIDIA GTC Berlin on October 20–22, 2026, to explore the latest developments in physical AI and robotics and connect with developers, researchers, and industry leaders.