NVIDIA Introduces Alpamayo 2 Super Reasoning VLA Model for Autonomous Driving

NVIDIA Introduces Alpamayo 2 Super Reasoning VLA Model for Autonomous Driving NVIDIA has released Alpamayo 2 Super, an open 34-billion-parameter reasoning vision-language-action (VLA) model designed to unify trajectory planning, scene reasoning, visual question answering, and auto-labeling for autonomous vehicle development. Alpamayo 2 Super merges the 32B Cosmos 3 Super Reasoner with a 2B diffusion-based Action Expert, post-trained via reinforcement learning. The model offers 360-degree surround-view perception across up to seven cameras and outputs future trajectories alongside Chain-of-Causation (CoC) reasoning traces. It ranks first on the LingoQA benchmark with a 79.2 score and achieves a 0.911 m minADE_6 on the Physical AI AV Dataset. Released under the OpenMDW-1.1 license, the model supports fine-tuning, commercial redistribution, and distillation into embedded targets like DRIVE AGX Thor. Architecture and Core Workflows AV development frequently relies on disconnected models for trajectory generation, intent prediction, and scene understanding. Alpamayo 2 Super unifies these stages under a single 34-billion-parameter foundation model. Its 32B reasoner processes multi-camera feeds, language, and historical motion, while a 2B diffusion-based Action Expert maps internal states into ego-vehicle trajectories. The system handles four primary tasks: jointly generating trajectories with Chain-of-Causation (CoC) traces, predicting high-level meta-actions, conducting visual question answering (VQA) with 2D spatial grounding, and generating structured auto-labels. Open-Loop and Closed-Loop Performance Across 1,434 challenging clips from the Physical AI AV Dataset, the model achieved a 6.4-second minADE_6 of 0.911 m and a score of 0.433 on the Physical AI AV Reasoning Benchmark. In LingoQA, it scored 79.2, leading competing models such as Qwen3-VL 32B, Gemini 2.5 Pro, and GPT-4o. To counter the limitations of static open-loop testing, NVIDIA tested the policy in the reactive AlpaSim simulator across 913 reconstructed scenes, where it attained an AlpaSim Score of 1.50 ± 0.13, revealing interactions such as collisions and near-misses over time. Scene Understanding and Auto-Labeling In meta-action classification across 94K clips, the model recorded IoU scores of 74.59 lateral, 61.91 longitudinal, and 73.55 lane-wise. For surround-view visual question answering across 8K pairs, Alpamayo 2 Super achieved an answer similarity score of 0.652 and a 2D bounding-box IoU of 0.71, outperforming Qwen3-VL 32B. Additionally, the system serves as an offline data engine, generating structured CoC auto-labels with or without pre-existing future trajectories, matching expert human annotations at a 0.652 similarity rating.

NVIDIA 推出开源 34B 自动驾驶模型 Alpamayo 2 Super NVIDIA 发布了开源的 340 亿参数推理视觉-语言-动作（VLA）模型 Alpamayo 2 Super。该模型结合了 320 亿参数的 Cosmos 3 Super 推理器与 20 亿参数基于扩散的动作专家，具备 360 度多相机感知能力，旨在统一轨迹规划、元动作预测、场景理解及自动标注工作流。 Alpamayo 2 Super 采用 32B 推理器搭配 2B 扩散动作专家架构，经强化学习后训练，支持最多 7 个相机的全景输入。 模型权重与推理代码已开源，遵循 Linux 基金会 OpenMDW-1.1 许可，允许商业再分发与蒸馏部署，输出无许可限制。 在开环基准 LingoQA 中取得 79.2 分排名第一，并在闭环仿真平台 AlpaSim 中取得 1.50 ± 0.13 的评分。 支持输出因果链（CoC）推理轨迹、高层元动作、带 2D 空间定位的场景问答及结构化因果自动标注。 模型架构与开源许可 针对自动驾驶研发中轨迹生成、意图预测、场景理解和数据标注模型相互割裂的问题，Alpamayo 2 Super 提供了统一的基础模型。它由负责解析多相机视频、语言和历史轨迹的 32B Cosmos 3 Super 推理器，以及将内部表征转化为本车轨迹的 2B 扩散动作专家组成。模型基于 Linux 基金会 OpenMDW-1.1 协议开源，涵盖微调、衍生模型开发与商业再分发，蒸馏模型可在车载 NVIDIA DRIVE AGX Thor 上部署，且模型输出不附加任何许可条件。 轨迹生成与开环/闭环评测 模型能够联合生成未来轨迹与因果链（CoC）推理轨迹，解释决策原因并支持安全验证。在开环评测中，针对 Physical AI AV 数据集的 1,434 个长尾困难样本，其 6.4 秒 minADE_6 为 0.911 米；在 Physical AI AV Reasoning 基准中得分 0.433；在 LingoQA 基准测试中取得 79.2 分，居 37 个参评模型首位，优于 Qwen2.5-VL、Qwen3-VL、Gemini 2.5 Pro 及 GPT-4o。为弥补开环评估无法反应动作后动态变化的缺陷，模型在 NVIDIA AlpaSim 仿真平台的 913 个重建场景闭环评测中获得了 1.50 ± 0.13 的 AlpaSim 评分。 元动作预测与高层意图对齐 除轨迹外，模型还可输出让行、变道、停车或加速等高层元动作，便于下游规划器使用、意图几何一致性检查以及长尾数据检索。在包含 94,000 个片段的内部验证集上，Alpamayo 2 Super 在横向、纵向和车道维度的分类交并比（IoU）分别达到 74.59、61.91 和 73.55。 场景问答与 2D 空间定位 通过视觉问答（VQA），开发者可用自然语言查询全景场景中的关键交通要素及决策逻辑。该模型不仅生成文字回答，还能在相关相机画面中预测 2D 边界框实现空间定位（2D Grounding）。在 8,000 个问答对的内部测试中，其回答相似度得分为 0.652（Qwen3-VL 32B 为 0.450），2D 定位 IoU 达到 0.71（Qwen3-VL 32B 为 0.17）。 因果链自动数据标注 长尾场景的人工标注耗时昂贵，Alpamayo 2 Super 可作为离线自动标注引擎，按关键帧输出结构化 CoC 标注。若数据缺少真实的未来运动信息，模型还可先自主预测未来轨迹再辅助标注。在 8,000 个内部片段与专家标注的对比评估中，该模型取得了 0.652 的相似度得分，优于 Qwen3-VL 32B 的 0.450，显著压缩数据标注周期。

Autonomous vehicle (AV) development often relies on separate models for trajectory generation, high-level intent prediction, scene understanding, and data labeling. This separation makes it hard to compare related outputs, investigate model behavior, and reuse the same representations across the development workflow. NVIDIA Alpamayo 2 Super is an open 34-billion-parameter reasoning vision-language-action (VLA) model designed to accelerate autonomous vehicle (AV)  development. It combines the 32-billion-parameter NVIDIA Cosmos 3 Super Reasoner with a 2-billion-parameter diffusion-based Action Expert and is post-trained with reinforcement learning. The reasoner interprets multi-camera video, language context, and prior motion history, while the Action Expert converts the model’s resulting internal representation into a future ego-vehicle trajectory. Alpamayo 2 Super’s perception expands to 360-degree coverage across up to seven cameras and can return multiple complementary outputs: future trajectories, Chain-of-Causation (CoC) reasoning traces, high-level meta-actions, grounded answers to questions about the scene, and reasoning auto-labels. This multi-task design gives AV developers a common foundation across multiple stages of the development workflow. The same foundation model can be used as an offline policy teacher, an evaluation critic, a data engine, or a starting point for new task customization, instead of maintaining a separate model for each stage of the workflow. This post provides a hands-on introduction to four Alpamayo 2 Super-enabled workflows: Generate trajectories and CoC reasoning traces, evaluating the results in open-loop and closed-loop benchmarks. Predict meta-actions such as yield, change lanes, and stop alongside a trajectory. Ask natural-language questions about a multi-camera driving scene. Generate CoC auto-labels with 2D grounding on your own clips. The model weights are available on Hugging Face and the inference notebooks on GitHub . The model is released under OpenMDW-1.1, the Linux Foundation permissive license for open model distributions, which covers fine-tuning, derivative models, and commercial redistribution. Distilled models can be deployed commercially without further permission from NVIDIA, and model outputs carry no license conditions. Figure 1. Alpamayo 2 Super processes multimodal inputs through a 32B Cosmos 3 Super Reasoner and a 2B Action Expert Planning and reasoning Reasoning through new scenarios is a fundamental problem in autonomous driving. Navigating construction zones, partially occluded pedestrians, unusual right-of-way interactions, and objects entering the roadway requires more than matching a common trajectory pattern. A useful driving model must identify scene context that matters, connect it to the appropriate driving decision, and produce an action consistent with that decision. Trajectories and CoC traces: What and why Alpamayo 2 Super, like its predecessors, jointly produces output trajectories and CoC reasoning traces. The trajectory expresses what the ego vehicle could do next. The CoC reasoning trace provides insights into why a driving decision was made from observed scene context. Returning both outputs makes it easier to understand the model’s decision-making, curate difficult cases, compare a deployed policy with a larger teacher, and diagnose whether a failure originated in perception, reasoning, or action generation. CoC traces also feed into the NVIDIA Halos safety validation workflows by enabling introspection into the model’s understanding of the scene. The Alpamayo 2 Super repository’s inference notebook loads a surround-view clip, prepares ego-motion history, and samples a trajectory with its associated CoC trace. The core inference step and outputs are shown below. from alpamayo2_super import helper from alpamayo2_super.load_physical_aiavdataset import load_physical_aiavdataset from alpamayo2_super.models.alpamayo2_super import Alpamayo2Super from alpamayo2_super.visualization import plot_inference_result data = load_physical_aiavdataset( "030c760c-ae38-49aa-9ad8-f5650a545d26", t0_us=2000000, ) model = Alpamayo2Super.from_pretrained("nvidia/Alpamayo2-Super", dtype=torch.bfloat16, device_map="cuda:0") model_inputs = helper.prepare_model_inputs(data, model.config, model.tokenizer) model_inputs = helper.to_device(model_inputs, "cuda") torch.cuda.manual_seed_all(42) with torch.autocast("cuda", dtype=torch.bfloat16): pred_xyz, pred_rot, logprob, extra = model.sample_trajectories_from_data( data=model_inputs, top_p=0.98, temperature=0.6, num_traj_samples=1, diffusion_kwargs={"inference_step": 10}, return_extra=True, ) fig, metadata = plot_inference_result( data=data, pred_xyz=pred_xyz, extra=extra, ) Figure 2. Alpamayo 2 Super planning in a challenging scene Evaluation methods To evaluate the model’s output reasoning and trajectory quality, we can use open-loop and closed-loop evaluation methods. Open-loop evaluation measures trajectory and reasoning quality on recorded scenes by comparing them to ground-truth labels. Alpamayo 2 Super achieves the following results: Trajectory prediction: Across 1,434 challenging samples from the Physical AI AV Dataset , it records a 6.4-second minADE_6 of 0.911 m. AV reasoning: It scores 0.433 on the Physical AI AV Reasoning Benchmark . LingoQA: Alpamayo 2 Super achieves 79.2 on the LingoQA benchmark , ranking first among 37 evaluated models. With 34 billion parameters, it leads Qwen2.5-VL (72B) by 17.0 points, Qwen3-VL (32B) by 7.0 points, Gemini 2.5 Pro by 15.1 points, and GPT-4o by 23.2 points. Lower minADE_6 values indicate better trajectory predictions; higher reasoning scores indicate better performance. The main challenge with open-loop metrics, however, is that they evaluate predictions against a fixed, prerecorded future and therefore don’t capture what happens after the model’s first action, which may affect the rest of the scene. For example, if the ego vehicle changes lanes, an open-loop replay may continue moving an adjacent vehicle along its recorded trajectory rather than accounting for how it would respond to the ego vehicle. Closed-loop simulation executes each predicted action within the scene and, when the simulator includes reactive behavior models, captures how surrounding agents may react. NVIDIA AlpaSim enables this by repeatedly rendering observations, querying the policy, and applying its actions so developers can measure these closed-loop effects over time. To run Alpamayo 2 Super on an AlpaSim evaluation suite, use the corresponding shell command: uv run alpasim_wizard deploy=local topology=2gpu driver=alpamayo2 wizard.log_dir=$PWD/tutorial eval.video.video_layouts=[REASONING_OVERLAY] with the following AlpaSim wizard configuration: # Should be used in defaults list, e.g. # - /driver: alpamayo2 # Type validation happens at driver runtime via OmegaConf.structured merge defaults: - alpamayo_configs # Camera and simulation configs for 4-cam 10Hz - _self_ # YAML values override schema defaults # Alpamayo 2 Super Driver Configuration for Alpasim # Logging level (uses wizard's global setting) log_level: ${wizard.log_level} # Model configuration model: model_type: alpamayo2 # Entry-point name in alpasim.models registry # HuggingFace model ID (requires cached download or hf authentication in the driver container) checkpoint_path: "nvidia/Alpamayo2-Super" # # Alternative local path to a pre-downloaded model # checkpoint_path: "/mnt/drivers/alpamayo2/Alpamayo2-Super" device: "cuda" # Enable classifier-free guidance navigation sampling (NOTE: this requires 2 GPUs with at least 70 GB VRAM). # Set to true only when sufficient GPU memory is available. use_classifier_free_guidance_nav: false # Server configuration host: "0.0.0.0" port: ??? # Inference configuration inference: use_cameras: - camera_cross_left_120fov - camera_front_wide_120fov - camera_cross_right_120fov - camera_front_tele_30fov max_batch_size: 1 # A2Super is memory intensive, start with batch size 1 subsample_factor: 1 context_length: 4 # A2Super uses 4 temporal frames per camera # Route configuration — A2Super uses language-only navigation, not waypoint commands route: use_waypoint_commands: false # Output configuration output_dir: "/mnt/output/driver" # Trajectory optimization (disabled by default for A2Super) trajectory_optimizer: enabled: false plot_debug_images: false Video 1. Alpamayo 2 Super driving closed-loop within AlpaSim, navigating heavy traffic next to a construction zone On 913 reconstructed scenes, Alpamayo 2 Super obtains an AlpaSim Score of 1.50 ± 0.13. This closed-loop score complements open-loop results by revealing collisions, road departures, close encounters, and other failures that can emerge only after the policy influences future observations. Evaluation Benchmark (set) Metric Alpamayo 2 Super Reference scores Better Open-loop Physical AI AV Dataset (1,434 challenging samples) minADE_6 @ 6.4 s (m) 0.911m Alpamayo 1.5 Nano: 0.916m Lower Open-loop Physical AI AV Reasoning Benchmark Reasoning score 0.433 Alpamayo 1.5 Nano: 0.414 GPT-5.5: 0.502 Higher Open-loop LingoQA Lingo-Judge Score 79.2 Alpamayo 1.5 Nano: 74.2 Qwen3-VL 32B: 72.2Qwen2.5-VL 72B: 62.2 Gemini 2.5 Pro: 64.1GPT-4o: 56.0 Higher Closed-loop AlpaSim (913 reconstructed scenes from the Physical AI AV NuRec dataset) AlpaSim Score 1.50 ± 0.13 Alpamayo 1.5 Nano: 1.37 ± 0.10 Higher Table 1. Open-loop and closed-loop evaluation of Alpamayo 2 Super on long-tail driving scenarios A trajectory is precise, but it doesn’t always provide a compact description of intent. Meta-actions summarize a plan in terms of high-level decisions such as yield, change lanes, stop , or accelerate . These outputs can help bridge an end-to-end foundation model and a modular AV stack: a downstream planner can consume the decision, an evaluator can check whether trajectory geometry agrees with intent, and teams can search their data corpus for particular maneuvers. Generate meta-actions The meta-actions notebook shows how to produce meta-action outputs with an example scene. For a detailed list of supported meta-actions, refer to these lists . The core inference step and outputs are shown below. from alpamayo2_super import helper from alpamayo2_super.load_physical_aiavdataset import load_physical_aiavdataset from alpamayo2_super.models.alpamayo2_super import Alpamayo2Super from alpamayo2_super.text_tasks import generate_text, prepare_text_generation_inputs data = load_physical_aiavdataset( "030c760c-ae38-49aa-9ad8-f5650a545d26", t0_us=2000000, ) model = Alpamayo2Super.from_pretrained("nvidia/Alpamayo2-Super", dtype=torch.bfloat16, device_map="cuda:0") task_inputs = prepare_text_generation_inputs( data=data, model_config=model.config, tokenizer=model.tokenizer, task="meta_action", ) task_inputs = helper.to_device(task_inputs, "cuda") torch.cuda.manual_seed_all(42) with torch.autocast("cuda", dtype=torch.bfloat16): result = generate_text( model, task_inputs, top_p=0.98, temperature=0.6, max_new_tokens=512, ) cot = result["cot"][0] meta_action = result["meta_action"][0] print("Chain-of-Causation:\n", cot) print("\nMeta-action:\n", meta_action) Evaluate meta-action accuracy To evaluate meta-action accuracy, we compare Alpamayo 2 Super’s output with ground-truth labels and report the resulting classification accuracy in terms of intersection-over-union (IoU) for the three components of its meta-action taxonomy: lateral, longitudinal, and lane-wise. On an internal set of 94K clips with ground-truth meta-action data, Alpamayo 2 Super achieves 74.59 lateral IoU, 61.91 longitudinal IoU, and 73.55 lane-wise IoU across its meta-action taxonomy. Figure 3. An output showing a meta-action for the corresponding scenario Scene understanding Planning is only one way to use a driving foundation model. Visual question answering (VQA) exposes the model’s scene understanding directly through natural language. Developers can ask about key elements in the scene, how they affect driving behavior, why the ego vehicle should slow, or about other aspects of scenarios. ne-change gap. VQA is valuable for interactive debugging and data operations. It can help a developer inspect why a policy behaved a certain way, build semantic filters over large clip collections, and generate candidate annotations for human review. With surround-view input, questions can reference side and rear context that a front-view-only model would miss. Querying multi-camera scenes with VQA and 2D grounding For each clip, Alpamayo 2 Super can generate answers and spatially localize referenced actors by predicting 2D bounding boxes in the relevant camera frames. This grounding makes outputs more useful than free-form text alone. Reviewers can verify which specific object the model is referring to, automated checks can flag missing or inconsistent boxes, and downstream models can leverage (e.g., through distillation) these tighter links between visual evidence, reasoning, and action. The scene understanding and VQA notebook shows how to perform VQA with the same inputs. The core inference step and outputs are shown below. from alpamayo2_super import helper from alpamayo2_super.load_physical_aiavdataset import load_physical_aiavdataset from alpamayo2_super.models.alpamayo2_super import Alpamayo2Super from alpamayo2_super.text_tasks import generate_text, prepare_text_generation_inputs data = load_physical_aiavdataset( "ea4a6729-bf33-4997-905b-cd58774a3580", t0_us=7500000, ) model = Alpamayo2Super.from_pretrained("nvidia/Alpamayo2-Super", dtype=torch.bfloat16, device_map="cuda:0") task_inputs = prepare_vqa_inputs( data=data, model_config=model.config, tokenizer=model.tokenizer, question="Describe the driving scene and identify the key traffic elements that should influence ego behavior.", ) task_inputs = helper.to_device(task_inputs, "cuda") with torch.autocast("cuda", dtype=torch.bfloat16): result = generate_text( model, task_inputs, top_p=1.0, temperature=0.1, max_new_tokens=1024, ) answer = result["answer"][0] print(answer) Figure 4. Alpamayo 2 Super answering a question about the observed scene The model can return two types of output for the same input: Figure 4, above, shows a natural-language description of the scene, while Figure 5, below, shows the same model spatially localizing referenced objects by predicting 2D bounding boxes in the relevant camera frame. Figure 5. An output showing 2D grounding as an answer to a question Evaluate VQA response and grounding quality To evaluate VQA performance, we compare Alpamayo 2 Super’s generated responses with ground truth answers across an internal set of 8K question-answer pairs. The model achieves 0.652 answer similarity (higher is better), compared with Qwen3-VL 32B at 0.450. For 2D grounding, we compare the predicted bounding boxes to ground-truth annotations using IoU and obtain 0.71 compared to 0.17 for Qwen3-VL 32B. Together, these measurements evaluate whether the model both answers questions accurately and associates its responses with the correct visual evidence across the surround-view cameras. Auto-labeling Reasoning models need decision-grounded reasoning data, but labeling long-tail driving clips by hand is expensive and slow. Annotators must inspect temporal and multi-camera context, identify the causal actors, describe how they affect the ego vehicle, and keep the label consistent with the intended maneuver. Alpamayo 2 Super can serve as an offline auto-labeler that proposes this structure at scale. This can compress annotation cycles from months to days. Generate structured CoC auto-labels The CoC auto-labeling notebook accepts clips in the released schema and writes one structured record per selected keyframe. from alpamayo2_super import helper from alpamayo2_super.load_physical_aiavdataset import load_physical_aiavdataset from alpamayo2_super.models.alpamayo2_super import Alpamayo2Super from alpamayo2_super.text_tasks import generate_text, prepare_text_generation_inputs data = load_physical_aiavdataset( "b5f3756c-4f0e-4298-a1ff-cc92ed392ae0", t0_us=11000000, ) model = Alpamayo2Super.from_pretrained("nvidia/Alpamayo2-Super", dtype=torch.bfloat16, device_map="cuda:0") task_inputs = prepare_text_generation_inputs( data=data, model_config=model.config, tokenizer=model.tokenizer, task="auto_labeling", ) task_inputs = helper.to_device(task_inputs, "cuda") torch.cuda.manual_seed_all(42) with torch.autocast("cuda", dtype=torch.bfloat16): result = generate_text( model, task_inputs, top_p=0.98, temperature=0.6, max_new_tokens=1024, ) auto_labeling_text = result["cot_auto_labeling"][0] auto_labeling_json = result["cot_auto_labeling_json"][0] print(json.dumps(auto_labeling_json, indent=2)) By default, Alpamayo 2 Super assumes access to future ego-trajectory information. However, it can also auto-label data that does not contain future trajectory information: # ... same imports and model loading as above ... # Get the model to predict a future trajectory of its own, and use that in auto-labeling as the "observed" future motion. trajectory_inputs = helper.prepare_model_inputs(data, model.config, model.tokenizer) trajectory_inputs = helper.to_device(trajectory_inputs, "cuda") torch.cuda.manual_seed_all(42) with torch.autocast("cuda", dtype=torch.bfloat16): pred_xyz, pred_rot, _, extra = model.sample_trajectories_from_data( data=trajectory_inputs, top_p=0.98, temperature=0.6, num_traj_samples=1, diffusion_kwargs={"inference_step": 10}, return_extra=True, ) future_xyz = pred_xyz[:, 0, 0].detach().cpu() future_rot = pred_rot[:, 0, 0].detach().cpu() # In case you want to see the model's reasoning, uncomment these: # trajectory_cot = str(extra["cot"].reshape(-1)[0]) # print("trajectory_cot:\n", trajectory_cot) task_inputs = prepare_text_generation_inputs( data=data, model_config=model.config, tokenizer=model.tokenizer, task="auto_labeling", # This is where the predicted trajectories are passed in: future_xyz=future_xyz, future_rot=future_rot, ) task_inputs = helper.to_device(task_inputs, "cuda") torch.cuda.manual_seed_all(42) with torch.autocast("cuda", dtype=torch.bfloat16): result = generate_text( model, task_inputs, top_p=0.98, temperature=0.6, max_new_tokens=1024, ) auto_labeling_text = result["cot_auto_labeling"][0] auto_labeling_json = result["cot_auto_labeling_json"][0] print(json.dumps(auto_labeling_json, indent=2)) Figure 6. Reasoning auto-label output with CoC trace and multi-camera input Evaluate CoC auto-labeling quality To evaluate CoC auto-labeling quality, we compare Alpamayo 2 Super’s generated labels with expert annotations on 8k internal clips. We use an internal judge model to assess similarity. Alpamayo 2 Super achieves a 0.652 similarity score, compared with Qwen3-VL 32B at 0.450. These results show its potential to generate structured reasoning labels at scale while maintaining consistency with expert-authored annotations. Build with Alpamayo 2 Super Alpamayo 2 Super brings surround-view perception, reasoning, planning, scene understanding, and data auto-labeling into one open model workflow. These capabilities give developers a practical foundation for building teacher models, curating long-tail data, inspecting policy decisions, and evaluating AV systems beyond a single open-loop trajectory metric. As a teacher model, it can be distilled into compact models that run on NVIDIA DRIVE AGX Thor inside the vehicle. Explore the model on Hugging Face , run the inference notebooks , and share what you build on the Alpamayo developer forum . For more details on the broader updates introduced as part of the Alpamayo 2 release, see the associated Hugging Face blog post .