NVIDIA Introduces Nemotron 3.5 Super VL for Multimodal Reasoning and Specialized Agents

NVIDIA Introduces Nemotron 3.5 Super VL for Multimodal Reasoning and Specialized Agents NVIDIA has released Nemotron 3.5 Super VL, an open, customizable multimodal mixture-of-experts model designed to power specialized enterprise agents. The 120-billion-parameter model activates 12 billion parameters per token and unifies reasoning, instruction following, tool usage, text, image, document, and video understanding into a single workflow. Uses a 120B MoE architecture activating 12B parameters per token, delivering up to 5x higher throughput for agentic tasks on a single GPU compared to peer open models. Eliminates the need for separate vision and text models by integrating document, OCR, chart, and video analysis with reasoning and tool calling. Outperforms Nemotron 3 Ultra on Hermes Bench (63.44% vs. 56.33%) and LangChain behavioral evaluations (87.05% vs. 84.3%). Introduces efficiency techniques such as LiLiCorr speculative decoding, NVFP4 mixed-precision quantization, multi-token prediction, and context caching. Released alongside open datasets on Hugging Face, including a 2.8T-token Indic-language dataset developed with IIT Madras and AI4Bharat. Model Architecture and Performance Nemotron 3.5 Super VL features a 120B-parameter MoE design with 12B active parameters per token. Distilled from Nemotron 3 Ultra using SkewReverse loss and multi-teacher on-policy distillation (MOPD), it achieves over 600 tokens per second on Baseten endpoints during Artificial Analysis Intelligence Index tasks. Evaluations place the model ahead of Nemotron 3 Ultra on Hermes Bench (63.44%) and LangChain evals (87.05%), with improvements on coding benchmarks such as SWE-bench and Terminal-Bench. Multimodal Capabilities and Chart Understanding The model incorporates a two-stage supervised fine-tuning pipeline covering text, images, tables, OCR, multipage documents, and video understanding. On the CharXiv chart-reasoning benchmark, Nemotron 3.5 Super VL reached an 80.8% score (84% with Python code interpretation), up from 63% scored by Nemotron 3 Nano Omni. Training benefited from synthetic datasets generated via NeMo Data Designer, data contributions from Perplexity, and environments from Prime Intellect. Enterprise Customization and Case Studies Using NVIDIA NeMo libraries, developers can customize the model with SFT, LoRA, and reinforcement learning via NeMo Gym and NeMo RL. Early implementations demonstrate operational improvements: Scale AI's deployment for Skydance reduced vendor credit processing inference cost by 40x and latency by 5x; Ramp projected up to 50% higher throughput and 10x lower latency on receipt image processing; and AI21 with Alice reported a 41% accuracy gain in vulnerability analysis. Inference and Optimization Techniques Inference speed is boosted by LiLiCorr, an open speculative decoding technique that evaluates candidate drafts jointly to deliver up to 2.3x faster token generation on DGX Spark. The model also leverages Multi-Token Prediction (MTP) and Model Optimizer's NVFP4 quantization-aware distillation, which allows running 4-bit weights and activations on Blackwell GPUs while preserving accuracy. Context reuse preserves attention caches and Mamba recurrent states across sequential agent turns. Production Deployment and Open Assets For scaled serving, NVIDIA Dynamo provides KV-cache-aware routing and disaggregated serving, supporting up to 8x more users and an 85% drop in first-token latency across 16 B200 GPUs. Production deployment is packaged through NVIDIA NIM. Alongside base, post-trained, and quantized weights, NVIDIA published open datasets on Hugging Face, spanning finance, agent tool use, conversational data, and a 2.8-trillion-token Indic corpus covering 22 languages created with IIT Madras and AI4Bharat.

NVIDIA 推出开源多模态模型 Nemotron 3.5 Super VL，赋能专用智能体推理与复杂工作流 NVIDIA 发布了开源、可定制的多模态模型 Nemotron 3.5 Super VL。该模型拥有 120B 参数并采用 MoE 架构（每 token 激活 12B），单 GPU 上智能体任务吞吐量提升最高达 5 倍，旨在融合多模态理解与推理工具调用以服务企业智能体。 采用 120B MoE 架构且单 token 仅激活 12B 参数，在单 GPU 上的智能体吞吐量比同体量开源模型高出最高 5 倍，并在 Hermes Bench 和 LangChain 评测中超越 Nemotron 3 Ultra。 整合文本、图像、长文档、图表及视频理解与代码推理，CharXiv 图表推理得分达 80.8%（配合 Python 代码执行可达 84%），并位列 OCRBench v2 前列。 从 Nemotron 3 Ultra 蒸馏并采用多教师在策略蒸馏（MOPD），结合包含 10B 印地语 Token 的增强预训练以及开源的 2.8T Token 跨 22 种印度语言数据集。 引入推测解码技术 LiLiCorr（生成提速达 2.3 倍）、NVFP4 4位量化感知蒸馏、多 token 预测（MTP）以及跨轮次 Prefix Caching 和 Mamba 状态复用。 企业案例显示 Scale AI 推理成本降低 40 倍、延迟降 5 倍；Ramp 吞吐量提升高达 50%、延迟降 10 倍；AI21 与 Alice 的威胁情报分析准确率提升 41%。 专用智能体能力与行业定制案例 Nemotron 3.5 Super VL 支持编码调试、网络调研、文档智能、网络安全威胁分流及视频理解等工作流。企业可利用 NVIDIA NeMo 进行 SFT、LoRA 及强化学习以融入自身专有数据和流程。Scale AI 在 Skydance 自动化供应商信用处理中实现推理成本降低 40 倍、延迟降低 5 倍；Ramp 在收据结构化提取中吞吐量预计提升至多 50%、延迟降 10 倍；AI21 与 Alice 则将其用于威胁情报分析并将准确率提升 41%。 单工作流融合文本与视觉理解 该模型消除了分别部署文本与视觉语言模型的需求。通过两阶段 SFT 管线训练，涵盖图表、表格、OCR、多页文档与视频。得益于 NeMo Data Designer 生成的训练数据，在 CharXiv 图表基准上取得 80.8% 的得分，结合 Python 代码解释可达到 84%。在智能体任务基准中，Nemotron 3.5 Super VL 在 Hermes Bench 达到 63.44%、LangChain 评测达到 87.05%，均高于 Nemotron 3 Ultra。 模型底层训练与印地语能力扩展 模型基础从 Nemotron 3 Ultra 蒸馏而来，后训练阶段采用多教师在策略蒸馏（MOPD），融合不同教师在视频推理、视觉工具调用及对齐上的指导，并在 SWE-bench 和 Terminal-Bench 上超越 Nemotron 3 Super。此外，模型在持续预训练中加入了来自 12 种语言子集的 10B 印地语 Token；NVIDIA 亦联合 IIT Madras 及 AI4Bharat 开放了涵盖 22 种印度语言、约 2.8T Token 的完整数据集。 推理加速与高效生产部署 针对智能体反复调用的开销，模型结合了 LiLiCorr 并行推测解码（在 DGX Spark 上提速至多 2.3 倍）、NVFP4 混合精度与量化感知蒸馏（QAD，支持 NVFP4 KV 缓存）、多 token 预测（MTP），以及针对上下文复用的 Prefix Caching 与 Mamba 状态保留。生产部署可通过 NVIDIA NIM 封装，并借助 NVIDIA Dynamo 优化；在 16 张 B200 GPU 测试中，Dynamo 使并发用户承载量提升至多 8 倍，首 token 延迟降低 85%。

Always-on agents are taking on longer, more complex enterprise workflows. A single request can involve reading a document, checking information across sources, following company policy, and calling several tools. NVIDIA Nemotron 3.5 Super VL is an open, customizable multimodal model for these specialized workflows. Its 120B-parameter mixture-of-experts (MoE) architecture activates 12B parameters per token, balancing model capacity with inference efficiency. On a single GPU, this model delivers up to 5x higher throughput for agentic tasks compared to other open models in its size category. Figure 1. Nemotron 3.5 Super VL delivers over 600 TPS on Artificial Analysis Intelligence Index agentic tasks on the Baseten endpoint The model combines reasoning, instruction following, and tool use with text, image, document, and video understanding. By combining these capabilities into one model, teams can handle text and visual tasks without deploying separate text and vision-language models or passing task context between them. Developers can post-train Nemotron 3.5 Super VL with NVIDIA NeMo using their own data, tools, policies, and workflows. This post explains how to use that foundation to build specialized enterprise agents and describes the training and inference techniques that power it. Why Nemotron 3.5 Super VL for specialized agents? Agents need to understand both their tasks and operating environments. A customer-service agent must follow company return policies. Research agents connect evidence across sources, while document agents interpret tables, charts, and page layouts. Nemotron 3.5 Super VL combines reasoning and visual understanding across a range of agent workflows, including: Coding: Test and validate code, analyze errors, and recommend fixes. Web research: Search across websites, connect information, and verify answers. Document intelligence: Extract information from tables, charts, and page layouts to produce summaries and reports. Cybersecurity: Identify security vulnerabilities and triage threats to prioritize remediation. Video understanding: Extract information for search, content review, and incident analysis. Within a system of models, Nemotron 3.5 Super VL handles specialized, multistep work that needs reasoning, tool use, and text and visual understanding together. Smaller models, such as NVIDIA Nemotron 3.5 Lightning , handle high-volume text tasks, while frontier models such as NVIDIA Nemotron 3 Ultra tackle complex planning and orchestration. Customize Nemotron 3.5 Super VL for domain data, tools, and policies Specialized agents must learn how an organization works. Domain terminology, tool interfaces, operating procedures, and policy requirements shape how agents complete tasks successfully. Organizations in industries ranging from cybersecurity and finance to healthcare are post-training Nemotron 3.5 Super VL to outperform frontier models in accuracy and efficiency across their production workflows. For example: Scale AI, an AI data and technology provider, post-trained Nemotron 3.5 Super VL for automated vendor credit processing at Skydance Corporation, a media and entertainment company. Inference cost for this workload will decrease by 40x and latency by 5x. Ramp, a financial operations platform provider, is converting receipt images into structured JSON containing merchant information, dates, and payment data. Ramp expects up to 50% higher throughput and 10x lower latency. AI21 and Alice post-trained Nemotron 3.5 Super VL for threat intelligence and vulnerability analysis, improving accuracy by 41%. Figure 2. Post-trained Nemotron 3.5 Super VL achieves leading performance for specialized tasks across industries Teams can use NVIDIA NeMo open libraries to post-train models to meet these requirements. Developers can use NVIDIA NeMo AutoModel for supervised fine-tuning (SFT) and low-rank adaptation (LoRA) to teach domain-specific behavior. Reinforcement learning (RL) improves performance in agentic environments that reflect organizational workflows. Alongside Nemotron 3.5 Super VL, NVIDIA is releasing open datasets on Hugging Face. These include agentic tool-use and conversational data, finance reasoning and tool-calling datasets, instruction-following and chat data, long-context transduction and translation data, fairness data, and 40 vision-language datasets. These datasets can supplement proprietary data for SFT and RL. For multimodal data, start with NeMo Data Designer resources for chart question answering and long-document understanding . Explore NVIDIA NeMo RL and NVIDIA NeMo Gym for RL and environment-based evaluation. Organizations can adapt and deploy the model’s open weights in supported local, data center, or cloud environments to meet their infrastructure and data requirements. For customization resources, see the Nemotron 3.5 Super VL usage cookbook on GitHub. Combine text and visual understanding in one workflow Nemotron 3.5 Super VL combines text, image, document, and video understanding with reasoning and instruction following to answer questions across these inputs. Its visual training covers charts, tables, optical character recognition (OCR), multipage documents, and video understanding. A two-stage SFT pipeline trains the language and vision components, combining visual evidence with reasoning, chat, safety, long-context data, and agent trajectories. For document intelligence, the same model can interpret a chart, connect it with a table elsewhere in a document, and summarize both. Developers can tailor the workflow to their own documents and evaluation criteria. Diverse training data generated with NeMo Data Designer helped the model better understand complex charts. On CharXiv, a chart-reasoning benchmark, the model scored 80.8%, compared with 63% for Nemotron 3 Nano Omni . With Python code interpretation, it reached 84%. For the approach and lessons learned, read the chart-understanding deep dive and the open-source chart question-answering pipeline . This model was built with support and expertise from Nemotron Coalition organizations. For instance, high-quality datasets from Perplexity helped improve and streamline the model’s mid-training. Environments from Prime Intellect enabled building better teacher models. Nemotron 3.5 Super VL is also one of the top open models on the OCRBench v2 leaderboard . Accuracy and efficiency across agent tasks Evaluations of Nemotron 3.5 Super VL span Hermes, LangChain, OpenClaw, OpenCode, Pi, Codex, and Claude. In addition to coding tasks, agent harness providers use their own evaluations to test a wide range of agent tasks. On Hermes Bench and LangChain behavioral evaluations, Nemotron 3.5 Super VL scores higher than Nemotron 3 Ultra. Model Hermes Bench LangChain Evals Nemotron 3 Ultra 56.33% 84.3% Nemotron 3.5 Super VL 63.44% 87.05% Table 1. Nemotron 3.5 Super VL scores higher than Nemotron 3 Ultra on Hermes Bench and LangChain evaluations For always-on agents, efficiency determines how much work a deployment can complete. Throughput, token usage, tool calls, and the steps needed to reach a correct result all affect task completion time. Figure 3. Nemotron 3.5 Super VL uses higher throughput and token efficiency to deliver 4x faster task completion This model was also optimized to run locally on NVIDIA DGX Spark . An independent evaluation by EXO Labs placed Nemotron 3.5 Super VL on the Pareto frontier for accuracy and task completion time. Figure 4. Nemotron 3.5 Super VL balances accuracy and speed for agent tasks on DGX Spark Nemotron 3.5 Super VL under the hood The model builds on a strong foundation and uses a hybrid architecture, speculative decoding, context reuse, and mixed precision to reduce inference work across repeated agent calls. Building a strong base model Nemotron 3.5 Super VL was distilled from Nemotron 3 Ultra. Ultra’s coding capabilities, combined with the SkewReverse loss used during distillation, helped improve coding accuracy. During post-training, multi-teacher on-policy distillation (MOPD) combines guidance from specialized teachers across text and vision. The model generates responses, and the teachers score them for video reasoning and tracking, vision tool use, and alignment. Additional training on examples of agent actions, code repositories, and software engineering tasks improved the model’s performance on agent tasks. Nemotron 3.5 Super VL outperforms Nemotron 3 Super on SWE-bench and Terminal-Bench across multiple evaluation setups. Continued pretraining with high-quality natural and synthetic data improved Indic-language capabilities and contributed to higher overall scores after post-training. This stage added 10B Indic tokens, drawn from a 12-language subset of Nemotron-pretraining-Indic-v1 and natural Indic text. NVIDIA is releasing the full dataset, which contains about 2.8T tokens across 22 Indian languages and was developed with IIT Madras and its AI4Bharat lab. The following results are based on the post-trained Nemotron 3.5 Super VL. Figure 5. Nemotron 3.5 Super VL delivers leading accuracy on Indic benchmarks, thanks to specialized Indic data Developers can use NeMo to build on this foundation and customize Nemotron 3.5 Super VL for their applications, adapting its reasoning and agentic capabilities to their data, tools, and workflows. LiLiCorr: An open speculative decoding technique LiLiCorr improves parallel speculative decoding by jointly evaluating token candidates across an entire draft, capturing dependencies that independent token predictions miss. A lightweight network computes compatibility scores in a single pass to produce more coherent drafts, leaving only a low-cost, sequential final selection step. On DGX Spark, LiLiCorr is the recommended drafter for low-latency inference with Nemotron 3.5 Super VL. It delivers up to 2.3x faster token generation than decoding without speculation. NVFP4 and mixed precision The model uses NVIDIA Model Optimizer for post-training quantization and quantization-aware distillation (QAD). The NVFP4 recipe uses 4-bit weights and activations on the NVIDIA Blackwell architecture . The same checkpoint runs on non-Blackwell platforms with 4-bit weights and 16-bit activations. Precision is selected separately for sensitive components. NVFP4 makes the model smaller, and QAD helps it retain its accuracy at lower precision. During QAD, the quantized model learns to match the outputs of the original BF16 model, which closes the accuracy gap that quantization introduces. QAD also supports storing the key-value (KV) cache in NVFP4 instead of FP8 while still meeting accuracy targets. Because QAD happens during training, it adds no inference overhead. Multi-token prediction Multi-token prediction (MTP) gives Nemotron 3.5 Super VL an internal drafter that proposes several tokens for the main model to verify in one forward pass. Accepting multiple tokens at once reduces the number of compute-intensive verification steps needed to generate a response. A dedicated MTP-boosting training phase further aligned the drafter with the final model. Context reuse across agent turns Agents repeatedly pass system instructions, tool definitions, and conversation history back to the model. Prefix caching reuses computation from those shared inputs. The model’s serving optimizations preserve both attention caches and Mamba recurrent states. Additional Mamba checkpoints reduce the replay needed to recover inputs for speculative decoding, while internal checkpointing avoids extra model passes during prompt processing. Simplify production deployment with NVIDIA NIM and NVIDIA Dynamo NVIDIA NIM packages models with optimized inference engines, validated configurations, and standard APIs for cloud and data center deployment. NIM receives ongoing security, feature, and performance updates. Enterprise support is available through NVIDIA AI Enterprise. Get started with NIM . NVIDIA Dynamo supports distributed inference optimizations to increase throughput and reduce latency, including KV-cache-aware routing, KV cache management, and disaggregated serving. On 16 NVIDIA B200 GPUs at 50 tokens per second per user, KV-aware routing with session affinity and KV cache offload serves up to 8x as many users while reducing first-token latency by 85%. Get started with the Dynamo deployment recipes . Figure 6. NVIDIA Dynamo serves 8x as many users and reduces first-token latency by 85% at the same per-user token rate Get started with Nemotron 3.5 Super VL Build specialized agents with multimodal reasoning and tool use. Customize the model for specialized workflows, then deploy it in a supported environment. The release includes base, post-trained, and NVFP4-quantized checkpoints with training recipes, data, and reinforcement-learning environments. See the Nemotron 3.5 technical report for model details and Hugging Face or ModelScope for deployment instructions. For hosted model access, see OpenRouter . Partner ecosystem Partners support customization and deployment of Nemotron 3.5 Super VL for domain-specific workflows. View partners: Post-training AfterQuery AgileRL AI21 Alice Applied Compute Baseten Bespoke Labs Databricks distil labs Fastino Labs Fireworks AI Handshake Mercor Mindbeam ms-swift Multiverse Computing Prime Intellect Reasonable River AI Scale AI Thinking Machines Lab (Tinker) TNG Trajectory Global systems integrators HCLTech Infosys Tata Consultancy Services Thoughtworks AI natives APMIC Cognition Daily Domyn Dream Eon Factory AI H Company Harvey Inflection AI Navan NextLM NYB.AI Orca Security Perplexity Ramp Terray Two Delta Versos