NVIDIA Releases Nemotron 3.5 Lightning for High-Volume AI Agent Execution

NVIDIA Releases Nemotron 3.5 Lightning for High-Volume AI Agent Execution NVIDIA has launched Nemotron 3.5 Lightning, an open 30B mixture-of-experts model designed with 3B active parameters to handle high-frequency execution tasks for always-on AI agents. Nemotron 3.5 Lightning utilizes an open 30B MoE architecture with 3B active parameters to handle high-volume tool calls, formatting, and subagent tasks at low latency. The model includes multi-token prediction pretraining, draft models (DSpark and DFlash), and an NVFP4 quantization checkpoint alongside BF16. On PinchBench, the model achieved 86% accuracy while completing 10,000 tasks 30% faster than Qwen3.6 35B. NVIDIA introduced NeMo Switchyard to route high-level planning to frontier reasoning models while directing routine execution workloads down to Nemotron 3.5 Lightning. Architecture and Agent Workload Execution NVIDIA Nemotron 3.5 Lightning targets the execution layer of autonomous agents, such as tool invocations and output validations, which typically consume large portions of long-running token budgets. Its mixture-of-experts design activates 3B parameters per token out of 30B total, matching the capacity of larger dense models with lower compute overhead. The model is trained for frameworks including OpenClaw and Hermes Agent, and is integrated with NVIDIA's open-source NemoClaw security and management stack. Decoding Optimizations and Benchmarks To achieve low latency, the model incorporates multi-token prediction (MTP) introduced during pretraining and boosted post-training, accompanied by DSpark and DFlash draft models for speculative decoding. It is released with an NVFP4 checkpoint that runs across NVIDIA Blackwell, Hopper, and Ampere architectures. According to NVIDIA, the model achieves up to 4x the output speed of similar-sized alternatives on the Artificial Analysis Intelligence Index and completes 10,000 PinchBench tasks 30% faster than Qwen3.6 35B at comparable accuracy. Model Orchestration and Open Licensing NVIDIA introduced NeMo Switchyard, an orchestration library designed to split tasks by directing planning to frontier reasoning models like Nemotron 3 Ultra and execution steps to Nemotron 3.5 Lightning. Weights, training recipes, and data—including the Nemotron-RL Agentic Terminal Pivot dataset—are licensed under OpenMDW-1.1. The model can run locally on hardware such as NVIDIA Jetson, DGX Spark, and GeForce RTX 5090 using tools like Ollama, llama.cpp, and LM Studio.

英伟达推出开源模型 Nemotron 3.5 Lightning 优化长时运行智能体的高频任务执行 英伟达发布了开源混合专家（MoE）模型 Nemotron 3.5 Lightning，总参数量为 30B，活跃参数为 3B，专门面向全天候运行的 AI 智能体执行层，以低延迟和低成本处理工具调用、结果验证等高频常规任务。 Nemotron 3.5 Lightning 采用 30B 总参数及 3B 活跃参数的 MoE 架构，模型权重、数据与训练配方基于 OpenMDW-1.1 协议开源发布。 配套发布智能路由库 NeMo Switchyard，可将复杂规划交由顶尖推理模型，将高频执行任务分配给 Lightning 模型。 在 PinchBench 基准测试中达到 86% 准确率，处理 10,000 项任务的速度比同等精度的 Qwen3.6 35B 快 30%，并在同尺寸模型中提供最高达 4 倍的输出速度。 通过多标记预测（MTP）、DSpark/DFlash 草稿模型及 NVFP4 量化实现推理加速，可部署于数据中心以及 DGX Spark、Jetson、RTX 5090 等本地硬件。 定位于智能体执行层与模型协同路由 长时运行的 AI 智能体会消耗大量 Token 用于常规的高频任务。Nemotron 3.5 Lightning 专为 OpenClaw 和 Hermes Agent 等智能体测试框架优化，作为执行层降低推理成本。英伟达同步推出了 NeMo Switchyard 路由库，使开发者能够构建多模型系统，将规划交由如 Nemotron 3 Ultra 等前沿模型，将格式化和工具验证等具体任务交由 Lightning 模型处理。 性能表现与基准测试 在整合了智能体、编码及科学推理等 9 项评测的 Artificial Analysis Intelligence Index 榜单中，Nemotron 3.5 Lightning 确立了小型开源模型的准确率-速度帕累托前沿，输出速度达到同类尺寸模型的最高 4 倍。在 PinchBench 测试中，它以 86% 的准确率比 Qwen3.6 35B 提速 30% 完成 1 万项任务。 投机解码与本地化部署优化 该模型在预训练及后训练阶段融合了多标记预测技术，并附带针对 DGX Spark 及低并发场景的 DSpark 和 DFlash 两款草稿模型以实现投机解码。模型同时提供 BF16 和针对 Blackwell、Hopper、Ampere 优化的 NVFP4 量化版本，支持在 LM Studio、llama.cpp、Ollama 等本地工具上运行。

Long-running AI agents spend most of their time on high-volume execution: tool calls, result validation, and subagent delegation. Using a frontier reasoning model for every execution step adds cost and latency. NVIDIA Nemotron 3.5 Lightning is an open 30B mixture-of-experts (MoE) model with 3B active parameters built for that execution layer of always-on agents. It is designed for harnesses like OpenClaw and Hermes Agent—all supported by the NVIDIA NemoClaw open source security and management stack for running always-on AI agents. The NVIDIA Nemotron open model family is like a software library, with each release continuously improving accuracy and speed. As these models evolve, the rapid maturation of model routing and orchestration is also underway. This is important because developers increasingly build applications with a system of models . Frontier reasoning models such as Nemotron 3 Ultra handle orchestration and complex planning while smaller, more efficient models handle the high-volume execution layer. This post introduces NVIDIA Nemotron 3.5 Lightning and explains how its smaller MoE design is optimized for high-volume, low-latency execution in autonomous agents. It also details the inference and training innovations that power it. Finally, the post also introduces NVIDIA NeMo Switchyard , a library that intelligently routes each task to the best model for the job. Why is Nemotron 3.5 Lightning ideal for long-running AI agents? Nemotron 3.5 Lightning is a customizable open 30B MoE model with 3B active parameters, providing optimal high-volume execution for autonomous agents. MoEs are fast and efficient because a router sends each token to just a few of its many experts, so only a fraction of the model’s parameters run per token. This provides the capacity of a larger dense model at the compute cost of a small one. Nemotron 3.5 Lightning is the smallest member of the Nemotron 3 model family and ships with many of the same techniques proven across the family, including: Speculative decoding : Multi-token prediction was included during Nemotron 3.5 Lightning training (as for Nemotron 3 Super and Nemotron 3 Ultra). Nemotron 3.5 Lightning also ships with DFlash and DSpark, enabling more comprehensive inference optimization across a range of serving scenarios. Harness-optimized training: The model is trained for popular agent harnesses, enabling agents to make more accurate calls while reducing latency for high-volume tasks. The result is a model built for execution-focused, high call volumes, and low latency—all at a size that deploys anywhere from an NVIDIA DGX Spark to a data center. Video 1. Learn how to deploy NVIDIA Nemotron 3.5 Lightning on DGX Spark and use it for fast, high-volume agentic workloads Customize Nemotron 3.5 Lightning out of the box Models earn their place in specialized AI agent systems when they’re adapted to the job. And Lightning-class models are highly customizable: small models fine-tune faster, cheaper, and on far more modest hardware than their larger counterparts. You can customize Nemotron 3.5 Lightning out of the box to fit your workload. As with every Nemotron open model launch, the weights, training data, and recipes are released as permissively as possible under OpenMDW-1.1, so you can: Fine-tune with LoRA or full SFT using NeMo Automodel and NeMo Megatron Bridge Run reinforcement learning and environment-based evaluations and rollouts with NeMo RL and NeMo Gym This release includes Nemotron-RL Agentic Terminal Pivot , an open agentic reinforcement learning dataset used to train some of the coding agent capabilities. Route work to the right model using NeMo Switchyard While frontier models may win the headlines, models like Nemotron 3.5 Lightning earn their medals in the trenches. They handle requests like git pull, validate tool outputs, format results, and run the routine calls that dominate any long-running agent’s token budget. Model routing and orchestration help make this division of labor more accessible. They are now  available through NVIDIA NeMo Switchyard . Switchyard can expose Nemotron 3.5 Lightning as a routing target alongside your open and closed models, so every request lands on the most capable and efficient model that can handle it. Plans route up to the frontier, execution routes down to Lightning, ensuring that your tokens are spent efficiently and effectively. How does Nemotron 3.5 Lightning perform on the accuracy-speed Pareto frontier? Nemotron 3.5 Lightning delivers leading accuracy at the highest output speed in its class, winning the accuracy-versus-speed Pareto frontier on the Artificial Analysis Intelligence Index. This index combines nine evaluations to measure model performance across agentic tasks, coding, scientific reasoning, and general intelligence. Nemotron 3.5 Lightning combines strong intelligence with up to 4x output speed of similar-sized models, placing it on the accuracy-speed Pareto frontier for high-volume agent workloads. Figure 1. Nemotron 3.5 Lightning defines the accuracy-speed Pareto frontier for small open models on the Artificial Analysis Intelligence Index leaderboard Agent efficiency ultimately comes down to how quickly a model completes useful work and not simply how fast it generates tokens. On PinchBench, Nemotron 3.5 Lightning reaches 86% accuracy while completing 10,000 tasks 30% faster than Qwen3.6 35B at similar accuracy. Higher inference throughput and token efficiency places Nemotron 3.5 Lightning on the efficiency frontier, helping always-on agents finish high-volume work faster. Figure 2. Nemotron 3.5 Lightning leads the efficiency frontier by completing agentic tasks up to 30% faster at comparable accuracies How does Nemotron 3.5 Lightning deliver speed without compromising accuracy? Nemotron 3.5 Lightning delivers speed and customization without compromising accuracy through speculative decoding, and quantization. Speculative decoding Nemotron 3.5 Lightning is built to quickly generate tokens and has the ability to generate multiple tokens through speculative decoding. This is a process whereby the model, or draft model, will draft some number of tokens which are efficiently reviewed. Nemotron 3.5 Lightning underwent a dedicated pretraining stage to bake multi-token prediction (MTP) into the model, as with Nemotron 3 Super and Ultra. After training, a dedicated MTP-boosting phase further improved MTP accuracy. Beyond MTP, two draft models are provided with Nemotron 3.5 Lightning: DSpark , which is recommended for DGX Spark inference workloads and low concurrency data center workloads. MTP is best suited for medium to high concurrency, with the optimal draft length decreasing as concurrency increases. NVIDIA is also releasing a DFlash draft model, which can be measured against the others and may perform best for your workloads. Quantization Nemotron 3.5 Lightning ships with an NVFP4 checkpoint alongside BF16, using the same specialized NVFP4 kernels that power Nemotron 3 Ultra across NVIDIA Blackwell, NVIDIA Hopper, and NVIDIA Ampere GPUs. The same file serves just as well in data centers as it does on your desktop DGX Spark. How is Nemotron 3.5 Lightning ideal for local AI? Nemotron 3.5 Lightning makes capable agentic AI accessible on local systems including NVIDIA Jetson, GeForce RTX 5090, and DGX Spark. NVIDIA has worked with a number of teams including EXO Labs to understand how this model performs on DGX Spark. Figure 3. Nemotron 3.5 Lightning sits right on the Pareto frontier for small open models on the EXO Labs local.ai leaderboard In addition, you can run Nemotron 3.5 Lightning across the industry standard set of tools, including LM Studio, llama.cpp, Ollama, and Unsloth. Partner ecosystem Nemotron 3.5 Lightning is supported by a growing ecosystem of partners across harnesses, customization, deployment, and inference, including: Post-training: AgileRL , Applied Compute, Deep Cogito, distil labs , Fastino Labs , Locai Labs , Prime Intellect, Reasonable , Thinking Machines Lab, Thoughtworks , Trajectory , Uniphore Inference software: Ollama , Exo, Canonical , LM Studio , Unsloth Harnesses and agent frameworks: Aible , Cline , Factory AI , Hermes Agent, Kilo Code , LangChain , LM Studio Bionic, OpenClaw, OpenCode , OpenHands , Pi Cloud service provider platforms : Amazon SageMaker JumpStart , Google Cloud Gemini Enterprise Agent Platform , MSFT Foundry, OCI Enterprise AI GSI: Accenture, Tata Consultancy Services , Tech Mahindra , Wipro AI natives: Arcos Labs , CodeRabbit , Dream, Harvey Hosted inference service providers: Baseten , Bitdeer AI , BlackBox AI, CoreWeave , Crusoe , DeepInfra , Fireworks AI , FriendliAI , GMI Cloud , Modal , Nebius , Together AI Start building with Nemotron 3.5 Lightning Nemotron 3.5 Lightning is fully open—weights, data, and recipes—so you can adapt it to your workflows and deploy it anywhere. To get started, try it on build.nvidia.com or through OpenRouter . Download the weights from Hugging Face , and ModelScope . Want to dive deeper? Read the Nemotron 3.5 Lightning cookbook Deploy with the vLLM , SGLang , and TensorRT-LLM deployment guides Set up routing with Switchyard docs Stay up to date on NVIDIA Nemotron by subscribing to NVIDIA news and following NVIDIA AI on LinkedIn , X , Discord , and YouTube . Visit the Nemotron developer page for resources to get started. Explore open Nemotron models and datasets on Hugging Face , ModelScope , and Blueprints on build.nvidia.com . Engage with Nemotron livestreams , tutorials , and the developer community on the NVIDIA forum and Discord .