Customizing NVIDIA Nemotron 3 Nano with Prime Intellect Lab

Customizing NVIDIA Nemotron 3 Nano with Prime Intellect Lab A guide demonstrates how to customize NVIDIA Nemotron 3 Nano using Prime Intellect Lab's hosted reinforcement learning infrastructure, achieving significant task performance improvements at low cost. The tutorial uses reinforcement learning with verifiable rewards (RLVR) in a Python Math environment with tool access capped at five turns. Evaluating a 100-step training run showed model accuracy improving from a 21.9% baseline to 90.6% on 32 test examples. The entire customization process cost less than $5 in compute and yields a deployable LoRA adapter. The identical workflow can be scaled to larger models like Nemotron 3 Super and Nemotron 3 Ultra by modifying the model identifier in configuration files. Baseline Evaluation in Python Math Using Prime Intellect Lab's CLI, developers configure an environment to evaluate Nemotron 3 Nano on mathematical reasoning tasks using Python libraries such as numpy, sympy, and scipy. In the baseline run, the unadapted model achieved an average reward of 0.219 (21.9% accuracy), frequently failing by exhausting turns on unsupported tool calls without outputting a final boxed answer. Reinforcement Learning and LoRA Training A TOML configuration sets up 100 steps of reinforcement learning with verifiable rewards, using batches of 32 rollouts per example and tracking rewards via the Prime Intellect dashboard. The platform manages the underlying compute infrastructure without requiring local GPU cluster management. Evaluation Results and Scaling Deploying the generated LoRA adapter raised evaluation accuracy to 90.6% on the identical 32-problem test set, reflecting a 68.75 percentage point gain for under $5. The tutorial notes that developers can apply the exact same evaluation and RLVR training pipeline to larger family models, including Nemotron 3 Super and Nemotron 3 Ultra, by updating the model identifier in the training configuration.

借助 Prime Intellect Lab 快速定制 NVIDIA Nemotron 3 Nano 模型 NVIDIA 介绍了如何结合 Prime Intellect Lab 托管训练平台，利用可验证奖励强化学习（RLVR）定制开源模型 Nemotron 3 Nano。整个工作流涵盖建立基线、100 步强化学习微调以及部署重新评估三个核心步骤，显著提升了模型在 Python 数学任务上的表现。 本地环境仅需约 5 到 15 分钟配置，依托托管训练无需自行管理本地 GPU 集群即可产出 LoRA 适配器。 在 32 个测试样本的数学任务中，Nemotron 3 Nano 的准确率从基线的 21.9% 提升至训练后的 90.6%，总花费低于 5 美元。 相同的工作流程可无缝复用至 Nemotron 3 Super（120B）和 Ultra（550B）等更大规模的模型上。 定制流程三步走 开发者首先通过 Prime CLI 搭建工作区并在 Python 数学环境中评测基线；随后配置包含 100 步训练、批大小为 32 的强化学习任务（RLVR），并在云端仪表盘监控奖励曲线与轨迹；训练完成后生成 LoRA 适配器并部署为在线推理端点。 基线与微调结果对比 基线模型因在 5 轮上限内调用不支持的工具而频繁失败，平均奖励仅为 0.219。经过微调后，平均奖励升至 0.906，准确率绝对值提升 68.75 个百分点，24 个产生结果差异的测试用例中仅有 1 个出现退步。 扩展至超大模型 得益于开源权重和托管基础设施，该流程同样支持 Nemotron 3 Super 以及运行在 NVIDIA Blackwell 架构上的 Nemotron 3 Ultra。开发者只需替换配置文件中的模型标识符，即可复用整套定制与评估体系。

Customization is what enables developers to take a general model and tailor it to use cases, domains, languages, and more. However, customization comes with a few challenges. It requires infrastructure , technical expertise, and software specific to the workflow, as well as resources such as GPUs and the ability to use them effectively. It also depends on specialized domain knowledge: What algorithm should I use? What environment should I use? How do I know whether the model actually learned? Customization can appear daunting. With open models such as the NVIDIA Nemotron 3 family—including Nano , Super , and Ultra —combined with platforms such as Prime Intellect Lab that offer training as a service, the complete customization loop is much more accessible. In this tutorial, you’ll use Prime Intellect Lab to customize NVIDIA Nemotron 3 Nano with hosted reinforcement learning and produce a downloadable LoRA adapter. The local setup takes about five minutes—that’s all. By the end of this blog, you’ll learn two things: Customization is more accessible than ever. Getting started with open models is easier than ever. What you’ll build We’re going to follow a simple flow for this tutorial: Baseline, train, and reevaluate—all using Nemotron 3 Nano—and resulting in a customized model for our task. To show how easy it is to get started with the robust customization tools we have at our disposal, we’ll be using a standard “hello world” example: Python Math. This tutorial applies to Nemotron 3 Super and Nemotron 3 Ultra. The video will also walk you through the more involved coding task that’s part of a companion notebook to this tutorial: Video 1. This tutorial walks through a complete customization workflow: baseline Nemotron 3 Nano in the math-python environment, train a LoRA adapter with reinforcement learning with verifiable rewards, deploy it, and compare results What customization means Customization means taking a model and changing or adapting its trainable parameters so that it becomes more likely to do the kinds of tasks you want it to do successfully. There are many ways to do this. Depending on the problem, you might use supervised fine-tuning, parameter-efficient fine-tuning, preference optimization, reinforcement learning, or a combination of techniques. For a broader treatment, see the Selecting Large Language Model Customization Techniques post. This tutorial focuses on reinforcement learning with verifiable rewards (RLVR) on the Python Math environment . The basic idea of the environment is simple: Use Python tools (Python, numpy , sympy and scipy ) to do some math. Here’s an example of a prompt: # System: Use Python for all calculations. Give your answer inside \boxed{}. In addition to the Python standard library, you have access to: numpy sympy scipy. # User: If $2^8=4^x$, what is the value of $x$? > NOTE: The environment has a default of 100 turns. For this experiment, we cap each rollout at five assistant turns. A direct response is one turn, and so a tool call with the final response is two of the 5 turns. This penalizes the model heavily for repeatedly calling tools without producing a final response. For a deeper introduction to reinforcement learning for agents , see the post Mastering Agentic Techniques: AI Agent Reinforcement Learning . Why openness matters Open weights are an important part of the equation, but they aren’t the entire equation. Data, evaluation code, and training recipes also determine what you can understand, reproduce, and deploy. The NVIDIA Nemotron 3 family is published with open model resources, including weights, data, and recipes. You can explore the NVIDIA Nemotron developer resources , read Inside NVIDIA Nemotron 3 , and inspect the NVIDIA Nemotron repository . These resources give a better view into how Nemotron was built and post-trained, the kinds of data and environments that were used, and where your own customization should differ. Prerequisites For this tutorial, you’ll need: A development environment with curl and a supported Python 3 installation. A Prime Intellect account with access to Hosted Training and billing configured. Internet access for the Prime CLI, environment package, hosted model, and adapter deployment. About 5-15 minutes. Prime Intellect handles the hosted rollout, training, and inference infrastructure; you don’t need to manage a local GPU cluster. Note: Model availability and pricing change over time. Treat the live CLI catalog as the source of truth rather than copying an identifier or price from a static article. Customize Nemotron 3 Nano in three steps Let’s get into the simple steps required to customize your very own Nemotron 3 Nano. Step 1: Get a Baseline First, install the Prime CLI, authenticate, and create a Prime Intellect Lab workspace: curl -LsSf https://astral.sh/uv/install.sh | sh uv python install 3.13 uv tool install --python 3.13 --force "prime==0.6.17" uv --version prime --version prime login mkdir -p nemotron-customization cd nemotron-customization prime lab setup uv run python --version Prime Intellect Lab creates the workspace structure and installs the verifiers tooling. The longer-running evaluation and training steps come next. Inspect the environment before using it: prime env info primeintellect/math-python --version 0.1.10 prime env inspect primeintellect/math-python@0.1.10 README.md Pin the exact environment package in the workspace: uv add "math_python==0.1.10" \ --index https://hub.primeintellect.ai/primeintellect/simple/ uv lock --check Now check the live hosted training catalog: prime train models prime inference models Checkpoint: Continue only if the output contains nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16 and shows the current training, input, and output prices. Use the following command to grab a quick baseline on our task so we can show if the model improved after training: export NANO="nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16" export MATH_EVAL_ARGS='{"dataset_name":"math","dataset_split":"test","num_train_examples":-1,"max_turns":5,"sandbox_client_max_workers":8}' export MATH_SAMPLING='{"max_tokens":2048,"temperature":1.0,"extra_body":{"chat_template_kwargs":{"enable_thinking":false}}}' prime eval run math-python \ --provider prime \ --model "$NANO" \ --num-examples 32 \ --rollouts-per-example 1 \ --max-concurrent 8 \ --env-args "$MATH_EVAL_ARGS" \ --sampling-args "$MATH_SAMPLING" \ --timeout 300 \ --max-retries 2 \ --save-results \ --disable-tui \ --abbreviated-summary Your results should look similar to the following, which showcases that the model starts off doing rather poorly and has a fairly low accuracy. ... --- All --- Rewards: reward: avg - 0.219, std - 0.413... This is an example of a failing prompt for our baseline model, in which the model fails to produce a boxed response because it exhausted five turns calling unsupported tools: # System: Use Python for all calculations. Give your answer inside \boxed{}. In addition to the Python standard library, you have access to: numpy sympy scipy. # User: If $2^8=4^x$, what is the value of $x$? # Assistant: [[EMPTY RESPONSE]] Figure 1 is an example of our reward distribution on this initial evaluation. The reward is binary: 1 means the model produced a correct response, and 0 means the model produced an incorrect or missing response. Figure 1. Reward distribution for 32 examples of the Math Python task, with most of the entries showing as unsolved or incorrect Step 2: Take Nemotron 3 Nano to school To get started on customizing Nemotron 3 Nano, the first thing we’ll do is create configs/rl/nemotron-3-nano-math-python-100step.toml : name = "nemotron-3-nano-math-python-100step" model = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16" loss = "rl" max_steps = 100 batch_size = 32 rollouts_per_example = 8 max_inflight_rollouts = 32 learning_rate = 2e-5 lora_alpha = 16 [sampling] max_tokens = 2048 temperature = 1.0 enable_thinking = false [adapters] interval = 0 keep_last = 1 [[env]] id = "primeintellect/math-python@0.1.10" args = { dataset_name = "math", dataset_split = "train", num_train_examples = -1, max_turns = 5 } This config outlines how we want to train our model. The trainer samples eight attempts for each selected task, giving it a comparative signal inside each rollout group. Four groups form a batch of 32, and caps the number of rollouts being assembled concurrently for this small run. Launch the run: prime train configs/rl/nemotron-3-nano-math-python-100step.toml Prime prints the parsed configuration, current price, and environment action status before asking for confirmation. After you confirm, the CLI returns a run ID and a dashboard URL. Evaluation and training continue after that point. Checkpoint: Save the run ID. The launch output should show the exact Nano model, the pinned Math Python environment, live pricing, and a successful environment action check. The dashboard shows reward curves, reward distributions, and individual rollouts. The CLI exposes the same evidence: export RUN_ID="<run-id>" export STEP="<step>" prime train logs "$RUN_ID" --follow prime train progress "$RUN_ID" prime train distributions "$RUN_ID" --type rewards prime train usage "$RUN_ID" --watch prime train rollouts "$RUN_ID" --step "$STEP" prime train checkpoints "$RUN_ID" When experimenting, begin with small variations. If every attempt receives the same score, the trainer has little signal to work with. Next, check a high-, middle-, and low-reward rollout and ask whether the score reflects genuinely better math problem-solving. Finally, look at factors like token use and truncation to ensure the configuration is not interfering with the training signal. If the reward curve tells us if the model got more or less reward; the rollouts help tell you why. Figure 2. Nemotron 3 Nano’s reward curve over 100 steps Step 3: Check how much Nemotron 3 Nano learned Training reward is‌ like measuring the model’s performance on homework. It’s not real until you give it a test. When the run completes successfully, Prime Intellect Lab uploads the final LoRA adapter. Confirm that the training artifact is ready, and then inspect its deployment state: prime deployments list A ready adapter that isn’t currently serving can appear as NOT_DEPLOYED . After deployment is requested, its serving state moves through DEPLOYING to DEPLOYED . Checkpoint: Continue only after the final adapter exists and is ready to deploy. A completed trainer process without a usable adapter isn’t the artifact this tutorial promises. You can download the adapter from the training dashboard. To query it through hosted inference, deploy it: prime deployments list prime train checkpoints "$RUN_ID" --status READY export ADAPTER_ID="<adapter-id>" export ADAPTED_MODEL="$NANO:$ADAPTER_ID" prime deployments create "$ADAPTER_ID" --plain COLUMNS=300 NO_COLOR=1 prime deployments list --plain Deployment creates a billable inference endpoint. Once its status is DEPLOYED , the model identifier combines Nano with the adapter ID: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16:<adapter-id> Now repeat the baseline command, changing only the model identifier Keep the environment, task count, rollout count, sampling settings, and reward unchanged. Changing more than the model makes the comparison harder to interpret. # Continue only after the adapter reports DEPLOYED. prime eval run math-python \ --provider prime \ --model "$ADAPTED_MODEL" \ --num-examples 32 \ --rollouts-per-example 1 \ --max-concurrent 8 \ --env-args "$MATH_EVAL_ARGS" \ --sampling-args "$MATH_SAMPLING" \ --timeout 300 \ --max-retries 2 \ --save-results \ --disable-tui \ --abbreviated-summary Your results should contain the following, which showcases the same evaluation configuration used above—showing an increase in the average reward as compared to our baseline model on the same example set—this result shows that our model has learned to be meaningfully better at our task: ... --- All --- Rewards: reward: avg - 0.906, std - 0.291 ... Here’s the same prompt the initial model was unable to correctly solve: # System: Use Python for all calculations. Give your answer inside \boxed{}. In addition to the Python standard library, you have access to: numpy sympy scipy. # User: If $2^8=4^x$, what is the value of $x$? # Assistant Tool Turn: sp.solve(sp.Eq(2**8, 4**x), x) ...# Assistant: We can rewrite the equation using the same base. \[ 2^8 = 4^x \quad\Longrightarrow\quad 2^8 = (2^2)^x = 2^{2x}. \] Since the bases are equal, the exponents must be equal: \[ 8 = 2x \;\Longrightarrow\; x = \frac{8}{2}=4. \] \[ \boxed{4} \] Figure 3 shows the reward distribution on the same set of evaluation prompts we saw before, and as you can see, the model can get a significantly higher number of them correct: Figure 3. Reward distribution for 32 examples of the Math Python task with most of the problems now showing as solved On the same 32 held-out problems, accuracy increased from 21.9% (7/32) to 90.6% (29/32), an absolute increase of 68.75 percentage points. Of the 24 prompts that had different outcomes between baseline and final, 23 improved, and one regressed. Additionally, Nemotron 3 Nano got better at this task with a total spend of less than $5. To remove the adapter deployment, you can issue the following command as cleanup: prime deployments delete "$ADAPTER_ID" --plain See the Prime Intellect guide to deploying LoRA adapters for inference for deployment states and OpenAI-compatible inference examples. Training larger models with Prime Intellect Lab: Now that we’ve gone through an example of how we can use Prime Intellect Lab to run a short training of Nemotron 3 Nano, you may be asking yourself: “What if I wanted to train larger models? Is it much harder?” The answer is: not at all. With the convenience of hosted training, we can simply modify the model identifier and run the exact same workflow using the following: Train Nemotron 3 Super Run the live catalog check again: prime train models Confirm availability and current pricing, then replace the model line with: model = "nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16" Keep the same experimental discipline: baseline first, inspect reward variation, train, and retest under unchanged settings. Train Nemotron 3 Ultra Nemotron 3 Ultra is the high-capacity option in the family, and Prime Intellect has support for hosted reinforcement learning on NVIDIA Blackwell infrastructure. Verify you have access to managed training of Ultra using the following command: prime train models If you see it, replace the model line with: model = "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4" Begin customizing Nemotron 3 Customization is how to custom-tailor AI for our use cases. Without customization, we’re limited to whatever behavior a model has out of the box, and we may overpay for a more broadly capable model when a lower-cost model could be adapted to the task we need. Open models such as NVIDIA Nemotron 3, combined with hosted training platforms such as Prime Intellect Lab, make that process much easier to start. The most valuable part isn’t simply that the weights are available or that the infrastructure is managed. It’s that the entire improvement loop can be inspected and repeated. Visit the Nemotron developer page for resources to get started. Explore open Nemotron models and datasets on Hugging Face and Blueprints on build.nvidia.com . Engage with Nemotron livestreams , tutorials , and the developer community on the NVIDIA forum and the Nemotron channel on Discord . Stay up to date on NVIDIA Nemotron by subscribing to NVIDIA news and following NVIDIA AI on LinkedIn , X , Discord , and YouTube .