nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16

NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16


Description

NVIDIA Nemotron 3.5 Bolt 74B-A9B is a vision-language model (VLM) designed to run everyday agentic tasks locally. This is the full-precision (BF16) release of this model, intended primarily for customization and post-training rather than direct production inference.

The 24GB version of this model uses a native 2-bit quantization to fit in 24GB of memory and run alongside other applications. It aims to achieve the optimal balance between speed and intelligence, being faster than most popular dense models while smarter than most popular MoE models with a similar memory footprint. For latency- and throughput-optimized inference, use this 24GB version for llama.cpp or vLLM instead.

This model is ready for commercial or non‑commercial use.

Model Summary

| | | |:---|:---| | Total Parameters | 74B (9B active) | | Architecture | Mamba-2–Transformer hybrid LatentMoE with a C-RADIO v4-H vision encoder | | Precision | BF16 (full-precision weights) | | Context Length | Up to 256K tokens | | Inputs / Outputs | Text, images, and video / text | | Deployment | vLLM on NVIDIA Ampere (A100), Hopper (H100-80GB), or Blackwell | | Speculative Decoding | LiLiCorr drafter via DFlash; 7 speculative tokens in the supplied vLLM recipes | | Reasoning Mode | Chat-template controls (enable_thinking=True/False), inherited from the Super 3.5 VL client | | Recommended Sampling | Text: temperature 1.0, top_p 0.95 (inherited from the Super 3.5 VL client) | | Best For | Customization, post-training, and producing quantized variants for local agentic tasks | | License | OpenMDW-1.1 |

Model Overview

Model Developer: NVIDIA Corporation

Model Dates: December 2025 - May 2026

Data Freshness:

• The post-training data has a cutoff date of May 2026.

What is Nemotron?

NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

Quick Start

This model requires the following vLLM version or newer. Please use this command.

wget https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16/resolve/main/ultra_v3_reasoning_parser.py

export VLLM_USE_V2_MODEL_RUNNER=1

vllm serve nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16 \
--trust-remote-code \
--load-format auto \
--tensor-parallel-size 1 \
--max-model-len 262144 \
--max-num-seqs 1 \
--enable-chunked-prefill \
--attention-backend FLASHINFER \
--mamba-backend flashinfer \
--mamba-cache-mode align \
--seed 42 \
--allowed-local-media-path / \
--limit-mm-per-prompt '{"video":1,"image":1}' \
--video-pruning-rate 0.5 \
--media-io-kwargs '{"video":{"fps":2,"num_frames":256}}' \
--enable-auto-tool-choice \
--tool-call-parser qwen3_coder \
--reasoning-parser-plugin ultra_v3_reasoning_parser.py \
--reasoning-parser ultra_v3

For more details on how to deploy and use the model — see the [Quick Start Guide](#quick-start-guide) below!

License/Terms of Use

Governing Terms: Use of the models is governed by the OpenMDW License Agreement, version 1.1.

Benchmarks

| Benchmark | Qwen3.6-35B-A3B-BF16 | Gemma4 26B-A4B-BF16 | Nemotron 3.5 Lightning 30B-A3B-BF16 | Nemotron 3.5 Bolt 74B-A9B-24GB (IQ2) | Nemotron 3.5 Bolt 74B-A9B (BF16) | |---|---:|---:|---:|---:|---:| | GDPVal-AA-V2 | 25.75 | 15.33 | 18.26 | 28.47 | 31.13 | | Terminal-Bench 2.1 | 44.38 | 37.22 | 23.46 | 50.98 | 53.93 | | Tau3 Banking | 10.52 | 14.02 | 9.48 | 17.01 | 17.01 | | HLE (no tools) | 19.56 | 17.42 | 10.47 | 25.30 | 28.73 | | AA-Omniscience accuracy | 19.47 | 22.17 | 16.63 | 16.90 | 21.80 | | SciCode | 35.33 | 40.28 | 31.38 | 41.02 | 44.49 | | GPQA Diamond | 83.40 | 79.61 | 75.57 | 86.43 | 88.83 | | AA-LCR | 61.06 | 57.56 | 49.19 | 64.06 | 67.38 | | AA-Omniscience non-hallucination | 65.85 | 14.22 | 68.81 | 61.27 | 56.14 | | PinchBench | 88.07 | 74.70 | 83.43 | 87.82 | 90.35 | | MCP-Atlas | 51.20* | — | 46.4 | 52.20 | 55.60 | | BrowseComp | 48.74 | 26.30 | 36.81 | 66.75 | 70.00 | | WebArena (vision) | 36.60 | 38.2 | — | 50.86 | 50.25 |

\*The reported MCP-Atlas score for Qwen3.6-35B-A3B-BF16 is 62.8; we measured 51.2 on the 500 public MCP-Atlas tasks.

Deployment Geography: Global

Use Case

Nemotron-3.5-Bolt-74B-A9B-BF16 is the full-precision (BF16) release of this model, intended primarily for customization and post-training rather than direct production inference. It is intended to be used by developers who want to: post-train the model on their own data (SFT, RL via NeMo RL and NeMo Gym, or distillation), adapt it to a domain or task, produce quantized variants (NVFP4, W4A16, GGUF) for their own deployment targets, or run full-precision research and evaluation. English and coding languages are the primary languages, with Spanish, French, German, Italian, and Japanese also being supported.

For latency- and throughput-optimized inference, use the 24GB version for llama.cpp or vLLM instead.

Release Date

Hugging Face 10/15/2026 via https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16

Reference(s)

• NVIDIA Nemotron 3 model family on Hugging Face
• NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB Technical Report
• NVIDIA Nemotron 3.5 Bolt 74B A9B 24GB Technical Blog

Model Architecture

• Architecture Type: Mamba2-Transformer Hybrid Mixture of Experts (MoE) with vision encoder
• Network Architecture: Nemotron Hybrid LatentMoE
• Nemotron 3.5 Super LLM — hybrid Mamba-2 + MoE + Attention backbone
• C-RADIO v4-H — vision encoder for image and video frames, connected to the backbone...