nvidia/NVIDIA-Nemotron-Labs-Teacher-STEM

NVIDIA-Nemotron-Labs-Teacher-STEM


Model Summary

| | | |:---|:---| | Total Parameters | 550B (55B active) | | Architecture | LatentMoE - Mamba-2 + MoE + Attention hybrid with Multi-Token Prediction (MTP) | | Context Length | Up to 1M tokens | | Minimum GPU Requirement | NVIDIA Grace Blackwell (4xGB200) NVIDIA Blackwell (4xB200), NVIDIA Grace Blackwell Ultra (4xGB300), NVIDIA Blackwell Ultra (4xB300), NVIDIA Hopper (8xH100) | | Supported Languages | English, French, Spanish, Italian, German, Japanese, Hindi, Korean, Brazilian Portuguese, and Chinese | | Best For | Reasoning, including multi-step, across math, code, natural sciences, and humanities; knowledge-intensive QA; reasoning-trace and synthetic-data generation; serving as a distillation teacher | | Reasoning Mode | Configurable on/off via chat template (enable_thinking=True/False) | | License | OpenMDW License Agreement, version 1.1 | | Release Date | August, 2026 | | Quickstart | [Click here!](#quick-start-guide) |

Model Overview

Model Developer: NVIDIA Corporation

Model Dates: December 2025 - May 2026

Data Freshness:

• The post-training data has a cutoff date of May 2026.
• The pre-training data has a cutoff date of September 2025.

What is Nemotron?

NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

Description

NVIDIA-Nemotron-Labs-Teacher-STEM is a specialized general-reasoning model in the Nemotron 3 Ultra family, trained by NVIDIA. It is produced by taking the post-trained Nemotron 3 Ultra student and applying an additional round of reasoning-focused supervised fine-tuning and reinforcement learning, yielding a model with strong general reasoning capabilities across a wide range of subjects — including math, code, the natural sciences, humanities, sociology, and tool use within these domains (including code execution and search). Within the broader recipe, this is one of more than ten domain-specialized teacher models that supply training signal to Multi-Teacher On-Policy Distillation (MOPD), the stage used to produce the final Nemotron 3 Ultra. It is released as a standalone checkpoint because it is a strong reasoning model in its own right, matching or outperforming DeepSeek V4 Pro (High) on benchmarks such as GPQA, MMLU-Pro, LiveCodeBench v6, IMOAnswerBench, and Apex Shortlist. Like other models in the family, it responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response. The model's reasoning capabilities can be configured through a flag in the chat template.

The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. Like the Super model, the Ultra model incorporates Multi-Token Prediction (MTP) layers for faster text generation and improved quality, and it is trained using an NVFP4 pre-training recipe to maximize compute efficiency. The model has 55B active parameters and 550B parameters in total.

The supported languages include: English, French, Spanish, Italian, German, Japanese, Korean, Hindi, Brazilian Portuguese, and Chinese.

This model is ready for commercial and non-commercial use.

License/Terms of Use

Governing Download Terms: Use of this model is governed by the OpenMDW-1.1 model license.

Deployment Geography: Global

Use Case

NVIDIA-Nemotron-Labs-Teacher-STEM is a specialized general-reasoning model intended to be used in English, Code, and supported multilingual contexts. Its primary role is to serve as a domain-specialized teacher for Multi-Teacher On-Policy Distillation (MOPD) in the Nemotron 3 Ultra recipe. It is released so developers and researchers can use it for synthetic-data generation, reasoning-trace distillation, knowledge-intensive question answering, competitive math and coding, and scientific reasoning, as well as a strong teacher or grader within their own distillation and data-generation pipelines. It is well suited to generating high-quality, verifiable reasoning traces across math, code, the natural sciences, and humanities.

Release Date

Hugging Face - 08/14/2026 via Hugging Face

Reference(s)

• NVIDIA Nemotron 3 model family on Hugging Face
• NVIDIA Nemotron 3 Ultra Technical Report

Model Architecture

• Architecture Type: Mamba2-Transformer Hybrid Latent Mixture of Experts (LatentMoE) with Multi-Token Prediction (MTP)
• Network Architecture: Nemotron Hybrid LatentMoE
• Number of model parameters: 550B Total / 55B Active

Computational Load (Internal Only: For NVIDIA Models Only; please add as an HTML comment and remove the fields below from the published model card)

Cumulative Compute: Pre-Training (7.18e+24 FLOPS) - Post-Training (1.15e+23 FLOPs) Estimated Energy and Emissions for Model Training: Pre-Training (11,890,852 kWh, 3841) - Post-Training (299,600 kWh, 102)

Model Design

The model utilizes the LatentMoE architecture, where tokens are projected into a smaller latent dimension for expert routing and computation, improving accuracy per byte. The Ultra model is pre-trained using an NVFP4 recipe — sharing the quantization-aware pre-training approach pioneered in the Nemotron 3 family. The majority of linear layers use NVFP4 for weights, activations, and gradients, while select layers (including latent projections, MTP layers, QKV/attention projections, and embeddings) are maintained in BF16 or MXFP8 for training stability. The model includes Multi-Token Prediction (MTP) layers using a shared-weight design across prediction heads. This improves training signal quality, enables faster inference via native speculative decoding, and supports more stable autoregressive drafting at longer draft lengths compared to independently trained offset heads.

Training Methodology

Stage 1:...