nvidia/NVIDIA-Nemotron-Labs-Teacher-Competition-Coding

NVIDIA-Nemotron-Labs-Teacher-Competition-Coding


Model Summary

| | | |:---|:---| | Total Parameters | 550B (55B active) | | Architecture | LatentMoE - Mamba-2 + MoE + Attention hybrid with Multi-Token Prediction (MTP) | | Context Length | Up to 1M tokens | | Minimum GPU Requirement | 4xGB200, 4xB200, 4x GB300, 4x B300, 8xH100 | | Supported Languages | English, French, Spanish, Italian, German, Japanese, Hindi, Korean, Brazilian Portuguese, and Chinese | | Best For | Competitive programming, algorithmic problem solving, and code reasoning; verified coding-solution and reasoning-trace generation; serving as a distillation teacher | | Reasoning Mode | Configurable on/off via chat template (enable_thinking=True/False) | | License | OpenMDW License Agreement, version 1.1 | | Release Date | August, 2026 | | Quickstart | [Click here!](#quick-start-guide) |

Model Overview

Model Developer: NVIDIA Corporation

Model Dates: December 2025 - May 2026

Data Freshness:

• The post-training data has a cutoff date of May 2026.
• The pre-training data has a cutoff date of September 2025.

What is Nemotron?

NVIDIA Nemotron™ is a family of open models with open weights, training data, and recipes, delivering leading efficiency and accuracy for building specialized AI agents.

Description

NVIDIA-Nemotron-Labs-Teacher-Competition-Coding is a specialized programming model in the Nemotron 3 Ultra family, trained by NVIDIA. It is produced by taking the post-trained Nemotron 3 Ultra student and applying an additional round of coding-focused supervised fine-tuning and reinforcement learning, yielding a model with strong algorithmic reasoning and solution-generation capabilities across competitive programming tasks. Within the broader recipe, this is one of more than ten domain-specialized teacher models that supply training signal to Multi-Teacher On-Policy Distillation (MOPD), the stage used to produce the final Nemotron 3 Ultra. It is released as a standalone checkpoint because it is a strong coding model in its own right, achieving leading results on competition coding benchmarks. Like other models in the family, it responds to user queries and tasks by first generating a reasoning trace and then concluding with a final response. The model's reasoning capabilities can be configured through a flag in the chat template.

The model employs a hybrid Latent Mixture-of-Experts (LatentMoE) architecture, utilizing interleaved Mamba-2 and MoE layers, along with select Attention layers. Like the Super model, the Ultra model incorporates Multi-Token Prediction (MTP) layers for faster text generation and improved quality, and it is trained using an NVFP4 pre-training recipe to maximize compute efficiency. The model has 55B active parameters and 550B parameters in total.

The supported languages include: English, French, Spanish, Italian, German, Japanese, Korean, Hindi, Brazilian Portuguese, and Chinese.

This model is ready for commercial and non-commercial use.

License/Terms of Use

Governing Download Terms: Use of this model is governed by the OpenMDW-1.1 model license.

Deployment Geography: Global

Use Case

NVIDIA-Nemotron-Labs-Teacher-Competition-Coding is a specialized programming model intended to be used in English, Code, and supported multilingual contexts. Its primary role is to serve as a domain-specialized teacher for Multi-Teacher On-Policy Distillation (MOPD) in the Nemotron 3 Ultra recipe. It is released so developers and researchers can use it for competitive coding, algorithmic problem solving, generating verified code solutions and reasoning traces, and as a teacher or grader within their own distillation and data-generation pipelines. It is well suited to producing execution-verified solutions to hard algorithmic problems.

Release Date

Hugging Face - 08/14/2026 via Hugging Face

Reference(s)

• NVIDIA Nemotron 3 model family on Hugging Face
• NVIDIA Nemotron 3 Ultra Technical Report

Model Architecture

• Architecture Type: Mamba2-Transformer Hybrid Latent Mixture of Experts (LatentMoE) with Multi-Token Prediction (MTP)
• Network Architecture: Nemotron Hybrid LatentMoE
• Number of model parameters: 550B Total / 55B Active

Model Design

The model utilizes the LatentMoE architecture, where tokens are projected into a smaller latent dimension for expert routing and computation, improving accuracy per byte. The Ultra model is pre-trained using an NVFP4 recipe — sharing the quantization-aware pre-training approach pioneered in the Nemotron 3 family. The majority of linear layers use NVFP4 for weights, activations, and gradients, while select layers (including latent projections, MTP layers, QKV/attention projections, and embeddings) are maintained in BF16 or MXFP8 for training stability. The model includes Multi-Token Prediction (MTP) layers using a shared-weight design across prediction heads. This improves training signal quality, enables faster inference via native speculative decoding, and supports more stable autoregressive drafting at longer draft lengths compared to independently trained offset heads.

Training Methodology

Stage 1: Pre-Training

• NVIDIA-Nemotron-3-Ultra-550B-A55B-Base-BF16 model was pre-trained for approximately 20T tokens using crawled and synthetic code, math, science, and general knowledge data. Training leveraged an NVFP4 recipe for efficiency. All datasets are disclosed in the [Training and Evaluation Datasets](#training-and-evaluation-datasets) section of this document. Major portions of the pre-training corpus are released in the Nemotron-Pre-Training-Datasets collection.
• Software used for pre-training: Megatron-LM

Stage 2: Supervised Fine-Tuning

• The model...