NVIDIA/aicr v0.21.0

v0.21.0

Repository: NVIDIA/aicr

Tag: v0.21.0

Published: 2026-09-08T23:57:15Z

Prerelease: no

Release notes: This release focuses on API surface hardening, GB300 and Vera Rubin support, major GPU Operator and Dynamo version bumps, and Kubernetes 1.37 with broad validation and bundler hardening.

Highlights

New Hardware and Platform Support \- GB300 lands on both the cloud and bare-metal (NCP) paths: a generic gb300-generic-ubuntu-training recipe for self-managed Kubernetes (\#2568), EKS training and inference recipes (\#2382) with signed evidence (\#2410), and a Slurm variant (\#2544). Vera Rubin (VR200) arrives as preview overlays on RKE2 (\#2520), with published evidence (\#2573).

Component Upgrades \- GPU Operator moves to v26.7.0 and the DRA driver to 0.5.0, with a workaround for the ComputeDomain CRD conflict v26.7.0 introduces and the driver held at 580 (\#2439). dynamo-platform moves to 1.4.2: the request plane defaults to TCP and KV events to ZMQ, so bundled NATS is no longer deployed \- re-enabling it is opt-in for standing clusters. The same bump takes Grove to v0.1.0-alpha.12 and picks up a NIXL loader-path fix (\#1983).

Recipes and Coverage

• GKE self-installed driver pools now carry the driver in the bundle: gpuStack=bundle-installer replaces driver-installer, pinning the version in the recipe so upgrades roll with the bundle (\#2360)
• OKE gains a gpuStack profile (\#2355), with RDMA fabric wiring for L40S RoCE and GB200 InfiniBand landing alongside it (\#2356)
• An l40-any accelerator overlay and a declared RTX PRO 6000 Server Edition driver floor
• Per-value readiness constraints for configuration profiles

Validation Hardening \- Every NCCL benchmark run gets its own namespace, and cleanup fails the run when that namespace will not terminate. The validator rejects an incompatible StorageClass before creating the model-cache PVC, tolerates tainted nodes with a wider Trainer readiness timeout, retries transient reads while polling gang-scheduling pods, and resolves the GPU-node universe label per service. DaemonSet checks are guarded against stale rollouts, and concurrent snapshot runs no longer collide.

API Surface Hardening \- The four surfaces AICR freezes for v1 \- REST, Go SDK, CLI, and artifact schemas \- are now each held to a committed baseline. The profile-aware /v2 REST family folds into a single /v1 (\#2112) \- a pre-adoption restructure with no consumers to notify. JSON Schemas for the v1 artifacts are published and gated, the per-deployer bundle layout is frozen, and artifacts read both their alpha apiVersion and the maturity-appropriate target from ADR-022. pkg/client/v1 is now the only path the CLI and server take into business logic, enforced by an architecture test. Backing all of it: gates that replay documented REST examples and CLI invocations against real handlers and assert the spec and server agree on routes.

Bundler and Deployer Reliability \- A readiness gate landed for the Helm network-operator, DRA eviction is wired for GPU driver upgrades behind an opt-in node label, and OCP gates the GPU Operator on network readiness. Bundles missing a required node selector now fail instead of shipping, generated wrappers are stamped with both the AICR and payload versions, and Argo CD is forced to replace the readiness-gate Job on upgrade.

Other Improvements \- Kubernetes moves to 1.37: recipe floors are raised to clear the DRA chart's kubeVersion, agentgateway moves to v1.5.0 for the tightened CRD cost budget, client libraries are on v0.37.0, and Kind and KWOK lanes run 1.37 with Argo CD server-side diff. Toolchain moves to Go 1.27.1. CycloneDX SBOMs and OpenVEX documents are published per platform, and vendor/ is gone in favor of Go proxy resolution verified reproducible on every push to main (ADR-023), and ADR-025 proposes an NVIDIA Cluster Readiness Engine component.

*Thanks to* @atif1996, @ayuskauskas, @ezhang3333, @framsouza, @giuliocalzo, @haarchri, @Kevin-Hawkins, @lalitadithya, @lockwobr, @MDhamani, @mikecook, @mohityadav8, @njhensley, @ntheanh201, @rorajani, @srao-nv, @varmesh, @xdu31, @yankay, @yuanchen8911, and @mchmarny.

Changelog

New Features

• dc3818e5ab400822dc6e3f1119d0ad8f47340988: feat(agent): isolate concurrent snapshot runs with per-run resources (#2357) (@ayuskauskas)
• d3801134c0994f99ebb8ad1959f77a8cc1beda4b: feat(api)!: collapse the REST families into a single /v1 (#2464) (@mchmarny)
• 54f31b4fceef11d3e98649547f2e6da1a1dc612d: feat(api): deprecation channel, CLI surface gate, ADR-022 emit guards (#2436) (@mchmarny)
• 3f70c95c81db58b68991cc26348881e5e5cbcfdd: feat(api): gate the REST contract against a committed baseline (#2468) (@mchmarny)
• 482f670cff6217559472328b21016a85db18a628: feat(api): publish and gate JSON Schemas for the v1 artifacts (#2470) (@mchmarny)
• 05b75b79c5f6110e557262daa5b2bd384decc19a: feat(bundler): add readiness gate for the Helm network-operator (#2337) (@framsouza)
• d85b4bafe44531011f3931fc016d720704075c50: feat(bundler): freeze the per-deployer bundle layout (#2475) (@mchmarny)
• cefd89dc8d013cd575093566d0c3f9e7c90d60e2: feat(bundler): wire DRA eviction for GPU driver upgrades (#2401) (@MDhamani)
• 3e8bef2d19790c11b168f692bb2461952cefac34: feat(recipe): per-value readiness constraints for configuration profiles (#2347) (@atif1996)
• 33f0860fcbe583bb57ec54ada090267788dc9328: feat(recipes)!: GKE bundle-installer replaces driver-installer (#2360) (@atif1996)
• ffe0cc27a1ec9106bcb319b53467f6f9c904c458: feat(recipes): OKE RDMA fabric wiring...