NVIDIA/TensorRT-LLM v1.1.0

v1.1.0

Repository: NVIDIA/TensorRT-LLM

Tag: v1.1.0

Published: 2025-12-19T13:55:05Z

Prerelease: no

Release notes:

Know Issue

• If users create project with tensorrt-llm==1.1.0 in pyproject.toml file as dependency as below:

dependencies = [
"tensorrt-llm==1.1.0",
]

when users install project dependencies with command uv sync, error will happend with message:

No solution found when resolving dependencies for split (markers: python_full_version >= '3.13' and sys_platform == 'darwin'):
╰─▶ Because patchelf==0.18.0.0 was yanked (reason: https://github.com/mayeut/patchelf-pypi/issues/87) and tensorrt-llm==1.1.0 depends on patchelf==0.18.0, we can
conclude that tensorrt-llm==1.1.0 cannot be used.
And because your project depends on tensorrt-llm==1.1.0, we can conclude that your project's requirements are unsatisfiable.".

That's because patchelf 0.18.0 was yanked by author.

A valid work around for this issue is to add block in pyproject.toml:

[tool.uv]
override-dependencies = [
"patchelf==0.17.2.4",
]

Key Features and Enhancements

• Model Support
• Add model supports of GPT-OSS, Hunyuan-Dense (contribution from @sorenwu), Hunyuan-MoE (contribution from @qianbiaoxiang), Seed-OSS (contribution from @Nekofish-Li).
• Features
• Connector API: Introduced a new KV Cache Connector API for state transfer in disaggregated serving.
• Reuse & Offloading: Enabled KV cache reuse for MLA (Multi-Head Latent Attention) and added examples for host offloading.
• Salting: Implemented KV cache salting for secure cache reuse.
• Guided Decoding Integration: Enabled guided decoding to work in conjunction with speculative decoding (including 2-model and draft model chunked prefill).
• Eagle: Added multi-layer Eagle support and optimizations.
• CuteDSL: Integrated CuteDSL NVFP4 grouped GEMM for Blackwell.
• B300/GB300: Added support for B300/GB300.
• Documentation
• Deployment Guides: Added comprehensive deployment guides for GPT-OSS, DeepSeek-R1, and VDR 1.0.
• Feature Documentation: Created new documentation for KV Cache Connector, LoRA feature usage, and AutoDeploy.
• Tech Blogs: Published blogs on “Combining Guided Decoding and Speculative Decoding” and “ADP Balance Strategy”.
• Quick Start: Refined Quick Start guides with new links to ModelOpt checkpoints and updated installation steps (Linux/Windows).
• API Reference: Enhanced LLM API documentation by explicitly labeling stable vs. unstable APIs.
• Performance: Updated online benchmarking documentation and performance overview pages.
• Examples: Refined Slurm examples and added K2 tool calling examples.
• Infrastructure Changes
• The base Docker image for TensorRT-LLM: nvcr.io/nvidia/pytorch:25.10-py3.
• The base Docker image for TensorRT-LLM Backend: nvcr.io/nvidia/tritonserver:25.10-py3.
• The dependent public PyTorch version: 2.9.0.
• The dependent NVIDIA ModelOpt version: 0.37.
• The dependent xgrammar version: 0.1.25.
• The dependent transformers version: 4.56.0.
• The dependent NIXL version: 0.5.0.
• API Changes
• Breaking Change: The C++ TRTLLM sampler is now enabled by default, replacing the legacy implementation.
• KV Cache Connector API: Introduced a new KV Cache Connector API.
• Standardized topk logprob returns across TRT and PyTorch backends.
• Added stable labels to arguments in the LLM class to better indicate API stability.
• Wait and Cancel API: Added tests and support for handling non-existent and completed request cancellations in the executor.
• Fixed multiple Issues
• illegal memory access, weight loading issues for DeepSeek-R1 W4A8, CUDA graph warmup issues with speculative decoding, memory leaks
• Known Issues
• GB300 Multi-Node: Support for GB300 in multi-node configurations is currently in beta and not fully validated in this release. GB300 multi-node configurations have been validated in 1.2.0rc4+.

What's Changed

• [None][chore] Bump version to 1.1.0rc0 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/6651
• [TRTLLM-6683][feat] Support LoRA reload CPU cache evicted adapter by @amitz-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/6510
• [None][test] correct test-db context for perf yaml file by @ruodil in https://github.com/NVIDIA/TensorRT-LLM/pull/6686
• [None] [feat] Add model gpt-oss by @hlu1 in https://github.com/NVIDIA/TensorRT-LLM/pull/6645
• [https://nvbugs/5409414][fix] fix Not registered specs by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/6660
• [None][feat] : Add FP8 context MLA support for SM120 by @peaceh-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/6059
• [TRTLLM-6092][doc] Add LoRA feature usage doc by @shaharmor98 in https://github.com/NVIDIA/TensorRT-LLM/pull/6603
• [TRTLLM-6409][feat] Enable guided decoding with speculative decoding (part 1: two-model engine) by @syuoni in https://github.com/NVIDIA/TensorRT-LLM/pull/6300
• [TRTLLM-6881][feat] Include attention dp rank info with KV cache events by @pcastonguay in https://github.com/NVIDIA/TensorRT-LLM/pull/6563
• [None][infra] Fix guardwords by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/6711
• [None][package] Pin cuda-python version to >=12,<13 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/6702
• [None][doc] Add deployment guide section to the official doc website by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/6669
• [None][fix] disagg ctx pp4 + gen pp4 integ test by @raayandhar in https://github.com/NVIDIA/TensorRT-LLM/pull/6489
• [None][feat] Clean up ngram auto mode, add max_concurrency to configs by @mikeiovine in https://github.com/NVIDIA/TensorRT-LLM/pull/6676
• [None][chore] Remove py_executor from disagg gh team by @pcastonguay in https://github.com/NVIDIA/TensorRT-LLM/pull/6716
• [https://nvbugs/5423962][fix] Address broken links by @chenopis in https://github.com/NVIDIA/TensorRT-LLM/pull/6531
• [None][fix] Migrate to new cuda binding package name by @tongyuantongyu in https://github.com/NVIDIA/TensorRT-LLM/pull/6700
• [https://nvbugs/5410687][fix] Hopper w4a8 groupwise MoE interleave by @symphonylyh in https://github.com/NVIDIA/TensorRT-LLM/pull/6708
• [None][feat] Add NCCL Symmetric Integration for All Reduce by @Tabrizian in https://github.com/NVIDIA/TensorRT-LLM/pull/4500
• [TRTLLM-6785][feat] BREAKING CHANGE Enable TRTLLM sampler by default by @dcampora in https://github.com/NVIDIA/TensorRT-LLM/pull/6216
• [TRTQA-2920][fix] Add failed cases into waives.txt by @xinhe-nv in...