NVIDIA/TensorRT-LLM v1.2.0rc0.post1

v1.2.0rc0.post1

Repository: NVIDIA/TensorRT-LLM

Tag: v1.2.0rc0.post1

Published: 2025-10-14T09:17:22Z

Prerelease: yes

Release notes:

Announcement Highlights

• Model Support
• Support Qwen3 next (#7892)

• API
• Return topk logprobs in torch backend (#7976)
• Add chunked return_generation_logits logic (#7831)

• Benchmark
• Lock gpu clocks in test_perf.py to reliably detect perf regressions (#8099)
• Fixed the kv cache size parsing in test_perf.py AD backend (#8092)
• Improve perf_metrics endpoint functionality (#8005)

• Feature
• AutoDeploy: Linear Attention Support (SSM + causal_conv + Bamba + Nemotron-H) (#8068)
• add W4A8 NVFP4 FP8 fused moe (#7968)
• Add ModelOPT INT4 awq fake quant support in AutoDeploy (#7770)
• Save state first pass for speculative decoding (#7012)
• Executor changes to support helix parallelism (#7972)
• Support CUDA graph for DeepEP (#7514)
• Integrate tinygemm2 for gpt-oss (#7916)
• Support for cancelling requests with disaggregation (#8114)
• Update TRT-LLM Gen MoE kernels (#7970)
• AutoDeploy: dive deeper into token generation bugs + enable_block_reuse (#8108)
• AutoDeploy add autotuning when capturing cudagraphs (#8120)
• AutoDeploy: compiler backends based on nn.Module (#8126)
• Add heuristics for lm head tp size when enable_lm_head_tp_in_adp=True (#7891)
• Improve batched sampling perf for contiguous batches (#7908)

• Documentation
• Add more description on EXAONE usage (#8089)

• Fix & Infra
• Fix CUDA graph for Qwen2.5-VL (#8047)
• Refine qwen3-next implementation (#8064)
• Patched incorrect starcoder tp config (#8118)
• Fix Qwen3 FP8 per-tensor when requesting TRTLLM-GEN MoE backend (#8075)
• Fix TRT-python multi LoRA TP=2 test arguments (#8059)
• Fix the non-determinism issue in the mm_encoder test (#8033)
• Checking connection to etcd server in unit test (#8006)
• Add MNNVL AlltoAll tests to pre-merge (#7466)
• Add test cases into QA test list (#8081)
• Avoid downloading Tiny llama from HF (#8071)
• Fix OOM issue when dp padding is enabled (#8052)
• Fix unwaiving disagg pp tests (#8069)
• Fix shape propagation after TP sharding (#7912)
• Fix patchelf version issue (#8112)
• Fix device id assignment for some vision models (#8070)
• Do not explicitly pass temperature=0 to select greedy sampling (#8110)
• Fix access to new tokens in sampler (#7958)
• Adding install_tensorrt.sh script to pip wheel (#8116)
• Fix flakey unit test for dynamic spec decoding (#8129)
• Minor cleanup and improvements (#7619)
• Reserve an extra slot for padded batch (#7998)
• Fix MTP 2-model (#8115)
• Add LoRa Torch tests for the latest NIM model list (#6806)

What's Changed

• [TRTLLM-7728][perf] improve batched sampling perf for contiguous batches by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/7908
• [None][feat] Support Qwen3 next by @byshiue in https://github.com/NVIDIA/TensorRT-LLM/pull/7892
• [TRTLLM-8269][test] do not explicitly pass temperature=0 to select greedy sampling by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/7909
• [None][fix] Fix TRT-python multi LoRA TP=2 test arguments by @amitz-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/8059
• [https://nvbugs/5542867][fix] Fix the non-determinism issue in the mm_encoder test by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/8033
• [https://nvbugs/5538098][fix] Checking connection to etcd server in unit test by @pcastonguay in https://github.com/NVIDIA/TensorRT-LLM/pull/8006
• [TRTLLM-6741][fix] Add heuristics for lm head tp size when enable_lm_head_tp_in_adp=True by @Njuapp in https://github.com/NVIDIA/TensorRT-LLM/pull/7891
• [None][feat] Return topk logprobs in torch backend by @dcaox in https://github.com/NVIDIA/TensorRT-LLM/pull/7976
• [#4593][feat] AutoDeploy: Linear Attention Support (SSM + causal_conv + Bamba + Nemotron-H) by @lucaslie in https://github.com/NVIDIA/TensorRT-LLM/pull/8068
• [None] [test] Add MNNVL AlltoAll tests to pre-merge by @kaiyux in https://github.com/NVIDIA/TensorRT-LLM/pull/7466
• [TRTLLM-6239][feat] add test cases into QA test list by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/8081
• [None][fix] Fix CUDA graph for Qwen2.5-VL by @yechank-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/8047
• [None][chore] Bump version to 1.2.0rc1 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/8097
• [https://nvbugs/5547414][fix] avoid downloading Tiny llama from HF by @Tabrizian in https://github.com/NVIDIA/TensorRT-LLM/pull/8071
• [None][chore] Refine qwen3-next implementation. by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/8064
• [TRTLLM-8269][fix] Revert "do not explicitly pass temperature=0 to select greedy sampling" by @ixlmar in https://github.com/NVIDIA/TensorRT-LLM/pull/8103
• [None][chore] Revert MNNVL alltoall MR by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/8106
• [None][fix] : Fix OOM issue when dp padding is enabled by @peaceh-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/8052
• [None][doc] Add more description on EXAONE example by @yechank-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/8089
• [None][infra] Skip failed tests in post-merge for main by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/8102
• [https://nvbugs/5434320][fix] fix: Unwaiving disagg pp tests by @pcastonguay in https://github.com/NVIDIA/TensorRT-LLM/pull/8069
• [OMNIML-2336][feat] add W4A8 NVFP4 FP8 fused moe by @sychen52 in https://github.com/NVIDIA/TensorRT-LLM/pull/7968
• [TRTLLM-6342][bug] Fix shape propagation after TP sharding by @greg-kwasniewski1 in https://github.com/NVIDIA/TensorRT-LLM/pull/7912
• [TRTLLM-8031][feat] Add chunked return_generation_logits logic by @yibinl-nvidia in https://github.com/NVIDIA/TensorRT-LLM/pull/7831
• [#5860][feat] Add ModelOPT INT4 awq fake quant support in AutoDeploy by @Fridah-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/7770
• [None][fix] fix patchelf version issue by @bo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/8112
• [None][feat] Save state first pass by @IzzyPutterman in https://github.com/NVIDIA/TensorRT-LLM/pull/7012
• [TRTLLM-7733][feat] Executor changes to support helix parallelism by @brb-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/7972
• [https://nvbugs/5549081][fix] Fix device id assignment for some vision models by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/8070
• [#7588][feat] lock gpu clocks in test_perf.py to reliably detect perf regressions by @MrGeva in...