NVIDIA/TensorRT-LLM v1.1.0rc5

v1.1.0rc5

Repository: NVIDIA/TensorRT-LLM

Tag: v1.1.0rc5

Published: 2025-09-18T01:49:03Z

Prerelease: yes

Release notes:

Announcement Highlights

• Model Support
• Enable NvFP4/FP8 quantization for Nemotron-H architecture (#7589)
• Enable KV-cache reuse and add E2E tests for llava-next (#7349)
• Support gpt-oss with fp8 kv cache (#7612)
• Support kvcache reuse for phi4mm (#7563)
• API
• Add TorchLlmArgs to the connector api (#7493)
• Benchmark
• Extend test_perf.py to add disagg-serving perf tests (#7503)
• Add accuracy test for deepseek-r1 with chunked_prefill (#7365)
• Feature
• Optimize MLA kernels with separate reduction kernels (#7597)
• Wrap MOE with custom op (#7277)
• Make the should_use_spec_decode logic a bit smarter (#7112)
• Use a shell context to install dependancies (#7383)
• Topk logprobs for TRT backend and top1 logprob for PyT backend (#6097)
• Support chunked prefill for multimodal models (#6843)
• Optimize MLA chunked prefill && support fp8 mla chunked prefill (#7477)
• Disable deep_gemm for Qwen3 QKNormRoPEAttention and Linear layers due to accuracy issues (#7616)
• Add deepseek r1-w4afp8 quickstart (#7645)
• Nanobind: Allow none types for fields in result (#7672)
• Using arrival time in llmapi when creating LlmRequest in pytorch workflow (#7553)
• UCX zmq ip support ipv6 (#7530)
• Refactor: Quantization Transforms with Inheritance (#7227)

What's Changed

• [None][chore] Remove closed bugs by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/7591
• [https://nvbugs/5454559][fix] handle bias term in fuse_gate_mlp by @Linda-Stadter in https://github.com/NVIDIA/TensorRT-LLM/pull/7449
• [None][fix] enable NvFP4/FP8 quantization for Nemotron-H architecture by @tomeras91 in https://github.com/NVIDIA/TensorRT-LLM/pull/7589
• [None][feat] Optimize MLA kernels with separate reduction kernels by @PerkzZheng in https://github.com/NVIDIA/TensorRT-LLM/pull/7597
• [https://nvbugs/5445466][fix] unwaive DS R1 test cases with bug already fixed by @lancelly in https://github.com/NVIDIA/TensorRT-LLM/pull/7429
• [#6798][fix] fix compilation error in ub_allocator in single device build by @WilliamTambellini in https://github.com/NVIDIA/TensorRT-LLM/pull/6874
• [https://nvbugs/5434424][fix] A quick fix for the wrong output issue of SM89 blocked scaling batched GEMM when the input tensor is non-contiguous. by @StudyingShao in https://github.com/NVIDIA/TensorRT-LLM/pull/7615
• [None][chore] add TorchLlmArgs to the connector api by @richardhuo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/7493
• [TRTLLM-6707][fix] nanobind fix for executor exit call by @Linda-Stadter in https://github.com/NVIDIA/TensorRT-LLM/pull/7565
• [None][ci] add DGX_H100-2_GPUs-PyTorch-Others-1 pipeline by @QiJune in https://github.com/NVIDIA/TensorRT-LLM/pull/7629
• [TRTLLM-7408][feat] Wrap MOE with custom op. by @liji-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/7277
• [TRTLLM-5059][feat] Enable KV-cache reuse and add E2E tests for llava-next by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/7349
• [None][fix] fix post-merge issue raised by #5488 by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/7655
• [https://nvbugs/5410687][test] Add deepseek r1-w4afp8 quickstart by @fredricz-20070104 in https://github.com/NVIDIA/TensorRT-LLM/pull/7645
• [None][fix]UCX zmq ip support ipv6 by @chuangz0 in https://github.com/NVIDIA/TensorRT-LLM/pull/7530
• [None][feat] Make the should_use_spec_decode logic a bit smarter by @zheyuf in https://github.com/NVIDIA/TensorRT-LLM/pull/7112
• [#5861][autodeploy] Refactor: Quantization Transforms with Inheritance by @Fridah-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/7227
• [#7208][fix] Fix config type of MedusaConfig by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/7320
• [None][infra] Bump version to 1.1.0rc5 by @yiqingy0 in https://github.com/NVIDIA/TensorRT-LLM/pull/7668
• [TRTLLM-7871][infra] Extend test_perf.py to add disagg-serving perf tests. by @bo-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/7503
• [https://nvbugs/5494698][fix] skip gemma3 27b on blackwell by @xinhe-nv in https://github.com/NVIDIA/TensorRT-LLM/pull/7505
• [https://nvbugs/5477359][fix] Nanobind: Allow none types for fields in result by @Linda-Stadter in https://github.com/NVIDIA/TensorRT-LLM/pull/7672
• [None][chore] remove executor config in kv cache creator by @leslie-fang25 in https://github.com/NVIDIA/TensorRT-LLM/pull/7526
• [https://nvbugs/5488212][waive] Waive failed tests for L20 by @nvamyt in https://github.com/NVIDIA/TensorRT-LLM/pull/7664
• [None][feat] Use a shell context to install dependancies by @v-shobhit in https://github.com/NVIDIA/TensorRT-LLM/pull/7383
• [https://nvbugs/5505402] [fix] Disable deep_gemm for Qwen3 QKNormRoPEAttention and Linear layers due to accuracy issues by @DomBrown in https://github.com/NVIDIA/TensorRT-LLM/pull/7616
• [None][infra] Waive failed cases on main 0910 by @EmmaQiaoCh in https://github.com/NVIDIA/TensorRT-LLM/pull/7676
• [None][infra] Adjust labeling llm prompt for bug issues by @karljang in https://github.com/NVIDIA/TensorRT-LLM/pull/7385
• [None][ci] move some test cases from l40s to a30 by @QiJune in https://github.com/NVIDIA/TensorRT-LLM/pull/7684
• [None][fix] Fix the incorrect header file import in dataType.h by @Fan-Yunfan in https://github.com/NVIDIA/TensorRT-LLM/pull/7133
• [https://nvbugs/5498165][fix] fix permission error for config file lock by @chang-l in https://github.com/NVIDIA/TensorRT-LLM/pull/7656
• [https://nvbugs/5513192][fix] Add the missing param for kv_cache_tran… by @nv-guomingz in https://github.com/NVIDIA/TensorRT-LLM/pull/7679
• [TRTLLM-1302][feat] Topk logprobs for TRT backend and top1 logprob for PyT backend by @LinPoly in https://github.com/NVIDIA/TensorRT-LLM/pull/6097
• [TRTLLM-7169][infra] Fix Slurm multi-node test showing "Submit Test Results" in the test name by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/6856
• [TRTLLM-6791][infra] Add check for uploading stage name and avoid overriding test result tar file by @ZhanruiSunCh in https://github.com/NVIDIA/TensorRT-LLM/pull/6742
• [None][ci] Some improvements for Slurm CI by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/7689
• [None][ci] Test waives for the main branch 09/14 by @chzblych in https://github.com/NVIDIA/TensorRT-LLM/pull/7698
• [None][feat] support gpt-oss with fp8 kv cache by @PerkzZheng in https://github.com/NVIDIA/TensorRT-LLM/pull/7612
• [TRTLLM-6903][feat]...