NVIDIA/TensorRT

NVIDIA/TensorRT

Description: NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

Language: C++

License: Apache-2.0

Stars: 13102

Forks: 2380

Open issues: 607

Created: 2019-05-02T22:02:08Z

Pushed: 2026-06-25T05:58:05Z

Default branch: main

Fork: no

Archived: no

README:

:mega::mega: Announcement :mega::mega:

TensorRT 11.X is now released with powerful new capabilities designed to accelerate your AI inference workflows. With this major version bump, TensorRT's API has been streamlined and a few legacy features from 10.X have been removed.

Below provides migration guides for the following features:

• Weakly-typed networks and related APIs have been removed, replaced by Strongly Typed Networks.
• Implicit quantization and related APIs have been removed, replaced by Explicit Quantization
• IPluginV2 and related APIs have been removed, replaced by IPluginV3
• TREX tool has been removed, replaced by Nsight Deep Learning Designer
• Python bindings for Python 3.9 and older versions have been removed. RPM packages for RHEL/Rocky Linux 8 and RHEL/Rocky Linux 9 now depend on Python 3.12.

TensorRT Open Source Software

This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. It includes the sources for TensorRT plugins and ONNX parser, as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.

• For step-by-step walkthroughs of the TensorRT import paths (ONNX, Torch-TensorRT, HuggingFace/Optimum, Network Definition API) with examples and tooling tips, see the [Import Workflows Guide](documents/import_workflows.md).
• For the per-model support matrix across import paths (LLM, encoder-NLP, vision, audio, diffusion, multimodal), see [Supported Models](documents/supported_models.md).
• For code contributions to TensorRT-OSS, please see our [Contribution Guide](CONTRIBUTING.md) and [Coding Guidelines](CODING-GUIDELINES.md).
• For a summary of new additions and updates shipped with TensorRT-OSS releases, please refer to the [Changelog](CHANGELOG.md).
• For business inquiries, please contact [researchinquiries@nvidia.com](mailto:researchinquiries@nvidia.com)
• For press and other inquiries, please contact Hector Marinez at [hmarinez@nvidia.com](mailto:hmarinez@nvidia.com)

Need enterprise support? NVIDIA global support is available for TensorRT with the NVIDIA AI Enterprise software suite. Check out NVIDIA LaunchPad for free access to a set of hands-on labs with TensorRT hosted on NVIDIA infrastructure.

Join the TensorRT and Triton community and stay current on the latest product updates, bug fixes, content, best practices, and more.

Agentic Coding Skills

Various skills related to TensorRT usage and benchmarking are available [here](.agents/skills). For installation, refer to the instructions of your preferred coding agent.

Prebuilt TensorRT Python Package

We provide the TensorRT Python package for an easy installation. \ To install:

pip install tensorrt

You can skip the Build section to enjoy TensorRT with Python.

Build

Prerequisites

To build the TensorRT-OSS components, you will first need the following software packages.

TensorRT GA build

• TensorRT v11.1.0.106
• Available from direct download links listed below

System Packages

• CUDA
• Recommended versions:
• cuda-13.3.0
• cuda-12.9.0
• CUDNN (optional)
• cuDNN 8.9
• GNU make >= v4.1
• cmake >= v3.31
• python >= v3.10, = v19.0
• Essential utilities
• git, pkg-config, wget

Optional Packages

• NCCL >= v2.19, = 19.03
• NVIDIA Container Toolkit
• PyPI packages (for demo applications/tests)
• onnx
• onnxruntime
• tensorflow-gpu >= 2.5.1
• Pillow >= 9.0.1
• pycuda NOTE: onnx-tensorrt, cub, and protobuf packages are downloaded along with TensorRT OSS, and not required to be installed.

Downloading TensorRT Build

1. #### Download TensorRT OSS

git clone -b main https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive

2. #### (Optional - if not using TensorRT container) Specify the TensorRT GA release build path

If using the TensorRT OSS build container, TensorRT libraries are preinstalled under /usr/lib/x86_64-linux-gnu and you may skip this step.

Else download and extract the TensorRT GA build from NVIDIA Developer Zone with the direct links below:

• TensorRT 11.1.0.106 for CUDA 13.3, Linux x86_64
• TensorRT 11.1.0.106 for CUDA 12.9, Linux x86_64
• [TensorRT 11.1.0.106...