NVIDIA/TensorRT-RTX-EP-ABI v0.1.0

TensorRT RTX Execution Provider ABI Package (trt-rtx-ep-abi) v0.1.0

Repository: NVIDIA/TensorRT-RTX-EP-ABI

Tag: v0.1.0

Published: 2026-04-09T19:14:59Z

Prerelease: no

Release notes:

TensorRT RTX Execution Provider ABI Package (trt-rtx-ep-abi) v0.1.0

onnxruntime TensorRT RTX EP ABI (trt-rtx-ep-abi)

The NVIDIA TensorRT RTX Execution Provider (EP) ABI package is a standalone plugin (onnxruntime_providers_nv_tensorrt_rtx.dll / libonnxruntime_providers_nv_tensorrt_rtx.so) that implements the ORT EP ABI interfaces introduced in ORT 1.23.0. It is designed specifically for NVIDIA RTX GPUs and optimized for client-centric use cases.

Unlike the in-tree EP bundled with ONNX Runtime, this ABI package is built and versioned independently — it does not need to be built together with ONNX Runtime.

The TensorRT RTX EP leverages NVIDIA's TensorRT for RTX engine to accelerate ONNX models on RTX GPUs. It supports RTX GPUs based on Ampere and later architectures (NVIDIA GeForce RTX 30xx and above).

Benefits:

• Small package footprint: Optimized resource usage on end-user systems at just under 200 MB.
• Faster model compile and load times: Leverages just-in-time compilation to build RTX hardware-optimized engines on end-user devices in seconds.
• Portability: Seamlessly use cached models across multiple RTX GPUs.
• Independent versioning: Ship EP updates without requiring a new ONNX Runtime build.

Feature Support

• CUDA Graph — Reduced CPU overhead through CUDA graph capture and replay.
• EP Context Model — Save and reload pre-compiled engine context for faster session initialization.
• Runtime Cache — Cache compiled engines across sessions to avoid redundant compilation.
• GPU I/O Binding — Bind inputs and outputs directly on GPU memory to eliminate host-device copies.
• Optimized Memory Management — Efficient GPU memory allocation and reuse.
• Intermediate Memory Sharing Across Sessions — Multiple inference sessions running on the same device can share intermediate GPU memory buffers, significantly reducing overall GPU memory consumption when hosting multiple models concurrently.
• Device Compatibility API (IHV Support) — Implements the ORT Device Compatibility API (GetHardwareDeviceSupportStatus in OrtEpFactory). The EP reports machine-readable compatibility verdicts for each hardware device, including structured incompatibility reasons (driver version, unsupported device, missing features, policy blocks, runtime conflicts) via OrtEpDeviceCompatibilityDetails. GetEpDevices() only returns devices that are truly supported under current system conditions.
• V2 Device-Based EP API — Uses the modern SessionOptionsAppendExecutionProvider_V2 API with device enumeration.
• Secure DLL Loading — Delay-loaded TensorRT RTX DLLs with production signature verification (Windows).

Known Limitations

• D3D12 Interop is not supported — The EP ABI package does not currently support Direct3D 12 resource interop. Applications requiring D3D12 tensor sharing or D3D12 fence synchronization should use the in-tree TensorRT RTX EP bundled with ONNX Runtime instead.

Dependencies

| Component | Minimum Version | |-----------|-----------------| | ONNX Runtime SDK | 1.24.0+ | | CUDA Toolkit | 12.9+ | | TensorRT RTX SDK | 1.4.x.x | | CMake | 3.15+ | | Visual Studio (Windows) | 2019 or 2022 | | GCC / Clang (Linux) | C++17-capable |

Getting Started

Run via Python

import onnxruntime as ort

# 1. Register the EP plugin DLL
ort.register_execution_provider_library(
"NvTensorRTRTXExecutionProvider",
"onnxruntime_providers_nv_tensorrt_rtx.dll")

# 2. Discover the TensorRT RTX EP device
ep_devices = ort.get_ep_devices()
trt_device = None
for ep_device in ep_devices:
if ep_device.ep_name == "NvTensorRTRTXExecutionProvider":
trt_device = ep_device
break

# 3. Add EP device to session options with provider options
session_options = ort.SessionOptions()
session_options.add_provider_for_devices(
[trt_device],
{"nv_runtime_cache_path": "./cache"})

# 4. Create session and run inference
session = ort.InferenceSession("model.onnx", sess_options=session_options)
result = session.run([], {"input": input_data})

# 5. Cleanup: delete session before unregistering
del session
ort.unregister_execution_provider_library("NvTensorRTRTXExecutionProvider")

Run via C++

#include

OrtApi const& ortApi = Ort::GetApi();
Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "MyApp");
Ort::SessionOptions session_options;

// 1. Register the EP plugin library
ortApi.RegisterExecutionProviderLibrary(
env, "NvTensorRTRTXExecutionProvider",
ORT_TSTR("onnxruntime_providers_nv_tensorrt_rtx.dll"));

// 2. Enumerate available EP devices and find TensorRT RTX
const OrtEpDevice* const* ep_devices = nullptr;
size_t num_ep_devices;
ortApi.GetEpDevices(env, &ep_devices, &num_ep_devices);

const OrtEpDevice* trt_device = nullptr;
for (size_t i = 0; i < num_ep_devices; i++) {
if (strcmp(ortApi.EpDevice_EpName(ep_devices[i]),
"NvTensorRTRTXExecutionProvider") == 0) {
trt_device = ep_devices[i];
break;
}
}

// 3. Append the EP with provider options
const char* keys[] = {"enable_cuda_graph"};
const char* values[] = {"1"};
ortApi.SessionOptionsAppendExecutionProvider_V2(
session_options, env, &trt_device, 1,
keys, values, 1);

// 4. Create session
Ort::Session session(env, ORT_TSTR("model.onnx"), session_options);

See the [C++ examples](examples/cxx/README.md) for full samples demonstrating EP device selection, device tensors, CUDA stream interop, and EP context model workflows.

Build from Source

Clone the Repository

git clone https://github.com/NVIDIA/TensorRT-RTX-EP-ABI.git
cd TensorRT-RTX-EP-ABI

Build

Windows

cmake -B build -G "Visual Studio 17 2022" -A x64 `
-DONNXRUNTIME_ROOT="C:\SDK\onnxruntime-win-x64-1.24.0" `
-DTRT_RTX_ROOT="C:\SDK\TensorRT-RTX-1.1.1.36"
cmake --build build --config Release

Linux

cmake -B build \
-DONNXRUNTIME_ROOT=/path/to/onnxruntime \
-DTRT_RTX_ROOT=/path/to/tensorrt-rtx
cmake --build build

The output library is at:

• Windows: build\Release\onnxruntime_providers_nv_tensorrt_rtx.dll
• Linux: build/libonnxruntime_providers_nv_tensorrt_rtx.so

Documentation

For detailed documentation on features, execution provider options, and API usage, see the official…