NVIDIA/go-nvml

NVIDIA/go-nvml

Description: Go Bindings for the NVIDIA Management Library (NVML)

Language: C

License: Apache-2.0

Stars: 444

Forks: 96

Open issues: 24

Created: 2020-12-02T22:08:04Z

Pushed: 2026-06-08T16:15:04Z

Default branch: main

Fork: no

Archived: no

README:

Go Bindings for the NVIDIA Management Library (NVML)

Table of Contents

• [Overview](#overview)
• [Quick Start](#quick-start)
• [How the bindings are generated](#how-the-bindings-are-generated)
• [Code Structure](#code-structure)
• [Code defining the NVML API](#code-defining-the-nvml-api)
• [Code to load libnvidia-ml.so](#code-to-load-libnvidia-mlso)
• [Code to bridge the auto-generated and manual bindings](#code-to-bridge-the-auto-generated-and-manual-bindings)
• [Manual wrappers around the auto-generated bindings from c-for-go](#manual-wrappers-around-the-auto-generated-bindings-from-c-for-go)
• [Test code](#test-code)
• [Building and Testing](#building-and-testing)
• [Updating the Code](#updating-the-code)
• [Update nvml.h](#update-nvmlh)
• [Add new versioned APIs](#add-new-versioned-apis)
• [Add manual wrappers](#add-manual-wrappers)
• [Releasing](#releasing)
• [Contributing](#contributing)

Overview

This repository provides Go bindings for the NVIDIA Management Library API (NVML).

At present, these bindings are only supported on Linux.

These bindings are not a reimplementation of NVML in Go, but rather a set of wrappers around the C API provided by libnvidia-ml.so. This library is part of the standard NVIDIA driver distribution, and should be available on any Linux system that has the NVIDIA driver installed. The API is designed to be backwards compatible, so the latest bindings should work with any version of libnvidia-ml.so installed on your system.

Note: A working NVIDIA driver with libnvidia-ml.so is not required to compile code that imports these bindings. However, you will get a runtime error if libnvidia-ml.so is not available in your library path at runtime.

Please see the following link for documentation on the full NVML Go API:

Quick Start

All you need is a simple import and a call to nvml.Init() to start using these bindings.

The code below shows an example of using these bindings to query all of the GPUs on your system and print out their UUIDs.

package main

import (
"fmt"
"log"

"github.com/NVIDIA/go-nvml/pkg/nvml"
)

func main() {
ret := nvml.Init()
if ret != nvml.SUCCESS {
log.Fatalf("Unable to initialize NVML: %v", nvml.ErrorString(ret))
}
defer func() {
ret := nvml.Shutdown()
if ret != nvml.SUCCESS {
log.Fatalf("Unable to shutdown NVML: %v", nvml.ErrorString(ret))
}
}()

count, ret := nvml.DeviceGetCount()
if ret != nvml.SUCCESS {
log.Fatalf("Unable to get device count: %v", nvml.ErrorString(ret))
}

for i := 0; i )
1. A third-party tool called `c-for-go` ()

Using these tools, we are able to generate a set of Go bindings for NVML, given
nothing more than a specific version of the `nvml.h` header file (which defines
the full NVML API). Most of the process to generate these bindings is
automated, but a few manual steps are required in order to make the generated
bindings more useful from an end user's perspective.

The basic flow to generate the bindings is therefore to:

1. Take the `nvml.h` file and pass it through `c-for-go`
1. Take each of the low-level Go bindings generated by `c-for-go` and wrap them
in a more user-friendly API

As an example, consider the Go bindings generated for the
`nvmlDeviceGetAccountingPids()` API call below:

Original API in `nvml.h`:

nvmlReturn_t nvmlDeviceGetAccountingPids(nvmlDevice_t device, unsigned int *count, unsigned int *pids);

Auto-generated Go bindings from `c-for-go`:

func nvmlDeviceGetAccountingPids(Device Device, Count *uint32, Pids *uint32) Return { cDevice, _ := *(*C.nvmlDevice_t)(unsafe.Pointer(&Device)), cgoAllocsUnknown cCount, _ := (*C.uint)(unsafe.Pointer(Count)), cgoAllocsUnknown cPids, _ := (*C.uint)(unsafe.Pointer(Pids)), cgoAllocsUnknown __ret := C.nvmlDeviceGetAccountingPids(cDevice, cCount, cPids) __v := (Return)(__ret) return __v }

Manual wrapper around the auto-generated bindings:

package nvml

func DeviceGetAccountingPids(Device Device) ([]int, Return) { var Count uint32 = 1 // Will be reduced upon returning for { Pids := make([]uint32, Count) ret := nvmlDeviceGetAccountingPids(Device, &Count, &Pids[0]) if ret == SUCCESS { return uint32SliceToIntSlice(Pids[:Count]), ret } if ret != ERROR_INSUFFICIENT_SIZE { return nil, ret } Count *= 2 } }

func (Device Device) GetAccountingPids() ([]int, Return) { return DeviceGetAccountingPids(Device) }

This manual wrapper makes it so that users don't have to write the boiler-plate
code of figuring out the correct `count` to pass into the API while at the same
time growing the `Pids` array and turning into a slice. It would be used as
follows:

device, _ := nvml.DeviceGetHandleByIndex(0) pids, _ := device.GetAccountingPids() ...

This is actually one of the more complicated examples. Most of the
manual wrappers are very simple and look similar to the following:

Original API in `nvml.h`:

nvmlReturn_t nvmlDeviceGetUUID(nvmlDevice_t device, char *uuid, unsigned int length);

Auto-generated Go bindings from `c-for-go`:

func nvmlDeviceGetUUID(Device Device, Uuid *byte, Length uint32) Return { cDevice, _ := *(*C.nvmlDevice_t)(unsafe.Pointer(&Device)), cgoAllocsUnknown cUuid, _ := (*C.char)(unsafe.Pointer(Uuid)), cgoAllocsUnknown cLength, _ := (C.uint)(Length), cgoAllocsUnknown __ret := C.nvmlDeviceGetUUID(cDevice, cUuid, cLength) __v := (Return)(__ret) return __v }

Manual wrapper around the auto-generated bindings:

package nvml

func DeviceGetUUID(Device Device) (string, Return) { Uuid := make([]byte, DEVICE_UUID_BUFFER_SIZE) ret := nvmlDeviceGetUUID(Device, &Uuid[0], DEVICE_UUID_BUFFER_SIZE) return string(Uuid[:clen(Uuid)]), ret }

func (Device Device) GetUUID() (string, Return) { return DeviceGetUUID(Device) }

While it does take some effort to take the auto-generated bindings and manually
wrap them in the more user-friendly API, this only has to be done once per API
call and then never touched again. As such, as new release of NVML come out,
only the new API calls will need to be added.

The following section goes into the details of how the code is structured, and
what each…