NVIDIA/k8s-nim-operator

NVIDIA/k8s-nim-operator

Description: An Operator for deployment and maintenance of NVIDIA NIMs and NeMo microservices in a Kubernetes environment.

Language: Go

License: Apache-2.0

Stars: 161

Forks: 45

Open issues: 23

Created: 2024-07-30T16:23:54Z

Pushed: 2026-06-10T13:43:02Z

Default branch: main

Fork: no

Archived: no

README:

NVIDIA NIM Operator

NVIDIA NIM Operator is a Kubernetes Operator that is designed to facilitate the deployment, management, and scaling of NVIDIA NIM microservices on Kubernetes clusters.

NVIDIA NIM microservices deliver AI foundation models as accelerated inference microservices that are portable across data center, workstation, and cloud, accelerating flexible generative AI development, deployment and time to value.

To use the Operator in your cluster, refer to docs for installation and configuration information.

Developing the Operator

Prerequisites

• Kubernetes v1.28 and higher.
• NVIDIA GPUs that are supported by the NIM microservices to deploy.

Deploying the Operator on the Cluster

Build and push your image to the location specified by `IMAGE_NAME`:`VERSION`:

make build IMAGE_NAME=/k8s-nim-operator VERSION= -f deployments/container/Makefile

> Publish the image to a personal registry. > You must be able to pull the image from the working environment. > Make sure you have the proper permission to the registry if the preceding commands result in an error.

Install the CRDs into the cluster:

make install

Deploy the manager to the cluster with the image specified by `IMG`:

make deploy IMG=/k8s-nim-operator:

> If you encounter RBAC errors, you might need to grant yourself cluster-admin privileges or be logged in as admin. > Ensure that the samples have default values.

Deploying Sample NIMCache, NIMService, NIMPipeline, NemoCustomizer, NemoEvaluator and NemoGuardrail Resources

Follow the guides in the docs to deploy sample CR instances.

Uninstalling the Operator

Delete the instances (CRs) from the cluster:

kubectl delete -k config/samples/

Delete the APIs (CRDs) from the cluster:

make uninstall

Undeploy the controller from the cluster:

make undeploy

Project Distribution

Perform the following steps to build the installation manifests and distribute this project to users.

1. Build the manifests for the image built and published in the registry:

make build-installer IMG=/k8s-nim-operator:

The preceding Makefile target generates a dist/install.yaml file. This file is built with Kustomize and contains the manifests for the CRDs and resources that are necessary to install this project without its dependencies.

2. Run the installer:

kubectl apply -f https://raw.githubusercontent.com//k8s-nim-operator//dist/install.yaml

Contributing

NVIDIA can work with partners to add platform support for the NIM Operator. The NIM Operator is open-source and permissively [licensed](LICENSE.md) with only minimal requirements for source code [contributions](CONTRIBUTING.md).

To get started with building the NIM Operator, follow these steps:

git clone git@github.com:NVIDIA/k8s-nim-operator.git
cd k8s-nim-operator
make build IMAGE_NAME=/k8s-nim-operator VERSION= -f deployments/container/Makefile

Run make help for more information about additional make targets.

More information can be found in the Kubebuilder Documentation.