coreweave/inference-cookbook

coreweave/inference-cookbook

Description: Runnable RL recipes (miles, slime, NeMo-RL) and the cwtrain library for training against CoreWeave Dedicated Inference with RL Rollouts.

Language: Python

License: Apache-2.0

Stars: 2

Forks: 0

Open issues: 1

Created: 2026-10-12T21:11:14Z

Pushed: 2026-10-14T01:48:46Z

Default branch: main

Fork: no

Archived: no

README:

CoreWeave Inference Cookbook

Runnable reinforcement-learning (RL) recipes for training against a live CoreWeave Dedicated Inference deployment, and cwtrain, the Python library and CLI that the recipes share.

The recipes use RL Rollouts, the Dedicated Inference capability for serving rollouts with checkpoint hot-load. After every training step the trainer publishes its new weights. The deployment swaps them in place on every replica, and the next rollout batch comes from its OpenAI-compatible endpoint. The trainer runs no inference engines of its own.

cwtrain does the CoreWeave side of a run. It creates the deployment that the run samples from, and deletes it at the end. It publishes the trainer's weights to the deployment by checkpoint hot-load and samples rollouts from its endpoint. It integrates with miles, slime and NeMo-RL. The NeMo Gym evaluation recipe uses it to create and clean up its deployment.


Why train against a hot-load deployment

• No inference engines in the trainer. Rollouts come from a deployment that runs and scales

on its own GPUs, behind the same OpenAI-compatible API as any other Dedicated Inference deployment.

• On-policy rollouts. A hot-load completes only when every replica serves the new weights, and

the integrations wait for that before the next rollout batch.

• Small, verified updates. After one full checkpoint, each step uploads only the tensors that

changed. It sends them as a compressed delta with a checksum per tensor. The engine verifies each changed tensor before it serves the new weights. A full checkpoint every N steps keeps the chain of deltas short.

• Your framework, or your own loop. The library integrates with miles, slime and NeMo-RL, and

its CLI and Python API work from any other training loop.

Recipes

| Recipe | Framework | Algorithm | Upstream changes | Tested with | | --- | --- | --- | --- | --- | | [miles](recipes/training/miles/README.md) | miles | GRPO, synchronous or one step ahead | None: miles calls the cwtrain hook after each weight sync | radixark/miles:v0.1.1 | | [slime](recipes/training/slime/README.md) | slime | GRPO | Three small patches, applied by the launcher, so slime publishes weights without inference engines of its own | slime 8ee9c1e | | [NeMo-RL](recipes/training/nemo-rl/README.md) | NeMo-RL | GRPO | Patches, applied by the launcher, that register the coreweave backend and optional Gym rollouts | NeMo-RL v0.7.0 | | [NeMo Gym](recipes/evaluation/nemo-gym/README.md) | NeMo Gym | Evaluation with scored rollouts | None: cwtrain provisions and cleans up the gateway and deployment; Gym runs the agent and scores responses | NeMo Gym v0.6.0 |

The training recipes use DAPO-Math-17k and run on Kubernetes. NeMo-RL uses Nemotron 3.5 Lightning BF16; slime and miles use Qwen3-4B. Each ships manifests, an example deployment request and an example bucket policy.

Before you start

• A CoreWeave account with Dedicated Inference access, and **RL Rollouts enabled for your

organization**. Until it is enabled, creating a deployment with a hotLoad section fails with hot-load is not enabled for this organization.

• Inference GPU quota for the instance type you deploy on.
• A CoreWeave API token with the Inference Admin role, to create deployments and start

hot-loads. Polling hot-loads needs only the Inference Viewer role.

• An AI Object Storage bucket that

holds the base model. Its bucket policy must let the Dedicated Inference service account read the bucket (Grant inference access to your bucket) and let your trainer write snapshots under the hot-load prefix. The trainer needs no delete or policy rights, and a principal of its own: the recipes run it as a Kubernetes service account with Workload Identity Federation. A bucket policy makes the bucket explicit-allow only. Keep your organization's statement in the policy, or all other access to the bucket stops. Each recipe ships a bucket-policy.example.json with this structure.

• A gateway, and a deployment created with engine dynamo-vllm and a hotLoad section naming the

bucket and path prefix for snapshots ([example request](recipes/training/miles/deployment.example.json)). The recipes use cwtrain deployment create to create it from such a request. You can also use the API, or the Hot loading section of the Cloud Console's create drawer. You can enable hot-load only when you create a deployment; you cannot add it later. All of the deployment's gateways must be in one zone.

• Python 3.12 or newer.
• For the recipes: kubectl access to a CoreWeave Kubernetes cluster with GPU nodes for the

trainer.

Quick start: create a deployment and hot-load a checkpoint

Install the library and its CLI from PyPI:

pip install cwtrain

Point it at your account and bucket:

export COREWEAVE_API_TOKEN=... # API token with the Inference Admin role
export AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=... # AI Object Storage key that can write the bucket
export AWS_REGION= # the bucket's Availability Zone, e.g. US-EAST-04A

cwtrain also reads CW_API_TOKEN, the name that the CoreWeave documentation uses, when COREWEAVE_API_TOKEN is not set.

Create the deployment from a request body like a recipe's [deployment.example.json](recipes/training/miles/deployment.example.json), with every `` filled in:

cwtrain deployment create --file deployment.json --wait

The...