replicate/cog v0.17.0

v0.17.0

Repository: replicate/cog

Tag: v0.17.0

Published: 2026-03-25T23:36:37Z

Prerelease: no

Release notes: This is a big release. The prediction server has been rewritten in Rust, Pydantic dependency conflicts are a thing of the past, and several long-requested QoL features are here. We've tested extensively against models on Replicate and the vast majority work without any changes. Under the hood, though, this is the foundation for what's coming next. A few things did change... see the breaking changes section at the bottom.

Highlights

New prediction server (coglet)

The Python HTTP server that ran inside Cog containers has been replaced with a Rust-based server called coglet. It uses a two-process architecture -- a Rust parent handling HTTP and orchestration, and a Python worker subprocess running your predict function. You don't need to change anything in your code. Predictions are faster to start, the server handles concurrency better, and worker crashes no longer take down the whole container. More importantly, this is the runtime we'll be building on -- it unlocks things like native streaming, smarter scheduling, and tighter hardware integration that weren't possible with the old Python server.

Custom metrics API

You can now emit custom metrics from your predict function:

from cog import current_scope

def predict(self, prompt: str) -> str:
scope = current_scope()
scope.record_metric("tokens_generated", 128)
scope.record_metric("timing.inference", 0.42)
...

Metrics appear in the prediction response alongside predict_time. Supports "replace" (default), "incr", and "append" accumulation modes. Dot-path keys create nested objects.

User-defined healthcheck

Add a healthcheck() method to your Predictor to inject custom validation into the /health-check endpoint:

def healthcheck(self) -> bool:
# check GPU is responsive, weights are loaded, etc.
return True

Runs with a 5-second timeout, even when the model is busy processing predictions.

Per-prediction context

Callers can now pass a context dict with prediction requests, accessible in your predict function via current_scope().context. Useful for forwarding metadata like API tokens, region hints, or prediction IDs without changing your function signature.

Pydantic removed from default stack

Cog's type system was built on Pydantic, which meant the SDK pinned a specific Pydantic version in every container. If a package you depended on needed a different version, you were stuck. That's gone now -- cog.BaseModel is a standard Python dataclass, and Pydantic isn't installed at all unless you add it yourself. The API is unchanged (class Output(BaseModel): text: str still works), and if your predict function returns a Pydantic model it'll still be serialized correctly.

Multi-registry support

cog push and cog login now work with any OCI-compliant registry -- GHCR, GCR, ECR, Docker Hub, self-hosted. The provider is selected automatically based on the registry host in your image name.

SDK version decoupling

New build.sdk_version field in cog.yaml lets you pin the Python SDK version independently from the CLI:

build:
sdk_version: "0.16.6"

Omit it to get the latest stable release. Set "prerelease" to opt into pre-release builds. Minimum supported version is 0.16.0.

Faster image pushes

cog push can now upload layers directly to the registry in parallel with automatic chunking (96 MB chunks, 5 concurrent uploads), blob deduplication, and retry with exponential backoff. Set COG_PUSH_OCI=1 to enable. Falls back to docker push if anything goes wrong.

uv for package management

Dockerfiles now use uv instead of pip for faster, more reliable dependency installation inside containers.

Setup timeout changes

The old hardcoded 5-minute setup timeout has been removed. You can now configure your own timeout with the COG_SETUP_TIMEOUT environment variable (in seconds). If unset, there's no internal timeout.

Pretty CLI output

CLI output now uses color-coded prefixes (✔ green for success, ⚠ yellow for warnings, ✗ red for errors) and auto-detects color support from TTY/environment. Use --no-color or NO_COLOR=1 to disable.

---

Breaking changes

• `python_version` is now required in the build: section of cog.yaml. Builds fail with a clear error if it's missing. Add python_version: "3.13" (or 3.10/3.11/3.12).
• Python 3.8 and 3.9 are no longer supported. Minimum is 3.10.
• Pydantic is no longer installed by default. If you depend on it, add it to your python_packages.
• `cog train` is deprecated. It still works but prints a warning. Will be removed in a future release so we can replace it with something better.
• `emit_metric()` is deprecated in favor of current_scope().record_metric(). The old function still works as a compat shim.