NVIDIA/OSMO 6.3.0

6.3.0

Repository: NVIDIA/OSMO

Tag: 6.3.0

Published: 2026-05-05T20:44:05Z

Prerelease: no

Release notes:

Highlights

• ConfigMap-based configuration — All service configs (pools, backends, pod templates, roles, and more) can now be managed as Helm values via a Kubernetes ConfigMap, following standard K8s patterns and enabling GitOps workflows.
• TLS support — The service chart now terminates TLS at the gateway, with values for cert/key, redirect from HTTP, and SAN configuration.
• Service chart consolidation — The standalone router and web-ui Helm charts have been folded into the service chart, making a full deployment a single Helm release.
• Multi-provider deploy scripts — deploy-k8s.sh now provisions OSMO on Azure AKS, AWS EKS, microk8s, or any existing Kubernetes cluster, with idempotent installers for KAI Scheduler, GPU Operator, MinIO, and configurable storage backends (MinIO, Azure Blob, AWS S3, BYO S3).
• Per-group timeouts — exec_timeout and queue_timeout now meter each group independently instead of running against the workflow as a whole, so a stuck simulation group no longer kills the rest of the workflow.
• Dataset CLI and API deprecated — osmo dataset commands and the /datasets API endpoints are deprecated and will be removed in 6.4. Migrate to workflow-managed dataset outputs.
• Rsync download support — Pull files from running workflow tasks to your local machine with osmo workflow rsync download, complementing the existing upload capability.
• Visual transfer progress — File sync operations now display a progress bar showing bytes transferred, percentage, rate, and ETA.
• Workload identity for core services — Run OSMO services under a cloud-issued federated identity (Azure Workload Identity on AKS/Arc, AWS IRSA / EKS Pod Identity) via new cloud-neutral serviceAccount annotations and per-component extraPodLabels hooks, removing the need to mount cloud storage keys as Kubernetes Secrets.
• Privilege escalation fix — Policies with empty resources lists no longer grant access to resource-scoped endpoints.

Breaking Changes

• Router chart removed: The standalone router Helm chart is gone. Router pods now deploy as part of the service chart. Existing router resources (osmo-router, osmo-router-headless) continue to work, but you must remove the separate router Helm release before upgrading. See the 6.2 to 6.3 upgrade guide for migration steps. (#897)
• Web UI chart removed: The standalone web-ui Helm chart has been merged into the service chart. Set ui.enabled: true in service values to deploy the UI alongside the API. Remove the separate web-ui release before upgrading. (#907)
• Squid proxy removed from backend operator: The egress allowlist and squid-proxy sidecar have been removed from the backend operator chart. Network policies now restrict pod-to-pod access directly. (#823)
• Per-group timeout semantics: exec_timeout and queue_timeout are now enforced per group (clock starts on the group's RUNNING or SCHEDULING transition) instead of per workflow. An expired group is marked FAILED_EXEC_TIMEOUT or FAILED_QUEUE_TIMEOUT; sibling groups continue and the workflow status aggregates only after all groups finish. (#925)
• Dataset CLI and API deprecated: All osmo dataset subcommands print a stderr deprecation warning, and the /datasets REST endpoints are marked deprecated in the OpenAPI schema. The Datasets page in the UI shows a deprecation banner. Both will be removed in 6.4. (#872)
• S3 addressing default: For S3-compatible backends with a custom endpoint_url, the addressing style now defaults to virtual-hosted instead of boto3's auto-selection (which picks path style for custom endpoints), fixing compatibility with providers that require virtual hosts. If a backend requires path addressing, set the addressing_style attribute to path, or force OSMO to always use path addressing via the AWS_S3_FORCE_PATH_STYLE environment variable. (#950)

Helm Charts

• ConfigMap configuration mode: Set services.configs.enabled: true to manage all service configs via Helm values. CLI/API writes return HTTP 409 when active. The chart ships with default roles, pod templates, resource validations, backend, and pool. (#822)
• ConfigMap mode for worker, agent, and logger: The ConfigMapWatcher now runs in the worker, agent, and logger services. Previously only the API service watched the ConfigMap, so workflow pods built by the worker could be constructed from stale config. (#926)
• TLS termination at the gateway: Configure a serving cert/key, optional HTTP-to-HTTPS redirect, and SAN list via gateway.tls. The gateway template generates the matching Envoy listener config. (#953)
• Cloud workload identity: New top-level serviceAccount block (create, name, annotations) and per-component extraPodLabels on agent, api, worker, logger, router, and delayedJobMonitor. The hooks are cloud-neutral — set the annotations and labels your CSP's identity webhook expects:
• Azure (AKS / Arc): annotate the SA with azure.workload.identity/client-id: and label pods with azure.workload.identity/use: "true". The Azure storage backend falls back to DefaultAzureCredential when no static connection string is supplied.
• AWS (EKS IRSA / Pod Identity): annotate the SA with eks.amazonaws.com/role-arn: . The S3 backend picks up the federated token from boto3's default credential chain — no pod labels required.

• Gateway consolidation: A unified gateway now handles load balancing for all service types (API, router, UI), simplifying ingress configuration. (#817, #799)
• Gateway extension hooks: Inject custom Envoy filters and additive auth-skip paths via gateway.envoy.extensions and gateway.envoy.authSkipPaths, useful for sidecar integrations and bypassing authz on specific endpoints. (#1009)
• Default identity headers: Minimal deployments can now inject default x-osmo-user, x-osmo-roles, and x-osmo-allowed-pools headers for unauthenticated browser requests via gateway.envoy.defaultIdentity values. (#902)
• oauth2-proxy extraEnv: Expose environment variables on the oauth2-proxy container via gateway.oauth2Proxy.extraEnv, needed for Redis AUTH when using session storage. (#898)
• Custom HPA metrics: Specify custom metrics…