AI Runtime is Generally Available

AI Runtime is Generally Available | Databricks Blog Skip to main content

Summary

AI Runtime is now generally available, bringing high-performance GPU infrastructure directly into Databricks.

Scale reliably and experiment faster with seconds-level launches, flexible environments, and integrated observability — without the overhead of managing GPU infrastructure.

Keep every training workload secure and governed with built-in Unity Catalog controls and workload isolation — in a new era of agent-led training.

Today, we’re excited to announce the General Availability of AI Runtime (AIR),  bringing fully managed GPU infrastructure directly into Databricks so customers can train their most demanding models where their enterprise data already lives. Too often, AI teams spend more time securing GPU capacity, managing failures, and waiting for experiments to launch than improving their models. Running GPU infrastructure separately from their Lakehouse adds governance gaps and puts sensitive training data at risk. As  coding agents drive more training experiments , these bottlenecks and  risks grow — making reliable, fast, and governed infrastructure essential. AIR brings GPU infrastructure, developer tools, and data governance together so teams can train at scale, iterate faster, and secure every workload without managing a separate GPU stack. Hundreds of companies, including Robinhood, PicPay, Merck,  Flo Health , TinyFish , and  FactSet , already use AIR to train and ship deep learning models to production, with use cases spanning healthcare, manufacturing, and consumer-facing agents. Our own  Databricks AI Research team uses AIR to train the models behind KARL and AI Parse. With AIR, customers can: Train at scale with hassle-free GPU infrastructure:  Serverless, on-demand access to advanced GPUs backed by  reliable multinode training , optimized  performance , and battle-tested by 100,000+ GPU-hour training runs. Experiment faster with near-instant launch latency and easy debugging:  Bring your own code, containers, and Ray workloads – and obtain seconds-level launch latency on RDMA-enabled GPUs. Environment caching and built-in debugging tools to save time on every training loop. Secure and govern every GPU workload: AIR operates within the governance perimeter of Databricks and Unity Catalog, and is backed by years of investment in Serverless security. AIR helps ensure GPU workloads protect valuable training data, comply with access policies, and are tracked/auditable.

Hassle-free GPU infrastructure Managing GPU infrastructure requires significant operational overhead, from capacity planning and node maintenance to networking and monitoring. With AIR, Databricks takes on the operational work of provisioning GPUs, maintaining healthy infrastructure, and optimizing the workload, so your team can focus on training: Get the GPU capacity you need, on demand. Databricks handles GPU procurement, cluster configuration, and networking so your team can start training without building and operating a full GPU platform. Scale with confidence on healthy infrastructure. Databricks runs  dozens of checks for GPU and network health before and during training , so your workloads can scale to hundreds of GPUs reliably. Efficient training by default: Built-in data loaders stream and cache training data to reduce time spent waiting on I/O. Use tools such as  UCVolumeDataset to improve data throughput without building a custom loading pipeline.

AI Runtime validates GPUs before scheduling, continuously monitors active workloads, and periodically tests idle GPUs to detect and remove unhealthy nodes.

Using Databricks AI Runtime, we moved our large-scale training from a single-node setup with 8 A100 GPUs to a multi-node setup with 16 H100 GPUs. This reduced training time from roughly seven days to 48 hours, a 3.5x speedup, making it much easier for our team to iterate on larger model experiments and train more efficiently. —Kostia Omelianchuk, Applied Research Scientist and Area Tech Lead, Superhuman Use GPUs wherever you work Access GPUs via CLI or interactively within Notebooks or IDEs. Your first workload just requires a single CLI command. With  Databricks AIR Skills , migrating from a K8S or SLURM cluster just takes a single natural language prompt.

Orchestrate GPU workloads with confidence with Lakeflow AIR integrates natively with  Lakeflow so you can keep GPU workloads in sync with upstream and downstream pipelines. Moving to production requires just a few extra lines of YAML, with built in priority/pre-emption that minimize idle GPU time and monitoring tools like  Genie ZeroOps that lower operational toil.

Experiment, debug, and iterate faster Distributed training workloads are painful to set up and launch. AI teams waste time configuring training environments, waiting for GPU clusters to spin up, and debugging issues across the stack. AIR helps you get from code to training in seconds. You can use pre-built environments or  bring your own container , and benefit from highly optimized environment caching so you just need to build environments once. AIR is compatible with all major frameworks, including Ray, so you can bring code over with minimal configuration. Databricks AI Runtime lets us train where our data already lives, with the reliability, security, and performance we need for large-scale model training. We can spend less time managing GPU infrastructure and more time improving the models themselves. —Vladislav Nedosekin, Director of Engineering, Data & AI Platform, Flo Health Launch your next experiment in seconds When you iterate and develop a training script, every second of delay is felt repeatedly. And so, we’ve  optimized every layer of the provisioning stack to make workloads, even large scale RDMA-enabled clusters with dozens of GPUs, spin up in seconds.

Unlike legacy GPU stacks that require full container image downloads before running code, AI Runtime uses warm serverless GPU pools and lazy image loading to start workloads in seconds, while continuing to load the remaining image data in the background.

Debug distributed training in one place We’ve built an entirely new observability experience in MLflow, the industry’s leading experiment tracking tool. With this new experience, debugging your logs, which stream seconds after submission, is as simple as adding  –watch  to your AIR command. For deeper investigation, you can use our new logs viewer, tailor-built for...