microsoft/physical-ai-toolchain

microsoft/physical-ai-toolchain

Language: Python

License: MIT

Stars: 80

Forks: 41

Open issues: 238

Created: 2026-03-02T14:39:02Z

Pushed: 2026-06-11T00:37:03Z

Default branch: main

Fork: no

Archived: no

README:

Physical AI Toolchain

   

Overview

Physical AI and robotics are moving from headlines and experimentation into real-world industrial deployment. The shift creates practical implications for how human-robot-AI collaboration becomes an operational capability in manufacturing, logistics, healthcare, and autonomous systems. Operationalizing physical intelligence at scale — across fleets and federations of intelligent systems — is a challenge no single OEM or software vendor can deliver alone.

Physical AI is the strategic inflection point for AI platforms, and robotics is the hero use case. It sits at the intersection of cloud, edge, data, and agentic AI.

Physical AI Toolchain is an open-source, production-ready framework that integrates Microsoft Azure cloud services with NVIDIA's physical AI stack, accelerating robotics and physical AI developers to automate and scale data curation, augmentation, and evaluation across perception, mobility, imitation learning, and reinforcement learning pipelines. It provides:

• Accelerate physical AI innovation. From edge data capture on NVIDIA Jetson devices through cloud-based training on GPU clusters to model deployment at the edge, every stage of the physical AI lifecycle is addressed with tested, repeatable automation.
• Operationalize physical intelligence. Built on Azure Machine Learning, Azure Kubernetes Service, Azure Arc, and Azure Storage with Entra ID authentication, managed identities, and Infrastructure as Code, so workloads meet the security, compliance, and governance requirements of production environments.
• Scale through ecosystem collaboration. Native support for NVIDIA Isaac Sim and Isaac Lab for simulation and reinforcement learning, NVIDIA OSMO for workflow orchestration, and the NVIDIA Jetson platform for edge inference — a hardware-accelerated path from research to deployment enabled by deep partnership across the ecosystem.
• Human-robot-AI agent collaboration. Agentic engineering lets teams move from isolated machines to coordinated, instruction-driven workflows. AI agents can turn high-level instructions into executed pipelines — but they are a convenience layer, not a requirement. Start with manual workflows, introduce agents when you are ready, and customize their behavior to match your team's trust boundaries.
• Broad physical AI applicability. While robotics is the hero use case, the architecture supports any physical AI workload that follows the simulate → train → evaluate → deploy pattern, including autonomous mobile robots, robotic manipulation, industrial inspection, and embodied AI research.

Whether you are evaluating Azure and NVIDIA as a platform for physical AI, planning a proof of concept, or scaling to production, this toolchain provides a tested solution and working code to accelerate your timeline.

Who This Is For

• Robotics researchers moving from Isaac Sim prototypes to production-grade training and deployment pipelines
• Platform engineers standardizing physical AI pipelines across teams with Infrastructure as Code and repeatable workflows
• Enterprise teams piloting Jetson + Azure deployments and need security, compliance, and scalability from day one

> [!NOTE] > Who it's not for (yet): This toolchain targets production and pre-production workloads. It is not currently designed for hobbyist projects, ROS beginners learning the basics, or single-robot desktop demos. We welcome contributions that broaden accessibility over time.

> [!TIP] > Get started in under 2 hours. By the end of the [Quickstart Guide](docs/getting-started/quickstart.md), you will have: > > - A pick-and-place RL policy trained in Isaac Lab on Azure GPU compute > - Experiment metrics and checkpoints tracked in MLflow > - A containerized model deployed to a Jetson device via GitOps

What's Inside


| Capability | Description | |---------------------------------|-------------------------------------------------------------------------------------------------------------| | Simulation & Synthetic Data | Isaac Sim and Isaac Lab environments for RL task training and synthetic data generation | | Edge Data Capture | ROS 2 demonstration recording on Jetson with chunking, compression, and cloud upload | | Cloud Data Pipeline | Automated ROS-to-LeRobot conversion, quality validation, and event-driven orchestration | | Training Infrastructure | OSMO + Azure ML integration for scalable RL and IL training with experiment tracking | | Model Evaluation | Offline replay evaluation, Isaac Sim evaluation, and evaluation dashboards | | Model Deployment | ONNX/TensorRT conversion, container packaging, and GitOps-based edge deployment | | Agentic Workflows | Instruction-driven agents that orchestrate data collection, training, evaluation, and deployment end-to-end | | Hybrid Architecture | Azure Arc, air-gapped training support, and MQTT telemetry for connected and disconnected sites |

Key Features

• Infrastructure as Code — Terraform modules for reproducible Azure deployments
• Containerized Workflows — Docker-based Isaac Lab training with NVIDIA GPU support
• MLflow Integration — Automatic experiment tracking and model versioning
• Scalable Compute — Auto-scaling GPU nodes with pay-per-use cost optimization

-…