NVIDIA/DreamDojo

NVIDIA/DreamDojo

Description: Official Codebase for "DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos" (ICML 2026)

Language: Python

License: Apache-2.0

Stars: 926

Forks: 59

Open issues: 12

Created: 2026-02-09T14:51:34Z

Pushed: 2026-03-21T19:03:33Z

Default branch: main

Fork: no

Archived: no

README:

💭 DreamDojo

🔥 Highlights

*DreamDojo* is an interactive world model that learns from large-scale human videos. In short, we made the following key contributions:

• A large-scale video dataset. 44k hours of diverse human egocentric videos, the largest dataset to date for world model pretraining.
• A foundation world model. The first robot world model of its kind that demonstrates strong generalization to diverse objects and environments after post-training.
• A distillation pipeline. After distillation, our model can achieve long-horizon autoregressive generation, with stable real-time interactions at 10 FPS for over 1 minute.

📢 News

• [2026/02/18] We released both pretraining and post-training code.
• [2026/02/18] We released all pretrained and post-trained checkpoints (2B and 14B).
• [2026/02/18] We released the GR-1 post-training datasets and the evaluation sets.
• [2026/02/09] We released our paper on arXiv.

🕹️ Quick Start

• Setup
• Latent Action Model Training
• DreamDojo Pretraining
• DreamDojo Post-Training
• DreamDojo Distillation
• Evaluation
• Trouble Shooting

⭐ Citation

If you find our work useful, please consider citing us and giving a star to our repo.

@article{gao2026dreamdojo,
title={DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos},
author={Shenyuan Gao and William Liang and Kaiyuan Zheng and Ayaan Malik and Seonghyeon Ye and Sihyun Yu and Wei-Cheng Tseng and Yuzhu Dong and Kaichun Mo and Chen-Hsuan Lin and Qianli Ma and Seungjun Nah and Loic Magne and Jiannan Xiang and Yuqi Xie and Ruijie Zheng and Dantong Niu and You Liang Tan and K.R. Zentner and George Kurian and Suneel Indupuru and Pooya Jannaty and Jinwei Gu and Jun Zhang and Jitendra Malik and Pieter Abbeel and Ming-Yu Liu and Yuke Zhu and Joel Jang and Linxi "Jim" Fan},
journal={arXiv preprint arXiv:2602.06949},
year={2026}
}

⚖️ License

DreamDojo source code is released under the Apache-2.0 license.