ByteDance-Seed/Puppeteer

ByteDance-Seed/Puppeteer

Language: Python

License: Apache-2.0

Stars: 7

Forks: 1

Open issues: 0

Created: 2025-08-28T07:41:01Z

Pushed: 2025-09-09T14:30:57Z

Default branch: master

Fork: no

Archived: no

README:

Project | Paper | Video | Data: Articulation-XL2.0

Puppeteer is proposed for automatic rigging and animation of 3D objects. Given a 3D object, Puppeteer first automatically generates skeletal structures and skinning weights, then animates the rigged model with video guidance through a differentiable optimization pipeline. This comprehensive approach aims to enable fully automated transformation of static 3D models into dynamically animated assets, eliminating the need for manual rigging expertise and significantly streamlining 3D content creation workflows.

🔥 News

• Sep 09, 2025: We uploaded the video for Puppeteer.
• Sep 04, 2025: Release the inference codes and model checkpoints.
• Aug 15, 2025: Release paper of Puppeteer!

🔧 Installtation

We use Python 3.10 with PyTorch 2.1.1 and CUDA 11.8. The environment and required packages can be installed as follows:

git clone https://github.com/ByteDance-Seed/Puppeteer.git --recursive && cd Puppeteer
conda create -n puppeteer python==3.10.13 -y
conda activate puppeteer
pip install torch==2.1.1 torchvision==0.16.1 torchaudio==2.1.1 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt
pip install flash-attn==2.6.3 --no-build-isolation
pip install torch-scatter -f https://data.pyg.org/whl/torch-2.1.1+cu118.html
pip install --no-index --no-cache-dir pytorch3d -f https://dl.fbaipublicfiles.com/pytorch3d/packaging/wheels/py310_cu118_pyt211/download.html

🚀 Demo

We provide a complete pipeline for rigging and animating 3D models. Before running the pipeline, visit each folder (skeleton, skinning, animation) to download the necessary model checkpoints. Example data is available in the examples folder.

Rigging

Given 3D meshes, we first predict the skeleton and skinning weights:

bash demo_rigging.sh

The final rig files will be saved in results/final_rigging. To evaluate the skeleton and skinning components separately, refer to their respective folders.

Video-guided 3D animation

To animate the rigged model using video guidance, run:

bash demo_animation.sh

The rendered 3D animation sequence from different views will be saved in results/animation. Refer to the animation folder for comprehensive details on data processing and structure.

😊 Acknowledgment

The code builds upon MagicArticulate, MeshAnything, Functional Diffusion, RigNet, Michelangelo, PartField, AnyMole and Lab4D. We gratefully acknowledge the authors for making their work publicly available.

📚 Citation

@article{song2025puppeteer,
title={Puppeteer: Rig and Animate Your 3D Models},
author={Chaoyue Song and Xiu Li and Fan Yang and Zhongcong Xu and Jiacheng Wei and Fayao Liu and Jiashi Feng and Guosheng Lin and Jianfeng Zhang},
journal={arXiv preprint arXiv:2508.10898},
year={2025}
}