replicate/cog-wan-2.1

replicate/cog-wan-2.1

Description: Wan: Open and Advanced Large-Scale Video Generative Models

Language: Python

License: Apache-2.0

Stars: 10

Forks: 2

Open issues: 1

Created: 2025-02-25T16:48:26Z

Pushed: 2025-02-26T18:05:34Z

Default branch: replicate

Fork: yes

Parent repository: Wan-Video/Wan2.1

Archived: no

README:

Wan2.1

💜 Wan &nbsp&nbsp ｜ &nbsp&nbsp 🖥️ GitHub &nbsp&nbsp | &nbsp&nbsp🤗 Hugging Face&nbsp&nbsp | &nbsp&nbsp🤖 ModelScope&nbsp&nbsp | &nbsp&nbsp 📑 Paper (Coming soon) &nbsp&nbsp | &nbsp&nbsp 📑 Blog &nbsp&nbsp | &nbsp&nbsp💬 WeChat Group&nbsp&nbsp | &nbsp&nbsp 📖 Discord&nbsp&nbsp | &nbsp&nbsp 🚀 Replicate&nbsp&nbsp

-----

[Wan: Open and Advanced Large-Scale Video Generative Models]("")

In this repository, we present Wan2.1, a comprehensive and open suite of video foundation models that pushes the boundaries of video generation. Wan2.1 offers these key features:

• 👍 SOTA Performance: Wan2.1 consistently outperforms existing open-source models and state-of-the-art commercial solutions across multiple benchmarks.
• 👍 Supports Consumer-grade GPUs: The T2V-1.3B model requires only 8.19 GB VRAM, making it compatible with almost all consumer-grade GPUs. It can generate a 5-second 480P video on an RTX 4090 in about 4 minutes (without optimization techniques like quantization). Its performance is even comparable to some closed-source models.
• 👍 Multiple Tasks: Wan2.1 excels in Text-to-Video, Image-to-Video, Video Editing, Text-to-Image, and Video-to-Audio, advancing the field of video generation.
• 👍 Visual Text Generation: Wan2.1 is the first video model capable of generating both Chinese and English text, featuring robust text generation that enhances its practical applications.
• 👍 Powerful Video VAE: Wan-VAE delivers exceptional efficiency and performance, encoding and decoding 1080P videos of any length while preserving temporal information, making it an ideal foundation for video and image generation.

Video Demos

🔥 Latest News!!

• Feb 25, 2025: 👋 We've released the inference code and weights of Wan2.1.

📑 Todo List

• Wan2.1 Text-to-Video
• [x] Multi-GPU Inference code of the 14B and 1.3B models
• [x] Checkpoints of the 14B and 1.3B models
• [x] Gradio demo
• [ ] Diffusers integration
• [ ] ComfyUI integration
• Wan2.1 Image-to-Video
• [x] Multi-GPU Inference code of the 14B model
• [x] Checkpoints of the 14B model
• [x] Gradio demo
• [ ] Diffusers integration
• [ ] ComfyUI integration

Quickstart

Installation

Clone the repo:

git clone https://github.com/Wan-Video/Wan2.1.git
cd Wan2.1

Install dependencies:

# Ensure torch >= 2.4.0
pip install -r requirements.txt

Model Download

| Models | Download Link | Notes | | --------------|-------------------------------------------------------------------------------|-------------------------------| | T2V-14B | 🤗 Huggingface 🤖 ModelScope | Supports both 480P and 720P | I2V-14B-720P | 🤗 Huggingface 🤖 ModelScope | Supports 720P | I2V-14B-480P | 🤗 Huggingface 🤖 ModelScope | Supports 480P | T2V-1.3B | 🤗 Huggingface 🤖 ModelScope | Supports 480P

> 💡Note: The 1.3B model is capable of generating videos at 720P resolution. However, due to limited training at this resolution, the results are generally less stable compared to 480P. For optimal performance, we recommend using 480P resolution.

Download models using huggingface-cli:

pip install "huggingface_hub[cli]"
huggingface-cli download Wan-AI/Wan2.1-T2V-14B --local-dir ./Wan2.1-T2V-14B

Run Text-to-Video Generation

This repository supports two Text-to-Video models (1.3B and 14B) and two resolutions (480P and 720P). The parameters and configurations for these models are as follows:

Task Resolution Model

480P 720P

t2v-14B ✔️ ✔️ Wan2.1-T2V-14B

t2v-1.3B ✔️ ❌ Wan2.1-T2V-1.3B

##### (1) Without Prompt Extention

To facilitate implementation, we will start with a basic version of the inference process that skips the [prompt extension](#2-using-prompt-extention) step.

• Single-GPU inference

python generate.py --task t2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-T2V-14B --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

If you encounter OOM (Out-of-Memory) issues, you can use the --offload_model True and --t5_cpu options to reduce GPU memory usage. For example, on an RTX 4090 GPU:

python generate.py --task t2v-1.3B --size 832*480 --ckpt_dir ./Wan2.1-T2V-1.3B --offload_model True --t5_cpu --sample_shift 8 --sample_guide_scale 6 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

> 💡Note: If you are using the T2V-1.3B model, we recommend setting the parameter --sample_guide_scale 6. The --sample_shift parameter can be adjusted within the range of 8 to 12 based on the performance.

• Multi-GPU inference using FSDP + xDiT USP

pip install "xfuser>=0.4.1"
torchrun --nproc_per_node=8 generate.py --task t2v-14B --size 1280*720 --ckpt_dir ./Wan2.1-T2V-14B --dit_fsdp --t5_fsdp --ulysses_size 8 --prompt "Two anthropomorphic cats in comfy boxing gear and bright gloves fight intensely on a spotlighted stage."

##### (2) Using Prompt Extention

Extending the prompts can effectively enrich the details in the generated videos, further enhancing the video quality. Therefore, we recommend enabling prompt extension. We provide the following two methods for prompt extension:

• Use the Dashscope API for extension.
• Apply for a dashscope.api_key in advance (EN | CN).
• Configure the environment variable DASH_API_KEY to specify the Dashscope API key. For users of Alibaba Cloud's international site, you also need to set the environment variable DASH_API_URL to…