Tencent-Hunyuan/HunyuanCustom

Tencent-Hunyuan/HunyuanCustom

Description: HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation

Language: Python

License: NOASSERTION

Stars: 1224

Forks: 109

Open issues: 36

Created: 2025-05-07T07:48:42Z

Pushed: 2025-10-15T06:54:23Z

Default branch: main

Fork: no

Archived: no

README:

HunyuanCustom 🌅

> **HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation**

🔥🔥🔥 News!!

• June 13, 2025: 🚀 HunyuanCustom supports single GPU with only 8GB VRAM, many thanks to WanGP.
• June 6, 2025: 💃 We release the inference code and model weights of audio-driven and video-driven powered by OmniV2V.
• May 13, 2025: 🎉 HunyuanCustom has been integrated into ComfyUI-HunyuanVideoWrapper by Kijai.
• May 12, 2025: 🔥 HunyuanCustom is available in Cloud-Native-Build (CNB) HunyuanCustom.
• May 8, 2025: 👋 We release the inference code and model weights of HunyuanCustom. [Download](models/README.md).

📑 Open-source Plan

• HunyuanCustom
• Single-Subject Video Customization
• [x] Inference
• [x] Checkpoints
• [x] ComfyUI
• Audio-Driven Video Customization
• [x] Inference
• [x] Checkpoints
• [ ] ComfyUI
• Video-Driven Video Customization
• [x] Inference
• [x] Checkpoints
• [ ] ComfyUI
• Multi-Subject Video Customization

Contents

• [HunyuanCustom 🌅](#hunyuancustom-)
• [🔥🔥🔥 News!!](#-news)
• [📑 Open-source Plan](#-open-source-plan)
• [Contents](#contents)
• [Abstract](#abstract)
• [HunyuanCustom Overall Architecture](#hunyuancustom-overall-architecture)
• [🎉 HunyuanCustom Key Features](#-hunyuancustom-key-features)
• [Multimodal Video customization](#multimodal-video-customization)
• [Various Applications](#various-applications)
• [📈 Comparisons](#-comparisons)
• [📜 Requirements](#-requirements)
• [🛠️ Dependencies and Installation](#️-dependencies-and-installation)
• [Installation Guide for Linux](#installation-guide-for-linux)
• [🧱 Download Pretrained Models](#-download-pretrained-models)
• [🚀 Parallel Inference on Multiple GPUs](#-parallel-inference-on-multiple-gpus)
• [🔑 Single-gpu Inference](#-single-gpu-inference)
• [Run with very low VRAM](#run-with-very-low-vram)
• [Run a Gradio Server](#run-a-gradio-server)
• [🔗 BibTeX](#-bibtex)
• [Acknowledgements](#acknowledgements)

---

Abstract

Customized video generation aims to produce videos featuring specific subjects under flexible user-defined conditions, yet existing methods often struggle with identity consistency and limited input modalities. In this paper, we propose HunyuanCustom, a multi-modal customized video generation framework that emphasizes subject consistency while supporting image, audio, video, and text conditions. Built upon HunyuanVideo, our model first addresses the image-text conditioned generation task by introducing a text-image fusion module based on LLaVA for enhanced multi-modal understanding, along with an image ID enhancement module that leverages temporal concatenation to reinforce identity features across frames. To enable audio- and video-conditioned generation, we further propose modality-specific condition injection mechanisms: an AudioNet module that achieves hierarchical alignment via spatial cross-attention, and a video-driven injection module that integrates latent-compressed conditional video through a patchify-based feature-alignment network. Extensive experiments on single- and multi-subject scenarios demonstrate that HunyuanCustom significantly outperforms state-of-the-art open- and closed-source methods in terms of ID consistency, realism, and text-video alignment. Moreover, we validate its robustness across downstream tasks, including audio and video-driven customized video generation. Our results highlight the effectiveness of multi-modal conditioning and identity-preserving strategies in advancing controllable video generation.

HunyuanCustom Overall Architecture


We propose HunyuanCustom, a multi-modal, conditional, and controllable generation model centered on subject consistency, built upon the Hunyuan Video generation framework. It enables the generation of subject-consistent videos conditioned on text, images, audio, and video inputs.

🎉 HunyuanCustom Key Features

Multimodal Video customization

HunyuanCustom supports inputs in the form of text, images, audio, and video. Specifically, it can handle single or multiple image inputs to enable customized video generation for one or more subjects. Additionally, it can incorporate extra audio inputs to drive the subject to speak the corresponding audio. Lastly, HunyuanCustom supports video input, allowing for the replacement of specified objects in the video with subjects from a given image. 

Various Applications

With the multi-modal capabilities of HunyuanCustom, numerous downstream tasks can be accomplished. For instance, by taking multiple images as input, HunyuanCustom can facilitate virtual human advertisements and virtual try-on. Additionally, with image and audio inputs, it can create singing avatars. Furthermore, by using an image and a video as inputs, HunyuanCustom supports video editing by replacing subjects in the video with those in the provided image. More applications await your exploration! 

📈 Comparisons

To evaluate the performance of HunyuanCustom, we compared it with state-of-the-art video customization methods, including VACE, Skyreels, Pika, Vidu, Keling, and Hailuo. The comparison focused on face/subject consistency, video-text alignment, and overall video quality.

| Models | Face-Sim | CLIP-B-T | DINO-Sim | Temp-Consis | DD | |-------------------|----------|----------|----------|-------------|------| | VACE-1.3B | 0.204 | _0.308_ | 0.569 | 0.967 | 0.53 | | Skyreels | 0.402 | 0.295 | 0.579 | 0.942 | 0.72 | | Pika | 0.363 | 0.305 | 0.485 | 0.928 | _0.89_ | | Vidu2.0 | 0.424 | 0.300 | 0.537 | _0.961_ | 0.43 | | Keling1.6 | 0.505 | 0.285 | _0.580_ | 0.914 | 0.78 | | Hailuo | _0.526_ | 0.314| 0.433 | 0.937 | 0.94 | |…