Tencent-Hunyuan/InstantCharacter

Tencent-Hunyuan/InstantCharacter

Language: Python

License: NOASSERTION

Stars: 1049

Forks: 95

Open issues: 36

Created: 2025-04-16T09:29:52Z

Pushed: 2025-05-14T03:33:35Z

Default branch: main

Fork: no

Archived: no

README:

InstantCharacter is an innovative, tuning-free method designed to achieve character-preserving generation from a single image, supporting a variety of downstream tasks.

Release

• [2025/05/14] 🔥 Thanks to Zeyu Long for optimizing VRAM usage - now supports [offload-inference](./offload_infer_demo.py) under 22GB VRAM.
• [2025/04/21] 🔥 Thanks to jax-explorer for providing the ComfyUI Wrapper.
• [2025/04/18] 🔥 We release the demo checkpoints and code.

• [2025/04/02] 🔥 We launch the project page.

Download

You can directly download the model from Huggingface.

huggingface-cli download --resume-download Tencent/InstantCharacter --local-dir checkpoints --local-dir-use-symlinks False

If you cannot access to Huggingface, you can use hf-mirror to download models.

export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --resume-download Tencent/InstantCharacter --local-dir checkpoints --local-dir-use-symlinks False

Once you have prepared all models, the folder tree should be like:

.
├── assets
├── checkpoints
├── models
├── infer_demo.py
├── pipeline.py
└── README.md

Usage

# !pip install transformers accelerate diffusers huggingface_cli
import torch
from PIL import Image
from pipeline import InstantCharacterFluxPipeline

# Step 1 Load base model and adapter
ip_adapter_path = 'checkpoints/instantcharacter_ip-adapter.bin'
base_model = 'black-forest-labs/FLUX.1-dev'
image_encoder_path = 'google/siglip-so400m-patch14-384'
image_encoder_2_path = 'facebook/dinov2-giant'
seed = 123456
pipe = InstantCharacterFluxPipeline.from_pretrained(base_model, torch_dtype=torch.bfloat16)
pipe.to("cuda")
pipe.init_adapter(
image_encoder_path=image_encoder_path,
image_encoder_2_path=image_encoder_2_path,
subject_ipadapter_cfg=dict(subject_ip_adapter_path=ip_adapter_path, nb_token=1024),
)

# Step 2 Load reference image
ref_image_path = 'assets/girl.jpg' # white background
ref_image = Image.open(ref_image_path).convert('RGB')

# Step 3 Inference without style
prompt = "A girl is playing a guitar in street"
image = pipe(
prompt=prompt,
num_inference_steps=28,
guidance_scale=3.5,
subject_image=ref_image,
subject_scale=0.9,
generator=torch.manual_seed(seed),
).images[0]
image.save("flux_instantcharacter.png")

You can use style lora

# download style lora
huggingface-cli download --resume-download InstantX/FLUX.1-dev-LoRA-Ghibli --local-dir checkpoints/style_lora/ --local-dir-use-symlinks False
huggingface-cli download --resume-download InstantX/FLUX.1-dev-LoRA-Makoto-Shinkai --local-dir checkpoints/style_lora/ --local-dir-use-symlinks False

# You can also use other style lora

# Step 3 Inference with style
lora_file_path = 'checkpoints/style_lora/ghibli_style.safetensors'
trigger = 'ghibli style'
prompt = "A girl is playing a guitar in street"
image = pipe.with_style_lora(
lora_file_path=lora_file_path,
trigger=trigger,
prompt=prompt,
num_inference_steps=28,
guidance_scale=3.5,
subject_image=ref_image,
subject_scale=0.9,
generator=torch.manual_seed(seed),
).images[0]
image.save("flux_instantcharacter_style_ghibli.png")

# Step 3 Inference with style
lora_file_path = 'checkpoints/style_lora/Makoto_Shinkai_style.safetensors'
trigger = 'Makoto Shinkai style'
prompt = "A girl is playing a guitar in street"
image = pipe.with_style_lora(
lora_file_path=lora_file_path,
trigger=trigger,
prompt=prompt,
num_inference_steps=28,
guidance_scale=3.5,
subject_image=ref_image,
subject_scale=0.9,
generator=torch.manual_seed(seed),
).images[0]
image.save("flux_instantcharacter_style_Makoto.png")

More case

Animal character are relatively unstable.

Acknowledgment

• Our work is sponsored by HuggingFace and fal.ai.

• Thanks to the model JY Duan.

Cite

If you find InstantCharacter useful for your research and applications, please cite us using this BibTeX:

@article{tao2025instantcharacter,
title={InstantCharacter: Personalize Any Characters with a Scalable Diffusion Transformer Framework},
author={Tao, Jiale and Zhang, Yanbing and Wang, Qixun and Cheng, Yiji and Wang, Haofan and Bai, Xu and Zhou, Zhengguang and Li, Ruihuang and Wang, Linqing and Wang, Chunyu and others},
journal={arXiv preprint arXiv:2504.12395},
year={2025}
}