Tencent-Hunyuan/AuK

Tencent-Hunyuan/AuK

Description: AuK: An Open-Source Foundational Model for Speech Generation and Editing

Language: Python

License: NOASSERTION

Stars: 41

Forks: 3

Open issues: 0

Created: 2026-08-19T01:39:24Z

Pushed: 2026-09-09T05:02:18Z

Default branch: main

Fork: no

Archived: no

README:

💻 Try our model on the HuggingFace Space · ModelScope Space!

News

• [2026/09/09] 🎉 We open-source AuK. Code and model weights are publicly available. Try it on the 🤗 Demo Space or the 🤖 ModelScope Space!

Contents

• [News](#news)
• [Introduction](#introduction)
• [Performance](#performance)
• [Model Architecture](#model-architecture)
• [Supported Tasks](#supported-tasks)
• [Quick Start](#quick-start)
• [Installation](#installation)
• [uv](#uv)
• [Conda](#conda)
• [Download the weights](#download-the-weights)
• [Command-line inference](#command-line-inference)
• [Interactive Gradio demo](#interactive-gradio-demo)
• [ComfyUI](#comfyui)
• [Prompt Enhancer](#prompt-enhancer)
• [Python API](#python-api)
• [Fine-tuning](#fine-tuning)
• [Contributing](#contributing)
• [Citation](#citation)
• [License](#license)

Introduction

AuK is a 1.5B foundation model for speech generation and editing. Trained on millions of hours of diverse audio data, AuK supports zero-shot and instruction-based TTS, content and acoustic editing, paralinguistic editing, speech enhancement, and source separation through a unified natural-language instruction interface. AuK has two variants:

| Model | Description | Weight | | --- | --- | --- | | AuK | Base model for high-quality generation | 🤗 Hugging Face · 🤖 ModelScope | | AuK-Flash | Distilled model for fast 4-step inference | 🤗 Hugging Face · 🤖 ModelScope |

Performance


Model Architecture


Supported Tasks

AuK exposes every task through the same natural-language instruction interface. The table below groups the supported tasks by category, with a short description and a link to its section in the [Cookbook](docs/COOKBOOK.md), where you'll find instruction templates plus CLI and Python examples.

Category Task Description Cookbook

Speech Generation Zero-shot TTS Speak the target text in the voice of the reference audio. Zero-shot TTS

Instruct TTS Generate speech from a voice description alone — no reference audio. Instruct TTS

Content Editing Speech Content Editing Rewrite what is said — replace, insert, or remove text. Speech Content Editing

Lyric Editing Rewrite lyrics in a singing recording while preserving the melody and voice. Lyric Editing

Acoustic Editing Pitch Editing Raise or lower the pitch by semitones. Pitch Editing

Speed Editing Adjust the speaking rate; output length scales with the speed factor. Speed Editing

Volume Editing Raise or lower the volume by decibels. Volume Editing

Paralinguistic Editing Emotion Change the emotion while preserving content and voice. Emotion

Timbre Change the timbre to a description while keeping the content unchanged. Timbre

De-accent Remove a regional accent while preserving the speaker's voice and content. De-accent

Nonverbal Editing Remove or add nonverbal sounds such as breaths, laughs, or coughs. Nonverbal Editing

Whisper Conversion Convert between normal speech and whisper while preserving speaker and content. Whisper Conversion

Enhancement & Separation Speech Enhancement Denoise, dereverberate, or restore natural, clear speech. Speech Enhancement

Speech Separation Keep one speaker by talking order and remove the others. Speech Separation

Music Separation Extract the singing voice from a mix, or keep all human voices. Music Separation

Target Speaker Extraction Keep the target speaker identified by what they say. Target Speaker Extraction

Quick Start

Installation

Clone the repository, then choose either uv or Conda to create an isolated Python 3.10 environment.

git clone https://github.com/Tencent-Hunyuan/AuK
cd AuK

uv

# Create and activate a project-local environment.
uv venv --python 3.10
source .venv/bin/activate

# Choose one installation target:
# Core inference and CLI only
uv pip install -e .

# Core inference + Gradio + Prompt Enhancer + ASR
uv pip install -e ".[gradio]"

# Core inference + ComfyUI nodes + Prompt Enhancer + ASR
uv pip install -e ".[comfyui]"

# Core inference + fine-tuning
uv pip install -e ".[train]"

# Everything
uv pip install -e ".[gradio,train]"

Conda

conda create -n auk python=3.10 -y
conda activate auk

# Choose one installation target:
# Core inference and CLI only
pip install -e .

# Core inference + Gradio + Prompt Enhancer + ASR
pip install -e ".[gradio]"

# Core inference + ComfyUI nodes + Prompt Enhancer + ASR
pip install -e ".[comfyui]"

# Core inference + fine-tuning
pip install -e ".[train]"

# Everything
pip install -e ".[gradio,train]"

The default installation includes PyTorch, TorchAudio, and TorchVision. If your platform requires a specific CPU or CUDA build, install a matching PyTorch stack for your platform first, then install AuK with either command above.

Download the weights

🤗 HuggingFace

pip install -U "huggingface_hub[cli]"

# AuK-Base
hf download tencent/AuK --local-dir ./ckpts/AuK

# AuK-Flash (4-step distilled)
hf download tencent/AuK-Flash --local-dir ./ckpts/AuK-Flash

# MLLM Encoder
hf download Qwen/Qwen2.5-Omni-3B --local-dir ./ckpts/Qwen2.5-Omni-3B

🤖 ModelScope

pip install -U modelscope

# AuK-Base
modelscope download --model Tencent-Hunyuan/AuK --local_dir ./ckpts/AuK

# AuK-Flash (4-step distilled)
modelscope download --model Tencent-Hunyuan/AuK-Flash --local_dir ./ckpts/AuK-Flash

# MLLM Encoder
modelscope download --model Qwen/Qwen2.5-Omni-3B --local_dir ./ckpts/Qwen2.5-Omni-3B

The expected directory structure is:

ckpts/
├── AuK/
├── AuK-Flash/ # optional
└── Qwen2.5-Omni-3B/

The model checkpoint contains the diffusion transformer and layer-fusion weights. The MLLM encoder and VAE are loaded from separate...