Tencent-Hunyuan/Hy-SimulMT

Tencent-Hunyuan/Hy-SimulMT

Language: Python

License: NOASSERTION

Stars: 95

Forks: 10

Open issues: 2

Created: 2026-10-22T07:24:21Z

Pushed: 2026-10-27T03:47:54Z

Default branch: main

Fork: no

Archived: no

README: English | [简体中文](README.zh-CN.md)

Hy-SimulMT

A Low-Latency, High-Quality Model for Real-Time Simultaneous Translation

Introduction

Hy-SimulMT is a real-time simultaneous translation model developed by Tencent Hunyuan, supporting bidirectional speech-to-speech translation between Chinese and English. It continuously receives incoming audio and translates as the speaker talks, producing both translated text and speech. With real-time voice cloning, it preserves the speaker's vocal characteristics while conveying the message accurately.

Designed for live communication settings such as meetings, speeches, and interviews, Hy-SimulMT aims to convey information accurately while reducing the disruption that waiting introduces into a conversation. To evaluate this balance between translation quality and response speed, we combine public benchmarks, real-world audio, and comparisons with professional simultaneous interpreters.

In our RealSI long-form audio evaluation, Hy-SimulMT achieved a StreamLAAL of approximately 3 seconds, the lowest among the systems shown in the comparison. Its GEMBA score exceeded those of the compared open-source models and four of the five commercial APIs.

This repository provides the SimulInfer inference service and SimulTrain training tools. Speech output and voice cloning require a configured TTS service. You can also try Hy-SimulMT in the Hy Translate App.

Scan to download the Hy Translate App

Android iOS

News

• 2026.10.26 | Release: Hy-SimulMT is released with bidirectional real-time simultaneous translation between Chinese and English and real-time voice cloning. Project and model resources are linked below.

Results

Keeping Up Faster, Conveying Meaning More Accurately

The challenge of simultaneous translation is not simply to convert one language into another, but to understand and formulate a translation while the speaker is still talking. Waiting for more information often helps resolve the complete meaning, but also makes listeners wait longer. Speaking earlier, on the other hand, places greater demands on incremental understanding and translation generation.

Hy-SimulMT focuses on this balance between quality and latency: translating as a continuous audio stream arrives, so that listeners receive information promptly while the original meaning is preserved as accurately as possible.

We compared Hy-SimulMT with several open-source simultaneous translation models and commercial APIs on the open-source RealSI long-form audio benchmark. The results show that Hy-SimulMT achieves competitive translation quality at lower latency.

*The horizontal axis shows StreamLAAL, a measure of streaming translation lag, in seconds (lower is better); the vertical axis shows the GEMBA translation quality score (higher is better). StreamLAAL is not the same as the time to first speech output.*

In this evaluation, Hy-SimulMT achieved a StreamLAAL of approximately 3 seconds, the lowest among the systems shown. Its GEMBA score exceeded those of the compared open-source models and four of the five commercial APIs.

These results suggest that lower latency does not necessarily require a substantial sacrifice in translation quality. Hy-SimulMT offers a more timely way to access cross-language information during extended speeches, interviews, and meetings.

From Public Benchmarks to Real-World Settings

Translation Quality on Multiple Short-Form Audio Benchmarks

In addition to long-form audio, we evaluated the model on four public short-form audio benchmarks—FLEURS, CoVoST2, ACL6060, and RealSI-Sentence—to examine translation performance under different data conditions.

| API / Model | FLEURS | CoVoST2 | ACL6060 | RealSI-Sentence | | --- | ---: | ---: | ---: | ---: | | Qwen3.8-LiveTranslate | 88.4 | 82.1 | 80.5 | 84.4 | | Hy-SimulMT | 90.3 | 75.5 | 81.2 | 84.7 | | Gemini3.5-LiveTranslate | 92.4 | 74.4 | 82.6 | 82.2 | | Confucius4 R2T2 → T3PO | 89.4 | 76.8 | 76.1 | 77.7 | | Seed-LiveInterpret2.0 | 85.8 | 67.7 | 74.8 | 80.0 | | SimulS2ST-Omni | 78.6 | 66.6 | 73.1 | 77.4 | | GPT-Realtime-Translate | 77.7 | 50.8 | 76.4 | 76.7 | | XunFei-Simult-Interpretation | 73.4 | 57.0 | 57.5 | 70.9 | | SeamlessStreaming | 72.8 | 60.9 | 49.8 | 56.9 |

*Higher scores are better. The entire Hy-SimulMT row is bolded to highlight our model; other bolded values indicate the highest score in the corresponding column.*

Hy-SimulMT scored higher than the other open-source models in the table on FLEURS, ACL6060, and RealSI-Sentence. Its 84.7 on RealSI-Sentence was the highest among all listed systems. Results across these benchmarks also help us identify the model's strengths and areas for improvement.

A Matched Comparison with Professional Simultaneous Interpreters

To further examine performance on real-world content, we collected 18 Chinese and English audio recordings newly appearing online from September 2026 onward, each approximately 6 minutes long. The recordings cover speeches, interviews, and meetings across ten domains: software and AI, medicine, manufacturing and energy, natural sciences, agriculture and food, business management, law, education, culture and tourism, and sports and daily life.

Alongside Hy-SimulMT, we invited six interpreters with years of practical simultaneous interpreting experience to interpret these recordings. Three held CATTI Level 1 interpreting certificates, and three held CATTI Level 2 interpreting certificates. For each certification level, the highest score among the three interpreters was used for comparison.

As this small-scale capability assessment shows, Hy-SimulMT achieves translation quality on par with expert human simultaneous interpreters, while often responding faster (with lower latency).

*These findings apply only to the samples, participating interpreters, and evaluation settings used in this test. They do not constitute a comprehensive assessment of speech quality, voice similarity, or other dimensions.*

##### Examples Comparing Hy-SimulMT with Human Interpreters

Beyond the quantitative results, the following four videos illustrate how Hy-SimulMT and human interpreters compare in...