Better speech synthesis for audiobooks
Captured source
source ↗Better speech synthesis for audiobooks - Amazon Science
Close
Close
Social
bluesky
threads
youtube
github
rss
Menu
Research
Research areas
Automated reasoning
Cloud and systems
Computer vision
Conversational AI
Economics
Information and knowledge management
Machine learning
Operations research and optimization
Quantum technologies
Robotics
Search and information retrieval
Security, privacy, and abuse prevention
Sustainability
Our scientific contributions
Publications
Research from our scientists and collaborators.
Conferences
Our experts present and discuss cutting-edge research at scientific meetings globally.
Research areas
Automated reasoning
Cloud and systems
Computer vision
Conversational AI
Economics
Information and knowledge management
Machine learning
Operations research and optimization
Quantum technologies
Robotics
Search and information retrieval
Security, privacy, and abuse prevention
Sustainability
Our scientific contributions
Publications
Research from our scientists and collaborators.
Conferences
Our experts present and discuss cutting-edge research at scientific meetings globally.
News & blog
The latest from Amazon researchers
Amazon Science Blog
Technical deep-dives and perspectives from our scientists.
News
Research milestones and recent achievements.
The latest from Amazon researchers
Amazon Science Blog
Technical deep-dives and perspectives from our scientists.
News
Research milestones and recent achievements.
Collaborations
Amazon Research Awards
Overview
Call for proposals
Latest news
Research stories
Recipients
Amazon Nova AI Challenge
Overview
Rules
FAQs
Teams
Research collaborations
Overview
Carnegie Mellon University
Columbia University
Hampton University
Howard University
IIT Bombay
Johns Hopkins University
Max Planck Society
MIT
Tennessee State University
University of California, Los Angeles
University of Illinois Urbana-Champaign
University of Southern California
University of Texas at Austin
Virginia Tech
University of Washington
Amazon Research Awards
Overview
Call for proposals
Latest news
Research stories
Recipients
Amazon Nova AI Challenge
Overview
Rules
FAQs
Teams
Research collaborations
Overview
Carnegie Mellon University
Columbia University
Hampton University
Howard University
IIT Bombay
Johns Hopkins University
Max Planck Society
MIT
Tennessee State University
University of California, Los Angeles
University of Illinois Urbana-Champaign
University of Southern California
University of Texas at Austin
Virginia Tech
University of Washington
Resources
Code and datasets
Amazon Nova
Try Amazon’s frontier foundation models.
Code and datasets
Amazon Nova
Try Amazon’s frontier foundation models.
Careers
Careers
Explore our open roles.
Amazon Scholars
Faculty research opportunities on industry-scale technical challenges.
Postdoctoral Science Program
Early-career research opportunities alongside experienced industry scientists.
Careers
Explore our open roles.
Amazon Scholars
Faculty research opportunities on industry-scale technical challenges.
Postdoctoral Science Program
Early-career research opportunities alongside experienced industry scientists.
Search
Submit Search
Conversational AI
Better speech synthesis for audiobooks
“Controllable text to speech” makes speech more expressive, character voices more distinct and consistent.
By Arent van Korlaar
October 23, 2026
6 min read
Share
Share
Copy link
X
Line
QZone
Sina Weibo
分享到微信
x
Listen • 8:53 min
Key takeaways
Audible's controllable text-to-speech (TTS) system profiles book characters and attributes dialogue with 97% accuracy in English (94% in Spanish). Rendering distinct character voices improved dialogue differentiation by 3.7 points on a MUSHRA test (a 0–100 listener-rating scale), with statistically significant listening-experience gains for 8 of 10 voices across both English and Spanish. The system controls delivery via natural-language captions (e.g., "whispering nervously", "shouting in anger") and fixes each character's voice profile once at the book level, so the model renders that character consistently from beginning to end.
Was this answer helpful?
Text-to-speech systems — which convert written text into synthetic speech — display impressive naturalness in applications such as chatbots and virtual assistants, where interactions last seconds. But they tend not to fare as well with longer-form content. When applied to audiobooks, for instance, conventional text-to-speech (TTS) models fail to reliably change speaking style between characters, as a human narrator would. This creates two problems for listeners: (1) distinguishing narration from dialogue and (2) identifying which character speaks which lines. To address this problem, the Audible technology team is using controllable TTS, meaning the ability to control dialogue delivery with natural-language speech descriptions (e.g., "speak in an enthusiastic, high-pitched voice"). This allows the TTS model to leverage the deep textual understanding of large language models (LLMs) to synthesize high-quality speech. Our approach involves three key innovations. First, we developed a novel data synthesis pipeline to produce speech data captioned with high-quality descriptions of speech characteristics. Training on this data enables our model to understand captions and produce audio consistent with them at inference time. Second, we created a system that automatically profiles characters in books and identifies their dialogue with 97% accuracy. It also provides captions for that dialogue, which act as stage directions for the model. Third, we used the first two techniques to synthesize audiobooks featuring distinct character voices, improving dialogue differentiation by 3.7 points on a MUSHRA (multiple stimuli with hidden reference and anchor) test, a standardized method where listeners rate audio quality on a scale from 0 to 100. Across tests in both English and Spanish, we observe statistically significant improvement in listening experience for eight out of ten voices.
Controllable text to speech assigns more self-consistent and distinct speaking styles to different characters in an audiobook — officers on the bridge of a spaceship, for example.
Spaceship dialogue
TTS for audiobooks...
Excerpt shown — open the source for the full document.
Notability
notability 6.0/10Substantive post from Amazon on improved TTS for audiobooks