Chatterbox – Text-to-Speech Model

What is Chatterbox?

Chatterbox is an open-source, high-fidelity text-to-speech (TTS) model developed by Resemble AI. Built on a 0.5 billion-parameter Llama backbone, it transforms plain text into natural, expressive speech. Trained on 0.5 million hours of cleaned audio, Chatterbox leverages alignment-informed synthesis to maintain precise lip-sync and timing. Unique to Chatterbox is its emotion exaggeration control, enabling developers to dial up or tone down expressiveness for dramatic narration, character voices, and dynamic AI agents. Outputs include a subtle watermark to promote ethical usage and traceability.

Key Features

• 0.5 Billion Parameter Llama Backbone: Balances model size with ultra-natural speech quality.
• Emotion Exaggeration Control: User-adjustable “exaggeration” slider (0–2) for varied expressive styles.
• Alignment-Informed Synthesis: Stable, consistent timing between text and audio.
• Watermarked Outputs: Embedded inaudible watermark for responsible AI deployment.
• Voice Conversion Support: Match or clone voices using a reference audio clip.
• Ultra-Stable Generation: Outperforms leading commercial TTS like ElevenLabs in stability and nuance.
• Advanced Sampling Controls: Temperature, CFG weight, top_p, min_p, and repetition penalty for fine-tuning.

Best Use Cases

• Interactive AI Agents & Chatbots: Lifelike responses with adjustable emotion.
• Game Dialogue & Cinematics: Character voices with dynamic intensity control.
• Video Narration & Explainers: Professional voiceover with rich expressiveness.
• Memes & Social Clips: Create humorous or dramatic one-liners instantly.
• Podcasts & Audiobooks: Long-form narration with consistent tone and pacing.

Prompt Tips and Output Quality

• Input Text Length: Use longer passages for storytelling; shorter prompts for concise alerts.
• Reference Audio: Supply a sample clip (e.g., MP3 URL) to match tone and timbre.
• Exaggeration (0–2):
  • 0–0.5 for neutral/flat delivery
  • 0.7 (default) for mild expressiveness
  • 1.5–2.0 for theatrical or character voices
• Temperature (0–2): Lower values (0.2–0.5) yield consistent, predictable speech; higher (1.0–1.5) adds variation.
• CFG Weight (0–2): Balances strict adherence to text (lower) vs. creative interpretation (higher).
• Top_p & Min_p: Tailor randomness—reduce top_p (0.7–0.9) for focused output; raise for more diversity.
• Repetition Penalty (1–2): Increase to avoid word repetition in verbose content.

FAQs

Q: How do I control emotion intensity?
Use the exaggeration parameter: values below 0.7 tone down expression, values above 1.0 heighten drama.

Q: Can I match a custom voice?
Yes. Provide a reference_audio URL to steer Chatterbox toward the same style and pitch.

Q: Is Chatterbox multilingual?
Chatterbox is optimized for English. Community contributions are welcome to extend language support.

Q: How does the watermark work?
An inaudible digital watermark is embedded in each output to ensure traceability and discourage misuse.

Q: Is Chatterbox open source?
Absolutely. Chatterbox’s code and model checkpoints are available under an open-source license on Resemble AI’s GitHub.