Zihan428 commited on
Commit
8d76d15
·
verified ·
1 Parent(s): beb087b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +51 -4
README.md CHANGED
@@ -1,12 +1,36 @@
1
  ---
2
  license: mit
3
  language:
 
 
 
 
4
  - en
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  tags:
6
  - text-to-speech
7
- - speech generation
 
8
  - voice-cloning
9
- pipeline_tag: text-to-speech
10
  library_name: chatterbox
11
  ---
12
 
@@ -31,15 +55,17 @@ library_name: chatterbox
31
  <img width="100" alt="resemble-logo-horizontal" src="https://github.com/user-attachments/assets/35cf756b-3506-4943-9c72-c05ddfa4e525" />
32
  </div>
33
 
 
34
 
35
- We're excited to introduce Chatterbox, [Resemble AI's](https://resemble.ai) first production-grade open source TTS model. Licensed under MIT, Chatterbox has been benchmarked against leading closed-source systems like ElevenLabs, and is consistently preferred in side-by-side evaluations.
36
 
37
  Whether you're working on memes, videos, games, or AI agents, Chatterbox brings your content to life. It's also the first open source TTS model to support **emotion exaggeration control**, a powerful feature that makes your voices stand out. Try it now on our [Hugging Face Gradio app.](https://huggingface.co/spaces/ResembleAI/Chatterbox)
38
 
39
  If you like the model but need to scale or tune it for higher accuracy, check out our competitively priced TTS service (<a href="https://resemble.ai">link</a>). It delivers reliable performance with ultra-low latency of sub 200ms—ideal for production use in agents, applications, or interactive media.
40
 
41
  # Key Details
42
- - SoTA zeroshot TTS
 
43
  - 0.5B Llama backbone
44
  - Unique exaggeration/intensity control
45
  - Ultra-stable with alignment-informed inference
@@ -57,6 +83,10 @@ If you like the model but need to scale or tune it for higher accuracy, check ou
57
  - Try lower `cfg` values (e.g. `~0.3`) and increase `exaggeration` to around `0.7` or higher.
58
  - Higher `exaggeration` tends to speed up speech; reducing `cfg` helps compensate with slower, more deliberate pacing.
59
 
 
 
 
 
60
 
61
  # Installation
62
  ```
@@ -80,8 +110,25 @@ AUDIO_PROMPT_PATH="YOUR_FILE.wav"
80
  wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH)
81
  ta.save("test-2.wav", wav, model.sr)
82
  ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
83
  See `example_tts.py` for more examples.
84
 
 
85
  # Acknowledgements
86
  - [Cosyvoice](https://github.com/FunAudioLLM/CosyVoice)
87
  - [HiFT-GAN](https://github.com/yl4579/HiFTNet)
 
1
  ---
2
  license: mit
3
  language:
4
+ - ar
5
+ - da
6
+ - de
7
+ - el
8
  - en
9
+ - es
10
+ - fi
11
+ - fr
12
+ - he
13
+ - hi
14
+ - it
15
+ - ja
16
+ - ko
17
+ - ms
18
+ - nl
19
+ - no
20
+ - pl
21
+ - pt
22
+ - ru
23
+ - sv
24
+ - sw
25
+ - tr
26
+ - zh
27
+ pipeline_tag: text-to-speech
28
  tags:
29
  - text-to-speech
30
+ - speech
31
+ - speech-generation
32
  - voice-cloning
33
+ - multilingual-tts
34
  library_name: chatterbox
35
  ---
36
 
 
55
  <img width="100" alt="resemble-logo-horizontal" src="https://github.com/user-attachments/assets/35cf756b-3506-4943-9c72-c05ddfa4e525" />
56
  </div>
57
 
58
+ **09/04 🔥 Introducing Chatterbox Multilingual in 23 Languages!**
59
 
60
+ We're excited to introduce **Chatterbox** and **Chatterbox Multilingual**, [Resemble AI's](https://resemble.ai) production-grade open source TTS models. Chatterbox Multilingual supports **Arabic, Danish, German, Greek, English, Spanish, Finnish, French, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese, Russian, Swedish, Swahili, Turkish, Chinese** out of the box. Licensed under MIT, Chatterbox has been benchmarked against leading closed-source systems like ElevenLabs, and is consistently preferred in side-by-side evaluations.
61
 
62
  Whether you're working on memes, videos, games, or AI agents, Chatterbox brings your content to life. It's also the first open source TTS model to support **emotion exaggeration control**, a powerful feature that makes your voices stand out. Try it now on our [Hugging Face Gradio app.](https://huggingface.co/spaces/ResembleAI/Chatterbox)
63
 
64
  If you like the model but need to scale or tune it for higher accuracy, check out our competitively priced TTS service (<a href="https://resemble.ai">link</a>). It delivers reliable performance with ultra-low latency of sub 200ms—ideal for production use in agents, applications, or interactive media.
65
 
66
  # Key Details
67
+ - Multilingual, zero-shot TTS supporting 23 languages
68
+ - SoTA zeroshot English TTS
69
  - 0.5B Llama backbone
70
  - Unique exaggeration/intensity control
71
  - Ultra-stable with alignment-informed inference
 
83
  - Try lower `cfg` values (e.g. `~0.3`) and increase `exaggeration` to around `0.7` or higher.
84
  - Higher `exaggeration` tends to speed up speech; reducing `cfg` helps compensate with slower, more deliberate pacing.
85
 
86
+
87
+ ***Note:*** Ensure that the reference clip matches the specified language tag. Otherwise, language transfer outputs may inherit the accent of the reference clip’s language.
88
+ ***To mitigate this, set the CFG weight to 0.***
89
+
90
 
91
  # Installation
92
  ```
 
110
  wav = model.generate(text, audio_prompt_path=AUDIO_PROMPT_PATH)
111
  ta.save("test-2.wav", wav, model.sr)
112
  ```
113
+
114
+ # Multilingual Quickstart
115
+ ```python
116
+ import torchaudio as ta
117
+ from chatterbox.mtl_tts import ChatterboxMultilingualTTS
118
+
119
+ multilingual_model = ChatterboxMultilingualTTS.from_pretrained(device="cuda")
120
+
121
+ french_text = "Bonjour, comment ça va? Ceci est le modèle de synthèse vocale multilingue Chatterbox, il prend en charge 23 langues."
122
+ wav_french = multilingual_model.generate(spanish_text, language_id="fr")
123
+ ta.save("test-french.wav", wav_french, model.sr)
124
+
125
+ chinese_text = "你好,今天天气真不错,希望你有一个愉快的周末。"
126
+ wav_chinese = multilingual_model.generate(chinese_text, language_id="zh")
127
+ ta.save("test-chinese.wav", wav_chinese, model.sr)
128
+ ```
129
  See `example_tts.py` for more examples.
130
 
131
+
132
  # Acknowledgements
133
  - [Cosyvoice](https://github.com/FunAudioLLM/CosyVoice)
134
  - [HiFT-GAN](https://github.com/yl4579/HiFTNet)