File size: 2,857 Bytes
bb58fce |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
---
tags:
- text-to-speech
license: cc-by-nc-sa-4.0
language:
- zh
- en
- de
- ja
- fr
- es
- ko
- ar
- nl
- ru
- it
- pl
- pt
pipeline_tag: text-to-speech
inference: false
extra_gated_prompt: >-
You agree to not use the model to generate contents that violate DMCA or local
laws.
extra_gated_fields:
Country: country
Specific date: date_picker
I agree to use this model for non-commercial use ONLY: checkbox
---
# OpenAudio S1
**OpenAudio S1** is a leading text-to-speech (TTS) model trained on more than 2 million hours of audio data in multiple languages.
Supported languages:
- English (en)
- Chinese (zh)
- Japanese (ja)
- German (de)
- French (fr)
- Spanish (es)
- Korean (ko)
- Arabic (ar)
- Russian (ru)
- Dutch (nl)
- Italian (it)
- Polish (pl)
- Portuguese (pt)
Please refer to [Fish Speech Github](https://github.com/fishaudio/fish-speech) for more info.
Demo available at [Fish Audio Playground](https://fish.audio).
Visit the [OpenAudio website](https://openaudio.com) for blog & tech report.
## Emotion and Tone Support
OpenAudio S1 supports a variety of emotional, tone, and special markers to enhance speech synthesis:
**1. Emotional markers:**
(angry) (sad) (disdainful) (excited) (surprised) (satisfied) (unhappy) (anxious) (hysterical) (delighted) (scared) (worried) (indifferent) (upset) (impatient) (nervous) (guilty) (scornful) (frustrated) (depressed) (panicked) (furious) (empathetic) (embarrassed) (reluctant) (disgusted) (keen) (moved) (proud) (relaxed) (grateful) (confident) (interested) (curious) (confused) (joyful) (disapproving) (negative) (denying) (astonished) (serious) (sarcastic) (conciliative) (comforting) (sincere) (sneering) (hesitating) (yielding) (painful) (awkward) (amused)
**2. Tone markers:**
(in a hurry tone) (shouting) (screaming) (whispering) (soft tone)
**3. Special markers:**
(laughing) (chuckling) (sobbing) (crying loudly) (sighing) (panting) (groaning) (crowd laughing) (background laughter) (audience laughing)
**Special markers with corresponding onomatopoeia:**
- Laughing: Ha,ha,ha
- Chuckling: Hmm,hmm
## Model Variants and Performance
OpenAudio S1 includes the following models:
- **S1 (4B, proprietary):** The full-sized model.
- **S1-mini (0.5B):** A distilled version of S1.
Both S1 and S1-mini incorporate online Reinforcement Learning from Human Feedback (RLHF).
**Seed TTS Eval Metrics (English, auto eval, based on OpenAI gpt-4o-transcribe, speaker distance using Revai/pyannote-wespeaker-voxceleb-resnet34-LM):**
- **S1:**
- WER (Word Error Rate): **0.008**
- CER (Character Error Rate): **0.004**
- Distance: **0.332**
- **S1-mini:**
- WER (Word Error Rate): **0.011**
- CER (Character Error Rate): **0.005**
- Distance: **0.380**
## License
This model is permissively licensed under the CC-BY-NC-SA-4.0 license.
|