Update README.md

README.md

@@ -217,40 +217,75 @@ Stage 1 employed low threshold values (0 to 0.30 BLEU depending on dataset), whe
 | NST         | 250                      | 250          |
 | **Total**   | **56,514**               | **8,533**    |
 
-The default when loading our models through Hugging Face is **Stage 2**. We have however also uploaded continued pretraining checkpoints and tagged them. You can load these other checkpoints by specifying the `revision` in `.from_pretrained()`. The pretrained checkpoints tag can for example be found here: [`pretrained-checkpoint`](https://huggingface.co/KBLab/kb-whisper-large/tree/pretrained-checkpoint). The Stage 2 default model tag is named `standard`. We supply a different stage 2 checkpoint -- with a more condensed style of transcribing -- under the name `subtitle`. 
+The default when loading our models through Hugging Face is **Stage 2**. We have however also uploaded continued pretraining checkpoints and tagged them. You can load these other checkpoints by specifying the `revision` in `.from_pretrained()`. The pretrained checkpoints tag can for example be found here: [`pretrained-checkpoint`](https://huggingface.co/KBLab/kb-whisper-large/tree/pretrained-checkpoint). The Stage 2 default model tag is named `standard`. We supply two different stage 2 checkpoints -- one with a more condensed style of transcribing -- under the name `subtitle`, and one more verbose called `strict`. 
 
 ### Evaluation
 
 
-#### WER
+#### WER compared to OpenAI
 | Model size  |  | FLEURS | CommonVoice | NST  |
 |------------|---------|--------|-------------|------|
 | [tiny](https://huggingface.co/KBLab/kb-whisper-tiny)       | **KBLab**   | **13.2**  | **12.9**  | **11.2**  |
 |            | OpenAI  | 59.2   | 67.8   | 85.2   |
 | [base](https://huggingface.co/KBLab/kb-whisper-base)       | **KBLab**   | **9.1**   | **8.7**   | **7.8**   |
 |            | OpenAI  | 39.6   | 52.1   | 53.4   |
-| [small](https://huggingface.co/KBLab/kb-whisper-small)      | **KBLab**   | **7.3**   | **6.4**   | **6.6**   |
+| [small](https://huggingface.co/KBLab/kb-whisper-small)     | **KBLab**   | **7.3**   | **6.4**   | **6.6**   |
 |            | OpenAI  | 20.6   | 26.4   | 26.4   |
-| [medium](https://huggingface.co/KBLab/kb-whisper-medium)     | **KBLab**   | **6.6**   | **5.4**   | **5.8**   |
+| [medium](https://huggingface.co/KBLab/kb-whisper-medium)   | **KBLab**   | **6.6**   | **5.4**   | **5.8**   |
 |            | OpenAI  | 12.1   | 15.8   | 17.1   |
-| [large-v3](https://huggingface.co/KBLab/kb-whisper-large)   | **KBLab**   | **5.4**   | **4.1**   | **5.2**   |
+| [large-v3](https://huggingface.co/KBLab/kb-whisper-large)  | **KBLab**   | **5.4**   | **4.1**   | **5.2**   |
 |            | OpenAI  | 7.8    | 9.5    | 11.3    |
 
+#### WER for different KBLab stage2 versions
 
-#### BLEU Score
+| Model size  |  | FLEURS | CommonVoice | NST  |
+|------------|---------|--------|-------------|------|
+| [tiny](https://huggingface.co/KBLab/kb-whisper-tiny)       | **standard**   | **13.2**  | **12.9**  | **11.2**  |
+|            | strict    | 14.1   | 13.4   | 11.0   |
+|            | subtitle  | 13.3   | 12.9   | 11.4   |
+| [base](https://huggingface.co/KBLab/kb-whisper-base)       | **standard**   | **9.1**   | **8.7**   | **7.8**   |
+|            | strict    | 10.4   | 9.6    | 8.4    |
+|            | subtitle  | 9.1    | 8.7    | 7.9    |
+| [small](https://huggingface.co/KBLab/kb-whisper-small)     | **standard**   | **7.3**   | **6.4**   | **6.6**   |
+|            | strict    | 8.2    | 7.0    | 6.7    |
+|            | subtitle  | 7.3    | 6.4    | 6.6    |
+| [medium](https://huggingface.co/KBLab/kb-whisper-medium)   | **standard**   | **6.6**   | **5.4**   | **5.8**   |
+|            | strict    | 6.8    | 5.4    | 6.0    |
+| [large-v3](https://huggingface.co/KBLab/kb-whisper-large)  | **standard**   | **5.4**   | **4.1**   | **5.2**   |
+|            | strict    | 5.3    | 4.0    | 5.1    |
+
+
+#### BLEU Score compared to OpenAI
 | Model size  |   | FLEURS | CommonVoice | NST  |
 |------------|---------|--------|-------------|------|
-| tiny       | KBLab   | **76.6**  | **73.7**  | **74.3**  |
+| [tiny](https://huggingface.co/KBLab/kb-whisper-tiny)       | **KBLab**   | **76.6**  | **73.7**  | **74.3**  |
 |            | OpenAI  | 26.9   | 21.1   | 24.0   |
-| base       | KBLab   | **83.2**   | **79.9**   | **78.3**   |
+| [base](https://huggingface.co/KBLab/kb-whisper-base)       | **KBLab**   | **83.2**   | **79.9**   | **78.3**   |
 |            | OpenAI  | 41.1   | 32.5   | 36.9   |
-| small      | KBLab   | **86.6**   | **83.5**   | **79.6**   |
+| [small](https://huggingface.co/KBLab/kb-whisper-small)     | **KBLab**   | **86.6**   | **83.5**   | **79.6**   |
 |            | OpenAI  | 64.0   | 56.5   | 58.2   |
-| medium     | KBLab   | **87.6**   | **85.0**   | **80.2**   |
+| [medium](https://huggingface.co/KBLab/kb-whisper-medium)   | **KBLab**   | **87.6**   | **85.0**   | **80.2**   |
 |            | OpenAI  | 77.1   | 70.1   | 68.9   |
-| large-v3   | KBLab   | **89.8**   | **87.2**   | **81.1**   |
+| [large-v3](https://huggingface.co/KBLab/kb-whisper-large)  | **KBLab**   | **89.8**   | **87.2**   | **81.1**   |
 |            | OpenAI  | 84.9    | 79.1    | 75.1    |
 
+#### BLEU Score compared to OpenAI
+| Model size  |   | FLEURS | CommonVoice | NST  |
+|------------|---------|--------|-------------|------|
+| [tiny](https://huggingface.co/KBLab/kb-whisper-tiny)       | **standard**   | **76.6**  | **73.7**  | **74.3**  |
+|            | strict      | 75.3    | 72.9    | 74.6    |
+|            | subtitle    | 76.6    | 73.7    | 74.1    |
+| [base](https://huggingface.co/KBLab/kb-whisper-base)       | **standard**   | **83.2**   | **79.9**   | **78.3**   |
+|            | strict      | 81.0    | 78.4    | 77.5    |
+|            | subtitle    | 83.2    | 79.8    | 78.2    |
+| [small](https://huggingface.co/KBLab/kb-whisper-small)     | **standard**   | **86.6**   | **83.5**   | **79.6**   |
+|            | strict      | 84.9    | 82.4    | 79.3    |
+|            | subtitle    | 86.6    | 83.5    | 79.6    |
+| [medium](https://huggingface.co/KBLab/kb-whisper-medium)   | **standard**   | **87.6**   | **85.0**   | **80.2**   |
+|            | strict      | 87.3    | 84.9    | 80.1    |
+| [large-v3](https://huggingface.co/KBLab/kb-whisper-large)  | **standard**   | **89.8**   | **87.2**   | **81.1**   |
+|            | strict      | 90.0    | 87.4    | 81.2    |
+
 
 ### Acknowledgements