drepic commited on
Commit
923cc9e
·
verified ·
1 Parent(s): d2aa50d

Add benchmark results

Browse files
Files changed (1) hide show
  1. README.md +102 -92
README.md CHANGED
@@ -1,92 +1,102 @@
1
- ---
2
- library_name: ctranslate2
3
- license: apache-2.0
4
- base_model: openai/whisper-small
5
- tags:
6
- - audio
7
- - automatic-speech-recognition
8
- - ctranslate2
9
- - faster-whisper
10
- - generated_from_trainer
11
- - whisper
12
- metrics:
13
- - wer
14
- model-index:
15
- - name: whisper-small-jp
16
- results: []
17
- ---
18
-
19
- > **This repository contains the CTranslate2 export of the fine-tuned model.**
20
- >
21
- > • Base Transformers model: [drepic/whisper-small-jp](https://huggingface.co/drepic/whisper-small-jp)
22
- > • Use with `faster-whisper`:
23
- >
24
- > ```python
25
- > from faster_whisper import WhisperModel
26
- > model = WhisperModel("drepic/whisper-small-jp-ct2", device="cuda", compute_type="float16")
27
- > ```
28
-
29
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
30
- should probably proofread and complete it, then remove this comment. -->
31
-
32
- # whisper-small-jp
33
-
34
- This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on an unknown dataset.
35
- It achieves the following results on the evaluation set:
36
- - Loss: 0.6168
37
- - Wer: 0.2600
38
- - Cer: 0.2600
39
-
40
- ## Model description
41
-
42
- Better suited for transcribing japanese youtube content.
43
-
44
- ## Intended uses & limitations
45
-
46
- More information needed
47
-
48
- ## Training and evaluation data
49
-
50
- More information needed
51
-
52
- ## Training procedure
53
-
54
- ### Training hyperparameters
55
-
56
- The following hyperparameters were used during training:
57
- - learning_rate: 5e-06
58
- - train_batch_size: 8
59
- - eval_batch_size: 4
60
- - seed: 42
61
- - distributed_type: multi-GPU
62
- - num_devices: 2
63
- - total_train_batch_size: 16
64
- - total_eval_batch_size: 8
65
- - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
66
- - lr_scheduler_type: linear
67
- - lr_scheduler_warmup_steps: 300
68
- - num_epochs: 10
69
- - mixed_precision_training: Native AMP
70
-
71
- ### Training results
72
-
73
- | Training Loss | Epoch | Step | Validation Loss | Wer | Cer |
74
- |:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|
75
- | 0.6589 | 1.0 | 7154 | 0.6615 | 0.2735 | 0.2735 |
76
- | 0.6273 | 2.0 | 14308 | 0.6457 | 0.2699 | 0.2699 |
77
- | 0.6251 | 3.0 | 21462 | 0.6359 | 0.2660 | 0.2660 |
78
- | 0.6427 | 4.0 | 28616 | 0.6283 | 0.2642 | 0.2642 |
79
- | 0.6389 | 5.0 | 35770 | 0.6243 | 0.2631 | 0.2631 |
80
- | 0.6078 | 6.0 | 42924 | 0.6242 | 0.2615 | 0.2615 |
81
- | 0.5788 | 7.0 | 50078 | 0.6195 | 0.2603 | 0.2603 |
82
- | 0.5801 | 8.0 | 57232 | 0.6180 | 0.2596 | 0.2596 |
83
- | 0.5866 | 9.0 | 64386 | 0.6145 | 0.2598 | 0.2598 |
84
- | 0.6052 | 10.0 | 71540 | 0.6168 | 0.2600 | 0.2600 |
85
-
86
-
87
- ### Framework versions
88
-
89
- - Transformers 4.56.1
90
- - Pytorch 2.8.0+cu128
91
- - Datasets 4.0.0
92
- - Tokenizers 0.22.0
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: ctranslate2
3
+ license: apache-2.0
4
+ base_model: openai/whisper-small
5
+ tags:
6
+ - audio
7
+ - automatic-speech-recognition
8
+ - ctranslate2
9
+ - faster-whisper
10
+ - generated_from_trainer
11
+ - whisper
12
+ metrics:
13
+ - wer
14
+ model-index:
15
+ - name: whisper-small-jp
16
+ results: []
17
+ ---
18
+
19
+ > **This repository contains the CTranslate2 export of the fine-tuned model.**
20
+ >
21
+ > • Base Transformers model: [drepic/whisper-small-jp](https://huggingface.co/drepic/whisper-small-jp)
22
+ > • Use with `faster-whisper`:
23
+ >
24
+ > ```python
25
+ > from faster_whisper import WhisperModel
26
+ > model = WhisperModel("drepic/whisper-small-jp-ct2", device="cuda", compute_type="float16")
27
+ > ```
28
+
29
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
30
+ should probably proofread and complete it, then remove this comment. -->
31
+
32
+ # whisper-small-jp
33
+
34
+ This model is a fine-tuned version of [openai/whisper-small](https://huggingface.co/openai/whisper-small) on an unknown dataset.
35
+ It achieves the following results on the evaluation set:
36
+ - Loss: 0.6168
37
+ - Wer: 0.2600
38
+ - Cer: 0.2600
39
+
40
+ ## Model description
41
+
42
+ Better suited for transcribing japanese youtube content.
43
+
44
+ ## Intended uses & limitations
45
+
46
+ More information needed
47
+
48
+ ## Training and evaluation data
49
+
50
+ More information needed
51
+
52
+ ## Training procedure
53
+
54
+ ### Training hyperparameters
55
+
56
+ The following hyperparameters were used during training:
57
+ - learning_rate: 5e-06
58
+ - train_batch_size: 8
59
+ - eval_batch_size: 4
60
+ - seed: 42
61
+ - distributed_type: multi-GPU
62
+ - num_devices: 2
63
+ - total_train_batch_size: 16
64
+ - total_eval_batch_size: 8
65
+ - optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
66
+ - lr_scheduler_type: linear
67
+ - lr_scheduler_warmup_steps: 300
68
+ - num_epochs: 10
69
+ - mixed_precision_training: Native AMP
70
+
71
+ ### Training results
72
+
73
+ | Training Loss | Epoch | Step | Validation Loss | Wer | Cer |
74
+ |:-------------:|:-----:|:-----:|:---------------:|:------:|:------:|
75
+ | 0.6589 | 1.0 | 7154 | 0.6615 | 0.2735 | 0.2735 |
76
+ | 0.6273 | 2.0 | 14308 | 0.6457 | 0.2699 | 0.2699 |
77
+ | 0.6251 | 3.0 | 21462 | 0.6359 | 0.2660 | 0.2660 |
78
+ | 0.6427 | 4.0 | 28616 | 0.6283 | 0.2642 | 0.2642 |
79
+ | 0.6389 | 5.0 | 35770 | 0.6243 | 0.2631 | 0.2631 |
80
+ | 0.6078 | 6.0 | 42924 | 0.6242 | 0.2615 | 0.2615 |
81
+ | 0.5788 | 7.0 | 50078 | 0.6195 | 0.2603 | 0.2603 |
82
+ | 0.5801 | 8.0 | 57232 | 0.6180 | 0.2596 | 0.2596 |
83
+ | 0.5866 | 9.0 | 64386 | 0.6145 | 0.2598 | 0.2598 |
84
+ | 0.6052 | 10.0 | 71540 | 0.6168 | 0.2600 | 0.2600 |
85
+
86
+
87
+ ### Framework versions
88
+
89
+ - Transformers 4.56.1
90
+ - Pytorch 2.8.0+cu128
91
+ - Datasets 4.0.0
92
+ - Tokenizers 0.22.0
93
+
94
+
95
+ ## Evaluation Results
96
+
97
+ ### mozilla-foundation/common_voice_17_0 (ja) — split: test
98
+
99
+ - **CER**: 0.2302
100
+ - **Dataset**: mozilla-foundation/common_voice_17_0
101
+ - **Language/Subset**: ja
102
+ - **Split**: test