File size: 2,949 Bytes
21e3286 1a79db7 180ac67 1a79db7 21e3286 180ac67 21e3286 180ac67 a4d7ac8 67901b5 a4d7ac8 40c0754 21e3286 a4d7ac8 f73eaa9 21e3286 b9314d5 a4d7ac8 21e3286 1a79db7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
---
library_name: transformers
license: mit
base_model: openai/whisper-large-v3-turbo
tags:
- generated_from_trainer
metrics:
- bleu
model-index:
- name: whisper-large-v3-turbo-gl-en
results: []
datasets:
- juanjucm/OpenHQ-SpeechT-GL-EN
language:
- gl
- en
---
# whisper-large-v3-turbo-OpenHQ-GL-EN
This model is a fine-tuned version of [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo) trained on [juanjucm/OpenHQ-SpeechT-GL-EN](https://huggingface.co/datasets/juanjucm/OpenHQ-SpeechT-GL-EN) for **Galician-to-English Text to Speech Translation** task. It takes galician speech audios as input and generates the correspondant translated transcription in English.
The motivation behind this work is to increase the visibility of the Galician language, making it more accessible for non-Galician speakers to understand and engage with Galician audio content.
This model was developed during a 3-week Speech Translation workshop organised by [Yasmin Moslem](https://huggingface.co/ymoslem).
### Performance and training details
Baseline model achieved a BLEU score of **3.38** on the evaluation dataset.
After fine-tuning, it achieves the following results on the evaluation set:
- Loss: 0.9360
- **BLEU: 55.6535**
- **ChrF++: 72.19**
The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 16
- eval_batch_size: 8
- seed: 42
- distributed_type: multi-GPU
- num_devices: 2
- total_train_batch_size: 32
- total_eval_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- training_steps: 3500
- mixed_precision_training: Native AMP
### Training results
We used [BLEU Score](https://en.wikipedia.org/wiki/BLEU) as our reference translation metric for selecting the best checkpoint after training.
| Training Loss | Epoch | Step | Validation Loss | Bleu |
|:-------------:|:-------:|:----:|:---------------:|:-------:|
| 0.2758 | 1.6667 | 250 | 0.7646 | 50.6055 |
| 0.0592 | 3.3333 | 500 | 0.7730 | 53.1258 |
| 0.0406 | 5.0 | 750 | 0.7860 | 53.3406 |
| 0.0173 | 6.6667 | 1000 | 0.8358 | 51.9789 |
| 0.0091 | 8.3333 | 1250 | 0.8909 | 54.4806 |
| 0.0071 | 10.0 | 1500 | 0.8862 | 54.2655 |
| 0.0039 | 11.6667 | 1750 | 0.9216 | 52.5119 |
| 0.0014 | 13.3333 | 2000 | 0.9281 | 54.5752 |
| 0.0013 | 15.0 | 2250 | 0.9471 | 54.5791 |
| 0.0009 | 16.6667 | 2500 | 0.9541 | 54.8725 |
| 0.0006 | 18.3333 | 2750 | 0.9614 | 53.1879 |
| 0.0006 | 20.0 | 3000 | 0.9701 | 54.6499 |
| 0.0006 | 21.6667 | 3250 | 0.9739 | 54.4341 |
| 0.0006 | 23.3333 | 3500 | 0.9747 | 54.5311 |
### Framework versions
- Transformers 4.45.1
- Pytorch 2.4.1+cu121
- Datasets 3.0.1
- Tokenizers 0.20.0 |