AfriHuBERT-nyagen-female-42

This model is a fine-tuned version of ajesujoba/AfriHuBERT on the NYAGEN - NYA dataset. It achieves the following results on the evaluation set:

  • Loss: 2.8695
  • Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.001
  • train_batch_size: 8
  • eval_batch_size: 4
  • seed: 42
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 16
  • optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 30.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
5.5512 1.1117 100 2.8784 1.0
2.8606 2.2235 200 2.8762 1.0
2.8621 3.3352 300 2.8734 1.0
2.8607 4.4469 400 2.8671 1.0
2.8577 5.5587 500 2.8753 1.0
2.8559 6.6704 600 2.8659 1.0
2.8618 7.7821 700 2.8682 1.0
2.8547 8.8939 800 2.8689 1.0
2.8429 10.0 900 2.8722 1.0
2.8572 11.1117 1000 2.8646 1.0
2.8581 12.2235 1100 2.8732 1.0
2.8604 13.3352 1200 2.8693 1.0
2.8567 14.4469 1300 2.8832 1.0
2.8592 15.5587 1400 2.8699 1.0
2.8602 16.6704 1500 2.8732 1.0
2.8628 17.7821 1600 2.8689 1.0
2.8623 18.8939 1700 2.8656 1.0
2.849 20.0 1800 2.8894 1.0
2.861 21.1117 1900 2.8637 1.0
2.8586 22.2235 2000 2.8661 1.0
2.8578 23.3352 2100 2.8656 1.0
2.8571 24.4469 2200 2.8626 1.0
2.8551 25.5587 2300 2.8677 1.0
2.8565 26.6704 2400 2.8655 1.0
2.8537 27.7821 2500 2.8667 1.0
2.8542 28.8939 2600 2.8642 1.0
2.8391 30.0 2700 2.8650 1.0

Framework versions

  • Transformers 4.53.0.dev0
  • Pytorch 2.6.0+cu124
  • Datasets 3.6.0
  • Tokenizers 0.21.0
Downloads last month
4
Safetensors
Model size
94.4M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for csikasote/AfriHuBERT-nyagen-female-42

Finetuned
(2)
this model