git-base-pokemon-captioning-generate

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.1551
• Wer Score: 4.4399

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 40
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer Score
4.6973	4.1702	100	0.4374	15.2827
0.1366	8.3404	200	0.1118	4.2933
0.0431	12.5106	300	0.1225	4.5509
0.0115	16.6809	400	0.1358	5.0092
0.003	20.8511	500	0.1445	5.2642
0.0015	25.0	600	0.1506	4.6816
0.0012	29.1702	700	0.1537	4.3778
0.0011	33.3404	800	0.1542	4.1532
0.001	37.5106	900	0.1552	4.3699

Framework versions

• Transformers 4.56.1
• Pytorch 2.6.0+cu124
• Datasets 3.6.0
• Tokenizers 0.22.0