train_wic_1753094169

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on the wic dataset. It achieves the following results on the evaluation set:

• Loss: 0.2264
• Num Input Tokens Seen: 4213808

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 123
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10.0

Training results

Training Loss	Epoch	Step	Validation Loss	Input Tokens Seen
0.3973	0.5	611	0.2512	210240
0.1679	1.0	1222	0.2457	421528
0.2039	1.5	1833	0.2264	632632
0.1124	2.0	2444	0.2628	843368
0.0416	2.5	3055	0.3458	1054024
0.1077	3.0	3666	0.2453	1264408
0.1233	3.5	4277	0.3687	1475000
0.3016	4.0	4888	0.3397	1685768
0.0943	4.5	5499	0.4715	1895752
0.2191	5.0	6110	0.4199	2106968
0.0003	5.5	6721	0.5726	2318136
0.0	6.0	7332	0.6758	2528648
0.0	6.5	7943	0.8199	2739720
0.0	7.0	8554	0.7548	2949592
0.0	7.5	9165	0.7159	3160056
0.0	8.0	9776	0.7366	3371056
0.0	8.5	10387	0.7989	3581616
0.0	9.0	10998	0.8500	3792672
0.0	9.5	11609	0.8643	4003136
0.0	10.0	12220	0.8656	4213808

Framework versions

• PEFT 0.15.2
• Transformers 4.51.3
• Pytorch 2.7.1+cu126
• Datasets 3.6.0
• Tokenizers 0.21.1