fr_childes_42

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.8718

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 40000
• training_steps: 100000

Training results

Training Loss	Epoch	Step	Validation Loss
No log	2.5	2000	7.2323
7.2283	5.0	4000	5.9768
7.2283	7.5	6000	5.8268
5.6892	10.0	8000	5.7444
5.6892	12.5	10000	5.6708
5.4694	15.0	12000	5.5713
5.4694	17.5	14000	5.5270
5.3091	20.0	16000	5.4483
5.3091	22.5	18000	5.3926
5.1707	25.0	20000	5.2315
5.1707	27.5	22000	4.9059
4.6992	30.0	24000	4.1680
4.6992	32.5	26000	3.6409
3.5699	35.0	28000	3.2064
3.5699	37.5	30000	3.0010
2.9074	40.0	32000	2.8509
2.9074	42.5	34000	2.7339
2.6073	45.0	36000	2.6182
2.6073	47.5	38000	2.5613
2.4166	50.0	40000	2.4946
2.4166	52.5	42000	2.4197
2.2667	55.0	44000	2.3687
2.2667	57.5	46000	2.2802
2.146	60.0	48000	2.2621
2.146	62.5	50000	2.2170
2.0465	65.0	52000	2.1907
2.0465	67.5	54000	2.1659
1.969	70.0	56000	2.1273
1.969	72.5	58000	2.0874
1.9056	75.0	60000	2.0743
1.9056	77.5	62000	2.0583
1.8492	80.0	64000	2.0371
1.8492	82.5	66000	2.0039
1.8024	85.0	68000	1.9901
1.8024	87.5	70000	1.9754
1.7652	90.0	72000	1.9566
1.7652	92.5	74000	1.9404
1.7344	95.0	76000	1.9128
1.7344	97.5	78000	1.9396
1.7055	100.0	80000	1.9591
1.7055	102.5	82000	1.9078
1.6779	105.0	84000	1.9178
1.6779	107.5	86000	1.9046
1.6529	110.0	88000	1.8918
1.6529	112.5	90000	1.9010
1.6347	115.0	92000	1.8959
1.6347	117.5	94000	1.9094
1.6225	120.0	96000	1.8838
1.6225	122.5	98000	1.8972
1.6132	125.0	100000	1.8718

Framework versions

• Transformers 4.45.2
• Pytorch 2.5.1+cu124
• Datasets 3.0.1
• Tokenizers 0.20.1