lemonhat/Qwen2.5-7B-Instruct-2and3_apps_30k_v6_new

2and3_apps_30k_v6_new

This model is a fine-tuned version of Qwen/Qwen2.5-7B-Instruct on the 2and3_apps_30k_v6_new dataset. It achieves the following results on the evaluation set:

• Loss: 0.1566

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• total_eval_batch_size: 4
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
0.1978	0.2049	100	0.1760
0.1779	0.4098	200	0.1688
0.1786	0.6148	300	0.1669
0.1736	0.8197	400	0.1641
0.1626	1.0246	500	0.1595
0.1573	1.2295	600	0.1593
0.1605	1.4344	700	0.1585
0.1596	1.6393	800	0.1576
0.1583	1.8443	900	0.1570

Framework versions

• Transformers 4.46.1
• Pytorch 2.6.0+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3