win10/Llama-3.2-3B-Instruct-24-9-29

my DC sever

https://discord.gg/yaTfFF6Ut2

我正在計畫微調64K指令模型，請幫助我進行計畫

Support me here if you're interested:

Ko-fi: https://ko-fi.com/ogodwin10

Llama-3.2-3B-Instruct-24-9-29

This model is a fine-tuned version of unsloth/Llama-3.2-3B-Instruct on the lmsys_chat dataset. It achieves the following results on the evaluation set:

• Loss: 1.1817

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 2
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• training_steps: 1000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
1.256	0.0160	100	1.1817
1.236	0.0320	200	1.1817
1.2212	0.0480	300	1.1817
1.1804	0.0641	400	1.1817
1.2801	0.0801	500	1.1817
1.2232	0.0961	600	1.1817
1.2433	0.1121	700	1.1817
1.2231	0.1281	800	1.1817
1.2272	0.1441	900	1.1817
1.2843	0.1602	1000	1.1817

Framework versions

• Transformers 4.45.0
• Pytorch 2.4.0+cu124
• Datasets 2.19.1
• Tokenizers 0.20.0