Update README.md

README.md

@@ -3,16 +3,23 @@ base_model: meta-llama/Llama-3.1-8B-Instruct
 library_name: transformers
 model_name: Llama-3.1-8B-KAM
 tags:
-- generated_from_trainer
 - trl
+- Llama
 - sft
+- generated_from_trainer
 licence: license
 ---
 
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
 # Model Card for Llama-3.1-8B-KAM
 
-This model is a fine-tuned version of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct).
-It has been trained using [TRL](https://github.com/huggingface/trl).
+This model is a fine-tuned version of [meta-llama/Llama-3.1-8B-Instruct](https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct) on the None dataset.
+
+## Model description
+
+More information needed
 
 ## Quick start
 
@@ -27,9 +34,35 @@ print(output["generated_text"])
 
 ## Training procedure
 
+This model was trained with SFT.
+
+### Training hyperparameters
 
+The following hyperparameters were used during training:
+- learning_rate: 0.0002
+- train_batch_size: 1
+- eval_batch_size: 8
+- seed: 3407
+- gradient_accumulation_steps: 16
+- total_train_batch_size: 8
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 20
+- training_steps: 500
+- mixed_precision_training: Native AMP
 
-This model was trained with SFT.
+### Training results
+#### Step &nbsp;&nbsp;	Training Loss
+  - 50	 &nbsp;&nbsp;&nbsp;   2.158200
+  - 100	 &nbsp;&nbsp;&nbsp;   1.845900
+  - 150	 &nbsp;&nbsp;&nbsp;   1.832200
+  - 200	 &nbsp;&nbsp;&nbsp;   1.805300
+  - 250	 &nbsp;&nbsp;&nbsp;   1.783800
+  - 300	 &nbsp;&nbsp;&nbsp;   1.767500
+  - 350	 &nbsp;&nbsp;&nbsp;   1.744800
+  - 400	 &nbsp;&nbsp;&nbsp;   1.745600
+  - 450	 &nbsp;&nbsp;&nbsp;   1.749500
+  - 500	 &nbsp;&nbsp;&nbsp;   1.756100
 
 ### Framework versions
 
@@ -37,21 +70,4 @@ This model was trained with SFT.
 - Transformers: 4.46.2
 - Pytorch: 2.4.0
 - Datasets: 3.0.1
-- Tokenizers: 0.20.0
-
-## Citations
-
-
-
-Cite TRL as:
-    
-```bibtex
-@misc{vonwerra2022trl,
-	title        = {{TRL: Transformer Reinforcement Learning}},
-	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},
-	year         = 2020,
-	journal      = {GitHub repository},
-	publisher    = {GitHub},
-	howpublished = {\url{https://github.com/huggingface/trl}}
-}
-```
+- Tokenizers: 0.20.0