Model save

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [meta-llama/Llama-3.1-8B](https://huggingface.co/meta-llama/Llama-3.1-8B) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.5224
+- Loss: 0.6825
 
 ## Model description
 
@@ -38,70 +38,25 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 0.0002
-- train_batch_size: 6
+- train_batch_size: 2
 - eval_batch_size: 8
 - seed: 42
-- gradient_accumulation_steps: 6
-- total_train_batch_size: 36
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 4
 - optimizer: Use paged_adamw_32bit with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
-- training_steps: 100
+- training_steps: 75
 - mixed_precision_training: Native AMP
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| No log        | 1.0   | 2    | 1.1349          |
-| No log        | 2.0   | 4    | 1.1026          |
-| No log        | 3.0   | 6    | 1.0619          |
-| No log        | 4.0   | 8    | 1.0198          |
-| 1.3117        | 5.0   | 10   | 0.9727          |
-| 1.3117        | 6.0   | 12   | 0.9251          |
-| 1.3117        | 7.0   | 14   | 0.8814          |
-| 1.3117        | 8.0   | 16   | 0.8433          |
-| 1.3117        | 9.0   | 18   | 0.8134          |
-| 1.0107        | 10.0  | 20   | 0.7857          |
-| 1.0107        | 11.0  | 22   | 0.7571          |
-| 1.0107        | 12.0  | 24   | 0.7359          |
-| 1.0107        | 13.0  | 26   | 0.7097          |
-| 1.0107        | 14.0  | 28   | 0.6873          |
-| 0.8097        | 15.0  | 30   | 0.6693          |
-| 0.8097        | 16.0  | 32   | 0.6487          |
-| 0.8097        | 17.0  | 34   | 0.6325          |
-| 0.8097        | 18.0  | 36   | 0.6193          |
-| 0.8097        | 19.0  | 38   | 0.6013          |
-| 0.6699        | 20.0  | 40   | 0.5934          |
-| 0.6699        | 21.0  | 42   | 0.5781          |
-| 0.6699        | 22.0  | 44   | 0.5656          |
-| 0.6699        | 23.0  | 46   | 0.5601          |
-| 0.6699        | 24.0  | 48   | 0.5424          |
-| 0.5545        | 25.0  | 50   | 0.5361          |
-| 0.5545        | 26.0  | 52   | 0.5246          |
-| 0.5545        | 27.0  | 54   | 0.5220          |
-| 0.5545        | 28.0  | 56   | 0.5176          |
-| 0.5545        | 29.0  | 58   | 0.5143          |
-| 0.4559        | 30.0  | 60   | 0.5167          |
-| 0.4559        | 31.0  | 62   | 0.5079          |
-| 0.4559        | 32.0  | 64   | 0.5148          |
-| 0.4559        | 33.0  | 66   | 0.5128          |
-| 0.4559        | 34.0  | 68   | 0.5095          |
-| 0.3905        | 35.0  | 70   | 0.5110          |
-| 0.3905        | 36.0  | 72   | 0.5153          |
-| 0.3905        | 37.0  | 74   | 0.5135          |
-| 0.3905        | 38.0  | 76   | 0.5136          |
-| 0.3905        | 39.0  | 78   | 0.5196          |
-| 0.3472        | 40.0  | 80   | 0.5182          |
-| 0.3472        | 41.0  | 82   | 0.5164          |
-| 0.3472        | 42.0  | 84   | 0.5180          |
-| 0.3472        | 43.0  | 86   | 0.5214          |
-| 0.3472        | 44.0  | 88   | 0.5235          |
-| 0.328         | 45.0  | 90   | 0.5234          |
-| 0.328         | 46.0  | 92   | 0.5228          |
-| 0.328         | 47.0  | 94   | 0.5224          |
-| 0.328         | 48.0  | 96   | 0.5225          |
-| 0.328         | 49.0  | 98   | 0.5225          |
-| 0.3182        | 50.0  | 100  | 0.5224          |
+| Training Loss | Epoch  | Step | Validation Loss |
+|:-------------:|:------:|:----:|:---------------:|
+| 1.1149        | 0.9714 | 17   | 0.9204          |
+| 1.1694        | 2.0    | 35   | 0.7661          |
+| 0.9686        | 2.9714 | 52   | 0.6987          |
+| 0.876         | 4.0    | 70   | 0.6828          |
+| 0.6826        | 4.2857 | 75   | 0.6825          |
 
 
 ### Framework versions

runs/Oct29_12-37-06_pest/events.out.tfevents.1730205428.pest.1.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1f50b88b10df8eb4da7d5a23818ce5a8a70818c16046a857a5801315ed15797e
-size 10309
+oid sha256:40d9098b401e0319194401fdc228e28a7ec53300c3a803bb1473dcbeecdf7ade
+size 10923