End of training

README.md

@@ -93,7 +93,7 @@ wandb_name: 861c0084-b845-4e35-9b2f-ffbd4ba5fae8
 wandb_project: Gradients-On-Demand
 wandb_run: unknown
 wandb_runid: 861c0084-b845-4e35-9b2f-ffbd4ba5fae8
-warmup_steps: 10
+warmup_steps: 80
 weight_decay: 0.01
 xformers_attention: null
 
@@ -105,7 +105,7 @@ xformers_attention: null
 
 This model is a fine-tuned version of [EleutherAI/pythia-160m](https://huggingface.co/EleutherAI/pythia-160m) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 2.6273
+- Loss: 2.6158
 
 ## Model description
 
@@ -132,7 +132,7 @@ The following hyperparameters were used during training:
 - total_train_batch_size: 32
 - optimizer: Use OptimizerNames.ADAMW_BNB with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: cosine
-- lr_scheduler_warmup_steps: 10
+- lr_scheduler_warmup_steps: 80
 - training_steps: 200
 
 ### Training results
@@ -140,11 +140,11 @@ The following hyperparameters were used during training:
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
 | No log        | 0.0001 | 1    | 2.7746          |
-| 43.6122       | 0.0050 | 40   | 2.7383          |
-| 43.0998       | 0.0100 | 80   | 2.6859          |
-| 42.6668       | 0.0150 | 120  | 2.6449          |
-| 41.707        | 0.0200 | 160  | 2.6281          |
-| 41.4304       | 0.0250 | 200  | 2.6273          |
+| 43.8063       | 0.0050 | 40   | 2.7672          |
+| 43.9191       | 0.0100 | 80   | 2.7285          |
+| 43.3074       | 0.0150 | 120  | 2.6594          |
+| 41.7411       | 0.0200 | 160  | 2.6218          |
+| 41.2435       | 0.0250 | 200  | 2.6158          |
 
 
 ### Framework versions

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c39c4c8541a10e7e07c3b71fea07fcccb673ef6d6496f6c0e743482927139f5c
+oid sha256:04bffbfad5424a541e12026012cd40446f21a484be199609b296b47a7fae0caa
 size 4753742