cpt core 4

README.md

@@ -430,15 +430,17 @@ O^O/ \_/ \    Batch size per device = 1 | Gradient accumulation steps = 1
 wandb: Using wandb-core as the SDK backend.  Please refer to https://wandb.me/wandb-core for more information.
 wandb: Currently logged in as: mtasic85 to https://api.wandb.ai. Use `wandb login --relogin` to force relogin
 wandb: Tracking run with wandb version 0.19.8
-wandb: Run data is saved locally in /home/tangled/tangled-alpha-0.9-core/scripts/wandb/run-20250315_163140-55plsawx
+wandb: Run data is saved locally in /home/tangled/tangled-alpha-0.9-core/scripts/wandb/run-20250315_170030-4hx8rh26
 wandb: Run `wandb offline` to turn off syncing.
 wandb: Syncing run cpt-core-4
 wandb: ⭐️ View project at https://wandb.ai/mtasic85/tangled-alpha-0.9-core
-wandb: 🚀 View run at https://wandb.ai/mtasic85/tangled-alpha-0.9-core/runs/55plsawx
-{'loss': 1.9674, 'grad_norm': 3.058457851409912, 'learning_rate': 4.999999991182871e-05, 'epoch': 0.0}
-  0%|                                                                                                                                                                                             | 1/37406 [00:03<32:07:27,  3.09s/it]
+wandb: 🚀 View run at https://wandb.ai/mtasic85/tangled-alpha-0.9-core/runs/4hx8rh26
+{'loss': 1.7227, 'grad_norm': 0.4360397756099701, 'learning_rate': 4.999999991182871e-05, 'epoch': 0.0}
+  0%|                                                                                                                                                                                             | 1/37406 [00:03<32:10:32,  3.10s/it]
 Unsloth: Will smartly offload gradients to save VRAM!
-{'loss': 3.4846, 'grad_norm': 1.5452028512954712, 'learning_rate': 4.999999964731482e-05, 'epoch': 0.0}
-  0%|▎                                                                                                                                                                                           | 67/37406 [02:27<23:06:36,  2.23s/it]
+{'loss': 3.296, 'grad_norm': 0.6972701549530029, 'learning_rate': 4.999999964731482e-05, 'epoch': 0.0}
+{'loss': 1.7377, 'grad_norm': 0.5415557026863098, 'learning_rate': 4.9999999206458345e-05, 'epoch': 0.0}
+{'loss': 1.774, 'grad_norm': 0.3695605993270874, 'learning_rate': 4.99999985892593e-05, 'epoch': 0.0}
+  0%|                                                                                                                                                                                             | 4/37406 [00:09<24:03:55,  2.32s/it]
 # ...
 ```