End of training

README.md

@@ -15,14 +15,14 @@ This student model is distilled from the teacher model [roneneldan/TinyStories-3
 The [Distily](https://github.com/lapp0/distily) library was used for this distillation.
 
 It achieves the following results on the evaluation set:
-- eval_enwikippl: 19431.7090
-- eval_frwikippl: 76316.3438
-- eval_zhwikippl: 60793.4219
-- eval_tinystoriesppl: 10653.4062
-- eval_loss: 6.2558
-- eval_runtime: 6.5566
-- eval_samples_per_second: 76.259
-- eval_steps_per_second: 9.609
+- eval_enwikippl: 104.6652
+- eval_frwikippl: 13772.8643
+- eval_zhwikippl: 74161.4531
+- eval_tinystoriesppl: 5.5260
+- eval_loss: 0.7819
+- eval_runtime: 6.5261
+- eval_samples_per_second: 76.616
+- eval_steps_per_second: 9.654
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment.
@@ -62,17 +62,17 @@ Peak GPU Memory: 6.6064 GB
 | step | epoch | enwikippl | frwikippl | loss | runtime | samples_per_second | steps_per_second | tinystoriesppl | zhwikippl |
 | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 | **teacher eval** |  | 169.9865 | 47377.9414 |  |  |  |  | 3.9789 | 4998.1294 |
-| 0 | 0 | 24265.9961 | 83952.2266 | 6.4532 | 6.5311 | 76.556 | 9.646 | 14059.6133 | 62337.3242 |
-| 5000 | 0.1010 | 19431.7090 | 76316.3438 | 6.2558 | 6.5566 | 76.259 | 9.609 | 10653.4062 | 60793.4219 |
-| 10000 | 0.2020 | 19431.7090 | 76316.3438 | 6.2558 | 6.5391 | 76.464 | 9.634 | 10653.4062 | 60793.4219 |
-| 15000 | 0.3030 | 19431.7090 | 76316.3438 | 6.2558 | 6.5321 | 76.546 | 9.645 | 10653.4062 | 60793.4219 |
-| 20000 | 0.4040 | 19431.7090 | 76316.3438 | 6.2558 | 6.5334 | 76.53 | 9.643 | 10653.4062 | 60793.4219 |
-| 25000 | 0.5051 | 19431.7090 | 76316.3438 | 6.2558 | 6.5891 | 75.883 | 9.561 | 10653.4062 | 60793.4219 |
-| 30000 | 0.6061 | 19431.7090 | 76316.3438 | 6.2558 | 6.6149 | 75.587 | 9.524 | 10653.4062 | 60793.4219 |
-| 35000 | 0.7071 | 19431.7090 | 76316.3438 | 6.2558 | 6.5246 | 76.633 | 9.656 | 10653.4062 | 60793.4219 |
-| 40000 | 0.8081 | 19431.7090 | 76316.3438 | 6.2558 | 6.5481 | 76.358 | 9.621 | 10653.4062 | 60793.4219 |
-| 45000 | 0.9091 | 19431.7090 | 76316.3438 | 6.2558 | 6.5442 | 76.403 | 9.627 | 10653.4062 | 60793.4219 |
-| 49500 | 1.0 | 19431.7090 | 76316.3438 | 6.2558 | 6.5486 | 76.352 | 9.62 | 10653.4062 | 60793.4219 |
+| 0 | 0 | 25306.5312 | 80342.6562 | 6.4738 | 6.54 | 76.453 | 9.633 | 14565.9658 | 71518.8438 |
+| 5000 | 0.1010 | 104.6652 | 13772.8643 | 0.7819 | 6.5261 | 76.616 | 9.654 | 5.5260 | 74161.4531 |
+| 10000 | 0.2020 | 144.5553 | 13569.7109 | 0.7842 | 6.5111 | 76.792 | 9.676 | 8.9185 | 62270.8359 |
+| 15000 | 0.3030 | 105.4526 | 12598.8818 | 0.7708 | 6.5186 | 76.704 | 9.665 | 5.6194 | 53872.625 |
+| 20000 | 0.4040 | 121.5509 | 12060.5781 | 0.7610 | 6.5194 | 76.694 | 9.663 | 7.1313 | 52133.4336 |
+| 25000 | 0.5051 | 111.6548 | 13016.4775 | 0.7537 | 6.5166 | 76.727 | 9.668 | 6.0700 | 53485.9688 |
+| 30000 | 0.6061 | 101.6823 | 11441.6719 | 0.7577 | 6.5294 | 76.577 | 9.649 | 5.7104 | 48007.9258 |
+| 35000 | 0.7071 | 97.8760 | 10992.2207 | 0.7519 | 6.5151 | 76.745 | 9.67 | 5.5543 | 47549.0430 |
+| 40000 | 0.8081 | 114.8546 | 11104.2744 | 0.7378 | 6.5634 | 76.18 | 9.599 | 6.9089 | 42804.5273 |
+| 45000 | 0.9091 | 112.3336 | 11524.9678 | 0.7228 | 6.5648 | 76.164 | 9.597 | 6.6096 | 46781.4727 |
+| 49500 | 1.0 | 110.1023 | 10899.7119 | 0.7097 | 6.5487 | 76.351 | 9.62 | 6.5616 | 49450.9141 |
 
 ### Framework versions
 - Distily 0.2.0

logs/copy_teacher_modules=_(_lm_head___True)_, dropout=0, learning_rate=4e-05, warmup_ratio=0, weight_decay=0.01/events.out.tfevents.1723998203.5f530b1cf724

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8813fe3fae697f2f0640c456e3db71924747a0db2c576e2494376e348b7e060
+size 312