Hugging Face's logo

floflodebilbao
/
Lora_LED_sum_outcome

PEFT
TensorBoard
Safetensors
Generated from Trainer
Model card Files Metrics Community
floflodebilbao commited on
Commit
8650000
·
verified ·
1 Parent(s): 58a683a

End of training

Browse files
Files changed (5) hide show
  1. README.md +29 -28
  2. adapter_config.json +3 -3
  3. adapter_model.safetensors +1 -1
  4. runs/Jul29_13-03-40_tardis/events.out.tfevents.1753787022.tardis.19624.0 +3 -0
  5. training_args.bin +1 -1
README.md CHANGED
@@ -22,21 +22,21 @@ should probably proofread and complete it, then remove this comment. -->
22
 
23
  This model is a fine-tuned version of [allenai/led-base-16384](https://huggingface.co/allenai/led-base-16384) on an unknown dataset.
24
  It achieves the following results on the evaluation set:
25
- - Loss: 3.7163
26
- - Rouge1: 0.3771
27
- - Rouge2: 0.1486
28
- - Rougel: 0.3048
29
- - Rougelsum: 0.3041
30
- - Gen Len: 26.78
31
- - Bleu: 0.0676
32
- - Precisions: 0.145
33
- - Brevity Penalty: 0.7751
34
- - Length Ratio: 0.7969
35
- - Translation Length: 934.0
36
  - Reference Length: 1172.0
37
- - Precision: 0.8956
38
- - Recall: 0.8849
39
- - F1: 0.8901
40
  - Hashcode: roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1)
41
 
42
  ## Model description
@@ -56,29 +56,30 @@ More information needed
56
  ### Training hyperparameters
57
 
58
  The following hyperparameters were used during training:
59
- - learning_rate: 0.001
60
- - train_batch_size: 8
61
- - eval_batch_size: 8
62
  - seed: 42
 
 
63
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
64
  - lr_scheduler_type: linear
65
  - num_epochs: 10
66
- - mixed_precision_training: Native AMP
67
 
68
  ### Training results
69
 
70
  | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len | Bleu | Precisions | Brevity Penalty | Length Ratio | Translation Length | Reference Length | Precision | Recall | F1 | Hashcode |
71
  |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|:------:|:----------:|:---------------:|:------------:|:------------------:|:----------------:|:---------:|:------:|:------:|:---------------------------------------------------------:|
72
- | 8.5991 | 1.0 | 13 | 6.3567 | 0.296 | 0.0973 | 0.2568 | 0.2557 | 31.66 | 0.0456 | 0.0891 | 1.0 | 1.0751 | 1260.0 | 1172.0 | 0.8683 | 0.8666 | 0.8674 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
73
- | 5.6132 | 2.0 | 26 | 4.4329 | 0.3194 | 0.1255 | 0.2695 | 0.2708 | 27.78 | 0.0667 | 0.1161 | 0.9275 | 0.93 | 1090.0 | 1172.0 | 0.8859 | 0.8748 | 0.8802 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
74
- | 4.4029 | 3.0 | 39 | 3.8761 | 0.3541 | 0.147 | 0.2867 | 0.2869 | 26.3 | 0.0667 | 0.1404 | 0.7466 | 0.7739 | 907.0 | 1172.0 | 0.8938 | 0.8809 | 0.8871 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
75
- | 4.0321 | 4.0 | 52 | 3.7464 | 0.3456 | 0.1434 | 0.2907 | 0.2907 | 23.88 | 0.0623 | 0.1454 | 0.6813 | 0.7227 | 847.0 | 1172.0 | 0.897 | 0.8791 | 0.8878 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
76
- | 3.8606 | 5.0 | 65 | 3.7107 | 0.377 | 0.1448 | 0.2955 | 0.2954 | 28.08 | 0.0704 | 0.133 | 0.8931 | 0.8985 | 1053.0 | 1172.0 | 0.8909 | 0.8825 | 0.8866 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
77
- | 3.7497 | 6.0 | 78 | 3.7107 | 0.4101 | 0.177 | 0.3294 | 0.3289 | 26.58 | 0.0952 | 0.1707 | 0.7947 | 0.8131 | 953.0 | 1172.0 | 0.9017 | 0.8886 | 0.895 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
78
- | 3.6478 | 7.0 | 91 | 3.7065 | 0.3755 | 0.1524 | 0.3027 | 0.3025 | 27.6 | 0.0654 | 0.1417 | 0.81 | 0.8259 | 968.0 | 1172.0 | 0.8893 | 0.8847 | 0.8869 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
79
- | 3.5904 | 8.0 | 104 | 3.7081 | 0.3692 | 0.1483 | 0.3055 | 0.3046 | 27.86 | 0.0669 | 0.1372 | 0.8311 | 0.8439 | 989.0 | 1172.0 | 0.8911 | 0.8834 | 0.8871 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
80
- | 3.5673 | 9.0 | 117 | 3.7045 | 0.3724 | 0.1591 | 0.3096 | 0.3081 | 26.8 | 0.0701 | 0.1492 | 0.7667 | 0.7901 | 926.0 | 1172.0 | 0.8958 | 0.8851 | 0.8903 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
81
- | 3.5516 | 10.0 | 130 | 3.7163 | 0.3771 | 0.1486 | 0.3048 | 0.3041 | 26.78 | 0.0676 | 0.145 | 0.7751 | 0.7969 | 934.0 | 1172.0 | 0.8956 | 0.8849 | 0.8901 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
82
 
83
 
84
  ### Framework versions
 
22
 
23
  This model is a fine-tuned version of [allenai/led-base-16384](https://huggingface.co/allenai/led-base-16384) on an unknown dataset.
24
  It achieves the following results on the evaluation set:
25
+ - Loss: 3.7571
26
+ - Rouge1: 0.3651
27
+ - Rouge2: 0.142
28
+ - Rougel: 0.2963
29
+ - Rougelsum: 0.2972
30
+ - Gen Len: 27.12
31
+ - Bleu: 0.065
32
+ - Precisions: 0.1385
33
+ - Brevity Penalty: 0.7854
34
+ - Length Ratio: 0.8055
35
+ - Translation Length: 944.0
36
  - Reference Length: 1172.0
37
+ - Precision: 0.8932
38
+ - Recall: 0.8832
39
+ - F1: 0.8881
40
  - Hashcode: roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1)
41
 
42
  ## Model description
 
56
  ### Training hyperparameters
57
 
58
  The following hyperparameters were used during training:
59
+ - learning_rate: 0.002
60
+ - train_batch_size: 1
61
+ - eval_batch_size: 1
62
  - seed: 42
63
+ - gradient_accumulation_steps: 16
64
+ - total_train_batch_size: 16
65
  - optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
66
  - lr_scheduler_type: linear
67
  - num_epochs: 10
 
68
 
69
  ### Training results
70
 
71
  | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len | Bleu | Precisions | Brevity Penalty | Length Ratio | Translation Length | Reference Length | Precision | Recall | F1 | Hashcode |
72
  |:-------------:|:-----:|:----:|:---------------:|:------:|:------:|:------:|:---------:|:-------:|:------:|:----------:|:---------------:|:------------:|:------------------:|:----------------:|:---------:|:------:|:------:|:---------------------------------------------------------:|
73
+ | 8.123 | 1.0 | 7 | 7.3532 | 0.2921 | 0.0954 | 0.2408 | 0.2414 | 32.0 | 0.0493 | 0.0883 | 1.0 | 1.1058 | 1296.0 | 1172.0 | 0.8636 | 0.8648 | 0.8641 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
74
+ | 5.947 | 2.0 | 14 | 5.1102 | 0.335 | 0.1361 | 0.2865 | 0.2889 | 22.94 | 0.0612 | 0.1526 | 0.6419 | 0.6928 | 812.0 | 1172.0 | 0.8992 | 0.8779 | 0.8883 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
75
+ | 4.4536 | 3.0 | 21 | 4.2132 | 0.3429 | 0.1435 | 0.2928 | 0.2937 | 22.68 | 0.0531 | 0.1512 | 0.5815 | 0.6485 | 760.0 | 1172.0 | 0.902 | 0.8794 | 0.8904 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
76
+ | 3.8549 | 4.0 | 28 | 3.9066 | 0.3534 | 0.1446 | 0.2939 | 0.2952 | 24.72 | 0.0661 | 0.146 | 0.7252 | 0.7568 | 887.0 | 1172.0 | 0.8976 | 0.8819 | 0.8896 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
77
+ | 3.5956 | 5.0 | 35 | 3.8415 | 0.3878 | 0.1685 | 0.3168 | 0.3168 | 25.48 | 0.0649 | 0.1552 | 0.7176 | 0.7509 | 880.0 | 1172.0 | 0.9023 | 0.8863 | 0.8941 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
78
+ | 3.4355 | 6.0 | 42 | 3.7729 | 0.3825 | 0.1597 | 0.3131 | 0.3135 | 26.08 | 0.0581 | 0.1496 | 0.7317 | 0.7619 | 893.0 | 1172.0 | 0.8975 | 0.8856 | 0.8914 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
79
+ | 3.3337 | 7.0 | 49 | 3.7560 | 0.3597 | 0.1448 | 0.2961 | 0.2973 | 27.3 | 0.0629 | 0.1368 | 0.8069 | 0.8234 | 965.0 | 1172.0 | 0.8941 | 0.8827 | 0.8882 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
80
+ | 3.2635 | 8.0 | 56 | 3.7481 | 0.3491 | 0.1411 | 0.2928 | 0.2938 | 25.9 | 0.046 | 0.1253 | 0.7519 | 0.7782 | 912.0 | 1172.0 | 0.8932 | 0.8811 | 0.887 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
81
+ | 3.2095 | 9.0 | 63 | 3.7572 | 0.3672 | 0.1422 | 0.3015 | 0.3019 | 26.68 | 0.0582 | 0.1374 | 0.7771 | 0.7986 | 936.0 | 1172.0 | 0.8958 | 0.885 | 0.8903 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
82
+ | 3.2068 | 10.0 | 70 | 3.7571 | 0.3651 | 0.142 | 0.2963 | 0.2972 | 27.12 | 0.065 | 0.1385 | 0.7854 | 0.8055 | 944.0 | 1172.0 | 0.8932 | 0.8832 | 0.8881 | roberta-large_L17_no-idf_version=0.3.12(hug_trans=4.53.1) |
83
 
84
 
85
  ### Framework versions
adapter_config.json CHANGED
@@ -24,10 +24,10 @@
24
  "rank_pattern": {},
25
  "revision": null,
26
  "target_modules": [
27
- "k_proj",
28
  "v_proj",
29
- "q_proj",
30
- "out_proj"
31
  ],
32
  "task_type": "SEQ_2_SEQ_LM",
33
  "trainable_token_indices": null,
 
24
  "rank_pattern": {},
25
  "revision": null,
26
  "target_modules": [
27
+ "out_proj",
28
  "v_proj",
29
+ "k_proj",
30
+ "q_proj"
31
  ],
32
  "task_type": "SEQ_2_SEQ_LM",
33
  "trainable_token_indices": null,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0cb64013c389c9e784e2b907eda5c9e83eebee7c74e98e54c35790deb099b4fc
3
  size 2372496
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1f8c4506324eec4c71a13ad43d29486641f13b1988308a52b0897aa8488b4b3
3
  size 2372496
runs/Jul29_13-03-40_tardis/events.out.tfevents.1753787022.tardis.19624.0 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:10c078360b248e89d627f615b678fab6bb9afff5822cd9215913b9e8d5c8a919
3
+ size 19368
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1f1ae0c722f8db17053cde485f3cd3eddff23c1cd47b814449e423d681e58dbf
3
  size 5905
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:985a1055ce59fe10f0fec7e9527bd7332efcc0d24b5b7edab8dce1072d2fb8c3
3
  size 5905