File size: 5,007 Bytes
d368d2e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
---
license: mit
base_model: severinsimmler/xlm-roberta-longformer-base-16384
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: longformer_pos_neg
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# longformer_pos_neg
This model is a fine-tuned version of [severinsimmler/xlm-roberta-longformer-base-16384](https://huggingface.co/severinsimmler/xlm-roberta-longformer-base-16384) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.5549
- Precision: 0.5599
- Recall: 0.5786
- F1: 0.5691
- Accuracy: 0.9030
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 100
### Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|:-------------:|:-----:|:----:|:---------------:|:---------:|:------:|:------:|:--------:|
| No log | 1.35 | 50 | 0.7729 | 0.0 | 0.0 | 0.0 | 0.7762 |
| No log | 2.7 | 100 | 0.5497 | 0.0220 | 0.0078 | 0.0115 | 0.8017 |
| No log | 4.05 | 150 | 0.4143 | 0.0706 | 0.0698 | 0.0702 | 0.8383 |
| No log | 5.41 | 200 | 0.3607 | 0.2329 | 0.2578 | 0.2447 | 0.8632 |
| No log | 6.76 | 250 | 0.3320 | 0.3628 | 0.3101 | 0.3344 | 0.8807 |
| No log | 8.11 | 300 | 0.3261 | 0.5108 | 0.4574 | 0.4826 | 0.8939 |
| No log | 9.46 | 350 | 0.3190 | 0.4229 | 0.5950 | 0.4944 | 0.8826 |
| No log | 10.81 | 400 | 0.2662 | 0.4821 | 0.6008 | 0.5349 | 0.9014 |
| No log | 12.16 | 450 | 0.2714 | 0.5901 | 0.5775 | 0.5837 | 0.9137 |
| 0.3792 | 13.51 | 500 | 0.2852 | 0.5769 | 0.5891 | 0.5829 | 0.9105 |
| 0.3792 | 14.86 | 550 | 0.3868 | 0.5876 | 0.5329 | 0.5589 | 0.9082 |
| 0.3792 | 16.22 | 600 | 0.3218 | 0.5444 | 0.6531 | 0.5938 | 0.9129 |
| 0.3792 | 17.57 | 650 | 0.3022 | 0.5645 | 0.6357 | 0.5980 | 0.9112 |
| 0.3792 | 18.92 | 700 | 0.3737 | 0.5419 | 0.6764 | 0.6017 | 0.9025 |
| 0.3792 | 20.27 | 750 | 0.3730 | 0.5411 | 0.6628 | 0.5958 | 0.9119 |
| 0.3792 | 21.62 | 800 | 0.4021 | 0.6145 | 0.6240 | 0.6192 | 0.9109 |
| 0.3792 | 22.97 | 850 | 0.3358 | 0.5159 | 0.6298 | 0.5672 | 0.9008 |
| 0.3792 | 24.32 | 900 | 0.3779 | 0.6065 | 0.6124 | 0.6095 | 0.9138 |
| 0.3792 | 25.68 | 950 | 0.4435 | 0.5293 | 0.6298 | 0.5752 | 0.9063 |
| 0.0755 | 27.03 | 1000 | 0.4230 | 0.6333 | 0.6124 | 0.6227 | 0.9169 |
| 0.0755 | 28.38 | 1050 | 0.3666 | 0.5911 | 0.6415 | 0.6152 | 0.9163 |
| 0.0755 | 29.73 | 1100 | 0.3335 | 0.6098 | 0.6512 | 0.6298 | 0.9178 |
| 0.0755 | 31.08 | 1150 | 0.4606 | 0.5725 | 0.6202 | 0.5953 | 0.9075 |
| 0.0755 | 32.43 | 1200 | 0.4280 | 0.5656 | 0.6434 | 0.6020 | 0.9065 |
| 0.0755 | 33.78 | 1250 | 0.4003 | 0.5833 | 0.6376 | 0.6093 | 0.9158 |
| 0.0755 | 35.14 | 1300 | 0.5802 | 0.6422 | 0.5775 | 0.6082 | 0.9020 |
| 0.0755 | 36.49 | 1350 | 0.4503 | 0.6014 | 0.6550 | 0.6271 | 0.9172 |
| 0.0755 | 37.84 | 1400 | 0.5614 | 0.6643 | 0.5523 | 0.6032 | 0.9044 |
| 0.0755 | 39.19 | 1450 | 0.5082 | 0.628 | 0.6085 | 0.6181 | 0.9119 |
| 0.0407 | 40.54 | 1500 | 0.3964 | 0.6072 | 0.6531 | 0.6293 | 0.9165 |
| 0.0407 | 41.89 | 1550 | 0.5447 | 0.4572 | 0.6938 | 0.5512 | 0.8799 |
| 0.0407 | 43.24 | 1600 | 0.5303 | 0.4816 | 0.6589 | 0.5565 | 0.8947 |
| 0.0407 | 44.59 | 1650 | 0.4461 | 0.6409 | 0.6260 | 0.6333 | 0.9138 |
| 0.0407 | 45.95 | 1700 | 0.6884 | 0.5561 | 0.4031 | 0.4674 | 0.8766 |
| 0.0407 | 47.3 | 1750 | 0.4556 | 0.5431 | 0.6105 | 0.5748 | 0.9097 |
| 0.0407 | 48.65 | 1800 | 0.4272 | 0.6771 | 0.5853 | 0.6279 | 0.9183 |
| 0.0407 | 50.0 | 1850 | 0.4904 | 0.5603 | 0.6570 | 0.6048 | 0.9015 |
| 0.0407 | 51.35 | 1900 | 0.4206 | 0.5655 | 0.6357 | 0.5985 | 0.9135 |
### Framework versions
- Transformers 4.38.2
- Pytorch 2.1.2
- Datasets 2.1.0
- Tokenizers 0.15.2
|