README.md

---
license: mit
base_model: vinai/bertweet-base
tags:
- generated_from_trainer
metrics:
- f1
- precision
- recall
- accuracy
model-index:
- name: bertweet-base_ordinal_7_seed42_EN-NL
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# bertweet-base_ordinal_7_seed42_EN-NL

This model is a fine-tuned version of [vinai/bertweet-base](https://huggingface.co/vinai/bertweet-base) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 3.9585
- Mse: 6.1125
- Rmse: 2.4723
- Mae: 1.5109
- R2: 0.1525
- F1: 0.7447
- Precision: 0.7446
- Recall: 0.7479
- Accuracy: 0.7479

## Model description

This is the best-performing model for Dutch irony detection. The model was fine-tuned both a mix of English and Dutch tweets. 
The model predicts one of 7 labels indicating for irony likelihood, where 0 is not ironic and 6 is ironic.
When merging for binary classification, we advise mapping labels 0,1,2,3 as not-ironic and labels 4,5,6 as ironic.

## Intended uses & limitations

More information needed

## Training and evaluation data

The model was trained and evaluated on the TRIC dataset.

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-06
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 200
- num_epochs: 10

### Training results

| Training Loss | Epoch  | Step | Validation Loss | Mse    | Rmse   | Mae    | R2      | F1     | Precision | Recall | Accuracy |
|:-------------:|:------:|:----:|:---------------:|:------:|:------:|:------:|:-------:|:------:|:---------:|:------:|:--------:|
| 5.7557        | 0.2141 | 100  | 5.6476          | 7.6297 | 2.7622 | 2.6574 | -0.0806 | 0.4761 | 0.3859    | 0.6212 | 0.6212   |
| 5.2388        | 0.4283 | 200  | 5.2492          | 7.1086 | 2.6662 | 2.4741 | -0.0068 | 0.4761 | 0.3859    | 0.6212 | 0.6212   |
| 4.9773        | 0.6424 | 300  | 5.0558          | 6.8733 | 2.6217 | 2.3016 | 0.0266  | 0.4761 | 0.3859    | 0.6212 | 0.6212   |
| 4.7427        | 0.8565 | 400  | 4.8666          | 6.6212 | 2.5732 | 2.1990 | 0.0623  | 0.4761 | 0.3859    | 0.6212 | 0.6212   |
| 4.6378        | 1.0707 | 500  | 4.6806          | 6.0772 | 2.4652 | 2.0941 | 0.1393  | 0.6773 | 0.6795    | 0.6888 | 0.6888   |
| 4.3851        | 1.2848 | 600  | 4.6153          | 6.2799 | 2.5060 | 1.9928 | 0.1106  | 0.6915 | 0.6964    | 0.6888 | 0.6888   |
| 4.3077        | 1.4989 | 700  | 4.5016          | 6.2147 | 2.4929 | 1.9276 | 0.1198  | 0.6882 | 0.6928    | 0.7008 | 0.7008   |
| 4.2337        | 1.7131 | 800  | 4.2877          | 5.5862 | 2.3635 | 1.8854 | 0.2088  | 0.7183 | 0.7218    | 0.7274 | 0.7274   |
| 4.2273        | 1.9272 | 900  | 4.3769          | 5.9397 | 2.4371 | 1.8601 | 0.1588  | 0.6994 | 0.6991    | 0.6996 | 0.6996   |
| 4.0563        | 2.1413 | 1000 | 4.2168          | 6.1013 | 2.4701 | 1.7033 | 0.1359  | 0.7088 | 0.7203    | 0.7238 | 0.7238   |
| 3.7778        | 2.3555 | 1100 | 4.1356          | 6.1098 | 2.4718 | 1.6562 | 0.1347  | 0.7260 | 0.7269    | 0.7322 | 0.7322   |
| 3.7206        | 2.5696 | 1200 | 4.2222          | 6.1062 | 2.4711 | 1.7394 | 0.1352  | 0.7245 | 0.7326    | 0.7214 | 0.7214   |
| 3.7175        | 2.7837 | 1300 | 4.0073          | 5.7021 | 2.3879 | 1.6224 | 0.1924  | 0.7277 | 0.7345    | 0.7382 | 0.7382   |
| 3.8003        | 2.9979 | 1400 | 4.1116          | 5.8166 | 2.4118 | 1.7346 | 0.1762  | 0.7258 | 0.7268    | 0.7250 | 0.7250   |
| 3.6247        | 3.2120 | 1500 | 4.1286          | 6.0663 | 2.4630 | 1.6876 | 0.1409  | 0.7309 | 0.7355    | 0.7286 | 0.7286   |
| 3.4364        | 3.4261 | 1600 | 4.2100          | 6.3353 | 2.5170 | 1.7467 | 0.1028  | 0.7235 | 0.7329    | 0.7201 | 0.7201   |
| 3.3301        | 3.6403 | 1700 | 4.0403          | 6.0483 | 2.4593 | 1.6357 | 0.1434  | 0.7436 | 0.7442    | 0.7431 | 0.7431   |
| 3.3634        | 3.8544 | 1800 | 3.9496          | 5.5790 | 2.3620 | 1.6297 | 0.2099  | 0.7259 | 0.7282    | 0.7334 | 0.7334   |
| 3.4602        | 4.0685 | 1900 | 3.8729          | 5.7334 | 2.3945 | 1.5597 | 0.1880  | 0.7402 | 0.7410    | 0.7455 | 0.7455   |
| 3.1223        | 4.2827 | 2000 | 4.0417          | 6.3812 | 2.5261 | 1.5875 | 0.0963  | 0.7144 | 0.7394    | 0.7346 | 0.7346   |
| 3.1337        | 4.4968 | 2100 | 4.0039          | 5.9493 | 2.4391 | 1.6285 | 0.1574  | 0.7389 | 0.7421    | 0.7370 | 0.7370   |
| 3.1321        | 4.7109 | 2200 | 3.9092          | 5.8926 | 2.4275 | 1.5742 | 0.1655  | 0.7347 | 0.7339    | 0.7358 | 0.7358   |
| 3.1927        | 4.9251 | 2300 | 4.0312          | 5.9928 | 2.4480 | 1.6140 | 0.1513  | 0.7459 | 0.7540    | 0.7431 | 0.7431   |
| 2.9806        | 5.1392 | 2400 | 3.9638          | 6.0145 | 2.4524 | 1.5633 | 0.1482  | 0.7524 | 0.7536    | 0.7515 | 0.7515   |
| 2.9582        | 5.3533 | 2500 | 3.9413          | 5.9409 | 2.4374 | 1.5549 | 0.1586  | 0.7539 | 0.7539    | 0.7539 | 0.7539   |
| 2.7418        | 5.5675 | 2600 | 3.9578          | 5.9843 | 2.4463 | 1.5525 | 0.1525  | 0.7456 | 0.7476    | 0.7443 | 0.7443   |
| 2.9866        | 5.7816 | 2700 | 3.8793          | 5.8070 | 2.4098 | 1.5416 | 0.1776  | 0.7426 | 0.7425    | 0.7467 | 0.7467   |
| 2.8627        | 5.9957 | 2800 | 3.8625          | 5.7805 | 2.4043 | 1.5103 | 0.1813  | 0.7615 | 0.7609    | 0.7624 | 0.7624   |
| 2.8191        | 6.2099 | 2900 | 3.9342          | 5.9964 | 2.4488 | 1.5211 | 0.1508  | 0.7628 | 0.7622    | 0.7636 | 0.7636   |
| 2.6259        | 6.4240 | 3000 | 3.9203          | 6.0893 | 2.4676 | 1.5006 | 0.1376  | 0.7487 | 0.7478    | 0.7503 | 0.7503   |
| 2.8785        | 6.6381 | 3100 | 3.8633          | 5.8444 | 2.4175 | 1.4946 | 0.1723  | 0.7600 | 0.7601    | 0.7600 | 0.7600   |
| 2.6016        | 6.8522 | 3200 | 4.0736          | 6.2654 | 2.5031 | 1.5923 | 0.1127  | 0.7456 | 0.7518    | 0.7431 | 0.7431   |
| 2.5155        | 7.0664 | 3300 | 3.9459          | 6.0688 | 2.4635 | 1.5211 | 0.1405  | 0.7584 | 0.7597    | 0.7575 | 0.7575   |
| 2.6918        | 7.2805 | 3400 | 3.9312          | 6.0072 | 2.4510 | 1.5271 | 0.1492  | 0.7541 | 0.7534    | 0.7551 | 0.7551   |


### Framework versions

- Transformers 4.40.2
- Pytorch 2.1.2
- Datasets 2.18.0
- Tokenizers 0.19.1