Improve model card: license, tags, paper & GitHub links, usage, and description

#1
by nielsr HF Staff - opened
Files changed (1) hide show
  1. README.md +65 -14
README.md CHANGED
@@ -1,25 +1,32 @@
1
  ---
2
- library_name: transformers
3
- license: apache-2.0
4
  base_model: answerdotai/ModernBERT-base
5
- tags:
6
- - generated_from_trainer
 
 
7
  metrics:
8
  - accuracy
9
  - f1
 
 
 
 
 
10
  model-index:
11
  - name: ModernBERT-base-subjectivity-english
12
  results: []
13
- language:
14
- - en
15
  ---
16
 
17
- <!-- This model card has been generated automatically according to the information the Trainer had access to. You
18
- should probably proofread and complete it, then remove this comment. -->
19
-
20
  # ModernBERT-base-subjectivity-english
21
 
22
- This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the [CheckThat! Lab Task 1 Subjectivity Detection at CLEF 2025](arxiv.org/abs/2507.11764).
 
 
 
 
 
23
  It achieves the following results on the evaluation set:
24
  - Loss: 1.0478
25
  - Macro F1: 0.7034
@@ -32,15 +39,38 @@ It achieves the following results on the evaluation set:
32
 
33
  ## Model description
34
 
35
- More information needed
36
 
37
  ## Intended uses & limitations
38
 
39
- More information needed
40
 
41
  ## Training and evaluation data
42
 
43
- More information needed
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
44
 
45
  ## Training procedure
46
 
@@ -72,4 +102,25 @@ The following hyperparameters were used during training:
72
  - Transformers 4.49.0
73
  - Pytorch 2.5.1+cu121
74
  - Datasets 3.3.1
75
- - Tokenizers 0.21.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
2
  base_model: answerdotai/ModernBERT-base
3
+ language:
4
+ - en
5
+ library_name: transformers
6
+ license: cc-by-4.0
7
  metrics:
8
  - accuracy
9
  - f1
10
+ pipeline_tag: text-classification
11
+ tags:
12
+ - generated_from_trainer
13
+ - modernbert
14
+ - subjectivity-detection
15
  model-index:
16
  - name: ModernBERT-base-subjectivity-english
17
  results: []
18
+ datasets:
19
+ - MatteoFasulo/clef2025_checkthat_task1_subjectivity
20
  ---
21
 
 
 
 
22
  # ModernBERT-base-subjectivity-english
23
 
24
+ This model is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) on the [CheckThat! Lab Task 1 Subjectivity Detection at CLEF 2025](https://arxiv.org/abs/2507.11764).
25
+
26
+ The model was presented in the paper [AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles](https://huggingface.co/papers/2507.11764).
27
+
28
+ The official code repository can be found at: [https://github.com/MatteoFasulo/clef2025-checkthat](https://github.com/MatteoFasulo/clef2025-checkthat)
29
+
30
  It achieves the following results on the evaluation set:
31
  - Loss: 1.0478
32
  - Macro F1: 0.7034
 
39
 
40
  ## Model description
41
 
42
+ This model, `ModernBERT-base-subjectivity-english`, is a fine-tuned version of [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) designed for subjectivity detection in news articles. It was developed as part of AI Wizards' participation in the CLEF 2025 CheckThat! Lab Task 1, aiming to classify sentences as subjective or objective. The core innovation of this model lies in enhancing transformer-based embeddings by integrating sentiment scores, derived from an auxiliary model, with sentence representations. This approach has shown to significantly boost performance, particularly the subjective F1 score, and aims to improve upon standard fine-tuning methods. To address prevalent class imbalance across languages, the model also employs decision threshold calibration optimized on the development set.
43
 
44
  ## Intended uses & limitations
45
 
46
+ This model is intended for classifying sentences in news articles as subjective (opinion-laden) or objective. This capability is crucial for applications such as combating misinformation, improving fact-checking pipelines, and supporting journalistic efforts. While this specific model is tailored for English, the broader research explored its effectiveness across monolingual (Arabic, German, Italian, Bulgarian) and zero-shot transfer settings (Greek, Polish, Romanian, Ukrainian). A key strength is its use of decision threshold calibration to mitigate class imbalance. However, users should note that the original submission had an issue with skewed class distribution which was later corrected, indicating the importance of proper data splits and calibration for optimal performance.
47
 
48
  ## Training and evaluation data
49
 
50
+ The `ModernBERT-base-subjectivity-english` model was fine-tuned on the English portion of the CheckThat! Lab Task 1: Subjectivity Detection in News Articles dataset provided for CLEF 2025. The training and development datasets included sentences in English (among other languages like Arabic, German, Italian, and Bulgarian). For final evaluation, the broader project also assessed generalization on unseen languages like Greek, Romanian, Polish, and Ukrainian. The training strategy involved augmenting transformer embeddings with sentiment signals and employing decision threshold calibration to improve performance and handle class imbalance.
51
+
52
+ ## How to use
53
+
54
+ You can use this model directly with the `transformers` library for text classification:
55
+
56
+ ```python
57
+ from transformers import pipeline
58
+
59
+ # Load the text classification pipeline
60
+ classifier = pipeline(
61
+ "text-classification",
62
+ model="MatteoFasulo/ModernBERT-base-subjectivity-english",
63
+ tokenizer="answerdotai/ModernBERT-base",
64
+ )
65
+
66
+ text1 = "The company reported a 10% increase in profits in the last quarter."
67
+ result1 = classifier(text1)
68
+ print(f"Text: '{text1}' Classification: {result1}")
69
+
70
+ text2 = "This product is absolutely amazing and everyone should try it!"
71
+ result2 = classifier(text2)
72
+ print(f"Text: '{text2}' Classification: {result2}")
73
+ ```
74
 
75
  ## Training procedure
76
 
 
102
  - Transformers 4.49.0
103
  - Pytorch 2.5.1+cu121
104
  - Datasets 3.3.1
105
+ - Tokenizers 0.21.0
106
+
107
+ ## Code
108
+
109
+ The official code and materials for this submission are available on GitHub:
110
+ [https://github.com/MatteoFasulo/clef2025-checkthat](https://github.com/MatteoFasulo/clef2025-checkthat)
111
+
112
+ ## Citation
113
+
114
+ If you find our work helpful or inspiring, please feel free to cite it:
115
+
116
+ ```bibtex
117
+ @misc{fasulo2025aiwizardscheckthat2025,
118
+ title={AI Wizards at CheckThat! 2025: Enhancing Transformer-Based Embeddings with Sentiment for Subjectivity Detection in News Articles},
119
+ author={Matteo Fasulo and Luca Babboni and Luca Tedeschini},
120
+ year={2025},
121
+ eprint={2507.11764},
122
+ archivePrefix={arXiv},
123
+ primaryClass={cs.CL},
124
+ url={https://arxiv.org/abs/2507.11764},
125
+ }
126
+ ```