File size: 9,995 Bytes
8083b5b 119c285 cf7a0b7 119c285 8083b5b 6cf1912 8083b5b 119c285 8083b5b cf7a0b7 8083b5b 9c32588 119c285 8083b5b 119c285 8083b5b cf7a0b7 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 cf7a0b7 c3ab8b2 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 c3ab8b2 8083b5b 119c285 c3ab8b2 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 c3ab8b2 cf7a0b7 119c285 cf7a0b7 119c285 c3ab8b2 119c285 c3ab8b2 cf7a0b7 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 cf7a0b7 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 8083b5b 119c285 d1b8929 119c285 cf7a0b7 119c285 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 |
---
language: en
tags:
- text-classification
- pytorch
- ModernBERT
- emotions
- multi-class-classification
- multi-label-classification
datasets:
- go_emotions
license: mit
metrics:
- accuracy
- f1
- precision
- recall
- matthews_correlation
base_model:
- answerdotai/ModernBERT-large
widget:
- text: I am thrilled to be a part of this amazing journey!
- text: I feel so disappointed with the results.
- text: This is a neutral statement about cake.
library_name: transformers
---

### Overview
This model was fine-tuned from [ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large) on the [GoEmotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) dataset for multi-label classification. It predicts emotional states in text, with a total of 28 possible labels. Each input text can have one or more associated labels, reflecting the multi-label nature of the task.
Try it out [here](https://huggingface.co/spaces/cirimus/modernbert-go-emotions).
---
### Model Details
- **Base Model**: [ModernBERT-large](https://huggingface.co/answerdotai/ModernBERT-large)
- **Fine-Tuning Dataset**: [GoEmotions](https://huggingface.co/datasets/go_emotions)
- **Number of Labels**: 28
- **Problem Type**: Multi-label classification
- **Language**: English
- **License**: [MIT](https://opensource.org/licenses/MIT)
- **Fine-Tuning Framework**: Hugging Face Transformers
---
### Example Usage
Here’s how to use the model with Hugging Face Transformers:
```python
from transformers import pipeline
import torch
# Load the model
classifier = pipeline(
"text-classification",
model="cirimus/modernbert-large-go-emotions",
top_k=5
)
text = "I am so happy and excited about this opportunity!"
predictions = classifier(text)
# Print top 5 detected emotions
sorted_preds = sorted(predictions[0], key=lambda x: x['score'], reverse=True)
top_5 = sorted_preds[:5]
print("\nTop 5 emotions detected:")
for pred in top_5:
print(f"\t{pred['label']:10s} : {pred['score']:.3f}")
# Example output:
# Top 5 emotions detected:
# joy : 0.784
# excitement : 0.735
# admiration : 0.013
# gratitude : 0.003
# amusement : 0.003
```
### How the Model Was Created
The model was fine-tuned for 3 epochs using the following hyperparameters:
- **Learning Rate**: `2e-5`
- **Batch Size**: 16
- **Weight Decay**: `0.01`
- **Optimizer**: AdamW
- **Evaluation Metrics**: Precision, Recall, F1 Score (weighted), Accuracy
---
### Dataset
The [GoEmotions](https://huggingface.co/datasets/google-research-datasets/go_emotions) dataset is a multi-label emotion classification dataset derived from Reddit comments. It contains 58,000 examples with 28 emotion labels (e.g., admiration, amusement, anger, etc.), and it is annotated for multi-label classification.
---
### Evaluation Results
The model was evaluated on the test split of the GoEmotions dataset, using a threshold of `0.5` for binarizing predictions. The overall metrics were:
**Standard Results**:
Using the default threshold of 0.5.
| Label | Accuracy | Precision | Recall | F1 | MCC | Support | Threshold |
|----------------|----------|-----------|--------|-------|-------|---------|-----------|
| **macro avg** | 0.971 | 0.611 | 0.410 | 0.472 | 0.475 | 5427 | 0.5 |
| admiration | 0.946 | 0.739 | 0.653 | 0.693 | 0.666 | 504 | 0.5 |
| amusement | 0.982 | 0.817 | 0.814 | 0.816 | 0.807 | 264 | 0.5 |
| anger | 0.968 | 0.671 | 0.237 | 0.351 | 0.387 | 198 | 0.5 |
| annoyance | 0.938 | 0.449 | 0.191 | 0.268 | 0.265 | 320 | 0.5 |
| approval | 0.940 | 0.564 | 0.302 | 0.393 | 0.384 | 351 | 0.5 |
| caring | 0.977 | 0.581 | 0.319 | 0.411 | 0.420 | 135 | 0.5 |
| confusion | 0.973 | 0.553 | 0.307 | 0.395 | 0.400 | 153 | 0.5 |
| curiosity | 0.952 | 0.551 | 0.454 | 0.498 | 0.476 | 284 | 0.5 |
| desire | 0.988 | 0.702 | 0.398 | 0.508 | 0.523 | 83 | 0.5 |
| disappointment | 0.972 | 0.500 | 0.152 | 0.234 | 0.265 | 151 | 0.5 |
| disapproval | 0.951 | 0.503 | 0.315 | 0.387 | 0.374 | 267 | 0.5 |
| disgust | 0.981 | 0.685 | 0.301 | 0.418 | 0.446 | 123 | 0.5 |
| embarrassment | 0.995 | 0.800 | 0.324 | 0.462 | 0.507 | 37 | 0.5 |
| excitement | 0.983 | 0.649 | 0.233 | 0.343 | 0.382 | 103 | 0.5 |
| fear | 0.991 | 0.738 | 0.577 | 0.647 | 0.648 | 78 | 0.5 |
| gratitude | 0.990 | 0.955 | 0.895 | 0.924 | 0.919 | 352 | 0.5 |
| grief | 0.999 | 0.000 | 0.000 | 0.000 | 0.000 | 6 | 0.5 |
| joy | 0.980 | 0.658 | 0.646 | 0.652 | 0.642 | 161 | 0.5 |
| love | 0.983 | 0.795 | 0.815 | 0.805 | 0.796 | 238 | 0.5 |
| nervousness | 0.996 | 0.556 | 0.435 | 0.488 | 0.490 | 23 | 0.5 |
| optimism | 0.973 | 0.702 | 0.392 | 0.503 | 0.513 | 186 | 0.5 |
| pride | 0.998 | 0.800 | 0.250 | 0.381 | 0.446 | 16 | 0.5 |
| realization | 0.972 | 0.405 | 0.117 | 0.182 | 0.207 | 145 | 0.5 |
| relief | 0.998 | 0.000 | 0.000 | 0.000 | 0.000 | 11 | 0.5 |
| remorse | 0.992 | 0.566 | 0.839 | 0.676 | 0.686 | 56 | 0.5 |
| sadness | 0.980 | 0.764 | 0.436 | 0.555 | 0.568 | 156 | 0.5 |
| surprise | 0.980 | 0.692 | 0.447 | 0.543 | 0.547 | 141 | 0.5 |
| neutral | 0.796 | 0.716 | 0.628 | 0.669 | 0.525 | 1787 | 0.5 |
**Optimal Results**:
Using the best threshold for each label based on the training set (tuned on F1), tested on the test set:
| Label | Accuracy | Precision | Recall | F1 | MCC | Support | Threshold |
|----------------|----------|-----------|--------|-------|-------|---------|-----------|
| **macro avg** | 0.968 | 0.591 | 0.528 | 0.550 | 0.536 | 5427 | various |
| admiration | 0.947 | 0.722 | 0.702 | 0.712 | 0.683 | 504 | 0.40 |
| amusement | 0.983 | 0.812 | 0.848 | 0.830 | 0.821 | 264 | 0.45 |
| anger | 0.966 | 0.548 | 0.460 | 0.500 | 0.485 | 198 | 0.25 |
| annoyance | 0.926 | 0.378 | 0.403 | 0.390 | 0.351 | 320 | 0.30 |
| approval | 0.928 | 0.445 | 0.470 | 0.457 | 0.419 | 351 | 0.30 |
| caring | 0.975 | 0.496 | 0.430 | 0.460 | 0.449 | 135 | 0.35 |
| confusion | 0.966 | 0.417 | 0.510 | 0.459 | 0.444 | 153 | 0.30 |
| curiosity | 0.950 | 0.522 | 0.588 | 0.553 | 0.528 | 284 | 0.40 |
| desire | 0.988 | 0.673 | 0.422 | 0.519 | 0.527 | 83 | 0.40 |
| disappointment | 0.964 | 0.338 | 0.305 | 0.321 | 0.303 | 151 | 0.30 |
| disapproval | 0.948 | 0.468 | 0.416 | 0.440 | 0.414 | 267 | 0.35 |
| disgust | 0.978 | 0.529 | 0.447 | 0.485 | 0.475 | 123 | 0.25 |
| embarrassment | 0.994 | 0.650 | 0.351 | 0.456 | 0.475 | 37 | 0.35 |
| excitement | 0.978 | 0.419 | 0.427 | 0.423 | 0.412 | 103 | 0.25 |
| fear | 0.990 | 0.662 | 0.628 | 0.645 | 0.640 | 78 | 0.40 |
| gratitude | 0.990 | 0.955 | 0.895 | 0.924 | 0.919 | 352 | 0.50 |
| grief | 0.999 | 0.750 | 0.500 | 0.600 | 0.612 | 6 | 0.35 |
| joy | 0.980 | 0.660 | 0.640 | 0.650 | 0.639 | 161 | 0.50 |
| love | 0.982 | 0.774 | 0.836 | 0.804 | 0.795 | 238 | 0.45 |
| nervousness | 0.995 | 0.435 | 0.435 | 0.435 | 0.432 | 23 | 0.45 |
| optimism | 0.972 | 0.597 | 0.565 | 0.580 | 0.566 | 186 | 0.25 |
| pride | 0.998 | 0.667 | 0.375 | 0.480 | 0.499 | 16 | 0.15 |
| realization | 0.962 | 0.273 | 0.248 | 0.260 | 0.241 | 145 | 0.25 |
| relief | 0.999 | 0.800 | 0.364 | 0.500 | 0.539 | 11 | 0.25 |
| remorse | 0.993 | 0.641 | 0.732 | 0.683 | 0.681 | 56 | 0.65 |
| sadness | 0.978 | 0.646 | 0.538 | 0.587 | 0.579 | 156 | 0.30 |
| surprise | 0.979 | 0.603 | 0.518 | 0.557 | 0.548 | 141 | 0.40 |
| neutral | 0.791 | 0.669 | 0.722 | 0.695 | 0.537 | 1787 | 0.40 |
---
### Intended Use
The model is designed for emotion classification in English-language text, particularly in domains such as:
- Social media sentiment analysis
- Customer feedback evaluation
- Behavioral or psychological research
---
### Limitations and Biases
- **Data Bias**: The dataset is based on Reddit comments, which may not generalize well to other domains or cultural contexts.
- **Underrepresented Classes**: Certain labels like "grief" and "relief" have very few examples, leading to lower performance for those classes.
- **Ambiguity**: Some training data contain annotation inconsistencies or ambiguities that may impact predictions.
---
---
### Environmental Impact
- **Hardware Used**: NVIDIA RTX4090
- **Training Time**: <1 hour
- **Carbon Emissions**: ~0.06 kg CO2 (calculated via [ML CO2 Impact Calculator](https://mlco2.github.io/impact)).
---
### Citation
If you use this model, please cite it as follows:
```bibtex
@inproceedings{JdFE2025c,
title = {Emotion Classification with ModernBERT},
author = {Enric Junqu\'e de Fortuny},
year = {2025},
howpublished = {\url{https://huggingface.co/cirimus/modernbert-large-go-emotions}},
}
|