Update README.md
Browse files
README.md
CHANGED
@@ -60,7 +60,7 @@ import torch
|
|
60 |
classifier = pipeline(
|
61 |
"text-classification",
|
62 |
model="cirimus/modernbert-large-go-emotions",
|
63 |
-
|
64 |
)
|
65 |
|
66 |
text = "I am so happy and excited about this opportunity!"
|
@@ -72,15 +72,15 @@ top_5 = sorted_preds[:5]
|
|
72 |
|
73 |
print("\nTop 5 emotions detected:")
|
74 |
for pred in top_5:
|
75 |
-
print(f"{pred['label']}: {pred['score']:.3f}")
|
76 |
|
77 |
# Example output:
|
78 |
# Top 5 emotions detected:
|
79 |
-
#
|
80 |
-
#
|
81 |
-
#
|
82 |
-
#
|
83 |
-
#
|
84 |
```
|
85 |
|
86 |
### How the Model Was Created
|
@@ -110,17 +110,9 @@ The model was evaluated on the test split of the GoEmotions dataset, using a thr
|
|
110 |
|
111 |
Using the default threshold of 0.5.
|
112 |
|
113 |
-
*Macro Averages (test)*
|
114 |
-
- Accuracy: `0.971`
|
115 |
-
- Precision: `0.611`
|
116 |
-
- Recall: `0.410`
|
117 |
-
- F1: `0.472`
|
118 |
-
- MCC: `0.475`
|
119 |
-
|
120 |
-
*Per-Label Results (test)*
|
121 |
-
|
122 |
| Label | Accuracy | Precision | Recall | F1 | MCC | Support | Threshold |
|
123 |
|----------------|----------|-----------|--------|-------|-------|---------|-----------|
|
|
|
124 |
| admiration | 0.946 | 0.739 | 0.653 | 0.693 | 0.666 | 504 | 0.5 |
|
125 |
| amusement | 0.982 | 0.817 | 0.814 | 0.816 | 0.807 | 264 | 0.5 |
|
126 |
| anger | 0.968 | 0.671 | 0.237 | 0.351 | 0.387 | 198 | 0.5 |
|
@@ -152,19 +144,12 @@ Using the default threshold of 0.5.
|
|
152 |
|
153 |
**Optimal Results**:
|
154 |
|
155 |
-
Using the best threshold for each label based on the training set (tuned on F1)
|
156 |
-
|
157 |
-
*Macro Averages (test)*
|
158 |
-
- Accuracy: `0.968`
|
159 |
-
- Precision: `0.591`
|
160 |
-
- Recall: `0.528`
|
161 |
-
- F1: `0.550`
|
162 |
-
- MCC: `0.536`
|
163 |
|
164 |
-
*Per-Label Results (test)*
|
165 |
|
166 |
| Label | Accuracy | Precision | Recall | F1 | MCC | Support | Threshold |
|
167 |
|----------------|----------|-----------|--------|-------|-------|---------|-----------|
|
|
|
168 |
| admiration | 0.947 | 0.722 | 0.702 | 0.712 | 0.683 | 504 | 0.40 |
|
169 |
| amusement | 0.983 | 0.812 | 0.848 | 0.830 | 0.821 | 264 | 0.45 |
|
170 |
| anger | 0.966 | 0.548 | 0.460 | 0.500 | 0.485 | 198 | 0.25 |
|
|
|
60 |
classifier = pipeline(
|
61 |
"text-classification",
|
62 |
model="cirimus/modernbert-large-go-emotions",
|
63 |
+
top_k=5
|
64 |
)
|
65 |
|
66 |
text = "I am so happy and excited about this opportunity!"
|
|
|
72 |
|
73 |
print("\nTop 5 emotions detected:")
|
74 |
for pred in top_5:
|
75 |
+
print(f"\t{pred['label']:10s} : {pred['score']:.3f}")
|
76 |
|
77 |
# Example output:
|
78 |
# Top 5 emotions detected:
|
79 |
+
# joy : 0.784
|
80 |
+
# excitement : 0.735
|
81 |
+
# admiration : 0.013
|
82 |
+
# gratitude : 0.003
|
83 |
+
# amusement : 0.003
|
84 |
```
|
85 |
|
86 |
### How the Model Was Created
|
|
|
110 |
|
111 |
Using the default threshold of 0.5.
|
112 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
113 |
| Label | Accuracy | Precision | Recall | F1 | MCC | Support | Threshold |
|
114 |
|----------------|----------|-----------|--------|-------|-------|---------|-----------|
|
115 |
+
| **macro avg** | 0.971 | 0.611 | 0.410 | 0.472 | 0.475 | 5427 | 0.5 |
|
116 |
| admiration | 0.946 | 0.739 | 0.653 | 0.693 | 0.666 | 504 | 0.5 |
|
117 |
| amusement | 0.982 | 0.817 | 0.814 | 0.816 | 0.807 | 264 | 0.5 |
|
118 |
| anger | 0.968 | 0.671 | 0.237 | 0.351 | 0.387 | 198 | 0.5 |
|
|
|
144 |
|
145 |
**Optimal Results**:
|
146 |
|
147 |
+
Using the best threshold for each label based on the training set (tuned on F1), tested on the test set:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
148 |
|
|
|
149 |
|
150 |
| Label | Accuracy | Precision | Recall | F1 | MCC | Support | Threshold |
|
151 |
|----------------|----------|-----------|--------|-------|-------|---------|-----------|
|
152 |
+
| **macro avg** | 0.968 | 0.591 | 0.528 | 0.550 | 0.536 | 5427 | various |
|
153 |
| admiration | 0.947 | 0.722 | 0.702 | 0.712 | 0.683 | 504 | 0.40 |
|
154 |
| amusement | 0.983 | 0.812 | 0.848 | 0.830 | 0.821 | 264 | 0.45 |
|
155 |
| anger | 0.966 | 0.548 | 0.460 | 0.500 | 0.485 | 198 | 0.25 |
|