Add SetFit ABSA model
Browse files- README.md +49 -49
- config.json +1 -2
- config_sentence_transformers.json +2 -2
- config_setfit.json +4 -6
- model.safetensors +1 -1
- model_head.pkl +2 -2
README.md
CHANGED
@@ -6,19 +6,20 @@ tags:
|
|
6 |
- text-classification
|
7 |
- generated_from_setfit_trainer
|
8 |
widget:
|
9 |
-
- text:
|
10 |
-
|
11 |
-
|
12 |
-
|
13 |
-
|
14 |
-
|
15 |
-
and
|
16 |
-
|
17 |
-
|
18 |
-
|
19 |
-
|
20 |
-
|
21 |
-
|
|
|
22 |
metrics:
|
23 |
- accuracy
|
24 |
pipeline_tag: text-classification
|
@@ -26,7 +27,7 @@ library_name: setfit
|
|
26 |
inference: false
|
27 |
base_model: sentence-transformers/all-MiniLM-L6-v2
|
28 |
model-index:
|
29 |
-
- name: SetFit
|
30 |
results:
|
31 |
- task:
|
32 |
type: text-classification
|
@@ -37,13 +38,13 @@ model-index:
|
|
37 |
split: test
|
38 |
metrics:
|
39 |
- type: accuracy
|
40 |
-
value: 0.
|
41 |
name: Accuracy
|
42 |
---
|
43 |
|
44 |
-
# SetFit
|
45 |
|
46 |
-
This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. In particular, this model is in charge of
|
47 |
|
48 |
The model has been trained using an efficient few-shot learning technique that involves:
|
49 |
|
@@ -53,8 +54,8 @@ The model has been trained using an efficient few-shot learning technique that i
|
|
53 |
This model was trained within the context of a larger system for ABSA, which looks like so:
|
54 |
|
55 |
1. Use a spaCy model to select possible aspect span candidates.
|
56 |
-
2. Use
|
57 |
-
3.
|
58 |
|
59 |
## Model Details
|
60 |
|
@@ -63,10 +64,10 @@ This model was trained within the context of a larger system for ABSA, which loo
|
|
63 |
- **Sentence Transformer body:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
|
64 |
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
|
65 |
- **spaCy Model:** en_core_web_lg
|
66 |
-
- **SetFitABSA Aspect Model:** [setfit-absa-aspect](https://huggingface.co/setfit-absa-aspect)
|
67 |
-
- **SetFitABSA Polarity Model:** [ronalhung/setfit-absa-restaurants-
|
68 |
- **Maximum Sequence Length:** 256 tokens
|
69 |
-
- **Number of Classes:**
|
70 |
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
|
71 |
<!-- - **Language:** Unknown -->
|
72 |
<!-- - **License:** Unknown -->
|
@@ -78,19 +79,17 @@ This model was trained within the context of a larger system for ABSA, which loo
|
|
78 |
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
|
79 |
|
80 |
### Model Labels
|
81 |
-
| Label
|
82 |
-
|
83 |
-
|
|
84 |
-
|
|
85 |
-
| neutral | <ul><li>"'s on the menu or not.:The food is uniformly exceptional, with a very capable kitchen which will proudly whip up whatever you feel like eating, whether it's on the menu or not."</li><li>'to sample both meats).:Our agreed favorite is the orrechiete with sausage and chicken (usually the waiters are kind enough to split the dish in half so you get to sample both meats).'</li><li>'to split the dish in half so:Our agreed favorite is the orrechiete with sausage and chicken (usually the waiters are kind enough to split the dish in half so you get to sample both meats).'</li></ul> |
|
86 |
-
| conflict | <ul><li>'The food was delicious but:The food was delicious but do not come here on a empty stomach.'</li><li>"The service varys from day:The service varys from day to day- sometimes they're very nice, and sometimes not."</li></ul> |
|
87 |
|
88 |
## Evaluation
|
89 |
|
90 |
### Metrics
|
91 |
| Label | Accuracy |
|
92 |
|:--------|:---------|
|
93 |
-
| **all** | 0.
|
94 |
|
95 |
## Uses
|
96 |
|
@@ -109,8 +108,8 @@ from setfit import AbsaModel
|
|
109 |
|
110 |
# Download from the 🤗 Hub
|
111 |
model = AbsaModel.from_pretrained(
|
112 |
-
"setfit-absa-aspect",
|
113 |
"ronalhung/setfit-absa-restaurants-aspect",
|
|
|
114 |
)
|
115 |
# Run inference
|
116 |
preds = model("The food was great, but the venue is just way too busy.")
|
@@ -145,14 +144,12 @@ preds = model("The food was great, but the venue is just way too busy.")
|
|
145 |
### Training Set Metrics
|
146 |
| Training set | Min | Median | Max |
|
147 |
|:-------------|:----|:--------|:----|
|
148 |
-
| Word count |
|
149 |
|
150 |
-
| Label
|
151 |
-
|
152 |
-
|
|
153 |
-
|
|
154 |
-
| neutral | 25 |
|
155 |
-
| positive | 82 |
|
156 |
|
157 |
### Training Hyperparameters
|
158 |
- batch_size: (128, 128)
|
@@ -175,23 +172,26 @@ preds = model("The food was great, but the venue is just way too busy.")
|
|
175 |
### Training Results
|
176 |
| Epoch | Step | Training Loss | Validation Loss |
|
177 |
|:------:|:----:|:-------------:|:---------------:|
|
178 |
-
| 0.
|
179 |
-
| 0.
|
180 |
-
|
|
181 |
-
|
|
182 |
-
|
|
183 |
-
|
|
184 |
-
|
|
|
|
|
|
|
|
185 |
|
186 |
### Framework Versions
|
187 |
-
- Python: 3.11.
|
188 |
- SetFit: 1.1.2
|
189 |
- Sentence Transformers: 3.4.1
|
190 |
-
- spaCy: 3.
|
191 |
-
- Transformers: 4.
|
192 |
-
- PyTorch: 2.
|
193 |
- Datasets: 3.5.0
|
194 |
-
- Tokenizers: 0.21.
|
195 |
|
196 |
## Citation
|
197 |
|
|
|
6 |
- text-classification
|
7 |
- generated_from_setfit_trainer
|
8 |
widget:
|
9 |
+
- text: food:What is even better, is that the prices are very affordable as well,
|
10 |
+
and the food is really good.
|
11 |
+
- text: 'cuisine:An oasis of refinement: Food, though somewhat uneven, often reaches
|
12 |
+
the pinnacles of new American fine cuisine - chef''s passion (and kitchen''s precise
|
13 |
+
execution) is most evident in the fish dishes and soups.'
|
14 |
+
- text: check:Had we been stalling I could understand where they were coming from,
|
15 |
+
but we had been there less than an hour and they hadn't even brought us a check
|
16 |
+
yet!
|
17 |
+
- text: back waiters:I understand the area and folks you need not come here for the
|
18 |
+
romantic, alluring ambiance or the five star service featuring a sommlier and
|
19 |
+
a complicated maze of captain and back waiters - you come for the authentic foods,
|
20 |
+
the tastes, the experiance.
|
21 |
+
- text: wine selection:The wine selection (by the glass and bottle) is wonderful and
|
22 |
+
I always recommend that friends make a reservation if they're going to be in town.
|
23 |
metrics:
|
24 |
- accuracy
|
25 |
pipeline_tag: text-classification
|
|
|
27 |
inference: false
|
28 |
base_model: sentence-transformers/all-MiniLM-L6-v2
|
29 |
model-index:
|
30 |
+
- name: SetFit Aspect Model with sentence-transformers/all-MiniLM-L6-v2
|
31 |
results:
|
32 |
- task:
|
33 |
type: text-classification
|
|
|
38 |
split: test
|
39 |
metrics:
|
40 |
- type: accuracy
|
41 |
+
value: 0.8539823008849557
|
42 |
name: Accuracy
|
43 |
---
|
44 |
|
45 |
+
# SetFit Aspect Model with sentence-transformers/all-MiniLM-L6-v2
|
46 |
|
47 |
+
This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. In particular, this model is in charge of filtering aspect span candidates.
|
48 |
|
49 |
The model has been trained using an efficient few-shot learning technique that involves:
|
50 |
|
|
|
54 |
This model was trained within the context of a larger system for ABSA, which looks like so:
|
55 |
|
56 |
1. Use a spaCy model to select possible aspect span candidates.
|
57 |
+
2. **Use this SetFit model to filter these possible aspect span candidates.**
|
58 |
+
3. Use a SetFit model to classify the filtered aspect span candidates.
|
59 |
|
60 |
## Model Details
|
61 |
|
|
|
64 |
- **Sentence Transformer body:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
|
65 |
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
|
66 |
- **spaCy Model:** en_core_web_lg
|
67 |
+
- **SetFitABSA Aspect Model:** [ronalhung/setfit-absa-restaurants-aspect](https://huggingface.co/ronalhung/setfit-absa-restaurants-aspect)
|
68 |
+
- **SetFitABSA Polarity Model:** [ronalhung/setfit-absa-restaurants-polarity](https://huggingface.co/ronalhung/setfit-absa-restaurants-polarity)
|
69 |
- **Maximum Sequence Length:** 256 tokens
|
70 |
+
- **Number of Classes:** 2 classes
|
71 |
<!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
|
72 |
<!-- - **Language:** Unknown -->
|
73 |
<!-- - **License:** Unknown -->
|
|
|
79 |
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
|
80 |
|
81 |
### Model Labels
|
82 |
+
| Label | Examples |
|
83 |
+
|:----------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
|
84 |
+
| aspect | <ul><li>'staff:But the staff was so horrible to us.'</li><li>"food:To be completely fair, the only redeeming factor was the food, which was above average, but couldn't make up for all the other deficiencies of Teodora."</li><li>"food:The food is uniformly exceptional, with a very capable kitchen which will proudly whip up whatever you feel like eating, whether it's on the menu or not."</li></ul> |
|
85 |
+
| no aspect | <ul><li>"factor:To be completely fair, the only redeeming factor was the food, which was above average, but couldn't make up for all the other deficiencies of Teodora."</li><li>"deficiencies:To be completely fair, the only redeeming factor was the food, which was above average, but couldn't make up for all the other deficiencies of Teodora."</li><li>"Teodora:To be completely fair, the only redeeming factor was the food, which was above average, but couldn't make up for all the other deficiencies of Teodora."</li></ul> |
|
|
|
|
|
86 |
|
87 |
## Evaluation
|
88 |
|
89 |
### Metrics
|
90 |
| Label | Accuracy |
|
91 |
|:--------|:---------|
|
92 |
+
| **all** | 0.8540 |
|
93 |
|
94 |
## Uses
|
95 |
|
|
|
108 |
|
109 |
# Download from the 🤗 Hub
|
110 |
model = AbsaModel.from_pretrained(
|
|
|
111 |
"ronalhung/setfit-absa-restaurants-aspect",
|
112 |
+
"ronalhung/setfit-absa-restaurants-polarity",
|
113 |
)
|
114 |
# Run inference
|
115 |
preds = model("The food was great, but the venue is just way too busy.")
|
|
|
144 |
### Training Set Metrics
|
145 |
| Training set | Min | Median | Max |
|
146 |
|:-------------|:----|:--------|:----|
|
147 |
+
| Word count | 4 | 17.9296 | 37 |
|
148 |
|
149 |
+
| Label | Training Sample Count |
|
150 |
+
|:----------|:----------------------|
|
151 |
+
| no aspect | 71 |
|
152 |
+
| aspect | 128 |
|
|
|
|
|
153 |
|
154 |
### Training Hyperparameters
|
155 |
- batch_size: (128, 128)
|
|
|
172 |
### Training Results
|
173 |
| Epoch | Step | Training Loss | Validation Loss |
|
174 |
|:------:|:----:|:-------------:|:---------------:|
|
175 |
+
| 0.0059 | 1 | 0.311 | - |
|
176 |
+
| 0.2959 | 50 | 0.2716 | 0.2559 |
|
177 |
+
| 0.5917 | 100 | 0.1808 | 0.2302 |
|
178 |
+
| 0.8876 | 150 | 0.0201 | 0.2321 |
|
179 |
+
| 1.1834 | 200 | 0.0027 | 0.2249 |
|
180 |
+
| 1.4793 | 250 | 0.0017 | 0.2325 |
|
181 |
+
| 1.7751 | 300 | 0.0008 | 0.2361 |
|
182 |
+
| 2.0710 | 350 | 0.0007 | 0.2496 |
|
183 |
+
| 2.3669 | 400 | 0.0006 | 0.2306 |
|
184 |
+
| 2.6627 | 450 | 0.0005 | 0.2370 |
|
185 |
|
186 |
### Framework Versions
|
187 |
+
- Python: 3.11.12
|
188 |
- SetFit: 1.1.2
|
189 |
- Sentence Transformers: 3.4.1
|
190 |
+
- spaCy: 3.8.5
|
191 |
+
- Transformers: 4.51.3
|
192 |
+
- PyTorch: 2.6.0+cu124
|
193 |
- Datasets: 3.5.0
|
194 |
+
- Tokenizers: 0.21.1
|
195 |
|
196 |
## Citation
|
197 |
|
config.json
CHANGED
@@ -1,5 +1,4 @@
|
|
1 |
{
|
2 |
-
"_name_or_path": "sentence-transformers/all-MiniLM-L6-v2",
|
3 |
"architectures": [
|
4 |
"BertModel"
|
5 |
],
|
@@ -19,7 +18,7 @@
|
|
19 |
"pad_token_id": 0,
|
20 |
"position_embedding_type": "absolute",
|
21 |
"torch_dtype": "float32",
|
22 |
-
"transformers_version": "4.
|
23 |
"type_vocab_size": 2,
|
24 |
"use_cache": true,
|
25 |
"vocab_size": 30522
|
|
|
1 |
{
|
|
|
2 |
"architectures": [
|
3 |
"BertModel"
|
4 |
],
|
|
|
18 |
"pad_token_id": 0,
|
19 |
"position_embedding_type": "absolute",
|
20 |
"torch_dtype": "float32",
|
21 |
+
"transformers_version": "4.51.3",
|
22 |
"type_vocab_size": 2,
|
23 |
"use_cache": true,
|
24 |
"vocab_size": 30522
|
config_sentence_transformers.json
CHANGED
@@ -1,8 +1,8 @@
|
|
1 |
{
|
2 |
"__version__": {
|
3 |
"sentence_transformers": "3.4.1",
|
4 |
-
"transformers": "4.
|
5 |
-
"pytorch": "2.
|
6 |
},
|
7 |
"prompts": {},
|
8 |
"default_prompt_name": null,
|
|
|
1 |
{
|
2 |
"__version__": {
|
3 |
"sentence_transformers": "3.4.1",
|
4 |
+
"transformers": "4.51.3",
|
5 |
+
"pytorch": "2.6.0+cu124"
|
6 |
},
|
7 |
"prompts": {},
|
8 |
"default_prompt_name": null,
|
config_setfit.json
CHANGED
@@ -1,11 +1,9 @@
|
|
1 |
{
|
|
|
2 |
"labels": [
|
3 |
-
"
|
4 |
-
"
|
5 |
-
"neutral",
|
6 |
-
"positive"
|
7 |
],
|
8 |
"spacy_model": "en_core_web_lg",
|
9 |
-
"normalize_embeddings": false
|
10 |
-
"span_context": 3
|
11 |
}
|
|
|
1 |
{
|
2 |
+
"span_context": 0,
|
3 |
"labels": [
|
4 |
+
"no aspect",
|
5 |
+
"aspect"
|
|
|
|
|
6 |
],
|
7 |
"spacy_model": "en_core_web_lg",
|
8 |
+
"normalize_embeddings": false
|
|
|
9 |
}
|
model.safetensors
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 90864192
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3b18d16aca34ce420c1788dc1b084dc9dae7d95dc3d0656b272a85c8107817d1
|
3 |
size 90864192
|
model_head.pkl
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:5fe8b5e479477a6226d5bce3f8d401bd7bdfa0c71b6e66857b1d6fe4ce24e75b
|
3 |
+
size 3919
|