Add SetFit ABSA model

1_Pooling/config.json

@@ -1,5 +1,5 @@
 {
-  "word_embedding_dimension": 384,
+  "word_embedding_dimension": 768,
   "pooling_mode_cls_token": false,
   "pooling_mode_mean_tokens": true,
   "pooling_mode_max_tokens": false,

README.md

@@ -27,9 +27,9 @@ metrics:
 pipeline_tag: text-classification
 library_name: setfit
 inference: false
-base_model: sentence-transformers/all-MiniLM-L6-v2
+base_model: sentence-transformers/all-mpnet-base-v2
 model-index:
-- name: SetFit Polarity Model with sentence-transformers/all-MiniLM-L6-v2
+- name: SetFit Polarity Model with sentence-transformers/all-mpnet-base-v2
   results:
   - task:
       type: text-classification
@@ -40,34 +40,34 @@ model-index:
       split: test
     metrics:
     - type: accuracy
-      value: 0.6956390066626287
+      value: 0.7401574803149606
       name: Accuracy
     - type: f1_micro
-      value: 0.6956390066626287
+      value: 0.7401574803149606
       name: F1_Micro
     - type: f1_macro
-      value: 0.4329017416100218
+      value: 0.49431392855326756
       name: F1_Macro
     - type: f1_weighted
-      value: 0.6632256738950839
+      value: 0.7257118530125051
       name: F1_Weighted
     - type: precision_micro
-      value: 0.6956390066626287
+      value: 0.7401574803149606
       name: Precision_Micro
     - type: precision_macro
-      value: 0.45635478257069084
+      value: 0.493835965235845
       name: Precision_Macro
     - type: recall_micro
-      value: 0.6956390066626287
+      value: 0.7401574803149606
       name: Recall_Micro
     - type: recall_macro
-      value: 0.43398523441592457
+      value: 0.5050369958093243
       name: Recall_Macro
 ---
 
-# SetFit Polarity Model with sentence-transformers/all-MiniLM-L6-v2
+# SetFit Polarity Model with sentence-transformers/all-mpnet-base-v2
 
-This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. In particular, this model is in charge of classifying aspect polarities.
+This is a [SetFit](https://github.com/huggingface/setfit) model that can be used for Aspect Based Sentiment Analysis (ABSA). This SetFit model uses [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification. In particular, this model is in charge of classifying aspect polarities.
 
 The model has been trained using an efficient few-shot learning technique that involves:
 
@@ -84,12 +84,12 @@ This model was trained within the context of a larger system for ABSA, which loo
 
 ### Model Description
 - **Model Type:** SetFit
-- **Sentence Transformer body:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)
+- **Sentence Transformer body:** [sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
 - **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
-- **spaCy Model:** en_core_web_lg
+- **spaCy Model:** en_core_web_trf
 - **SetFitABSA Aspect Model:** [ronalhung/setfit-absa-restaurants-aspect](https://huggingface.co/ronalhung/setfit-absa-restaurants-aspect)
 - **SetFitABSA Polarity Model:** [ronalhung/setfit-absa-restaurants-polarity](https://huggingface.co/ronalhung/setfit-absa-restaurants-polarity)
-- **Maximum Sequence Length:** 256 tokens
+- **Maximum Sequence Length:** 384 tokens
 - **Number of Classes:** 4 classes
 <!-- - **Training Dataset:** [Unknown](https://huggingface.co/datasets/unknown) -->
 <!-- - **Language:** Unknown -->
@@ -114,7 +114,7 @@ This model was trained within the context of a larger system for ABSA, which loo
 ### Metrics
 | Label   | Accuracy | F1_Micro | F1_Macro | F1_Weighted | Precision_Micro | Precision_Macro | Recall_Micro | Recall_Macro |
 |:--------|:---------|:---------|:---------|:------------|:----------------|:----------------|:-------------|:-------------|
-| **all** | 0.6956   | 0.6956   | 0.4329   | 0.6632      | 0.6956          | 0.4564          | 0.6956       | 0.4340       |
+| **all** | 0.7402   | 0.7402   | 0.4943   | 0.7257      | 0.7402          | 0.4938          | 0.7402       | 0.5050       |
 
 ## Uses
 
@@ -179,7 +179,7 @@ preds = model("The food was great, but the venue is just way too busy.")
 | positive | 169                   |
 
 ### Training Hyperparameters
-- batch_size: (128, 128)
+- batch_size: (64, 64)
 - num_epochs: (5, 5)
 - max_steps: -1
 - sampling_strategy: oversampling
@@ -199,14 +199,16 @@ preds = model("The food was great, but the venue is just way too busy.")
 ### Training Results
 | Epoch  | Step | Training Loss | Validation Loss |
 |:------:|:----:|:-------------:|:---------------:|
-| 0.0039 | 1    | 0.3102        | -               |
-| 0.1946 | 50   | 0.272         | 0.2236          |
-| 0.3891 | 100  | 0.1589        | 0.2166          |
-| 0.5837 | 150  | 0.0554        | 0.2199          |
-| 0.7782 | 200  | 0.0267        | 0.2241          |
-| 0.9728 | 250  | 0.0191        | 0.2276          |
-| 1.1673 | 300  | 0.0095        | 0.2284          |
-| 1.3619 | 350  | 0.0045        | 0.2320          |
+| 0.0019 | 1    | 0.2601        | -               |
+| 0.0975 | 50   | 0.2588        | 0.2241          |
+| 0.1949 | 100  | 0.1555        | 0.1764          |
+| 0.2924 | 150  | 0.0436        | 0.1602          |
+| 0.3899 | 200  | 0.0213        | 0.1544          |
+| 0.4873 | 250  | 0.0097        | 0.1627          |
+| 0.5848 | 300  | 0.0037        | 0.1749          |
+| 0.6823 | 350  | 0.0012        | 0.2009          |
+| 0.7797 | 400  | 0.0006        | 0.1869          |
+| 0.8772 | 450  | 0.0011        | 0.1804          |
 
 ### Framework Versions
 - Python: 3.11.13

config.json

@@ -1,25 +1,23 @@
 {
   "architectures": [
-    "BertModel"
+    "MPNetModel"
   ],
   "attention_probs_dropout_prob": 0.1,
-  "classifier_dropout": null,
-  "gradient_checkpointing": false,
+  "bos_token_id": 0,
+  "eos_token_id": 2,
   "hidden_act": "gelu",
   "hidden_dropout_prob": 0.1,
-  "hidden_size": 384,
+  "hidden_size": 768,
   "initializer_range": 0.02,
-  "intermediate_size": 1536,
-  "layer_norm_eps": 1e-12,
-  "max_position_embeddings": 512,
-  "model_type": "bert",
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "mpnet",
   "num_attention_heads": 12,
-  "num_hidden_layers": 6,
-  "pad_token_id": 0,
-  "position_embedding_type": "absolute",
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "relative_attention_num_buckets": 32,
   "torch_dtype": "float32",
   "transformers_version": "4.52.4",
-  "type_vocab_size": 2,
-  "use_cache": true,
-  "vocab_size": 30522
+  "vocab_size": 30527
 }

config_setfit.json

@@ -1,11 +1,11 @@
 {
+  "spacy_model": "en_core_web_trf",
+  "span_context": 3,
   "normalize_embeddings": false,
   "labels": [
     "conflict",
     "negative",
     "neutral",
     "positive"
-  ],
-  "spacy_model": "en_core_web_lg",
-  "span_context": 3
+  ]
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3e0b83e777578291820c332350637200bef46c737f18c4d369adb775ac1c5a66
-size 90864192
+oid sha256:751c6b9fbfc279267a4948042ba89985aa7d2272ce712b6f5dad079d81916555
+size 437967672

model_head.pkl

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e1094f805f1b8676e0c432024f7143e780d71c3f72e2a10280f4817ec9ed7b10
-size 13287
+oid sha256:cb2173c537259d51f82ef9a3f5f66e8964a7f24581f78380f1ce73232e37b25b
+size 25575

sentence_bert_config.json

@@ -1,4 +1,4 @@
 {
-  "max_seq_length": 256,
+  "max_seq_length": 384,
   "do_lower_case": false
 }

special_tokens_map.json

@@ -1,27 +1,41 @@
 {
+  "bos_token": {
+    "content": "<s>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
   "cls_token": {
-    "content": "[CLS]",
+    "content": "<s>",
     "lstrip": false,
     "normalized": false,
     "rstrip": false,
     "single_word": false
   },
-  "mask_token": {
-    "content": "[MASK]",
+  "eos_token": {
+    "content": "</s>",
     "lstrip": false,
     "normalized": false,
     "rstrip": false,
     "single_word": false
   },
+  "mask_token": {
+    "content": "<mask>",
+    "lstrip": true,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
   "pad_token": {
-    "content": "[PAD]",
+    "content": "<pad>",
     "lstrip": false,
     "normalized": false,
     "rstrip": false,
     "single_word": false
   },
   "sep_token": {
-    "content": "[SEP]",
+    "content": "</s>",
     "lstrip": false,
     "normalized": false,
     "rstrip": false,

tokenizer_config.json

@@ -1,64 +1,72 @@
 {
   "added_tokens_decoder": {
     "0": {
-      "content": "[PAD]",
+      "content": "<s>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "single_word": false,
       "special": true
     },
-    "100": {
-      "content": "[UNK]",
+    "1": {
+      "content": "<pad>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "single_word": false,
       "special": true
     },
-    "101": {
-      "content": "[CLS]",
+    "2": {
+      "content": "</s>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "single_word": false,
       "special": true
     },
-    "102": {
-      "content": "[SEP]",
+    "3": {
+      "content": "<unk>",
       "lstrip": false,
-      "normalized": false,
+      "normalized": true,
       "rstrip": false,
       "single_word": false,
       "special": true
     },
-    "103": {
-      "content": "[MASK]",
+    "104": {
+      "content": "[UNK]",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,
       "single_word": false,
       "special": true
+    },
+    "30526": {
+      "content": "<mask>",
+      "lstrip": true,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
     }
   },
+  "bos_token": "<s>",
   "clean_up_tokenization_spaces": false,
-  "cls_token": "[CLS]",
-  "do_basic_tokenize": true,
+  "cls_token": "<s>",
   "do_lower_case": true,
+  "eos_token": "</s>",
   "extra_special_tokens": {},
-  "mask_token": "[MASK]",
+  "mask_token": "<mask>",
   "max_length": 128,
-  "model_max_length": 256,
-  "never_split": null,
+  "model_max_length": 384,
   "pad_to_multiple_of": null,
-  "pad_token": "[PAD]",
+  "pad_token": "<pad>",
   "pad_token_type_id": 0,
   "padding_side": "right",
-  "sep_token": "[SEP]",
+  "sep_token": "</s>",
   "stride": 0,
   "strip_accents": null,
   "tokenize_chinese_chars": true,
-  "tokenizer_class": "BertTokenizer",
+  "tokenizer_class": "MPNetTokenizer",
   "truncation_side": "right",
   "truncation_strategy": "longest_first",
   "unk_token": "[UNK]"

vocab.txt

@@ -1,3 +1,7 @@
+<s>
+<pad>
+</s>
+<unk>
 [PAD]
 [unused0]
 [unused1]
@@ -30520,3 +30524,4 @@ necessitated
 ##：
 ##？
 ##～
+<mask>