metadata
base_model: sentence-transformers/paraphrase-MiniLM-L3-v2
library_name: setfit
metrics:
- accuracy
pipeline_tag: text-classification
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
widget:
- text: >-
metrics.statistics.polysyllables: 4603, 102, 339, 2604, 397, 1180, 555,
1488, 1226, 378, 6639, 1978, 4088, 7005, 3256, 86, 2338, 1905, 1647, 16369
- text: >-
company.relationship: founder, None, founder/chairman, Relation, relation,
CEO, chairman, investor, founder and CEO, founder/CEO, owner, chairman of
management committee, founder and chairman, Chairman and Chief Executive
Officer, general director, executive chairman, Chairman/founder, founder,
chairman, ceo, former chairman and CEO, relation and chairman
- text: 'variety: Western, Eastern'
- text: >-
Data.Fat.Saturated Fat: 2.009, 1.164, 1.86, 2.154, 0.568, 0.117, 1.11,
0.049, 0.66, 1.242, 1.899, 0.596, 2.667, 0.044, 2.554, 0.633, 4.591,
1.214, 0.121, 5.486
- text: >-
Date.Full: 8/26/1990, 3/24/1991, 3/31/1991, 4/7/1991, 4/14/1991,
4/21/1991, 4/28/1991, 5/5/1991, 5/12/1991, 5/19/1991, 5/26/1991, 6/2/1991,
6/9/1991, 6/16/1991, 6/23/1991, 6/30/1991, 7/7/1991, 7/14/1991, 7/21/1991,
7/28/1991
inference: true
model-index:
- name: SetFit with sentence-transformers/paraphrase-MiniLM-L3-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: Unknown
type: unknown
split: test
metrics:
- type: accuracy
value: 0.7512388503468781
name: Accuracy
SetFit with sentence-transformers/paraphrase-MiniLM-L3-v2
This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-MiniLM-L3-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
- Fine-tuning a Sentence Transformer with contrastive learning.
- Training a classification head with features from the fine-tuned Sentence Transformer.
Model Details
Model Description
- Model Type: SetFit
- Sentence Transformer body: sentence-transformers/paraphrase-MiniLM-L3-v2
- Classification head: a LogisticRegression instance
- Maximum Sequence Length: 128 tokens
- Number of Classes: 39 classes
Model Sources
- Repository: SetFit on GitHub
- Paper: Efficient Few-Shot Learning Without Prompts
- Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts
Model Labels
Label | Examples |
---|---|
Month Number |
|
Date |
|
Categorical |
|
Year |
|
Longitude |
|
Floating Point Number |
|
Slug |
|
U.S. State Abbreviation |
|
Month Name |
|
Day of Month |
|
Currency Code |
|
Last Name |
|
Timestamp |
|
Day of Week |
|
Integer |
|
Street Address |
|
URL |
|
U.S. State |
|
Zip Code |
|
Country Name |
|
Boolean |
|
Short text |
|
Occupation |
|
Partial timestamp |
|
Street Name |
|
Full Name |
|
Very short text |
|
URI |
|
Latitude |
|
Time |
|
Postal Code |
|
Country ISO Code |
|
First Name |
|
City Name |
|
Color |
|
License Plate |
|
AM/PM |
|
Company Name |
|
Secondary Address |
|
Evaluation
Metrics
Label | Accuracy |
---|---|
all | 0.7512 |
Uses
Direct Use for Inference
First install the SetFit library:
pip install setfit
Then you can load this model and run inference.
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("quantisan/paraphrase-MiniLM-L3-v2-93dataset")
# Run inference
preds = model("variety: Western, Eastern")
Training Details
Training Set Metrics
Training set | Min | Median | Max |
---|---|---|---|
Word count | 2 | 22.3314 | 85 |
Label | Training Sample Count |
---|---|
Categorical | 8 |
Timestamp | 5 |
Date | 8 |
Integer | 8 |
Partial timestamp | 4 |
Short text | 8 |
Very short text | 3 |
AM/PM | 1 |
Boolean | 8 |
City Name | 1 |
Color | 3 |
Company Name | 1 |
Country ISO Code | 2 |
Country Name | 8 |
Currency Code | 1 |
Day of Month | 4 |
Day of Week | 4 |
First Name | 1 |
Floating Point Number | 8 |
Full Name | 8 |
Last Name | 2 |
Latitude | 4 |
License Plate | 1 |
Longitude | 4 |
Month Name | 6 |
Month Number | 4 |
Occupation | 3 |
Postal Code | 1 |
Secondary Address | 1 |
Slug | 8 |
Street Address | 3 |
Street Name | 3 |
Time | 3 |
U.S. State | 8 |
U.S. State Abbreviation | 6 |
URI | 1 |
URL | 8 |
Year | 8 |
Zip Code | 4 |
Training Hyperparameters
- batch_size: (8, 8)
- num_epochs: (4, 4)
- max_steps: -1
- sampling_strategy: oversampling
- body_learning_rate: (2e-05, 1e-05)
- head_learning_rate: 0.01
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- l2_weight: 0.01
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: True
Training Results
Epoch | Step | Training Loss | Validation Loss |
---|---|---|---|
0.0003 | 1 | 0.3882 | - |
0.0140 | 50 | 0.1864 | - |
0.0280 | 100 | 0.1588 | - |
0.0421 | 150 | 0.15 | - |
0.0561 | 200 | 0.1537 | - |
0.0701 | 250 | 0.1325 | - |
0.0841 | 300 | 0.132 | - |
0.0981 | 350 | 0.1149 | - |
0.1121 | 400 | 0.1198 | - |
0.1262 | 450 | 0.1035 | - |
0.1402 | 500 | 0.0907 | - |
0.1542 | 550 | 0.0917 | - |
0.1682 | 600 | 0.0875 | - |
0.1822 | 650 | 0.0803 | - |
0.1962 | 700 | 0.0669 | - |
0.2103 | 750 | 0.0671 | - |
0.2243 | 800 | 0.0614 | - |
0.2383 | 850 | 0.0642 | - |
0.2523 | 900 | 0.0481 | - |
0.2663 | 950 | 0.0548 | - |
0.2803 | 1000 | 0.0346 | - |
0.2944 | 1050 | 0.0406 | - |
0.3084 | 1100 | 0.0403 | - |
0.3224 | 1150 | 0.0349 | - |
0.3364 | 1200 | 0.0312 | - |
0.3504 | 1250 | 0.0378 | - |
0.3645 | 1300 | 0.0335 | - |
0.3785 | 1350 | 0.0323 | - |
0.3925 | 1400 | 0.0234 | - |
0.4065 | 1450 | 0.0313 | - |
0.4205 | 1500 | 0.022 | - |
0.4345 | 1550 | 0.0326 | - |
0.4486 | 1600 | 0.0233 | - |
0.4626 | 1650 | 0.0195 | - |
0.4766 | 1700 | 0.0254 | - |
0.4906 | 1750 | 0.0211 | - |
0.5046 | 1800 | 0.0198 | - |
0.5186 | 1850 | 0.0201 | - |
0.5327 | 1900 | 0.0216 | - |
0.5467 | 1950 | 0.0174 | - |
0.5607 | 2000 | 0.0176 | - |
0.5747 | 2050 | 0.0234 | - |
0.5887 | 2100 | 0.0172 | - |
0.6027 | 2150 | 0.0129 | - |
0.6168 | 2200 | 0.0151 | - |
0.6308 | 2250 | 0.015 | - |
0.6448 | 2300 | 0.0164 | - |
0.6588 | 2350 | 0.0137 | - |
0.6728 | 2400 | 0.014 | - |
0.6869 | 2450 | 0.0154 | - |
0.7009 | 2500 | 0.0135 | - |
0.7149 | 2550 | 0.0164 | - |
0.7289 | 2600 | 0.0139 | - |
0.7429 | 2650 | 0.0164 | - |
0.7569 | 2700 | 0.0106 | - |
0.7710 | 2750 | 0.0084 | - |
0.7850 | 2800 | 0.0133 | - |
0.7990 | 2850 | 0.0114 | - |
0.8130 | 2900 | 0.0066 | - |
0.8270 | 2950 | 0.0091 | - |
0.8410 | 3000 | 0.0126 | - |
0.8551 | 3050 | 0.0107 | - |
0.8691 | 3100 | 0.0068 | - |
0.8831 | 3150 | 0.006 | - |
0.8971 | 3200 | 0.007 | - |
0.9111 | 3250 | 0.0155 | - |
0.9251 | 3300 | 0.0111 | - |
0.9392 | 3350 | 0.0049 | - |
0.9532 | 3400 | 0.0076 | - |
0.9672 | 3450 | 0.0092 | - |
0.9812 | 3500 | 0.0086 | - |
0.9952 | 3550 | 0.0061 | - |
1.0 | 3567 | - | 0.1341 |
1.0093 | 3600 | 0.0073 | - |
1.0233 | 3650 | 0.0065 | - |
1.0373 | 3700 | 0.0063 | - |
1.0513 | 3750 | 0.0094 | - |
1.0653 | 3800 | 0.0114 | - |
1.0793 | 3850 | 0.0084 | - |
1.0934 | 3900 | 0.0098 | - |
1.1074 | 3950 | 0.0058 | - |
1.1214 | 4000 | 0.0045 | - |
1.1354 | 4050 | 0.018 | - |
1.1494 | 4100 | 0.0077 | - |
1.1634 | 4150 | 0.0067 | - |
1.1775 | 4200 | 0.0061 | - |
1.1915 | 4250 | 0.0037 | - |
1.2055 | 4300 | 0.0045 | - |
1.2195 | 4350 | 0.0033 | - |
1.2335 | 4400 | 0.0067 | - |
1.2475 | 4450 | 0.0054 | - |
1.2616 | 4500 | 0.0057 | - |
1.2756 | 4550 | 0.004 | - |
1.2896 | 4600 | 0.0033 | - |
1.3036 | 4650 | 0.0076 | - |
1.3176 | 4700 | 0.0045 | - |
1.3317 | 4750 | 0.0068 | - |
1.3457 | 4800 | 0.0043 | - |
1.3597 | 4850 | 0.0049 | - |
1.3737 | 4900 | 0.0045 | - |
1.3877 | 4950 | 0.0055 | - |
1.4017 | 5000 | 0.0065 | - |
1.4158 | 5050 | 0.0029 | - |
1.4298 | 5100 | 0.0041 | - |
1.4438 | 5150 | 0.0064 | - |
1.4578 | 5200 | 0.0031 | - |
1.4718 | 5250 | 0.0078 | - |
1.4858 | 5300 | 0.0031 | - |
1.4999 | 5350 | 0.004 | - |
1.5139 | 5400 | 0.0035 | - |
1.5279 | 5450 | 0.0062 | - |
1.5419 | 5500 | 0.0062 | - |
1.5559 | 5550 | 0.0065 | - |
1.5699 | 5600 | 0.0036 | - |
1.5840 | 5650 | 0.0037 | - |
1.5980 | 5700 | 0.0047 | - |
1.6120 | 5750 | 0.0037 | - |
1.6260 | 5800 | 0.0028 | - |
1.6400 | 5850 | 0.0052 | - |
1.6541 | 5900 | 0.0043 | - |
1.6681 | 5950 | 0.0029 | - |
1.6821 | 6000 | 0.0064 | - |
1.6961 | 6050 | 0.0031 | - |
1.7101 | 6100 | 0.0023 | - |
1.7241 | 6150 | 0.002 | - |
1.7382 | 6200 | 0.0041 | - |
1.7522 | 6250 | 0.0033 | - |
1.7662 | 6300 | 0.0043 | - |
1.7802 | 6350 | 0.0023 | - |
1.7942 | 6400 | 0.0036 | - |
1.8082 | 6450 | 0.0024 | - |
1.8223 | 6500 | 0.0016 | - |
1.8363 | 6550 | 0.003 | - |
1.8503 | 6600 | 0.0043 | - |
1.8643 | 6650 | 0.0043 | - |
1.8783 | 6700 | 0.0017 | - |
1.8923 | 6750 | 0.0018 | - |
1.9064 | 6800 | 0.0029 | - |
1.9204 | 6850 | 0.0026 | - |
1.9344 | 6900 | 0.0039 | - |
1.9484 | 6950 | 0.0019 | - |
1.9624 | 7000 | 0.0041 | - |
1.9765 | 7050 | 0.0019 | - |
1.9905 | 7100 | 0.0023 | - |
2.0 | 7134 | - | 0.1286 |
2.0045 | 7150 | 0.0016 | - |
2.0185 | 7200 | 0.0017 | - |
2.0325 | 7250 | 0.0016 | - |
2.0465 | 7300 | 0.0019 | - |
2.0606 | 7350 | 0.0015 | - |
2.0746 | 7400 | 0.0016 | - |
2.0886 | 7450 | 0.0015 | - |
2.1026 | 7500 | 0.0015 | - |
2.1166 | 7550 | 0.0034 | - |
2.1306 | 7600 | 0.0043 | - |
2.1447 | 7650 | 0.0016 | - |
2.1587 | 7700 | 0.0016 | - |
2.1727 | 7750 | 0.0015 | - |
2.1867 | 7800 | 0.0015 | - |
2.2007 | 7850 | 0.0017 | - |
2.2147 | 7900 | 0.0013 | - |
2.2288 | 7950 | 0.0016 | - |
2.2428 | 8000 | 0.0013 | - |
2.2568 | 8050 | 0.0039 | - |
2.2708 | 8100 | 0.0053 | - |
2.2848 | 8150 | 0.0025 | - |
2.2989 | 8200 | 0.0015 | - |
2.3129 | 8250 | 0.0012 | - |
2.3269 | 8300 | 0.006 | - |
2.3409 | 8350 | 0.0014 | - |
2.3549 | 8400 | 0.0014 | - |
2.3689 | 8450 | 0.0028 | - |
2.3830 | 8500 | 0.0015 | - |
2.3970 | 8550 | 0.0019 | - |
2.4110 | 8600 | 0.0017 | - |
2.4250 | 8650 | 0.002 | - |
2.4390 | 8700 | 0.0016 | - |
2.4530 | 8750 | 0.0014 | - |
2.4671 | 8800 | 0.0021 | - |
2.4811 | 8850 | 0.0012 | - |
2.4951 | 8900 | 0.0015 | - |
2.5091 | 8950 | 0.0012 | - |
2.5231 | 9000 | 0.0012 | - |
2.5371 | 9050 | 0.0016 | - |
2.5512 | 9100 | 0.0016 | - |
2.5652 | 9150 | 0.0013 | - |
2.5792 | 9200 | 0.0028 | - |
2.5932 | 9250 | 0.0013 | - |
2.6072 | 9300 | 0.0011 | - |
2.6213 | 9350 | 0.0035 | - |
2.6353 | 9400 | 0.0013 | - |
2.6493 | 9450 | 0.0012 | - |
2.6633 | 9500 | 0.0037 | - |
2.6773 | 9550 | 0.0012 | - |
2.6913 | 9600 | 0.0011 | - |
2.7054 | 9650 | 0.0037 | - |
2.7194 | 9700 | 0.0012 | - |
2.7334 | 9750 | 0.0013 | - |
2.7474 | 9800 | 0.0013 | - |
2.7614 | 9850 | 0.001 | - |
2.7754 | 9900 | 0.0011 | - |
2.7895 | 9950 | 0.0012 | - |
2.8035 | 10000 | 0.0012 | - |
2.8175 | 10050 | 0.001 | - |
2.8315 | 10100 | 0.001 | - |
2.8455 | 10150 | 0.0011 | - |
2.8595 | 10200 | 0.0009 | - |
2.8736 | 10250 | 0.0018 | - |
2.8876 | 10300 | 0.0013 | - |
2.9016 | 10350 | 0.0009 | - |
2.9156 | 10400 | 0.0033 | - |
2.9296 | 10450 | 0.0034 | - |
2.9437 | 10500 | 0.0011 | - |
2.9577 | 10550 | 0.0013 | - |
2.9717 | 10600 | 0.0009 | - |
2.9857 | 10650 | 0.0009 | - |
2.9997 | 10700 | 0.0011 | - |
3.0 | 10701 | - | 0.1205 |
3.0137 | 10750 | 0.0009 | - |
3.0278 | 10800 | 0.0009 | - |
3.0418 | 10850 | 0.0032 | - |
3.0558 | 10900 | 0.0008 | - |
3.0698 | 10950 | 0.0013 | - |
3.0838 | 11000 | 0.0033 | - |
3.0978 | 11050 | 0.0011 | - |
3.1119 | 11100 | 0.0008 | - |
3.1259 | 11150 | 0.0009 | - |
3.1399 | 11200 | 0.0008 | - |
3.1539 | 11250 | 0.0033 | - |
3.1679 | 11300 | 0.0032 | - |
3.1819 | 11350 | 0.0008 | - |
3.1960 | 11400 | 0.0008 | - |
3.2100 | 11450 | 0.001 | - |
3.2240 | 11500 | 0.0009 | - |
3.2380 | 11550 | 0.0008 | - |
3.2520 | 11600 | 0.0008 | - |
3.2660 | 11650 | 0.0008 | - |
3.2801 | 11700 | 0.0009 | - |
3.2941 | 11750 | 0.0008 | - |
3.3081 | 11800 | 0.0007 | - |
3.3221 | 11850 | 0.0008 | - |
3.3361 | 11900 | 0.0008 | - |
3.3502 | 11950 | 0.0009 | - |
3.3642 | 12000 | 0.0008 | - |
3.3782 | 12050 | 0.0007 | - |
3.3922 | 12100 | 0.0009 | - |
3.4062 | 12150 | 0.0008 | - |
3.4202 | 12200 | 0.0008 | - |
3.4343 | 12250 | 0.0009 | - |
3.4483 | 12300 | 0.0008 | - |
3.4623 | 12350 | 0.0008 | - |
3.4763 | 12400 | 0.0008 | - |
3.4903 | 12450 | 0.0009 | - |
3.5043 | 12500 | 0.0007 | - |
3.5184 | 12550 | 0.0008 | - |
3.5324 | 12600 | 0.0009 | - |
3.5464 | 12650 | 0.0031 | - |
3.5604 | 12700 | 0.0009 | - |
3.5744 | 12750 | 0.0008 | - |
3.5884 | 12800 | 0.0007 | - |
3.6025 | 12850 | 0.0007 | - |
3.6165 | 12900 | 0.0007 | - |
3.6305 | 12950 | 0.0008 | - |
3.6445 | 13000 | 0.0007 | - |
3.6585 | 13050 | 0.0008 | - |
3.6726 | 13100 | 0.0007 | - |
3.6866 | 13150 | 0.0007 | - |
3.7006 | 13200 | 0.0008 | - |
3.7146 | 13250 | 0.0007 | - |
3.7286 | 13300 | 0.0031 | - |
3.7426 | 13350 | 0.0006 | - |
3.7567 | 13400 | 0.0008 | - |
3.7707 | 13450 | 0.0007 | - |
3.7847 | 13500 | 0.0006 | - |
3.7987 | 13550 | 0.0007 | - |
3.8127 | 13600 | 0.0008 | - |
3.8267 | 13650 | 0.0007 | - |
3.8408 | 13700 | 0.0008 | - |
3.8548 | 13750 | 0.0007 | - |
3.8688 | 13800 | 0.0007 | - |
3.8828 | 13850 | 0.0007 | - |
3.8968 | 13900 | 0.0007 | - |
3.9108 | 13950 | 0.0007 | - |
3.9249 | 14000 | 0.0031 | - |
3.9389 | 14050 | 0.003 | - |
3.9529 | 14100 | 0.0007 | - |
3.9669 | 14150 | 0.0007 | - |
3.9809 | 14200 | 0.0007 | - |
3.9950 | 14250 | 0.0007 | - |
4.0 | 14268 | - | 0.1155 |
Framework Versions
- Python: 3.11.10
- SetFit: 1.1.0
- Sentence Transformers: 3.2.0
- Transformers: 4.45.2
- PyTorch: 2.4.1+cu124
- Datasets: 3.0.1
- Tokenizers: 0.20.1
Citation
BibTeX
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}