SetFit with sentence-transformers/stsb-xlm-r-multilingual

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/stsb-xlm-r-multilingual as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/stsb-xlm-r-multilingual
Classification head: a LogisticRegression instance
Maximum Sequence Length: 128 tokens
Number of Classes: 12 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
optimism	"in spite of all that he's witnessed, remains surprisingly idealistic" "but on the whole, you're gon na like this movie." 'phrase life affirming'
discomfort	'visceral and dangerously honest revelations about the men and machines behind the curtains of our planet' 'the subtlest and most complexly evil uncle ralph' 'the characters, cast in impossibly contrived situations, are totally estranged from reality.'
anger	'crime drama' 'a gritty police thriller with all the dysfunctional family dynamics one could wish for' 'atrocities'
joy	'nice' 'sets itself apart by forming a chain of relationships that come full circle to end on a positive (if tragic) note' 'earnest movie'
sadness	'consumed by lust and love and crushed by betrayal that it conjures up the intoxicating fumes and emotional ghosts of a freshly painted rembrandt' 'has the stomach-knotting suspense of a legal thriller, while the testimony of witnesses lends the film a resonant undertone of tragedy' 'a bittersweet drama about the limbo of grief and how truth-telling can open the door to liberation.'
neutral	'bound' 'feels impersonal, almost generic.' 'gr'
disappointment	'his fake backdrops' 'i did go back and check out the last 10 minutes, but these were more repulsive than the first 30 or 40 minutes.' "though many of the actors throw off a spark or two when they first appear, they can'tgenerate enough heat in this cold vacuum of a comedy to start a reaction."
frustration	'just such a dungpile' 'of this tacky nonsense' 'exactly how bad it is'
admiration	'of extraordinary journalism' 'classical actress' 'does its predecessors proud.'
excitement	'some strong supporting players' 'outrageously creative action' 'gangs excels in spectacle and pacing.'
amusement	'fascinating, ludicrous, provocative and vainglorious' 'will amuse or entertain them' '4ever has the same sledgehammer appeal as pokemon videos'
confusion	'muddled and derivative that few will bother thinking it all through' 'leaves vague impressions and a nasty aftertaste but little clear memory of its operational mechanics' 'trying to cope with the mysterious and brutal nature of adults'

Evaluation

Metrics

Label	Accuracy
all	0.4645

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("jovemexausto/setfit-xlmr-emotions")
# Run inference
preds = model("is flat.")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	8.0486	31

Label	Training Sample Count
neutral	12
admiration	12
amusement	12
anger	12
confusion	12
disappointment	12
discomfort	12
excitement	12
frustration	12
joy	12
optimism	12
sadness	12

Training Hyperparameters

batch_size: (72, 72)
num_epochs: (1, 1)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0038	1	0.2344	-
0.1894	50	0.1877	-
0.3788	100	0.0875	-
0.5682	150	0.0436	-
0.7576	200	0.0259	-
0.9470	250	0.0196	-
1.0	264	-	0.2041

Framework Versions

Python: 3.11.13
SetFit: 1.1.2
Sentence Transformers: 4.1.0
Transformers: 4.52.4
PyTorch: 2.6.0+cu124
Datasets: 3.6.0
Tokenizers: 0.21.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

jovemexausto
/

setfit-xlmr-emotions