SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

Fine-tuning a Sentence Transformer with contrastive learning.
Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Type: SetFit
Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
Classification head: a LogisticRegression instance
Maximum Sequence Length: 128 tokens
Number of Classes: 6 classes

Model Sources

Repository: SetFit on GitHub
Paper: Efficient Few-Shot Learning Without Prompts
Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
matches-match_time	'Norwich City vs Newcastle United' 'will Manchester United play with chelsea' 'est-ce que Manchester United jouera avec chelsea'
matches-match_result	'Liverpool and West Ham result' 'what is the score of Wolverhampton match' 'who won in Liverpool vs Newcastle United match'
greet-who_are_you	'how can you help me' "pourquoi j'ai besoin de toi" 'je ne te comprends pas'
matches-team_next_match	'Real Madrid fixtures' 'quels sont les prochains matchs de Borussia Dortmund' 'próximos partidos de Atletico Madrid'
greet-good_bye	'See you later' 'A plus tard' 'stop'
greet-hi	'Hello buddy' 'Salut' 'Hey'

Evaluation

Metrics

Label	Accuracy
all	1.0

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("Ah7med/setfit-football_bootpress_paraph-multi-v2")
# Run inference
preds = model("why do I need you")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	5.2	10

Label	Training Sample Count
greet-hi	5
greet-who_are_you	7
greet-good_bye	5
matches-team_next_match	21
matches-match_time	12
matches-match_result	15

Training Hyperparameters

batch_size: (4, 4)
num_epochs: (4, 4)
max_steps: -1
sampling_strategy: oversampling
body_learning_rate: (2e-05, 1e-05)
head_learning_rate: 0.01
loss: CosineSimilarityLoss
distance_metric: cosine_distance
margin: 0.25
end_to_end: False
use_amp: False
warmup_proportion: 0.1
l2_weight: 0.01
seed: 42
eval_max_steps: -1
load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0012	1	0.1308	-
0.0603	50	0.1596	-
0.1206	100	0.1399	-
0.1809	150	0.089	-
0.2413	200	0.0461	-
0.3016	250	0.026	-
0.3619	300	0.0081	-
0.4222	350	0.0048	-
0.4825	400	0.0039	-
0.5428	450	0.0018	-
0.6031	500	0.002	-
0.6634	550	0.0015	-
0.7238	600	0.0011	-
0.7841	650	0.0009	-
0.8444	700	0.0008	-
0.9047	750	0.0005	-
0.9650	800	0.0007	-
1.0	829	-	0.0211
1.0253	850	0.0006	-
1.0856	900	0.0005	-
1.1460	950	0.0005	-
1.2063	1000	0.0003	-
1.2666	1050	0.0003	-
1.3269	1100	0.0004	-
1.3872	1150	0.0003	-
1.4475	1200	0.0004	-
1.5078	1250	0.0002	-
1.5682	1300	0.0003	-
1.6285	1350	0.0003	-
1.6888	1400	0.0003	-
1.7491	1450	0.0003	-
1.8094	1500	0.0003	-
1.8697	1550	0.0003	-
1.9300	1600	0.0002	-
1.9903	1650	0.0002	-
2.0	1658	-	0.0190
2.0507	1700	0.0003	-
2.1110	1750	0.0002	-
2.1713	1800	0.0002	-
2.2316	1850	0.0002	-
2.2919	1900	0.0002	-
2.3522	1950	0.0002	-
2.4125	2000	0.0002	-
2.4729	2050	0.0002	-
2.5332	2100	0.0002	-
2.5935	2150	0.0002	-
2.6538	2200	0.0001	-
2.7141	2250	0.0002	-
2.7744	2300	0.0001	-
2.8347	2350	0.0002	-
2.8951	2400	0.0001	-
2.9554	2450	0.0002	-
3.0	2487	-	0.0181
3.0157	2500	0.0002	-
3.0760	2550	0.0001	-
3.1363	2600	0.0001	-
3.1966	2650	0.0001	-
3.2569	2700	0.0001	-
3.3172	2750	0.0001	-
3.3776	2800	0.0001	-
3.4379	2850	0.0001	-
3.4982	2900	0.0001	-
3.5585	2950	0.0001	-
3.6188	3000	0.0001	-
3.6791	3050	0.0001	-
3.7394	3100	0.0001	-
3.7998	3150	0.0001	-
3.8601	3200	0.0001	-
3.9204	3250	0.0001	-
3.9807	3300	0.0001	-
4.0	3316	-	0.0176

Framework Versions

Python: 3.11.12
SetFit: 1.1.2
Sentence Transformers: 3.4.1
Transformers: 4.51.3
PyTorch: 2.6.0+cu124
Datasets: 3.5.1
Tokenizers: 0.21.1

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}

Ah7med
/

setfit-football_bootpress_paraph-multi-v2