ali170506/chab

SetFit with sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 128 tokens
• Number of Classes: 9 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
schedule_a_visit	'I’d like to schedule a visit' 'Je voudrais planifier une visite' 'Puis-je programmer une visite?'
check_availability	'Est-ce encore disponible?' 'Is this still available?' 'Can I check availability?'
amenities_and_features	'Parlez-moi des fonctionnalités du bien' 'Tell me the features of the property' 'Quels sont les équipements disponibles?'
payment_plan	'Pouvez-vous me parler du plan de paiement?' 'Quels sont les modes de paiement disponibles?' 'What are the payment options?'
reservation_process	'Tell me about the reservation process' 'Pouvez-vous m’expliquer le processus de réservation?' 'Comment puis-je faire une réservation?'
location_details	'Où est-ce situé?' 'Can you tell me the location details?' 'What’s the address?'
pricing_details	'How much does it cost?' 'Tell me the pricing details' 'Combien ça coûte?'
option_process	'Tell me about the option process' 'Parlez-moi du processus des options' 'Quels sont mes choix?'
information_on_projects	'Can you give me information about the projects?' 'I need details on the available projects' 'Quels sont les projets disponibles ?'

Evaluation

Metrics

Label	Accuracy
all	1.0

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("ali170506/chab")
# Run inference
preds = model("Is it available?")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	3	5.2222	8

Label	Training Sample Count
information_on_projects	3
pricing_details	3
location_details	3
amenities_and_features	3
check_availability	3
schedule_a_visit	3
reservation_process	3
option_process	3
payment_plan	3

Training Hyperparameters

• batch_size: (4, 4)
• num_epochs: (4, 4)
• max_steps: -1
• sampling_strategy: oversampling
• body_learning_rate: (2e-05, 1e-05)
• head_learning_rate: 0.01
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: True

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0062	1	0.0311	-
0.0617	10	0.0989	-
0.1235	20	0.0036	-
0.1852	30	0.0121	-
0.2469	40	0.0209	-
0.3086	50	0.001	-
0.3704	60	0.0067	-
0.4321	70	0.017	-
0.4938	80	0.0037	-
0.5556	90	0.012	-
0.6173	100	0.0009	-
0.6790	110	0.0044	-
0.7407	120	0.0014	-
0.8025	130	0.0006	-
0.8642	140	0.0016	-
0.9259	150	0.0024	-
0.9877	160	0.0011	-
1.0	162	-	0.0164
1.0494	170	0.0019	-
1.1111	180	0.0017	-
1.1728	190	0.0004	-
1.2346	200	0.0008	-
1.2963	210	0.0012	-
1.3580	220	0.0009	-
1.4198	230	0.0006	-
1.4815	240	0.001	-
1.5432	250	0.0009	-
1.6049	260	0.0015	-
1.6667	270	0.0016	-
1.7284	280	0.0009	-
1.7901	290	0.0005	-
1.8519	300	0.0009	-
1.9136	310	0.0009	-
1.9753	320	0.0008	-
2.0	324	-	0.0138
2.0370	330	0.0011	-
2.0988	340	0.0016	-
2.1605	350	0.0006	-
2.2222	360	0.0012	-
2.2840	370	0.0014	-
2.3457	380	0.0009	-
2.4074	390	0.0008	-
2.4691	400	0.0003	-
2.5309	410	0.0002	-
2.5926	420	0.0007	-
2.6543	430	0.001	-
2.7160	440	0.0008	-
2.7778	450	0.0008	-
2.8395	460	0.0003	-
2.9012	470	0.0004	-
2.9630	480	0.0003	-
3.0	486	-	0.0129
3.0247	490	0.0013	-
3.0864	500	0.0006	-
3.1481	510	0.0008	-
3.2099	520	0.0001	-
3.2716	530	0.0007	-
3.3333	540	0.0004	-
3.3951	550	0.0004	-
3.4568	560	0.0003	-
3.5185	570	0.0003	-
3.5802	580	0.0002	-
3.6420	590	0.0002	-
3.7037	600	0.0002	-
3.7654	610	0.0007	-
3.8272	620	0.0007	-
3.8889	630	0.0007	-
3.9506	640	0.0003	-
4.0	648	-	0.0129

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.12
• SetFit: 1.0.3
• Sentence Transformers: 3.0.1
• Transformers: 4.37.0
• PyTorch: 2.4.1+cu121
• Datasets: 3.0.1
• Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}