Быстрая модель BERT для русского языка с размером ембеддинга 256 и длиной контекста 512. Модель получена методом последовательной дистилляции моделей sergeyzh/rubert-tiny-turbo и BAAI/bge-m3. Выигрывает по скорости у rubert-tiny-turbo при аналогичном качестве на CPU в ~x1.4, на GPU в ~x1.2 раза.

Использование

from sentence_transformers import SentenceTransformer

model = SentenceTransformer('sergeyzh/rubert-tiny-lite')

sentences = ["привет мир", "hello world", "здравствуй вселенная"]
embeddings = model.encode(sentences)

print(model.similarity(embeddings, embeddings))

Метрики

Оценки модели на бенчмарке encodechka:

model STS PI NLI SA TI
BAAI/bge-m3 0.864 0.749 0.510 0.819 0.973
intfloat/multilingual-e5-large 0.862 0.727 0.473 0.810 0.979
sergeyzh/rubert-tiny-lite 0.839 0.712 0.488 0.788 0.949
intfloat/multilingual-e5-base 0.835 0.704 0.459 0.796 0.964
sergeyzh/rubert-tiny-turbo 0.828 0.722 0.476 0.787 0.955
intfloat/multilingual-e5-small 0.822 0.714 0.457 0.758 0.957
cointegrated/rubert-tiny2 0.750 0.651 0.417 0.737 0.937

Оценки модели на бенчмарке ruMTEB:

Model Name Metric rubert-tiny2 rubert-tiny-turbo rubert-tiny-lite multilingual-e5-small multilingual-e5-base multilingual-e5-large
CEDRClassification Accuracy 0.369 0.390 0.407 0.401 0.423 0.448
GeoreviewClassification Accuracy 0.396 0.414 0.423 0.447 0.461 0.497
GeoreviewClusteringP2P V-measure 0.442 0.597 0.611 0.586 0.545 0.605
HeadlineClassification Accuracy 0.742 0.686 0.652 0.732 0.757 0.758
InappropriatenessClassification Accuracy 0.586 0.591 0.588 0.592 0.588 0.616
KinopoiskClassification Accuracy 0.491 0.505 0.507 0.500 0.509 0.566
RiaNewsRetrieval NDCG@10 0.140 0.513 0.617 0.700 0.702 0.807
RuBQReranking MAP@10 0.461 0.622 0.631 0.715 0.720 0.756
RuBQRetrieval NDCG@10 0.109 0.517 0.511 0.685 0.696 0.741
RuReviewsClassification Accuracy 0.570 0.607 0.615 0.612 0.630 0.653
RuSTSBenchmarkSTS Pearson correlation 0.694 0.787 0.799 0.781 0.796 0.831
RuSciBenchGRNTIClassification Accuracy 0.456 0.529 0.544 0.550 0.563 0.582
RuSciBenchGRNTIClusteringP2P V-measure 0.414 0.481 0.510 0.511 0.516 0.520
RuSciBenchOECDClassification Accuracy 0.355 0.415 0.424 0.427 0.423 0.445
RuSciBenchOECDClusteringP2P V-measure 0.381 0.411 0.438 0.443 0.448 0.450
SensitiveTopicsClassification Accuracy 0.220 0.244 0.282 0.228 0.234 0.257
TERRaClassification Average Precision 0.519 0.563 0.574 0.551 0.550 0.584
Model Name Metric rubert-tiny2 rubert-tiny-turbo rubert-tiny-lite multilingual-e5-small multilingual-e5-base multilingual-e5-large
Classification Accuracy 0.514 0.535 0.536 0.551 0.561 0.588
Clustering V-measure 0.412 0.496 0.520 0.513 0.503 0.525
MultiLabelClassification Accuracy 0.294 0.317 0.344 0.314 0.329 0.353
PairClassification Average Precision 0.519 0.563 0.574 0.551 0.550 0.584
Reranking MAP@10 0.461 0.622 0.631 0.715 0.720 0.756
Retrieval NDCG@10 0.124 0.515 0.564 0.697 0.699 0.774
STS Pearson correlation 0.694 0.787 0.799 0.781 0.796 0.831
Average Average 0.431 0.548 0.567 0.588 0.594 0.630
Downloads last month
19
Safetensors
Model size
23M params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Datasets used to train sergeyzh/rubert-tiny-lite

Collection including sergeyzh/rubert-tiny-lite