Update README.md
Browse files
README.md
CHANGED
@@ -105,8 +105,8 @@ You can finetune this model on your own dataset.
|
|
105 |
## Encodechka
|
106 |
| Model | Model Parameters | STS | PI | NLI | SA | TI | IC | ICX | NEI1 | NEI2 | AVG |
|
107 |
|---------------------------------------|------------------|------|------|------|------|------|------|------|------|------|------|
|
108 |
-
| insuperabile/rumodernbert-hnp | 149M | 0.78 | 0.67 | 0.49 | 0.79 | 0.97 | 0.73 | 0.68 | 0.26 | 0.34 | 0.63 |
|
109 |
| insuperabile/rumodernbert-solyanka | 149M | 0.8 | 0.56 | 0.4 | 0.76 | 0.98 | 0.73 | 0.67 | 0.33 | 0.36 | 0.62 |
|
|
|
110 |
| insuperabile/rumodernbert-solyanka-QP | 149M | 0.81 | 0.65 | 0.4 | 0.81 | 0.98 | 0.79 | 0.74 | 0.35 | 0.41 | 0.66 |
|
111 |
| deepvk/USER-base | 124M | 0.85 | 0.74 | 0.48 | 0.81 | 0.99 | 0.8 | 0.7 | 0.29 | 0.41 | 0.68 |
|
112 |
| paraphrase-multilingual-MiniLM-L12-v2 | 118M | 0.84 | 0.62 | 0.5 | 0.76 | 0.92 | 0.77 | 0.72 | - | - | - |
|
@@ -116,7 +116,6 @@ You can finetune this model on your own dataset.
|
|
116 |
## RuMTEB
|
117 |
| model | avg | CEDRClass | GeoreviewClassification | GeoreviewClustering | HeadlineClassif | InappClassif | Kinopoisk | RiaRetrieval | RuBQReranking | RubqRetrieval | RuReviewsClass | RuSTSBench | RSBGClassif | RSBGCluster | RSBOClassif | RSBOCluster | SensitiveClassif | TERRa |
|
118 |
|:--------------------------------------|--------:|------------:|--------------------------:|----------------------:|------------------:|---------------:|------------:|---------------:|----------------:|----------------:|-----------------:|-------------:|--------------:|--------------:|--------------:|--------------:|-------------------:|--------:|
|
119 |
-
| rumodernbert-hnp | 51.2224 | 45.12 | 40.24 | 52.05 | 70.1 | 59.58 | 51.13 | 45.54 | 60.98 | 40 | 58.61 | 78.53 | 47.12 | 45.71 | 36.7 | 41.45 | 40.81 | 57.11 |
|
120 |
| rumodernbert-solyanka | 53.2006 | 38.34 | 33.79 | 66.68 | 79.36 | 60.71 | 44.78 | 50.67 | 63.57 | 53.58 | 51.05 | 80.07 | 52.31 | 51.17 | 41.01 | 45.21 | 41.39 | 50.72 |
|
121 |
| rumodernbert-solyanka-qp | 56.5847 | 39.44 | 37.72 | 71.23 | 73.85 | 59.97 | 50.37 | 73.09 | 68.07 | 62.65 | 56.59 | 81.64 | 56.04 | 53.40 | 44.48 | 46.80 | 32.82 | 53.78 |
|
122 |
| user-base | 57.6429 | 46.78 | 46.88 | 63.41 | 75 | 61.83 | 56.03 | 77.72 | 64.42 | 56.86 | 65.48 | 81.91 | 55.55 | 51.5 | 43.28 | 44.87 | 28.65 | 59.76 |
|
|
|
105 |
## Encodechka
|
106 |
| Model | Model Parameters | STS | PI | NLI | SA | TI | IC | ICX | NEI1 | NEI2 | AVG |
|
107 |
|---------------------------------------|------------------|------|------|------|------|------|------|------|------|------|------|
|
|
|
108 |
| insuperabile/rumodernbert-solyanka | 149M | 0.8 | 0.56 | 0.4 | 0.76 | 0.98 | 0.73 | 0.67 | 0.33 | 0.36 | 0.62 |
|
109 |
+
| insuperabile/SimBERT_RU | 149M | 0.79 | 0.73 | 0.51 | 0.80 | 0.98 | 0.78 | 0.74 | 0.28 | 0.37 | 0.66 |
|
110 |
| insuperabile/rumodernbert-solyanka-QP | 149M | 0.81 | 0.65 | 0.4 | 0.81 | 0.98 | 0.79 | 0.74 | 0.35 | 0.41 | 0.66 |
|
111 |
| deepvk/USER-base | 124M | 0.85 | 0.74 | 0.48 | 0.81 | 0.99 | 0.8 | 0.7 | 0.29 | 0.41 | 0.68 |
|
112 |
| paraphrase-multilingual-MiniLM-L12-v2 | 118M | 0.84 | 0.62 | 0.5 | 0.76 | 0.92 | 0.77 | 0.72 | - | - | - |
|
|
|
116 |
## RuMTEB
|
117 |
| model | avg | CEDRClass | GeoreviewClassification | GeoreviewClustering | HeadlineClassif | InappClassif | Kinopoisk | RiaRetrieval | RuBQReranking | RubqRetrieval | RuReviewsClass | RuSTSBench | RSBGClassif | RSBGCluster | RSBOClassif | RSBOCluster | SensitiveClassif | TERRa |
|
118 |
|:--------------------------------------|--------:|------------:|--------------------------:|----------------------:|------------------:|---------------:|------------:|---------------:|----------------:|----------------:|-----------------:|-------------:|--------------:|--------------:|--------------:|--------------:|-------------------:|--------:|
|
|
|
119 |
| rumodernbert-solyanka | 53.2006 | 38.34 | 33.79 | 66.68 | 79.36 | 60.71 | 44.78 | 50.67 | 63.57 | 53.58 | 51.05 | 80.07 | 52.31 | 51.17 | 41.01 | 45.21 | 41.39 | 50.72 |
|
120 |
| rumodernbert-solyanka-qp | 56.5847 | 39.44 | 37.72 | 71.23 | 73.85 | 59.97 | 50.37 | 73.09 | 68.07 | 62.65 | 56.59 | 81.64 | 56.04 | 53.40 | 44.48 | 46.80 | 32.82 | 53.78 |
|
121 |
| user-base | 57.6429 | 46.78 | 46.88 | 63.41 | 75 | 61.83 | 56.03 | 77.72 | 64.42 | 56.86 | 65.48 | 81.91 | 55.55 | 51.5 | 43.28 | 44.87 | 28.65 | 59.76 |
|