insuperabile commited on
Commit
73b50e1
·
verified ·
1 Parent(s): cb98252

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -2
README.md CHANGED
@@ -105,8 +105,8 @@ You can finetune this model on your own dataset.
105
  ## Encodechka
106
  | Model | Model Parameters | STS | PI | NLI | SA | TI | IC | ICX | NEI1 | NEI2 | AVG |
107
  |---------------------------------------|------------------|------|------|------|------|------|------|------|------|------|------|
108
- | insuperabile/rumodernbert-hnp | 149M | 0.78 | 0.67 | 0.49 | 0.79 | 0.97 | 0.73 | 0.68 | 0.26 | 0.34 | 0.63 |
109
  | insuperabile/rumodernbert-solyanka | 149M | 0.8 | 0.56 | 0.4 | 0.76 | 0.98 | 0.73 | 0.67 | 0.33 | 0.36 | 0.62 |
 
110
  | insuperabile/rumodernbert-solyanka-QP | 149M | 0.81 | 0.65 | 0.4 | 0.81 | 0.98 | 0.79 | 0.74 | 0.35 | 0.41 | 0.66 |
111
  | deepvk/USER-base | 124M | 0.85 | 0.74 | 0.48 | 0.81 | 0.99 | 0.8 | 0.7 | 0.29 | 0.41 | 0.68 |
112
  | paraphrase-multilingual-MiniLM-L12-v2 | 118M | 0.84 | 0.62 | 0.5 | 0.76 | 0.92 | 0.77 | 0.72 | - | - | - |
@@ -116,7 +116,6 @@ You can finetune this model on your own dataset.
116
  ## RuMTEB
117
  | model | avg | CEDRClass | GeoreviewClassification | GeoreviewClustering | HeadlineClassif | InappClassif | Kinopoisk | RiaRetrieval | RuBQReranking | RubqRetrieval | RuReviewsClass | RuSTSBench | RSBGClassif | RSBGCluster | RSBOClassif | RSBOCluster | SensitiveClassif | TERRa |
118
  |:--------------------------------------|--------:|------------:|--------------------------:|----------------------:|------------------:|---------------:|------------:|---------------:|----------------:|----------------:|-----------------:|-------------:|--------------:|--------------:|--------------:|--------------:|-------------------:|--------:|
119
- | rumodernbert-hnp | 51.2224 | 45.12 | 40.24 | 52.05 | 70.1 | 59.58 | 51.13 | 45.54 | 60.98 | 40 | 58.61 | 78.53 | 47.12 | 45.71 | 36.7 | 41.45 | 40.81 | 57.11 |
120
  | rumodernbert-solyanka | 53.2006 | 38.34 | 33.79 | 66.68 | 79.36 | 60.71 | 44.78 | 50.67 | 63.57 | 53.58 | 51.05 | 80.07 | 52.31 | 51.17 | 41.01 | 45.21 | 41.39 | 50.72 |
121
  | rumodernbert-solyanka-qp | 56.5847 | 39.44 | 37.72 | 71.23 | 73.85 | 59.97 | 50.37 | 73.09 | 68.07 | 62.65 | 56.59 | 81.64 | 56.04 | 53.40 | 44.48 | 46.80 | 32.82 | 53.78 |
122
  | user-base | 57.6429 | 46.78 | 46.88 | 63.41 | 75 | 61.83 | 56.03 | 77.72 | 64.42 | 56.86 | 65.48 | 81.91 | 55.55 | 51.5 | 43.28 | 44.87 | 28.65 | 59.76 |
 
105
  ## Encodechka
106
  | Model | Model Parameters | STS | PI | NLI | SA | TI | IC | ICX | NEI1 | NEI2 | AVG |
107
  |---------------------------------------|------------------|------|------|------|------|------|------|------|------|------|------|
 
108
  | insuperabile/rumodernbert-solyanka | 149M | 0.8 | 0.56 | 0.4 | 0.76 | 0.98 | 0.73 | 0.67 | 0.33 | 0.36 | 0.62 |
109
+ | insuperabile/SimBERT_RU | 149M | 0.79 | 0.73 | 0.51 | 0.80 | 0.98 | 0.78 | 0.74 | 0.28 | 0.37 | 0.66 |
110
  | insuperabile/rumodernbert-solyanka-QP | 149M | 0.81 | 0.65 | 0.4 | 0.81 | 0.98 | 0.79 | 0.74 | 0.35 | 0.41 | 0.66 |
111
  | deepvk/USER-base | 124M | 0.85 | 0.74 | 0.48 | 0.81 | 0.99 | 0.8 | 0.7 | 0.29 | 0.41 | 0.68 |
112
  | paraphrase-multilingual-MiniLM-L12-v2 | 118M | 0.84 | 0.62 | 0.5 | 0.76 | 0.92 | 0.77 | 0.72 | - | - | - |
 
116
  ## RuMTEB
117
  | model | avg | CEDRClass | GeoreviewClassification | GeoreviewClustering | HeadlineClassif | InappClassif | Kinopoisk | RiaRetrieval | RuBQReranking | RubqRetrieval | RuReviewsClass | RuSTSBench | RSBGClassif | RSBGCluster | RSBOClassif | RSBOCluster | SensitiveClassif | TERRa |
118
  |:--------------------------------------|--------:|------------:|--------------------------:|----------------------:|------------------:|---------------:|------------:|---------------:|----------------:|----------------:|-----------------:|-------------:|--------------:|--------------:|--------------:|--------------:|-------------------:|--------:|
 
119
  | rumodernbert-solyanka | 53.2006 | 38.34 | 33.79 | 66.68 | 79.36 | 60.71 | 44.78 | 50.67 | 63.57 | 53.58 | 51.05 | 80.07 | 52.31 | 51.17 | 41.01 | 45.21 | 41.39 | 50.72 |
120
  | rumodernbert-solyanka-qp | 56.5847 | 39.44 | 37.72 | 71.23 | 73.85 | 59.97 | 50.37 | 73.09 | 68.07 | 62.65 | 56.59 | 81.64 | 56.04 | 53.40 | 44.48 | 46.80 | 32.82 | 53.78 |
121
  | user-base | 57.6429 | 46.78 | 46.88 | 63.41 | 75 | 61.83 | 56.03 | 77.72 | 64.42 | 56.86 | 65.48 | 81.91 | 55.55 | 51.5 | 43.28 | 44.87 | 28.65 | 59.76 |