Hugging Face's logo

ai-forever
/
FRIDA

Feature Extraction
sentence-transformers
Safetensors
Transformers
Russian
English
t5
mteb
Eval Results
text-generation-inference
Model card Files Community
6
ai-forever commited on
Commit
aed004d
·
verified ·
1 Parent(s): fb57740

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1561 -1559
README.md CHANGED
@@ -1,1561 +1,1563 @@
1
- ---
2
- model-index:
3
- - name: FRIDA
4
- results:
5
- - dataset:
6
- config: default
7
- name: MTEB CEDRClassification (default)
8
- revision: c0ba03d058e3e1b2f3fd20518875a4563dd12db4
9
- split: test
10
- type: ai-forever/cedr-classification
11
- metrics:
12
- - type: accuracy
13
- value: 64.60148777895856
14
- - type: f1
15
- value: 70.36630348039266
16
- - type: lrap
17
- value: 92.47290116896953
18
- - type: main_score
19
- value: 64.60148777895856
20
- task:
21
- type: MultilabelClassification
22
- - dataset:
23
- config: default
24
- name: MTEB GeoreviewClassification (default)
25
- revision: 3765c0d1de6b7d264bc459433c45e5a75513839c
26
- split: test
27
- type: ai-forever/georeview-classification
28
- metrics:
29
- - type: accuracy
30
- value: 57.70996093750001
31
- - type: f1
32
- value: 53.18542982057098
33
- - type: f1_weighted
34
- value: 53.17663229582108
35
- - type: main_score
36
- value: 57.70996093750001
37
- task:
38
- type: Classification
39
- - dataset:
40
- config: default
41
- name: MTEB GeoreviewClusteringP2P (default)
42
- revision: 97a313c8fc85b47f13f33e7e9a95c1ad888c7fec
43
- split: test
44
- type: ai-forever/georeview-clustering-p2p
45
- metrics:
46
- - type: main_score
47
- value: 78.25468393043356
48
- - type: v_measure
49
- value: 78.25468393043356
50
- - type: v_measure_std
51
- value: 0.5094366871364238
52
- task:
53
- type: Clustering
54
- - dataset:
55
- config: default
56
- name: MTEB HeadlineClassification (default)
57
- revision: 2fe05ee6b5832cda29f2ef7aaad7b7fe6a3609eb
58
- split: test
59
- type: ai-forever/headline-classification
60
- metrics:
61
- - type: accuracy
62
- value: 89.0185546875
63
- - type: f1
64
- value: 88.993933120612
65
- - type: f1_weighted
66
- value: 88.99276764225768
67
- - type: main_score
68
- value: 89.0185546875
69
- task:
70
- type: Classification
71
- - dataset:
72
- config: default
73
- name: MTEB InappropriatenessClassification (default)
74
- revision: 601651fdc45ef243751676e62dd7a19f491c0285
75
- split: test
76
- type: ai-forever/inappropriateness-classification
77
- metrics:
78
- - type: accuracy
79
- value: 78.330078125
80
- - type: ap
81
- value: 73.17856750532495
82
- - type: ap_weighted
83
- value: 73.17856750532495
84
- - type: f1
85
- value: 78.20169867599041
86
- - type: f1_weighted
87
- value: 78.20169867599041
88
- - type: main_score
89
- value: 78.330078125
90
- task:
91
- type: Classification
92
- - dataset:
93
- config: default
94
- name: MTEB KinopoiskClassification (default)
95
- revision: 5911f26666ac11af46cb9c6849d0dc80a378af24
96
- split: test
97
- type: ai-forever/kinopoisk-sentiment-classification
98
- metrics:
99
- - type: accuracy
100
- value: 70.46666666666665
101
- - type: f1
102
- value: 65.83951766538878
103
- - type: f1_weighted
104
- value: 65.83951766538878
105
- - type: main_score
106
- value: 70.46666666666665
107
- task:
108
- type: Classification
109
- - dataset:
110
- config: ru
111
- name: MTEB MIRACLReranking (ru)
112
- revision: 6d1962c527217f8927fca80f890f14f36b2802af
113
- split: dev
114
- type: miracl/mmteb-miracl-reranking
115
- metrics:
116
- - type: MAP@1(MIRACL)
117
- value: 39.023
118
- - type: MAP@10(MIRACL)
119
- value: 60.208
120
- - type: MAP@100(MIRACL)
121
- value: 61.672000000000004
122
- - type: MAP@1000(MIRACL)
123
- value: 61.672000000000004
124
- - type: MAP@20(MIRACL)
125
- value: 61.30799999999999
126
- - type: MAP@3(MIRACL)
127
- value: 53.33
128
- - type: MAP@5(MIRACL)
129
- value: 57.289
130
- - type: NDCG@1(MIRACL)
131
- value: 63.352
132
- - type: NDCG@10(MIRACL)
133
- value: 66.042
134
- - type: NDCG@100(MIRACL)
135
- value: 68.702
136
- - type: NDCG@1000(MIRACL)
137
- value: 68.702
138
- - type: NDCG@20(MIRACL)
139
- value: 67.768
140
- - type: NDCG@3(MIRACL)
141
- value: 61.925
142
- - type: NDCG@5(MIRACL)
143
- value: 63.327
144
- - type: P@1(MIRACL)
145
- value: 63.352
146
- - type: P@10(MIRACL)
147
- value: 16.512
148
- - type: P@100(MIRACL)
149
- value: 1.9529999999999998
150
- - type: P@1000(MIRACL)
151
- value: 0.19499999999999998
152
- - type: P@20(MIRACL)
153
- value: 9.13
154
- - type: P@3(MIRACL)
155
- value: 37.878
156
- - type: P@5(MIRACL)
157
- value: 27.586
158
- - type: Recall@1(MIRACL)
159
- value: 39.023
160
- - type: Recall@10(MIRACL)
161
- value: 72.35000000000001
162
- - type: Recall@100(MIRACL)
163
- value: 79.952
164
- - type: Recall@1000(MIRACL)
165
- value: 79.952
166
- - type: Recall@20(MIRACL)
167
- value: 76.828
168
- - type: Recall@3(MIRACL)
169
- value: 57.769999999999996
170
- - type: Recall@5(MIRACL)
171
- value: 64.91900000000001
172
- - type: main_score
173
- value: 66.042
174
- - type: nAUC_MAP@1000_diff1(MIRACL)
175
- value: 27.150388833033052
176
- - type: nAUC_MAP@1000_max(MIRACL)
177
- value: 55.15672274267081
178
- - type: nAUC_MAP@1000_std(MIRACL)
179
- value: 30.088939934575553
180
- - type: nAUC_MAP@100_diff1(MIRACL)
181
- value: 27.150388833033052
182
- - type: nAUC_MAP@100_max(MIRACL)
183
- value: 55.15672274267081
184
- - type: nAUC_MAP@100_std(MIRACL)
185
- value: 30.088939934575553
186
- - type: nAUC_MAP@10_diff1(MIRACL)
187
- value: 27.853691773641742
188
- - type: nAUC_MAP@10_max(MIRACL)
189
- value: 52.89390350055654
190
- - type: nAUC_MAP@10_std(MIRACL)
191
- value: 28.08732516551691
192
- - type: nAUC_MAP@1_diff1(MIRACL)
193
- value: 43.23179150244192
194
- - type: nAUC_MAP@1_max(MIRACL)
195
- value: 29.923943954188864
196
- - type: nAUC_MAP@1_std(MIRACL)
197
- value: 7.447084370195121
198
- - type: nAUC_MAP@20_diff1(MIRACL)
199
- value: 27.328384072311675
200
- - type: nAUC_MAP@20_max(MIRACL)
201
- value: 54.60286379835721
202
- - type: nAUC_MAP@20_std(MIRACL)
203
- value: 29.8084128980043
204
- - type: nAUC_MAP@3_diff1(MIRACL)
205
- value: 31.244971536944554
206
- - type: nAUC_MAP@3_max(MIRACL)
207
- value: 43.63984692803854
208
- - type: nAUC_MAP@3_std(MIRACL)
209
- value: 18.609234683765887
210
- - type: nAUC_MAP@5_diff1(MIRACL)
211
- value: 29.088760492638286
212
- - type: nAUC_MAP@5_max(MIRACL)
213
- value: 48.30474364461509
214
- - type: nAUC_MAP@5_std(MIRACL)
215
- value: 23.817514353844224
216
- - type: nAUC_NDCG@1000_diff1(MIRACL)
217
- value: 23.12754356408408
218
- - type: nAUC_NDCG@1000_max(MIRACL)
219
- value: 64.24894553363303
220
- - type: nAUC_NDCG@1000_std(MIRACL)
221
- value: 38.19318050598967
222
- - type: nAUC_NDCG@100_diff1(MIRACL)
223
- value: 23.12754356408408
224
- - type: nAUC_NDCG@100_max(MIRACL)
225
- value: 64.24894553363303
226
- - type: nAUC_NDCG@100_std(MIRACL)
227
- value: 38.19318050598967
228
- - type: nAUC_NDCG@10_diff1(MIRACL)
229
- value: 24.779856373697275
230
- - type: nAUC_NDCG@10_max(MIRACL)
231
- value: 60.4054459738118
232
- - type: nAUC_NDCG@10_std(MIRACL)
233
- value: 35.148950441182784
234
- - type: nAUC_NDCG@1_diff1(MIRACL)
235
- value: 35.605865569438556
236
- - type: nAUC_NDCG@1_max(MIRACL)
237
- value: 65.77787399715454
238
- - type: nAUC_NDCG@1_std(MIRACL)
239
- value: 34.34726892885082
240
- - type: nAUC_NDCG@20_diff1(MIRACL)
241
- value: 23.71231783125691
242
- - type: nAUC_NDCG@20_max(MIRACL)
243
- value: 62.89676599488004
244
- - type: nAUC_NDCG@20_std(MIRACL)
245
- value: 37.697052941884316
246
- - type: nAUC_NDCG@3_diff1(MIRACL)
247
- value: 26.109027741640865
248
- - type: nAUC_NDCG@3_max(MIRACL)
249
- value: 56.22356793638693
250
- - type: nAUC_NDCG@3_std(MIRACL)
251
- value: 29.9437568508688
252
- - type: nAUC_NDCG@5_diff1(MIRACL)
253
- value: 25.98644715327336
254
- - type: nAUC_NDCG@5_max(MIRACL)
255
- value: 56.25032008404774
256
- - type: nAUC_NDCG@5_std(MIRACL)
257
- value: 31.581899860862578
258
- - type: nAUC_P@1000_diff1(MIRACL)
259
- value: -18.29912787064644
260
- - type: nAUC_P@1000_max(MIRACL)
261
- value: 31.811344878776087
262
- - type: nAUC_P@1000_std(MIRACL)
263
- value: 30.163820183304914
264
- - type: nAUC_P@100_diff1(MIRACL)
265
- value: -18.299127870646405
266
- - type: nAUC_P@100_max(MIRACL)
267
- value: 31.811344878776133
268
- - type: nAUC_P@100_std(MIRACL)
269
- value: 30.163820183304956
270
- - type: nAUC_P@10_diff1(MIRACL)
271
- value: -15.96416268531149
272
- - type: nAUC_P@10_max(MIRACL)
273
- value: 36.989578896466526
274
- - type: nAUC_P@10_std(MIRACL)
275
- value: 34.54507111688143
276
- - type: nAUC_P@1_diff1(MIRACL)
277
- value: 35.605865569438556
278
- - type: nAUC_P@1_max(MIRACL)
279
- value: 65.77787399715454
280
- - type: nAUC_P@1_std(MIRACL)
281
- value: 34.34726892885082
282
- - type: nAUC_P@20_diff1(MIRACL)
283
- value: -17.443963421383287
284
- - type: nAUC_P@20_max(MIRACL)
285
- value: 34.309618168778385
286
- - type: nAUC_P@20_std(MIRACL)
287
- value: 33.38820956485373
288
- - type: nAUC_P@3_diff1(MIRACL)
289
- value: -8.533621861815652
290
- - type: nAUC_P@3_max(MIRACL)
291
- value: 45.90408386776497
292
- - type: nAUC_P@3_std(MIRACL)
293
- value: 34.50459351305535
294
- - type: nAUC_P@5_diff1(MIRACL)
295
- value: -13.207968899314865
296
- - type: nAUC_P@5_max(MIRACL)
297
- value: 40.37718282248973
298
- - type: nAUC_P@5_std(MIRACL)
299
- value: 35.601417332196206
300
- - type: nAUC_Recall@1000_diff1(MIRACL)
301
- value: 7.907304198177226
302
- - type: nAUC_Recall@1000_max(MIRACL)
303
- value: 77.82197832361145
304
- - type: nAUC_Recall@1000_std(MIRACL)
305
- value: 52.66957487246724
306
- - type: nAUC_Recall@100_diff1(MIRACL)
307
- value: 7.907304198177226
308
- - type: nAUC_Recall@100_max(MIRACL)
309
- value: 77.82197832361145
310
- - type: nAUC_Recall@100_std(MIRACL)
311
- value: 52.66957487246724
312
- - type: nAUC_Recall@10_diff1(MIRACL)
313
- value: 15.498121023488693
314
- - type: nAUC_Recall@10_max(MIRACL)
315
- value: 62.24320529338724
316
- - type: nAUC_Recall@10_std(MIRACL)
317
- value: 40.60221460946224
318
- - type: nAUC_Recall@1_diff1(MIRACL)
319
- value: 43.23179150244192
320
- - type: nAUC_Recall@1_max(MIRACL)
321
- value: 29.923943954188864
322
- - type: nAUC_Recall@1_std(MIRACL)
323
- value: 7.447084370195121
324
- - type: nAUC_Recall@20_diff1(MIRACL)
325
- value: 11.457044176116248
326
- - type: nAUC_Recall@20_max(MIRACL)
327
- value: 70.3493054342368
328
- - type: nAUC_Recall@20_std(MIRACL)
329
- value: 49.27124296325928
330
- - type: nAUC_Recall@3_diff1(MIRACL)
331
- value: 25.12077828977941
332
- - type: nAUC_Recall@3_max(MIRACL)
333
- value: 42.903379317937166
334
- - type: nAUC_Recall@3_std(MIRACL)
335
- value: 20.324501722161497
336
- - type: nAUC_Recall@5_diff1(MIRACL)
337
- value: 20.925701235197977
338
- - type: nAUC_Recall@5_max(MIRACL)
339
- value: 49.85323960390812
340
- - type: nAUC_Recall@5_std(MIRACL)
341
- value: 29.04484539530469
342
- task:
343
- type: Reranking
344
- - dataset:
345
- config: ru
346
- name: MTEB MIRACLRetrieval (ru)
347
- revision: main
348
- split: dev
349
- type: miracl/mmteb-miracl
350
- metrics:
351
- - type: main_score
352
- value: 71.882
353
- - type: map_at_1
354
- value: 37.913000000000004
355
- - type: map_at_10
356
- value: 62.604000000000006
357
- - type: map_at_100
358
- value: 64.925
359
- - type: map_at_1000
360
- value: 64.992
361
- - type: map_at_20
362
- value: 64.081
363
- - type: map_at_3
364
- value: 55.212
365
- - type: map_at_5
366
- value: 59.445
367
- - type: mrr_at_1
368
- value: 73.24281150159744
369
- - type: mrr_at_10
370
- value: 81.65043866321825
371
- - type: mrr_at_100
372
- value: 81.85391378818977
373
- - type: mrr_at_1000
374
- value: 81.85753390802569
375
- - type: mrr_at_20
376
- value: 81.81045606130179
377
- - type: mrr_at_3
378
- value: 80.56443024494146
379
- - type: mrr_at_5
380
- value: 81.30724174653893
381
- - type: nauc_map_at_1000_diff1
382
- value: 26.962150235593356
383
- - type: nauc_map_at_1000_max
384
- value: 29.234958037854568
385
- - type: nauc_map_at_1000_std
386
- value: -2.4294465103633884
387
- - type: nauc_map_at_100_diff1
388
- value: 26.92990252114163
389
- - type: nauc_map_at_100_max
390
- value: 29.206328533120118
391
- - type: nauc_map_at_100_std
392
- value: -2.437371090941197
393
- - type: nauc_map_at_10_diff1
394
- value: 25.758265691179226
395
- - type: nauc_map_at_10_max
396
- value: 26.949978490795317
397
- - type: nauc_map_at_10_std
398
- value: -5.484961002106038
399
- - type: nauc_map_at_1_diff1
400
- value: 34.70849461278043
401
- - type: nauc_map_at_1_max
402
- value: 12.778570893623042
403
- - type: nauc_map_at_1_std
404
- value: -13.018292652743938
405
- - type: nauc_map_at_20_diff1
406
- value: 26.659923008218268
407
- - type: nauc_map_at_20_max
408
- value: 28.341440871568185
409
- - type: nauc_map_at_20_std
410
- value: -3.614549844913084
411
- - type: nauc_map_at_3_diff1
412
- value: 27.197629021438203
413
- - type: nauc_map_at_3_max
414
- value: 20.701094874050856
415
- - type: nauc_map_at_3_std
416
- value: -12.062992301112041
417
- - type: nauc_map_at_5_diff1
418
- value: 25.51793537203295
419
- - type: nauc_map_at_5_max
420
- value: 23.80396771243794
421
- - type: nauc_map_at_5_std
422
- value: -8.920465695323575
423
- - type: nauc_mrr_at_1000_diff1
424
- value: 45.14819989592967
425
- - type: nauc_mrr_at_1000_max
426
- value: 53.29202156141053
427
- - type: nauc_mrr_at_1000_std
428
- value: 18.037336462510524
429
- - type: nauc_mrr_at_100_diff1
430
- value: 45.15287600228451
431
- - type: nauc_mrr_at_100_max
432
- value: 53.29979751928615
433
- - type: nauc_mrr_at_100_std
434
- value: 18.04996604778386
435
- - type: nauc_mrr_at_10_diff1
436
- value: 44.96865105944474
437
- - type: nauc_mrr_at_10_max
438
- value: 53.53323465323092
439
- - type: nauc_mrr_at_10_std
440
- value: 18.25001344917689
441
- - type: nauc_mrr_at_1_diff1
442
- value: 46.16604946873163
443
- - type: nauc_mrr_at_1_max
444
- value: 48.573651103547874
445
- - type: nauc_mrr_at_1_std
446
- value: 13.764871626330915
447
- - type: nauc_mrr_at_20_diff1
448
- value: 45.11925458479102
449
- - type: nauc_mrr_at_20_max
450
- value: 53.35685123898342
451
- - type: nauc_mrr_at_20_std
452
- value: 18.127344968819905
453
- - type: nauc_mrr_at_3_diff1
454
- value: 45.377195452730234
455
- - type: nauc_mrr_at_3_max
456
- value: 53.35146309217089
457
- - type: nauc_mrr_at_3_std
458
- value: 17.47105877186237
459
- - type: nauc_mrr_at_5_diff1
460
- value: 45.00525578771549
461
- - type: nauc_mrr_at_5_max
462
- value: 53.76227254707128
463
- - type: nauc_mrr_at_5_std
464
- value: 18.437290060746957
465
- - type: nauc_ndcg_at_1000_diff1
466
- value: 31.19215594457491
467
- - type: nauc_ndcg_at_1000_max
468
- value: 38.09555406458668
469
- - type: nauc_ndcg_at_1000_std
470
- value: 7.225628621238009
471
- - type: nauc_ndcg_at_100_diff1
472
- value: 30.726331247999934
473
- - type: nauc_ndcg_at_100_max
474
- value: 37.81369589418277
475
- - type: nauc_ndcg_at_100_std
476
- value: 7.242855238555071
477
- - type: nauc_ndcg_at_10_diff1
478
- value: 27.514048333744835
479
- - type: nauc_ndcg_at_10_max
480
- value: 33.10990399385253
481
- - type: nauc_ndcg_at_10_std
482
- value: 0.3051899572112002
483
- - type: nauc_ndcg_at_1_diff1
484
- value: 47.06089085235751
485
- - type: nauc_ndcg_at_1_max
486
- value: 47.7300872370495
487
- - type: nauc_ndcg_at_1_std
488
- value: 12.468605493613916
489
- - type: nauc_ndcg_at_20_diff1
490
- value: 29.404215438764496
491
- - type: nauc_ndcg_at_20_max
492
- value: 35.26967886796471
493
- - type: nauc_ndcg_at_20_std
494
- value: 3.7214697890813353
495
- - type: nauc_ndcg_at_3_diff1
496
- value: 29.448848639643067
497
- - type: nauc_ndcg_at_3_max
498
- value: 33.85912412370657
499
- - type: nauc_ndcg_at_3_std
500
- value: 0.895453646819452
501
- - type: nauc_ndcg_at_5_diff1
502
- value: 26.916649012613526
503
- - type: nauc_ndcg_at_5_max
504
- value: 30.899005979291644
505
- - type: nauc_ndcg_at_5_std
506
- value: -1.0001575639156615
507
- - type: nauc_precision_at_1000_diff1
508
- value: -8.492004667432635
509
- - type: nauc_precision_at_1000_max
510
- value: 14.970190384017679
511
- - type: nauc_precision_at_1000_std
512
- value: 32.871386621137816
513
- - type: nauc_precision_at_100_diff1
514
- value: -8.287314133999967
515
- - type: nauc_precision_at_100_max
516
- value: 17.794821961284736
517
- - type: nauc_precision_at_100_std
518
- value: 35.092483550562
519
- - type: nauc_precision_at_10_diff1
520
- value: -7.594128993028063
521
- - type: nauc_precision_at_10_max
522
- value: 24.691446370325732
523
- - type: nauc_precision_at_10_std
524
- value: 30.126552282608493
525
- - type: nauc_precision_at_1_diff1
526
- value: 47.06089085235751
527
- - type: nauc_precision_at_1_max
528
- value: 47.7300872370495
529
- - type: nauc_precision_at_1_std
530
- value: 12.468605493613916
531
- - type: nauc_precision_at_20_diff1
532
- value: -6.503872195775146
533
- - type: nauc_precision_at_20_max
534
- value: 21.789730053141312
535
- - type: nauc_precision_at_20_std
536
- value: 32.61349377558794
537
- - type: nauc_precision_at_3_diff1
538
- value: 0.67417079971061
539
- - type: nauc_precision_at_3_max
540
- value: 30.793871354370662
541
- - type: nauc_precision_at_3_std
542
- value: 18.35266479252011
543
- - type: nauc_precision_at_5_diff1
544
- value: -7.088881730215777
545
- - type: nauc_precision_at_5_max
546
- value: 26.539771712769006
547
- - type: nauc_precision_at_5_std
548
- value: 24.116262291865834
549
- - type: nauc_recall_at_1000_diff1
550
- value: 34.53263588412461
551
- - type: nauc_recall_at_1000_max
552
- value: 63.54157869100173
553
- - type: nauc_recall_at_1000_std
554
- value: 64.19854844792808
555
- - type: nauc_recall_at_100_diff1
556
- value: 22.86564728642275
557
- - type: nauc_recall_at_100_max
558
- value: 40.350507162549825
559
- - type: nauc_recall_at_100_std
560
- value: 29.24492545863015
561
- - type: nauc_recall_at_10_diff1
562
- value: 15.384818367225009
563
- - type: nauc_recall_at_10_max
564
- value: 24.41108571453699
565
- - type: nauc_recall_at_10_std
566
- value: -3.9216160585776323
567
- - type: nauc_recall_at_1_diff1
568
- value: 34.70849461278043
569
- - type: nauc_recall_at_1_max
570
- value: 12.778570893623042
571
- - type: nauc_recall_at_1_std
572
- value: -13.018292652743938
573
- - type: nauc_recall_at_20_diff1
574
- value: 18.122499000084208
575
- - type: nauc_recall_at_20_max
576
- value: 26.63104220179424
577
- - type: nauc_recall_at_20_std
578
- value: 3.969217732521512
579
- - type: nauc_recall_at_3_diff1
580
- value: 21.413050725250116
581
- - type: nauc_recall_at_3_max
582
- value: 16.18894988386887
583
- - type: nauc_recall_at_3_std
584
- value: -15.24884339282375
585
- - type: nauc_recall_at_5_diff1
586
- value: 16.35673072212927
587
- - type: nauc_recall_at_5_max
588
- value: 18.607003829267846
589
- - type: nauc_recall_at_5_std
590
- value: -10.463525876945454
591
- - type: ndcg_at_1
592
- value: 72.923
593
- - type: ndcg_at_10
594
- value: 71.882
595
- - type: ndcg_at_100
596
- value: 77.09899999999999
597
- - type: ndcg_at_1000
598
- value: 77.835
599
- - type: ndcg_at_20
600
- value: 74.497
601
- - type: ndcg_at_3
602
- value: 68.504
603
- - type: ndcg_at_5
604
- value: 69.068
605
- - type: precision_at_1
606
- value: 72.923
607
- - type: precision_at_10
608
- value: 19.936
609
- - type: precision_at_100
610
- value: 2.6310000000000002
611
- - type: precision_at_1000
612
- value: 0.27799999999999997
613
- - type: precision_at_20
614
- value: 11.33
615
- - type: precision_at_3
616
- value: 45.927
617
- - type: precision_at_5
618
- value: 33.131
619
- - type: recall_at_1
620
- value: 37.913000000000004
621
- - type: recall_at_10
622
- value: 78.365
623
- - type: recall_at_100
624
- value: 94.348
625
- - type: recall_at_1000
626
- value: 98.187
627
- - type: recall_at_20
628
- value: 85.229
629
- - type: recall_at_3
630
- value: 61.42999999999999
631
- - type: recall_at_5
632
- value: 69.56700000000001
633
- task:
634
- type: Retrieval
635
- - dataset:
636
- config: ru
637
- name: MTEB MassiveIntentClassification (ru)
638
- revision: 4672e20407010da34463acc759c162ca9734bca6
639
- split: test
640
- type: mteb/amazon_massive_intent
641
- metrics:
642
- - type: accuracy
643
- value: 79.11903160726294
644
- - type: f1
645
- value: 76.22609082694545
646
- - type: f1_weighted
647
- value: 77.81461248063566
648
- - type: main_score
649
- value: 79.11903160726294
650
- task:
651
- type: Classification
652
- - dataset:
653
- config: ru
654
- name: MTEB MassiveScenarioClassification (ru)
655
- revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
656
- split: test
657
- type: mteb/amazon_massive_scenario
658
- metrics:
659
- - type: accuracy
660
- value: 88.80632145258912
661
- - type: f1
662
- value: 87.53157475314829
663
- - type: f1_weighted
664
- value: 88.22733432521495
665
- - type: main_score
666
- value: 88.80632145258912
667
- task:
668
- type: Classification
669
- - dataset:
670
- config: default
671
- name: MTEB RUParaPhraserSTS (default)
672
- revision: 43265056790b8f7c59e0139acb4be0a8dad2c8f4
673
- split: test
674
- type: merionum/ru_paraphraser
675
- metrics:
676
- - type: cosine_pearson
677
- value: 72.70307124858925
678
- - type: cosine_spearman
679
- value: 78.09439086920204
680
- - type: euclidean_pearson
681
- value: 76.2033672014715
682
- - type: euclidean_spearman
683
- value: 78.09439086920204
684
- - type: main_score
685
- value: 78.09439086920204
686
- - type: manhattan_pearson
687
- value: 76.11750470223116
688
- - type: manhattan_spearman
689
- value: 78.01081063503413
690
- - type: pearson
691
- value: 72.70307124858925
692
- - type: spearman
693
- value: 78.09439086920204
694
- task:
695
- type: STS
696
- - dataset:
697
- config: default
698
- name: MTEB RiaNewsRetrieval (default)
699
- revision: 82374b0bbacda6114f39ff9c5b925fa1512ca5d7
700
- split: test
701
- type: ai-forever/ria-news-retrieval
702
- metrics:
703
- - type: main_score
704
- value: 86.819
705
- - type: map_at_1
706
- value: 78.79
707
- - type: map_at_10
708
- value: 84.516
709
- - type: map_at_100
710
- value: 84.68
711
- - type: map_at_1000
712
- value: 84.685
713
- - type: map_at_20
714
- value: 84.624
715
- - type: map_at_3
716
- value: 83.722
717
- - type: map_at_5
718
- value: 84.246
719
- - type: mrr_at_1
720
- value: 78.78
721
- - type: mrr_at_10
722
- value: 84.51815476190441
723
- - type: mrr_at_100
724
- value: 84.68390840473289
725
- - type: mrr_at_1000
726
- value: 84.68947095200002
727
- - type: mrr_at_20
728
- value: 84.62958130822527
729
- - type: mrr_at_3
730
- value: 83.74499999999964
731
- - type: mrr_at_5
732
- value: 84.23849999999955
733
- - type: nauc_map_at_1000_diff1
734
- value: 82.09914867708899
735
- - type: nauc_map_at_1000_max
736
- value: 43.02024854784386
737
- - type: nauc_map_at_1000_std
738
- value: -22.919695880762777
739
- - type: nauc_map_at_100_diff1
740
- value: 82.09705922783733
741
- - type: nauc_map_at_100_max
742
- value: 43.02697379581718
743
- - type: nauc_map_at_100_std
744
- value: -22.90719212899522
745
- - type: nauc_map_at_10_diff1
746
- value: 82.04404594672894
747
- - type: nauc_map_at_10_max
748
- value: 43.06752103182731
749
- - type: nauc_map_at_10_std
750
- value: -23.007870153273576
751
- - type: nauc_map_at_1_diff1
752
- value: 83.89134152210333
753
- - type: nauc_map_at_1_max
754
- value: 38.083626428503415
755
- - type: nauc_map_at_1_std
756
- value: -25.817960401194252
757
- - type: nauc_map_at_20_diff1
758
- value: 82.08534662247806
759
- - type: nauc_map_at_20_max
760
- value: 43.074305042312346
761
- - type: nauc_map_at_20_std
762
- value: -22.91785703613217
763
- - type: nauc_map_at_3_diff1
764
- value: 81.7967508697558
765
- - type: nauc_map_at_3_max
766
- value: 42.90927479098251
767
- - type: nauc_map_at_3_std
768
- value: -24.01312203859392
769
- - type: nauc_map_at_5_diff1
770
- value: 81.90704517505098
771
- - type: nauc_map_at_5_max
772
- value: 43.05204677044616
773
- - type: nauc_map_at_5_std
774
- value: -23.267331507554896
775
- - type: nauc_mrr_at_1000_diff1
776
- value: 82.11902348082472
777
- - type: nauc_mrr_at_1000_max
778
- value: 43.04118936353063
779
- - type: nauc_mrr_at_1000_std
780
- value: -22.858804296830773
781
- - type: nauc_mrr_at_100_diff1
782
- value: 82.11685562002263
783
- - type: nauc_mrr_at_100_max
784
- value: 43.0482537895494
785
- - type: nauc_mrr_at_100_std
786
- value: -22.84431127787993
787
- - type: nauc_mrr_at_10_diff1
788
- value: 82.06909958688058
789
- - type: nauc_mrr_at_10_max
790
- value: 43.07921689466605
791
- - type: nauc_mrr_at_10_std
792
- value: -22.957623576663234
793
- - type: nauc_mrr_at_1_diff1
794
- value: 83.91147637794326
795
- - type: nauc_mrr_at_1_max
796
- value: 37.91917159543152
797
- - type: nauc_mrr_at_1_std
798
- value: -26.141868289283266
799
- - type: nauc_mrr_at_20_diff1
800
- value: 82.10314004731809
801
- - type: nauc_mrr_at_20_max
802
- value: 43.09295406509764
803
- - type: nauc_mrr_at_20_std
804
- value: -22.862091782178787
805
- - type: nauc_mrr_at_3_diff1
806
- value: 81.82117067269036
807
- - type: nauc_mrr_at_3_max
808
- value: 42.94628953323521
809
- - type: nauc_mrr_at_3_std
810
- value: -23.852510312400714
811
- - type: nauc_mrr_at_5_diff1
812
- value: 81.92857441701598
813
- - type: nauc_mrr_at_5_max
814
- value: 43.129719354492934
815
- - type: nauc_mrr_at_5_std
816
- value: -23.145342272624085
817
- - type: nauc_ndcg_at_1000_diff1
818
- value: 81.75015729717991
819
- - type: nauc_ndcg_at_1000_max
820
- value: 44.7266586308995
821
- - type: nauc_ndcg_at_1000_std
822
- value: -20.60663899715267
823
- - type: nauc_ndcg_at_100_diff1
824
- value: 81.6897808298767
825
- - type: nauc_ndcg_at_100_max
826
- value: 44.99492791287099
827
- - type: nauc_ndcg_at_100_std
828
- value: -20.09637266506936
829
- - type: nauc_ndcg_at_10_diff1
830
- value: 81.46290312197337
831
- - type: nauc_ndcg_at_10_max
832
- value: 45.30218378452244
833
- - type: nauc_ndcg_at_10_std
834
- value: -20.70393523891777
835
- - type: nauc_ndcg_at_1_diff1
836
- value: 83.89134152210333
837
- - type: nauc_ndcg_at_1_max
838
- value: 38.083626428503415
839
- - type: nauc_ndcg_at_1_std
840
- value: -25.817960401194252
841
- - type: nauc_ndcg_at_20_diff1
842
- value: 81.61080772657213
843
- - type: nauc_ndcg_at_20_max
844
- value: 45.36571800492172
845
- - type: nauc_ndcg_at_20_std
846
- value: -20.278763852504042
847
- - type: nauc_ndcg_at_3_diff1
848
- value: 80.95965359410461
849
- - type: nauc_ndcg_at_3_max
850
- value: 44.756971949205834
851
- - type: nauc_ndcg_at_3_std
852
- value: -23.07797617717319
853
- - type: nauc_ndcg_at_5_diff1
854
- value: 81.12417712163976
855
- - type: nauc_ndcg_at_5_max
856
- value: 45.15727381406512
857
- - type: nauc_ndcg_at_5_std
858
- value: -21.52861766165519
859
- - type: nauc_precision_at_1000_diff1
860
- value: 76.80566850396093
861
- - type: nauc_precision_at_1000_max
862
- value: 82.45685370922442
863
- - type: nauc_precision_at_1000_std
864
- value: 46.93570976777808
865
- - type: nauc_precision_at_100_diff1
866
- value: 77.21645520953484
867
- - type: nauc_precision_at_100_max
868
- value: 73.43604108309935
869
- - type: nauc_precision_at_100_std
870
- value: 31.978176891671367
871
- - type: nauc_precision_at_10_diff1
872
- value: 77.88251664302092
873
- - type: nauc_precision_at_10_max
874
- value: 60.58112638995018
875
- - type: nauc_precision_at_10_std
876
- value: -3.674424315180332
877
- - type: nauc_precision_at_1_diff1
878
- value: 83.89134152210333
879
- - type: nauc_precision_at_1_max
880
- value: 38.083626428503415
881
- - type: nauc_precision_at_1_std
882
- value: -25.817960401194252
883
- - type: nauc_precision_at_20_diff1
884
- value: 78.16426786697438
885
- - type: nauc_precision_at_20_max
886
- value: 66.0723612699222
887
- - type: nauc_precision_at_20_std
888
- value: 6.121527084555938
889
- - type: nauc_precision_at_3_diff1
890
- value: 77.43122492166451
891
- - type: nauc_precision_at_3_max
892
- value: 52.50727288548085
893
- - type: nauc_precision_at_3_std
894
- value: -19.036076920799427
895
- - type: nauc_precision_at_5_diff1
896
- value: 77.1127254320532
897
- - type: nauc_precision_at_5_max
898
- value: 56.100901899221135
899
- - type: nauc_precision_at_5_std
900
- value: -12.009191140844198
901
- - type: nauc_recall_at_1000_diff1
902
- value: 76.80566850396035
903
- - type: nauc_recall_at_1000_max
904
- value: 82.45685370922577
905
- - type: nauc_recall_at_1000_std
906
- value: 46.93570976777776
907
- - type: nauc_recall_at_100_diff1
908
- value: 77.21645520953459
909
- - type: nauc_recall_at_100_max
910
- value: 73.43604108310011
911
- - type: nauc_recall_at_100_std
912
- value: 31.978176891671993
913
- - type: nauc_recall_at_10_diff1
914
- value: 77.88251664302089
915
- - type: nauc_recall_at_10_max
916
- value: 60.58112638994999
917
- - type: nauc_recall_at_10_std
918
- value: -3.6744243151805427
919
- - type: nauc_recall_at_1_diff1
920
- value: 83.89134152210333
921
- - type: nauc_recall_at_1_max
922
- value: 38.083626428503415
923
- - type: nauc_recall_at_1_std
924
- value: -25.817960401194252
925
- - type: nauc_recall_at_20_diff1
926
- value: 78.16426786697409
927
- - type: nauc_recall_at_20_max
928
- value: 66.07236126992217
929
- - type: nauc_recall_at_20_std
930
- value: 6.121527084555941
931
- - type: nauc_recall_at_3_diff1
932
- value: 77.43122492166454
933
- - type: nauc_recall_at_3_max
934
- value: 52.507272885480816
935
- - type: nauc_recall_at_3_std
936
- value: -19.036076920799776
937
- - type: nauc_recall_at_5_diff1
938
- value: 77.11272543205318
939
- - type: nauc_recall_at_5_max
940
- value: 56.10090189922128
941
- - type: nauc_recall_at_5_std
942
- value: -12.009191140843809
943
- - type: ndcg_at_1
944
- value: 78.79
945
- - type: ndcg_at_10
946
- value: 86.819
947
- - type: ndcg_at_100
948
- value: 87.599
949
- - type: ndcg_at_1000
950
- value: 87.761
951
- - type: ndcg_at_20
952
- value: 87.208
953
- - type: ndcg_at_3
954
- value: 85.222
955
- - type: ndcg_at_5
956
- value: 86.164
957
- - type: precision_at_1
958
- value: 78.79
959
- - type: precision_at_10
960
- value: 9.384
961
- - type: precision_at_100
962
- value: 0.975
963
- - type: precision_at_1000
964
- value: 0.099
965
- - type: precision_at_20
966
- value: 4.769
967
- - type: precision_at_3
968
- value: 29.842999999999996
969
- - type: precision_at_5
970
- value: 18.362000000000002
971
- - type: recall_at_1
972
- value: 78.79
973
- - type: recall_at_10
974
- value: 93.84
975
- - type: recall_at_100
976
- value: 97.45
977
- - type: recall_at_1000
978
- value: 98.76
979
- - type: recall_at_20
980
- value: 95.37
981
- - type: recall_at_3
982
- value: 89.53
983
- - type: recall_at_5
984
- value: 91.81
985
- task:
986
- type: Retrieval
987
- - dataset:
988
- config: default
989
- name: MTEB RuBQReranking (default)
990
- revision: 2e96b8f098fa4b0950fc58eacadeb31c0d0c7fa2
991
- split: test
992
- type: ai-forever/rubq-reranking
993
- metrics:
994
- - type: main_score
995
- value: 77.07394404835635
996
- - type: map
997
- value: 77.07394404835635
998
- - type: mrr
999
- value: 82.53144412718882
1000
- - type: nAUC_map_diff1
1001
- value: 45.29805217456628
1002
- - type: nAUC_map_max
1003
- value: 34.39894042439188
1004
- - type: nAUC_map_std
1005
- value: 21.11309674418275
1006
- - type: nAUC_mrr_diff1
1007
- value: 54.783994737367046
1008
- - type: nAUC_mrr_max
1009
- value: 45.68526733900048
1010
- - type: nAUC_mrr_std
1011
- value: 28.22466385500339
1012
- task:
1013
- type: Reranking
1014
- - dataset:
1015
- config: default
1016
- name: MTEB RuBQRetrieval (default)
1017
- revision: e19b6ffa60b3bc248e0b41f4cc37c26a55c2a67b
1018
- split: test
1019
- type: ai-forever/rubq-retrieval
1020
- metrics:
1021
- - type: main_score
1022
- value: 72.392
1023
- - type: map_at_1
1024
- value: 47.370000000000005
1025
- - type: map_at_10
1026
- value: 65.503
1027
- - type: map_at_100
1028
- value: 66.38
1029
- - type: map_at_1000
1030
- value: 66.42099999999999
1031
- - type: map_at_20
1032
- value: 66.071
1033
- - type: map_at_3
1034
- value: 61.439
1035
- - type: map_at_5
1036
- value: 63.922999999999995
1037
- - type: mrr_at_1
1038
- value: 67.37588652482269
1039
- - type: mrr_at_10
1040
- value: 76.0066747345116
1041
- - type: mrr_at_100
1042
- value: 76.25754138969413
1043
- - type: mrr_at_1000
1044
- value: 76.26968825657428
1045
- - type: mrr_at_20
1046
- value: 76.17548265904622
1047
- - type: mrr_at_3
1048
- value: 74.61583924349881
1049
- - type: mrr_at_5
1050
- value: 75.46690307328608
1051
- - type: nauc_map_at_1000_diff1
1052
- value: 42.52570720187294
1053
- - type: nauc_map_at_1000_max
1054
- value: 37.40318318724238
1055
- - type: nauc_map_at_1000_std
1056
- value: 0.6037788201535506
1057
- - type: nauc_map_at_100_diff1
1058
- value: 42.493410029691226
1059
- - type: nauc_map_at_100_max
1060
- value: 37.39802489244377
1061
- - type: nauc_map_at_100_std
1062
- value: 0.6071359951887154
1063
- - type: nauc_map_at_10_diff1
1064
- value: 42.09833519659916
1065
- - type: nauc_map_at_10_max
1066
- value: 37.1184138958874
1067
- - type: nauc_map_at_10_std
1068
- value: 0.4063543094010351
1069
- - type: nauc_map_at_1_diff1
1070
- value: 49.56605205141156
1071
- - type: nauc_map_at_1_max
1072
- value: 26.251096698710384
1073
- - type: nauc_map_at_1_std
1074
- value: -4.580748485387834
1075
- - type: nauc_map_at_20_diff1
1076
- value: 42.33372393482018
1077
- - type: nauc_map_at_20_max
1078
- value: 37.416955604649985
1079
- - type: nauc_map_at_20_std
1080
- value: 0.6050577802787294
1081
- - type: nauc_map_at_3_diff1
1082
- value: 42.362234475441845
1083
- - type: nauc_map_at_3_max
1084
- value: 34.56001379838821
1085
- - type: nauc_map_at_3_std
1086
- value: -1.507636598929042
1087
- - type: nauc_map_at_5_diff1
1088
- value: 42.0202264882535
1089
- - type: nauc_map_at_5_max
1090
- value: 36.64306050200848
1091
- - type: nauc_map_at_5_std
1092
- value: -0.09509025708798424
1093
- - type: nauc_mrr_at_1000_diff1
1094
- value: 58.99601742026931
1095
- - type: nauc_mrr_at_1000_max
1096
- value: 49.61561872452777
1097
- - type: nauc_mrr_at_1000_std
1098
- value: 2.3956102974352356
1099
- - type: nauc_mrr_at_100_diff1
1100
- value: 58.9865943101085
1101
- - type: nauc_mrr_at_100_max
1102
- value: 49.6248111507265
1103
- - type: nauc_mrr_at_100_std
1104
- value: 2.411155095066369
1105
- - type: nauc_mrr_at_10_diff1
1106
- value: 58.81758131092919
1107
- - type: nauc_mrr_at_10_max
1108
- value: 49.780365572616695
1109
- - type: nauc_mrr_at_10_std
1110
- value: 2.7068696565195944
1111
- - type: nauc_mrr_at_1_diff1
1112
- value: 61.67036882487055
1113
- - type: nauc_mrr_at_1_max
1114
- value: 45.455271042821714
1115
- - type: nauc_mrr_at_1_std
1116
- value: -0.9370526815458349
1117
- - type: nauc_mrr_at_20_diff1
1118
- value: 58.93674818203478
1119
- - type: nauc_mrr_at_20_max
1120
- value: 49.703218108625215
1121
- - type: nauc_mrr_at_20_std
1122
- value: 2.4473106598190415
1123
- - type: nauc_mrr_at_3_diff1
1124
- value: 59.046856598788445
1125
- - type: nauc_mrr_at_3_max
1126
- value: 49.37161726123392
1127
- - type: nauc_mrr_at_3_std
1128
- value: 1.5110936686701506
1129
- - type: nauc_mrr_at_5_diff1
1130
- value: 58.92289378915668
1131
- - type: nauc_mrr_at_5_max
1132
- value: 49.847638994134144
1133
- - type: nauc_mrr_at_5_std
1134
- value: 2.420421880131702
1135
- - type: nauc_ndcg_at_1000_diff1
1136
- value: 45.56062215161734
1137
- - type: nauc_ndcg_at_1000_max
1138
- value: 41.507152286702
1139
- - type: nauc_ndcg_at_1000_std
1140
- value: 2.79388283208751
1141
- - type: nauc_ndcg_at_100_diff1
1142
- value: 44.84064192570408
1143
- - type: nauc_ndcg_at_100_max
1144
- value: 41.50353573562353
1145
- - type: nauc_ndcg_at_100_std
1146
- value: 3.1804999773629357
1147
- - type: nauc_ndcg_at_10_diff1
1148
- value: 43.341482144213614
1149
- - type: nauc_ndcg_at_10_max
1150
- value: 41.159590898395074
1151
- - type: nauc_ndcg_at_10_std
1152
- value: 2.945242338240843
1153
- - type: nauc_ndcg_at_1_diff1
1154
- value: 62.23623985611396
1155
- - type: nauc_ndcg_at_1_max
1156
- value: 45.04945770947091
1157
- - type: nauc_ndcg_at_1_std
1158
- value: -0.8804967656575725
1159
- - type: nauc_ndcg_at_20_diff1
1160
- value: 43.905372612093664
1161
- - type: nauc_ndcg_at_20_max
1162
- value: 41.797709837872446
1163
- - type: nauc_ndcg_at_20_std
1164
- value: 3.1853356915569653
1165
- - type: nauc_ndcg_at_3_diff1
1166
- value: 44.18163998834299
1167
- - type: nauc_ndcg_at_3_max
1168
- value: 38.352891017864636
1169
- - type: nauc_ndcg_at_3_std
1170
- value: -0.8235767021150929
1171
- - type: nauc_ndcg_at_5_diff1
1172
- value: 43.41374688421302
1173
- - type: nauc_ndcg_at_5_max
1174
- value: 40.390365601593956
1175
- - type: nauc_ndcg_at_5_std
1176
- value: 1.6743650108127537
1177
- - type: nauc_precision_at_1000_diff1
1178
- value: -9.711058370691381
1179
- - type: nauc_precision_at_1000_max
1180
- value: 6.97321343449286
1181
- - type: nauc_precision_at_1000_std
1182
- value: 7.933531916622121
1183
- - type: nauc_precision_at_100_diff1
1184
- value: -8.247029644152319
1185
- - type: nauc_precision_at_100_max
1186
- value: 10.86740140944616
1187
- - type: nauc_precision_at_100_std
1188
- value: 9.581885544675918
1189
- - type: nauc_precision_at_10_diff1
1190
- value: -2.409043695429943
1191
- - type: nauc_precision_at_10_max
1192
- value: 21.04733206074314
1193
- - type: nauc_precision_at_10_std
1194
- value: 10.03334651647101
1195
- - type: nauc_precision_at_1_diff1
1196
- value: 62.23623985611396
1197
- - type: nauc_precision_at_1_max
1198
- value: 45.04945770947091
1199
- - type: nauc_precision_at_1_std
1200
- value: -0.8804967656575725
1201
- - type: nauc_precision_at_20_diff1
1202
- value: -5.230303656931621
1203
- - type: nauc_precision_at_20_max
1204
- value: 17.77799716919181
1205
- - type: nauc_precision_at_20_std
1206
- value: 10.739127998618654
1207
- - type: nauc_precision_at_3_diff1
1208
- value: 10.40376424999862
1209
- - type: nauc_precision_at_3_max
1210
- value: 30.933333400254035
1211
- - type: nauc_precision_at_3_std
1212
- value: 6.126209127968004
1213
- - type: nauc_precision_at_5_diff1
1214
- value: 3.147398101830739
1215
- - type: nauc_precision_at_5_max
1216
- value: 27.1746309955971
1217
- - type: nauc_precision_at_5_std
1218
- value: 8.874723615388788
1219
- - type: nauc_recall_at_1000_diff1
1220
- value: 5.055940692380908
1221
- - type: nauc_recall_at_1000_max
1222
- value: 22.42031123370267
1223
- - type: nauc_recall_at_1000_std
1224
- value: 27.75476692527869
1225
- - type: nauc_recall_at_100_diff1
1226
- value: 17.86391178198642
1227
- - type: nauc_recall_at_100_max
1228
- value: 34.776134863678955
1229
- - type: nauc_recall_at_100_std
1230
- value: 18.96377158778504
1231
- - type: nauc_recall_at_10_diff1
1232
- value: 24.863097695413597
1233
- - type: nauc_recall_at_10_max
1234
- value: 37.697411651507444
1235
- - type: nauc_recall_at_10_std
1236
- value: 9.519849994253967
1237
- - type: nauc_recall_at_1_diff1
1238
- value: 49.56605205141156
1239
- - type: nauc_recall_at_1_max
1240
- value: 26.251096698710384
1241
- - type: nauc_recall_at_1_std
1242
- value: -4.580748485387834
1243
- - type: nauc_recall_at_20_diff1
1244
- value: 22.440602811005636
1245
- - type: nauc_recall_at_20_max
1246
- value: 39.538861316515
1247
- - type: nauc_recall_at_20_std
1248
- value: 11.363269553121468
1249
- - type: nauc_recall_at_3_diff1
1250
- value: 32.80302839873736
1251
- - type: nauc_recall_at_3_max
1252
- value: 32.53105685012729
1253
- - type: nauc_recall_at_3_std
1254
- value: -0.7140166410605693
1255
- - type: nauc_recall_at_5_diff1
1256
- value: 29.375386639154865
1257
- - type: nauc_recall_at_5_max
1258
- value: 36.91045781164083
1259
- - type: nauc_recall_at_5_std
1260
- value: 4.725419050262578
1261
- - type: ndcg_at_1
1262
- value: 67.13900000000001
1263
- - type: ndcg_at_10
1264
- value: 72.392
1265
- - type: ndcg_at_100
1266
- value: 75.25800000000001
1267
- - type: ndcg_at_1000
1268
- value: 75.982
1269
- - type: ndcg_at_20
1270
- value: 73.783
1271
- - type: ndcg_at_3
1272
- value: 67.269
1273
- - type: ndcg_at_5
1274
- value: 69.807
1275
- - type: precision_at_1
1276
- value: 67.13900000000001
1277
- - type: precision_at_10
1278
- value: 13.327
1279
- - type: precision_at_100
1280
- value: 1.5559999999999998
1281
- - type: precision_at_1000
1282
- value: 0.164
1283
- - type: precision_at_20
1284
- value: 7.119000000000001
1285
- - type: precision_at_3
1286
- value: 35.599
1287
- - type: precision_at_5
1288
- value: 23.936
1289
- - type: recall_at_1
1290
- value: 47.370000000000005
1291
- - type: recall_at_10
1292
- value: 82.16
1293
- - type: recall_at_100
1294
- value: 93.34
1295
- - type: recall_at_1000
1296
- value: 98.202
1297
- - type: recall_at_20
1298
- value: 86.687
1299
- - type: recall_at_3
1300
- value: 69.319
1301
- - type: recall_at_5
1302
- value: 75.637
1303
- task:
1304
- type: Retrieval
1305
- - dataset:
1306
- config: default
1307
- name: MTEB RuReviewsClassification (default)
1308
- revision: f6d2c31f4dc6b88f468552750bfec05b4b41b05a
1309
- split: test
1310
- type: ai-forever/ru-reviews-classification
1311
- metrics:
1312
- - type: accuracy
1313
- value: 75.0537109375
1314
- - type: f1
1315
- value: 74.00523205209554
1316
- - type: f1_weighted
1317
- value: 74.00436782840376
1318
- - type: main_score
1319
- value: 75.0537109375
1320
- task:
1321
- type: Classification
1322
- - dataset:
1323
- config: default
1324
- name: MTEB RuSTSBenchmarkSTS (default)
1325
- revision: 7cf24f325c6da6195df55bef3d86b5e0616f3018
1326
- split: test
1327
- type: ai-forever/ru-stsbenchmark-sts
1328
- metrics:
1329
- - type: cosine_pearson
1330
- value: 81.10255413476487
1331
- - type: cosine_spearman
1332
- value: 81.40020843157141
1333
- - type: euclidean_pearson
1334
- value: 81.25155479902466
1335
- - type: euclidean_spearman
1336
- value: 81.40020831064922
1337
- - type: main_score
1338
- value: 81.40020843157141
1339
- - type: manhattan_pearson
1340
- value: 81.1493715249014
1341
- - type: manhattan_spearman
1342
- value: 81.30973667941649
1343
- - type: pearson
1344
- value: 81.10255413476487
1345
- - type: spearman
1346
- value: 81.40020843157141
1347
- task:
1348
- type: STS
1349
- - dataset:
1350
- config: default
1351
- name: MTEB RuSciBenchGRNTIClassification (default)
1352
- revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
1353
- split: test
1354
- type: ai-forever/ru-scibench-grnti-classification
1355
- metrics:
1356
- - type: accuracy
1357
- value: 69.8974609375
1358
- - type: f1
1359
- value: 68.57837564785511
1360
- - type: f1_weighted
1361
- value: 68.59030489460784
1362
- - type: main_score
1363
- value: 69.8974609375
1364
- task:
1365
- type: Classification
1366
- - dataset:
1367
- config: default
1368
- name: MTEB RuSciBenchGRNTIClusteringP2P (default)
1369
- revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
1370
- split: test
1371
- type: ai-forever/ru-scibench-grnti-classification
1372
- metrics:
1373
- - type: main_score
1374
- value: 67.03880348548029
1375
- - type: v_measure
1376
- value: 67.03880348548029
1377
- - type: v_measure_std
1378
- value: 0.6126278133139618
1379
- task:
1380
- type: Clustering
1381
- - dataset:
1382
- config: default
1383
- name: MTEB RuSciBenchOECDClassification (default)
1384
- revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
1385
- split: test
1386
- type: ai-forever/ru-scibench-oecd-classification
1387
- metrics:
1388
- - type: accuracy
1389
- value: 54.63378906250001
1390
- - type: f1
1391
- value: 51.34306420274629
1392
- - type: f1_weighted
1393
- value: 51.33495867493914
1394
- - type: main_score
1395
- value: 54.63378906250001
1396
- task:
1397
- type: Classification
1398
- - dataset:
1399
- config: default
1400
- name: MTEB RuSciBenchOECDClusteringP2P (default)
1401
- revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
1402
- split: test
1403
- type: ai-forever/ru-scibench-oecd-classification
1404
- metrics:
1405
- - type: main_score
1406
- value: 56.55947121159027
1407
- - type: v_measure
1408
- value: 56.55947121159027
1409
- - type: v_measure_std
1410
- value: 0.5498882006880662
1411
- task:
1412
- type: Clustering
1413
- - dataset:
1414
- config: ru
1415
- name: MTEB STS22 (ru)
1416
- revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
1417
- split: test
1418
- type: mteb/sts22-crosslingual-sts
1419
- metrics:
1420
- - type: cosine_pearson
1421
- value: 61.833294921667914
1422
- - type: cosine_spearman
1423
- value: 63.53967536726357
1424
- - type: euclidean_pearson
1425
- value: 60.382865218855805
1426
- - type: euclidean_spearman
1427
- value: 63.53967536726357
1428
- - type: main_score
1429
- value: 63.53967536726357
1430
- - type: manhattan_pearson
1431
- value: 60.24879015304578
1432
- - type: manhattan_spearman
1433
- value: 63.42305760430092
1434
- - type: pearson
1435
- value: 61.833294921667914
1436
- - type: spearman
1437
- value: 63.53967536726357
1438
- task:
1439
- type: STS
1440
- - dataset:
1441
- config: default
1442
- name: MTEB SensitiveTopicsClassification (default)
1443
- revision: 416b34a802308eac30e4192afc0ff99bb8dcc7f2
1444
- split: test
1445
- type: ai-forever/sensitive-topics-classification
1446
- metrics:
1447
- - type: accuracy
1448
- value: 39.8193359375
1449
- - type: f1
1450
- value: 55.46591740935434
1451
- - type: lrap
1452
- value: 66.50980631510454
1453
- - type: main_score
1454
- value: 39.8193359375
1455
- task:
1456
- type: MultilabelClassification
1457
- - dataset:
1458
- config: default
1459
- name: MTEB TERRa (default)
1460
- revision: 7b58f24536063837d644aab9a023c62199b2a612
1461
- split: dev
1462
- type: ai-forever/terra-pairclassification
1463
- metrics:
1464
- - type: cosine_accuracy
1465
- value: 66.77524429967427
1466
- - type: cosine_accuracy_threshold
1467
- value: 55.58975338935852
1468
- - type: cosine_ap
1469
- value: 66.4567219323658
1470
- - type: cosine_f1
1471
- value: 70.64676616915423
1472
- - type: cosine_f1_threshold
1473
- value: 45.55969536304474
1474
- - type: cosine_precision
1475
- value: 57.028112449799195
1476
- - type: cosine_recall
1477
- value: 92.81045751633987
1478
- - type: dot_accuracy
1479
- value: 66.77524429967427
1480
- - type: dot_accuracy_threshold
1481
- value: 55.589759349823
1482
- - type: dot_ap
1483
- value: 66.4567219323658
1484
- - type: dot_f1
1485
- value: 70.64676616915423
1486
- - type: dot_f1_threshold
1487
- value: 45.55969536304474
1488
- - type: dot_precision
1489
- value: 57.028112449799195
1490
- - type: dot_recall
1491
- value: 92.81045751633987
1492
- - type: euclidean_accuracy
1493
- value: 66.77524429967427
1494
- - type: euclidean_accuracy_threshold
1495
- value: 94.24455165863037
1496
- - type: euclidean_ap
1497
- value: 66.4567219323658
1498
- - type: euclidean_f1
1499
- value: 70.64676616915423
1500
- - type: euclidean_f1_threshold
1501
- value: 104.34587001800537
1502
- - type: euclidean_precision
1503
- value: 57.028112449799195
1504
- - type: euclidean_recall
1505
- value: 92.81045751633987
1506
- - type: main_score
1507
- value: 66.4567219323658
1508
- - type: manhattan_accuracy
1509
- value: 66.77524429967427
1510
- - type: manhattan_accuracy_threshold
1511
- value: 2865.5345916748047
1512
- - type: manhattan_ap
1513
- value: 66.26659863769075
1514
- - type: manhattan_f1
1515
- value: 70.8542713567839
1516
- - type: manhattan_f1_threshold
1517
- value: 3212.3912811279297
1518
- - type: manhattan_precision
1519
- value: 57.55102040816327
1520
- - type: manhattan_recall
1521
- value: 92.15686274509804
1522
- - type: max_accuracy
1523
- value: 66.77524429967427
1524
- - type: max_ap
1525
- value: 66.4567219323658
1526
- - type: max_f1
1527
- value: 70.8542713567839
1528
- - type: max_precision
1529
- value: 57.55102040816327
1530
- - type: max_recall
1531
- value: 92.81045751633987
1532
- - type: similarity_accuracy
1533
- value: 66.77524429967427
1534
- - type: similarity_accuracy_threshold
1535
- value: 55.58975338935852
1536
- - type: similarity_ap
1537
- value: 66.4567219323658
1538
- - type: similarity_f1
1539
- value: 70.64676616915423
1540
- - type: similarity_f1_threshold
1541
- value: 45.55969536304474
1542
- - type: similarity_precision
1543
- value: 57.028112449799195
1544
- - type: similarity_recall
1545
- value: 92.81045751633987
1546
- task:
1547
- type: PairClassification
1548
- license: mit
1549
- language:
1550
- - ru
1551
- - en
1552
- tags:
1553
- - mteb
1554
- - transformers
1555
- - sentence-transformers
1556
- base_model: ai-forever/FRED-T5-1.7B
1557
- pipeline_tag: feature-extraction
1558
- ---
 
 
1559
 
1560
  # Model Card for FRIDA
1561
 
@@ -1565,7 +1567,7 @@ pipeline_tag: feature-extraction
1565
 
1566
  FRIDA is a full-scale finetuned general text embedding model inspired by denoising architecture based on T5. The model is based on the encoder part of [FRED-T5](https://arxiv.org/abs/2309.10931) model and continues research of text embedding models ([ruMTEB](https://arxiv.org/abs/2408.12503), [ru-en-RoSBERTa](https://huggingface.co/ai-forever/ru-en-RoSBERTa)). It has been pre-trained on a Russian-English dataset and fine-tuned for improved performance on the target task.
1567
 
1568
- For more model details please refer to our technical report [TODO].
1569
 
1570
  ## Usage
1571
 
 
1
+ ---
2
+ model-index:
3
+ - name: FRIDA
4
+ results:
5
+ - dataset:
6
+ config: default
7
+ name: MTEB CEDRClassification (default)
8
+ revision: c0ba03d058e3e1b2f3fd20518875a4563dd12db4
9
+ split: test
10
+ type: ai-forever/cedr-classification
11
+ metrics:
12
+ - type: accuracy
13
+ value: 64.60148777895856
14
+ - type: f1
15
+ value: 70.36630348039266
16
+ - type: lrap
17
+ value: 92.47290116896953
18
+ - type: main_score
19
+ value: 64.60148777895856
20
+ task:
21
+ type: MultilabelClassification
22
+ - dataset:
23
+ config: default
24
+ name: MTEB GeoreviewClassification (default)
25
+ revision: 3765c0d1de6b7d264bc459433c45e5a75513839c
26
+ split: test
27
+ type: ai-forever/georeview-classification
28
+ metrics:
29
+ - type: accuracy
30
+ value: 57.70996093750001
31
+ - type: f1
32
+ value: 53.18542982057098
33
+ - type: f1_weighted
34
+ value: 53.17663229582108
35
+ - type: main_score
36
+ value: 57.70996093750001
37
+ task:
38
+ type: Classification
39
+ - dataset:
40
+ config: default
41
+ name: MTEB GeoreviewClusteringP2P (default)
42
+ revision: 97a313c8fc85b47f13f33e7e9a95c1ad888c7fec
43
+ split: test
44
+ type: ai-forever/georeview-clustering-p2p
45
+ metrics:
46
+ - type: main_score
47
+ value: 78.25468393043356
48
+ - type: v_measure
49
+ value: 78.25468393043356
50
+ - type: v_measure_std
51
+ value: 0.5094366871364238
52
+ task:
53
+ type: Clustering
54
+ - dataset:
55
+ config: default
56
+ name: MTEB HeadlineClassification (default)
57
+ revision: 2fe05ee6b5832cda29f2ef7aaad7b7fe6a3609eb
58
+ split: test
59
+ type: ai-forever/headline-classification
60
+ metrics:
61
+ - type: accuracy
62
+ value: 89.0185546875
63
+ - type: f1
64
+ value: 88.993933120612
65
+ - type: f1_weighted
66
+ value: 88.99276764225768
67
+ - type: main_score
68
+ value: 89.0185546875
69
+ task:
70
+ type: Classification
71
+ - dataset:
72
+ config: default
73
+ name: MTEB InappropriatenessClassification (default)
74
+ revision: 601651fdc45ef243751676e62dd7a19f491c0285
75
+ split: test
76
+ type: ai-forever/inappropriateness-classification
77
+ metrics:
78
+ - type: accuracy
79
+ value: 78.330078125
80
+ - type: ap
81
+ value: 73.17856750532495
82
+ - type: ap_weighted
83
+ value: 73.17856750532495
84
+ - type: f1
85
+ value: 78.20169867599041
86
+ - type: f1_weighted
87
+ value: 78.20169867599041
88
+ - type: main_score
89
+ value: 78.330078125
90
+ task:
91
+ type: Classification
92
+ - dataset:
93
+ config: default
94
+ name: MTEB KinopoiskClassification (default)
95
+ revision: 5911f26666ac11af46cb9c6849d0dc80a378af24
96
+ split: test
97
+ type: ai-forever/kinopoisk-sentiment-classification
98
+ metrics:
99
+ - type: accuracy
100
+ value: 70.46666666666665
101
+ - type: f1
102
+ value: 65.83951766538878
103
+ - type: f1_weighted
104
+ value: 65.83951766538878
105
+ - type: main_score
106
+ value: 70.46666666666665
107
+ task:
108
+ type: Classification
109
+ - dataset:
110
+ config: ru
111
+ name: MTEB MIRACLReranking (ru)
112
+ revision: 6d1962c527217f8927fca80f890f14f36b2802af
113
+ split: dev
114
+ type: miracl/mmteb-miracl-reranking
115
+ metrics:
116
+ - type: MAP@1(MIRACL)
117
+ value: 39.023
118
+ - type: MAP@10(MIRACL)
119
+ value: 60.208
120
+ - type: MAP@100(MIRACL)
121
+ value: 61.672000000000004
122
+ - type: MAP@1000(MIRACL)
123
+ value: 61.672000000000004
124
+ - type: MAP@20(MIRACL)
125
+ value: 61.30799999999999
126
+ - type: MAP@3(MIRACL)
127
+ value: 53.33
128
+ - type: MAP@5(MIRACL)
129
+ value: 57.289
130
+ - type: NDCG@1(MIRACL)
131
+ value: 63.352
132
+ - type: NDCG@10(MIRACL)
133
+ value: 66.042
134
+ - type: NDCG@100(MIRACL)
135
+ value: 68.702
136
+ - type: NDCG@1000(MIRACL)
137
+ value: 68.702
138
+ - type: NDCG@20(MIRACL)
139
+ value: 67.768
140
+ - type: NDCG@3(MIRACL)
141
+ value: 61.925
142
+ - type: NDCG@5(MIRACL)
143
+ value: 63.327
144
+ - type: P@1(MIRACL)
145
+ value: 63.352
146
+ - type: P@10(MIRACL)
147
+ value: 16.512
148
+ - type: P@100(MIRACL)
149
+ value: 1.9529999999999998
150
+ - type: P@1000(MIRACL)
151
+ value: 0.19499999999999998
152
+ - type: P@20(MIRACL)
153
+ value: 9.13
154
+ - type: P@3(MIRACL)
155
+ value: 37.878
156
+ - type: P@5(MIRACL)
157
+ value: 27.586
158
+ - type: Recall@1(MIRACL)
159
+ value: 39.023
160
+ - type: Recall@10(MIRACL)
161
+ value: 72.35000000000001
162
+ - type: Recall@100(MIRACL)
163
+ value: 79.952
164
+ - type: Recall@1000(MIRACL)
165
+ value: 79.952
166
+ - type: Recall@20(MIRACL)
167
+ value: 76.828
168
+ - type: Recall@3(MIRACL)
169
+ value: 57.769999999999996
170
+ - type: Recall@5(MIRACL)
171
+ value: 64.91900000000001
172
+ - type: main_score
173
+ value: 66.042
174
+ - type: nAUC_MAP@1000_diff1(MIRACL)
175
+ value: 27.150388833033052
176
+ - type: nAUC_MAP@1000_max(MIRACL)
177
+ value: 55.15672274267081
178
+ - type: nAUC_MAP@1000_std(MIRACL)
179
+ value: 30.088939934575553
180
+ - type: nAUC_MAP@100_diff1(MIRACL)
181
+ value: 27.150388833033052
182
+ - type: nAUC_MAP@100_max(MIRACL)
183
+ value: 55.15672274267081
184
+ - type: nAUC_MAP@100_std(MIRACL)
185
+ value: 30.088939934575553
186
+ - type: nAUC_MAP@10_diff1(MIRACL)
187
+ value: 27.853691773641742
188
+ - type: nAUC_MAP@10_max(MIRACL)
189
+ value: 52.89390350055654
190
+ - type: nAUC_MAP@10_std(MIRACL)
191
+ value: 28.08732516551691
192
+ - type: nAUC_MAP@1_diff1(MIRACL)
193
+ value: 43.23179150244192
194
+ - type: nAUC_MAP@1_max(MIRACL)
195
+ value: 29.923943954188864
196
+ - type: nAUC_MAP@1_std(MIRACL)
197
+ value: 7.447084370195121
198
+ - type: nAUC_MAP@20_diff1(MIRACL)
199
+ value: 27.328384072311675
200
+ - type: nAUC_MAP@20_max(MIRACL)
201
+ value: 54.60286379835721
202
+ - type: nAUC_MAP@20_std(MIRACL)
203
+ value: 29.8084128980043
204
+ - type: nAUC_MAP@3_diff1(MIRACL)
205
+ value: 31.244971536944554
206
+ - type: nAUC_MAP@3_max(MIRACL)
207
+ value: 43.63984692803854
208
+ - type: nAUC_MAP@3_std(MIRACL)
209
+ value: 18.609234683765887
210
+ - type: nAUC_MAP@5_diff1(MIRACL)
211
+ value: 29.088760492638286
212
+ - type: nAUC_MAP@5_max(MIRACL)
213
+ value: 48.30474364461509
214
+ - type: nAUC_MAP@5_std(MIRACL)
215
+ value: 23.817514353844224
216
+ - type: nAUC_NDCG@1000_diff1(MIRACL)
217
+ value: 23.12754356408408
218
+ - type: nAUC_NDCG@1000_max(MIRACL)
219
+ value: 64.24894553363303
220
+ - type: nAUC_NDCG@1000_std(MIRACL)
221
+ value: 38.19318050598967
222
+ - type: nAUC_NDCG@100_diff1(MIRACL)
223
+ value: 23.12754356408408
224
+ - type: nAUC_NDCG@100_max(MIRACL)
225
+ value: 64.24894553363303
226
+ - type: nAUC_NDCG@100_std(MIRACL)
227
+ value: 38.19318050598967
228
+ - type: nAUC_NDCG@10_diff1(MIRACL)
229
+ value: 24.779856373697275
230
+ - type: nAUC_NDCG@10_max(MIRACL)
231
+ value: 60.4054459738118
232
+ - type: nAUC_NDCG@10_std(MIRACL)
233
+ value: 35.148950441182784
234
+ - type: nAUC_NDCG@1_diff1(MIRACL)
235
+ value: 35.605865569438556
236
+ - type: nAUC_NDCG@1_max(MIRACL)
237
+ value: 65.77787399715454
238
+ - type: nAUC_NDCG@1_std(MIRACL)
239
+ value: 34.34726892885082
240
+ - type: nAUC_NDCG@20_diff1(MIRACL)
241
+ value: 23.71231783125691
242
+ - type: nAUC_NDCG@20_max(MIRACL)
243
+ value: 62.89676599488004
244
+ - type: nAUC_NDCG@20_std(MIRACL)
245
+ value: 37.697052941884316
246
+ - type: nAUC_NDCG@3_diff1(MIRACL)
247
+ value: 26.109027741640865
248
+ - type: nAUC_NDCG@3_max(MIRACL)
249
+ value: 56.22356793638693
250
+ - type: nAUC_NDCG@3_std(MIRACL)
251
+ value: 29.9437568508688
252
+ - type: nAUC_NDCG@5_diff1(MIRACL)
253
+ value: 25.98644715327336
254
+ - type: nAUC_NDCG@5_max(MIRACL)
255
+ value: 56.25032008404774
256
+ - type: nAUC_NDCG@5_std(MIRACL)
257
+ value: 31.581899860862578
258
+ - type: nAUC_P@1000_diff1(MIRACL)
259
+ value: -18.29912787064644
260
+ - type: nAUC_P@1000_max(MIRACL)
261
+ value: 31.811344878776087
262
+ - type: nAUC_P@1000_std(MIRACL)
263
+ value: 30.163820183304914
264
+ - type: nAUC_P@100_diff1(MIRACL)
265
+ value: -18.299127870646405
266
+ - type: nAUC_P@100_max(MIRACL)
267
+ value: 31.811344878776133
268
+ - type: nAUC_P@100_std(MIRACL)
269
+ value: 30.163820183304956
270
+ - type: nAUC_P@10_diff1(MIRACL)
271
+ value: -15.96416268531149
272
+ - type: nAUC_P@10_max(MIRACL)
273
+ value: 36.989578896466526
274
+ - type: nAUC_P@10_std(MIRACL)
275
+ value: 34.54507111688143
276
+ - type: nAUC_P@1_diff1(MIRACL)
277
+ value: 35.605865569438556
278
+ - type: nAUC_P@1_max(MIRACL)
279
+ value: 65.77787399715454
280
+ - type: nAUC_P@1_std(MIRACL)
281
+ value: 34.34726892885082
282
+ - type: nAUC_P@20_diff1(MIRACL)
283
+ value: -17.443963421383287
284
+ - type: nAUC_P@20_max(MIRACL)
285
+ value: 34.309618168778385
286
+ - type: nAUC_P@20_std(MIRACL)
287
+ value: 33.38820956485373
288
+ - type: nAUC_P@3_diff1(MIRACL)
289
+ value: -8.533621861815652
290
+ - type: nAUC_P@3_max(MIRACL)
291
+ value: 45.90408386776497
292
+ - type: nAUC_P@3_std(MIRACL)
293
+ value: 34.50459351305535
294
+ - type: nAUC_P@5_diff1(MIRACL)
295
+ value: -13.207968899314865
296
+ - type: nAUC_P@5_max(MIRACL)
297
+ value: 40.37718282248973
298
+ - type: nAUC_P@5_std(MIRACL)
299
+ value: 35.601417332196206
300
+ - type: nAUC_Recall@1000_diff1(MIRACL)
301
+ value: 7.907304198177226
302
+ - type: nAUC_Recall@1000_max(MIRACL)
303
+ value: 77.82197832361145
304
+ - type: nAUC_Recall@1000_std(MIRACL)
305
+ value: 52.66957487246724
306
+ - type: nAUC_Recall@100_diff1(MIRACL)
307
+ value: 7.907304198177226
308
+ - type: nAUC_Recall@100_max(MIRACL)
309
+ value: 77.82197832361145
310
+ - type: nAUC_Recall@100_std(MIRACL)
311
+ value: 52.66957487246724
312
+ - type: nAUC_Recall@10_diff1(MIRACL)
313
+ value: 15.498121023488693
314
+ - type: nAUC_Recall@10_max(MIRACL)
315
+ value: 62.24320529338724
316
+ - type: nAUC_Recall@10_std(MIRACL)
317
+ value: 40.60221460946224
318
+ - type: nAUC_Recall@1_diff1(MIRACL)
319
+ value: 43.23179150244192
320
+ - type: nAUC_Recall@1_max(MIRACL)
321
+ value: 29.923943954188864
322
+ - type: nAUC_Recall@1_std(MIRACL)
323
+ value: 7.447084370195121
324
+ - type: nAUC_Recall@20_diff1(MIRACL)
325
+ value: 11.457044176116248
326
+ - type: nAUC_Recall@20_max(MIRACL)
327
+ value: 70.3493054342368
328
+ - type: nAUC_Recall@20_std(MIRACL)
329
+ value: 49.27124296325928
330
+ - type: nAUC_Recall@3_diff1(MIRACL)
331
+ value: 25.12077828977941
332
+ - type: nAUC_Recall@3_max(MIRACL)
333
+ value: 42.903379317937166
334
+ - type: nAUC_Recall@3_std(MIRACL)
335
+ value: 20.324501722161497
336
+ - type: nAUC_Recall@5_diff1(MIRACL)
337
+ value: 20.925701235197977
338
+ - type: nAUC_Recall@5_max(MIRACL)
339
+ value: 49.85323960390812
340
+ - type: nAUC_Recall@5_std(MIRACL)
341
+ value: 29.04484539530469
342
+ task:
343
+ type: Reranking
344
+ - dataset:
345
+ config: ru
346
+ name: MTEB MIRACLRetrieval (ru)
347
+ revision: main
348
+ split: dev
349
+ type: miracl/mmteb-miracl
350
+ metrics:
351
+ - type: main_score
352
+ value: 71.882
353
+ - type: map_at_1
354
+ value: 37.913000000000004
355
+ - type: map_at_10
356
+ value: 62.604000000000006
357
+ - type: map_at_100
358
+ value: 64.925
359
+ - type: map_at_1000
360
+ value: 64.992
361
+ - type: map_at_20
362
+ value: 64.081
363
+ - type: map_at_3
364
+ value: 55.212
365
+ - type: map_at_5
366
+ value: 59.445
367
+ - type: mrr_at_1
368
+ value: 73.24281150159744
369
+ - type: mrr_at_10
370
+ value: 81.65043866321825
371
+ - type: mrr_at_100
372
+ value: 81.85391378818977
373
+ - type: mrr_at_1000
374
+ value: 81.85753390802569
375
+ - type: mrr_at_20
376
+ value: 81.81045606130179
377
+ - type: mrr_at_3
378
+ value: 80.56443024494146
379
+ - type: mrr_at_5
380
+ value: 81.30724174653893
381
+ - type: nauc_map_at_1000_diff1
382
+ value: 26.962150235593356
383
+ - type: nauc_map_at_1000_max
384
+ value: 29.234958037854568
385
+ - type: nauc_map_at_1000_std
386
+ value: -2.4294465103633884
387
+ - type: nauc_map_at_100_diff1
388
+ value: 26.92990252114163
389
+ - type: nauc_map_at_100_max
390
+ value: 29.206328533120118
391
+ - type: nauc_map_at_100_std
392
+ value: -2.437371090941197
393
+ - type: nauc_map_at_10_diff1
394
+ value: 25.758265691179226
395
+ - type: nauc_map_at_10_max
396
+ value: 26.949978490795317
397
+ - type: nauc_map_at_10_std
398
+ value: -5.484961002106038
399
+ - type: nauc_map_at_1_diff1
400
+ value: 34.70849461278043
401
+ - type: nauc_map_at_1_max
402
+ value: 12.778570893623042
403
+ - type: nauc_map_at_1_std
404
+ value: -13.018292652743938
405
+ - type: nauc_map_at_20_diff1
406
+ value: 26.659923008218268
407
+ - type: nauc_map_at_20_max
408
+ value: 28.341440871568185
409
+ - type: nauc_map_at_20_std
410
+ value: -3.614549844913084
411
+ - type: nauc_map_at_3_diff1
412
+ value: 27.197629021438203
413
+ - type: nauc_map_at_3_max
414
+ value: 20.701094874050856
415
+ - type: nauc_map_at_3_std
416
+ value: -12.062992301112041
417
+ - type: nauc_map_at_5_diff1
418
+ value: 25.51793537203295
419
+ - type: nauc_map_at_5_max
420
+ value: 23.80396771243794
421
+ - type: nauc_map_at_5_std
422
+ value: -8.920465695323575
423
+ - type: nauc_mrr_at_1000_diff1
424
+ value: 45.14819989592967
425
+ - type: nauc_mrr_at_1000_max
426
+ value: 53.29202156141053
427
+ - type: nauc_mrr_at_1000_std
428
+ value: 18.037336462510524
429
+ - type: nauc_mrr_at_100_diff1
430
+ value: 45.15287600228451
431
+ - type: nauc_mrr_at_100_max
432
+ value: 53.29979751928615
433
+ - type: nauc_mrr_at_100_std
434
+ value: 18.04996604778386
435
+ - type: nauc_mrr_at_10_diff1
436
+ value: 44.96865105944474
437
+ - type: nauc_mrr_at_10_max
438
+ value: 53.53323465323092
439
+ - type: nauc_mrr_at_10_std
440
+ value: 18.25001344917689
441
+ - type: nauc_mrr_at_1_diff1
442
+ value: 46.16604946873163
443
+ - type: nauc_mrr_at_1_max
444
+ value: 48.573651103547874
445
+ - type: nauc_mrr_at_1_std
446
+ value: 13.764871626330915
447
+ - type: nauc_mrr_at_20_diff1
448
+ value: 45.11925458479102
449
+ - type: nauc_mrr_at_20_max
450
+ value: 53.35685123898342
451
+ - type: nauc_mrr_at_20_std
452
+ value: 18.127344968819905
453
+ - type: nauc_mrr_at_3_diff1
454
+ value: 45.377195452730234
455
+ - type: nauc_mrr_at_3_max
456
+ value: 53.35146309217089
457
+ - type: nauc_mrr_at_3_std
458
+ value: 17.47105877186237
459
+ - type: nauc_mrr_at_5_diff1
460
+ value: 45.00525578771549
461
+ - type: nauc_mrr_at_5_max
462
+ value: 53.76227254707128
463
+ - type: nauc_mrr_at_5_std
464
+ value: 18.437290060746957
465
+ - type: nauc_ndcg_at_1000_diff1
466
+ value: 31.19215594457491
467
+ - type: nauc_ndcg_at_1000_max
468
+ value: 38.09555406458668
469
+ - type: nauc_ndcg_at_1000_std
470
+ value: 7.225628621238009
471
+ - type: nauc_ndcg_at_100_diff1
472
+ value: 30.726331247999934
473
+ - type: nauc_ndcg_at_100_max
474
+ value: 37.81369589418277
475
+ - type: nauc_ndcg_at_100_std
476
+ value: 7.242855238555071
477
+ - type: nauc_ndcg_at_10_diff1
478
+ value: 27.514048333744835
479
+ - type: nauc_ndcg_at_10_max
480
+ value: 33.10990399385253
481
+ - type: nauc_ndcg_at_10_std
482
+ value: 0.3051899572112002
483
+ - type: nauc_ndcg_at_1_diff1
484
+ value: 47.06089085235751
485
+ - type: nauc_ndcg_at_1_max
486
+ value: 47.7300872370495
487
+ - type: nauc_ndcg_at_1_std
488
+ value: 12.468605493613916
489
+ - type: nauc_ndcg_at_20_diff1
490
+ value: 29.404215438764496
491
+ - type: nauc_ndcg_at_20_max
492
+ value: 35.26967886796471
493
+ - type: nauc_ndcg_at_20_std
494
+ value: 3.7214697890813353
495
+ - type: nauc_ndcg_at_3_diff1
496
+ value: 29.448848639643067
497
+ - type: nauc_ndcg_at_3_max
498
+ value: 33.85912412370657
499
+ - type: nauc_ndcg_at_3_std
500
+ value: 0.895453646819452
501
+ - type: nauc_ndcg_at_5_diff1
502
+ value: 26.916649012613526
503
+ - type: nauc_ndcg_at_5_max
504
+ value: 30.899005979291644
505
+ - type: nauc_ndcg_at_5_std
506
+ value: -1.0001575639156615
507
+ - type: nauc_precision_at_1000_diff1
508
+ value: -8.492004667432635
509
+ - type: nauc_precision_at_1000_max
510
+ value: 14.970190384017679
511
+ - type: nauc_precision_at_1000_std
512
+ value: 32.871386621137816
513
+ - type: nauc_precision_at_100_diff1
514
+ value: -8.287314133999967
515
+ - type: nauc_precision_at_100_max
516
+ value: 17.794821961284736
517
+ - type: nauc_precision_at_100_std
518
+ value: 35.092483550562
519
+ - type: nauc_precision_at_10_diff1
520
+ value: -7.594128993028063
521
+ - type: nauc_precision_at_10_max
522
+ value: 24.691446370325732
523
+ - type: nauc_precision_at_10_std
524
+ value: 30.126552282608493
525
+ - type: nauc_precision_at_1_diff1
526
+ value: 47.06089085235751
527
+ - type: nauc_precision_at_1_max
528
+ value: 47.7300872370495
529
+ - type: nauc_precision_at_1_std
530
+ value: 12.468605493613916
531
+ - type: nauc_precision_at_20_diff1
532
+ value: -6.503872195775146
533
+ - type: nauc_precision_at_20_max
534
+ value: 21.789730053141312
535
+ - type: nauc_precision_at_20_std
536
+ value: 32.61349377558794
537
+ - type: nauc_precision_at_3_diff1
538
+ value: 0.67417079971061
539
+ - type: nauc_precision_at_3_max
540
+ value: 30.793871354370662
541
+ - type: nauc_precision_at_3_std
542
+ value: 18.35266479252011
543
+ - type: nauc_precision_at_5_diff1
544
+ value: -7.088881730215777
545
+ - type: nauc_precision_at_5_max
546
+ value: 26.539771712769006
547
+ - type: nauc_precision_at_5_std
548
+ value: 24.116262291865834
549
+ - type: nauc_recall_at_1000_diff1
550
+ value: 34.53263588412461
551
+ - type: nauc_recall_at_1000_max
552
+ value: 63.54157869100173
553
+ - type: nauc_recall_at_1000_std
554
+ value: 64.19854844792808
555
+ - type: nauc_recall_at_100_diff1
556
+ value: 22.86564728642275
557
+ - type: nauc_recall_at_100_max
558
+ value: 40.350507162549825
559
+ - type: nauc_recall_at_100_std
560
+ value: 29.24492545863015
561
+ - type: nauc_recall_at_10_diff1
562
+ value: 15.384818367225009
563
+ - type: nauc_recall_at_10_max
564
+ value: 24.41108571453699
565
+ - type: nauc_recall_at_10_std
566
+ value: -3.9216160585776323
567
+ - type: nauc_recall_at_1_diff1
568
+ value: 34.70849461278043
569
+ - type: nauc_recall_at_1_max
570
+ value: 12.778570893623042
571
+ - type: nauc_recall_at_1_std
572
+ value: -13.018292652743938
573
+ - type: nauc_recall_at_20_diff1
574
+ value: 18.122499000084208
575
+ - type: nauc_recall_at_20_max
576
+ value: 26.63104220179424
577
+ - type: nauc_recall_at_20_std
578
+ value: 3.969217732521512
579
+ - type: nauc_recall_at_3_diff1
580
+ value: 21.413050725250116
581
+ - type: nauc_recall_at_3_max
582
+ value: 16.18894988386887
583
+ - type: nauc_recall_at_3_std
584
+ value: -15.24884339282375
585
+ - type: nauc_recall_at_5_diff1
586
+ value: 16.35673072212927
587
+ - type: nauc_recall_at_5_max
588
+ value: 18.607003829267846
589
+ - type: nauc_recall_at_5_std
590
+ value: -10.463525876945454
591
+ - type: ndcg_at_1
592
+ value: 72.923
593
+ - type: ndcg_at_10
594
+ value: 71.882
595
+ - type: ndcg_at_100
596
+ value: 77.09899999999999
597
+ - type: ndcg_at_1000
598
+ value: 77.835
599
+ - type: ndcg_at_20
600
+ value: 74.497
601
+ - type: ndcg_at_3
602
+ value: 68.504
603
+ - type: ndcg_at_5
604
+ value: 69.068
605
+ - type: precision_at_1
606
+ value: 72.923
607
+ - type: precision_at_10
608
+ value: 19.936
609
+ - type: precision_at_100
610
+ value: 2.6310000000000002
611
+ - type: precision_at_1000
612
+ value: 0.27799999999999997
613
+ - type: precision_at_20
614
+ value: 11.33
615
+ - type: precision_at_3
616
+ value: 45.927
617
+ - type: precision_at_5
618
+ value: 33.131
619
+ - type: recall_at_1
620
+ value: 37.913000000000004
621
+ - type: recall_at_10
622
+ value: 78.365
623
+ - type: recall_at_100
624
+ value: 94.348
625
+ - type: recall_at_1000
626
+ value: 98.187
627
+ - type: recall_at_20
628
+ value: 85.229
629
+ - type: recall_at_3
630
+ value: 61.42999999999999
631
+ - type: recall_at_5
632
+ value: 69.56700000000001
633
+ task:
634
+ type: Retrieval
635
+ - dataset:
636
+ config: ru
637
+ name: MTEB MassiveIntentClassification (ru)
638
+ revision: 4672e20407010da34463acc759c162ca9734bca6
639
+ split: test
640
+ type: mteb/amazon_massive_intent
641
+ metrics:
642
+ - type: accuracy
643
+ value: 79.11903160726294
644
+ - type: f1
645
+ value: 76.22609082694545
646
+ - type: f1_weighted
647
+ value: 77.81461248063566
648
+ - type: main_score
649
+ value: 79.11903160726294
650
+ task:
651
+ type: Classification
652
+ - dataset:
653
+ config: ru
654
+ name: MTEB MassiveScenarioClassification (ru)
655
+ revision: fad2c6e8459f9e1c45d9315f4953d921437d70f8
656
+ split: test
657
+ type: mteb/amazon_massive_scenario
658
+ metrics:
659
+ - type: accuracy
660
+ value: 88.80632145258912
661
+ - type: f1
662
+ value: 87.53157475314829
663
+ - type: f1_weighted
664
+ value: 88.22733432521495
665
+ - type: main_score
666
+ value: 88.80632145258912
667
+ task:
668
+ type: Classification
669
+ - dataset:
670
+ config: default
671
+ name: MTEB RUParaPhraserSTS (default)
672
+ revision: 43265056790b8f7c59e0139acb4be0a8dad2c8f4
673
+ split: test
674
+ type: merionum/ru_paraphraser
675
+ metrics:
676
+ - type: cosine_pearson
677
+ value: 72.70307124858925
678
+ - type: cosine_spearman
679
+ value: 78.09439086920204
680
+ - type: euclidean_pearson
681
+ value: 76.2033672014715
682
+ - type: euclidean_spearman
683
+ value: 78.09439086920204
684
+ - type: main_score
685
+ value: 78.09439086920204
686
+ - type: manhattan_pearson
687
+ value: 76.11750470223116
688
+ - type: manhattan_spearman
689
+ value: 78.01081063503413
690
+ - type: pearson
691
+ value: 72.70307124858925
692
+ - type: spearman
693
+ value: 78.09439086920204
694
+ task:
695
+ type: STS
696
+ - dataset:
697
+ config: default
698
+ name: MTEB RiaNewsRetrieval (default)
699
+ revision: 82374b0bbacda6114f39ff9c5b925fa1512ca5d7
700
+ split: test
701
+ type: ai-forever/ria-news-retrieval
702
+ metrics:
703
+ - type: main_score
704
+ value: 86.819
705
+ - type: map_at_1
706
+ value: 78.79
707
+ - type: map_at_10
708
+ value: 84.516
709
+ - type: map_at_100
710
+ value: 84.68
711
+ - type: map_at_1000
712
+ value: 84.685
713
+ - type: map_at_20
714
+ value: 84.624
715
+ - type: map_at_3
716
+ value: 83.722
717
+ - type: map_at_5
718
+ value: 84.246
719
+ - type: mrr_at_1
720
+ value: 78.78
721
+ - type: mrr_at_10
722
+ value: 84.51815476190441
723
+ - type: mrr_at_100
724
+ value: 84.68390840473289
725
+ - type: mrr_at_1000
726
+ value: 84.68947095200002
727
+ - type: mrr_at_20
728
+ value: 84.62958130822527
729
+ - type: mrr_at_3
730
+ value: 83.74499999999964
731
+ - type: mrr_at_5
732
+ value: 84.23849999999955
733
+ - type: nauc_map_at_1000_diff1
734
+ value: 82.09914867708899
735
+ - type: nauc_map_at_1000_max
736
+ value: 43.02024854784386
737
+ - type: nauc_map_at_1000_std
738
+ value: -22.919695880762777
739
+ - type: nauc_map_at_100_diff1
740
+ value: 82.09705922783733
741
+ - type: nauc_map_at_100_max
742
+ value: 43.02697379581718
743
+ - type: nauc_map_at_100_std
744
+ value: -22.90719212899522
745
+ - type: nauc_map_at_10_diff1
746
+ value: 82.04404594672894
747
+ - type: nauc_map_at_10_max
748
+ value: 43.06752103182731
749
+ - type: nauc_map_at_10_std
750
+ value: -23.007870153273576
751
+ - type: nauc_map_at_1_diff1
752
+ value: 83.89134152210333
753
+ - type: nauc_map_at_1_max
754
+ value: 38.083626428503415
755
+ - type: nauc_map_at_1_std
756
+ value: -25.817960401194252
757
+ - type: nauc_map_at_20_diff1
758
+ value: 82.08534662247806
759
+ - type: nauc_map_at_20_max
760
+ value: 43.074305042312346
761
+ - type: nauc_map_at_20_std
762
+ value: -22.91785703613217
763
+ - type: nauc_map_at_3_diff1
764
+ value: 81.7967508697558
765
+ - type: nauc_map_at_3_max
766
+ value: 42.90927479098251
767
+ - type: nauc_map_at_3_std
768
+ value: -24.01312203859392
769
+ - type: nauc_map_at_5_diff1
770
+ value: 81.90704517505098
771
+ - type: nauc_map_at_5_max
772
+ value: 43.05204677044616
773
+ - type: nauc_map_at_5_std
774
+ value: -23.267331507554896
775
+ - type: nauc_mrr_at_1000_diff1
776
+ value: 82.11902348082472
777
+ - type: nauc_mrr_at_1000_max
778
+ value: 43.04118936353063
779
+ - type: nauc_mrr_at_1000_std
780
+ value: -22.858804296830773
781
+ - type: nauc_mrr_at_100_diff1
782
+ value: 82.11685562002263
783
+ - type: nauc_mrr_at_100_max
784
+ value: 43.0482537895494
785
+ - type: nauc_mrr_at_100_std
786
+ value: -22.84431127787993
787
+ - type: nauc_mrr_at_10_diff1
788
+ value: 82.06909958688058
789
+ - type: nauc_mrr_at_10_max
790
+ value: 43.07921689466605
791
+ - type: nauc_mrr_at_10_std
792
+ value: -22.957623576663234
793
+ - type: nauc_mrr_at_1_diff1
794
+ value: 83.91147637794326
795
+ - type: nauc_mrr_at_1_max
796
+ value: 37.91917159543152
797
+ - type: nauc_mrr_at_1_std
798
+ value: -26.141868289283266
799
+ - type: nauc_mrr_at_20_diff1
800
+ value: 82.10314004731809
801
+ - type: nauc_mrr_at_20_max
802
+ value: 43.09295406509764
803
+ - type: nauc_mrr_at_20_std
804
+ value: -22.862091782178787
805
+ - type: nauc_mrr_at_3_diff1
806
+ value: 81.82117067269036
807
+ - type: nauc_mrr_at_3_max
808
+ value: 42.94628953323521
809
+ - type: nauc_mrr_at_3_std
810
+ value: -23.852510312400714
811
+ - type: nauc_mrr_at_5_diff1
812
+ value: 81.92857441701598
813
+ - type: nauc_mrr_at_5_max
814
+ value: 43.129719354492934
815
+ - type: nauc_mrr_at_5_std
816
+ value: -23.145342272624085
817
+ - type: nauc_ndcg_at_1000_diff1
818
+ value: 81.75015729717991
819
+ - type: nauc_ndcg_at_1000_max
820
+ value: 44.7266586308995
821
+ - type: nauc_ndcg_at_1000_std
822
+ value: -20.60663899715267
823
+ - type: nauc_ndcg_at_100_diff1
824
+ value: 81.6897808298767
825
+ - type: nauc_ndcg_at_100_max
826
+ value: 44.99492791287099
827
+ - type: nauc_ndcg_at_100_std
828
+ value: -20.09637266506936
829
+ - type: nauc_ndcg_at_10_diff1
830
+ value: 81.46290312197337
831
+ - type: nauc_ndcg_at_10_max
832
+ value: 45.30218378452244
833
+ - type: nauc_ndcg_at_10_std
834
+ value: -20.70393523891777
835
+ - type: nauc_ndcg_at_1_diff1
836
+ value: 83.89134152210333
837
+ - type: nauc_ndcg_at_1_max
838
+ value: 38.083626428503415
839
+ - type: nauc_ndcg_at_1_std
840
+ value: -25.817960401194252
841
+ - type: nauc_ndcg_at_20_diff1
842
+ value: 81.61080772657213
843
+ - type: nauc_ndcg_at_20_max
844
+ value: 45.36571800492172
845
+ - type: nauc_ndcg_at_20_std
846
+ value: -20.278763852504042
847
+ - type: nauc_ndcg_at_3_diff1
848
+ value: 80.95965359410461
849
+ - type: nauc_ndcg_at_3_max
850
+ value: 44.756971949205834
851
+ - type: nauc_ndcg_at_3_std
852
+ value: -23.07797617717319
853
+ - type: nauc_ndcg_at_5_diff1
854
+ value: 81.12417712163976
855
+ - type: nauc_ndcg_at_5_max
856
+ value: 45.15727381406512
857
+ - type: nauc_ndcg_at_5_std
858
+ value: -21.52861766165519
859
+ - type: nauc_precision_at_1000_diff1
860
+ value: 76.80566850396093
861
+ - type: nauc_precision_at_1000_max
862
+ value: 82.45685370922442
863
+ - type: nauc_precision_at_1000_std
864
+ value: 46.93570976777808
865
+ - type: nauc_precision_at_100_diff1
866
+ value: 77.21645520953484
867
+ - type: nauc_precision_at_100_max
868
+ value: 73.43604108309935
869
+ - type: nauc_precision_at_100_std
870
+ value: 31.978176891671367
871
+ - type: nauc_precision_at_10_diff1
872
+ value: 77.88251664302092
873
+ - type: nauc_precision_at_10_max
874
+ value: 60.58112638995018
875
+ - type: nauc_precision_at_10_std
876
+ value: -3.674424315180332
877
+ - type: nauc_precision_at_1_diff1
878
+ value: 83.89134152210333
879
+ - type: nauc_precision_at_1_max
880
+ value: 38.083626428503415
881
+ - type: nauc_precision_at_1_std
882
+ value: -25.817960401194252
883
+ - type: nauc_precision_at_20_diff1
884
+ value: 78.16426786697438
885
+ - type: nauc_precision_at_20_max
886
+ value: 66.0723612699222
887
+ - type: nauc_precision_at_20_std
888
+ value: 6.121527084555938
889
+ - type: nauc_precision_at_3_diff1
890
+ value: 77.43122492166451
891
+ - type: nauc_precision_at_3_max
892
+ value: 52.50727288548085
893
+ - type: nauc_precision_at_3_std
894
+ value: -19.036076920799427
895
+ - type: nauc_precision_at_5_diff1
896
+ value: 77.1127254320532
897
+ - type: nauc_precision_at_5_max
898
+ value: 56.100901899221135
899
+ - type: nauc_precision_at_5_std
900
+ value: -12.009191140844198
901
+ - type: nauc_recall_at_1000_diff1
902
+ value: 76.80566850396035
903
+ - type: nauc_recall_at_1000_max
904
+ value: 82.45685370922577
905
+ - type: nauc_recall_at_1000_std
906
+ value: 46.93570976777776
907
+ - type: nauc_recall_at_100_diff1
908
+ value: 77.21645520953459
909
+ - type: nauc_recall_at_100_max
910
+ value: 73.43604108310011
911
+ - type: nauc_recall_at_100_std
912
+ value: 31.978176891671993
913
+ - type: nauc_recall_at_10_diff1
914
+ value: 77.88251664302089
915
+ - type: nauc_recall_at_10_max
916
+ value: 60.58112638994999
917
+ - type: nauc_recall_at_10_std
918
+ value: -3.6744243151805427
919
+ - type: nauc_recall_at_1_diff1
920
+ value: 83.89134152210333
921
+ - type: nauc_recall_at_1_max
922
+ value: 38.083626428503415
923
+ - type: nauc_recall_at_1_std
924
+ value: -25.817960401194252
925
+ - type: nauc_recall_at_20_diff1
926
+ value: 78.16426786697409
927
+ - type: nauc_recall_at_20_max
928
+ value: 66.07236126992217
929
+ - type: nauc_recall_at_20_std
930
+ value: 6.121527084555941
931
+ - type: nauc_recall_at_3_diff1
932
+ value: 77.43122492166454
933
+ - type: nauc_recall_at_3_max
934
+ value: 52.507272885480816
935
+ - type: nauc_recall_at_3_std
936
+ value: -19.036076920799776
937
+ - type: nauc_recall_at_5_diff1
938
+ value: 77.11272543205318
939
+ - type: nauc_recall_at_5_max
940
+ value: 56.10090189922128
941
+ - type: nauc_recall_at_5_std
942
+ value: -12.009191140843809
943
+ - type: ndcg_at_1
944
+ value: 78.79
945
+ - type: ndcg_at_10
946
+ value: 86.819
947
+ - type: ndcg_at_100
948
+ value: 87.599
949
+ - type: ndcg_at_1000
950
+ value: 87.761
951
+ - type: ndcg_at_20
952
+ value: 87.208
953
+ - type: ndcg_at_3
954
+ value: 85.222
955
+ - type: ndcg_at_5
956
+ value: 86.164
957
+ - type: precision_at_1
958
+ value: 78.79
959
+ - type: precision_at_10
960
+ value: 9.384
961
+ - type: precision_at_100
962
+ value: 0.975
963
+ - type: precision_at_1000
964
+ value: 0.099
965
+ - type: precision_at_20
966
+ value: 4.769
967
+ - type: precision_at_3
968
+ value: 29.842999999999996
969
+ - type: precision_at_5
970
+ value: 18.362000000000002
971
+ - type: recall_at_1
972
+ value: 78.79
973
+ - type: recall_at_10
974
+ value: 93.84
975
+ - type: recall_at_100
976
+ value: 97.45
977
+ - type: recall_at_1000
978
+ value: 98.76
979
+ - type: recall_at_20
980
+ value: 95.37
981
+ - type: recall_at_3
982
+ value: 89.53
983
+ - type: recall_at_5
984
+ value: 91.81
985
+ task:
986
+ type: Retrieval
987
+ - dataset:
988
+ config: default
989
+ name: MTEB RuBQReranking (default)
990
+ revision: 2e96b8f098fa4b0950fc58eacadeb31c0d0c7fa2
991
+ split: test
992
+ type: ai-forever/rubq-reranking
993
+ metrics:
994
+ - type: main_score
995
+ value: 77.07394404835635
996
+ - type: map
997
+ value: 77.07394404835635
998
+ - type: mrr
999
+ value: 82.53144412718882
1000
+ - type: nAUC_map_diff1
1001
+ value: 45.29805217456628
1002
+ - type: nAUC_map_max
1003
+ value: 34.39894042439188
1004
+ - type: nAUC_map_std
1005
+ value: 21.11309674418275
1006
+ - type: nAUC_mrr_diff1
1007
+ value: 54.783994737367046
1008
+ - type: nAUC_mrr_max
1009
+ value: 45.68526733900048
1010
+ - type: nAUC_mrr_std
1011
+ value: 28.22466385500339
1012
+ task:
1013
+ type: Reranking
1014
+ - dataset:
1015
+ config: default
1016
+ name: MTEB RuBQRetrieval (default)
1017
+ revision: e19b6ffa60b3bc248e0b41f4cc37c26a55c2a67b
1018
+ split: test
1019
+ type: ai-forever/rubq-retrieval
1020
+ metrics:
1021
+ - type: main_score
1022
+ value: 72.392
1023
+ - type: map_at_1
1024
+ value: 47.370000000000005
1025
+ - type: map_at_10
1026
+ value: 65.503
1027
+ - type: map_at_100
1028
+ value: 66.38
1029
+ - type: map_at_1000
1030
+ value: 66.42099999999999
1031
+ - type: map_at_20
1032
+ value: 66.071
1033
+ - type: map_at_3
1034
+ value: 61.439
1035
+ - type: map_at_5
1036
+ value: 63.922999999999995
1037
+ - type: mrr_at_1
1038
+ value: 67.37588652482269
1039
+ - type: mrr_at_10
1040
+ value: 76.0066747345116
1041
+ - type: mrr_at_100
1042
+ value: 76.25754138969413
1043
+ - type: mrr_at_1000
1044
+ value: 76.26968825657428
1045
+ - type: mrr_at_20
1046
+ value: 76.17548265904622
1047
+ - type: mrr_at_3
1048
+ value: 74.61583924349881
1049
+ - type: mrr_at_5
1050
+ value: 75.46690307328608
1051
+ - type: nauc_map_at_1000_diff1
1052
+ value: 42.52570720187294
1053
+ - type: nauc_map_at_1000_max
1054
+ value: 37.40318318724238
1055
+ - type: nauc_map_at_1000_std
1056
+ value: 0.6037788201535506
1057
+ - type: nauc_map_at_100_diff1
1058
+ value: 42.493410029691226
1059
+ - type: nauc_map_at_100_max
1060
+ value: 37.39802489244377
1061
+ - type: nauc_map_at_100_std
1062
+ value: 0.6071359951887154
1063
+ - type: nauc_map_at_10_diff1
1064
+ value: 42.09833519659916
1065
+ - type: nauc_map_at_10_max
1066
+ value: 37.1184138958874
1067
+ - type: nauc_map_at_10_std
1068
+ value: 0.4063543094010351
1069
+ - type: nauc_map_at_1_diff1
1070
+ value: 49.56605205141156
1071
+ - type: nauc_map_at_1_max
1072
+ value: 26.251096698710384
1073
+ - type: nauc_map_at_1_std
1074
+ value: -4.580748485387834
1075
+ - type: nauc_map_at_20_diff1
1076
+ value: 42.33372393482018
1077
+ - type: nauc_map_at_20_max
1078
+ value: 37.416955604649985
1079
+ - type: nauc_map_at_20_std
1080
+ value: 0.6050577802787294
1081
+ - type: nauc_map_at_3_diff1
1082
+ value: 42.362234475441845
1083
+ - type: nauc_map_at_3_max
1084
+ value: 34.56001379838821
1085
+ - type: nauc_map_at_3_std
1086
+ value: -1.507636598929042
1087
+ - type: nauc_map_at_5_diff1
1088
+ value: 42.0202264882535
1089
+ - type: nauc_map_at_5_max
1090
+ value: 36.64306050200848
1091
+ - type: nauc_map_at_5_std
1092
+ value: -0.09509025708798424
1093
+ - type: nauc_mrr_at_1000_diff1
1094
+ value: 58.99601742026931
1095
+ - type: nauc_mrr_at_1000_max
1096
+ value: 49.61561872452777
1097
+ - type: nauc_mrr_at_1000_std
1098
+ value: 2.3956102974352356
1099
+ - type: nauc_mrr_at_100_diff1
1100
+ value: 58.9865943101085
1101
+ - type: nauc_mrr_at_100_max
1102
+ value: 49.6248111507265
1103
+ - type: nauc_mrr_at_100_std
1104
+ value: 2.411155095066369
1105
+ - type: nauc_mrr_at_10_diff1
1106
+ value: 58.81758131092919
1107
+ - type: nauc_mrr_at_10_max
1108
+ value: 49.780365572616695
1109
+ - type: nauc_mrr_at_10_std
1110
+ value: 2.7068696565195944
1111
+ - type: nauc_mrr_at_1_diff1
1112
+ value: 61.67036882487055
1113
+ - type: nauc_mrr_at_1_max
1114
+ value: 45.455271042821714
1115
+ - type: nauc_mrr_at_1_std
1116
+ value: -0.9370526815458349
1117
+ - type: nauc_mrr_at_20_diff1
1118
+ value: 58.93674818203478
1119
+ - type: nauc_mrr_at_20_max
1120
+ value: 49.703218108625215
1121
+ - type: nauc_mrr_at_20_std
1122
+ value: 2.4473106598190415
1123
+ - type: nauc_mrr_at_3_diff1
1124
+ value: 59.046856598788445
1125
+ - type: nauc_mrr_at_3_max
1126
+ value: 49.37161726123392
1127
+ - type: nauc_mrr_at_3_std
1128
+ value: 1.5110936686701506
1129
+ - type: nauc_mrr_at_5_diff1
1130
+ value: 58.92289378915668
1131
+ - type: nauc_mrr_at_5_max
1132
+ value: 49.847638994134144
1133
+ - type: nauc_mrr_at_5_std
1134
+ value: 2.420421880131702
1135
+ - type: nauc_ndcg_at_1000_diff1
1136
+ value: 45.56062215161734
1137
+ - type: nauc_ndcg_at_1000_max
1138
+ value: 41.507152286702
1139
+ - type: nauc_ndcg_at_1000_std
1140
+ value: 2.79388283208751
1141
+ - type: nauc_ndcg_at_100_diff1
1142
+ value: 44.84064192570408
1143
+ - type: nauc_ndcg_at_100_max
1144
+ value: 41.50353573562353
1145
+ - type: nauc_ndcg_at_100_std
1146
+ value: 3.1804999773629357
1147
+ - type: nauc_ndcg_at_10_diff1
1148
+ value: 43.341482144213614
1149
+ - type: nauc_ndcg_at_10_max
1150
+ value: 41.159590898395074
1151
+ - type: nauc_ndcg_at_10_std
1152
+ value: 2.945242338240843
1153
+ - type: nauc_ndcg_at_1_diff1
1154
+ value: 62.23623985611396
1155
+ - type: nauc_ndcg_at_1_max
1156
+ value: 45.04945770947091
1157
+ - type: nauc_ndcg_at_1_std
1158
+ value: -0.8804967656575725
1159
+ - type: nauc_ndcg_at_20_diff1
1160
+ value: 43.905372612093664
1161
+ - type: nauc_ndcg_at_20_max
1162
+ value: 41.797709837872446
1163
+ - type: nauc_ndcg_at_20_std
1164
+ value: 3.1853356915569653
1165
+ - type: nauc_ndcg_at_3_diff1
1166
+ value: 44.18163998834299
1167
+ - type: nauc_ndcg_at_3_max
1168
+ value: 38.352891017864636
1169
+ - type: nauc_ndcg_at_3_std
1170
+ value: -0.8235767021150929
1171
+ - type: nauc_ndcg_at_5_diff1
1172
+ value: 43.41374688421302
1173
+ - type: nauc_ndcg_at_5_max
1174
+ value: 40.390365601593956
1175
+ - type: nauc_ndcg_at_5_std
1176
+ value: 1.6743650108127537
1177
+ - type: nauc_precision_at_1000_diff1
1178
+ value: -9.711058370691381
1179
+ - type: nauc_precision_at_1000_max
1180
+ value: 6.97321343449286
1181
+ - type: nauc_precision_at_1000_std
1182
+ value: 7.933531916622121
1183
+ - type: nauc_precision_at_100_diff1
1184
+ value: -8.247029644152319
1185
+ - type: nauc_precision_at_100_max
1186
+ value: 10.86740140944616
1187
+ - type: nauc_precision_at_100_std
1188
+ value: 9.581885544675918
1189
+ - type: nauc_precision_at_10_diff1
1190
+ value: -2.409043695429943
1191
+ - type: nauc_precision_at_10_max
1192
+ value: 21.04733206074314
1193
+ - type: nauc_precision_at_10_std
1194
+ value: 10.03334651647101
1195
+ - type: nauc_precision_at_1_diff1
1196
+ value: 62.23623985611396
1197
+ - type: nauc_precision_at_1_max
1198
+ value: 45.04945770947091
1199
+ - type: nauc_precision_at_1_std
1200
+ value: -0.8804967656575725
1201
+ - type: nauc_precision_at_20_diff1
1202
+ value: -5.230303656931621
1203
+ - type: nauc_precision_at_20_max
1204
+ value: 17.77799716919181
1205
+ - type: nauc_precision_at_20_std
1206
+ value: 10.739127998618654
1207
+ - type: nauc_precision_at_3_diff1
1208
+ value: 10.40376424999862
1209
+ - type: nauc_precision_at_3_max
1210
+ value: 30.933333400254035
1211
+ - type: nauc_precision_at_3_std
1212
+ value: 6.126209127968004
1213
+ - type: nauc_precision_at_5_diff1
1214
+ value: 3.147398101830739
1215
+ - type: nauc_precision_at_5_max
1216
+ value: 27.1746309955971
1217
+ - type: nauc_precision_at_5_std
1218
+ value: 8.874723615388788
1219
+ - type: nauc_recall_at_1000_diff1
1220
+ value: 5.055940692380908
1221
+ - type: nauc_recall_at_1000_max
1222
+ value: 22.42031123370267
1223
+ - type: nauc_recall_at_1000_std
1224
+ value: 27.75476692527869
1225
+ - type: nauc_recall_at_100_diff1
1226
+ value: 17.86391178198642
1227
+ - type: nauc_recall_at_100_max
1228
+ value: 34.776134863678955
1229
+ - type: nauc_recall_at_100_std
1230
+ value: 18.96377158778504
1231
+ - type: nauc_recall_at_10_diff1
1232
+ value: 24.863097695413597
1233
+ - type: nauc_recall_at_10_max
1234
+ value: 37.697411651507444
1235
+ - type: nauc_recall_at_10_std
1236
+ value: 9.519849994253967
1237
+ - type: nauc_recall_at_1_diff1
1238
+ value: 49.56605205141156
1239
+ - type: nauc_recall_at_1_max
1240
+ value: 26.251096698710384
1241
+ - type: nauc_recall_at_1_std
1242
+ value: -4.580748485387834
1243
+ - type: nauc_recall_at_20_diff1
1244
+ value: 22.440602811005636
1245
+ - type: nauc_recall_at_20_max
1246
+ value: 39.538861316515
1247
+ - type: nauc_recall_at_20_std
1248
+ value: 11.363269553121468
1249
+ - type: nauc_recall_at_3_diff1
1250
+ value: 32.80302839873736
1251
+ - type: nauc_recall_at_3_max
1252
+ value: 32.53105685012729
1253
+ - type: nauc_recall_at_3_std
1254
+ value: -0.7140166410605693
1255
+ - type: nauc_recall_at_5_diff1
1256
+ value: 29.375386639154865
1257
+ - type: nauc_recall_at_5_max
1258
+ value: 36.91045781164083
1259
+ - type: nauc_recall_at_5_std
1260
+ value: 4.725419050262578
1261
+ - type: ndcg_at_1
1262
+ value: 67.13900000000001
1263
+ - type: ndcg_at_10
1264
+ value: 72.392
1265
+ - type: ndcg_at_100
1266
+ value: 75.25800000000001
1267
+ - type: ndcg_at_1000
1268
+ value: 75.982
1269
+ - type: ndcg_at_20
1270
+ value: 73.783
1271
+ - type: ndcg_at_3
1272
+ value: 67.269
1273
+ - type: ndcg_at_5
1274
+ value: 69.807
1275
+ - type: precision_at_1
1276
+ value: 67.13900000000001
1277
+ - type: precision_at_10
1278
+ value: 13.327
1279
+ - type: precision_at_100
1280
+ value: 1.5559999999999998
1281
+ - type: precision_at_1000
1282
+ value: 0.164
1283
+ - type: precision_at_20
1284
+ value: 7.119000000000001
1285
+ - type: precision_at_3
1286
+ value: 35.599
1287
+ - type: precision_at_5
1288
+ value: 23.936
1289
+ - type: recall_at_1
1290
+ value: 47.370000000000005
1291
+ - type: recall_at_10
1292
+ value: 82.16
1293
+ - type: recall_at_100
1294
+ value: 93.34
1295
+ - type: recall_at_1000
1296
+ value: 98.202
1297
+ - type: recall_at_20
1298
+ value: 86.687
1299
+ - type: recall_at_3
1300
+ value: 69.319
1301
+ - type: recall_at_5
1302
+ value: 75.637
1303
+ task:
1304
+ type: Retrieval
1305
+ - dataset:
1306
+ config: default
1307
+ name: MTEB RuReviewsClassification (default)
1308
+ revision: f6d2c31f4dc6b88f468552750bfec05b4b41b05a
1309
+ split: test
1310
+ type: ai-forever/ru-reviews-classification
1311
+ metrics:
1312
+ - type: accuracy
1313
+ value: 75.0537109375
1314
+ - type: f1
1315
+ value: 74.00523205209554
1316
+ - type: f1_weighted
1317
+ value: 74.00436782840376
1318
+ - type: main_score
1319
+ value: 75.0537109375
1320
+ task:
1321
+ type: Classification
1322
+ - dataset:
1323
+ config: default
1324
+ name: MTEB RuSTSBenchmarkSTS (default)
1325
+ revision: 7cf24f325c6da6195df55bef3d86b5e0616f3018
1326
+ split: test
1327
+ type: ai-forever/ru-stsbenchmark-sts
1328
+ metrics:
1329
+ - type: cosine_pearson
1330
+ value: 81.10255413476487
1331
+ - type: cosine_spearman
1332
+ value: 81.40020843157141
1333
+ - type: euclidean_pearson
1334
+ value: 81.25155479902466
1335
+ - type: euclidean_spearman
1336
+ value: 81.40020831064922
1337
+ - type: main_score
1338
+ value: 81.40020843157141
1339
+ - type: manhattan_pearson
1340
+ value: 81.1493715249014
1341
+ - type: manhattan_spearman
1342
+ value: 81.30973667941649
1343
+ - type: pearson
1344
+ value: 81.10255413476487
1345
+ - type: spearman
1346
+ value: 81.40020843157141
1347
+ task:
1348
+ type: STS
1349
+ - dataset:
1350
+ config: default
1351
+ name: MTEB RuSciBenchGRNTIClassification (default)
1352
+ revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
1353
+ split: test
1354
+ type: ai-forever/ru-scibench-grnti-classification
1355
+ metrics:
1356
+ - type: accuracy
1357
+ value: 69.8974609375
1358
+ - type: f1
1359
+ value: 68.57837564785511
1360
+ - type: f1_weighted
1361
+ value: 68.59030489460784
1362
+ - type: main_score
1363
+ value: 69.8974609375
1364
+ task:
1365
+ type: Classification
1366
+ - dataset:
1367
+ config: default
1368
+ name: MTEB RuSciBenchGRNTIClusteringP2P (default)
1369
+ revision: 673a610d6d3dd91a547a0d57ae1b56f37ebbf6a1
1370
+ split: test
1371
+ type: ai-forever/ru-scibench-grnti-classification
1372
+ metrics:
1373
+ - type: main_score
1374
+ value: 67.03880348548029
1375
+ - type: v_measure
1376
+ value: 67.03880348548029
1377
+ - type: v_measure_std
1378
+ value: 0.6126278133139618
1379
+ task:
1380
+ type: Clustering
1381
+ - dataset:
1382
+ config: default
1383
+ name: MTEB RuSciBenchOECDClassification (default)
1384
+ revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
1385
+ split: test
1386
+ type: ai-forever/ru-scibench-oecd-classification
1387
+ metrics:
1388
+ - type: accuracy
1389
+ value: 54.63378906250001
1390
+ - type: f1
1391
+ value: 51.34306420274629
1392
+ - type: f1_weighted
1393
+ value: 51.33495867493914
1394
+ - type: main_score
1395
+ value: 54.63378906250001
1396
+ task:
1397
+ type: Classification
1398
+ - dataset:
1399
+ config: default
1400
+ name: MTEB RuSciBenchOECDClusteringP2P (default)
1401
+ revision: 26c88e99dcaba32bb45d0e1bfc21902337f6d471
1402
+ split: test
1403
+ type: ai-forever/ru-scibench-oecd-classification
1404
+ metrics:
1405
+ - type: main_score
1406
+ value: 56.55947121159027
1407
+ - type: v_measure
1408
+ value: 56.55947121159027
1409
+ - type: v_measure_std
1410
+ value: 0.5498882006880662
1411
+ task:
1412
+ type: Clustering
1413
+ - dataset:
1414
+ config: ru
1415
+ name: MTEB STS22 (ru)
1416
+ revision: de9d86b3b84231dc21f76c7b7af1f28e2f57f6e3
1417
+ split: test
1418
+ type: mteb/sts22-crosslingual-sts
1419
+ metrics:
1420
+ - type: cosine_pearson
1421
+ value: 61.833294921667914
1422
+ - type: cosine_spearman
1423
+ value: 63.53967536726357
1424
+ - type: euclidean_pearson
1425
+ value: 60.382865218855805
1426
+ - type: euclidean_spearman
1427
+ value: 63.53967536726357
1428
+ - type: main_score
1429
+ value: 63.53967536726357
1430
+ - type: manhattan_pearson
1431
+ value: 60.24879015304578
1432
+ - type: manhattan_spearman
1433
+ value: 63.42305760430092
1434
+ - type: pearson
1435
+ value: 61.833294921667914
1436
+ - type: spearman
1437
+ value: 63.53967536726357
1438
+ task:
1439
+ type: STS
1440
+ - dataset:
1441
+ config: default
1442
+ name: MTEB SensitiveTopicsClassification (default)
1443
+ revision: 416b34a802308eac30e4192afc0ff99bb8dcc7f2
1444
+ split: test
1445
+ type: ai-forever/sensitive-topics-classification
1446
+ metrics:
1447
+ - type: accuracy
1448
+ value: 39.8193359375
1449
+ - type: f1
1450
+ value: 55.46591740935434
1451
+ - type: lrap
1452
+ value: 66.50980631510454
1453
+ - type: main_score
1454
+ value: 39.8193359375
1455
+ task:
1456
+ type: MultilabelClassification
1457
+ - dataset:
1458
+ config: default
1459
+ name: MTEB TERRa (default)
1460
+ revision: 7b58f24536063837d644aab9a023c62199b2a612
1461
+ split: dev
1462
+ type: ai-forever/terra-pairclassification
1463
+ metrics:
1464
+ - type: cosine_accuracy
1465
+ value: 66.77524429967427
1466
+ - type: cosine_accuracy_threshold
1467
+ value: 55.58975338935852
1468
+ - type: cosine_ap
1469
+ value: 66.4567219323658
1470
+ - type: cosine_f1
1471
+ value: 70.64676616915423
1472
+ - type: cosine_f1_threshold
1473
+ value: 45.55969536304474
1474
+ - type: cosine_precision
1475
+ value: 57.028112449799195
1476
+ - type: cosine_recall
1477
+ value: 92.81045751633987
1478
+ - type: dot_accuracy
1479
+ value: 66.77524429967427
1480
+ - type: dot_accuracy_threshold
1481
+ value: 55.589759349823
1482
+ - type: dot_ap
1483
+ value: 66.4567219323658
1484
+ - type: dot_f1
1485
+ value: 70.64676616915423
1486
+ - type: dot_f1_threshold
1487
+ value: 45.55969536304474
1488
+ - type: dot_precision
1489
+ value: 57.028112449799195
1490
+ - type: dot_recall
1491
+ value: 92.81045751633987
1492
+ - type: euclidean_accuracy
1493
+ value: 66.77524429967427
1494
+ - type: euclidean_accuracy_threshold
1495
+ value: 94.24455165863037
1496
+ - type: euclidean_ap
1497
+ value: 66.4567219323658
1498
+ - type: euclidean_f1
1499
+ value: 70.64676616915423
1500
+ - type: euclidean_f1_threshold
1501
+ value: 104.34587001800537
1502
+ - type: euclidean_precision
1503
+ value: 57.028112449799195
1504
+ - type: euclidean_recall
1505
+ value: 92.81045751633987
1506
+ - type: main_score
1507
+ value: 66.4567219323658
1508
+ - type: manhattan_accuracy
1509
+ value: 66.77524429967427
1510
+ - type: manhattan_accuracy_threshold
1511
+ value: 2865.5345916748047
1512
+ - type: manhattan_ap
1513
+ value: 66.26659863769075
1514
+ - type: manhattan_f1
1515
+ value: 70.8542713567839
1516
+ - type: manhattan_f1_threshold
1517
+ value: 3212.3912811279297
1518
+ - type: manhattan_precision
1519
+ value: 57.55102040816327
1520
+ - type: manhattan_recall
1521
+ value: 92.15686274509804
1522
+ - type: max_accuracy
1523
+ value: 66.77524429967427
1524
+ - type: max_ap
1525
+ value: 66.4567219323658
1526
+ - type: max_f1
1527
+ value: 70.8542713567839
1528
+ - type: max_precision
1529
+ value: 57.55102040816327
1530
+ - type: max_recall
1531
+ value: 92.81045751633987
1532
+ - type: similarity_accuracy
1533
+ value: 66.77524429967427
1534
+ - type: similarity_accuracy_threshold
1535
+ value: 55.58975338935852
1536
+ - type: similarity_ap
1537
+ value: 66.4567219323658
1538
+ - type: similarity_f1
1539
+ value: 70.64676616915423
1540
+ - type: similarity_f1_threshold
1541
+ value: 45.55969536304474
1542
+ - type: similarity_precision
1543
+ value: 57.028112449799195
1544
+ - type: similarity_recall
1545
+ value: 92.81045751633987
1546
+ task:
1547
+ type: PairClassification
1548
+ license: mit
1549
+ language:
1550
+ - ru
1551
+ - en
1552
+ tags:
1553
+ - mteb
1554
+ - transformers
1555
+ - sentence-transformers
1556
+ base_model: ai-forever/FRED-T5-1.7B
1557
+ pipeline_tag: feature-extraction
1558
+ datasets:
1559
+ - ai-forever/solyanka
1560
+ ---
1561
 
1562
  # Model Card for FRIDA
1563
 
 
1567
 
1568
  FRIDA is a full-scale finetuned general text embedding model inspired by denoising architecture based on T5. The model is based on the encoder part of [FRED-T5](https://arxiv.org/abs/2309.10931) model and continues research of text embedding models ([ruMTEB](https://arxiv.org/abs/2408.12503), [ru-en-RoSBERTa](https://huggingface.co/ai-forever/ru-en-RoSBERTa)). It has been pre-trained on a Russian-English dataset and fine-tuned for improved performance on the target task.
1569
 
1570
+ For more model details please refer to our [article](https://habr.com/ru/companies/sberdevices/articles/909924/) (RU).
1571
 
1572
  ## Usage
1573