phate334 commited on
Commit
6121c33
·
verified ·
1 Parent(s): 5532046

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +1113 -0
README.md ADDED
@@ -0,0 +1,1113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ tags:
3
+ - sentence-transformers
4
+ - feature-extraction
5
+ - sentence-similarity
6
+ - mteb
7
+ - transformers
8
+ - transformers.js
9
+ - llama-cpp
10
+ - gguf-my-repo
11
+ inference: false
12
+ license: apache-2.0
13
+ language:
14
+ - en
15
+ - zh
16
+ base_model: jinaai/jina-embeddings-v2-base-zh
17
+ model-index:
18
+ - name: jina-embeddings-v2-base-zh
19
+ results:
20
+ - task:
21
+ type: STS
22
+ dataset:
23
+ name: MTEB AFQMC
24
+ type: C-MTEB/AFQMC
25
+ config: default
26
+ split: validation
27
+ revision: None
28
+ metrics:
29
+ - type: cos_sim_pearson
30
+ value: 48.51403119231363
31
+ - type: cos_sim_spearman
32
+ value: 50.5928547846445
33
+ - type: euclidean_pearson
34
+ value: 48.750436310559074
35
+ - type: euclidean_spearman
36
+ value: 50.50950238691385
37
+ - type: manhattan_pearson
38
+ value: 48.7866189440328
39
+ - type: manhattan_spearman
40
+ value: 50.58692402017165
41
+ - task:
42
+ type: STS
43
+ dataset:
44
+ name: MTEB ATEC
45
+ type: C-MTEB/ATEC
46
+ config: default
47
+ split: test
48
+ revision: None
49
+ metrics:
50
+ - type: cos_sim_pearson
51
+ value: 50.25985700105725
52
+ - type: cos_sim_spearman
53
+ value: 51.28815934593989
54
+ - type: euclidean_pearson
55
+ value: 52.70329248799904
56
+ - type: euclidean_spearman
57
+ value: 50.94101139559258
58
+ - type: manhattan_pearson
59
+ value: 52.6647237400892
60
+ - type: manhattan_spearman
61
+ value: 50.922441325406176
62
+ - task:
63
+ type: Classification
64
+ dataset:
65
+ name: MTEB AmazonReviewsClassification (zh)
66
+ type: mteb/amazon_reviews_multi
67
+ config: zh
68
+ split: test
69
+ revision: 1399c76144fd37290681b995c656ef9b2e06e26d
70
+ metrics:
71
+ - type: accuracy
72
+ value: 34.944
73
+ - type: f1
74
+ value: 34.06478860660109
75
+ - task:
76
+ type: STS
77
+ dataset:
78
+ name: MTEB BQ
79
+ type: C-MTEB/BQ
80
+ config: default
81
+ split: test
82
+ revision: None
83
+ metrics:
84
+ - type: cos_sim_pearson
85
+ value: 65.15667035488342
86
+ - type: cos_sim_spearman
87
+ value: 66.07110142081
88
+ - type: euclidean_pearson
89
+ value: 60.447598102249714
90
+ - type: euclidean_spearman
91
+ value: 61.826575796578766
92
+ - type: manhattan_pearson
93
+ value: 60.39364279354984
94
+ - type: manhattan_spearman
95
+ value: 61.78743491223281
96
+ - task:
97
+ type: Clustering
98
+ dataset:
99
+ name: MTEB CLSClusteringP2P
100
+ type: C-MTEB/CLSClusteringP2P
101
+ config: default
102
+ split: test
103
+ revision: None
104
+ metrics:
105
+ - type: v_measure
106
+ value: 39.96714175391701
107
+ - task:
108
+ type: Clustering
109
+ dataset:
110
+ name: MTEB CLSClusteringS2S
111
+ type: C-MTEB/CLSClusteringS2S
112
+ config: default
113
+ split: test
114
+ revision: None
115
+ metrics:
116
+ - type: v_measure
117
+ value: 38.39863566717934
118
+ - task:
119
+ type: Reranking
120
+ dataset:
121
+ name: MTEB CMedQAv1
122
+ type: C-MTEB/CMedQAv1-reranking
123
+ config: default
124
+ split: test
125
+ revision: None
126
+ metrics:
127
+ - type: map
128
+ value: 83.63680381780644
129
+ - type: mrr
130
+ value: 86.16476190476192
131
+ - task:
132
+ type: Reranking
133
+ dataset:
134
+ name: MTEB CMedQAv2
135
+ type: C-MTEB/CMedQAv2-reranking
136
+ config: default
137
+ split: test
138
+ revision: None
139
+ metrics:
140
+ - type: map
141
+ value: 83.74350667859487
142
+ - type: mrr
143
+ value: 86.10388888888889
144
+ - task:
145
+ type: Retrieval
146
+ dataset:
147
+ name: MTEB CmedqaRetrieval
148
+ type: C-MTEB/CmedqaRetrieval
149
+ config: default
150
+ split: dev
151
+ revision: None
152
+ metrics:
153
+ - type: map_at_1
154
+ value: 22.072
155
+ - type: map_at_10
156
+ value: 32.942
157
+ - type: map_at_100
158
+ value: 34.768
159
+ - type: map_at_1000
160
+ value: 34.902
161
+ - type: map_at_3
162
+ value: 29.357
163
+ - type: map_at_5
164
+ value: 31.236000000000004
165
+ - type: mrr_at_1
166
+ value: 34.259
167
+ - type: mrr_at_10
168
+ value: 41.957
169
+ - type: mrr_at_100
170
+ value: 42.982
171
+ - type: mrr_at_1000
172
+ value: 43.042
173
+ - type: mrr_at_3
174
+ value: 39.722
175
+ - type: mrr_at_5
176
+ value: 40.898
177
+ - type: ndcg_at_1
178
+ value: 34.259
179
+ - type: ndcg_at_10
180
+ value: 39.153
181
+ - type: ndcg_at_100
182
+ value: 46.493
183
+ - type: ndcg_at_1000
184
+ value: 49.01
185
+ - type: ndcg_at_3
186
+ value: 34.636
187
+ - type: ndcg_at_5
188
+ value: 36.278
189
+ - type: precision_at_1
190
+ value: 34.259
191
+ - type: precision_at_10
192
+ value: 8.815000000000001
193
+ - type: precision_at_100
194
+ value: 1.474
195
+ - type: precision_at_1000
196
+ value: 0.179
197
+ - type: precision_at_3
198
+ value: 19.73
199
+ - type: precision_at_5
200
+ value: 14.174000000000001
201
+ - type: recall_at_1
202
+ value: 22.072
203
+ - type: recall_at_10
204
+ value: 48.484
205
+ - type: recall_at_100
206
+ value: 79.035
207
+ - type: recall_at_1000
208
+ value: 96.15
209
+ - type: recall_at_3
210
+ value: 34.607
211
+ - type: recall_at_5
212
+ value: 40.064
213
+ - task:
214
+ type: PairClassification
215
+ dataset:
216
+ name: MTEB Cmnli
217
+ type: C-MTEB/CMNLI
218
+ config: default
219
+ split: validation
220
+ revision: None
221
+ metrics:
222
+ - type: cos_sim_accuracy
223
+ value: 76.7047504509922
224
+ - type: cos_sim_ap
225
+ value: 85.26649874800871
226
+ - type: cos_sim_f1
227
+ value: 78.13528724646915
228
+ - type: cos_sim_precision
229
+ value: 71.57587548638132
230
+ - type: cos_sim_recall
231
+ value: 86.01823708206688
232
+ - type: dot_accuracy
233
+ value: 70.13830426939266
234
+ - type: dot_ap
235
+ value: 77.01510412382171
236
+ - type: dot_f1
237
+ value: 73.56710042713817
238
+ - type: dot_precision
239
+ value: 63.955094991364426
240
+ - type: dot_recall
241
+ value: 86.57937806873977
242
+ - type: euclidean_accuracy
243
+ value: 75.53818400481059
244
+ - type: euclidean_ap
245
+ value: 84.34668448241264
246
+ - type: euclidean_f1
247
+ value: 77.51741608613047
248
+ - type: euclidean_precision
249
+ value: 70.65614777756399
250
+ - type: euclidean_recall
251
+ value: 85.85457096095394
252
+ - type: manhattan_accuracy
253
+ value: 75.49007817197835
254
+ - type: manhattan_ap
255
+ value: 84.40297506704299
256
+ - type: manhattan_f1
257
+ value: 77.63185324160932
258
+ - type: manhattan_precision
259
+ value: 70.03949595636637
260
+ - type: manhattan_recall
261
+ value: 87.07037643207856
262
+ - type: max_accuracy
263
+ value: 76.7047504509922
264
+ - type: max_ap
265
+ value: 85.26649874800871
266
+ - type: max_f1
267
+ value: 78.13528724646915
268
+ - task:
269
+ type: Retrieval
270
+ dataset:
271
+ name: MTEB CovidRetrieval
272
+ type: C-MTEB/CovidRetrieval
273
+ config: default
274
+ split: dev
275
+ revision: None
276
+ metrics:
277
+ - type: map_at_1
278
+ value: 69.178
279
+ - type: map_at_10
280
+ value: 77.523
281
+ - type: map_at_100
282
+ value: 77.793
283
+ - type: map_at_1000
284
+ value: 77.79899999999999
285
+ - type: map_at_3
286
+ value: 75.878
287
+ - type: map_at_5
288
+ value: 76.849
289
+ - type: mrr_at_1
290
+ value: 69.44200000000001
291
+ - type: mrr_at_10
292
+ value: 77.55
293
+ - type: mrr_at_100
294
+ value: 77.819
295
+ - type: mrr_at_1000
296
+ value: 77.826
297
+ - type: mrr_at_3
298
+ value: 75.957
299
+ - type: mrr_at_5
300
+ value: 76.916
301
+ - type: ndcg_at_1
302
+ value: 69.44200000000001
303
+ - type: ndcg_at_10
304
+ value: 81.217
305
+ - type: ndcg_at_100
306
+ value: 82.45
307
+ - type: ndcg_at_1000
308
+ value: 82.636
309
+ - type: ndcg_at_3
310
+ value: 77.931
311
+ - type: ndcg_at_5
312
+ value: 79.655
313
+ - type: precision_at_1
314
+ value: 69.44200000000001
315
+ - type: precision_at_10
316
+ value: 9.357
317
+ - type: precision_at_100
318
+ value: 0.993
319
+ - type: precision_at_1000
320
+ value: 0.101
321
+ - type: precision_at_3
322
+ value: 28.1
323
+ - type: precision_at_5
324
+ value: 17.724
325
+ - type: recall_at_1
326
+ value: 69.178
327
+ - type: recall_at_10
328
+ value: 92.624
329
+ - type: recall_at_100
330
+ value: 98.209
331
+ - type: recall_at_1000
332
+ value: 99.684
333
+ - type: recall_at_3
334
+ value: 83.772
335
+ - type: recall_at_5
336
+ value: 87.882
337
+ - task:
338
+ type: Retrieval
339
+ dataset:
340
+ name: MTEB DuRetrieval
341
+ type: C-MTEB/DuRetrieval
342
+ config: default
343
+ split: dev
344
+ revision: None
345
+ metrics:
346
+ - type: map_at_1
347
+ value: 25.163999999999998
348
+ - type: map_at_10
349
+ value: 76.386
350
+ - type: map_at_100
351
+ value: 79.339
352
+ - type: map_at_1000
353
+ value: 79.39500000000001
354
+ - type: map_at_3
355
+ value: 52.959
356
+ - type: map_at_5
357
+ value: 66.59
358
+ - type: mrr_at_1
359
+ value: 87.9
360
+ - type: mrr_at_10
361
+ value: 91.682
362
+ - type: mrr_at_100
363
+ value: 91.747
364
+ - type: mrr_at_1000
365
+ value: 91.751
366
+ - type: mrr_at_3
367
+ value: 91.267
368
+ - type: mrr_at_5
369
+ value: 91.527
370
+ - type: ndcg_at_1
371
+ value: 87.9
372
+ - type: ndcg_at_10
373
+ value: 84.569
374
+ - type: ndcg_at_100
375
+ value: 87.83800000000001
376
+ - type: ndcg_at_1000
377
+ value: 88.322
378
+ - type: ndcg_at_3
379
+ value: 83.473
380
+ - type: ndcg_at_5
381
+ value: 82.178
382
+ - type: precision_at_1
383
+ value: 87.9
384
+ - type: precision_at_10
385
+ value: 40.605000000000004
386
+ - type: precision_at_100
387
+ value: 4.752
388
+ - type: precision_at_1000
389
+ value: 0.488
390
+ - type: precision_at_3
391
+ value: 74.9
392
+ - type: precision_at_5
393
+ value: 62.96000000000001
394
+ - type: recall_at_1
395
+ value: 25.163999999999998
396
+ - type: recall_at_10
397
+ value: 85.97399999999999
398
+ - type: recall_at_100
399
+ value: 96.63000000000001
400
+ - type: recall_at_1000
401
+ value: 99.016
402
+ - type: recall_at_3
403
+ value: 55.611999999999995
404
+ - type: recall_at_5
405
+ value: 71.936
406
+ - task:
407
+ type: Retrieval
408
+ dataset:
409
+ name: MTEB EcomRetrieval
410
+ type: C-MTEB/EcomRetrieval
411
+ config: default
412
+ split: dev
413
+ revision: None
414
+ metrics:
415
+ - type: map_at_1
416
+ value: 48.6
417
+ - type: map_at_10
418
+ value: 58.831
419
+ - type: map_at_100
420
+ value: 59.427
421
+ - type: map_at_1000
422
+ value: 59.44199999999999
423
+ - type: map_at_3
424
+ value: 56.383
425
+ - type: map_at_5
426
+ value: 57.753
427
+ - type: mrr_at_1
428
+ value: 48.6
429
+ - type: mrr_at_10
430
+ value: 58.831
431
+ - type: mrr_at_100
432
+ value: 59.427
433
+ - type: mrr_at_1000
434
+ value: 59.44199999999999
435
+ - type: mrr_at_3
436
+ value: 56.383
437
+ - type: mrr_at_5
438
+ value: 57.753
439
+ - type: ndcg_at_1
440
+ value: 48.6
441
+ - type: ndcg_at_10
442
+ value: 63.951
443
+ - type: ndcg_at_100
444
+ value: 66.72200000000001
445
+ - type: ndcg_at_1000
446
+ value: 67.13900000000001
447
+ - type: ndcg_at_3
448
+ value: 58.882
449
+ - type: ndcg_at_5
450
+ value: 61.373
451
+ - type: precision_at_1
452
+ value: 48.6
453
+ - type: precision_at_10
454
+ value: 8.01
455
+ - type: precision_at_100
456
+ value: 0.928
457
+ - type: precision_at_1000
458
+ value: 0.096
459
+ - type: precision_at_3
460
+ value: 22.033
461
+ - type: precision_at_5
462
+ value: 14.44
463
+ - type: recall_at_1
464
+ value: 48.6
465
+ - type: recall_at_10
466
+ value: 80.10000000000001
467
+ - type: recall_at_100
468
+ value: 92.80000000000001
469
+ - type: recall_at_1000
470
+ value: 96.1
471
+ - type: recall_at_3
472
+ value: 66.10000000000001
473
+ - type: recall_at_5
474
+ value: 72.2
475
+ - task:
476
+ type: Classification
477
+ dataset:
478
+ name: MTEB IFlyTek
479
+ type: C-MTEB/IFlyTek-classification
480
+ config: default
481
+ split: validation
482
+ revision: None
483
+ metrics:
484
+ - type: accuracy
485
+ value: 47.36437091188918
486
+ - type: f1
487
+ value: 36.60946954228577
488
+ - task:
489
+ type: Classification
490
+ dataset:
491
+ name: MTEB JDReview
492
+ type: C-MTEB/JDReview-classification
493
+ config: default
494
+ split: test
495
+ revision: None
496
+ metrics:
497
+ - type: accuracy
498
+ value: 79.5684803001876
499
+ - type: ap
500
+ value: 42.671935929201524
501
+ - type: f1
502
+ value: 73.31912729103752
503
+ - task:
504
+ type: STS
505
+ dataset:
506
+ name: MTEB LCQMC
507
+ type: C-MTEB/LCQMC
508
+ config: default
509
+ split: test
510
+ revision: None
511
+ metrics:
512
+ - type: cos_sim_pearson
513
+ value: 68.62670112113864
514
+ - type: cos_sim_spearman
515
+ value: 75.74009123170768
516
+ - type: euclidean_pearson
517
+ value: 73.93002595958237
518
+ - type: euclidean_spearman
519
+ value: 75.35222935003587
520
+ - type: manhattan_pearson
521
+ value: 73.89870445158144
522
+ - type: manhattan_spearman
523
+ value: 75.31714936339398
524
+ - task:
525
+ type: Reranking
526
+ dataset:
527
+ name: MTEB MMarcoReranking
528
+ type: C-MTEB/Mmarco-reranking
529
+ config: default
530
+ split: dev
531
+ revision: None
532
+ metrics:
533
+ - type: map
534
+ value: 31.5372713650176
535
+ - type: mrr
536
+ value: 30.163095238095238
537
+ - task:
538
+ type: Retrieval
539
+ dataset:
540
+ name: MTEB MMarcoRetrieval
541
+ type: C-MTEB/MMarcoRetrieval
542
+ config: default
543
+ split: dev
544
+ revision: None
545
+ metrics:
546
+ - type: map_at_1
547
+ value: 65.054
548
+ - type: map_at_10
549
+ value: 74.156
550
+ - type: map_at_100
551
+ value: 74.523
552
+ - type: map_at_1000
553
+ value: 74.535
554
+ - type: map_at_3
555
+ value: 72.269
556
+ - type: map_at_5
557
+ value: 73.41
558
+ - type: mrr_at_1
559
+ value: 67.24900000000001
560
+ - type: mrr_at_10
561
+ value: 74.78399999999999
562
+ - type: mrr_at_100
563
+ value: 75.107
564
+ - type: mrr_at_1000
565
+ value: 75.117
566
+ - type: mrr_at_3
567
+ value: 73.13499999999999
568
+ - type: mrr_at_5
569
+ value: 74.13499999999999
570
+ - type: ndcg_at_1
571
+ value: 67.24900000000001
572
+ - type: ndcg_at_10
573
+ value: 77.96300000000001
574
+ - type: ndcg_at_100
575
+ value: 79.584
576
+ - type: ndcg_at_1000
577
+ value: 79.884
578
+ - type: ndcg_at_3
579
+ value: 74.342
580
+ - type: ndcg_at_5
581
+ value: 76.278
582
+ - type: precision_at_1
583
+ value: 67.24900000000001
584
+ - type: precision_at_10
585
+ value: 9.466
586
+ - type: precision_at_100
587
+ value: 1.027
588
+ - type: precision_at_1000
589
+ value: 0.105
590
+ - type: precision_at_3
591
+ value: 27.955999999999996
592
+ - type: precision_at_5
593
+ value: 17.817
594
+ - type: recall_at_1
595
+ value: 65.054
596
+ - type: recall_at_10
597
+ value: 89.113
598
+ - type: recall_at_100
599
+ value: 96.369
600
+ - type: recall_at_1000
601
+ value: 98.714
602
+ - type: recall_at_3
603
+ value: 79.45400000000001
604
+ - type: recall_at_5
605
+ value: 84.06
606
+ - task:
607
+ type: Classification
608
+ dataset:
609
+ name: MTEB MassiveIntentClassification (zh-CN)
610
+ type: mteb/amazon_massive_intent
611
+ config: zh-CN
612
+ split: test
613
+ revision: 31efe3c427b0bae9c22cbb560b8f15491cc6bed7
614
+ metrics:
615
+ - type: accuracy
616
+ value: 68.1977135171486
617
+ - type: f1
618
+ value: 67.23114308718404
619
+ - task:
620
+ type: Classification
621
+ dataset:
622
+ name: MTEB MassiveScenarioClassification (zh-CN)
623
+ type: mteb/amazon_massive_scenario
624
+ config: zh-CN
625
+ split: test
626
+ revision: 7d571f92784cd94a019292a1f45445077d0ef634
627
+ metrics:
628
+ - type: accuracy
629
+ value: 71.92669804976462
630
+ - type: f1
631
+ value: 72.90628475628779
632
+ - task:
633
+ type: Retrieval
634
+ dataset:
635
+ name: MTEB MedicalRetrieval
636
+ type: C-MTEB/MedicalRetrieval
637
+ config: default
638
+ split: dev
639
+ revision: None
640
+ metrics:
641
+ - type: map_at_1
642
+ value: 49.2
643
+ - type: map_at_10
644
+ value: 54.539
645
+ - type: map_at_100
646
+ value: 55.135
647
+ - type: map_at_1000
648
+ value: 55.19199999999999
649
+ - type: map_at_3
650
+ value: 53.383
651
+ - type: map_at_5
652
+ value: 54.142999999999994
653
+ - type: mrr_at_1
654
+ value: 49.2
655
+ - type: mrr_at_10
656
+ value: 54.539
657
+ - type: mrr_at_100
658
+ value: 55.135999999999996
659
+ - type: mrr_at_1000
660
+ value: 55.19199999999999
661
+ - type: mrr_at_3
662
+ value: 53.383
663
+ - type: mrr_at_5
664
+ value: 54.142999999999994
665
+ - type: ndcg_at_1
666
+ value: 49.2
667
+ - type: ndcg_at_10
668
+ value: 57.123000000000005
669
+ - type: ndcg_at_100
670
+ value: 60.21300000000001
671
+ - type: ndcg_at_1000
672
+ value: 61.915
673
+ - type: ndcg_at_3
674
+ value: 54.772
675
+ - type: ndcg_at_5
676
+ value: 56.157999999999994
677
+ - type: precision_at_1
678
+ value: 49.2
679
+ - type: precision_at_10
680
+ value: 6.52
681
+ - type: precision_at_100
682
+ value: 0.8009999999999999
683
+ - type: precision_at_1000
684
+ value: 0.094
685
+ - type: precision_at_3
686
+ value: 19.6
687
+ - type: precision_at_5
688
+ value: 12.44
689
+ - type: recall_at_1
690
+ value: 49.2
691
+ - type: recall_at_10
692
+ value: 65.2
693
+ - type: recall_at_100
694
+ value: 80.10000000000001
695
+ - type: recall_at_1000
696
+ value: 93.89999999999999
697
+ - type: recall_at_3
698
+ value: 58.8
699
+ - type: recall_at_5
700
+ value: 62.2
701
+ - task:
702
+ type: Classification
703
+ dataset:
704
+ name: MTEB MultilingualSentiment
705
+ type: C-MTEB/MultilingualSentiment-classification
706
+ config: default
707
+ split: validation
708
+ revision: None
709
+ metrics:
710
+ - type: accuracy
711
+ value: 63.29333333333334
712
+ - type: f1
713
+ value: 63.03293854259612
714
+ - task:
715
+ type: PairClassification
716
+ dataset:
717
+ name: MTEB Ocnli
718
+ type: C-MTEB/OCNLI
719
+ config: default
720
+ split: validation
721
+ revision: None
722
+ metrics:
723
+ - type: cos_sim_accuracy
724
+ value: 75.69030860855442
725
+ - type: cos_sim_ap
726
+ value: 80.6157833772759
727
+ - type: cos_sim_f1
728
+ value: 77.87524366471735
729
+ - type: cos_sim_precision
730
+ value: 72.3076923076923
731
+ - type: cos_sim_recall
732
+ value: 84.37170010559663
733
+ - type: dot_accuracy
734
+ value: 67.78559826746074
735
+ - type: dot_ap
736
+ value: 72.00871467527499
737
+ - type: dot_f1
738
+ value: 72.58722247394654
739
+ - type: dot_precision
740
+ value: 63.57142857142857
741
+ - type: dot_recall
742
+ value: 84.58289334741288
743
+ - type: euclidean_accuracy
744
+ value: 75.20303194369248
745
+ - type: euclidean_ap
746
+ value: 80.98587256415605
747
+ - type: euclidean_f1
748
+ value: 77.26396917148362
749
+ - type: euclidean_precision
750
+ value: 71.03631532329496
751
+ - type: euclidean_recall
752
+ value: 84.68848996832101
753
+ - type: manhattan_accuracy
754
+ value: 75.20303194369248
755
+ - type: manhattan_ap
756
+ value: 80.93460699513219
757
+ - type: manhattan_f1
758
+ value: 77.124773960217
759
+ - type: manhattan_precision
760
+ value: 67.43083003952569
761
+ - type: manhattan_recall
762
+ value: 90.07391763463569
763
+ - type: max_accuracy
764
+ value: 75.69030860855442
765
+ - type: max_ap
766
+ value: 80.98587256415605
767
+ - type: max_f1
768
+ value: 77.87524366471735
769
+ - task:
770
+ type: Classification
771
+ dataset:
772
+ name: MTEB OnlineShopping
773
+ type: C-MTEB/OnlineShopping-classification
774
+ config: default
775
+ split: test
776
+ revision: None
777
+ metrics:
778
+ - type: accuracy
779
+ value: 87.00000000000001
780
+ - type: ap
781
+ value: 83.24372135949511
782
+ - type: f1
783
+ value: 86.95554191530607
784
+ - task:
785
+ type: STS
786
+ dataset:
787
+ name: MTEB PAWSX
788
+ type: C-MTEB/PAWSX
789
+ config: default
790
+ split: test
791
+ revision: None
792
+ metrics:
793
+ - type: cos_sim_pearson
794
+ value: 37.57616811591219
795
+ - type: cos_sim_spearman
796
+ value: 41.490259084930045
797
+ - type: euclidean_pearson
798
+ value: 38.9155043692188
799
+ - type: euclidean_spearman
800
+ value: 39.16056534305623
801
+ - type: manhattan_pearson
802
+ value: 38.76569892264335
803
+ - type: manhattan_spearman
804
+ value: 38.99891685590743
805
+ - task:
806
+ type: STS
807
+ dataset:
808
+ name: MTEB QBQTC
809
+ type: C-MTEB/QBQTC
810
+ config: default
811
+ split: test
812
+ revision: None
813
+ metrics:
814
+ - type: cos_sim_pearson
815
+ value: 35.44858610359665
816
+ - type: cos_sim_spearman
817
+ value: 38.11128146262466
818
+ - type: euclidean_pearson
819
+ value: 31.928644189822457
820
+ - type: euclidean_spearman
821
+ value: 34.384936631696554
822
+ - type: manhattan_pearson
823
+ value: 31.90586687414376
824
+ - type: manhattan_spearman
825
+ value: 34.35770153777186
826
+ - task:
827
+ type: STS
828
+ dataset:
829
+ name: MTEB STS22 (zh)
830
+ type: mteb/sts22-crosslingual-sts
831
+ config: zh
832
+ split: test
833
+ revision: 6d1ba47164174a496b7fa5d3569dae26a6813b80
834
+ metrics:
835
+ - type: cos_sim_pearson
836
+ value: 66.54931957553592
837
+ - type: cos_sim_spearman
838
+ value: 69.25068863016632
839
+ - type: euclidean_pearson
840
+ value: 50.26525596106869
841
+ - type: euclidean_spearman
842
+ value: 63.83352741910006
843
+ - type: manhattan_pearson
844
+ value: 49.98798282198196
845
+ - type: manhattan_spearman
846
+ value: 63.87649521907841
847
+ - task:
848
+ type: STS
849
+ dataset:
850
+ name: MTEB STSB
851
+ type: C-MTEB/STSB
852
+ config: default
853
+ split: test
854
+ revision: None
855
+ metrics:
856
+ - type: cos_sim_pearson
857
+ value: 82.52782476625825
858
+ - type: cos_sim_spearman
859
+ value: 82.55618986168398
860
+ - type: euclidean_pearson
861
+ value: 78.48190631687673
862
+ - type: euclidean_spearman
863
+ value: 78.39479731354655
864
+ - type: manhattan_pearson
865
+ value: 78.51176592165885
866
+ - type: manhattan_spearman
867
+ value: 78.42363787303265
868
+ - task:
869
+ type: Reranking
870
+ dataset:
871
+ name: MTEB T2Reranking
872
+ type: C-MTEB/T2Reranking
873
+ config: default
874
+ split: dev
875
+ revision: None
876
+ metrics:
877
+ - type: map
878
+ value: 67.36693873615643
879
+ - type: mrr
880
+ value: 77.83847701797939
881
+ - task:
882
+ type: Retrieval
883
+ dataset:
884
+ name: MTEB T2Retrieval
885
+ type: C-MTEB/T2Retrieval
886
+ config: default
887
+ split: dev
888
+ revision: None
889
+ metrics:
890
+ - type: map_at_1
891
+ value: 25.795
892
+ - type: map_at_10
893
+ value: 72.258
894
+ - type: map_at_100
895
+ value: 76.049
896
+ - type: map_at_1000
897
+ value: 76.134
898
+ - type: map_at_3
899
+ value: 50.697
900
+ - type: map_at_5
901
+ value: 62.324999999999996
902
+ - type: mrr_at_1
903
+ value: 86.634
904
+ - type: mrr_at_10
905
+ value: 89.792
906
+ - type: mrr_at_100
907
+ value: 89.91900000000001
908
+ - type: mrr_at_1000
909
+ value: 89.923
910
+ - type: mrr_at_3
911
+ value: 89.224
912
+ - type: mrr_at_5
913
+ value: 89.608
914
+ - type: ndcg_at_1
915
+ value: 86.634
916
+ - type: ndcg_at_10
917
+ value: 80.589
918
+ - type: ndcg_at_100
919
+ value: 84.812
920
+ - type: ndcg_at_1000
921
+ value: 85.662
922
+ - type: ndcg_at_3
923
+ value: 82.169
924
+ - type: ndcg_at_5
925
+ value: 80.619
926
+ - type: precision_at_1
927
+ value: 86.634
928
+ - type: precision_at_10
929
+ value: 40.389
930
+ - type: precision_at_100
931
+ value: 4.93
932
+ - type: precision_at_1000
933
+ value: 0.513
934
+ - type: precision_at_3
935
+ value: 72.104
936
+ - type: precision_at_5
937
+ value: 60.425
938
+ - type: recall_at_1
939
+ value: 25.795
940
+ - type: recall_at_10
941
+ value: 79.565
942
+ - type: recall_at_100
943
+ value: 93.24799999999999
944
+ - type: recall_at_1000
945
+ value: 97.595
946
+ - type: recall_at_3
947
+ value: 52.583999999999996
948
+ - type: recall_at_5
949
+ value: 66.175
950
+ - task:
951
+ type: Classification
952
+ dataset:
953
+ name: MTEB TNews
954
+ type: C-MTEB/TNews-classification
955
+ config: default
956
+ split: validation
957
+ revision: None
958
+ metrics:
959
+ - type: accuracy
960
+ value: 47.648999999999994
961
+ - type: f1
962
+ value: 46.28925837008413
963
+ - task:
964
+ type: Clustering
965
+ dataset:
966
+ name: MTEB ThuNewsClusteringP2P
967
+ type: C-MTEB/ThuNewsClusteringP2P
968
+ config: default
969
+ split: test
970
+ revision: None
971
+ metrics:
972
+ - type: v_measure
973
+ value: 54.07641891287953
974
+ - task:
975
+ type: Clustering
976
+ dataset:
977
+ name: MTEB ThuNewsClusteringS2S
978
+ type: C-MTEB/ThuNewsClusteringS2S
979
+ config: default
980
+ split: test
981
+ revision: None
982
+ metrics:
983
+ - type: v_measure
984
+ value: 53.423702062353954
985
+ - task:
986
+ type: Retrieval
987
+ dataset:
988
+ name: MTEB VideoRetrieval
989
+ type: C-MTEB/VideoRetrieval
990
+ config: default
991
+ split: dev
992
+ revision: None
993
+ metrics:
994
+ - type: map_at_1
995
+ value: 55.7
996
+ - type: map_at_10
997
+ value: 65.923
998
+ - type: map_at_100
999
+ value: 66.42
1000
+ - type: map_at_1000
1001
+ value: 66.431
1002
+ - type: map_at_3
1003
+ value: 63.9
1004
+ - type: map_at_5
1005
+ value: 65.225
1006
+ - type: mrr_at_1
1007
+ value: 55.60000000000001
1008
+ - type: mrr_at_10
1009
+ value: 65.873
1010
+ - type: mrr_at_100
1011
+ value: 66.36999999999999
1012
+ - type: mrr_at_1000
1013
+ value: 66.381
1014
+ - type: mrr_at_3
1015
+ value: 63.849999999999994
1016
+ - type: mrr_at_5
1017
+ value: 65.17500000000001
1018
+ - type: ndcg_at_1
1019
+ value: 55.7
1020
+ - type: ndcg_at_10
1021
+ value: 70.621
1022
+ - type: ndcg_at_100
1023
+ value: 72.944
1024
+ - type: ndcg_at_1000
1025
+ value: 73.25399999999999
1026
+ - type: ndcg_at_3
1027
+ value: 66.547
1028
+ - type: ndcg_at_5
1029
+ value: 68.93599999999999
1030
+ - type: precision_at_1
1031
+ value: 55.7
1032
+ - type: precision_at_10
1033
+ value: 8.52
1034
+ - type: precision_at_100
1035
+ value: 0.958
1036
+ - type: precision_at_1000
1037
+ value: 0.098
1038
+ - type: precision_at_3
1039
+ value: 24.733
1040
+ - type: precision_at_5
1041
+ value: 16
1042
+ - type: recall_at_1
1043
+ value: 55.7
1044
+ - type: recall_at_10
1045
+ value: 85.2
1046
+ - type: recall_at_100
1047
+ value: 95.8
1048
+ - type: recall_at_1000
1049
+ value: 98.3
1050
+ - type: recall_at_3
1051
+ value: 74.2
1052
+ - type: recall_at_5
1053
+ value: 80
1054
+ - task:
1055
+ type: Classification
1056
+ dataset:
1057
+ name: MTEB Waimai
1058
+ type: C-MTEB/waimai-classification
1059
+ config: default
1060
+ split: test
1061
+ revision: None
1062
+ metrics:
1063
+ - type: accuracy
1064
+ value: 84.54
1065
+ - type: ap
1066
+ value: 66.13603199670062
1067
+ - type: f1
1068
+ value: 82.61420654584116
1069
+ ---
1070
+
1071
+ # phate334/jina-embeddings-v2-base-zh-Q8_0-GGUF
1072
+ This model was converted to GGUF format from [`jinaai/jina-embeddings-v2-base-zh`](https://huggingface.co/jinaai/jina-embeddings-v2-base-zh) using llama.cpp via the ggml.ai's [GGUF-my-repo](https://huggingface.co/spaces/ggml-org/gguf-my-repo) space.
1073
+ Refer to the [original model card](https://huggingface.co/jinaai/jina-embeddings-v2-base-zh) for more details on the model.
1074
+
1075
+ ## Use with llama.cpp
1076
+ Install llama.cpp through brew (works on Mac and Linux)
1077
+
1078
+ ```bash
1079
+ brew install llama.cpp
1080
+
1081
+ ```
1082
+ Invoke the llama.cpp server or the CLI.
1083
+
1084
+ ### CLI:
1085
+ ```bash
1086
+ llama-cli --hf-repo phate334/jina-embeddings-v2-base-zh-Q8_0-GGUF --hf-file jina-embeddings-v2-base-zh-q8_0.gguf -p "The meaning to life and the universe is"
1087
+ ```
1088
+
1089
+ ### Server:
1090
+ ```bash
1091
+ llama-server --hf-repo phate334/jina-embeddings-v2-base-zh-Q8_0-GGUF --hf-file jina-embeddings-v2-base-zh-q8_0.gguf -c 2048
1092
+ ```
1093
+
1094
+ Note: You can also use this checkpoint directly through the [usage steps](https://github.com/ggerganov/llama.cpp?tab=readme-ov-file#usage) listed in the Llama.cpp repo as well.
1095
+
1096
+ Step 1: Clone llama.cpp from GitHub.
1097
+ ```
1098
+ git clone https://github.com/ggerganov/llama.cpp
1099
+ ```
1100
+
1101
+ Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
1102
+ ```
1103
+ cd llama.cpp && LLAMA_CURL=1 make
1104
+ ```
1105
+
1106
+ Step 3: Run inference through the main binary.
1107
+ ```
1108
+ ./llama-cli --hf-repo phate334/jina-embeddings-v2-base-zh-Q8_0-GGUF --hf-file jina-embeddings-v2-base-zh-q8_0.gguf -p "The meaning to life and the universe is"
1109
+ ```
1110
+ or
1111
+ ```
1112
+ ./llama-server --hf-repo phate334/jina-embeddings-v2-base-zh-Q8_0-GGUF --hf-file jina-embeddings-v2-base-zh-q8_0.gguf -c 2048
1113
+ ```