tomaarsen HF Staff commited on
Commit
d9d6c87
·
verified ·
1 Parent(s): cc31219

Add new SparseEncoder model

Browse files
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "word_embedding_dimension": 30522
5
+ }
README.md ADDED
@@ -0,0 +1,1731 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:99000
12
+ - loss:SpladeLoss
13
+ - loss:SparseMultipleNegativesRankingLoss
14
+ - loss:FlopsLoss
15
+ base_model: distilbert/distilbert-base-uncased
16
+ widget:
17
+ - text: 'The term emergent literacy signals a belief that, in a literate society,
18
+ young children even one and two year olds, are in the process of becoming literate”.
19
+ ... Gray (1956:21) notes: Functional literacy is used for the training of adults
20
+ to ''meet independently the reading and writing demands placed on them''.'
21
+ - text: Rey is seemingly confirmed as being The Chosen One per a quote by a Lucasfilm
22
+ production designer who worked on The Rise of Skywalker.
23
+ - text: are union gun safes fireproof?
24
+ - text: Fruit is an essential part of a healthy diet — and may aid weight loss. Most
25
+ fruits are low in calories while high in nutrients and fiber, which can boost
26
+ your fullness. Keep in mind that it's best to eat fruits whole rather than juiced.
27
+ What's more, simply eating fruit is not the key to weight loss.
28
+ - text: Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate
29
+ or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks
30
+ for chronic sinusitis.
31
+ datasets:
32
+ - sentence-transformers/gooaq
33
+ pipeline_tag: feature-extraction
34
+ library_name: sentence-transformers
35
+ metrics:
36
+ - dot_accuracy@1
37
+ - dot_accuracy@3
38
+ - dot_accuracy@5
39
+ - dot_accuracy@10
40
+ - dot_precision@1
41
+ - dot_precision@3
42
+ - dot_precision@5
43
+ - dot_precision@10
44
+ - dot_recall@1
45
+ - dot_recall@3
46
+ - dot_recall@5
47
+ - dot_recall@10
48
+ - dot_ndcg@10
49
+ - dot_mrr@10
50
+ - dot_map@100
51
+ - query_active_dims
52
+ - query_sparsity_ratio
53
+ - corpus_active_dims
54
+ - corpus_sparsity_ratio
55
+ co2_eq_emissions:
56
+ emissions: 13.27850984546157
57
+ energy_consumed: 0.03416115647838594
58
+ source: codecarbon
59
+ training_type: fine-tuning
60
+ on_cloud: false
61
+ cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
62
+ ram_total_size: 31.777088165283203
63
+ hours_used: 0.142
64
+ hardware_used: 1 x NVIDIA GeForce RTX 3090
65
+ model-index:
66
+ - name: splade-distilbert-base-uncased trained on GooAQ
67
+ results:
68
+ - task:
69
+ type: sparse-information-retrieval
70
+ name: Sparse Information Retrieval
71
+ dataset:
72
+ name: NanoMSMARCO
73
+ type: NanoMSMARCO
74
+ metrics:
75
+ - type: dot_accuracy@1
76
+ value: 0.3
77
+ name: Dot Accuracy@1
78
+ - type: dot_accuracy@3
79
+ value: 0.48
80
+ name: Dot Accuracy@3
81
+ - type: dot_accuracy@5
82
+ value: 0.62
83
+ name: Dot Accuracy@5
84
+ - type: dot_accuracy@10
85
+ value: 0.72
86
+ name: Dot Accuracy@10
87
+ - type: dot_precision@1
88
+ value: 0.3
89
+ name: Dot Precision@1
90
+ - type: dot_precision@3
91
+ value: 0.16
92
+ name: Dot Precision@3
93
+ - type: dot_precision@5
94
+ value: 0.124
95
+ name: Dot Precision@5
96
+ - type: dot_precision@10
97
+ value: 0.07200000000000001
98
+ name: Dot Precision@10
99
+ - type: dot_recall@1
100
+ value: 0.3
101
+ name: Dot Recall@1
102
+ - type: dot_recall@3
103
+ value: 0.48
104
+ name: Dot Recall@3
105
+ - type: dot_recall@5
106
+ value: 0.62
107
+ name: Dot Recall@5
108
+ - type: dot_recall@10
109
+ value: 0.72
110
+ name: Dot Recall@10
111
+ - type: dot_ndcg@10
112
+ value: 0.4923029036903167
113
+ name: Dot Ndcg@10
114
+ - type: dot_mrr@10
115
+ value: 0.4212222222222222
116
+ name: Dot Mrr@10
117
+ - type: dot_map@100
118
+ value: 0.4327140646742442
119
+ name: Dot Map@100
120
+ - type: query_active_dims
121
+ value: 129.6999969482422
122
+ name: Query Active Dims
123
+ - type: query_sparsity_ratio
124
+ value: 0.9957506062201611
125
+ name: Query Sparsity Ratio
126
+ - type: corpus_active_dims
127
+ value: 389.0450134277344
128
+ name: Corpus Active Dims
129
+ - type: corpus_sparsity_ratio
130
+ value: 0.9872536198994911
131
+ name: Corpus Sparsity Ratio
132
+ - type: dot_accuracy@1
133
+ value: 0.32
134
+ name: Dot Accuracy@1
135
+ - type: dot_accuracy@3
136
+ value: 0.48
137
+ name: Dot Accuracy@3
138
+ - type: dot_accuracy@5
139
+ value: 0.62
140
+ name: Dot Accuracy@5
141
+ - type: dot_accuracy@10
142
+ value: 0.7
143
+ name: Dot Accuracy@10
144
+ - type: dot_precision@1
145
+ value: 0.32
146
+ name: Dot Precision@1
147
+ - type: dot_precision@3
148
+ value: 0.16
149
+ name: Dot Precision@3
150
+ - type: dot_precision@5
151
+ value: 0.124
152
+ name: Dot Precision@5
153
+ - type: dot_precision@10
154
+ value: 0.07
155
+ name: Dot Precision@10
156
+ - type: dot_recall@1
157
+ value: 0.32
158
+ name: Dot Recall@1
159
+ - type: dot_recall@3
160
+ value: 0.48
161
+ name: Dot Recall@3
162
+ - type: dot_recall@5
163
+ value: 0.62
164
+ name: Dot Recall@5
165
+ - type: dot_recall@10
166
+ value: 0.7
167
+ name: Dot Recall@10
168
+ - type: dot_ndcg@10
169
+ value: 0.497755451311688
170
+ name: Dot Ndcg@10
171
+ - type: dot_mrr@10
172
+ value: 0.4339126984126984
173
+ name: Dot Mrr@10
174
+ - type: dot_map@100
175
+ value: 0.44682157418631346
176
+ name: Dot Map@100
177
+ - type: query_active_dims
178
+ value: 121.5999984741211
179
+ name: Query Active Dims
180
+ - type: query_sparsity_ratio
181
+ value: 0.9960159885173278
182
+ name: Query Sparsity Ratio
183
+ - type: corpus_active_dims
184
+ value: 397.7707824707031
185
+ name: Corpus Active Dims
186
+ - type: corpus_sparsity_ratio
187
+ value: 0.9869677353230226
188
+ name: Corpus Sparsity Ratio
189
+ - task:
190
+ type: sparse-information-retrieval
191
+ name: Sparse Information Retrieval
192
+ dataset:
193
+ name: NanoNFCorpus
194
+ type: NanoNFCorpus
195
+ metrics:
196
+ - type: dot_accuracy@1
197
+ value: 0.36
198
+ name: Dot Accuracy@1
199
+ - type: dot_accuracy@3
200
+ value: 0.44
201
+ name: Dot Accuracy@3
202
+ - type: dot_accuracy@5
203
+ value: 0.46
204
+ name: Dot Accuracy@5
205
+ - type: dot_accuracy@10
206
+ value: 0.54
207
+ name: Dot Accuracy@10
208
+ - type: dot_precision@1
209
+ value: 0.36
210
+ name: Dot Precision@1
211
+ - type: dot_precision@3
212
+ value: 0.2733333333333333
213
+ name: Dot Precision@3
214
+ - type: dot_precision@5
215
+ value: 0.21600000000000003
216
+ name: Dot Precision@5
217
+ - type: dot_precision@10
218
+ value: 0.182
219
+ name: Dot Precision@10
220
+ - type: dot_recall@1
221
+ value: 0.016695854443026216
222
+ name: Dot Recall@1
223
+ - type: dot_recall@3
224
+ value: 0.03106456335603726
225
+ name: Dot Recall@3
226
+ - type: dot_recall@5
227
+ value: 0.039511512430362564
228
+ name: Dot Recall@5
229
+ - type: dot_recall@10
230
+ value: 0.05434224491570579
231
+ name: Dot Recall@10
232
+ - type: dot_ndcg@10
233
+ value: 0.21590295298422338
234
+ name: Dot Ndcg@10
235
+ - type: dot_mrr@10
236
+ value: 0.41285714285714287
237
+ name: Dot Mrr@10
238
+ - type: dot_map@100
239
+ value: 0.07349060309007502
240
+ name: Dot Map@100
241
+ - type: query_active_dims
242
+ value: 221.5399932861328
243
+ name: Query Active Dims
244
+ - type: query_sparsity_ratio
245
+ value: 0.9927416292088942
246
+ name: Query Sparsity Ratio
247
+ - type: corpus_active_dims
248
+ value: 851.602783203125
249
+ name: Corpus Active Dims
250
+ - type: corpus_sparsity_ratio
251
+ value: 0.9720987227834637
252
+ name: Corpus Sparsity Ratio
253
+ - type: dot_accuracy@1
254
+ value: 0.32
255
+ name: Dot Accuracy@1
256
+ - type: dot_accuracy@3
257
+ value: 0.42
258
+ name: Dot Accuracy@3
259
+ - type: dot_accuracy@5
260
+ value: 0.48
261
+ name: Dot Accuracy@5
262
+ - type: dot_accuracy@10
263
+ value: 0.54
264
+ name: Dot Accuracy@10
265
+ - type: dot_precision@1
266
+ value: 0.32
267
+ name: Dot Precision@1
268
+ - type: dot_precision@3
269
+ value: 0.25999999999999995
270
+ name: Dot Precision@3
271
+ - type: dot_precision@5
272
+ value: 0.22399999999999998
273
+ name: Dot Precision@5
274
+ - type: dot_precision@10
275
+ value: 0.184
276
+ name: Dot Precision@10
277
+ - type: dot_recall@1
278
+ value: 0.011195854443026216
279
+ name: Dot Recall@1
280
+ - type: dot_recall@3
281
+ value: 0.0305213668117608
282
+ name: Dot Recall@3
283
+ - type: dot_recall@5
284
+ value: 0.04137763819101165
285
+ name: Dot Recall@5
286
+ - type: dot_recall@10
287
+ value: 0.054323362586038954
288
+ name: Dot Recall@10
289
+ - type: dot_ndcg@10
290
+ value: 0.21092767617388347
291
+ name: Dot Ndcg@10
292
+ - type: dot_mrr@10
293
+ value: 0.388079365079365
294
+ name: Dot Mrr@10
295
+ - type: dot_map@100
296
+ value: 0.07135787831660151
297
+ name: Dot Map@100
298
+ - type: query_active_dims
299
+ value: 199.74000549316406
300
+ name: Query Active Dims
301
+ - type: query_sparsity_ratio
302
+ value: 0.9934558677185911
303
+ name: Query Sparsity Ratio
304
+ - type: corpus_active_dims
305
+ value: 879.7700805664062
306
+ name: Corpus Active Dims
307
+ - type: corpus_sparsity_ratio
308
+ value: 0.9711758705010679
309
+ name: Corpus Sparsity Ratio
310
+ - task:
311
+ type: sparse-information-retrieval
312
+ name: Sparse Information Retrieval
313
+ dataset:
314
+ name: NanoNQ
315
+ type: NanoNQ
316
+ metrics:
317
+ - type: dot_accuracy@1
318
+ value: 0.24
319
+ name: Dot Accuracy@1
320
+ - type: dot_accuracy@3
321
+ value: 0.5
322
+ name: Dot Accuracy@3
323
+ - type: dot_accuracy@5
324
+ value: 0.56
325
+ name: Dot Accuracy@5
326
+ - type: dot_accuracy@10
327
+ value: 0.7
328
+ name: Dot Accuracy@10
329
+ - type: dot_precision@1
330
+ value: 0.24
331
+ name: Dot Precision@1
332
+ - type: dot_precision@3
333
+ value: 0.16666666666666663
334
+ name: Dot Precision@3
335
+ - type: dot_precision@5
336
+ value: 0.11200000000000002
337
+ name: Dot Precision@5
338
+ - type: dot_precision@10
339
+ value: 0.07
340
+ name: Dot Precision@10
341
+ - type: dot_recall@1
342
+ value: 0.23
343
+ name: Dot Recall@1
344
+ - type: dot_recall@3
345
+ value: 0.46
346
+ name: Dot Recall@3
347
+ - type: dot_recall@5
348
+ value: 0.52
349
+ name: Dot Recall@5
350
+ - type: dot_recall@10
351
+ value: 0.64
352
+ name: Dot Recall@10
353
+ - type: dot_ndcg@10
354
+ value: 0.43543954013123615
355
+ name: Dot Ndcg@10
356
+ - type: dot_mrr@10
357
+ value: 0.3874365079365079
358
+ name: Dot Mrr@10
359
+ - type: dot_map@100
360
+ value: 0.37672138136649846
361
+ name: Dot Map@100
362
+ - type: query_active_dims
363
+ value: 103.12000274658203
364
+ name: Query Active Dims
365
+ - type: query_sparsity_ratio
366
+ value: 0.9966214532879044
367
+ name: Query Sparsity Ratio
368
+ - type: corpus_active_dims
369
+ value: 603.1914672851562
370
+ name: Corpus Active Dims
371
+ - type: corpus_sparsity_ratio
372
+ value: 0.9802374855092997
373
+ name: Corpus Sparsity Ratio
374
+ - type: dot_accuracy@1
375
+ value: 0.28
376
+ name: Dot Accuracy@1
377
+ - type: dot_accuracy@3
378
+ value: 0.48
379
+ name: Dot Accuracy@3
380
+ - type: dot_accuracy@5
381
+ value: 0.56
382
+ name: Dot Accuracy@5
383
+ - type: dot_accuracy@10
384
+ value: 0.72
385
+ name: Dot Accuracy@10
386
+ - type: dot_precision@1
387
+ value: 0.28
388
+ name: Dot Precision@1
389
+ - type: dot_precision@3
390
+ value: 0.16
391
+ name: Dot Precision@3
392
+ - type: dot_precision@5
393
+ value: 0.11599999999999999
394
+ name: Dot Precision@5
395
+ - type: dot_precision@10
396
+ value: 0.07400000000000001
397
+ name: Dot Precision@10
398
+ - type: dot_recall@1
399
+ value: 0.26
400
+ name: Dot Recall@1
401
+ - type: dot_recall@3
402
+ value: 0.45
403
+ name: Dot Recall@3
404
+ - type: dot_recall@5
405
+ value: 0.52
406
+ name: Dot Recall@5
407
+ - type: dot_recall@10
408
+ value: 0.67
409
+ name: Dot Recall@10
410
+ - type: dot_ndcg@10
411
+ value: 0.45351079818093326
412
+ name: Dot Ndcg@10
413
+ - type: dot_mrr@10
414
+ value: 0.40330158730158727
415
+ name: Dot Mrr@10
416
+ - type: dot_map@100
417
+ value: 0.3886037227739609
418
+ name: Dot Map@100
419
+ - type: query_active_dims
420
+ value: 97.54000091552734
421
+ name: Query Active Dims
422
+ - type: query_sparsity_ratio
423
+ value: 0.996804272298161
424
+ name: Query Sparsity Ratio
425
+ - type: corpus_active_dims
426
+ value: 625.50048828125
427
+ name: Corpus Active Dims
428
+ - type: corpus_sparsity_ratio
429
+ value: 0.9795065694161179
430
+ name: Corpus Sparsity Ratio
431
+ - task:
432
+ type: sparse-nano-beir
433
+ name: Sparse Nano BEIR
434
+ dataset:
435
+ name: NanoBEIR mean
436
+ type: NanoBEIR_mean
437
+ metrics:
438
+ - type: dot_accuracy@1
439
+ value: 0.3
440
+ name: Dot Accuracy@1
441
+ - type: dot_accuracy@3
442
+ value: 0.47333333333333333
443
+ name: Dot Accuracy@3
444
+ - type: dot_accuracy@5
445
+ value: 0.5466666666666667
446
+ name: Dot Accuracy@5
447
+ - type: dot_accuracy@10
448
+ value: 0.6533333333333333
449
+ name: Dot Accuracy@10
450
+ - type: dot_precision@1
451
+ value: 0.3
452
+ name: Dot Precision@1
453
+ - type: dot_precision@3
454
+ value: 0.19999999999999998
455
+ name: Dot Precision@3
456
+ - type: dot_precision@5
457
+ value: 0.1506666666666667
458
+ name: Dot Precision@5
459
+ - type: dot_precision@10
460
+ value: 0.108
461
+ name: Dot Precision@10
462
+ - type: dot_recall@1
463
+ value: 0.18223195148100876
464
+ name: Dot Recall@1
465
+ - type: dot_recall@3
466
+ value: 0.32368818778534575
467
+ name: Dot Recall@3
468
+ - type: dot_recall@5
469
+ value: 0.39317050414345417
470
+ name: Dot Recall@5
471
+ - type: dot_recall@10
472
+ value: 0.47144741497190196
473
+ name: Dot Recall@10
474
+ - type: dot_ndcg@10
475
+ value: 0.3812151322685921
476
+ name: Dot Ndcg@10
477
+ - type: dot_mrr@10
478
+ value: 0.4071719576719577
479
+ name: Dot Mrr@10
480
+ - type: dot_map@100
481
+ value: 0.2943086830436059
482
+ name: Dot Map@100
483
+ - type: query_active_dims
484
+ value: 151.45333099365234
485
+ name: Query Active Dims
486
+ - type: query_sparsity_ratio
487
+ value: 0.9950378962389864
488
+ name: Query Sparsity Ratio
489
+ - type: corpus_active_dims
490
+ value: 576.610088197042
491
+ name: Corpus Active Dims
492
+ - type: corpus_sparsity_ratio
493
+ value: 0.9811083779504279
494
+ name: Corpus Sparsity Ratio
495
+ - type: dot_accuracy@1
496
+ value: 0.3591522762951334
497
+ name: Dot Accuracy@1
498
+ - type: dot_accuracy@3
499
+ value: 0.5289795918367346
500
+ name: Dot Accuracy@3
501
+ - type: dot_accuracy@5
502
+ value: 0.6121507064364207
503
+ name: Dot Accuracy@5
504
+ - type: dot_accuracy@10
505
+ value: 0.7046153846153848
506
+ name: Dot Accuracy@10
507
+ - type: dot_precision@1
508
+ value: 0.3591522762951334
509
+ name: Dot Precision@1
510
+ - type: dot_precision@3
511
+ value: 0.2408895866038723
512
+ name: Dot Precision@3
513
+ - type: dot_precision@5
514
+ value: 0.19369544740973316
515
+ name: Dot Precision@5
516
+ - type: dot_precision@10
517
+ value: 0.14295133437990584
518
+ name: Dot Precision@10
519
+ - type: dot_recall@1
520
+ value: 0.20232124722865225
521
+ name: Dot Recall@1
522
+ - type: dot_recall@3
523
+ value: 0.31785179276573494
524
+ name: Dot Recall@3
525
+ - type: dot_recall@5
526
+ value: 0.3843850663750471
527
+ name: Dot Recall@5
528
+ - type: dot_recall@10
529
+ value: 0.4668945859961475
530
+ name: Dot Recall@10
531
+ - type: dot_ndcg@10
532
+ value: 0.4149846764819604
533
+ name: Dot Ndcg@10
534
+ - type: dot_mrr@10
535
+ value: 0.46609898831327395
536
+ name: Dot Mrr@10
537
+ - type: dot_map@100
538
+ value: 0.34193786183886477
539
+ name: Dot Map@100
540
+ - type: query_active_dims
541
+ value: 207.32511528026893
542
+ name: Query Active Dims
543
+ - type: query_sparsity_ratio
544
+ value: 0.9932073548496079
545
+ name: Query Sparsity Ratio
546
+ - type: corpus_active_dims
547
+ value: 571.211992111076
548
+ name: Corpus Active Dims
549
+ - type: corpus_sparsity_ratio
550
+ value: 0.9812852371367842
551
+ name: Corpus Sparsity Ratio
552
+ - task:
553
+ type: sparse-information-retrieval
554
+ name: Sparse Information Retrieval
555
+ dataset:
556
+ name: NanoClimateFEVER
557
+ type: NanoClimateFEVER
558
+ metrics:
559
+ - type: dot_accuracy@1
560
+ value: 0.18
561
+ name: Dot Accuracy@1
562
+ - type: dot_accuracy@3
563
+ value: 0.32
564
+ name: Dot Accuracy@3
565
+ - type: dot_accuracy@5
566
+ value: 0.4
567
+ name: Dot Accuracy@5
568
+ - type: dot_accuracy@10
569
+ value: 0.52
570
+ name: Dot Accuracy@10
571
+ - type: dot_precision@1
572
+ value: 0.18
573
+ name: Dot Precision@1
574
+ - type: dot_precision@3
575
+ value: 0.12
576
+ name: Dot Precision@3
577
+ - type: dot_precision@5
578
+ value: 0.08800000000000001
579
+ name: Dot Precision@5
580
+ - type: dot_precision@10
581
+ value: 0.06199999999999999
582
+ name: Dot Precision@10
583
+ - type: dot_recall@1
584
+ value: 0.09166666666666666
585
+ name: Dot Recall@1
586
+ - type: dot_recall@3
587
+ value: 0.16333333333333333
588
+ name: Dot Recall@3
589
+ - type: dot_recall@5
590
+ value: 0.19
591
+ name: Dot Recall@5
592
+ - type: dot_recall@10
593
+ value: 0.24733333333333335
594
+ name: Dot Recall@10
595
+ - type: dot_ndcg@10
596
+ value: 0.20663969061747678
597
+ name: Dot Ndcg@10
598
+ - type: dot_mrr@10
599
+ value: 0.2785238095238095
600
+ name: Dot Mrr@10
601
+ - type: dot_map@100
602
+ value: 0.16465402486405106
603
+ name: Dot Map@100
604
+ - type: query_active_dims
605
+ value: 243.5399932861328
606
+ name: Query Active Dims
607
+ - type: query_sparsity_ratio
608
+ value: 0.9920208376487081
609
+ name: Query Sparsity Ratio
610
+ - type: corpus_active_dims
611
+ value: 704.9307250976562
612
+ name: Corpus Active Dims
613
+ - type: corpus_sparsity_ratio
614
+ value: 0.976904176492443
615
+ name: Corpus Sparsity Ratio
616
+ - task:
617
+ type: sparse-information-retrieval
618
+ name: Sparse Information Retrieval
619
+ dataset:
620
+ name: NanoDBPedia
621
+ type: NanoDBPedia
622
+ metrics:
623
+ - type: dot_accuracy@1
624
+ value: 0.54
625
+ name: Dot Accuracy@1
626
+ - type: dot_accuracy@3
627
+ value: 0.74
628
+ name: Dot Accuracy@3
629
+ - type: dot_accuracy@5
630
+ value: 0.82
631
+ name: Dot Accuracy@5
632
+ - type: dot_accuracy@10
633
+ value: 0.9
634
+ name: Dot Accuracy@10
635
+ - type: dot_precision@1
636
+ value: 0.54
637
+ name: Dot Precision@1
638
+ - type: dot_precision@3
639
+ value: 0.4666666666666666
640
+ name: Dot Precision@3
641
+ - type: dot_precision@5
642
+ value: 0.43200000000000005
643
+ name: Dot Precision@5
644
+ - type: dot_precision@10
645
+ value: 0.41200000000000003
646
+ name: Dot Precision@10
647
+ - type: dot_recall@1
648
+ value: 0.055010013413515094
649
+ name: Dot Recall@1
650
+ - type: dot_recall@3
651
+ value: 0.11580740263202617
652
+ name: Dot Recall@3
653
+ - type: dot_recall@5
654
+ value: 0.15730927980831147
655
+ name: Dot Recall@5
656
+ - type: dot_recall@10
657
+ value: 0.2871622888330674
658
+ name: Dot Recall@10
659
+ - type: dot_ndcg@10
660
+ value: 0.49104405587222694
661
+ name: Dot Ndcg@10
662
+ - type: dot_mrr@10
663
+ value: 0.6685238095238095
664
+ name: Dot Mrr@10
665
+ - type: dot_map@100
666
+ value: 0.34192873131869617
667
+ name: Dot Map@100
668
+ - type: query_active_dims
669
+ value: 180.3000030517578
670
+ name: Query Active Dims
671
+ - type: query_sparsity_ratio
672
+ value: 0.994092785431762
673
+ name: Query Sparsity Ratio
674
+ - type: corpus_active_dims
675
+ value: 549.1513671875
676
+ name: Corpus Active Dims
677
+ - type: corpus_sparsity_ratio
678
+ value: 0.9820080149666633
679
+ name: Corpus Sparsity Ratio
680
+ - task:
681
+ type: sparse-information-retrieval
682
+ name: Sparse Information Retrieval
683
+ dataset:
684
+ name: NanoFEVER
685
+ type: NanoFEVER
686
+ metrics:
687
+ - type: dot_accuracy@1
688
+ value: 0.54
689
+ name: Dot Accuracy@1
690
+ - type: dot_accuracy@3
691
+ value: 0.68
692
+ name: Dot Accuracy@3
693
+ - type: dot_accuracy@5
694
+ value: 0.76
695
+ name: Dot Accuracy@5
696
+ - type: dot_accuracy@10
697
+ value: 0.84
698
+ name: Dot Accuracy@10
699
+ - type: dot_precision@1
700
+ value: 0.54
701
+ name: Dot Precision@1
702
+ - type: dot_precision@3
703
+ value: 0.22666666666666666
704
+ name: Dot Precision@3
705
+ - type: dot_precision@5
706
+ value: 0.15600000000000003
707
+ name: Dot Precision@5
708
+ - type: dot_precision@10
709
+ value: 0.08599999999999998
710
+ name: Dot Precision@10
711
+ - type: dot_recall@1
712
+ value: 0.53
713
+ name: Dot Recall@1
714
+ - type: dot_recall@3
715
+ value: 0.6466666666666666
716
+ name: Dot Recall@3
717
+ - type: dot_recall@5
718
+ value: 0.7266666666666666
719
+ name: Dot Recall@5
720
+ - type: dot_recall@10
721
+ value: 0.8066666666666665
722
+ name: Dot Recall@10
723
+ - type: dot_ndcg@10
724
+ value: 0.6669183841429774
725
+ name: Dot Ndcg@10
726
+ - type: dot_mrr@10
727
+ value: 0.6358571428571428
728
+ name: Dot Mrr@10
729
+ - type: dot_map@100
730
+ value: 0.6246746886992789
731
+ name: Dot Map@100
732
+ - type: query_active_dims
733
+ value: 255.33999633789062
734
+ name: Query Active Dims
735
+ - type: query_sparsity_ratio
736
+ value: 0.9916342311664409
737
+ name: Query Sparsity Ratio
738
+ - type: corpus_active_dims
739
+ value: 861.5084228515625
740
+ name: Corpus Active Dims
741
+ - type: corpus_sparsity_ratio
742
+ value: 0.9717741818081527
743
+ name: Corpus Sparsity Ratio
744
+ - task:
745
+ type: sparse-information-retrieval
746
+ name: Sparse Information Retrieval
747
+ dataset:
748
+ name: NanoFiQA2018
749
+ type: NanoFiQA2018
750
+ metrics:
751
+ - type: dot_accuracy@1
752
+ value: 0.18
753
+ name: Dot Accuracy@1
754
+ - type: dot_accuracy@3
755
+ value: 0.36
756
+ name: Dot Accuracy@3
757
+ - type: dot_accuracy@5
758
+ value: 0.42
759
+ name: Dot Accuracy@5
760
+ - type: dot_accuracy@10
761
+ value: 0.52
762
+ name: Dot Accuracy@10
763
+ - type: dot_precision@1
764
+ value: 0.18
765
+ name: Dot Precision@1
766
+ - type: dot_precision@3
767
+ value: 0.15333333333333332
768
+ name: Dot Precision@3
769
+ - type: dot_precision@5
770
+ value: 0.12800000000000003
771
+ name: Dot Precision@5
772
+ - type: dot_precision@10
773
+ value: 0.086
774
+ name: Dot Precision@10
775
+ - type: dot_recall@1
776
+ value: 0.10933333333333334
777
+ name: Dot Recall@1
778
+ - type: dot_recall@3
779
+ value: 0.23572222222222222
780
+ name: Dot Recall@3
781
+ - type: dot_recall@5
782
+ value: 0.2821111111111111
783
+ name: Dot Recall@5
784
+ - type: dot_recall@10
785
+ value: 0.35740476190476195
786
+ name: Dot Recall@10
787
+ - type: dot_ndcg@10
788
+ value: 0.2731926938394576
789
+ name: Dot Ndcg@10
790
+ - type: dot_mrr@10
791
+ value: 0.28719047619047616
792
+ name: Dot Mrr@10
793
+ - type: dot_map@100
794
+ value: 0.22109942326282112
795
+ name: Dot Map@100
796
+ - type: query_active_dims
797
+ value: 88.83999633789062
798
+ name: Query Active Dims
799
+ - type: query_sparsity_ratio
800
+ value: 0.9970893127469402
801
+ name: Query Sparsity Ratio
802
+ - type: corpus_active_dims
803
+ value: 414.41741943359375
804
+ name: Corpus Active Dims
805
+ - type: corpus_sparsity_ratio
806
+ value: 0.9864223373490076
807
+ name: Corpus Sparsity Ratio
808
+ - task:
809
+ type: sparse-information-retrieval
810
+ name: Sparse Information Retrieval
811
+ dataset:
812
+ name: NanoHotpotQA
813
+ type: NanoHotpotQA
814
+ metrics:
815
+ - type: dot_accuracy@1
816
+ value: 0.66
817
+ name: Dot Accuracy@1
818
+ - type: dot_accuracy@3
819
+ value: 0.72
820
+ name: Dot Accuracy@3
821
+ - type: dot_accuracy@5
822
+ value: 0.78
823
+ name: Dot Accuracy@5
824
+ - type: dot_accuracy@10
825
+ value: 0.9
826
+ name: Dot Accuracy@10
827
+ - type: dot_precision@1
828
+ value: 0.66
829
+ name: Dot Precision@1
830
+ - type: dot_precision@3
831
+ value: 0.33333333333333326
832
+ name: Dot Precision@3
833
+ - type: dot_precision@5
834
+ value: 0.23199999999999996
835
+ name: Dot Precision@5
836
+ - type: dot_precision@10
837
+ value: 0.13799999999999998
838
+ name: Dot Precision@10
839
+ - type: dot_recall@1
840
+ value: 0.33
841
+ name: Dot Recall@1
842
+ - type: dot_recall@3
843
+ value: 0.5
844
+ name: Dot Recall@3
845
+ - type: dot_recall@5
846
+ value: 0.58
847
+ name: Dot Recall@5
848
+ - type: dot_recall@10
849
+ value: 0.69
850
+ name: Dot Recall@10
851
+ - type: dot_ndcg@10
852
+ value: 0.6058525828428769
853
+ name: Dot Ndcg@10
854
+ - type: dot_mrr@10
855
+ value: 0.7181269841269841
856
+ name: Dot Mrr@10
857
+ - type: dot_map@100
858
+ value: 0.5210794772668003
859
+ name: Dot Map@100
860
+ - type: query_active_dims
861
+ value: 147.39999389648438
862
+ name: Query Active Dims
863
+ - type: query_sparsity_ratio
864
+ value: 0.9951706967467242
865
+ name: Query Sparsity Ratio
866
+ - type: corpus_active_dims
867
+ value: 622.540283203125
868
+ name: Corpus Active Dims
869
+ - type: corpus_sparsity_ratio
870
+ value: 0.9796035553632422
871
+ name: Corpus Sparsity Ratio
872
+ - task:
873
+ type: sparse-information-retrieval
874
+ name: Sparse Information Retrieval
875
+ dataset:
876
+ name: NanoQuoraRetrieval
877
+ type: NanoQuoraRetrieval
878
+ metrics:
879
+ - type: dot_accuracy@1
880
+ value: 0.4
881
+ name: Dot Accuracy@1
882
+ - type: dot_accuracy@3
883
+ value: 0.58
884
+ name: Dot Accuracy@3
885
+ - type: dot_accuracy@5
886
+ value: 0.72
887
+ name: Dot Accuracy@5
888
+ - type: dot_accuracy@10
889
+ value: 0.82
890
+ name: Dot Accuracy@10
891
+ - type: dot_precision@1
892
+ value: 0.4
893
+ name: Dot Precision@1
894
+ - type: dot_precision@3
895
+ value: 0.19333333333333333
896
+ name: Dot Precision@3
897
+ - type: dot_precision@5
898
+ value: 0.15200000000000002
899
+ name: Dot Precision@5
900
+ - type: dot_precision@10
901
+ value: 0.08599999999999998
902
+ name: Dot Precision@10
903
+ - type: dot_recall@1
904
+ value: 0.3866666666666667
905
+ name: Dot Recall@1
906
+ - type: dot_recall@3
907
+ value: 0.5466666666666667
908
+ name: Dot Recall@3
909
+ - type: dot_recall@5
910
+ value: 0.6906666666666667
911
+ name: Dot Recall@5
912
+ - type: dot_recall@10
913
+ value: 0.7906666666666666
914
+ name: Dot Recall@10
915
+ - type: dot_ndcg@10
916
+ value: 0.5774223027465384
917
+ name: Dot Ndcg@10
918
+ - type: dot_mrr@10
919
+ value: 0.5223333333333333
920
+ name: Dot Mrr@10
921
+ - type: dot_map@100
922
+ value: 0.5159175593795703
923
+ name: Dot Map@100
924
+ - type: query_active_dims
925
+ value: 52.779998779296875
926
+ name: Query Active Dims
927
+ - type: query_sparsity_ratio
928
+ value: 0.9982707555606023
929
+ name: Query Sparsity Ratio
930
+ - type: corpus_active_dims
931
+ value: 59.61296081542969
932
+ name: Corpus Active Dims
933
+ - type: corpus_sparsity_ratio
934
+ value: 0.9980468854984789
935
+ name: Corpus Sparsity Ratio
936
+ - task:
937
+ type: sparse-information-retrieval
938
+ name: Sparse Information Retrieval
939
+ dataset:
940
+ name: NanoSCIDOCS
941
+ type: NanoSCIDOCS
942
+ metrics:
943
+ - type: dot_accuracy@1
944
+ value: 0.34
945
+ name: Dot Accuracy@1
946
+ - type: dot_accuracy@3
947
+ value: 0.52
948
+ name: Dot Accuracy@3
949
+ - type: dot_accuracy@5
950
+ value: 0.64
951
+ name: Dot Accuracy@5
952
+ - type: dot_accuracy@10
953
+ value: 0.78
954
+ name: Dot Accuracy@10
955
+ - type: dot_precision@1
956
+ value: 0.34
957
+ name: Dot Precision@1
958
+ - type: dot_precision@3
959
+ value: 0.22
960
+ name: Dot Precision@3
961
+ - type: dot_precision@5
962
+ value: 0.18
963
+ name: Dot Precision@5
964
+ - type: dot_precision@10
965
+ value: 0.14
966
+ name: Dot Precision@10
967
+ - type: dot_recall@1
968
+ value: 0.07266666666666667
969
+ name: Dot Recall@1
970
+ - type: dot_recall@3
971
+ value: 0.13666666666666666
972
+ name: Dot Recall@3
973
+ - type: dot_recall@5
974
+ value: 0.18566666666666662
975
+ name: Dot Recall@5
976
+ - type: dot_recall@10
977
+ value: 0.2876666666666667
978
+ name: Dot Recall@10
979
+ - type: dot_ndcg@10
980
+ value: 0.2663751860700744
981
+ name: Dot Ndcg@10
982
+ - type: dot_mrr@10
983
+ value: 0.453611111111111
984
+ name: Dot Mrr@10
985
+ - type: dot_map@100
986
+ value: 0.17985066244400083
987
+ name: Dot Map@100
988
+ - type: query_active_dims
989
+ value: 197.97999572753906
990
+ name: Query Active Dims
991
+ - type: query_sparsity_ratio
992
+ value: 0.9935135313633596
993
+ name: Query Sparsity Ratio
994
+ - type: corpus_active_dims
995
+ value: 729.0889892578125
996
+ name: Corpus Active Dims
997
+ - type: corpus_sparsity_ratio
998
+ value: 0.9761126731781072
999
+ name: Corpus Sparsity Ratio
1000
+ - task:
1001
+ type: sparse-information-retrieval
1002
+ name: Sparse Information Retrieval
1003
+ dataset:
1004
+ name: NanoArguAna
1005
+ type: NanoArguAna
1006
+ metrics:
1007
+ - type: dot_accuracy@1
1008
+ value: 0.02
1009
+ name: Dot Accuracy@1
1010
+ - type: dot_accuracy@3
1011
+ value: 0.16
1012
+ name: Dot Accuracy@3
1013
+ - type: dot_accuracy@5
1014
+ value: 0.22
1015
+ name: Dot Accuracy@5
1016
+ - type: dot_accuracy@10
1017
+ value: 0.24
1018
+ name: Dot Accuracy@10
1019
+ - type: dot_precision@1
1020
+ value: 0.02
1021
+ name: Dot Precision@1
1022
+ - type: dot_precision@3
1023
+ value: 0.05333333333333333
1024
+ name: Dot Precision@3
1025
+ - type: dot_precision@5
1026
+ value: 0.044000000000000004
1027
+ name: Dot Precision@5
1028
+ - type: dot_precision@10
1029
+ value: 0.024000000000000004
1030
+ name: Dot Precision@10
1031
+ - type: dot_recall@1
1032
+ value: 0.02
1033
+ name: Dot Recall@1
1034
+ - type: dot_recall@3
1035
+ value: 0.16
1036
+ name: Dot Recall@3
1037
+ - type: dot_recall@5
1038
+ value: 0.22
1039
+ name: Dot Recall@5
1040
+ - type: dot_recall@10
1041
+ value: 0.24
1042
+ name: Dot Recall@10
1043
+ - type: dot_ndcg@10
1044
+ value: 0.1308713212722807
1045
+ name: Dot Ndcg@10
1046
+ - type: dot_mrr@10
1047
+ value: 0.0955
1048
+ name: Dot Mrr@10
1049
+ - type: dot_map@100
1050
+ value: 0.10586268132966514
1051
+ name: Dot Map@100
1052
+ - type: query_active_dims
1053
+ value: 798.3800048828125
1054
+ name: Query Active Dims
1055
+ - type: query_sparsity_ratio
1056
+ value: 0.9738424741208698
1057
+ name: Query Sparsity Ratio
1058
+ - type: corpus_active_dims
1059
+ value: 696.6200561523438
1060
+ name: Corpus Active Dims
1061
+ - type: corpus_sparsity_ratio
1062
+ value: 0.9771764610395012
1063
+ name: Corpus Sparsity Ratio
1064
+ - task:
1065
+ type: sparse-information-retrieval
1066
+ name: Sparse Information Retrieval
1067
+ dataset:
1068
+ name: NanoSciFact
1069
+ type: NanoSciFact
1070
+ metrics:
1071
+ - type: dot_accuracy@1
1072
+ value: 0.44
1073
+ name: Dot Accuracy@1
1074
+ - type: dot_accuracy@3
1075
+ value: 0.58
1076
+ name: Dot Accuracy@3
1077
+ - type: dot_accuracy@5
1078
+ value: 0.64
1079
+ name: Dot Accuracy@5
1080
+ - type: dot_accuracy@10
1081
+ value: 0.68
1082
+ name: Dot Accuracy@10
1083
+ - type: dot_precision@1
1084
+ value: 0.44
1085
+ name: Dot Precision@1
1086
+ - type: dot_precision@3
1087
+ value: 0.20666666666666667
1088
+ name: Dot Precision@3
1089
+ - type: dot_precision@5
1090
+ value: 0.14
1091
+ name: Dot Precision@5
1092
+ - type: dot_precision@10
1093
+ value: 0.078
1094
+ name: Dot Precision@10
1095
+ - type: dot_recall@1
1096
+ value: 0.415
1097
+ name: Dot Recall@1
1098
+ - type: dot_recall@3
1099
+ value: 0.555
1100
+ name: Dot Recall@3
1101
+ - type: dot_recall@5
1102
+ value: 0.615
1103
+ name: Dot Recall@5
1104
+ - type: dot_recall@10
1105
+ value: 0.67
1106
+ name: Dot Recall@10
1107
+ - type: dot_ndcg@10
1108
+ value: 0.5536377846083319
1109
+ name: Dot Ndcg@10
1110
+ - type: dot_mrr@10
1111
+ value: 0.5218888888888888
1112
+ name: Dot Mrr@10
1113
+ - type: dot_map@100
1114
+ value: 0.5200703583453583
1115
+ name: Dot Map@100
1116
+ - type: query_active_dims
1117
+ value: 270.4800109863281
1118
+ name: Query Active Dims
1119
+ - type: query_sparsity_ratio
1120
+ value: 0.9911381950400915
1121
+ name: Query Sparsity Ratio
1122
+ - type: corpus_active_dims
1123
+ value: 766.060302734375
1124
+ name: Corpus Active Dims
1125
+ - type: corpus_sparsity_ratio
1126
+ value: 0.9749013726907028
1127
+ name: Corpus Sparsity Ratio
1128
+ - task:
1129
+ type: sparse-information-retrieval
1130
+ name: Sparse Information Retrieval
1131
+ dataset:
1132
+ name: NanoTouche2020
1133
+ type: NanoTouche2020
1134
+ metrics:
1135
+ - type: dot_accuracy@1
1136
+ value: 0.4489795918367347
1137
+ name: Dot Accuracy@1
1138
+ - type: dot_accuracy@3
1139
+ value: 0.8367346938775511
1140
+ name: Dot Accuracy@3
1141
+ - type: dot_accuracy@5
1142
+ value: 0.8979591836734694
1143
+ name: Dot Accuracy@5
1144
+ - type: dot_accuracy@10
1145
+ value: 1.0
1146
+ name: Dot Accuracy@10
1147
+ - type: dot_precision@1
1148
+ value: 0.4489795918367347
1149
+ name: Dot Precision@1
1150
+ - type: dot_precision@3
1151
+ value: 0.5782312925170067
1152
+ name: Dot Precision@3
1153
+ - type: dot_precision@5
1154
+ value: 0.5020408163265306
1155
+ name: Dot Precision@5
1156
+ - type: dot_precision@10
1157
+ value: 0.4183673469387756
1158
+ name: Dot Precision@10
1159
+ - type: dot_recall@1
1160
+ value: 0.028637012782604658
1161
+ name: Dot Recall@1
1162
+ - type: dot_recall@3
1163
+ value: 0.11168898095521147
1164
+ name: Dot Recall@3
1165
+ - type: dot_recall@5
1166
+ value: 0.168207833765178
1167
+ name: Dot Recall@5
1168
+ - type: dot_recall@10
1169
+ value: 0.26840587129271604
1170
+ name: Dot Recall@10
1171
+ - type: dot_ndcg@10
1172
+ value: 0.46065286658673993
1173
+ name: Dot Ndcg@10
1174
+ - type: dot_mrr@10
1175
+ value: 0.652437641723356
1176
+ name: Dot Mrr@10
1177
+ - type: dot_map@100
1178
+ value: 0.34327142171812286
1179
+ name: Dot Map@100
1180
+ - type: query_active_dims
1181
+ value: 37.918365478515625
1182
+ name: Query Active Dims
1183
+ - type: query_sparsity_ratio
1184
+ value: 0.9987576710085015
1185
+ name: Query Sparsity Ratio
1186
+ - type: corpus_active_dims
1187
+ value: 507.71820068359375
1188
+ name: Corpus Active Dims
1189
+ - type: corpus_sparsity_ratio
1190
+ value: 0.9833655002724724
1191
+ name: Corpus Sparsity Ratio
1192
+ ---
1193
+
1194
+ # splade-distilbert-base-uncased trained on GooAQ
1195
+
1196
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
1197
+ ## Model Details
1198
+
1199
+ ### Model Description
1200
+ - **Model Type:** SPLADE Sparse Encoder
1201
+ - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) <!-- at revision 12040accade4e8a0f71eabdb258fecc2e7e948be -->
1202
+ - **Maximum Sequence Length:** 256 tokens
1203
+ - **Output Dimensionality:** 30522 dimensions
1204
+ - **Similarity Function:** Dot Product
1205
+ - **Training Dataset:**
1206
+ - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
1207
+ - **Language:** en
1208
+ - **License:** apache-2.0
1209
+
1210
+ ### Model Sources
1211
+
1212
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
1213
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
1214
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
1215
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
1216
+
1217
+ ### Full Model Architecture
1218
+
1219
+ ```
1220
+ SparseEncoder(
1221
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'DistilBertForMaskedLM'})
1222
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
1223
+ )
1224
+ ```
1225
+
1226
+ ## Usage
1227
+
1228
+ ### Direct Usage (Sentence Transformers)
1229
+
1230
+ First install the Sentence Transformers library:
1231
+
1232
+ ```bash
1233
+ pip install -U sentence-transformers
1234
+ ```
1235
+
1236
+ Then you can load this model and run inference.
1237
+ ```python
1238
+ from sentence_transformers import SparseEncoder
1239
+
1240
+ # Download from the 🤗 Hub
1241
+ model = SparseEncoder("tomaarsen/splade-distilbert-base-uncased-gooaq-peft-r64")
1242
+ # Run inference
1243
+ queries = [
1244
+ "how many days for doxycycline to work on sinus infection?",
1245
+ ]
1246
+ documents = [
1247
+ 'Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks for chronic sinusitis.',
1248
+ 'Most engagements typically have a cocktail dress code, calling for dresses at, or slightly above, knee-length and high heels. If your party states a different dress code, however, such as semi-formal or dressy-casual, you may need to dress up or down accordingly.',
1249
+ 'The average service life of a gas furnace is about 15 years, but the actual life span of an individual unit can vary greatly. There are a number of contributing factors that determine the age a furnace reaches: The quality of the equipment.',
1250
+ ]
1251
+ query_embeddings = model.encode_query(queries)
1252
+ document_embeddings = model.encode_document(documents)
1253
+ print(query_embeddings.shape, document_embeddings.shape)
1254
+ # [1, 30522] [3, 30522]
1255
+
1256
+ # Get the similarity scores for the embeddings
1257
+ similarities = model.similarity(query_embeddings, document_embeddings)
1258
+ print(similarities)
1259
+ # tensor([[90.6936, 25.1699, 31.3902]])
1260
+ ```
1261
+
1262
+ <!--
1263
+ ### Direct Usage (Transformers)
1264
+
1265
+ <details><summary>Click to see the direct usage in Transformers</summary>
1266
+
1267
+ </details>
1268
+ -->
1269
+
1270
+ <!--
1271
+ ### Downstream Usage (Sentence Transformers)
1272
+
1273
+ You can finetune this model on your own dataset.
1274
+
1275
+ <details><summary>Click to expand</summary>
1276
+
1277
+ </details>
1278
+ -->
1279
+
1280
+ <!--
1281
+ ### Out-of-Scope Use
1282
+
1283
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
1284
+ -->
1285
+
1286
+ ## Evaluation
1287
+
1288
+ ### Metrics
1289
+
1290
+ #### Sparse Information Retrieval
1291
+
1292
+ * Datasets: `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
1293
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
1294
+
1295
+ | Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
1296
+ |:----------------------|:------------|:-------------|:-----------|:-----------------|:------------|:-----------|:-------------|:-------------|:-------------------|:------------|:------------|:------------|:---------------|
1297
+ | dot_accuracy@1 | 0.32 | 0.32 | 0.28 | 0.18 | 0.54 | 0.54 | 0.18 | 0.66 | 0.4 | 0.34 | 0.02 | 0.44 | 0.449 |
1298
+ | dot_accuracy@3 | 0.48 | 0.42 | 0.48 | 0.32 | 0.74 | 0.68 | 0.36 | 0.72 | 0.58 | 0.52 | 0.16 | 0.58 | 0.8367 |
1299
+ | dot_accuracy@5 | 0.62 | 0.48 | 0.56 | 0.4 | 0.82 | 0.76 | 0.42 | 0.78 | 0.72 | 0.64 | 0.22 | 0.64 | 0.898 |
1300
+ | dot_accuracy@10 | 0.7 | 0.54 | 0.72 | 0.52 | 0.9 | 0.84 | 0.52 | 0.9 | 0.82 | 0.78 | 0.24 | 0.68 | 1.0 |
1301
+ | dot_precision@1 | 0.32 | 0.32 | 0.28 | 0.18 | 0.54 | 0.54 | 0.18 | 0.66 | 0.4 | 0.34 | 0.02 | 0.44 | 0.449 |
1302
+ | dot_precision@3 | 0.16 | 0.26 | 0.16 | 0.12 | 0.4667 | 0.2267 | 0.1533 | 0.3333 | 0.1933 | 0.22 | 0.0533 | 0.2067 | 0.5782 |
1303
+ | dot_precision@5 | 0.124 | 0.224 | 0.116 | 0.088 | 0.432 | 0.156 | 0.128 | 0.232 | 0.152 | 0.18 | 0.044 | 0.14 | 0.502 |
1304
+ | dot_precision@10 | 0.07 | 0.184 | 0.074 | 0.062 | 0.412 | 0.086 | 0.086 | 0.138 | 0.086 | 0.14 | 0.024 | 0.078 | 0.4184 |
1305
+ | dot_recall@1 | 0.32 | 0.0112 | 0.26 | 0.0917 | 0.055 | 0.53 | 0.1093 | 0.33 | 0.3867 | 0.0727 | 0.02 | 0.415 | 0.0286 |
1306
+ | dot_recall@3 | 0.48 | 0.0305 | 0.45 | 0.1633 | 0.1158 | 0.6467 | 0.2357 | 0.5 | 0.5467 | 0.1367 | 0.16 | 0.555 | 0.1117 |
1307
+ | dot_recall@5 | 0.62 | 0.0414 | 0.52 | 0.19 | 0.1573 | 0.7267 | 0.2821 | 0.58 | 0.6907 | 0.1857 | 0.22 | 0.615 | 0.1682 |
1308
+ | dot_recall@10 | 0.7 | 0.0543 | 0.67 | 0.2473 | 0.2872 | 0.8067 | 0.3574 | 0.69 | 0.7907 | 0.2877 | 0.24 | 0.67 | 0.2684 |
1309
+ | **dot_ndcg@10** | **0.4978** | **0.2109** | **0.4535** | **0.2066** | **0.491** | **0.6669** | **0.2732** | **0.6059** | **0.5774** | **0.2664** | **0.1309** | **0.5536** | **0.4607** |
1310
+ | dot_mrr@10 | 0.4339 | 0.3881 | 0.4033 | 0.2785 | 0.6685 | 0.6359 | 0.2872 | 0.7181 | 0.5223 | 0.4536 | 0.0955 | 0.5219 | 0.6524 |
1311
+ | dot_map@100 | 0.4468 | 0.0714 | 0.3886 | 0.1647 | 0.3419 | 0.6247 | 0.2211 | 0.5211 | 0.5159 | 0.1799 | 0.1059 | 0.5201 | 0.3433 |
1312
+ | query_active_dims | 121.6 | 199.74 | 97.54 | 243.54 | 180.3 | 255.34 | 88.84 | 147.4 | 52.78 | 197.98 | 798.38 | 270.48 | 37.9184 |
1313
+ | query_sparsity_ratio | 0.996 | 0.9935 | 0.9968 | 0.992 | 0.9941 | 0.9916 | 0.9971 | 0.9952 | 0.9983 | 0.9935 | 0.9738 | 0.9911 | 0.9988 |
1314
+ | corpus_active_dims | 397.7708 | 879.7701 | 625.5005 | 704.9307 | 549.1514 | 861.5084 | 414.4174 | 622.5403 | 59.613 | 729.089 | 696.6201 | 766.0603 | 507.7182 |
1315
+ | corpus_sparsity_ratio | 0.987 | 0.9712 | 0.9795 | 0.9769 | 0.982 | 0.9718 | 0.9864 | 0.9796 | 0.998 | 0.9761 | 0.9772 | 0.9749 | 0.9834 |
1316
+
1317
+ #### Sparse Nano BEIR
1318
+
1319
+ * Dataset: `NanoBEIR_mean`
1320
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1321
+ ```json
1322
+ {
1323
+ "dataset_names": [
1324
+ "msmarco",
1325
+ "nfcorpus",
1326
+ "nq"
1327
+ ]
1328
+ }
1329
+ ```
1330
+
1331
+ | Metric | Value |
1332
+ |:----------------------|:-----------|
1333
+ | dot_accuracy@1 | 0.3 |
1334
+ | dot_accuracy@3 | 0.4733 |
1335
+ | dot_accuracy@5 | 0.5467 |
1336
+ | dot_accuracy@10 | 0.6533 |
1337
+ | dot_precision@1 | 0.3 |
1338
+ | dot_precision@3 | 0.2 |
1339
+ | dot_precision@5 | 0.1507 |
1340
+ | dot_precision@10 | 0.108 |
1341
+ | dot_recall@1 | 0.1822 |
1342
+ | dot_recall@3 | 0.3237 |
1343
+ | dot_recall@5 | 0.3932 |
1344
+ | dot_recall@10 | 0.4714 |
1345
+ | **dot_ndcg@10** | **0.3812** |
1346
+ | dot_mrr@10 | 0.4072 |
1347
+ | dot_map@100 | 0.2943 |
1348
+ | query_active_dims | 151.4533 |
1349
+ | query_sparsity_ratio | 0.995 |
1350
+ | corpus_active_dims | 576.6101 |
1351
+ | corpus_sparsity_ratio | 0.9811 |
1352
+
1353
+ #### Sparse Nano BEIR
1354
+
1355
+ * Dataset: `NanoBEIR_mean`
1356
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1357
+ ```json
1358
+ {
1359
+ "dataset_names": [
1360
+ "climatefever",
1361
+ "dbpedia",
1362
+ "fever",
1363
+ "fiqa2018",
1364
+ "hotpotqa",
1365
+ "msmarco",
1366
+ "nfcorpus",
1367
+ "nq",
1368
+ "quoraretrieval",
1369
+ "scidocs",
1370
+ "arguana",
1371
+ "scifact",
1372
+ "touche2020"
1373
+ ]
1374
+ }
1375
+ ```
1376
+
1377
+ | Metric | Value |
1378
+ |:----------------------|:----------|
1379
+ | dot_accuracy@1 | 0.3592 |
1380
+ | dot_accuracy@3 | 0.529 |
1381
+ | dot_accuracy@5 | 0.6122 |
1382
+ | dot_accuracy@10 | 0.7046 |
1383
+ | dot_precision@1 | 0.3592 |
1384
+ | dot_precision@3 | 0.2409 |
1385
+ | dot_precision@5 | 0.1937 |
1386
+ | dot_precision@10 | 0.143 |
1387
+ | dot_recall@1 | 0.2023 |
1388
+ | dot_recall@3 | 0.3179 |
1389
+ | dot_recall@5 | 0.3844 |
1390
+ | dot_recall@10 | 0.4669 |
1391
+ | **dot_ndcg@10** | **0.415** |
1392
+ | dot_mrr@10 | 0.4661 |
1393
+ | dot_map@100 | 0.3419 |
1394
+ | query_active_dims | 207.3251 |
1395
+ | query_sparsity_ratio | 0.9932 |
1396
+ | corpus_active_dims | 571.212 |
1397
+ | corpus_sparsity_ratio | 0.9813 |
1398
+
1399
+ <!--
1400
+ ## Bias, Risks and Limitations
1401
+
1402
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1403
+ -->
1404
+
1405
+ <!--
1406
+ ### Recommendations
1407
+
1408
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1409
+ -->
1410
+
1411
+ ## Training Details
1412
+
1413
+ ### Training Dataset
1414
+
1415
+ #### gooaq
1416
+
1417
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1418
+ * Size: 99,000 training samples
1419
+ * Columns: <code>question</code> and <code>answer</code>
1420
+ * Approximate statistics based on the first 1000 samples:
1421
+ | | question | answer |
1422
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1423
+ | type | string | string |
1424
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.79 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.02 tokens</li><li>max: 153 tokens</li></ul> |
1425
+ * Samples:
1426
+ | question | answer |
1427
+ |:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1428
+ | <code>what are the 5 characteristics of a star?</code> | <code>Key Concept: Characteristics used to classify stars include color, temperature, size, composition, and brightness.</code> |
1429
+ | <code>are copic markers alcohol ink?</code> | <code>Copic Ink is alcohol-based and flammable. Keep away from direct sunlight and extreme temperatures.</code> |
1430
+ | <code>what is the difference between appellate term and appellate division?</code> | <code>Appellate terms An appellate term is an intermediate appellate court that hears appeals from the inferior courts within their designated counties or judicial districts, and are intended to ease the workload on the Appellate Division and provide a less expensive forum closer to the people.</code> |
1431
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1432
+ ```json
1433
+ {
1434
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1435
+ "document_regularizer_weight": 3e-05,
1436
+ "query_regularizer_weight": 5e-05
1437
+ }
1438
+ ```
1439
+
1440
+ ### Evaluation Dataset
1441
+
1442
+ #### gooaq
1443
+
1444
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1445
+ * Size: 1,000 evaluation samples
1446
+ * Columns: <code>question</code> and <code>answer</code>
1447
+ * Approximate statistics based on the first 1000 samples:
1448
+ | | question | answer |
1449
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1450
+ | type | string | string |
1451
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.93 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.84 tokens</li><li>max: 127 tokens</li></ul> |
1452
+ * Samples:
1453
+ | question | answer |
1454
+ |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1455
+ | <code>should you take ibuprofen with high blood pressure?</code> | <code>In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.</code> |
1456
+ | <code>how old do you have to be to work in sc?</code> | <code>The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.</code> |
1457
+ | <code>how to write a topic proposal for a research paper?</code> | <code>['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']</code> |
1458
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1459
+ ```json
1460
+ {
1461
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1462
+ "document_regularizer_weight": 3e-05,
1463
+ "query_regularizer_weight": 5e-05
1464
+ }
1465
+ ```
1466
+
1467
+ ### Training Hyperparameters
1468
+ #### Non-Default Hyperparameters
1469
+
1470
+ - `eval_strategy`: steps
1471
+ - `per_device_train_batch_size`: 32
1472
+ - `per_device_eval_batch_size`: 32
1473
+ - `learning_rate`: 2e-05
1474
+ - `num_train_epochs`: 1
1475
+ - `bf16`: True
1476
+ - `load_best_model_at_end`: True
1477
+ - `batch_sampler`: no_duplicates
1478
+
1479
+ #### All Hyperparameters
1480
+ <details><summary>Click to expand</summary>
1481
+
1482
+ - `overwrite_output_dir`: False
1483
+ - `do_predict`: False
1484
+ - `eval_strategy`: steps
1485
+ - `prediction_loss_only`: True
1486
+ - `per_device_train_batch_size`: 32
1487
+ - `per_device_eval_batch_size`: 32
1488
+ - `per_gpu_train_batch_size`: None
1489
+ - `per_gpu_eval_batch_size`: None
1490
+ - `gradient_accumulation_steps`: 1
1491
+ - `eval_accumulation_steps`: None
1492
+ - `torch_empty_cache_steps`: None
1493
+ - `learning_rate`: 2e-05
1494
+ - `weight_decay`: 0.0
1495
+ - `adam_beta1`: 0.9
1496
+ - `adam_beta2`: 0.999
1497
+ - `adam_epsilon`: 1e-08
1498
+ - `max_grad_norm`: 1.0
1499
+ - `num_train_epochs`: 1
1500
+ - `max_steps`: -1
1501
+ - `lr_scheduler_type`: linear
1502
+ - `lr_scheduler_kwargs`: {}
1503
+ - `warmup_ratio`: 0.0
1504
+ - `warmup_steps`: 0
1505
+ - `log_level`: passive
1506
+ - `log_level_replica`: warning
1507
+ - `log_on_each_node`: True
1508
+ - `logging_nan_inf_filter`: True
1509
+ - `save_safetensors`: True
1510
+ - `save_on_each_node`: False
1511
+ - `save_only_model`: False
1512
+ - `restore_callback_states_from_checkpoint`: False
1513
+ - `no_cuda`: False
1514
+ - `use_cpu`: False
1515
+ - `use_mps_device`: False
1516
+ - `seed`: 42
1517
+ - `data_seed`: None
1518
+ - `jit_mode_eval`: False
1519
+ - `use_ipex`: False
1520
+ - `bf16`: True
1521
+ - `fp16`: False
1522
+ - `fp16_opt_level`: O1
1523
+ - `half_precision_backend`: auto
1524
+ - `bf16_full_eval`: False
1525
+ - `fp16_full_eval`: False
1526
+ - `tf32`: None
1527
+ - `local_rank`: 0
1528
+ - `ddp_backend`: None
1529
+ - `tpu_num_cores`: None
1530
+ - `tpu_metrics_debug`: False
1531
+ - `debug`: []
1532
+ - `dataloader_drop_last`: False
1533
+ - `dataloader_num_workers`: 0
1534
+ - `dataloader_prefetch_factor`: None
1535
+ - `past_index`: -1
1536
+ - `disable_tqdm`: False
1537
+ - `remove_unused_columns`: True
1538
+ - `label_names`: None
1539
+ - `load_best_model_at_end`: True
1540
+ - `ignore_data_skip`: False
1541
+ - `fsdp`: []
1542
+ - `fsdp_min_num_params`: 0
1543
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1544
+ - `fsdp_transformer_layer_cls_to_wrap`: None
1545
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1546
+ - `deepspeed`: None
1547
+ - `label_smoothing_factor`: 0.0
1548
+ - `optim`: adamw_torch
1549
+ - `optim_args`: None
1550
+ - `adafactor`: False
1551
+ - `group_by_length`: False
1552
+ - `length_column_name`: length
1553
+ - `ddp_find_unused_parameters`: None
1554
+ - `ddp_bucket_cap_mb`: None
1555
+ - `ddp_broadcast_buffers`: False
1556
+ - `dataloader_pin_memory`: True
1557
+ - `dataloader_persistent_workers`: False
1558
+ - `skip_memory_metrics`: True
1559
+ - `use_legacy_prediction_loop`: False
1560
+ - `push_to_hub`: False
1561
+ - `resume_from_checkpoint`: None
1562
+ - `hub_model_id`: None
1563
+ - `hub_strategy`: every_save
1564
+ - `hub_private_repo`: None
1565
+ - `hub_always_push`: False
1566
+ - `gradient_checkpointing`: False
1567
+ - `gradient_checkpointing_kwargs`: None
1568
+ - `include_inputs_for_metrics`: False
1569
+ - `include_for_metrics`: []
1570
+ - `eval_do_concat_batches`: True
1571
+ - `fp16_backend`: auto
1572
+ - `push_to_hub_model_id`: None
1573
+ - `push_to_hub_organization`: None
1574
+ - `mp_parameters`:
1575
+ - `auto_find_batch_size`: False
1576
+ - `full_determinism`: False
1577
+ - `torchdynamo`: None
1578
+ - `ray_scope`: last
1579
+ - `ddp_timeout`: 1800
1580
+ - `torch_compile`: False
1581
+ - `torch_compile_backend`: None
1582
+ - `torch_compile_mode`: None
1583
+ - `include_tokens_per_second`: False
1584
+ - `include_num_input_tokens_seen`: False
1585
+ - `neftune_noise_alpha`: None
1586
+ - `optim_target_modules`: None
1587
+ - `batch_eval_metrics`: False
1588
+ - `eval_on_start`: False
1589
+ - `use_liger_kernel`: False
1590
+ - `eval_use_gather_object`: False
1591
+ - `average_tokens_across_devices`: False
1592
+ - `prompts`: None
1593
+ - `batch_sampler`: no_duplicates
1594
+ - `multi_dataset_batch_sampler`: proportional
1595
+ - `router_mapping`: {}
1596
+ - `learning_rate_mapping`: {}
1597
+
1598
+ </details>
1599
+
1600
+ ### Training Logs
1601
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 |
1602
+ |:----------:|:--------:|:-------------:|:---------------:|:-----------------------:|:------------------------:|:------------------:|:-------------------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|
1603
+ | 0.0323 | 100 | 132.3714 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1604
+ | 0.0646 | 200 | 17.6927 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1605
+ | 0.0970 | 300 | 2.8291 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1606
+ | 0.1293 | 400 | 0.9365 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1607
+ | 0.1616 | 500 | 0.5878 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1608
+ | 0.1939 | 600 | 0.4041 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1609
+ | 0.1972 | 610 | - | 0.2847 | 0.3951 | 0.1965 | 0.3212 | 0.3043 | - | - | - | - | - | - | - | - | - | - |
1610
+ | 0.2262 | 700 | 0.3624 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1611
+ | 0.2586 | 800 | 0.295 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1612
+ | 0.2909 | 900 | 0.282 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1613
+ | 0.3232 | 1000 | 0.2947 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1614
+ | 0.3555 | 1100 | 0.2664 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1615
+ | 0.3878 | 1200 | 0.2387 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1616
+ | 0.3943 | 1220 | - | 0.2030 | 0.4818 | 0.2086 | 0.4084 | 0.3663 | - | - | - | - | - | - | - | - | - | - |
1617
+ | 0.4202 | 1300 | 0.2512 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1618
+ | 0.4525 | 1400 | 0.2117 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1619
+ | 0.4848 | 1500 | 0.2367 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1620
+ | 0.5171 | 1600 | 0.2085 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1621
+ | 0.5495 | 1700 | 0.1745 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1622
+ | 0.5818 | 1800 | 0.1958 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1623
+ | **0.5915** | **1830** | **-** | **0.1449** | **0.4978** | **0.2109** | **0.4535** | **0.3874** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** |
1624
+ | 0.6141 | 1900 | 0.1946 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1625
+ | 0.6464 | 2000 | 0.1801 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1626
+ | 0.6787 | 2100 | 0.1854 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1627
+ | 0.7111 | 2200 | 0.232 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1628
+ | 0.7434 | 2300 | 0.1798 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1629
+ | 0.7757 | 2400 | 0.1725 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1630
+ | 0.7886 | 2440 | - | 0.1437 | 0.4885 | 0.2206 | 0.4230 | 0.3774 | - | - | - | - | - | - | - | - | - | - |
1631
+ | 0.8080 | 2500 | 0.1598 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1632
+ | 0.8403 | 2600 | 0.1586 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1633
+ | 0.8727 | 2700 | 0.172 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1634
+ | 0.9050 | 2800 | 0.1875 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1635
+ | 0.9373 | 2900 | 0.1691 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1636
+ | 0.9696 | 3000 | 0.1473 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1637
+ | 0.9858 | 3050 | - | 0.1358 | 0.4923 | 0.2159 | 0.4354 | 0.3812 | - | - | - | - | - | - | - | - | - | - |
1638
+ | -1 | -1 | - | - | 0.4978 | 0.2109 | 0.4535 | 0.4150 | 0.2066 | 0.4910 | 0.6669 | 0.2732 | 0.6059 | 0.5774 | 0.2664 | 0.1309 | 0.5536 | 0.4607 |
1639
+
1640
+ * The bold row denotes the saved checkpoint.
1641
+
1642
+ ### Environmental Impact
1643
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
1644
+ - **Energy Consumed**: 0.034 kWh
1645
+ - **Carbon Emitted**: 0.013 kg of CO2
1646
+ - **Hours Used**: 0.142 hours
1647
+
1648
+ ### Training Hardware
1649
+ - **On Cloud**: No
1650
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3090
1651
+ - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
1652
+ - **RAM Size**: 31.78 GB
1653
+
1654
+ ### Framework Versions
1655
+ - Python: 3.11.6
1656
+ - Sentence Transformers: 4.2.0.dev0
1657
+ - Transformers: 4.52.4
1658
+ - PyTorch: 2.7.1+cu126
1659
+ - Accelerate: 1.5.1
1660
+ - Datasets: 2.21.0
1661
+ - Tokenizers: 0.21.1
1662
+
1663
+ ## Citation
1664
+
1665
+ ### BibTeX
1666
+
1667
+ #### Sentence Transformers
1668
+ ```bibtex
1669
+ @inproceedings{reimers-2019-sentence-bert,
1670
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1671
+ author = "Reimers, Nils and Gurevych, Iryna",
1672
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1673
+ month = "11",
1674
+ year = "2019",
1675
+ publisher = "Association for Computational Linguistics",
1676
+ url = "https://arxiv.org/abs/1908.10084",
1677
+ }
1678
+ ```
1679
+
1680
+ #### SpladeLoss
1681
+ ```bibtex
1682
+ @misc{formal2022distillationhardnegativesampling,
1683
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1684
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1685
+ year={2022},
1686
+ eprint={2205.04733},
1687
+ archivePrefix={arXiv},
1688
+ primaryClass={cs.IR},
1689
+ url={https://arxiv.org/abs/2205.04733},
1690
+ }
1691
+ ```
1692
+
1693
+ #### SparseMultipleNegativesRankingLoss
1694
+ ```bibtex
1695
+ @misc{henderson2017efficient,
1696
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
1697
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
1698
+ year={2017},
1699
+ eprint={1705.00652},
1700
+ archivePrefix={arXiv},
1701
+ primaryClass={cs.CL}
1702
+ }
1703
+ ```
1704
+
1705
+ #### FlopsLoss
1706
+ ```bibtex
1707
+ @article{paria2020minimizing,
1708
+ title={Minimizing flops to learn efficient sparse representations},
1709
+ author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
1710
+ journal={arXiv preprint arXiv:2004.05665},
1711
+ year={2020}
1712
+ }
1713
+ ```
1714
+
1715
+ <!--
1716
+ ## Glossary
1717
+
1718
+ *Clearly define terms in order to be accessible across audiences.*
1719
+ -->
1720
+
1721
+ <!--
1722
+ ## Model Card Authors
1723
+
1724
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1725
+ -->
1726
+
1727
+ <!--
1728
+ ## Model Card Contact
1729
+
1730
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1731
+ -->
adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "distilbert/distilbert-base-uncased",
5
+ "bias": "none",
6
+ "corda_config": null,
7
+ "eva_config": null,
8
+ "exclude_modules": null,
9
+ "fan_in_fan_out": false,
10
+ "inference_mode": false,
11
+ "init_lora_weights": true,
12
+ "layer_replication": null,
13
+ "layers_pattern": null,
14
+ "layers_to_transform": null,
15
+ "loftq_config": {},
16
+ "lora_alpha": 128,
17
+ "lora_bias": false,
18
+ "lora_dropout": 0.05,
19
+ "megatron_config": null,
20
+ "megatron_core": "megatron.core",
21
+ "modules_to_save": null,
22
+ "peft_type": "LORA",
23
+ "r": 64,
24
+ "rank_pattern": {},
25
+ "revision": null,
26
+ "target_modules": [
27
+ "v_lin",
28
+ "q_lin"
29
+ ],
30
+ "task_type": "FEATURE_EXTRACTION",
31
+ "trainable_token_indices": null,
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:65b27f929bb5dec1d99fcedbba445cf5d8193f74844389fca4768ce8cf14440c
3
+ size 4722120
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "SparseEncoder",
3
+ "__version__": {
4
+ "sentence_transformers": "4.2.0.dev0",
5
+ "transformers": "4.52.4",
6
+ "pytorch": "2.7.1+cu126"
7
+ },
8
+ "prompts": {
9
+ "query": "",
10
+ "document": ""
11
+ },
12
+ "default_prompt_name": null,
13
+ "similarity_fn_name": "dot"
14
+ }
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "DistilBertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff