Hugging Face's logo

tomaarsen
/
splade-distilbert-base-uncased-gooaq

Feature Extraction
sentence-transformers
Safetensors
English
distilbert
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:99000
loss:SpladeLoss
loss:SparseMultipleNegativesRankingLoss
loss:FlopsLoss
Eval Results
text-embeddings-inference
Model card Files
xet
Community
tomaarsen HF Staff commited on
Commit
7961881
·
verified ·
1 Parent(s): 061a499

Add new SparseEncoder model

Browse files
Files changed (11) hide show
  1. 1_SpladePooling/config.json +5 -0
  2. README.md +1731 -0
  3. config.json +23 -0
  4. config_sentence_transformers.json +14 -0
  5. model.safetensors +3 -0
  6. modules.json +14 -0
  7. sentence_bert_config.json +4 -0
  8. special_tokens_map.json +7 -0
  9. tokenizer.json +0 -0
  10. tokenizer_config.json +56 -0
  11. vocab.txt +0 -0
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "word_embedding_dimension": 30522
5
+ }
README.md ADDED
@@ -0,0 +1,1731 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:99000
12
+ - loss:SpladeLoss
13
+ - loss:SparseMultipleNegativesRankingLoss
14
+ - loss:FlopsLoss
15
+ base_model: distilbert/distilbert-base-uncased
16
+ widget:
17
+ - text: 'The term emergent literacy signals a belief that, in a literate society,
18
+ young children even one and two year olds, are in the process of becoming literate”.
19
+ ... Gray (1956:21) notes: Functional literacy is used for the training of adults
20
+ to ''meet independently the reading and writing demands placed on them''.'
21
+ - text: Rey is seemingly confirmed as being The Chosen One per a quote by a Lucasfilm
22
+ production designer who worked on The Rise of Skywalker.
23
+ - text: are union gun safes fireproof?
24
+ - text: Fruit is an essential part of a healthy diet — and may aid weight loss. Most
25
+ fruits are low in calories while high in nutrients and fiber, which can boost
26
+ your fullness. Keep in mind that it's best to eat fruits whole rather than juiced.
27
+ What's more, simply eating fruit is not the key to weight loss.
28
+ - text: Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate
29
+ or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks
30
+ for chronic sinusitis.
31
+ datasets:
32
+ - sentence-transformers/gooaq
33
+ pipeline_tag: feature-extraction
34
+ library_name: sentence-transformers
35
+ metrics:
36
+ - dot_accuracy@1
37
+ - dot_accuracy@3
38
+ - dot_accuracy@5
39
+ - dot_accuracy@10
40
+ - dot_precision@1
41
+ - dot_precision@3
42
+ - dot_precision@5
43
+ - dot_precision@10
44
+ - dot_recall@1
45
+ - dot_recall@3
46
+ - dot_recall@5
47
+ - dot_recall@10
48
+ - dot_ndcg@10
49
+ - dot_mrr@10
50
+ - dot_map@100
51
+ - query_active_dims
52
+ - query_sparsity_ratio
53
+ - corpus_active_dims
54
+ - corpus_sparsity_ratio
55
+ co2_eq_emissions:
56
+ emissions: 15.140869622791696
57
+ energy_consumed: 0.0389523841472174
58
+ source: codecarbon
59
+ training_type: fine-tuning
60
+ on_cloud: false
61
+ cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
62
+ ram_total_size: 31.777088165283203
63
+ hours_used: 0.154
64
+ hardware_used: 1 x NVIDIA GeForce RTX 3090
65
+ model-index:
66
+ - name: splade-distilbert-base-uncased trained on GooAQ
67
+ results:
68
+ - task:
69
+ type: sparse-information-retrieval
70
+ name: Sparse Information Retrieval
71
+ dataset:
72
+ name: NanoMSMARCO
73
+ type: NanoMSMARCO
74
+ metrics:
75
+ - type: dot_accuracy@1
76
+ value: 0.28
77
+ name: Dot Accuracy@1
78
+ - type: dot_accuracy@3
79
+ value: 0.5
80
+ name: Dot Accuracy@3
81
+ - type: dot_accuracy@5
82
+ value: 0.58
83
+ name: Dot Accuracy@5
84
+ - type: dot_accuracy@10
85
+ value: 0.74
86
+ name: Dot Accuracy@10
87
+ - type: dot_precision@1
88
+ value: 0.28
89
+ name: Dot Precision@1
90
+ - type: dot_precision@3
91
+ value: 0.16666666666666663
92
+ name: Dot Precision@3
93
+ - type: dot_precision@5
94
+ value: 0.11599999999999999
95
+ name: Dot Precision@5
96
+ - type: dot_precision@10
97
+ value: 0.07400000000000001
98
+ name: Dot Precision@10
99
+ - type: dot_recall@1
100
+ value: 0.28
101
+ name: Dot Recall@1
102
+ - type: dot_recall@3
103
+ value: 0.5
104
+ name: Dot Recall@3
105
+ - type: dot_recall@5
106
+ value: 0.58
107
+ name: Dot Recall@5
108
+ - type: dot_recall@10
109
+ value: 0.74
110
+ name: Dot Recall@10
111
+ - type: dot_ndcg@10
112
+ value: 0.4990149648564834
113
+ name: Dot Ndcg@10
114
+ - type: dot_mrr@10
115
+ value: 0.4237460317460318
116
+ name: Dot Mrr@10
117
+ - type: dot_map@100
118
+ value: 0.4349975076063166
119
+ name: Dot Map@100
120
+ - type: query_active_dims
121
+ value: 125.27999877929688
122
+ name: Query Active Dims
123
+ - type: query_sparsity_ratio
124
+ value: 0.9958954197372617
125
+ name: Query Sparsity Ratio
126
+ - type: corpus_active_dims
127
+ value: 312.56317138671875
128
+ name: Corpus Active Dims
129
+ - type: corpus_sparsity_ratio
130
+ value: 0.989759413819975
131
+ name: Corpus Sparsity Ratio
132
+ - type: dot_accuracy@1
133
+ value: 0.28
134
+ name: Dot Accuracy@1
135
+ - type: dot_accuracy@3
136
+ value: 0.56
137
+ name: Dot Accuracy@3
138
+ - type: dot_accuracy@5
139
+ value: 0.62
140
+ name: Dot Accuracy@5
141
+ - type: dot_accuracy@10
142
+ value: 0.72
143
+ name: Dot Accuracy@10
144
+ - type: dot_precision@1
145
+ value: 0.28
146
+ name: Dot Precision@1
147
+ - type: dot_precision@3
148
+ value: 0.18666666666666668
149
+ name: Dot Precision@3
150
+ - type: dot_precision@5
151
+ value: 0.124
152
+ name: Dot Precision@5
153
+ - type: dot_precision@10
154
+ value: 0.07200000000000001
155
+ name: Dot Precision@10
156
+ - type: dot_recall@1
157
+ value: 0.28
158
+ name: Dot Recall@1
159
+ - type: dot_recall@3
160
+ value: 0.56
161
+ name: Dot Recall@3
162
+ - type: dot_recall@5
163
+ value: 0.62
164
+ name: Dot Recall@5
165
+ - type: dot_recall@10
166
+ value: 0.72
167
+ name: Dot Recall@10
168
+ - type: dot_ndcg@10
169
+ value: 0.4890243338331294
170
+ name: Dot Ndcg@10
171
+ - type: dot_mrr@10
172
+ value: 0.41600000000000004
173
+ name: Dot Mrr@10
174
+ - type: dot_map@100
175
+ value: 0.43012741081376565
176
+ name: Dot Map@100
177
+ - type: query_active_dims
178
+ value: 111.45999908447266
179
+ name: Query Active Dims
180
+ - type: query_sparsity_ratio
181
+ value: 0.9963482078800711
182
+ name: Query Sparsity Ratio
183
+ - type: corpus_active_dims
184
+ value: 310.84136962890625
185
+ name: Corpus Active Dims
186
+ - type: corpus_sparsity_ratio
187
+ value: 0.9898158256461271
188
+ name: Corpus Sparsity Ratio
189
+ - task:
190
+ type: sparse-information-retrieval
191
+ name: Sparse Information Retrieval
192
+ dataset:
193
+ name: NanoNFCorpus
194
+ type: NanoNFCorpus
195
+ metrics:
196
+ - type: dot_accuracy@1
197
+ value: 0.32
198
+ name: Dot Accuracy@1
199
+ - type: dot_accuracy@3
200
+ value: 0.46
201
+ name: Dot Accuracy@3
202
+ - type: dot_accuracy@5
203
+ value: 0.56
204
+ name: Dot Accuracy@5
205
+ - type: dot_accuracy@10
206
+ value: 0.62
207
+ name: Dot Accuracy@10
208
+ - type: dot_precision@1
209
+ value: 0.32
210
+ name: Dot Precision@1
211
+ - type: dot_precision@3
212
+ value: 0.2866666666666666
213
+ name: Dot Precision@3
214
+ - type: dot_precision@5
215
+ value: 0.256
216
+ name: Dot Precision@5
217
+ - type: dot_precision@10
218
+ value: 0.21999999999999997
219
+ name: Dot Precision@10
220
+ - type: dot_recall@1
221
+ value: 0.010409862362909712
222
+ name: Dot Recall@1
223
+ - type: dot_recall@3
224
+ value: 0.027097764372744422
225
+ name: Dot Recall@3
226
+ - type: dot_recall@5
227
+ value: 0.04059644876604124
228
+ name: Dot Recall@5
229
+ - type: dot_recall@10
230
+ value: 0.07167100144175235
231
+ name: Dot Recall@10
232
+ - type: dot_ndcg@10
233
+ value: 0.24193545696769683
234
+ name: Dot Ndcg@10
235
+ - type: dot_mrr@10
236
+ value: 0.4157460317460317
237
+ name: Dot Mrr@10
238
+ - type: dot_map@100
239
+ value: 0.08023490333337685
240
+ name: Dot Map@100
241
+ - type: query_active_dims
242
+ value: 176.75999450683594
243
+ name: Query Active Dims
244
+ - type: query_sparsity_ratio
245
+ value: 0.994208767626406
246
+ name: Query Sparsity Ratio
247
+ - type: corpus_active_dims
248
+ value: 517.3054809570312
249
+ name: Corpus Active Dims
250
+ - type: corpus_sparsity_ratio
251
+ value: 0.9830513897858256
252
+ name: Corpus Sparsity Ratio
253
+ - type: dot_accuracy@1
254
+ value: 0.24
255
+ name: Dot Accuracy@1
256
+ - type: dot_accuracy@3
257
+ value: 0.46
258
+ name: Dot Accuracy@3
259
+ - type: dot_accuracy@5
260
+ value: 0.5
261
+ name: Dot Accuracy@5
262
+ - type: dot_accuracy@10
263
+ value: 0.58
264
+ name: Dot Accuracy@10
265
+ - type: dot_precision@1
266
+ value: 0.24
267
+ name: Dot Precision@1
268
+ - type: dot_precision@3
269
+ value: 0.29333333333333333
270
+ name: Dot Precision@3
271
+ - type: dot_precision@5
272
+ value: 0.252
273
+ name: Dot Precision@5
274
+ - type: dot_precision@10
275
+ value: 0.21400000000000002
276
+ name: Dot Precision@10
277
+ - type: dot_recall@1
278
+ value: 0.00781726182205936
279
+ name: Dot Recall@1
280
+ - type: dot_recall@3
281
+ value: 0.039232523010377704
282
+ name: Dot Recall@3
283
+ - type: dot_recall@5
284
+ value: 0.06598978333552999
285
+ name: Dot Recall@5
286
+ - type: dot_recall@10
287
+ value: 0.08530284648826633
288
+ name: Dot Recall@10
289
+ - type: dot_ndcg@10
290
+ value: 0.23949057652351813
291
+ name: Dot Ndcg@10
292
+ - type: dot_mrr@10
293
+ value: 0.3643571428571429
294
+ name: Dot Mrr@10
295
+ - type: dot_map@100
296
+ value: 0.0903057527304283
297
+ name: Dot Map@100
298
+ - type: query_active_dims
299
+ value: 156.66000366210938
300
+ name: Query Active Dims
301
+ - type: query_sparsity_ratio
302
+ value: 0.9948673087064377
303
+ name: Query Sparsity Ratio
304
+ - type: corpus_active_dims
305
+ value: 505.35760498046875
306
+ name: Corpus Active Dims
307
+ - type: corpus_sparsity_ratio
308
+ value: 0.9834428410661009
309
+ name: Corpus Sparsity Ratio
310
+ - task:
311
+ type: sparse-information-retrieval
312
+ name: Sparse Information Retrieval
313
+ dataset:
314
+ name: NanoNQ
315
+ type: NanoNQ
316
+ metrics:
317
+ - type: dot_accuracy@1
318
+ value: 0.22
319
+ name: Dot Accuracy@1
320
+ - type: dot_accuracy@3
321
+ value: 0.54
322
+ name: Dot Accuracy@3
323
+ - type: dot_accuracy@5
324
+ value: 0.66
325
+ name: Dot Accuracy@5
326
+ - type: dot_accuracy@10
327
+ value: 0.76
328
+ name: Dot Accuracy@10
329
+ - type: dot_precision@1
330
+ value: 0.22
331
+ name: Dot Precision@1
332
+ - type: dot_precision@3
333
+ value: 0.18
334
+ name: Dot Precision@3
335
+ - type: dot_precision@5
336
+ value: 0.14
337
+ name: Dot Precision@5
338
+ - type: dot_precision@10
339
+ value: 0.08
340
+ name: Dot Precision@10
341
+ - type: dot_recall@1
342
+ value: 0.21
343
+ name: Dot Recall@1
344
+ - type: dot_recall@3
345
+ value: 0.49
346
+ name: Dot Recall@3
347
+ - type: dot_recall@5
348
+ value: 0.63
349
+ name: Dot Recall@5
350
+ - type: dot_recall@10
351
+ value: 0.73
352
+ name: Dot Recall@10
353
+ - type: dot_ndcg@10
354
+ value: 0.4687690734978834
355
+ name: Dot Ndcg@10
356
+ - type: dot_mrr@10
357
+ value: 0.3937460317460317
358
+ name: Dot Mrr@10
359
+ - type: dot_map@100
360
+ value: 0.39105328798069705
361
+ name: Dot Map@100
362
+ - type: query_active_dims
363
+ value: 123.33999633789062
364
+ name: Query Active Dims
365
+ - type: query_sparsity_ratio
366
+ value: 0.9959589805275575
367
+ name: Query Sparsity Ratio
368
+ - type: corpus_active_dims
369
+ value: 371.6484680175781
370
+ name: Corpus Active Dims
371
+ - type: corpus_sparsity_ratio
372
+ value: 0.9878235873134926
373
+ name: Corpus Sparsity Ratio
374
+ - type: dot_accuracy@1
375
+ value: 0.36
376
+ name: Dot Accuracy@1
377
+ - type: dot_accuracy@3
378
+ value: 0.6
379
+ name: Dot Accuracy@3
380
+ - type: dot_accuracy@5
381
+ value: 0.68
382
+ name: Dot Accuracy@5
383
+ - type: dot_accuracy@10
384
+ value: 0.76
385
+ name: Dot Accuracy@10
386
+ - type: dot_precision@1
387
+ value: 0.36
388
+ name: Dot Precision@1
389
+ - type: dot_precision@3
390
+ value: 0.20666666666666664
391
+ name: Dot Precision@3
392
+ - type: dot_precision@5
393
+ value: 0.14
394
+ name: Dot Precision@5
395
+ - type: dot_precision@10
396
+ value: 0.08
397
+ name: Dot Precision@10
398
+ - type: dot_recall@1
399
+ value: 0.35
400
+ name: Dot Recall@1
401
+ - type: dot_recall@3
402
+ value: 0.58
403
+ name: Dot Recall@3
404
+ - type: dot_recall@5
405
+ value: 0.65
406
+ name: Dot Recall@5
407
+ - type: dot_recall@10
408
+ value: 0.72
409
+ name: Dot Recall@10
410
+ - type: dot_ndcg@10
411
+ value: 0.544210628480855
412
+ name: Dot Ndcg@10
413
+ - type: dot_mrr@10
414
+ value: 0.4959365079365079
415
+ name: Dot Mrr@10
416
+ - type: dot_map@100
417
+ value: 0.49453898976712246
418
+ name: Dot Map@100
419
+ - type: query_active_dims
420
+ value: 103.9000015258789
421
+ name: Query Active Dims
422
+ - type: query_sparsity_ratio
423
+ value: 0.9965958979907648
424
+ name: Query Sparsity Ratio
425
+ - type: corpus_active_dims
426
+ value: 356.2113342285156
427
+ name: Corpus Active Dims
428
+ - type: corpus_sparsity_ratio
429
+ value: 0.9883293580293391
430
+ name: Corpus Sparsity Ratio
431
+ - task:
432
+ type: sparse-nano-beir
433
+ name: Sparse Nano BEIR
434
+ dataset:
435
+ name: NanoBEIR mean
436
+ type: NanoBEIR_mean
437
+ metrics:
438
+ - type: dot_accuracy@1
439
+ value: 0.2733333333333334
440
+ name: Dot Accuracy@1
441
+ - type: dot_accuracy@3
442
+ value: 0.5
443
+ name: Dot Accuracy@3
444
+ - type: dot_accuracy@5
445
+ value: 0.6000000000000001
446
+ name: Dot Accuracy@5
447
+ - type: dot_accuracy@10
448
+ value: 0.7066666666666667
449
+ name: Dot Accuracy@10
450
+ - type: dot_precision@1
451
+ value: 0.2733333333333334
452
+ name: Dot Precision@1
453
+ - type: dot_precision@3
454
+ value: 0.2111111111111111
455
+ name: Dot Precision@3
456
+ - type: dot_precision@5
457
+ value: 0.17066666666666666
458
+ name: Dot Precision@5
459
+ - type: dot_precision@10
460
+ value: 0.12466666666666666
461
+ name: Dot Precision@10
462
+ - type: dot_recall@1
463
+ value: 0.16680328745430326
464
+ name: Dot Recall@1
465
+ - type: dot_recall@3
466
+ value: 0.33903258812424814
467
+ name: Dot Recall@3
468
+ - type: dot_recall@5
469
+ value: 0.4168654829220137
470
+ name: Dot Recall@5
471
+ - type: dot_recall@10
472
+ value: 0.5138903338139175
473
+ name: Dot Recall@10
474
+ - type: dot_ndcg@10
475
+ value: 0.4032398317740212
476
+ name: Dot Ndcg@10
477
+ - type: dot_mrr@10
478
+ value: 0.4110793650793651
479
+ name: Dot Mrr@10
480
+ - type: dot_map@100
481
+ value: 0.3020952329734635
482
+ name: Dot Map@100
483
+ - type: query_active_dims
484
+ value: 141.79332987467447
485
+ name: Query Active Dims
486
+ - type: query_sparsity_ratio
487
+ value: 0.995354389297075
488
+ name: Query Sparsity Ratio
489
+ - type: corpus_active_dims
490
+ value: 381.7902843249054
491
+ name: Corpus Active Dims
492
+ - type: corpus_sparsity_ratio
493
+ value: 0.9874913084226162
494
+ name: Corpus Sparsity Ratio
495
+ - type: dot_accuracy@1
496
+ value: 0.407032967032967
497
+ name: Dot Accuracy@1
498
+ - type: dot_accuracy@3
499
+ value: 0.6120565149136579
500
+ name: Dot Accuracy@3
501
+ - type: dot_accuracy@5
502
+ value: 0.6813500784929356
503
+ name: Dot Accuracy@5
504
+ - type: dot_accuracy@10
505
+ value: 0.7599058084772369
506
+ name: Dot Accuracy@10
507
+ - type: dot_precision@1
508
+ value: 0.407032967032967
509
+ name: Dot Precision@1
510
+ - type: dot_precision@3
511
+ value: 0.27516483516483514
512
+ name: Dot Precision@3
513
+ - type: dot_precision@5
514
+ value: 0.2152339089481947
515
+ name: Dot Precision@5
516
+ - type: dot_precision@10
517
+ value: 0.15498273155416015
518
+ name: Dot Precision@10
519
+ - type: dot_recall@1
520
+ value: 0.2321487370669766
521
+ name: Dot Recall@1
522
+ - type: dot_recall@3
523
+ value: 0.38956098353180224
524
+ name: Dot Recall@3
525
+ - type: dot_recall@5
526
+ value: 0.4491992568319475
527
+ name: Dot Recall@5
528
+ - type: dot_recall@10
529
+ value: 0.5287356542054744
530
+ name: Dot Recall@10
531
+ - type: dot_ndcg@10
532
+ value: 0.47211549606581404
533
+ name: Dot Ndcg@10
534
+ - type: dot_mrr@10
535
+ value: 0.5232897261468691
536
+ name: Dot Mrr@10
537
+ - type: dot_map@100
538
+ value: 0.3983099175669993
539
+ name: Dot Map@100
540
+ - type: query_active_dims
541
+ value: 180.88135636128337
542
+ name: Query Active Dims
543
+ - type: query_sparsity_ratio
544
+ value: 0.9940737384063533
545
+ name: Query Sparsity Ratio
546
+ - type: corpus_active_dims
547
+ value: 360.7380946225432
548
+ name: Corpus Active Dims
549
+ - type: corpus_sparsity_ratio
550
+ value: 0.9881810466344753
551
+ name: Corpus Sparsity Ratio
552
+ - task:
553
+ type: sparse-information-retrieval
554
+ name: Sparse Information Retrieval
555
+ dataset:
556
+ name: NanoClimateFEVER
557
+ type: NanoClimateFEVER
558
+ metrics:
559
+ - type: dot_accuracy@1
560
+ value: 0.24
561
+ name: Dot Accuracy@1
562
+ - type: dot_accuracy@3
563
+ value: 0.44
564
+ name: Dot Accuracy@3
565
+ - type: dot_accuracy@5
566
+ value: 0.52
567
+ name: Dot Accuracy@5
568
+ - type: dot_accuracy@10
569
+ value: 0.62
570
+ name: Dot Accuracy@10
571
+ - type: dot_precision@1
572
+ value: 0.24
573
+ name: Dot Precision@1
574
+ - type: dot_precision@3
575
+ value: 0.15333333333333332
576
+ name: Dot Precision@3
577
+ - type: dot_precision@5
578
+ value: 0.11199999999999999
579
+ name: Dot Precision@5
580
+ - type: dot_precision@10
581
+ value: 0.07400000000000001
582
+ name: Dot Precision@10
583
+ - type: dot_recall@1
584
+ value: 0.115
585
+ name: Dot Recall@1
586
+ - type: dot_recall@3
587
+ value: 0.20566666666666664
588
+ name: Dot Recall@3
589
+ - type: dot_recall@5
590
+ value: 0.254
591
+ name: Dot Recall@5
592
+ - type: dot_recall@10
593
+ value: 0.303
594
+ name: Dot Recall@10
595
+ - type: dot_ndcg@10
596
+ value: 0.25094049425975773
597
+ name: Dot Ndcg@10
598
+ - type: dot_mrr@10
599
+ value: 0.3527698412698412
600
+ name: Dot Mrr@10
601
+ - type: dot_map@100
602
+ value: 0.1922739754314787
603
+ name: Dot Map@100
604
+ - type: query_active_dims
605
+ value: 240.47999572753906
606
+ name: Query Active Dims
607
+ - type: query_sparsity_ratio
608
+ value: 0.992121093122091
609
+ name: Query Sparsity Ratio
610
+ - type: corpus_active_dims
611
+ value: 398.276123046875
612
+ name: Corpus Active Dims
613
+ - type: corpus_sparsity_ratio
614
+ value: 0.9869511787220079
615
+ name: Corpus Sparsity Ratio
616
+ - task:
617
+ type: sparse-information-retrieval
618
+ name: Sparse Information Retrieval
619
+ dataset:
620
+ name: NanoDBPedia
621
+ type: NanoDBPedia
622
+ metrics:
623
+ - type: dot_accuracy@1
624
+ value: 0.6
625
+ name: Dot Accuracy@1
626
+ - type: dot_accuracy@3
627
+ value: 0.78
628
+ name: Dot Accuracy@3
629
+ - type: dot_accuracy@5
630
+ value: 0.84
631
+ name: Dot Accuracy@5
632
+ - type: dot_accuracy@10
633
+ value: 0.9
634
+ name: Dot Accuracy@10
635
+ - type: dot_precision@1
636
+ value: 0.6
637
+ name: Dot Precision@1
638
+ - type: dot_precision@3
639
+ value: 0.4866666666666666
640
+ name: Dot Precision@3
641
+ - type: dot_precision@5
642
+ value: 0.44799999999999995
643
+ name: Dot Precision@5
644
+ - type: dot_precision@10
645
+ value: 0.38800000000000007
646
+ name: Dot Precision@10
647
+ - type: dot_recall@1
648
+ value: 0.07504886736842241
649
+ name: Dot Recall@1
650
+ - type: dot_recall@3
651
+ value: 0.14295532639016004
652
+ name: Dot Recall@3
653
+ - type: dot_recall@5
654
+ value: 0.17962338014181914
655
+ name: Dot Recall@5
656
+ - type: dot_recall@10
657
+ value: 0.2627232266447939
658
+ name: Dot Recall@10
659
+ - type: dot_ndcg@10
660
+ value: 0.4899447932345138
661
+ name: Dot Ndcg@10
662
+ - type: dot_mrr@10
663
+ value: 0.7126666666666667
664
+ name: Dot Mrr@10
665
+ - type: dot_map@100
666
+ value: 0.3806659729692501
667
+ name: Dot Map@100
668
+ - type: query_active_dims
669
+ value: 159.22000122070312
670
+ name: Query Active Dims
671
+ - type: query_sparsity_ratio
672
+ value: 0.9947834348594226
673
+ name: Query Sparsity Ratio
674
+ - type: corpus_active_dims
675
+ value: 347.9973449707031
676
+ name: Corpus Active Dims
677
+ - type: corpus_sparsity_ratio
678
+ value: 0.9885984750353612
679
+ name: Corpus Sparsity Ratio
680
+ - task:
681
+ type: sparse-information-retrieval
682
+ name: Sparse Information Retrieval
683
+ dataset:
684
+ name: NanoFEVER
685
+ type: NanoFEVER
686
+ metrics:
687
+ - type: dot_accuracy@1
688
+ value: 0.54
689
+ name: Dot Accuracy@1
690
+ - type: dot_accuracy@3
691
+ value: 0.78
692
+ name: Dot Accuracy@3
693
+ - type: dot_accuracy@5
694
+ value: 0.9
695
+ name: Dot Accuracy@5
696
+ - type: dot_accuracy@10
697
+ value: 0.9
698
+ name: Dot Accuracy@10
699
+ - type: dot_precision@1
700
+ value: 0.54
701
+ name: Dot Precision@1
702
+ - type: dot_precision@3
703
+ value: 0.26
704
+ name: Dot Precision@3
705
+ - type: dot_precision@5
706
+ value: 0.18
707
+ name: Dot Precision@5
708
+ - type: dot_precision@10
709
+ value: 0.09399999999999999
710
+ name: Dot Precision@10
711
+ - type: dot_recall@1
712
+ value: 0.5166666666666666
713
+ name: Dot Recall@1
714
+ - type: dot_recall@3
715
+ value: 0.7266666666666667
716
+ name: Dot Recall@3
717
+ - type: dot_recall@5
718
+ value: 0.8366666666666667
719
+ name: Dot Recall@5
720
+ - type: dot_recall@10
721
+ value: 0.8566666666666667
722
+ name: Dot Recall@10
723
+ - type: dot_ndcg@10
724
+ value: 0.7043014395888793
725
+ name: Dot Ndcg@10
726
+ - type: dot_mrr@10
727
+ value: 0.6723333333333333
728
+ name: Dot Mrr@10
729
+ - type: dot_map@100
730
+ value: 0.6525721659293088
731
+ name: Dot Map@100
732
+ - type: query_active_dims
733
+ value: 211.27999877929688
734
+ name: Query Active Dims
735
+ - type: query_sparsity_ratio
736
+ value: 0.9930777800019889
737
+ name: Query Sparsity Ratio
738
+ - type: corpus_active_dims
739
+ value: 428.28521728515625
740
+ name: Corpus Active Dims
741
+ - type: corpus_sparsity_ratio
742
+ value: 0.9859679831831087
743
+ name: Corpus Sparsity Ratio
744
+ - task:
745
+ type: sparse-information-retrieval
746
+ name: Sparse Information Retrieval
747
+ dataset:
748
+ name: NanoFiQA2018
749
+ type: NanoFiQA2018
750
+ metrics:
751
+ - type: dot_accuracy@1
752
+ value: 0.32
753
+ name: Dot Accuracy@1
754
+ - type: dot_accuracy@3
755
+ value: 0.48
756
+ name: Dot Accuracy@3
757
+ - type: dot_accuracy@5
758
+ value: 0.52
759
+ name: Dot Accuracy@5
760
+ - type: dot_accuracy@10
761
+ value: 0.62
762
+ name: Dot Accuracy@10
763
+ - type: dot_precision@1
764
+ value: 0.32
765
+ name: Dot Precision@1
766
+ - type: dot_precision@3
767
+ value: 0.21333333333333332
768
+ name: Dot Precision@3
769
+ - type: dot_precision@5
770
+ value: 0.15600000000000003
771
+ name: Dot Precision@5
772
+ - type: dot_precision@10
773
+ value: 0.102
774
+ name: Dot Precision@10
775
+ - type: dot_recall@1
776
+ value: 0.18219047619047618
777
+ name: Dot Recall@1
778
+ - type: dot_recall@3
779
+ value: 0.29224603174603175
780
+ name: Dot Recall@3
781
+ - type: dot_recall@5
782
+ value: 0.34507936507936504
783
+ name: Dot Recall@5
784
+ - type: dot_recall@10
785
+ value: 0.46135714285714285
786
+ name: Dot Recall@10
787
+ - type: dot_ndcg@10
788
+ value: 0.3638661029288861
789
+ name: Dot Ndcg@10
790
+ - type: dot_mrr@10
791
+ value: 0.40916666666666673
792
+ name: Dot Mrr@10
793
+ - type: dot_map@100
794
+ value: 0.3016646430756084
795
+ name: Dot Map@100
796
+ - type: query_active_dims
797
+ value: 103.12000274658203
798
+ name: Query Active Dims
799
+ - type: query_sparsity_ratio
800
+ value: 0.9966214532879044
801
+ name: Query Sparsity Ratio
802
+ - type: corpus_active_dims
803
+ value: 340.604248046875
804
+ name: Corpus Active Dims
805
+ - type: corpus_sparsity_ratio
806
+ value: 0.9888406969383764
807
+ name: Corpus Sparsity Ratio
808
+ - task:
809
+ type: sparse-information-retrieval
810
+ name: Sparse Information Retrieval
811
+ dataset:
812
+ name: NanoHotpotQA
813
+ type: NanoHotpotQA
814
+ metrics:
815
+ - type: dot_accuracy@1
816
+ value: 0.72
817
+ name: Dot Accuracy@1
818
+ - type: dot_accuracy@3
819
+ value: 0.78
820
+ name: Dot Accuracy@3
821
+ - type: dot_accuracy@5
822
+ value: 0.8
823
+ name: Dot Accuracy@5
824
+ - type: dot_accuracy@10
825
+ value: 0.92
826
+ name: Dot Accuracy@10
827
+ - type: dot_precision@1
828
+ value: 0.72
829
+ name: Dot Precision@1
830
+ - type: dot_precision@3
831
+ value: 0.4133333333333333
832
+ name: Dot Precision@3
833
+ - type: dot_precision@5
834
+ value: 0.25999999999999995
835
+ name: Dot Precision@5
836
+ - type: dot_precision@10
837
+ value: 0.152
838
+ name: Dot Precision@10
839
+ - type: dot_recall@1
840
+ value: 0.36
841
+ name: Dot Recall@1
842
+ - type: dot_recall@3
843
+ value: 0.62
844
+ name: Dot Recall@3
845
+ - type: dot_recall@5
846
+ value: 0.65
847
+ name: Dot Recall@5
848
+ - type: dot_recall@10
849
+ value: 0.76
850
+ name: Dot Recall@10
851
+ - type: dot_ndcg@10
852
+ value: 0.6875870542669431
853
+ name: Dot Ndcg@10
854
+ - type: dot_mrr@10
855
+ value: 0.7661031746031746
856
+ name: Dot Mrr@10
857
+ - type: dot_map@100
858
+ value: 0.6246470320917622
859
+ name: Dot Map@100
860
+ - type: query_active_dims
861
+ value: 132.77999877929688
862
+ name: Query Active Dims
863
+ - type: query_sparsity_ratio
864
+ value: 0.9956496953417437
865
+ name: Query Sparsity Ratio
866
+ - type: corpus_active_dims
867
+ value: 392.06817626953125
868
+ name: Corpus Active Dims
869
+ - type: corpus_sparsity_ratio
870
+ value: 0.987154571251244
871
+ name: Corpus Sparsity Ratio
872
+ - task:
873
+ type: sparse-information-retrieval
874
+ name: Sparse Information Retrieval
875
+ dataset:
876
+ name: NanoQuoraRetrieval
877
+ type: NanoQuoraRetrieval
878
+ metrics:
879
+ - type: dot_accuracy@1
880
+ value: 0.5
881
+ name: Dot Accuracy@1
882
+ - type: dot_accuracy@3
883
+ value: 0.74
884
+ name: Dot Accuracy@3
885
+ - type: dot_accuracy@5
886
+ value: 0.84
887
+ name: Dot Accuracy@5
888
+ - type: dot_accuracy@10
889
+ value: 0.96
890
+ name: Dot Accuracy@10
891
+ - type: dot_precision@1
892
+ value: 0.5
893
+ name: Dot Precision@1
894
+ - type: dot_precision@3
895
+ value: 0.26
896
+ name: Dot Precision@3
897
+ - type: dot_precision@5
898
+ value: 0.18799999999999997
899
+ name: Dot Precision@5
900
+ - type: dot_precision@10
901
+ value: 0.118
902
+ name: Dot Precision@10
903
+ - type: dot_recall@1
904
+ value: 0.49
905
+ name: Dot Recall@1
906
+ - type: dot_recall@3
907
+ value: 0.7066666666666666
908
+ name: Dot Recall@3
909
+ - type: dot_recall@5
910
+ value: 0.8013333333333332
911
+ name: Dot Recall@5
912
+ - type: dot_recall@10
913
+ value: 0.9133333333333334
914
+ name: Dot Recall@10
915
+ - type: dot_ndcg@10
916
+ value: 0.7141904974177148
917
+ name: Dot Ndcg@10
918
+ - type: dot_mrr@10
919
+ value: 0.6457777777777778
920
+ name: Dot Mrr@10
921
+ - type: dot_map@100
922
+ value: 0.6498892824353822
923
+ name: Dot Map@100
924
+ - type: query_active_dims
925
+ value: 63.400001525878906
926
+ name: Query Active Dims
927
+ - type: query_sparsity_ratio
928
+ value: 0.9979228097265619
929
+ name: Query Sparsity Ratio
930
+ - type: corpus_active_dims
931
+ value: 73.4577865600586
932
+ name: Corpus Active Dims
933
+ - type: corpus_sparsity_ratio
934
+ value: 0.9975932839735254
935
+ name: Corpus Sparsity Ratio
936
+ - task:
937
+ type: sparse-information-retrieval
938
+ name: Sparse Information Retrieval
939
+ dataset:
940
+ name: NanoSCIDOCS
941
+ type: NanoSCIDOCS
942
+ metrics:
943
+ - type: dot_accuracy@1
944
+ value: 0.36
945
+ name: Dot Accuracy@1
946
+ - type: dot_accuracy@3
947
+ value: 0.54
948
+ name: Dot Accuracy@3
949
+ - type: dot_accuracy@5
950
+ value: 0.68
951
+ name: Dot Accuracy@5
952
+ - type: dot_accuracy@10
953
+ value: 0.76
954
+ name: Dot Accuracy@10
955
+ - type: dot_precision@1
956
+ value: 0.36
957
+ name: Dot Precision@1
958
+ - type: dot_precision@3
959
+ value: 0.24666666666666667
960
+ name: Dot Precision@3
961
+ - type: dot_precision@5
962
+ value: 0.212
963
+ name: Dot Precision@5
964
+ - type: dot_precision@10
965
+ value: 0.152
966
+ name: Dot Precision@10
967
+ - type: dot_recall@1
968
+ value: 0.07566666666666669
969
+ name: Dot Recall@1
970
+ - type: dot_recall@3
971
+ value: 0.15266666666666667
972
+ name: Dot Recall@3
973
+ - type: dot_recall@5
974
+ value: 0.21866666666666668
975
+ name: Dot Recall@5
976
+ - type: dot_recall@10
977
+ value: 0.31366666666666665
978
+ name: Dot Recall@10
979
+ - type: dot_ndcg@10
980
+ value: 0.29765744924419957
981
+ name: Dot Ndcg@10
982
+ - type: dot_mrr@10
983
+ value: 0.47960317460317453
984
+ name: Dot Mrr@10
985
+ - type: dot_map@100
986
+ value: 0.21728096438859665
987
+ name: Dot Map@100
988
+ - type: query_active_dims
989
+ value: 247.55999755859375
990
+ name: Query Active Dims
991
+ - type: query_sparsity_ratio
992
+ value: 0.9918891292327306
993
+ name: Query Sparsity Ratio
994
+ - type: corpus_active_dims
995
+ value: 424.1747131347656
996
+ name: Corpus Active Dims
997
+ - type: corpus_sparsity_ratio
998
+ value: 0.9861026566694592
999
+ name: Corpus Sparsity Ratio
1000
+ - task:
1001
+ type: sparse-information-retrieval
1002
+ name: Sparse Information Retrieval
1003
+ dataset:
1004
+ name: NanoArguAna
1005
+ type: NanoArguAna
1006
+ metrics:
1007
+ - type: dot_accuracy@1
1008
+ value: 0.06
1009
+ name: Dot Accuracy@1
1010
+ - type: dot_accuracy@3
1011
+ value: 0.38
1012
+ name: Dot Accuracy@3
1013
+ - type: dot_accuracy@5
1014
+ value: 0.44
1015
+ name: Dot Accuracy@5
1016
+ - type: dot_accuracy@10
1017
+ value: 0.5
1018
+ name: Dot Accuracy@10
1019
+ - type: dot_precision@1
1020
+ value: 0.06
1021
+ name: Dot Precision@1
1022
+ - type: dot_precision@3
1023
+ value: 0.12666666666666665
1024
+ name: Dot Precision@3
1025
+ - type: dot_precision@5
1026
+ value: 0.08800000000000001
1027
+ name: Dot Precision@5
1028
+ - type: dot_precision@10
1029
+ value: 0.05000000000000001
1030
+ name: Dot Precision@10
1031
+ - type: dot_recall@1
1032
+ value: 0.06
1033
+ name: Dot Recall@1
1034
+ - type: dot_recall@3
1035
+ value: 0.38
1036
+ name: Dot Recall@3
1037
+ - type: dot_recall@5
1038
+ value: 0.44
1039
+ name: Dot Recall@5
1040
+ - type: dot_recall@10
1041
+ value: 0.5
1042
+ name: Dot Recall@10
1043
+ - type: dot_ndcg@10
1044
+ value: 0.29205612820937377
1045
+ name: Dot Ndcg@10
1046
+ - type: dot_mrr@10
1047
+ value: 0.22366666666666668
1048
+ name: Dot Mrr@10
1049
+ - type: dot_map@100
1050
+ value: 0.23474188188747466
1051
+ name: Dot Map@100
1052
+ - type: query_active_dims
1053
+ value: 477.05999755859375
1054
+ name: Query Active Dims
1055
+ - type: query_sparsity_ratio
1056
+ value: 0.9843699627298803
1057
+ name: Query Sparsity Ratio
1058
+ - type: corpus_active_dims
1059
+ value: 455.6429138183594
1060
+ name: Corpus Active Dims
1061
+ - type: corpus_sparsity_ratio
1062
+ value: 0.9850716560573239
1063
+ name: Corpus Sparsity Ratio
1064
+ - task:
1065
+ type: sparse-information-retrieval
1066
+ name: Sparse Information Retrieval
1067
+ dataset:
1068
+ name: NanoSciFact
1069
+ type: NanoSciFact
1070
+ metrics:
1071
+ - type: dot_accuracy@1
1072
+ value: 0.5
1073
+ name: Dot Accuracy@1
1074
+ - type: dot_accuracy@3
1075
+ value: 0.58
1076
+ name: Dot Accuracy@3
1077
+ - type: dot_accuracy@5
1078
+ value: 0.64
1079
+ name: Dot Accuracy@5
1080
+ - type: dot_accuracy@10
1081
+ value: 0.7
1082
+ name: Dot Accuracy@10
1083
+ - type: dot_precision@1
1084
+ value: 0.5
1085
+ name: Dot Precision@1
1086
+ - type: dot_precision@3
1087
+ value: 0.20666666666666664
1088
+ name: Dot Precision@3
1089
+ - type: dot_precision@5
1090
+ value: 0.136
1091
+ name: Dot Precision@5
1092
+ - type: dot_precision@10
1093
+ value: 0.08
1094
+ name: Dot Precision@10
1095
+ - type: dot_recall@1
1096
+ value: 0.465
1097
+ name: Dot Recall@1
1098
+ - type: dot_recall@3
1099
+ value: 0.545
1100
+ name: Dot Recall@3
1101
+ - type: dot_recall@5
1102
+ value: 0.605
1103
+ name: Dot Recall@5
1104
+ - type: dot_recall@10
1105
+ value: 0.69
1106
+ name: Dot Recall@10
1107
+ - type: dot_ndcg@10
1108
+ value: 0.5783252903985125
1109
+ name: Dot Ndcg@10
1110
+ - type: dot_mrr@10
1111
+ value: 0.5562222222222222
1112
+ name: Dot Mrr@10
1113
+ - type: dot_map@100
1114
+ value: 0.5447263194322018
1115
+ name: Dot Map@100
1116
+ - type: query_active_dims
1117
+ value: 280.32000732421875
1118
+ name: Query Active Dims
1119
+ - type: query_sparsity_ratio
1120
+ value: 0.9908158047531544
1121
+ name: Query Sparsity Ratio
1122
+ - type: corpus_active_dims
1123
+ value: 451.07366943359375
1124
+ name: Corpus Active Dims
1125
+ - type: corpus_sparsity_ratio
1126
+ value: 0.9852213593659134
1127
+ name: Corpus Sparsity Ratio
1128
+ - task:
1129
+ type: sparse-information-retrieval
1130
+ name: Sparse Information Retrieval
1131
+ dataset:
1132
+ name: NanoTouche2020
1133
+ type: NanoTouche2020
1134
+ metrics:
1135
+ - type: dot_accuracy@1
1136
+ value: 0.5714285714285714
1137
+ name: Dot Accuracy@1
1138
+ - type: dot_accuracy@3
1139
+ value: 0.8367346938775511
1140
+ name: Dot Accuracy@3
1141
+ - type: dot_accuracy@5
1142
+ value: 0.8775510204081632
1143
+ name: Dot Accuracy@5
1144
+ - type: dot_accuracy@10
1145
+ value: 0.9387755102040817
1146
+ name: Dot Accuracy@10
1147
+ - type: dot_precision@1
1148
+ value: 0.5714285714285714
1149
+ name: Dot Precision@1
1150
+ - type: dot_precision@3
1151
+ value: 0.5238095238095238
1152
+ name: Dot Precision@3
1153
+ - type: dot_precision@5
1154
+ value: 0.5020408163265306
1155
+ name: Dot Precision@5
1156
+ - type: dot_precision@10
1157
+ value: 0.4387755102040816
1158
+ name: Dot Precision@10
1159
+ - type: dot_recall@1
1160
+ value: 0.040543643156404505
1161
+ name: Dot Recall@1
1162
+ - type: dot_recall@3
1163
+ value: 0.11319223810019327
1164
+ name: Dot Recall@3
1165
+ - type: dot_recall@5
1166
+ value: 0.17323114359193661
1167
+ name: Dot Recall@5
1168
+ - type: dot_recall@10
1169
+ value: 0.2875136220142983
1170
+ name: Dot Recall@10
1171
+ - type: dot_ndcg@10
1172
+ value: 0.4859066604692993
1173
+ name: Dot Ndcg@10
1174
+ - type: dot_mrr@10
1175
+ value: 0.7081632653061225
1176
+ name: Dot Mrr@10
1177
+ - type: dot_map@100
1178
+ value: 0.36459453741861036
1179
+ name: Dot Map@100
1180
+ - type: query_active_dims
1181
+ value: 61.836734771728516
1182
+ name: Query Active Dims
1183
+ - type: query_sparsity_ratio
1184
+ value: 0.9979740274303215
1185
+ name: Query Sparsity Ratio
1186
+ - type: corpus_active_dims
1187
+ value: 380.6022644042969
1188
+ name: Corpus Active Dims
1189
+ - type: corpus_sparsity_ratio
1190
+ value: 0.9875302318195303
1191
+ name: Corpus Sparsity Ratio
1192
+ ---
1193
+
1194
+ # splade-distilbert-base-uncased trained on GooAQ
1195
+
1196
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
1197
+ ## Model Details
1198
+
1199
+ ### Model Description
1200
+ - **Model Type:** SPLADE Sparse Encoder
1201
+ - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) <!-- at revision 12040accade4e8a0f71eabdb258fecc2e7e948be -->
1202
+ - **Maximum Sequence Length:** 256 tokens
1203
+ - **Output Dimensionality:** 30522 dimensions
1204
+ - **Similarity Function:** Dot Product
1205
+ - **Training Dataset:**
1206
+ - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
1207
+ - **Language:** en
1208
+ - **License:** apache-2.0
1209
+
1210
+ ### Model Sources
1211
+
1212
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
1213
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
1214
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
1215
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
1216
+
1217
+ ### Full Model Architecture
1218
+
1219
+ ```
1220
+ SparseEncoder(
1221
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'DistilBertForMaskedLM'})
1222
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
1223
+ )
1224
+ ```
1225
+
1226
+ ## Usage
1227
+
1228
+ ### Direct Usage (Sentence Transformers)
1229
+
1230
+ First install the Sentence Transformers library:
1231
+
1232
+ ```bash
1233
+ pip install -U sentence-transformers
1234
+ ```
1235
+
1236
+ Then you can load this model and run inference.
1237
+ ```python
1238
+ from sentence_transformers import SparseEncoder
1239
+
1240
+ # Download from the 🤗 Hub
1241
+ model = SparseEncoder("tomaarsen/splade-distilbert-base-uncased-gooaq")
1242
+ # Run inference
1243
+ queries = [
1244
+ "how many days for doxycycline to work on sinus infection?",
1245
+ ]
1246
+ documents = [
1247
+ 'Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks for chronic sinusitis.',
1248
+ 'Most engagements typically have a cocktail dress code, calling for dresses at, or slightly above, knee-length and high heels. If your party states a different dress code, however, such as semi-formal or dressy-casual, you may need to dress up or down accordingly.',
1249
+ 'The average service life of a gas furnace is about 15 years, but the actual life span of an individual unit can vary greatly. There are a number of contributing factors that determine the age a furnace reaches: The quality of the equipment.',
1250
+ ]
1251
+ query_embeddings = model.encode_query(queries)
1252
+ document_embeddings = model.encode_document(documents)
1253
+ print(query_embeddings.shape, document_embeddings.shape)
1254
+ # [1, 30522] [3, 30522]
1255
+
1256
+ # Get the similarity scores for the embeddings
1257
+ similarities = model.similarity(query_embeddings, document_embeddings)
1258
+ print(similarities)
1259
+ # tensor([[103.7028, 26.2666, 35.3421]])
1260
+ ```
1261
+
1262
+ <!--
1263
+ ### Direct Usage (Transformers)
1264
+
1265
+ <details><summary>Click to see the direct usage in Transformers</summary>
1266
+
1267
+ </details>
1268
+ -->
1269
+
1270
+ <!--
1271
+ ### Downstream Usage (Sentence Transformers)
1272
+
1273
+ You can finetune this model on your own dataset.
1274
+
1275
+ <details><summary>Click to expand</summary>
1276
+
1277
+ </details>
1278
+ -->
1279
+
1280
+ <!--
1281
+ ### Out-of-Scope Use
1282
+
1283
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
1284
+ -->
1285
+
1286
+ ## Evaluation
1287
+
1288
+ ### Metrics
1289
+
1290
+ #### Sparse Information Retrieval
1291
+
1292
+ * Datasets: `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
1293
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
1294
+
1295
+ | Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
1296
+ |:----------------------|:------------|:-------------|:-----------|:-----------------|:------------|:-----------|:-------------|:-------------|:-------------------|:------------|:------------|:------------|:---------------|
1297
+ | dot_accuracy@1 | 0.28 | 0.24 | 0.36 | 0.24 | 0.6 | 0.54 | 0.32 | 0.72 | 0.5 | 0.36 | 0.06 | 0.5 | 0.5714 |
1298
+ | dot_accuracy@3 | 0.56 | 0.46 | 0.6 | 0.44 | 0.78 | 0.78 | 0.48 | 0.78 | 0.74 | 0.54 | 0.38 | 0.58 | 0.8367 |
1299
+ | dot_accuracy@5 | 0.62 | 0.5 | 0.68 | 0.52 | 0.84 | 0.9 | 0.52 | 0.8 | 0.84 | 0.68 | 0.44 | 0.64 | 0.8776 |
1300
+ | dot_accuracy@10 | 0.72 | 0.58 | 0.76 | 0.62 | 0.9 | 0.9 | 0.62 | 0.92 | 0.96 | 0.76 | 0.5 | 0.7 | 0.9388 |
1301
+ | dot_precision@1 | 0.28 | 0.24 | 0.36 | 0.24 | 0.6 | 0.54 | 0.32 | 0.72 | 0.5 | 0.36 | 0.06 | 0.5 | 0.5714 |
1302
+ | dot_precision@3 | 0.1867 | 0.2933 | 0.2067 | 0.1533 | 0.4867 | 0.26 | 0.2133 | 0.4133 | 0.26 | 0.2467 | 0.1267 | 0.2067 | 0.5238 |
1303
+ | dot_precision@5 | 0.124 | 0.252 | 0.14 | 0.112 | 0.448 | 0.18 | 0.156 | 0.26 | 0.188 | 0.212 | 0.088 | 0.136 | 0.502 |
1304
+ | dot_precision@10 | 0.072 | 0.214 | 0.08 | 0.074 | 0.388 | 0.094 | 0.102 | 0.152 | 0.118 | 0.152 | 0.05 | 0.08 | 0.4388 |
1305
+ | dot_recall@1 | 0.28 | 0.0078 | 0.35 | 0.115 | 0.075 | 0.5167 | 0.1822 | 0.36 | 0.49 | 0.0757 | 0.06 | 0.465 | 0.0405 |
1306
+ | dot_recall@3 | 0.56 | 0.0392 | 0.58 | 0.2057 | 0.143 | 0.7267 | 0.2922 | 0.62 | 0.7067 | 0.1527 | 0.38 | 0.545 | 0.1132 |
1307
+ | dot_recall@5 | 0.62 | 0.066 | 0.65 | 0.254 | 0.1796 | 0.8367 | 0.3451 | 0.65 | 0.8013 | 0.2187 | 0.44 | 0.605 | 0.1732 |
1308
+ | dot_recall@10 | 0.72 | 0.0853 | 0.72 | 0.303 | 0.2627 | 0.8567 | 0.4614 | 0.76 | 0.9133 | 0.3137 | 0.5 | 0.69 | 0.2875 |
1309
+ | **dot_ndcg@10** | **0.489** | **0.2395** | **0.5442** | **0.2509** | **0.4899** | **0.7043** | **0.3639** | **0.6876** | **0.7142** | **0.2977** | **0.2921** | **0.5783** | **0.4859** |
1310
+ | dot_mrr@10 | 0.416 | 0.3644 | 0.4959 | 0.3528 | 0.7127 | 0.6723 | 0.4092 | 0.7661 | 0.6458 | 0.4796 | 0.2237 | 0.5562 | 0.7082 |
1311
+ | dot_map@100 | 0.4301 | 0.0903 | 0.4945 | 0.1923 | 0.3807 | 0.6526 | 0.3017 | 0.6246 | 0.6499 | 0.2173 | 0.2347 | 0.5447 | 0.3646 |
1312
+ | query_active_dims | 111.46 | 156.66 | 103.9 | 240.48 | 159.22 | 211.28 | 103.12 | 132.78 | 63.4 | 247.56 | 477.06 | 280.32 | 61.8367 |
1313
+ | query_sparsity_ratio | 0.9963 | 0.9949 | 0.9966 | 0.9921 | 0.9948 | 0.9931 | 0.9966 | 0.9956 | 0.9979 | 0.9919 | 0.9844 | 0.9908 | 0.998 |
1314
+ | corpus_active_dims | 310.8414 | 505.3576 | 356.2113 | 398.2761 | 347.9973 | 428.2852 | 340.6042 | 392.0682 | 73.4578 | 424.1747 | 455.6429 | 451.0737 | 380.6023 |
1315
+ | corpus_sparsity_ratio | 0.9898 | 0.9834 | 0.9883 | 0.987 | 0.9886 | 0.986 | 0.9888 | 0.9872 | 0.9976 | 0.9861 | 0.9851 | 0.9852 | 0.9875 |
1316
+
1317
+ #### Sparse Nano BEIR
1318
+
1319
+ * Dataset: `NanoBEIR_mean`
1320
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1321
+ ```json
1322
+ {
1323
+ "dataset_names": [
1324
+ "msmarco",
1325
+ "nfcorpus",
1326
+ "nq"
1327
+ ]
1328
+ }
1329
+ ```
1330
+
1331
+ | Metric | Value |
1332
+ |:----------------------|:-----------|
1333
+ | dot_accuracy@1 | 0.2733 |
1334
+ | dot_accuracy@3 | 0.5 |
1335
+ | dot_accuracy@5 | 0.6 |
1336
+ | dot_accuracy@10 | 0.7067 |
1337
+ | dot_precision@1 | 0.2733 |
1338
+ | dot_precision@3 | 0.2111 |
1339
+ | dot_precision@5 | 0.1707 |
1340
+ | dot_precision@10 | 0.1247 |
1341
+ | dot_recall@1 | 0.1668 |
1342
+ | dot_recall@3 | 0.339 |
1343
+ | dot_recall@5 | 0.4169 |
1344
+ | dot_recall@10 | 0.5139 |
1345
+ | **dot_ndcg@10** | **0.4032** |
1346
+ | dot_mrr@10 | 0.4111 |
1347
+ | dot_map@100 | 0.3021 |
1348
+ | query_active_dims | 141.7933 |
1349
+ | query_sparsity_ratio | 0.9954 |
1350
+ | corpus_active_dims | 381.7903 |
1351
+ | corpus_sparsity_ratio | 0.9875 |
1352
+
1353
+ #### Sparse Nano BEIR
1354
+
1355
+ * Dataset: `NanoBEIR_mean`
1356
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1357
+ ```json
1358
+ {
1359
+ "dataset_names": [
1360
+ "climatefever",
1361
+ "dbpedia",
1362
+ "fever",
1363
+ "fiqa2018",
1364
+ "hotpotqa",
1365
+ "msmarco",
1366
+ "nfcorpus",
1367
+ "nq",
1368
+ "quoraretrieval",
1369
+ "scidocs",
1370
+ "arguana",
1371
+ "scifact",
1372
+ "touche2020"
1373
+ ]
1374
+ }
1375
+ ```
1376
+
1377
+ | Metric | Value |
1378
+ |:----------------------|:-----------|
1379
+ | dot_accuracy@1 | 0.407 |
1380
+ | dot_accuracy@3 | 0.6121 |
1381
+ | dot_accuracy@5 | 0.6814 |
1382
+ | dot_accuracy@10 | 0.7599 |
1383
+ | dot_precision@1 | 0.407 |
1384
+ | dot_precision@3 | 0.2752 |
1385
+ | dot_precision@5 | 0.2152 |
1386
+ | dot_precision@10 | 0.155 |
1387
+ | dot_recall@1 | 0.2321 |
1388
+ | dot_recall@3 | 0.3896 |
1389
+ | dot_recall@5 | 0.4492 |
1390
+ | dot_recall@10 | 0.5287 |
1391
+ | **dot_ndcg@10** | **0.4721** |
1392
+ | dot_mrr@10 | 0.5233 |
1393
+ | dot_map@100 | 0.3983 |
1394
+ | query_active_dims | 180.8814 |
1395
+ | query_sparsity_ratio | 0.9941 |
1396
+ | corpus_active_dims | 360.7381 |
1397
+ | corpus_sparsity_ratio | 0.9882 |
1398
+
1399
+ <!--
1400
+ ## Bias, Risks and Limitations
1401
+
1402
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1403
+ -->
1404
+
1405
+ <!--
1406
+ ### Recommendations
1407
+
1408
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1409
+ -->
1410
+
1411
+ ## Training Details
1412
+
1413
+ ### Training Dataset
1414
+
1415
+ #### gooaq
1416
+
1417
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1418
+ * Size: 99,000 training samples
1419
+ * Columns: <code>question</code> and <code>answer</code>
1420
+ * Approximate statistics based on the first 1000 samples:
1421
+ | | question | answer |
1422
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1423
+ | type | string | string |
1424
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.79 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.02 tokens</li><li>max: 153 tokens</li></ul> |
1425
+ * Samples:
1426
+ | question | answer |
1427
+ |:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1428
+ | <code>what are the 5 characteristics of a star?</code> | <code>Key Concept: Characteristics used to classify stars include color, temperature, size, composition, and brightness.</code> |
1429
+ | <code>are copic markers alcohol ink?</code> | <code>Copic Ink is alcohol-based and flammable. Keep away from direct sunlight and extreme temperatures.</code> |
1430
+ | <code>what is the difference between appellate term and appellate division?</code> | <code>Appellate terms An appellate term is an intermediate appellate court that hears appeals from the inferior courts within their designated counties or judicial districts, and are intended to ease the workload on the Appellate Division and provide a less expensive forum closer to the people.</code> |
1431
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1432
+ ```json
1433
+ {
1434
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1435
+ "document_regularizer_weight": 3e-05,
1436
+ "query_regularizer_weight": 5e-05
1437
+ }
1438
+ ```
1439
+
1440
+ ### Evaluation Dataset
1441
+
1442
+ #### gooaq
1443
+
1444
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1445
+ * Size: 1,000 evaluation samples
1446
+ * Columns: <code>question</code> and <code>answer</code>
1447
+ * Approximate statistics based on the first 1000 samples:
1448
+ | | question | answer |
1449
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1450
+ | type | string | string |
1451
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.93 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.84 tokens</li><li>max: 127 tokens</li></ul> |
1452
+ * Samples:
1453
+ | question | answer |
1454
+ |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1455
+ | <code>should you take ibuprofen with high blood pressure?</code> | <code>In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.</code> |
1456
+ | <code>how old do you have to be to work in sc?</code> | <code>The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.</code> |
1457
+ | <code>how to write a topic proposal for a research paper?</code> | <code>['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']</code> |
1458
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1459
+ ```json
1460
+ {
1461
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1462
+ "document_regularizer_weight": 3e-05,
1463
+ "query_regularizer_weight": 5e-05
1464
+ }
1465
+ ```
1466
+
1467
+ ### Training Hyperparameters
1468
+ #### Non-Default Hyperparameters
1469
+
1470
+ - `eval_strategy`: steps
1471
+ - `per_device_train_batch_size`: 32
1472
+ - `per_device_eval_batch_size`: 32
1473
+ - `learning_rate`: 2e-05
1474
+ - `num_train_epochs`: 1
1475
+ - `bf16`: True
1476
+ - `load_best_model_at_end`: True
1477
+ - `batch_sampler`: no_duplicates
1478
+
1479
+ #### All Hyperparameters
1480
+ <details><summary>Click to expand</summary>
1481
+
1482
+ - `overwrite_output_dir`: False
1483
+ - `do_predict`: False
1484
+ - `eval_strategy`: steps
1485
+ - `prediction_loss_only`: True
1486
+ - `per_device_train_batch_size`: 32
1487
+ - `per_device_eval_batch_size`: 32
1488
+ - `per_gpu_train_batch_size`: None
1489
+ - `per_gpu_eval_batch_size`: None
1490
+ - `gradient_accumulation_steps`: 1
1491
+ - `eval_accumulation_steps`: None
1492
+ - `torch_empty_cache_steps`: None
1493
+ - `learning_rate`: 2e-05
1494
+ - `weight_decay`: 0.0
1495
+ - `adam_beta1`: 0.9
1496
+ - `adam_beta2`: 0.999
1497
+ - `adam_epsilon`: 1e-08
1498
+ - `max_grad_norm`: 1.0
1499
+ - `num_train_epochs`: 1
1500
+ - `max_steps`: -1
1501
+ - `lr_scheduler_type`: linear
1502
+ - `lr_scheduler_kwargs`: {}
1503
+ - `warmup_ratio`: 0.0
1504
+ - `warmup_steps`: 0
1505
+ - `log_level`: passive
1506
+ - `log_level_replica`: warning
1507
+ - `log_on_each_node`: True
1508
+ - `logging_nan_inf_filter`: True
1509
+ - `save_safetensors`: True
1510
+ - `save_on_each_node`: False
1511
+ - `save_only_model`: False
1512
+ - `restore_callback_states_from_checkpoint`: False
1513
+ - `no_cuda`: False
1514
+ - `use_cpu`: False
1515
+ - `use_mps_device`: False
1516
+ - `seed`: 42
1517
+ - `data_seed`: None
1518
+ - `jit_mode_eval`: False
1519
+ - `use_ipex`: False
1520
+ - `bf16`: True
1521
+ - `fp16`: False
1522
+ - `fp16_opt_level`: O1
1523
+ - `half_precision_backend`: auto
1524
+ - `bf16_full_eval`: False
1525
+ - `fp16_full_eval`: False
1526
+ - `tf32`: None
1527
+ - `local_rank`: 0
1528
+ - `ddp_backend`: None
1529
+ - `tpu_num_cores`: None
1530
+ - `tpu_metrics_debug`: False
1531
+ - `debug`: []
1532
+ - `dataloader_drop_last`: False
1533
+ - `dataloader_num_workers`: 0
1534
+ - `dataloader_prefetch_factor`: None
1535
+ - `past_index`: -1
1536
+ - `disable_tqdm`: False
1537
+ - `remove_unused_columns`: True
1538
+ - `label_names`: None
1539
+ - `load_best_model_at_end`: True
1540
+ - `ignore_data_skip`: False
1541
+ - `fsdp`: []
1542
+ - `fsdp_min_num_params`: 0
1543
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1544
+ - `fsdp_transformer_layer_cls_to_wrap`: None
1545
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1546
+ - `deepspeed`: None
1547
+ - `label_smoothing_factor`: 0.0
1548
+ - `optim`: adamw_torch
1549
+ - `optim_args`: None
1550
+ - `adafactor`: False
1551
+ - `group_by_length`: False
1552
+ - `length_column_name`: length
1553
+ - `ddp_find_unused_parameters`: None
1554
+ - `ddp_bucket_cap_mb`: None
1555
+ - `ddp_broadcast_buffers`: False
1556
+ - `dataloader_pin_memory`: True
1557
+ - `dataloader_persistent_workers`: False
1558
+ - `skip_memory_metrics`: True
1559
+ - `use_legacy_prediction_loop`: False
1560
+ - `push_to_hub`: False
1561
+ - `resume_from_checkpoint`: None
1562
+ - `hub_model_id`: None
1563
+ - `hub_strategy`: every_save
1564
+ - `hub_private_repo`: None
1565
+ - `hub_always_push`: False
1566
+ - `gradient_checkpointing`: False
1567
+ - `gradient_checkpointing_kwargs`: None
1568
+ - `include_inputs_for_metrics`: False
1569
+ - `include_for_metrics`: []
1570
+ - `eval_do_concat_batches`: True
1571
+ - `fp16_backend`: auto
1572
+ - `push_to_hub_model_id`: None
1573
+ - `push_to_hub_organization`: None
1574
+ - `mp_parameters`:
1575
+ - `auto_find_batch_size`: False
1576
+ - `full_determinism`: False
1577
+ - `torchdynamo`: None
1578
+ - `ray_scope`: last
1579
+ - `ddp_timeout`: 1800
1580
+ - `torch_compile`: False
1581
+ - `torch_compile_backend`: None
1582
+ - `torch_compile_mode`: None
1583
+ - `include_tokens_per_second`: False
1584
+ - `include_num_input_tokens_seen`: False
1585
+ - `neftune_noise_alpha`: None
1586
+ - `optim_target_modules`: None
1587
+ - `batch_eval_metrics`: False
1588
+ - `eval_on_start`: False
1589
+ - `use_liger_kernel`: False
1590
+ - `eval_use_gather_object`: False
1591
+ - `average_tokens_across_devices`: False
1592
+ - `prompts`: None
1593
+ - `batch_sampler`: no_duplicates
1594
+ - `multi_dataset_batch_sampler`: proportional
1595
+ - `router_mapping`: {}
1596
+ - `learning_rate_mapping`: {}
1597
+
1598
+ </details>
1599
+
1600
+ ### Training Logs
1601
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 |
1602
+ |:----------:|:--------:|:-------------:|:---------------:|:-----------------------:|:------------------------:|:------------------:|:-------------------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|
1603
+ | 0.0323 | 100 | 11.4443 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1604
+ | 0.0646 | 200 | 0.2676 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1605
+ | 0.0970 | 300 | 0.1639 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1606
+ | 0.1293 | 400 | 0.1769 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1607
+ | 0.1616 | 500 | 0.1593 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1608
+ | 0.1939 | 600 | 0.1194 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1609
+ | 0.1972 | 610 | - | 0.1080 | 0.4260 | 0.2314 | 0.4303 | 0.3626 | - | - | - | - | - | - | - | - | - | - |
1610
+ | 0.2262 | 700 | 0.1351 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1611
+ | 0.2586 | 800 | 0.109 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1612
+ | 0.2909 | 900 | 0.1147 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1613
+ | 0.3232 | 1000 | 0.0994 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1614
+ | 0.3555 | 1100 | 0.0871 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1615
+ | 0.3878 | 1200 | 0.0891 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1616
+ | **0.3943** | **1220** | **-** | **0.0942** | **0.489** | **0.2395** | **0.5442** | **0.4242** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** |
1617
+ | 0.4202 | 1300 | 0.09 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1618
+ | 0.4525 | 1400 | 0.0902 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1619
+ | 0.4848 | 1500 | 0.1046 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1620
+ | 0.5171 | 1600 | 0.071 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1621
+ | 0.5495 | 1700 | 0.0783 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1622
+ | 0.5818 | 1800 | 0.0846 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1623
+ | 0.5915 | 1830 | - | 0.0804 | 0.4745 | 0.2537 | 0.4780 | 0.4021 | - | - | - | - | - | - | - | - | - | - |
1624
+ | 0.6141 | 1900 | 0.0572 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1625
+ | 0.6464 | 2000 | 0.0712 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1626
+ | 0.6787 | 2100 | 0.065 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1627
+ | 0.7111 | 2200 | 0.096 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1628
+ | 0.7434 | 2300 | 0.0764 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1629
+ | 0.7757 | 2400 | 0.0722 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1630
+ | 0.7886 | 2440 | - | 0.0716 | 0.4976 | 0.2348 | 0.4626 | 0.3983 | - | - | - | - | - | - | - | - | - | - |
1631
+ | 0.8080 | 2500 | 0.0579 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1632
+ | 0.8403 | 2600 | 0.0655 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1633
+ | 0.8727 | 2700 | 0.0612 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1634
+ | 0.9050 | 2800 | 0.0491 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1635
+ | 0.9373 | 2900 | 0.0496 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1636
+ | 0.9696 | 3000 | 0.0553 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1637
+ | 0.9858 | 3050 | - | 0.0746 | 0.4990 | 0.2419 | 0.4688 | 0.4032 | - | - | - | - | - | - | - | - | - | - |
1638
+ | -1 | -1 | - | - | 0.4890 | 0.2395 | 0.5442 | 0.4721 | 0.2509 | 0.4899 | 0.7043 | 0.3639 | 0.6876 | 0.7142 | 0.2977 | 0.2921 | 0.5783 | 0.4859 |
1639
+
1640
+ * The bold row denotes the saved checkpoint.
1641
+
1642
+ ### Environmental Impact
1643
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
1644
+ - **Energy Consumed**: 0.039 kWh
1645
+ - **Carbon Emitted**: 0.015 kg of CO2
1646
+ - **Hours Used**: 0.154 hours
1647
+
1648
+ ### Training Hardware
1649
+ - **On Cloud**: No
1650
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3090
1651
+ - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
1652
+ - **RAM Size**: 31.78 GB
1653
+
1654
+ ### Framework Versions
1655
+ - Python: 3.11.6
1656
+ - Sentence Transformers: 4.2.0.dev0
1657
+ - Transformers: 4.52.4
1658
+ - PyTorch: 2.7.1+cu126
1659
+ - Accelerate: 1.5.1
1660
+ - Datasets: 2.21.0
1661
+ - Tokenizers: 0.21.1
1662
+
1663
+ ## Citation
1664
+
1665
+ ### BibTeX
1666
+
1667
+ #### Sentence Transformers
1668
+ ```bibtex
1669
+ @inproceedings{reimers-2019-sentence-bert,
1670
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1671
+ author = "Reimers, Nils and Gurevych, Iryna",
1672
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1673
+ month = "11",
1674
+ year = "2019",
1675
+ publisher = "Association for Computational Linguistics",
1676
+ url = "https://arxiv.org/abs/1908.10084",
1677
+ }
1678
+ ```
1679
+
1680
+ #### SpladeLoss
1681
+ ```bibtex
1682
+ @misc{formal2022distillationhardnegativesampling,
1683
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1684
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1685
+ year={2022},
1686
+ eprint={2205.04733},
1687
+ archivePrefix={arXiv},
1688
+ primaryClass={cs.IR},
1689
+ url={https://arxiv.org/abs/2205.04733},
1690
+ }
1691
+ ```
1692
+
1693
+ #### SparseMultipleNegativesRankingLoss
1694
+ ```bibtex
1695
+ @misc{henderson2017efficient,
1696
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
1697
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
1698
+ year={2017},
1699
+ eprint={1705.00652},
1700
+ archivePrefix={arXiv},
1701
+ primaryClass={cs.CL}
1702
+ }
1703
+ ```
1704
+
1705
+ #### FlopsLoss
1706
+ ```bibtex
1707
+ @article{paria2020minimizing,
1708
+ title={Minimizing flops to learn efficient sparse representations},
1709
+ author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
1710
+ journal={arXiv preprint arXiv:2004.05665},
1711
+ year={2020}
1712
+ }
1713
+ ```
1714
+
1715
+ <!--
1716
+ ## Glossary
1717
+
1718
+ *Clearly define terms in order to be accessible across audiences.*
1719
+ -->
1720
+
1721
+ <!--
1722
+ ## Model Card Authors
1723
+
1724
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1725
+ -->
1726
+
1727
+ <!--
1728
+ ## Model Card Contact
1729
+
1730
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1731
+ -->
config.json ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "activation": "gelu",
3
+ "architectures": [
4
+ "DistilBertForMaskedLM"
5
+ ],
6
+ "attention_dropout": 0.1,
7
+ "dim": 768,
8
+ "dropout": 0.1,
9
+ "hidden_dim": 3072,
10
+ "initializer_range": 0.02,
11
+ "max_position_embeddings": 512,
12
+ "model_type": "distilbert",
13
+ "n_heads": 12,
14
+ "n_layers": 6,
15
+ "pad_token_id": 0,
16
+ "qa_dropout": 0.1,
17
+ "seq_classif_dropout": 0.2,
18
+ "sinusoidal_pos_embds": false,
19
+ "tie_weights_": true,
20
+ "torch_dtype": "float32",
21
+ "transformers_version": "4.52.4",
22
+ "vocab_size": 30522
23
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "SparseEncoder",
3
+ "__version__": {
4
+ "sentence_transformers": "4.2.0.dev0",
5
+ "transformers": "4.52.4",
6
+ "pytorch": "2.7.1+cu126"
7
+ },
8
+ "prompts": {
9
+ "query": "",
10
+ "document": ""
11
+ },
12
+ "default_prompt_name": null,
13
+ "similarity_fn_name": "dot"
14
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f3699fbbf37b61dcf06552b84aa8da4896549fdcb2e34ff0dc09c8e6379e7cd6
3
+ size 267954768
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "DistilBertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff