Hugging Face's logo

tomaarsen
/
splade-distilbert-base-uncased-gooaq-peft-r128

Feature Extraction
sentence-transformers
Safetensors
English
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:99000
loss:SpladeLoss
loss:SparseMultipleNegativesRankingLoss
loss:FlopsLoss
Eval Results
Model card Files
xet
Community
tomaarsen HF Staff commited on
Commit
ba9d67b
·
verified ·
1 Parent(s): 01b6a61

Add new SparseEncoder model

Browse files
Files changed (11) hide show
  1. 1_SpladePooling/config.json +5 -0
  2. README.md +1731 -0
  3. adapter_config.json +34 -0
  4. adapter_model.safetensors +3 -0
  5. config_sentence_transformers.json +14 -0
  6. modules.json +14 -0
  7. sentence_bert_config.json +4 -0
  8. special_tokens_map.json +7 -0
  9. tokenizer.json +0 -0
  10. tokenizer_config.json +56 -0
  11. vocab.txt +0 -0
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "word_embedding_dimension": 30522
5
+ }
README.md ADDED
@@ -0,0 +1,1731 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:99000
12
+ - loss:SpladeLoss
13
+ - loss:SparseMultipleNegativesRankingLoss
14
+ - loss:FlopsLoss
15
+ base_model: distilbert/distilbert-base-uncased
16
+ widget:
17
+ - text: 'The term emergent literacy signals a belief that, in a literate society,
18
+ young children even one and two year olds, are in the process of becoming literate”.
19
+ ... Gray (1956:21) notes: Functional literacy is used for the training of adults
20
+ to ''meet independently the reading and writing demands placed on them''.'
21
+ - text: Rey is seemingly confirmed as being The Chosen One per a quote by a Lucasfilm
22
+ production designer who worked on The Rise of Skywalker.
23
+ - text: are union gun safes fireproof?
24
+ - text: Fruit is an essential part of a healthy diet — and may aid weight loss. Most
25
+ fruits are low in calories while high in nutrients and fiber, which can boost
26
+ your fullness. Keep in mind that it's best to eat fruits whole rather than juiced.
27
+ What's more, simply eating fruit is not the key to weight loss.
28
+ - text: Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate
29
+ or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks
30
+ for chronic sinusitis.
31
+ datasets:
32
+ - sentence-transformers/gooaq
33
+ pipeline_tag: feature-extraction
34
+ library_name: sentence-transformers
35
+ metrics:
36
+ - dot_accuracy@1
37
+ - dot_accuracy@3
38
+ - dot_accuracy@5
39
+ - dot_accuracy@10
40
+ - dot_precision@1
41
+ - dot_precision@3
42
+ - dot_precision@5
43
+ - dot_precision@10
44
+ - dot_recall@1
45
+ - dot_recall@3
46
+ - dot_recall@5
47
+ - dot_recall@10
48
+ - dot_ndcg@10
49
+ - dot_mrr@10
50
+ - dot_map@100
51
+ - query_active_dims
52
+ - query_sparsity_ratio
53
+ - corpus_active_dims
54
+ - corpus_sparsity_ratio
55
+ co2_eq_emissions:
56
+ emissions: 16.638146863146233
57
+ energy_consumed: 0.04280437678001716
58
+ source: codecarbon
59
+ training_type: fine-tuning
60
+ on_cloud: false
61
+ cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
62
+ ram_total_size: 31.777088165283203
63
+ hours_used: 0.193
64
+ hardware_used: 1 x NVIDIA GeForce RTX 3090
65
+ model-index:
66
+ - name: splade-distilbert-base-uncased trained on GooAQ
67
+ results:
68
+ - task:
69
+ type: sparse-information-retrieval
70
+ name: Sparse Information Retrieval
71
+ dataset:
72
+ name: NanoMSMARCO
73
+ type: NanoMSMARCO
74
+ metrics:
75
+ - type: dot_accuracy@1
76
+ value: 0.3
77
+ name: Dot Accuracy@1
78
+ - type: dot_accuracy@3
79
+ value: 0.54
80
+ name: Dot Accuracy@3
81
+ - type: dot_accuracy@5
82
+ value: 0.68
83
+ name: Dot Accuracy@5
84
+ - type: dot_accuracy@10
85
+ value: 0.74
86
+ name: Dot Accuracy@10
87
+ - type: dot_precision@1
88
+ value: 0.3
89
+ name: Dot Precision@1
90
+ - type: dot_precision@3
91
+ value: 0.18
92
+ name: Dot Precision@3
93
+ - type: dot_precision@5
94
+ value: 0.136
95
+ name: Dot Precision@5
96
+ - type: dot_precision@10
97
+ value: 0.07400000000000001
98
+ name: Dot Precision@10
99
+ - type: dot_recall@1
100
+ value: 0.3
101
+ name: Dot Recall@1
102
+ - type: dot_recall@3
103
+ value: 0.54
104
+ name: Dot Recall@3
105
+ - type: dot_recall@5
106
+ value: 0.68
107
+ name: Dot Recall@5
108
+ - type: dot_recall@10
109
+ value: 0.74
110
+ name: Dot Recall@10
111
+ - type: dot_ndcg@10
112
+ value: 0.5061981336542133
113
+ name: Dot Ndcg@10
114
+ - type: dot_mrr@10
115
+ value: 0.43174603174603166
116
+ name: Dot Mrr@10
117
+ - type: dot_map@100
118
+ value: 0.44263003895418085
119
+ name: Dot Map@100
120
+ - type: query_active_dims
121
+ value: 118.5999984741211
122
+ name: Query Active Dims
123
+ - type: query_sparsity_ratio
124
+ value: 0.996114278275535
125
+ name: Query Sparsity Ratio
126
+ - type: corpus_active_dims
127
+ value: 397.6775817871094
128
+ name: Corpus Active Dims
129
+ - type: corpus_sparsity_ratio
130
+ value: 0.9869707888805744
131
+ name: Corpus Sparsity Ratio
132
+ - type: dot_accuracy@1
133
+ value: 0.32
134
+ name: Dot Accuracy@1
135
+ - type: dot_accuracy@3
136
+ value: 0.5
137
+ name: Dot Accuracy@3
138
+ - type: dot_accuracy@5
139
+ value: 0.64
140
+ name: Dot Accuracy@5
141
+ - type: dot_accuracy@10
142
+ value: 0.72
143
+ name: Dot Accuracy@10
144
+ - type: dot_precision@1
145
+ value: 0.32
146
+ name: Dot Precision@1
147
+ - type: dot_precision@3
148
+ value: 0.16666666666666663
149
+ name: Dot Precision@3
150
+ - type: dot_precision@5
151
+ value: 0.128
152
+ name: Dot Precision@5
153
+ - type: dot_precision@10
154
+ value: 0.07200000000000001
155
+ name: Dot Precision@10
156
+ - type: dot_recall@1
157
+ value: 0.32
158
+ name: Dot Recall@1
159
+ - type: dot_recall@3
160
+ value: 0.5
161
+ name: Dot Recall@3
162
+ - type: dot_recall@5
163
+ value: 0.64
164
+ name: Dot Recall@5
165
+ - type: dot_recall@10
166
+ value: 0.72
167
+ name: Dot Recall@10
168
+ - type: dot_ndcg@10
169
+ value: 0.5061402921245981
170
+ name: Dot Ndcg@10
171
+ - type: dot_mrr@10
172
+ value: 0.43823809523809515
173
+ name: Dot Mrr@10
174
+ - type: dot_map@100
175
+ value: 0.4500866595693115
176
+ name: Dot Map@100
177
+ - type: query_active_dims
178
+ value: 105.08000183105469
179
+ name: Query Active Dims
180
+ - type: query_sparsity_ratio
181
+ value: 0.996557237342538
182
+ name: Query Sparsity Ratio
183
+ - type: corpus_active_dims
184
+ value: 381.3874816894531
185
+ name: Corpus Active Dims
186
+ - type: corpus_sparsity_ratio
187
+ value: 0.9875045055471643
188
+ name: Corpus Sparsity Ratio
189
+ - task:
190
+ type: sparse-information-retrieval
191
+ name: Sparse Information Retrieval
192
+ dataset:
193
+ name: NanoNFCorpus
194
+ type: NanoNFCorpus
195
+ metrics:
196
+ - type: dot_accuracy@1
197
+ value: 0.34
198
+ name: Dot Accuracy@1
199
+ - type: dot_accuracy@3
200
+ value: 0.44
201
+ name: Dot Accuracy@3
202
+ - type: dot_accuracy@5
203
+ value: 0.48
204
+ name: Dot Accuracy@5
205
+ - type: dot_accuracy@10
206
+ value: 0.6
207
+ name: Dot Accuracy@10
208
+ - type: dot_precision@1
209
+ value: 0.34
210
+ name: Dot Precision@1
211
+ - type: dot_precision@3
212
+ value: 0.26
213
+ name: Dot Precision@3
214
+ - type: dot_precision@5
215
+ value: 0.23199999999999998
216
+ name: Dot Precision@5
217
+ - type: dot_precision@10
218
+ value: 0.19599999999999998
219
+ name: Dot Precision@10
220
+ - type: dot_recall@1
221
+ value: 0.01204138289831077
222
+ name: Dot Recall@1
223
+ - type: dot_recall@3
224
+ value: 0.028423242145972874
225
+ name: Dot Recall@3
226
+ - type: dot_recall@5
227
+ value: 0.04013720529494631
228
+ name: Dot Recall@5
229
+ - type: dot_recall@10
230
+ value: 0.06944452178864681
231
+ name: Dot Recall@10
232
+ - type: dot_ndcg@10
233
+ value: 0.2238211925399539
234
+ name: Dot Ndcg@10
235
+ - type: dot_mrr@10
236
+ value: 0.4057777777777777
237
+ name: Dot Mrr@10
238
+ - type: dot_map@100
239
+ value: 0.07440414426513103
240
+ name: Dot Map@100
241
+ - type: query_active_dims
242
+ value: 183.05999755859375
243
+ name: Query Active Dims
244
+ - type: query_sparsity_ratio
245
+ value: 0.9940023590341854
246
+ name: Query Sparsity Ratio
247
+ - type: corpus_active_dims
248
+ value: 823.3663940429688
249
+ name: Corpus Active Dims
250
+ - type: corpus_sparsity_ratio
251
+ value: 0.9730238387378621
252
+ name: Corpus Sparsity Ratio
253
+ - type: dot_accuracy@1
254
+ value: 0.3
255
+ name: Dot Accuracy@1
256
+ - type: dot_accuracy@3
257
+ value: 0.46
258
+ name: Dot Accuracy@3
259
+ - type: dot_accuracy@5
260
+ value: 0.48
261
+ name: Dot Accuracy@5
262
+ - type: dot_accuracy@10
263
+ value: 0.6
264
+ name: Dot Accuracy@10
265
+ - type: dot_precision@1
266
+ value: 0.3
267
+ name: Dot Precision@1
268
+ - type: dot_precision@3
269
+ value: 0.2733333333333333
270
+ name: Dot Precision@3
271
+ - type: dot_precision@5
272
+ value: 0.23199999999999998
273
+ name: Dot Precision@5
274
+ - type: dot_precision@10
275
+ value: 0.21400000000000002
276
+ name: Dot Precision@10
277
+ - type: dot_recall@1
278
+ value: 0.010708049564977435
279
+ name: Dot Recall@1
280
+ - type: dot_recall@3
281
+ value: 0.04042324214597287
282
+ name: Dot Recall@3
283
+ - type: dot_recall@5
284
+ value: 0.05817733939406678
285
+ name: Dot Recall@5
286
+ - type: dot_recall@10
287
+ value: 0.0849823575856454
288
+ name: Dot Recall@10
289
+ - type: dot_ndcg@10
290
+ value: 0.24157503472859507
291
+ name: Dot Ndcg@10
292
+ - type: dot_mrr@10
293
+ value: 0.3932222222222223
294
+ name: Dot Mrr@10
295
+ - type: dot_map@100
296
+ value: 0.08415340735361837
297
+ name: Dot Map@100
298
+ - type: query_active_dims
299
+ value: 150.77999877929688
300
+ name: Query Active Dims
301
+ - type: query_sparsity_ratio
302
+ value: 0.9950599567925006
303
+ name: Query Sparsity Ratio
304
+ - type: corpus_active_dims
305
+ value: 807.0741577148438
306
+ name: Corpus Active Dims
307
+ - type: corpus_sparsity_ratio
308
+ value: 0.9735576253943109
309
+ name: Corpus Sparsity Ratio
310
+ - task:
311
+ type: sparse-information-retrieval
312
+ name: Sparse Information Retrieval
313
+ dataset:
314
+ name: NanoNQ
315
+ type: NanoNQ
316
+ metrics:
317
+ - type: dot_accuracy@1
318
+ value: 0.28
319
+ name: Dot Accuracy@1
320
+ - type: dot_accuracy@3
321
+ value: 0.5
322
+ name: Dot Accuracy@3
323
+ - type: dot_accuracy@5
324
+ value: 0.58
325
+ name: Dot Accuracy@5
326
+ - type: dot_accuracy@10
327
+ value: 0.68
328
+ name: Dot Accuracy@10
329
+ - type: dot_precision@1
330
+ value: 0.28
331
+ name: Dot Precision@1
332
+ - type: dot_precision@3
333
+ value: 0.16666666666666663
334
+ name: Dot Precision@3
335
+ - type: dot_precision@5
336
+ value: 0.11600000000000002
337
+ name: Dot Precision@5
338
+ - type: dot_precision@10
339
+ value: 0.07
340
+ name: Dot Precision@10
341
+ - type: dot_recall@1
342
+ value: 0.27
343
+ name: Dot Recall@1
344
+ - type: dot_recall@3
345
+ value: 0.48
346
+ name: Dot Recall@3
347
+ - type: dot_recall@5
348
+ value: 0.54
349
+ name: Dot Recall@5
350
+ - type: dot_recall@10
351
+ value: 0.64
352
+ name: Dot Recall@10
353
+ - type: dot_ndcg@10
354
+ value: 0.45385561138570657
355
+ name: Dot Ndcg@10
356
+ - type: dot_mrr@10
357
+ value: 0.40454761904761893
358
+ name: Dot Mrr@10
359
+ - type: dot_map@100
360
+ value: 0.40238133013339067
361
+ name: Dot Map@100
362
+ - type: query_active_dims
363
+ value: 108.23999786376953
364
+ name: Query Active Dims
365
+ - type: query_sparsity_ratio
366
+ value: 0.9964537055938743
367
+ name: Query Sparsity Ratio
368
+ - type: corpus_active_dims
369
+ value: 581.3165893554688
370
+ name: Corpus Active Dims
371
+ - type: corpus_sparsity_ratio
372
+ value: 0.9809541776634733
373
+ name: Corpus Sparsity Ratio
374
+ - type: dot_accuracy@1
375
+ value: 0.24
376
+ name: Dot Accuracy@1
377
+ - type: dot_accuracy@3
378
+ value: 0.46
379
+ name: Dot Accuracy@3
380
+ - type: dot_accuracy@5
381
+ value: 0.54
382
+ name: Dot Accuracy@5
383
+ - type: dot_accuracy@10
384
+ value: 0.72
385
+ name: Dot Accuracy@10
386
+ - type: dot_precision@1
387
+ value: 0.24
388
+ name: Dot Precision@1
389
+ - type: dot_precision@3
390
+ value: 0.15333333333333332
391
+ name: Dot Precision@3
392
+ - type: dot_precision@5
393
+ value: 0.10800000000000001
394
+ name: Dot Precision@5
395
+ - type: dot_precision@10
396
+ value: 0.07400000000000001
397
+ name: Dot Precision@10
398
+ - type: dot_recall@1
399
+ value: 0.23
400
+ name: Dot Recall@1
401
+ - type: dot_recall@3
402
+ value: 0.44
403
+ name: Dot Recall@3
404
+ - type: dot_recall@5
405
+ value: 0.51
406
+ name: Dot Recall@5
407
+ - type: dot_recall@10
408
+ value: 0.67
409
+ name: Dot Recall@10
410
+ - type: dot_ndcg@10
411
+ value: 0.4431148339670733
412
+ name: Dot Ndcg@10
413
+ - type: dot_mrr@10
414
+ value: 0.3818015873015873
415
+ name: Dot Mrr@10
416
+ - type: dot_map@100
417
+ value: 0.3762054598208147
418
+ name: Dot Map@100
419
+ - type: query_active_dims
420
+ value: 97.18000030517578
421
+ name: Query Active Dims
422
+ - type: query_sparsity_ratio
423
+ value: 0.996816067089143
424
+ name: Query Sparsity Ratio
425
+ - type: corpus_active_dims
426
+ value: 564.0422973632812
427
+ name: Corpus Active Dims
428
+ - type: corpus_sparsity_ratio
429
+ value: 0.9815201396578442
430
+ name: Corpus Sparsity Ratio
431
+ - task:
432
+ type: sparse-nano-beir
433
+ name: Sparse Nano BEIR
434
+ dataset:
435
+ name: NanoBEIR mean
436
+ type: NanoBEIR_mean
437
+ metrics:
438
+ - type: dot_accuracy@1
439
+ value: 0.3066666666666667
440
+ name: Dot Accuracy@1
441
+ - type: dot_accuracy@3
442
+ value: 0.49333333333333335
443
+ name: Dot Accuracy@3
444
+ - type: dot_accuracy@5
445
+ value: 0.5800000000000001
446
+ name: Dot Accuracy@5
447
+ - type: dot_accuracy@10
448
+ value: 0.6733333333333333
449
+ name: Dot Accuracy@10
450
+ - type: dot_precision@1
451
+ value: 0.3066666666666667
452
+ name: Dot Precision@1
453
+ - type: dot_precision@3
454
+ value: 0.20222222222222222
455
+ name: Dot Precision@3
456
+ - type: dot_precision@5
457
+ value: 0.16133333333333333
458
+ name: Dot Precision@5
459
+ - type: dot_precision@10
460
+ value: 0.11333333333333334
461
+ name: Dot Precision@10
462
+ - type: dot_recall@1
463
+ value: 0.1940137942994369
464
+ name: Dot Recall@1
465
+ - type: dot_recall@3
466
+ value: 0.34947441404865764
467
+ name: Dot Recall@3
468
+ - type: dot_recall@5
469
+ value: 0.4200457350983155
470
+ name: Dot Recall@5
471
+ - type: dot_recall@10
472
+ value: 0.4831481739295489
473
+ name: Dot Recall@10
474
+ - type: dot_ndcg@10
475
+ value: 0.39462497919329126
476
+ name: Dot Ndcg@10
477
+ - type: dot_mrr@10
478
+ value: 0.4140238095238094
479
+ name: Dot Mrr@10
480
+ - type: dot_map@100
481
+ value: 0.3064718377842342
482
+ name: Dot Map@100
483
+ - type: query_active_dims
484
+ value: 136.63333129882812
485
+ name: Query Active Dims
486
+ - type: query_sparsity_ratio
487
+ value: 0.9955234476345315
488
+ name: Query Sparsity Ratio
489
+ - type: corpus_active_dims
490
+ value: 565.0999949325504
491
+ name: Corpus Active Dims
492
+ - type: corpus_sparsity_ratio
493
+ value: 0.9814854860450642
494
+ name: Corpus Sparsity Ratio
495
+ - type: dot_accuracy@1
496
+ value: 0.35158555729984303
497
+ name: Dot Accuracy@1
498
+ - type: dot_accuracy@3
499
+ value: 0.5366091051805337
500
+ name: Dot Accuracy@3
501
+ - type: dot_accuracy@5
502
+ value: 0.609105180533752
503
+ name: Dot Accuracy@5
504
+ - type: dot_accuracy@10
505
+ value: 0.7153218210361068
506
+ name: Dot Accuracy@10
507
+ - type: dot_precision@1
508
+ value: 0.35158555729984303
509
+ name: Dot Precision@1
510
+ - type: dot_precision@3
511
+ value: 0.24084772370486657
512
+ name: Dot Precision@3
513
+ - type: dot_precision@5
514
+ value: 0.195861852433281
515
+ name: Dot Precision@5
516
+ - type: dot_precision@10
517
+ value: 0.14448037676609105
518
+ name: Dot Precision@10
519
+ - type: dot_recall@1
520
+ value: 0.18710365017134828
521
+ name: Dot Recall@1
522
+ - type: dot_recall@3
523
+ value: 0.3166600122342838
524
+ name: Dot Recall@3
525
+ - type: dot_recall@5
526
+ value: 0.38032257819651705
527
+ name: Dot Recall@5
528
+ - type: dot_recall@10
529
+ value: 0.4791896835492342
530
+ name: Dot Recall@10
531
+ - type: dot_ndcg@10
532
+ value: 0.41388777461576925
533
+ name: Dot Ndcg@10
534
+ - type: dot_mrr@10
535
+ value: 0.4636842715108021
536
+ name: Dot Mrr@10
537
+ - type: dot_map@100
538
+ value: 0.33650048535941457
539
+ name: Dot Map@100
540
+ - type: query_active_dims
541
+ value: 195.48228298827937
542
+ name: Query Active Dims
543
+ - type: query_sparsity_ratio
544
+ value: 0.9935953645570972
545
+ name: Query Sparsity Ratio
546
+ - type: corpus_active_dims
547
+ value: 525.5023385946348
548
+ name: Corpus Active Dims
549
+ - type: corpus_sparsity_ratio
550
+ value: 0.9827828340674059
551
+ name: Corpus Sparsity Ratio
552
+ - task:
553
+ type: sparse-information-retrieval
554
+ name: Sparse Information Retrieval
555
+ dataset:
556
+ name: NanoClimateFEVER
557
+ type: NanoClimateFEVER
558
+ metrics:
559
+ - type: dot_accuracy@1
560
+ value: 0.2
561
+ name: Dot Accuracy@1
562
+ - type: dot_accuracy@3
563
+ value: 0.38
564
+ name: Dot Accuracy@3
565
+ - type: dot_accuracy@5
566
+ value: 0.42
567
+ name: Dot Accuracy@5
568
+ - type: dot_accuracy@10
569
+ value: 0.52
570
+ name: Dot Accuracy@10
571
+ - type: dot_precision@1
572
+ value: 0.2
573
+ name: Dot Precision@1
574
+ - type: dot_precision@3
575
+ value: 0.14
576
+ name: Dot Precision@3
577
+ - type: dot_precision@5
578
+ value: 0.09200000000000001
579
+ name: Dot Precision@5
580
+ - type: dot_precision@10
581
+ value: 0.06000000000000001
582
+ name: Dot Precision@10
583
+ - type: dot_recall@1
584
+ value: 0.08833333333333332
585
+ name: Dot Recall@1
586
+ - type: dot_recall@3
587
+ value: 0.18166666666666664
588
+ name: Dot Recall@3
589
+ - type: dot_recall@5
590
+ value: 0.19233333333333336
591
+ name: Dot Recall@5
592
+ - type: dot_recall@10
593
+ value: 0.2523333333333333
594
+ name: Dot Recall@10
595
+ - type: dot_ndcg@10
596
+ value: 0.2097369113981719
597
+ name: Dot Ndcg@10
598
+ - type: dot_mrr@10
599
+ value: 0.2989603174603175
600
+ name: Dot Mrr@10
601
+ - type: dot_map@100
602
+ value: 0.16798141398273245
603
+ name: Dot Map@100
604
+ - type: query_active_dims
605
+ value: 250.86000061035156
606
+ name: Query Active Dims
607
+ - type: query_sparsity_ratio
608
+ value: 0.9917810103987172
609
+ name: Query Sparsity Ratio
610
+ - type: corpus_active_dims
611
+ value: 643.326904296875
612
+ name: Corpus Active Dims
613
+ - type: corpus_sparsity_ratio
614
+ value: 0.9789225180428257
615
+ name: Corpus Sparsity Ratio
616
+ - task:
617
+ type: sparse-information-retrieval
618
+ name: Sparse Information Retrieval
619
+ dataset:
620
+ name: NanoDBPedia
621
+ type: NanoDBPedia
622
+ metrics:
623
+ - type: dot_accuracy@1
624
+ value: 0.62
625
+ name: Dot Accuracy@1
626
+ - type: dot_accuracy@3
627
+ value: 0.78
628
+ name: Dot Accuracy@3
629
+ - type: dot_accuracy@5
630
+ value: 0.86
631
+ name: Dot Accuracy@5
632
+ - type: dot_accuracy@10
633
+ value: 0.92
634
+ name: Dot Accuracy@10
635
+ - type: dot_precision@1
636
+ value: 0.62
637
+ name: Dot Precision@1
638
+ - type: dot_precision@3
639
+ value: 0.4733333333333334
640
+ name: Dot Precision@3
641
+ - type: dot_precision@5
642
+ value: 0.452
643
+ name: Dot Precision@5
644
+ - type: dot_precision@10
645
+ value: 0.39599999999999996
646
+ name: Dot Precision@10
647
+ - type: dot_recall@1
648
+ value: 0.06769969786296744
649
+ name: Dot Recall@1
650
+ - type: dot_recall@3
651
+ value: 0.14199136819511296
652
+ name: Dot Recall@3
653
+ - type: dot_recall@5
654
+ value: 0.192778624550143
655
+ name: Dot Recall@5
656
+ - type: dot_recall@10
657
+ value: 0.2816492423802407
658
+ name: Dot Recall@10
659
+ - type: dot_ndcg@10
660
+ value: 0.4998791588316728
661
+ name: Dot Ndcg@10
662
+ - type: dot_mrr@10
663
+ value: 0.7168571428571429
664
+ name: Dot Mrr@10
665
+ - type: dot_map@100
666
+ value: 0.3705445544087827
667
+ name: Dot Map@100
668
+ - type: query_active_dims
669
+ value: 146.02000427246094
670
+ name: Query Active Dims
671
+ - type: query_sparsity_ratio
672
+ value: 0.9952159096955487
673
+ name: Query Sparsity Ratio
674
+ - type: corpus_active_dims
675
+ value: 481.7581481933594
676
+ name: Corpus Active Dims
677
+ - type: corpus_sparsity_ratio
678
+ value: 0.9842160360332429
679
+ name: Corpus Sparsity Ratio
680
+ - task:
681
+ type: sparse-information-retrieval
682
+ name: Sparse Information Retrieval
683
+ dataset:
684
+ name: NanoFEVER
685
+ type: NanoFEVER
686
+ metrics:
687
+ - type: dot_accuracy@1
688
+ value: 0.44
689
+ name: Dot Accuracy@1
690
+ - type: dot_accuracy@3
691
+ value: 0.66
692
+ name: Dot Accuracy@3
693
+ - type: dot_accuracy@5
694
+ value: 0.78
695
+ name: Dot Accuracy@5
696
+ - type: dot_accuracy@10
697
+ value: 0.84
698
+ name: Dot Accuracy@10
699
+ - type: dot_precision@1
700
+ value: 0.44
701
+ name: Dot Precision@1
702
+ - type: dot_precision@3
703
+ value: 0.22
704
+ name: Dot Precision@3
705
+ - type: dot_precision@5
706
+ value: 0.156
707
+ name: Dot Precision@5
708
+ - type: dot_precision@10
709
+ value: 0.08599999999999998
710
+ name: Dot Precision@10
711
+ - type: dot_recall@1
712
+ value: 0.44
713
+ name: Dot Recall@1
714
+ - type: dot_recall@3
715
+ value: 0.64
716
+ name: Dot Recall@3
717
+ - type: dot_recall@5
718
+ value: 0.7366666666666666
719
+ name: Dot Recall@5
720
+ - type: dot_recall@10
721
+ value: 0.7966666666666665
722
+ name: Dot Recall@10
723
+ - type: dot_ndcg@10
724
+ value: 0.6190748153469672
725
+ name: Dot Ndcg@10
726
+ - type: dot_mrr@10
727
+ value: 0.5678888888888888
728
+ name: Dot Mrr@10
729
+ - type: dot_map@100
730
+ value: 0.5644736817593311
731
+ name: Dot Map@100
732
+ - type: query_active_dims
733
+ value: 253.3800048828125
734
+ name: Query Active Dims
735
+ - type: query_sparsity_ratio
736
+ value: 0.9916984468618435
737
+ name: Query Sparsity Ratio
738
+ - type: corpus_active_dims
739
+ value: 749.9185180664062
740
+ name: Corpus Active Dims
741
+ - type: corpus_sparsity_ratio
742
+ value: 0.9754302300613852
743
+ name: Corpus Sparsity Ratio
744
+ - task:
745
+ type: sparse-information-retrieval
746
+ name: Sparse Information Retrieval
747
+ dataset:
748
+ name: NanoFiQA2018
749
+ type: NanoFiQA2018
750
+ metrics:
751
+ - type: dot_accuracy@1
752
+ value: 0.22
753
+ name: Dot Accuracy@1
754
+ - type: dot_accuracy@3
755
+ value: 0.42
756
+ name: Dot Accuracy@3
757
+ - type: dot_accuracy@5
758
+ value: 0.46
759
+ name: Dot Accuracy@5
760
+ - type: dot_accuracy@10
761
+ value: 0.54
762
+ name: Dot Accuracy@10
763
+ - type: dot_precision@1
764
+ value: 0.22
765
+ name: Dot Precision@1
766
+ - type: dot_precision@3
767
+ value: 0.16666666666666663
768
+ name: Dot Precision@3
769
+ - type: dot_precision@5
770
+ value: 0.14400000000000002
771
+ name: Dot Precision@5
772
+ - type: dot_precision@10
773
+ value: 0.09
774
+ name: Dot Precision@10
775
+ - type: dot_recall@1
776
+ value: 0.13933333333333334
777
+ name: Dot Recall@1
778
+ - type: dot_recall@3
779
+ value: 0.26035714285714284
780
+ name: Dot Recall@3
781
+ - type: dot_recall@5
782
+ value: 0.31182539682539684
783
+ name: Dot Recall@5
784
+ - type: dot_recall@10
785
+ value: 0.3924047619047619
786
+ name: Dot Recall@10
787
+ - type: dot_ndcg@10
788
+ value: 0.3071601294876744
789
+ name: Dot Ndcg@10
790
+ - type: dot_mrr@10
791
+ value: 0.3309126984126985
792
+ name: Dot Mrr@10
793
+ - type: dot_map@100
794
+ value: 0.2510011498241125
795
+ name: Dot Map@100
796
+ - type: query_active_dims
797
+ value: 85.69999694824219
798
+ name: Query Active Dims
799
+ - type: query_sparsity_ratio
800
+ value: 0.9971921893405333
801
+ name: Query Sparsity Ratio
802
+ - type: corpus_active_dims
803
+ value: 416.93829345703125
804
+ name: Corpus Active Dims
805
+ - type: corpus_sparsity_ratio
806
+ value: 0.9863397453162627
807
+ name: Corpus Sparsity Ratio
808
+ - task:
809
+ type: sparse-information-retrieval
810
+ name: Sparse Information Retrieval
811
+ dataset:
812
+ name: NanoHotpotQA
813
+ type: NanoHotpotQA
814
+ metrics:
815
+ - type: dot_accuracy@1
816
+ value: 0.68
817
+ name: Dot Accuracy@1
818
+ - type: dot_accuracy@3
819
+ value: 0.78
820
+ name: Dot Accuracy@3
821
+ - type: dot_accuracy@5
822
+ value: 0.8
823
+ name: Dot Accuracy@5
824
+ - type: dot_accuracy@10
825
+ value: 0.88
826
+ name: Dot Accuracy@10
827
+ - type: dot_precision@1
828
+ value: 0.68
829
+ name: Dot Precision@1
830
+ - type: dot_precision@3
831
+ value: 0.35333333333333333
832
+ name: Dot Precision@3
833
+ - type: dot_precision@5
834
+ value: 0.24
835
+ name: Dot Precision@5
836
+ - type: dot_precision@10
837
+ value: 0.13799999999999998
838
+ name: Dot Precision@10
839
+ - type: dot_recall@1
840
+ value: 0.34
841
+ name: Dot Recall@1
842
+ - type: dot_recall@3
843
+ value: 0.53
844
+ name: Dot Recall@3
845
+ - type: dot_recall@5
846
+ value: 0.6
847
+ name: Dot Recall@5
848
+ - type: dot_recall@10
849
+ value: 0.69
850
+ name: Dot Recall@10
851
+ - type: dot_ndcg@10
852
+ value: 0.6197567693807055
853
+ name: Dot Ndcg@10
854
+ - type: dot_mrr@10
855
+ value: 0.7347142857142859
856
+ name: Dot Mrr@10
857
+ - type: dot_map@100
858
+ value: 0.540453368331375
859
+ name: Dot Map@100
860
+ - type: query_active_dims
861
+ value: 152.5399932861328
862
+ name: Query Active Dims
863
+ - type: query_sparsity_ratio
864
+ value: 0.9950022936476596
865
+ name: Query Sparsity Ratio
866
+ - type: corpus_active_dims
867
+ value: 553.4066772460938
868
+ name: Corpus Active Dims
869
+ - type: corpus_sparsity_ratio
870
+ value: 0.9818685971677447
871
+ name: Corpus Sparsity Ratio
872
+ - task:
873
+ type: sparse-information-retrieval
874
+ name: Sparse Information Retrieval
875
+ dataset:
876
+ name: NanoQuoraRetrieval
877
+ type: NanoQuoraRetrieval
878
+ metrics:
879
+ - type: dot_accuracy@1
880
+ value: 0.36
881
+ name: Dot Accuracy@1
882
+ - type: dot_accuracy@3
883
+ value: 0.52
884
+ name: Dot Accuracy@3
885
+ - type: dot_accuracy@5
886
+ value: 0.58
887
+ name: Dot Accuracy@5
888
+ - type: dot_accuracy@10
889
+ value: 0.78
890
+ name: Dot Accuracy@10
891
+ - type: dot_precision@1
892
+ value: 0.36
893
+ name: Dot Precision@1
894
+ - type: dot_precision@3
895
+ value: 0.1733333333333333
896
+ name: Dot Precision@3
897
+ - type: dot_precision@5
898
+ value: 0.124
899
+ name: Dot Precision@5
900
+ - type: dot_precision@10
901
+ value: 0.08199999999999999
902
+ name: Dot Precision@10
903
+ - type: dot_recall@1
904
+ value: 0.34666666666666673
905
+ name: Dot Recall@1
906
+ - type: dot_recall@3
907
+ value: 0.4706666666666666
908
+ name: Dot Recall@3
909
+ - type: dot_recall@5
910
+ value: 0.5506666666666666
911
+ name: Dot Recall@5
912
+ - type: dot_recall@10
913
+ value: 0.7506666666666666
914
+ name: Dot Recall@10
915
+ - type: dot_ndcg@10
916
+ value: 0.5326024015174656
917
+ name: Dot Ndcg@10
918
+ - type: dot_mrr@10
919
+ value: 0.4782936507936508
920
+ name: Dot Mrr@10
921
+ - type: dot_map@100
922
+ value: 0.4734890338060357
923
+ name: Dot Map@100
924
+ - type: query_active_dims
925
+ value: 52.900001525878906
926
+ name: Query Active Dims
927
+ - type: query_sparsity_ratio
928
+ value: 0.9982668238802871
929
+ name: Query Sparsity Ratio
930
+ - type: corpus_active_dims
931
+ value: 61.35552978515625
932
+ name: Corpus Active Dims
933
+ - type: corpus_sparsity_ratio
934
+ value: 0.9979897932709142
935
+ name: Corpus Sparsity Ratio
936
+ - task:
937
+ type: sparse-information-retrieval
938
+ name: Sparse Information Retrieval
939
+ dataset:
940
+ name: NanoSCIDOCS
941
+ type: NanoSCIDOCS
942
+ metrics:
943
+ - type: dot_accuracy@1
944
+ value: 0.28
945
+ name: Dot Accuracy@1
946
+ - type: dot_accuracy@3
947
+ value: 0.52
948
+ name: Dot Accuracy@3
949
+ - type: dot_accuracy@5
950
+ value: 0.62
951
+ name: Dot Accuracy@5
952
+ - type: dot_accuracy@10
953
+ value: 0.78
954
+ name: Dot Accuracy@10
955
+ - type: dot_precision@1
956
+ value: 0.28
957
+ name: Dot Precision@1
958
+ - type: dot_precision@3
959
+ value: 0.20666666666666667
960
+ name: Dot Precision@3
961
+ - type: dot_precision@5
962
+ value: 0.184
963
+ name: Dot Precision@5
964
+ - type: dot_precision@10
965
+ value: 0.13799999999999998
966
+ name: Dot Precision@10
967
+ - type: dot_recall@1
968
+ value: 0.059666666666666666
969
+ name: Dot Recall@1
970
+ - type: dot_recall@3
971
+ value: 0.12866666666666668
972
+ name: Dot Recall@3
973
+ - type: dot_recall@5
974
+ value: 0.18966666666666662
975
+ name: Dot Recall@5
976
+ - type: dot_recall@10
977
+ value: 0.2836666666666667
978
+ name: Dot Recall@10
979
+ - type: dot_ndcg@10
980
+ value: 0.2574919427490159
981
+ name: Dot Ndcg@10
982
+ - type: dot_mrr@10
983
+ value: 0.42540476190476184
984
+ name: Dot Mrr@10
985
+ - type: dot_map@100
986
+ value: 0.17688082476501285
987
+ name: Dot Map@100
988
+ - type: query_active_dims
989
+ value: 197.1999969482422
990
+ name: Query Active Dims
991
+ - type: query_sparsity_ratio
992
+ value: 0.9935390866604993
993
+ name: Query Sparsity Ratio
994
+ - type: corpus_active_dims
995
+ value: 676.0037231445312
996
+ name: Corpus Active Dims
997
+ - type: corpus_sparsity_ratio
998
+ value: 0.9778519191683201
999
+ name: Corpus Sparsity Ratio
1000
+ - task:
1001
+ type: sparse-information-retrieval
1002
+ name: Sparse Information Retrieval
1003
+ dataset:
1004
+ name: NanoArguAna
1005
+ type: NanoArguAna
1006
+ metrics:
1007
+ - type: dot_accuracy@1
1008
+ value: 0.02
1009
+ name: Dot Accuracy@1
1010
+ - type: dot_accuracy@3
1011
+ value: 0.14
1012
+ name: Dot Accuracy@3
1013
+ - type: dot_accuracy@5
1014
+ value: 0.22
1015
+ name: Dot Accuracy@5
1016
+ - type: dot_accuracy@10
1017
+ value: 0.38
1018
+ name: Dot Accuracy@10
1019
+ - type: dot_precision@1
1020
+ value: 0.02
1021
+ name: Dot Precision@1
1022
+ - type: dot_precision@3
1023
+ value: 0.04666666666666667
1024
+ name: Dot Precision@3
1025
+ - type: dot_precision@5
1026
+ value: 0.044000000000000004
1027
+ name: Dot Precision@5
1028
+ - type: dot_precision@10
1029
+ value: 0.038000000000000006
1030
+ name: Dot Precision@10
1031
+ - type: dot_recall@1
1032
+ value: 0.02
1033
+ name: Dot Recall@1
1034
+ - type: dot_recall@3
1035
+ value: 0.14
1036
+ name: Dot Recall@3
1037
+ - type: dot_recall@5
1038
+ value: 0.22
1039
+ name: Dot Recall@5
1040
+ - type: dot_recall@10
1041
+ value: 0.38
1042
+ name: Dot Recall@10
1043
+ - type: dot_ndcg@10
1044
+ value: 0.17464966621739791
1045
+ name: Dot Ndcg@10
1046
+ - type: dot_mrr@10
1047
+ value: 0.11243650793650796
1048
+ name: Dot Mrr@10
1049
+ - type: dot_map@100
1050
+ value: 0.11564322909400383
1051
+ name: Dot Map@100
1052
+ - type: query_active_dims
1053
+ value: 732.4600219726562
1054
+ name: Query Active Dims
1055
+ - type: query_sparsity_ratio
1056
+ value: 0.9760022271812904
1057
+ name: Query Sparsity Ratio
1058
+ - type: corpus_active_dims
1059
+ value: 648.47509765625
1060
+ name: Corpus Active Dims
1061
+ - type: corpus_sparsity_ratio
1062
+ value: 0.9787538464826602
1063
+ name: Corpus Sparsity Ratio
1064
+ - task:
1065
+ type: sparse-information-retrieval
1066
+ name: Sparse Information Retrieval
1067
+ dataset:
1068
+ name: NanoSciFact
1069
+ type: NanoSciFact
1070
+ metrics:
1071
+ - type: dot_accuracy@1
1072
+ value: 0.36
1073
+ name: Dot Accuracy@1
1074
+ - type: dot_accuracy@3
1075
+ value: 0.56
1076
+ name: Dot Accuracy@3
1077
+ - type: dot_accuracy@5
1078
+ value: 0.6
1079
+ name: Dot Accuracy@5
1080
+ - type: dot_accuracy@10
1081
+ value: 0.66
1082
+ name: Dot Accuracy@10
1083
+ - type: dot_precision@1
1084
+ value: 0.36
1085
+ name: Dot Precision@1
1086
+ - type: dot_precision@3
1087
+ value: 0.20666666666666667
1088
+ name: Dot Precision@3
1089
+ - type: dot_precision@5
1090
+ value: 0.132
1091
+ name: Dot Precision@5
1092
+ - type: dot_precision@10
1093
+ value: 0.078
1094
+ name: Dot Precision@10
1095
+ - type: dot_recall@1
1096
+ value: 0.335
1097
+ name: Dot Recall@1
1098
+ - type: dot_recall@3
1099
+ value: 0.535
1100
+ name: Dot Recall@3
1101
+ - type: dot_recall@5
1102
+ value: 0.575
1103
+ name: Dot Recall@5
1104
+ - type: dot_recall@10
1105
+ value: 0.66
1106
+ name: Dot Recall@10
1107
+ - type: dot_ndcg@10
1108
+ value: 0.5064687965907525
1109
+ name: Dot Ndcg@10
1110
+ - type: dot_mrr@10
1111
+ value: 0.4627222222222222
1112
+ name: Dot Mrr@10
1113
+ - type: dot_map@100
1114
+ value: 0.46039157929311386
1115
+ name: Dot Map@100
1116
+ - type: query_active_dims
1117
+ value: 276.20001220703125
1118
+ name: Query Active Dims
1119
+ - type: query_sparsity_ratio
1120
+ value: 0.9909507891944489
1121
+ name: Query Sparsity Ratio
1122
+ - type: corpus_active_dims
1123
+ value: 729.4652099609375
1124
+ name: Corpus Active Dims
1125
+ - type: corpus_sparsity_ratio
1126
+ value: 0.9761003469641264
1127
+ name: Corpus Sparsity Ratio
1128
+ - task:
1129
+ type: sparse-information-retrieval
1130
+ name: Sparse Information Retrieval
1131
+ dataset:
1132
+ name: NanoTouche2020
1133
+ type: NanoTouche2020
1134
+ metrics:
1135
+ - type: dot_accuracy@1
1136
+ value: 0.5306122448979592
1137
+ name: Dot Accuracy@1
1138
+ - type: dot_accuracy@3
1139
+ value: 0.7959183673469388
1140
+ name: Dot Accuracy@3
1141
+ - type: dot_accuracy@5
1142
+ value: 0.9183673469387755
1143
+ name: Dot Accuracy@5
1144
+ - type: dot_accuracy@10
1145
+ value: 0.9591836734693877
1146
+ name: Dot Accuracy@10
1147
+ - type: dot_precision@1
1148
+ value: 0.5306122448979592
1149
+ name: Dot Precision@1
1150
+ - type: dot_precision@3
1151
+ value: 0.5510204081632653
1152
+ name: Dot Precision@3
1153
+ - type: dot_precision@5
1154
+ value: 0.5102040816326532
1155
+ name: Dot Precision@5
1156
+ - type: dot_precision@10
1157
+ value: 0.4122448979591837
1158
+ name: Dot Precision@10
1159
+ - type: dot_recall@1
1160
+ value: 0.03493970479958239
1161
+ name: Dot Recall@1
1162
+ - type: dot_recall@3
1163
+ value: 0.10780840584746129
1164
+ name: Dot Recall@3
1165
+ - type: dot_recall@5
1166
+ value: 0.16707882245178216
1167
+ name: Dot Recall@5
1168
+ - type: dot_recall@10
1169
+ value: 0.26709619093606257
1170
+ name: Dot Recall@10
1171
+ - type: dot_ndcg@10
1172
+ value: 0.4628903176649091
1173
+ name: Dot Ndcg@10
1174
+ - type: dot_mrr@10
1175
+ value: 0.6864431486880467
1176
+ name: Dot Mrr@10
1177
+ - type: dot_map@100
1178
+ value: 0.34320194766414486
1179
+ name: Dot Map@100
1180
+ - type: query_active_dims
1181
+ value: 37.81632614135742
1182
+ name: Query Active Dims
1183
+ - type: query_sparsity_ratio
1184
+ value: 0.9987610141490939
1185
+ name: Query Sparsity Ratio
1186
+ - type: corpus_active_dims
1187
+ value: 493.48040771484375
1188
+ name: Corpus Active Dims
1189
+ - type: corpus_sparsity_ratio
1190
+ value: 0.9838319766819067
1191
+ name: Corpus Sparsity Ratio
1192
+ ---
1193
+
1194
+ # splade-distilbert-base-uncased trained on GooAQ
1195
+
1196
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
1197
+ ## Model Details
1198
+
1199
+ ### Model Description
1200
+ - **Model Type:** SPLADE Sparse Encoder
1201
+ - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) <!-- at revision 12040accade4e8a0f71eabdb258fecc2e7e948be -->
1202
+ - **Maximum Sequence Length:** 256 tokens
1203
+ - **Output Dimensionality:** 30522 dimensions
1204
+ - **Similarity Function:** Dot Product
1205
+ - **Training Dataset:**
1206
+ - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
1207
+ - **Language:** en
1208
+ - **License:** apache-2.0
1209
+
1210
+ ### Model Sources
1211
+
1212
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
1213
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
1214
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
1215
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
1216
+
1217
+ ### Full Model Architecture
1218
+
1219
+ ```
1220
+ SparseEncoder(
1221
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'DistilBertForMaskedLM'})
1222
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
1223
+ )
1224
+ ```
1225
+
1226
+ ## Usage
1227
+
1228
+ ### Direct Usage (Sentence Transformers)
1229
+
1230
+ First install the Sentence Transformers library:
1231
+
1232
+ ```bash
1233
+ pip install -U sentence-transformers
1234
+ ```
1235
+
1236
+ Then you can load this model and run inference.
1237
+ ```python
1238
+ from sentence_transformers import SparseEncoder
1239
+
1240
+ # Download from the 🤗 Hub
1241
+ model = SparseEncoder("tomaarsen/splade-distilbert-base-uncased-gooaq-peft-r128")
1242
+ # Run inference
1243
+ queries = [
1244
+ "how many days for doxycycline to work on sinus infection?",
1245
+ ]
1246
+ documents = [
1247
+ 'Treatment of suspected bacterial infection is with antibiotics, such as amoxicillin/clavulanate or doxycycline, given for 5 to 7 days for acute sinusitis and for up to 6 weeks for chronic sinusitis.',
1248
+ 'Most engagements typically have a cocktail dress code, calling for dresses at, or slightly above, knee-length and high heels. If your party states a different dress code, however, such as semi-formal or dressy-casual, you may need to dress up or down accordingly.',
1249
+ 'The average service life of a gas furnace is about 15 years, but the actual life span of an individual unit can vary greatly. There are a number of contributing factors that determine the age a furnace reaches: The quality of the equipment.',
1250
+ ]
1251
+ query_embeddings = model.encode_query(queries)
1252
+ document_embeddings = model.encode_document(documents)
1253
+ print(query_embeddings.shape, document_embeddings.shape)
1254
+ # [1, 30522] [3, 30522]
1255
+
1256
+ # Get the similarity scores for the embeddings
1257
+ similarities = model.similarity(query_embeddings, document_embeddings)
1258
+ print(similarities)
1259
+ # tensor([[85.3246, 22.8328, 29.6908]])
1260
+ ```
1261
+
1262
+ <!--
1263
+ ### Direct Usage (Transformers)
1264
+
1265
+ <details><summary>Click to see the direct usage in Transformers</summary>
1266
+
1267
+ </details>
1268
+ -->
1269
+
1270
+ <!--
1271
+ ### Downstream Usage (Sentence Transformers)
1272
+
1273
+ You can finetune this model on your own dataset.
1274
+
1275
+ <details><summary>Click to expand</summary>
1276
+
1277
+ </details>
1278
+ -->
1279
+
1280
+ <!--
1281
+ ### Out-of-Scope Use
1282
+
1283
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
1284
+ -->
1285
+
1286
+ ## Evaluation
1287
+
1288
+ ### Metrics
1289
+
1290
+ #### Sparse Information Retrieval
1291
+
1292
+ * Datasets: `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
1293
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
1294
+
1295
+ | Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
1296
+ |:----------------------|:------------|:-------------|:-----------|:-----------------|:------------|:-----------|:-------------|:-------------|:-------------------|:------------|:------------|:------------|:---------------|
1297
+ | dot_accuracy@1 | 0.32 | 0.3 | 0.24 | 0.2 | 0.62 | 0.44 | 0.22 | 0.68 | 0.36 | 0.28 | 0.02 | 0.36 | 0.5306 |
1298
+ | dot_accuracy@3 | 0.5 | 0.46 | 0.46 | 0.38 | 0.78 | 0.66 | 0.42 | 0.78 | 0.52 | 0.52 | 0.14 | 0.56 | 0.7959 |
1299
+ | dot_accuracy@5 | 0.64 | 0.48 | 0.54 | 0.42 | 0.86 | 0.78 | 0.46 | 0.8 | 0.58 | 0.62 | 0.22 | 0.6 | 0.9184 |
1300
+ | dot_accuracy@10 | 0.72 | 0.6 | 0.72 | 0.52 | 0.92 | 0.84 | 0.54 | 0.88 | 0.78 | 0.78 | 0.38 | 0.66 | 0.9592 |
1301
+ | dot_precision@1 | 0.32 | 0.3 | 0.24 | 0.2 | 0.62 | 0.44 | 0.22 | 0.68 | 0.36 | 0.28 | 0.02 | 0.36 | 0.5306 |
1302
+ | dot_precision@3 | 0.1667 | 0.2733 | 0.1533 | 0.14 | 0.4733 | 0.22 | 0.1667 | 0.3533 | 0.1733 | 0.2067 | 0.0467 | 0.2067 | 0.551 |
1303
+ | dot_precision@5 | 0.128 | 0.232 | 0.108 | 0.092 | 0.452 | 0.156 | 0.144 | 0.24 | 0.124 | 0.184 | 0.044 | 0.132 | 0.5102 |
1304
+ | dot_precision@10 | 0.072 | 0.214 | 0.074 | 0.06 | 0.396 | 0.086 | 0.09 | 0.138 | 0.082 | 0.138 | 0.038 | 0.078 | 0.4122 |
1305
+ | dot_recall@1 | 0.32 | 0.0107 | 0.23 | 0.0883 | 0.0677 | 0.44 | 0.1393 | 0.34 | 0.3467 | 0.0597 | 0.02 | 0.335 | 0.0349 |
1306
+ | dot_recall@3 | 0.5 | 0.0404 | 0.44 | 0.1817 | 0.142 | 0.64 | 0.2604 | 0.53 | 0.4707 | 0.1287 | 0.14 | 0.535 | 0.1078 |
1307
+ | dot_recall@5 | 0.64 | 0.0582 | 0.51 | 0.1923 | 0.1928 | 0.7367 | 0.3118 | 0.6 | 0.5507 | 0.1897 | 0.22 | 0.575 | 0.1671 |
1308
+ | dot_recall@10 | 0.72 | 0.085 | 0.67 | 0.2523 | 0.2816 | 0.7967 | 0.3924 | 0.69 | 0.7507 | 0.2837 | 0.38 | 0.66 | 0.2671 |
1309
+ | **dot_ndcg@10** | **0.5061** | **0.2416** | **0.4431** | **0.2097** | **0.4999** | **0.6191** | **0.3072** | **0.6198** | **0.5326** | **0.2575** | **0.1746** | **0.5065** | **0.4629** |
1310
+ | dot_mrr@10 | 0.4382 | 0.3932 | 0.3818 | 0.299 | 0.7169 | 0.5679 | 0.3309 | 0.7347 | 0.4783 | 0.4254 | 0.1124 | 0.4627 | 0.6864 |
1311
+ | dot_map@100 | 0.4501 | 0.0842 | 0.3762 | 0.168 | 0.3705 | 0.5645 | 0.251 | 0.5405 | 0.4735 | 0.1769 | 0.1156 | 0.4604 | 0.3432 |
1312
+ | query_active_dims | 105.08 | 150.78 | 97.18 | 250.86 | 146.02 | 253.38 | 85.7 | 152.54 | 52.9 | 197.2 | 732.46 | 276.2 | 37.8163 |
1313
+ | query_sparsity_ratio | 0.9966 | 0.9951 | 0.9968 | 0.9918 | 0.9952 | 0.9917 | 0.9972 | 0.995 | 0.9983 | 0.9935 | 0.976 | 0.991 | 0.9988 |
1314
+ | corpus_active_dims | 381.3875 | 807.0742 | 564.0423 | 643.3269 | 481.7581 | 749.9185 | 416.9383 | 553.4067 | 61.3555 | 676.0037 | 648.4751 | 729.4652 | 493.4804 |
1315
+ | corpus_sparsity_ratio | 0.9875 | 0.9736 | 0.9815 | 0.9789 | 0.9842 | 0.9754 | 0.9863 | 0.9819 | 0.998 | 0.9779 | 0.9788 | 0.9761 | 0.9838 |
1316
+
1317
+ #### Sparse Nano BEIR
1318
+
1319
+ * Dataset: `NanoBEIR_mean`
1320
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1321
+ ```json
1322
+ {
1323
+ "dataset_names": [
1324
+ "msmarco",
1325
+ "nfcorpus",
1326
+ "nq"
1327
+ ]
1328
+ }
1329
+ ```
1330
+
1331
+ | Metric | Value |
1332
+ |:----------------------|:-----------|
1333
+ | dot_accuracy@1 | 0.3067 |
1334
+ | dot_accuracy@3 | 0.4933 |
1335
+ | dot_accuracy@5 | 0.58 |
1336
+ | dot_accuracy@10 | 0.6733 |
1337
+ | dot_precision@1 | 0.3067 |
1338
+ | dot_precision@3 | 0.2022 |
1339
+ | dot_precision@5 | 0.1613 |
1340
+ | dot_precision@10 | 0.1133 |
1341
+ | dot_recall@1 | 0.194 |
1342
+ | dot_recall@3 | 0.3495 |
1343
+ | dot_recall@5 | 0.42 |
1344
+ | dot_recall@10 | 0.4831 |
1345
+ | **dot_ndcg@10** | **0.3946** |
1346
+ | dot_mrr@10 | 0.414 |
1347
+ | dot_map@100 | 0.3065 |
1348
+ | query_active_dims | 136.6333 |
1349
+ | query_sparsity_ratio | 0.9955 |
1350
+ | corpus_active_dims | 565.1 |
1351
+ | corpus_sparsity_ratio | 0.9815 |
1352
+
1353
+ #### Sparse Nano BEIR
1354
+
1355
+ * Dataset: `NanoBEIR_mean`
1356
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
1357
+ ```json
1358
+ {
1359
+ "dataset_names": [
1360
+ "climatefever",
1361
+ "dbpedia",
1362
+ "fever",
1363
+ "fiqa2018",
1364
+ "hotpotqa",
1365
+ "msmarco",
1366
+ "nfcorpus",
1367
+ "nq",
1368
+ "quoraretrieval",
1369
+ "scidocs",
1370
+ "arguana",
1371
+ "scifact",
1372
+ "touche2020"
1373
+ ]
1374
+ }
1375
+ ```
1376
+
1377
+ | Metric | Value |
1378
+ |:----------------------|:-----------|
1379
+ | dot_accuracy@1 | 0.3516 |
1380
+ | dot_accuracy@3 | 0.5366 |
1381
+ | dot_accuracy@5 | 0.6091 |
1382
+ | dot_accuracy@10 | 0.7153 |
1383
+ | dot_precision@1 | 0.3516 |
1384
+ | dot_precision@3 | 0.2408 |
1385
+ | dot_precision@5 | 0.1959 |
1386
+ | dot_precision@10 | 0.1445 |
1387
+ | dot_recall@1 | 0.1871 |
1388
+ | dot_recall@3 | 0.3167 |
1389
+ | dot_recall@5 | 0.3803 |
1390
+ | dot_recall@10 | 0.4792 |
1391
+ | **dot_ndcg@10** | **0.4139** |
1392
+ | dot_mrr@10 | 0.4637 |
1393
+ | dot_map@100 | 0.3365 |
1394
+ | query_active_dims | 195.4823 |
1395
+ | query_sparsity_ratio | 0.9936 |
1396
+ | corpus_active_dims | 525.5023 |
1397
+ | corpus_sparsity_ratio | 0.9828 |
1398
+
1399
+ <!--
1400
+ ## Bias, Risks and Limitations
1401
+
1402
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1403
+ -->
1404
+
1405
+ <!--
1406
+ ### Recommendations
1407
+
1408
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1409
+ -->
1410
+
1411
+ ## Training Details
1412
+
1413
+ ### Training Dataset
1414
+
1415
+ #### gooaq
1416
+
1417
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1418
+ * Size: 99,000 training samples
1419
+ * Columns: <code>question</code> and <code>answer</code>
1420
+ * Approximate statistics based on the first 1000 samples:
1421
+ | | question | answer |
1422
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1423
+ | type | string | string |
1424
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.79 tokens</li><li>max: 24 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.02 tokens</li><li>max: 153 tokens</li></ul> |
1425
+ * Samples:
1426
+ | question | answer |
1427
+ |:-----------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1428
+ | <code>what are the 5 characteristics of a star?</code> | <code>Key Concept: Characteristics used to classify stars include color, temperature, size, composition, and brightness.</code> |
1429
+ | <code>are copic markers alcohol ink?</code> | <code>Copic Ink is alcohol-based and flammable. Keep away from direct sunlight and extreme temperatures.</code> |
1430
+ | <code>what is the difference between appellate term and appellate division?</code> | <code>Appellate terms An appellate term is an intermediate appellate court that hears appeals from the inferior courts within their designated counties or judicial districts, and are intended to ease the workload on the Appellate Division and provide a less expensive forum closer to the people.</code> |
1431
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1432
+ ```json
1433
+ {
1434
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1435
+ "document_regularizer_weight": 3e-05,
1436
+ "query_regularizer_weight": 5e-05
1437
+ }
1438
+ ```
1439
+
1440
+ ### Evaluation Dataset
1441
+
1442
+ #### gooaq
1443
+
1444
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1445
+ * Size: 1,000 evaluation samples
1446
+ * Columns: <code>question</code> and <code>answer</code>
1447
+ * Approximate statistics based on the first 1000 samples:
1448
+ | | question | answer |
1449
+ |:--------|:----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
1450
+ | type | string | string |
1451
+ | details | <ul><li>min: 8 tokens</li><li>mean: 11.93 tokens</li><li>max: 25 tokens</li></ul> | <ul><li>min: 14 tokens</li><li>mean: 60.84 tokens</li><li>max: 127 tokens</li></ul> |
1452
+ * Samples:
1453
+ | question | answer |
1454
+ |:-----------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1455
+ | <code>should you take ibuprofen with high blood pressure?</code> | <code>In general, people with high blood pressure should use acetaminophen or possibly aspirin for over-the-counter pain relief. Unless your health care provider has said it's OK, you should not use ibuprofen, ketoprofen, or naproxen sodium. If aspirin or acetaminophen doesn't help with your pain, call your doctor.</code> |
1456
+ | <code>how old do you have to be to work in sc?</code> | <code>The general minimum age of employment for South Carolina youth is 14, although the state allows younger children who are performers to work in show business. If their families are agricultural workers, children younger than age 14 may also participate in farm labor.</code> |
1457
+ | <code>how to write a topic proposal for a research paper?</code> | <code>['Write down the main topic of your paper. ... ', 'Write two or three short sentences under the main topic that explain why you chose that topic. ... ', 'Write a thesis sentence that states the angle and purpose of your research paper. ... ', 'List the items you will cover in the body of the paper that support your thesis statement.']</code> |
1458
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1459
+ ```json
1460
+ {
1461
+ "loss": "SparseMultipleNegativesRankingLoss(scale=1.0, similarity_fct='dot_score')",
1462
+ "document_regularizer_weight": 3e-05,
1463
+ "query_regularizer_weight": 5e-05
1464
+ }
1465
+ ```
1466
+
1467
+ ### Training Hyperparameters
1468
+ #### Non-Default Hyperparameters
1469
+
1470
+ - `eval_strategy`: steps
1471
+ - `per_device_train_batch_size`: 32
1472
+ - `per_device_eval_batch_size`: 32
1473
+ - `learning_rate`: 2e-05
1474
+ - `num_train_epochs`: 1
1475
+ - `bf16`: True
1476
+ - `load_best_model_at_end`: True
1477
+ - `batch_sampler`: no_duplicates
1478
+
1479
+ #### All Hyperparameters
1480
+ <details><summary>Click to expand</summary>
1481
+
1482
+ - `overwrite_output_dir`: False
1483
+ - `do_predict`: False
1484
+ - `eval_strategy`: steps
1485
+ - `prediction_loss_only`: True
1486
+ - `per_device_train_batch_size`: 32
1487
+ - `per_device_eval_batch_size`: 32
1488
+ - `per_gpu_train_batch_size`: None
1489
+ - `per_gpu_eval_batch_size`: None
1490
+ - `gradient_accumulation_steps`: 1
1491
+ - `eval_accumulation_steps`: None
1492
+ - `torch_empty_cache_steps`: None
1493
+ - `learning_rate`: 2e-05
1494
+ - `weight_decay`: 0.0
1495
+ - `adam_beta1`: 0.9
1496
+ - `adam_beta2`: 0.999
1497
+ - `adam_epsilon`: 1e-08
1498
+ - `max_grad_norm`: 1.0
1499
+ - `num_train_epochs`: 1
1500
+ - `max_steps`: -1
1501
+ - `lr_scheduler_type`: linear
1502
+ - `lr_scheduler_kwargs`: {}
1503
+ - `warmup_ratio`: 0.0
1504
+ - `warmup_steps`: 0
1505
+ - `log_level`: passive
1506
+ - `log_level_replica`: warning
1507
+ - `log_on_each_node`: True
1508
+ - `logging_nan_inf_filter`: True
1509
+ - `save_safetensors`: True
1510
+ - `save_on_each_node`: False
1511
+ - `save_only_model`: False
1512
+ - `restore_callback_states_from_checkpoint`: False
1513
+ - `no_cuda`: False
1514
+ - `use_cpu`: False
1515
+ - `use_mps_device`: False
1516
+ - `seed`: 42
1517
+ - `data_seed`: None
1518
+ - `jit_mode_eval`: False
1519
+ - `use_ipex`: False
1520
+ - `bf16`: True
1521
+ - `fp16`: False
1522
+ - `fp16_opt_level`: O1
1523
+ - `half_precision_backend`: auto
1524
+ - `bf16_full_eval`: False
1525
+ - `fp16_full_eval`: False
1526
+ - `tf32`: None
1527
+ - `local_rank`: 0
1528
+ - `ddp_backend`: None
1529
+ - `tpu_num_cores`: None
1530
+ - `tpu_metrics_debug`: False
1531
+ - `debug`: []
1532
+ - `dataloader_drop_last`: False
1533
+ - `dataloader_num_workers`: 0
1534
+ - `dataloader_prefetch_factor`: None
1535
+ - `past_index`: -1
1536
+ - `disable_tqdm`: False
1537
+ - `remove_unused_columns`: True
1538
+ - `label_names`: None
1539
+ - `load_best_model_at_end`: True
1540
+ - `ignore_data_skip`: False
1541
+ - `fsdp`: []
1542
+ - `fsdp_min_num_params`: 0
1543
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1544
+ - `fsdp_transformer_layer_cls_to_wrap`: None
1545
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1546
+ - `deepspeed`: None
1547
+ - `label_smoothing_factor`: 0.0
1548
+ - `optim`: adamw_torch
1549
+ - `optim_args`: None
1550
+ - `adafactor`: False
1551
+ - `group_by_length`: False
1552
+ - `length_column_name`: length
1553
+ - `ddp_find_unused_parameters`: None
1554
+ - `ddp_bucket_cap_mb`: None
1555
+ - `ddp_broadcast_buffers`: False
1556
+ - `dataloader_pin_memory`: True
1557
+ - `dataloader_persistent_workers`: False
1558
+ - `skip_memory_metrics`: True
1559
+ - `use_legacy_prediction_loop`: False
1560
+ - `push_to_hub`: False
1561
+ - `resume_from_checkpoint`: None
1562
+ - `hub_model_id`: None
1563
+ - `hub_strategy`: every_save
1564
+ - `hub_private_repo`: None
1565
+ - `hub_always_push`: False
1566
+ - `gradient_checkpointing`: False
1567
+ - `gradient_checkpointing_kwargs`: None
1568
+ - `include_inputs_for_metrics`: False
1569
+ - `include_for_metrics`: []
1570
+ - `eval_do_concat_batches`: True
1571
+ - `fp16_backend`: auto
1572
+ - `push_to_hub_model_id`: None
1573
+ - `push_to_hub_organization`: None
1574
+ - `mp_parameters`:
1575
+ - `auto_find_batch_size`: False
1576
+ - `full_determinism`: False
1577
+ - `torchdynamo`: None
1578
+ - `ray_scope`: last
1579
+ - `ddp_timeout`: 1800
1580
+ - `torch_compile`: False
1581
+ - `torch_compile_backend`: None
1582
+ - `torch_compile_mode`: None
1583
+ - `include_tokens_per_second`: False
1584
+ - `include_num_input_tokens_seen`: False
1585
+ - `neftune_noise_alpha`: None
1586
+ - `optim_target_modules`: None
1587
+ - `batch_eval_metrics`: False
1588
+ - `eval_on_start`: False
1589
+ - `use_liger_kernel`: False
1590
+ - `eval_use_gather_object`: False
1591
+ - `average_tokens_across_devices`: False
1592
+ - `prompts`: None
1593
+ - `batch_sampler`: no_duplicates
1594
+ - `multi_dataset_batch_sampler`: proportional
1595
+ - `router_mapping`: {}
1596
+ - `learning_rate_mapping`: {}
1597
+
1598
+ </details>
1599
+
1600
+ ### Training Logs
1601
+ | Epoch | Step | Training Loss | Validation Loss | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 |
1602
+ |:----------:|:--------:|:-------------:|:---------------:|:-----------------------:|:------------------------:|:------------------:|:-------------------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|
1603
+ | 0.0323 | 100 | 81.7292 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1604
+ | 0.0646 | 200 | 4.3059 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1605
+ | 0.0970 | 300 | 0.8078 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1606
+ | 0.1293 | 400 | 0.4309 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1607
+ | 0.1616 | 500 | 0.3837 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1608
+ | 0.1939 | 600 | 0.282 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1609
+ | 0.1972 | 610 | - | 0.1867 | 0.4508 | 0.2059 | 0.3905 | 0.3491 | - | - | - | - | - | - | - | - | - | - |
1610
+ | 0.2262 | 700 | 0.2593 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1611
+ | 0.2586 | 800 | 0.2161 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1612
+ | 0.2909 | 900 | 0.2 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1613
+ | 0.3232 | 1000 | 0.2259 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1614
+ | 0.3555 | 1100 | 0.2161 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1615
+ | 0.3878 | 1200 | 0.1835 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1616
+ | 0.3943 | 1220 | - | 0.1368 | 0.4567 | 0.2373 | 0.4209 | 0.3717 | - | - | - | - | - | - | - | - | - | - |
1617
+ | 0.4202 | 1300 | 0.1936 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1618
+ | 0.4525 | 1400 | 0.1689 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1619
+ | 0.4848 | 1500 | 0.1858 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1620
+ | 0.5171 | 1600 | 0.1639 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1621
+ | 0.5495 | 1700 | 0.1376 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1622
+ | 0.5818 | 1800 | 0.1677 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1623
+ | **0.5915** | **1830** | **-** | **0.1138** | **0.5061** | **0.2416** | **0.4431** | **0.3969** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** | **-** |
1624
+ | 0.6141 | 1900 | 0.1483 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1625
+ | 0.6464 | 2000 | 0.1513 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1626
+ | 0.6787 | 2100 | 0.1449 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1627
+ | 0.7111 | 2200 | 0.193 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1628
+ | 0.7434 | 2300 | 0.1554 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1629
+ | 0.7757 | 2400 | 0.1372 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1630
+ | 0.7886 | 2440 | - | 0.1148 | 0.5084 | 0.2240 | 0.4428 | 0.3917 | - | - | - | - | - | - | - | - | - | - |
1631
+ | 0.8080 | 2500 | 0.1308 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1632
+ | 0.8403 | 2600 | 0.1284 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1633
+ | 0.8727 | 2700 | 0.1309 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1634
+ | 0.9050 | 2800 | 0.1458 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1635
+ | 0.9373 | 2900 | 0.1351 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1636
+ | 0.9696 | 3000 | 0.1135 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1637
+ | 0.9858 | 3050 | - | 0.1068 | 0.5062 | 0.2238 | 0.4539 | 0.3946 | - | - | - | - | - | - | - | - | - | - |
1638
+ | -1 | -1 | - | - | 0.5061 | 0.2416 | 0.4431 | 0.4139 | 0.2097 | 0.4999 | 0.6191 | 0.3072 | 0.6198 | 0.5326 | 0.2575 | 0.1746 | 0.5065 | 0.4629 |
1639
+
1640
+ * The bold row denotes the saved checkpoint.
1641
+
1642
+ ### Environmental Impact
1643
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
1644
+ - **Energy Consumed**: 0.043 kWh
1645
+ - **Carbon Emitted**: 0.017 kg of CO2
1646
+ - **Hours Used**: 0.193 hours
1647
+
1648
+ ### Training Hardware
1649
+ - **On Cloud**: No
1650
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3090
1651
+ - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
1652
+ - **RAM Size**: 31.78 GB
1653
+
1654
+ ### Framework Versions
1655
+ - Python: 3.11.6
1656
+ - Sentence Transformers: 4.2.0.dev0
1657
+ - Transformers: 4.52.4
1658
+ - PyTorch: 2.7.1+cu126
1659
+ - Accelerate: 1.5.1
1660
+ - Datasets: 2.21.0
1661
+ - Tokenizers: 0.21.1
1662
+
1663
+ ## Citation
1664
+
1665
+ ### BibTeX
1666
+
1667
+ #### Sentence Transformers
1668
+ ```bibtex
1669
+ @inproceedings{reimers-2019-sentence-bert,
1670
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1671
+ author = "Reimers, Nils and Gurevych, Iryna",
1672
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1673
+ month = "11",
1674
+ year = "2019",
1675
+ publisher = "Association for Computational Linguistics",
1676
+ url = "https://arxiv.org/abs/1908.10084",
1677
+ }
1678
+ ```
1679
+
1680
+ #### SpladeLoss
1681
+ ```bibtex
1682
+ @misc{formal2022distillationhardnegativesampling,
1683
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1684
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1685
+ year={2022},
1686
+ eprint={2205.04733},
1687
+ archivePrefix={arXiv},
1688
+ primaryClass={cs.IR},
1689
+ url={https://arxiv.org/abs/2205.04733},
1690
+ }
1691
+ ```
1692
+
1693
+ #### SparseMultipleNegativesRankingLoss
1694
+ ```bibtex
1695
+ @misc{henderson2017efficient,
1696
+ title={Efficient Natural Language Response Suggestion for Smart Reply},
1697
+ author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
1698
+ year={2017},
1699
+ eprint={1705.00652},
1700
+ archivePrefix={arXiv},
1701
+ primaryClass={cs.CL}
1702
+ }
1703
+ ```
1704
+
1705
+ #### FlopsLoss
1706
+ ```bibtex
1707
+ @article{paria2020minimizing,
1708
+ title={Minimizing flops to learn efficient sparse representations},
1709
+ author={Paria, Biswajit and Yeh, Chih-Kuan and Yen, Ian EH and Xu, Ning and Ravikumar, Pradeep and P{'o}czos, Barnab{'a}s},
1710
+ journal={arXiv preprint arXiv:2004.05665},
1711
+ year={2020}
1712
+ }
1713
+ ```
1714
+
1715
+ <!--
1716
+ ## Glossary
1717
+
1718
+ *Clearly define terms in order to be accessible across audiences.*
1719
+ -->
1720
+
1721
+ <!--
1722
+ ## Model Card Authors
1723
+
1724
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1725
+ -->
1726
+
1727
+ <!--
1728
+ ## Model Card Contact
1729
+
1730
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1731
+ -->
adapter_config.json ADDED
@@ -0,0 +1,34 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alpha_pattern": {},
3
+ "auto_mapping": null,
4
+ "base_model_name_or_path": "distilbert/distilbert-base-uncased",
5
+ "bias": "none",
6
+ "corda_config": null,
7
+ "eva_config": null,
8
+ "exclude_modules": null,
9
+ "fan_in_fan_out": false,
10
+ "inference_mode": false,
11
+ "init_lora_weights": true,
12
+ "layer_replication": null,
13
+ "layers_pattern": null,
14
+ "layers_to_transform": null,
15
+ "loftq_config": {},
16
+ "lora_alpha": 256,
17
+ "lora_bias": false,
18
+ "lora_dropout": 0.05,
19
+ "megatron_config": null,
20
+ "megatron_core": "megatron.core",
21
+ "modules_to_save": null,
22
+ "peft_type": "LORA",
23
+ "r": 128,
24
+ "rank_pattern": {},
25
+ "revision": null,
26
+ "target_modules": [
27
+ "q_lin",
28
+ "v_lin"
29
+ ],
30
+ "task_type": "FEATURE_EXTRACTION",
31
+ "trainable_token_indices": null,
32
+ "use_dora": false,
33
+ "use_rslora": false
34
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bfda768437c61da6f812fcce02918430ac38c3e5ca80af51821c270668d1b2a0
3
+ size 9440744
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "SparseEncoder",
3
+ "__version__": {
4
+ "sentence_transformers": "4.2.0.dev0",
5
+ "transformers": "4.52.4",
6
+ "pytorch": "2.7.1+cu126"
7
+ },
8
+ "prompts": {
9
+ "query": "",
10
+ "document": ""
11
+ },
12
+ "default_prompt_name": null,
13
+ "similarity_fn_name": "dot"
14
+ }
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "DistilBertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff