Hugging Face's logo

tomaarsen
/
splade-distilbert-base-uncased-gooaq

Feature Extraction
sentence-transformers
Safetensors
English
distilbert
sparse-encoder
sparse
splade
Generated from Trainer
dataset_size:3011496
loss:SpladeLoss
Eval Results
text-embeddings-inference
Model card Files
xet
Community
tomaarsen HF Staff commited on
Commit
53cc8a5
·
verified ·
1 Parent(s): 5ef2799

Add new SparseEncoder model

Browse files
Files changed (11) hide show
  1. 1_SpladePooling/config.json +5 -0
  2. README.md +1343 -0
  3. config.json +24 -0
  4. config_sentence_transformers.json +11 -0
  5. model.safetensors +3 -0
  6. modules.json +14 -0
  7. sentence_bert_config.json +4 -0
  8. special_tokens_map.json +7 -0
  9. tokenizer.json +0 -0
  10. tokenizer_config.json +56 -0
  11. vocab.txt +0 -0
1_SpladePooling/config.json ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ {
2
+ "pooling_strategy": "max",
3
+ "activation_function": "relu",
4
+ "word_embedding_dimension": 30522
5
+ }
README.md ADDED
@@ -0,0 +1,1343 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - sparse-encoder
8
+ - sparse
9
+ - splade
10
+ - generated_from_trainer
11
+ - dataset_size:3011496
12
+ - loss:SpladeLoss
13
+ base_model: distilbert/distilbert-base-uncased
14
+ widget:
15
+ - source_sentence: how much percent of alcohol is in scotch?
16
+ sentences:
17
+ - Our 24-hour day comes from the ancient Egyptians who divided day-time into 10
18
+ hours they measured with devices such as shadow clocks, and added a twilight hour
19
+ at the beginning and another one at the end of the day-time, says Lomb. "Night-time
20
+ was divided in 12 hours, based on the observations of stars.
21
+ - After distillation, a Scotch Whisky can be anywhere between 60-75% ABV, with American
22
+ Whiskey rocketing right into the 90% region. Before being placed in casks, Scotch
23
+ is usually diluted to around 63.5% ABV (68% for grain); welcome to the stage cask
24
+ strength Whisky.
25
+ - Money For Nothing. In season four Dominic West, the ostensible star of the series,
26
+ requested a reduced role so that he could spend more time with his family in London.
27
+ On the show it was explained that Jimmy McNulty had taken a patrol job which required
28
+ less strenuous work.
29
+ - source_sentence: what are the major causes of poor listening?
30
+ sentences:
31
+ - The four main causes of poor listening are due to not concentrating, listening
32
+ too hard, jumping to conclusions and focusing on delivery and personal appearance.
33
+ Sometimes we just don't feel attentive enough and hence don't concentrate.
34
+ - That's called being idle. “System Idle Process” is the software that runs when
35
+ the computer has absolutely nothing better to do. It has the lowest possible priority
36
+ and uses as few resources as possible, so that if anything at all comes along
37
+ for the CPU to work on, it can.
38
+ - 'No alcohol wine: how it''s made It''s not easy. There are three main methods
39
+ currently in use. Vacuum distillation sees alcohol and other volatiles removed
40
+ at a relatively low temperature (25°C-30°C), with aromatics blended back in afterwards.'
41
+ - source_sentence: are jess and justin still together?
42
+ sentences:
43
+ - Download photos and videos to your device On your iPhone, iPad, or iPod touch,
44
+ tap Settings > [your name] > iCloud > Photos. Then select Download and Keep Originals
45
+ and import the photos to your computer. On your Mac, open the Photos app. Select
46
+ the photos and videos you want to copy.
47
+ - Later, Justin reunites with Jessica at prom and the two get back together. ...
48
+ After a tearful goodbye to Jessica, the Jensens, and his friends, Justin dies
49
+ just before graduation.
50
+ - Incumbent president Muhammadu Buhari won his reelection bid, defeating his closest
51
+ rival Atiku Abubakar by over 3 million votes. He was issued a Certificate of Return,
52
+ and was sworn in on May 29, 2019, the former date of Democracy Day (Nigeria).
53
+ - source_sentence: when humans are depicted in hindu art?
54
+ sentences:
55
+ - 'Answer: Humans are depicted in Hindu art often in sensuous and erotic postures.'
56
+ - Bettas are carnivores. They require foods high in animal protein. Their preferred
57
+ diet in nature includes insects and insect larvae. In captivity, they thrive on
58
+ a varied diet of pellets or flakes made from fish meal, as well as frozen or freeze-dried
59
+ bloodworms.
60
+ - An active continental margin is found on the leading edge of the continent where
61
+ it is crashing into an oceanic plate. ... Passive continental margins are found
62
+ along the remaining coastlines.
63
+ - source_sentence: what is the difference between 18 and 20 inch tires?
64
+ sentences:
65
+ - '[''Alienware m17 R3. The best gaming laptop overall offers big power in slim,
66
+ redesigned chassis. ... '', ''Dell G3 15. ... '', ''Asus ROG Zephyrus G14. ...
67
+ '', ''Lenovo Legion Y545. ... '', ''Alienware Area 51m. ... '', ''Asus ROG Mothership.
68
+ ... '', ''Asus ROG Strix Scar III. ... '', ''HP Omen 17 (2019)'']'
69
+ - So extracurricular activities are just activities that you do outside of class.
70
+ The Common App says that extracurricular activities "include arts, athletics,
71
+ clubs, employment, personal commitments, and other pursuits."
72
+ - The only real difference is a 20" rim would be more likely to be damaged, as you
73
+ pointed out. Beyond looks, there is zero benefit for the 20" rim. Also, just the
74
+ availability of tires will likely be much more limited for the larger rim. ...
75
+ Tire selection is better for 18" wheels than 20" wheels.
76
+ datasets:
77
+ - sentence-transformers/gooaq
78
+ pipeline_tag: feature-extraction
79
+ library_name: sentence-transformers
80
+ metrics:
81
+ - dot_accuracy@1
82
+ - dot_accuracy@3
83
+ - dot_accuracy@5
84
+ - dot_accuracy@10
85
+ - dot_precision@1
86
+ - dot_precision@3
87
+ - dot_precision@5
88
+ - dot_precision@10
89
+ - dot_recall@1
90
+ - dot_recall@3
91
+ - dot_recall@5
92
+ - dot_recall@10
93
+ - dot_ndcg@10
94
+ - dot_mrr@10
95
+ - dot_map@100
96
+ co2_eq_emissions:
97
+ emissions: 520.0942452560889
98
+ energy_consumed: 1.3380282202203462
99
+ source: codecarbon
100
+ training_type: fine-tuning
101
+ on_cloud: false
102
+ cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
103
+ ram_total_size: 31.777088165283203
104
+ hours_used: 3.894
105
+ hardware_used: 1 x NVIDIA GeForce RTX 3090
106
+ model-index:
107
+ - name: splade-distilbert-base-uncased trained on GooAQ
108
+ results:
109
+ - task:
110
+ type: sparse-information-retrieval
111
+ name: Sparse Information Retrieval
112
+ dataset:
113
+ name: NanoClimateFEVER
114
+ type: NanoClimateFEVER
115
+ metrics:
116
+ - type: dot_accuracy@1
117
+ value: 0.22
118
+ name: Dot Accuracy@1
119
+ - type: dot_accuracy@3
120
+ value: 0.38
121
+ name: Dot Accuracy@3
122
+ - type: dot_accuracy@5
123
+ value: 0.48
124
+ name: Dot Accuracy@5
125
+ - type: dot_accuracy@10
126
+ value: 0.54
127
+ name: Dot Accuracy@10
128
+ - type: dot_precision@1
129
+ value: 0.22
130
+ name: Dot Precision@1
131
+ - type: dot_precision@3
132
+ value: 0.13333333333333333
133
+ name: Dot Precision@3
134
+ - type: dot_precision@5
135
+ value: 0.10800000000000001
136
+ name: Dot Precision@5
137
+ - type: dot_precision@10
138
+ value: 0.066
139
+ name: Dot Precision@10
140
+ - type: dot_recall@1
141
+ value: 0.11666666666666665
142
+ name: Dot Recall@1
143
+ - type: dot_recall@3
144
+ value: 0.17066666666666663
145
+ name: Dot Recall@3
146
+ - type: dot_recall@5
147
+ value: 0.21566666666666662
148
+ name: Dot Recall@5
149
+ - type: dot_recall@10
150
+ value: 0.254
151
+ name: Dot Recall@10
152
+ - type: dot_ndcg@10
153
+ value: 0.2261790676778388
154
+ name: Dot Ndcg@10
155
+ - type: dot_mrr@10
156
+ value: 0.3172460317460317
157
+ name: Dot Mrr@10
158
+ - type: dot_map@100
159
+ value: 0.18142380833103508
160
+ name: Dot Map@100
161
+ - task:
162
+ type: sparse-information-retrieval
163
+ name: Sparse Information Retrieval
164
+ dataset:
165
+ name: NanoDBPedia
166
+ type: NanoDBPedia
167
+ metrics:
168
+ - type: dot_accuracy@1
169
+ value: 0.52
170
+ name: Dot Accuracy@1
171
+ - type: dot_accuracy@3
172
+ value: 0.68
173
+ name: Dot Accuracy@3
174
+ - type: dot_accuracy@5
175
+ value: 0.72
176
+ name: Dot Accuracy@5
177
+ - type: dot_accuracy@10
178
+ value: 0.86
179
+ name: Dot Accuracy@10
180
+ - type: dot_precision@1
181
+ value: 0.52
182
+ name: Dot Precision@1
183
+ - type: dot_precision@3
184
+ value: 0.42666666666666664
185
+ name: Dot Precision@3
186
+ - type: dot_precision@5
187
+ value: 0.39599999999999996
188
+ name: Dot Precision@5
189
+ - type: dot_precision@10
190
+ value: 0.3600000000000001
191
+ name: Dot Precision@10
192
+ - type: dot_recall@1
193
+ value: 0.04167451005436889
194
+ name: Dot Recall@1
195
+ - type: dot_recall@3
196
+ value: 0.11687345791837826
197
+ name: Dot Recall@3
198
+ - type: dot_recall@5
199
+ value: 0.1514949553474152
200
+ name: Dot Recall@5
201
+ - type: dot_recall@10
202
+ value: 0.24492588020184664
203
+ name: Dot Recall@10
204
+ - type: dot_ndcg@10
205
+ value: 0.4347755184129968
206
+ name: Dot Ndcg@10
207
+ - type: dot_mrr@10
208
+ value: 0.6113253968253968
209
+ name: Dot Mrr@10
210
+ - type: dot_map@100
211
+ value: 0.3307600412459809
212
+ name: Dot Map@100
213
+ - task:
214
+ type: sparse-information-retrieval
215
+ name: Sparse Information Retrieval
216
+ dataset:
217
+ name: NanoFEVER
218
+ type: NanoFEVER
219
+ metrics:
220
+ - type: dot_accuracy@1
221
+ value: 0.68
222
+ name: Dot Accuracy@1
223
+ - type: dot_accuracy@3
224
+ value: 0.9
225
+ name: Dot Accuracy@3
226
+ - type: dot_accuracy@5
227
+ value: 0.92
228
+ name: Dot Accuracy@5
229
+ - type: dot_accuracy@10
230
+ value: 0.94
231
+ name: Dot Accuracy@10
232
+ - type: dot_precision@1
233
+ value: 0.68
234
+ name: Dot Precision@1
235
+ - type: dot_precision@3
236
+ value: 0.3066666666666667
237
+ name: Dot Precision@3
238
+ - type: dot_precision@5
239
+ value: 0.18799999999999997
240
+ name: Dot Precision@5
241
+ - type: dot_precision@10
242
+ value: 0.09999999999999998
243
+ name: Dot Precision@10
244
+ - type: dot_recall@1
245
+ value: 0.6566666666666666
246
+ name: Dot Recall@1
247
+ - type: dot_recall@3
248
+ value: 0.8566666666666666
249
+ name: Dot Recall@3
250
+ - type: dot_recall@5
251
+ value: 0.8766666666666667
252
+ name: Dot Recall@5
253
+ - type: dot_recall@10
254
+ value: 0.9166666666666667
255
+ name: Dot Recall@10
256
+ - type: dot_ndcg@10
257
+ value: 0.8076442104958218
258
+ name: Dot Ndcg@10
259
+ - type: dot_mrr@10
260
+ value: 0.7872222222222223
261
+ name: Dot Mrr@10
262
+ - type: dot_map@100
263
+ value: 0.7674244773257931
264
+ name: Dot Map@100
265
+ - task:
266
+ type: sparse-information-retrieval
267
+ name: Sparse Information Retrieval
268
+ dataset:
269
+ name: NanoFiQA2018
270
+ type: NanoFiQA2018
271
+ metrics:
272
+ - type: dot_accuracy@1
273
+ value: 0.34
274
+ name: Dot Accuracy@1
275
+ - type: dot_accuracy@3
276
+ value: 0.6
277
+ name: Dot Accuracy@3
278
+ - type: dot_accuracy@5
279
+ value: 0.6
280
+ name: Dot Accuracy@5
281
+ - type: dot_accuracy@10
282
+ value: 0.68
283
+ name: Dot Accuracy@10
284
+ - type: dot_precision@1
285
+ value: 0.34
286
+ name: Dot Precision@1
287
+ - type: dot_precision@3
288
+ value: 0.2733333333333333
289
+ name: Dot Precision@3
290
+ - type: dot_precision@5
291
+ value: 0.19199999999999995
292
+ name: Dot Precision@5
293
+ - type: dot_precision@10
294
+ value: 0.12
295
+ name: Dot Precision@10
296
+ - type: dot_recall@1
297
+ value: 0.1459126984126984
298
+ name: Dot Recall@1
299
+ - type: dot_recall@3
300
+ value: 0.36649206349206354
301
+ name: Dot Recall@3
302
+ - type: dot_recall@5
303
+ value: 0.39607142857142863
304
+ name: Dot Recall@5
305
+ - type: dot_recall@10
306
+ value: 0.5118174603174603
307
+ name: Dot Recall@10
308
+ - type: dot_ndcg@10
309
+ value: 0.40298090899579636
310
+ name: Dot Ndcg@10
311
+ - type: dot_mrr@10
312
+ value: 0.4699365079365079
313
+ name: Dot Mrr@10
314
+ - type: dot_map@100
315
+ value: 0.32554676606566874
316
+ name: Dot Map@100
317
+ - task:
318
+ type: sparse-information-retrieval
319
+ name: Sparse Information Retrieval
320
+ dataset:
321
+ name: NanoHotpotQA
322
+ type: NanoHotpotQA
323
+ metrics:
324
+ - type: dot_accuracy@1
325
+ value: 0.74
326
+ name: Dot Accuracy@1
327
+ - type: dot_accuracy@3
328
+ value: 0.9
329
+ name: Dot Accuracy@3
330
+ - type: dot_accuracy@5
331
+ value: 0.92
332
+ name: Dot Accuracy@5
333
+ - type: dot_accuracy@10
334
+ value: 0.96
335
+ name: Dot Accuracy@10
336
+ - type: dot_precision@1
337
+ value: 0.74
338
+ name: Dot Precision@1
339
+ - type: dot_precision@3
340
+ value: 0.44666666666666655
341
+ name: Dot Precision@3
342
+ - type: dot_precision@5
343
+ value: 0.3
344
+ name: Dot Precision@5
345
+ - type: dot_precision@10
346
+ value: 0.158
347
+ name: Dot Precision@10
348
+ - type: dot_recall@1
349
+ value: 0.37
350
+ name: Dot Recall@1
351
+ - type: dot_recall@3
352
+ value: 0.67
353
+ name: Dot Recall@3
354
+ - type: dot_recall@5
355
+ value: 0.75
356
+ name: Dot Recall@5
357
+ - type: dot_recall@10
358
+ value: 0.79
359
+ name: Dot Recall@10
360
+ - type: dot_ndcg@10
361
+ value: 0.735492134090369
362
+ name: Dot Ndcg@10
363
+ - type: dot_mrr@10
364
+ value: 0.8236666666666668
365
+ name: Dot Mrr@10
366
+ - type: dot_map@100
367
+ value: 0.6711040184164847
368
+ name: Dot Map@100
369
+ - task:
370
+ type: sparse-information-retrieval
371
+ name: Sparse Information Retrieval
372
+ dataset:
373
+ name: NanoMSMARCO
374
+ type: NanoMSMARCO
375
+ metrics:
376
+ - type: dot_accuracy@1
377
+ value: 0.26
378
+ name: Dot Accuracy@1
379
+ - type: dot_accuracy@3
380
+ value: 0.42
381
+ name: Dot Accuracy@3
382
+ - type: dot_accuracy@5
383
+ value: 0.56
384
+ name: Dot Accuracy@5
385
+ - type: dot_accuracy@10
386
+ value: 0.78
387
+ name: Dot Accuracy@10
388
+ - type: dot_precision@1
389
+ value: 0.26
390
+ name: Dot Precision@1
391
+ - type: dot_precision@3
392
+ value: 0.13999999999999999
393
+ name: Dot Precision@3
394
+ - type: dot_precision@5
395
+ value: 0.11200000000000002
396
+ name: Dot Precision@5
397
+ - type: dot_precision@10
398
+ value: 0.07800000000000001
399
+ name: Dot Precision@10
400
+ - type: dot_recall@1
401
+ value: 0.26
402
+ name: Dot Recall@1
403
+ - type: dot_recall@3
404
+ value: 0.42
405
+ name: Dot Recall@3
406
+ - type: dot_recall@5
407
+ value: 0.56
408
+ name: Dot Recall@5
409
+ - type: dot_recall@10
410
+ value: 0.78
411
+ name: Dot Recall@10
412
+ - type: dot_ndcg@10
413
+ value: 0.48566582103432865
414
+ name: Dot Ndcg@10
415
+ - type: dot_mrr@10
416
+ value: 0.39584126984126977
417
+ name: Dot Mrr@10
418
+ - type: dot_map@100
419
+ value: 0.4043469063460482
420
+ name: Dot Map@100
421
+ - task:
422
+ type: sparse-information-retrieval
423
+ name: Sparse Information Retrieval
424
+ dataset:
425
+ name: NanoNFCorpus
426
+ type: NanoNFCorpus
427
+ metrics:
428
+ - type: dot_accuracy@1
429
+ value: 0.4
430
+ name: Dot Accuracy@1
431
+ - type: dot_accuracy@3
432
+ value: 0.56
433
+ name: Dot Accuracy@3
434
+ - type: dot_accuracy@5
435
+ value: 0.62
436
+ name: Dot Accuracy@5
437
+ - type: dot_accuracy@10
438
+ value: 0.66
439
+ name: Dot Accuracy@10
440
+ - type: dot_precision@1
441
+ value: 0.4
442
+ name: Dot Precision@1
443
+ - type: dot_precision@3
444
+ value: 0.34
445
+ name: Dot Precision@3
446
+ - type: dot_precision@5
447
+ value: 0.316
448
+ name: Dot Precision@5
449
+ - type: dot_precision@10
450
+ value: 0.24999999999999997
451
+ name: Dot Precision@10
452
+ - type: dot_recall@1
453
+ value: 0.02367568043139258
454
+ name: Dot Recall@1
455
+ - type: dot_recall@3
456
+ value: 0.07666946243603708
457
+ name: Dot Recall@3
458
+ - type: dot_recall@5
459
+ value: 0.09651550012847633
460
+ name: Dot Recall@5
461
+ - type: dot_recall@10
462
+ value: 0.11965782081153208
463
+ name: Dot Recall@10
464
+ - type: dot_ndcg@10
465
+ value: 0.3006693982141158
466
+ name: Dot Ndcg@10
467
+ - type: dot_mrr@10
468
+ value: 0.48066666666666663
469
+ name: Dot Mrr@10
470
+ - type: dot_map@100
471
+ value: 0.130037101713329
472
+ name: Dot Map@100
473
+ - task:
474
+ type: sparse-information-retrieval
475
+ name: Sparse Information Retrieval
476
+ dataset:
477
+ name: NanoNQ
478
+ type: NanoNQ
479
+ metrics:
480
+ - type: dot_accuracy@1
481
+ value: 0.38
482
+ name: Dot Accuracy@1
483
+ - type: dot_accuracy@3
484
+ value: 0.6
485
+ name: Dot Accuracy@3
486
+ - type: dot_accuracy@5
487
+ value: 0.7
488
+ name: Dot Accuracy@5
489
+ - type: dot_accuracy@10
490
+ value: 0.82
491
+ name: Dot Accuracy@10
492
+ - type: dot_precision@1
493
+ value: 0.38
494
+ name: Dot Precision@1
495
+ - type: dot_precision@3
496
+ value: 0.21333333333333335
497
+ name: Dot Precision@3
498
+ - type: dot_precision@5
499
+ value: 0.14800000000000002
500
+ name: Dot Precision@5
501
+ - type: dot_precision@10
502
+ value: 0.08599999999999998
503
+ name: Dot Precision@10
504
+ - type: dot_recall@1
505
+ value: 0.36
506
+ name: Dot Recall@1
507
+ - type: dot_recall@3
508
+ value: 0.57
509
+ name: Dot Recall@3
510
+ - type: dot_recall@5
511
+ value: 0.66
512
+ name: Dot Recall@5
513
+ - type: dot_recall@10
514
+ value: 0.77
515
+ name: Dot Recall@10
516
+ - type: dot_ndcg@10
517
+ value: 0.5618580490206411
518
+ name: Dot Ndcg@10
519
+ - type: dot_mrr@10
520
+ value: 0.5104126984126982
521
+ name: Dot Mrr@10
522
+ - type: dot_map@100
523
+ value: 0.49679904702760114
524
+ name: Dot Map@100
525
+ - task:
526
+ type: sparse-information-retrieval
527
+ name: Sparse Information Retrieval
528
+ dataset:
529
+ name: NanoQuoraRetrieval
530
+ type: NanoQuoraRetrieval
531
+ metrics:
532
+ - type: dot_accuracy@1
533
+ value: 0.82
534
+ name: Dot Accuracy@1
535
+ - type: dot_accuracy@3
536
+ value: 0.96
537
+ name: Dot Accuracy@3
538
+ - type: dot_accuracy@5
539
+ value: 0.96
540
+ name: Dot Accuracy@5
541
+ - type: dot_accuracy@10
542
+ value: 1.0
543
+ name: Dot Accuracy@10
544
+ - type: dot_precision@1
545
+ value: 0.82
546
+ name: Dot Precision@1
547
+ - type: dot_precision@3
548
+ value: 0.3733333333333333
549
+ name: Dot Precision@3
550
+ - type: dot_precision@5
551
+ value: 0.23599999999999993
552
+ name: Dot Precision@5
553
+ - type: dot_precision@10
554
+ value: 0.13199999999999998
555
+ name: Dot Precision@10
556
+ - type: dot_recall@1
557
+ value: 0.7340000000000001
558
+ name: Dot Recall@1
559
+ - type: dot_recall@3
560
+ value: 0.9113333333333332
561
+ name: Dot Recall@3
562
+ - type: dot_recall@5
563
+ value: 0.922
564
+ name: Dot Recall@5
565
+ - type: dot_recall@10
566
+ value: 0.9833333333333333
567
+ name: Dot Recall@10
568
+ - type: dot_ndcg@10
569
+ value: 0.9062363336812763
570
+ name: Dot Ndcg@10
571
+ - type: dot_mrr@10
572
+ value: 0.8922222222222222
573
+ name: Dot Mrr@10
574
+ - type: dot_map@100
575
+ value: 0.8721868432072424
576
+ name: Dot Map@100
577
+ - task:
578
+ type: sparse-information-retrieval
579
+ name: Sparse Information Retrieval
580
+ dataset:
581
+ name: NanoSCIDOCS
582
+ type: NanoSCIDOCS
583
+ metrics:
584
+ - type: dot_accuracy@1
585
+ value: 0.38
586
+ name: Dot Accuracy@1
587
+ - type: dot_accuracy@3
588
+ value: 0.56
589
+ name: Dot Accuracy@3
590
+ - type: dot_accuracy@5
591
+ value: 0.62
592
+ name: Dot Accuracy@5
593
+ - type: dot_accuracy@10
594
+ value: 0.76
595
+ name: Dot Accuracy@10
596
+ - type: dot_precision@1
597
+ value: 0.38
598
+ name: Dot Precision@1
599
+ - type: dot_precision@3
600
+ value: 0.24
601
+ name: Dot Precision@3
602
+ - type: dot_precision@5
603
+ value: 0.188
604
+ name: Dot Precision@5
605
+ - type: dot_precision@10
606
+ value: 0.148
607
+ name: Dot Precision@10
608
+ - type: dot_recall@1
609
+ value: 0.08066666666666666
610
+ name: Dot Recall@1
611
+ - type: dot_recall@3
612
+ value: 0.14966666666666667
613
+ name: Dot Recall@3
614
+ - type: dot_recall@5
615
+ value: 0.19466666666666665
616
+ name: Dot Recall@5
617
+ - type: dot_recall@10
618
+ value: 0.3036666666666667
619
+ name: Dot Recall@10
620
+ - type: dot_ndcg@10
621
+ value: 0.29127983049304745
622
+ name: Dot Ndcg@10
623
+ - type: dot_mrr@10
624
+ value: 0.4865
625
+ name: Dot Mrr@10
626
+ - type: dot_map@100
627
+ value: 0.21683445069332832
628
+ name: Dot Map@100
629
+ - task:
630
+ type: sparse-information-retrieval
631
+ name: Sparse Information Retrieval
632
+ dataset:
633
+ name: NanoArguAna
634
+ type: NanoArguAna
635
+ metrics:
636
+ - type: dot_accuracy@1
637
+ value: 0.1
638
+ name: Dot Accuracy@1
639
+ - type: dot_accuracy@3
640
+ value: 0.48
641
+ name: Dot Accuracy@3
642
+ - type: dot_accuracy@5
643
+ value: 0.58
644
+ name: Dot Accuracy@5
645
+ - type: dot_accuracy@10
646
+ value: 0.72
647
+ name: Dot Accuracy@10
648
+ - type: dot_precision@1
649
+ value: 0.1
650
+ name: Dot Precision@1
651
+ - type: dot_precision@3
652
+ value: 0.15999999999999998
653
+ name: Dot Precision@3
654
+ - type: dot_precision@5
655
+ value: 0.11599999999999999
656
+ name: Dot Precision@5
657
+ - type: dot_precision@10
658
+ value: 0.07200000000000001
659
+ name: Dot Precision@10
660
+ - type: dot_recall@1
661
+ value: 0.1
662
+ name: Dot Recall@1
663
+ - type: dot_recall@3
664
+ value: 0.48
665
+ name: Dot Recall@3
666
+ - type: dot_recall@5
667
+ value: 0.58
668
+ name: Dot Recall@5
669
+ - type: dot_recall@10
670
+ value: 0.72
671
+ name: Dot Recall@10
672
+ - type: dot_ndcg@10
673
+ value: 0.4187747413908095
674
+ name: Dot Ndcg@10
675
+ - type: dot_mrr@10
676
+ value: 0.32185714285714284
677
+ name: Dot Mrr@10
678
+ - type: dot_map@100
679
+ value: 0.3357998070888097
680
+ name: Dot Map@100
681
+ - task:
682
+ type: sparse-information-retrieval
683
+ name: Sparse Information Retrieval
684
+ dataset:
685
+ name: NanoSciFact
686
+ type: NanoSciFact
687
+ metrics:
688
+ - type: dot_accuracy@1
689
+ value: 0.54
690
+ name: Dot Accuracy@1
691
+ - type: dot_accuracy@3
692
+ value: 0.66
693
+ name: Dot Accuracy@3
694
+ - type: dot_accuracy@5
695
+ value: 0.72
696
+ name: Dot Accuracy@5
697
+ - type: dot_accuracy@10
698
+ value: 0.78
699
+ name: Dot Accuracy@10
700
+ - type: dot_precision@1
701
+ value: 0.54
702
+ name: Dot Precision@1
703
+ - type: dot_precision@3
704
+ value: 0.2333333333333333
705
+ name: Dot Precision@3
706
+ - type: dot_precision@5
707
+ value: 0.15600000000000003
708
+ name: Dot Precision@5
709
+ - type: dot_precision@10
710
+ value: 0.08799999999999997
711
+ name: Dot Precision@10
712
+ - type: dot_recall@1
713
+ value: 0.505
714
+ name: Dot Recall@1
715
+ - type: dot_recall@3
716
+ value: 0.635
717
+ name: Dot Recall@3
718
+ - type: dot_recall@5
719
+ value: 0.69
720
+ name: Dot Recall@5
721
+ - type: dot_recall@10
722
+ value: 0.77
723
+ name: Dot Recall@10
724
+ - type: dot_ndcg@10
725
+ value: 0.6442911439119196
726
+ name: Dot Ndcg@10
727
+ - type: dot_mrr@10
728
+ value: 0.616
729
+ name: Dot Mrr@10
730
+ - type: dot_map@100
731
+ value: 0.603482712383199
732
+ name: Dot Map@100
733
+ - task:
734
+ type: sparse-information-retrieval
735
+ name: Sparse Information Retrieval
736
+ dataset:
737
+ name: NanoTouche2020
738
+ type: NanoTouche2020
739
+ metrics:
740
+ - type: dot_accuracy@1
741
+ value: 0.6530612244897959
742
+ name: Dot Accuracy@1
743
+ - type: dot_accuracy@3
744
+ value: 0.8979591836734694
745
+ name: Dot Accuracy@3
746
+ - type: dot_accuracy@5
747
+ value: 0.8979591836734694
748
+ name: Dot Accuracy@5
749
+ - type: dot_accuracy@10
750
+ value: 0.9795918367346939
751
+ name: Dot Accuracy@10
752
+ - type: dot_precision@1
753
+ value: 0.6530612244897959
754
+ name: Dot Precision@1
755
+ - type: dot_precision@3
756
+ value: 0.6326530612244898
757
+ name: Dot Precision@3
758
+ - type: dot_precision@5
759
+ value: 0.5346938775510205
760
+ name: Dot Precision@5
761
+ - type: dot_precision@10
762
+ value: 0.46122448979591835
763
+ name: Dot Precision@10
764
+ - type: dot_recall@1
765
+ value: 0.045158988646388926
766
+ name: Dot Recall@1
767
+ - type: dot_recall@3
768
+ value: 0.13170444661082067
769
+ name: Dot Recall@3
770
+ - type: dot_recall@5
771
+ value: 0.1831114698285018
772
+ name: Dot Recall@5
773
+ - type: dot_recall@10
774
+ value: 0.29974279420697125
775
+ name: Dot Recall@10
776
+ - type: dot_ndcg@10
777
+ value: 0.5207054661668302
778
+ name: Dot Ndcg@10
779
+ - type: dot_mrr@10
780
+ value: 0.7654680919987042
781
+ name: Dot Mrr@10
782
+ - type: dot_map@100
783
+ value: 0.3842460294701173
784
+ name: Dot Map@100
785
+ - task:
786
+ type: sparse-nano-beir
787
+ name: Sparse Nano BEIR
788
+ dataset:
789
+ name: NanoBEIR mean
790
+ type: NanoBEIR_mean
791
+ metrics:
792
+ - type: dot_accuracy@1
793
+ value: 0.4640816326530612
794
+ name: Dot Accuracy@1
795
+ - type: dot_accuracy@3
796
+ value: 0.6613814756671901
797
+ name: Dot Accuracy@3
798
+ - type: dot_accuracy@5
799
+ value: 0.7152276295133438
800
+ name: Dot Accuracy@5
801
+ - type: dot_accuracy@10
802
+ value: 0.8061224489795917
803
+ name: Dot Accuracy@10
804
+ - type: dot_precision@1
805
+ value: 0.4640816326530612
806
+ name: Dot Precision@1
807
+ - type: dot_precision@3
808
+ value: 0.3014861329147044
809
+ name: Dot Precision@3
810
+ - type: dot_precision@5
811
+ value: 0.2300533751962324
812
+ name: Dot Precision@5
813
+ - type: dot_precision@10
814
+ value: 0.16301726844583989
815
+ name: Dot Precision@10
816
+ - type: dot_recall@1
817
+ value: 0.26457091365729607
818
+ name: Dot Recall@1
819
+ - type: dot_recall@3
820
+ value: 0.42731328952235625
821
+ name: Dot Recall@3
822
+ - type: dot_recall@5
823
+ value: 0.482784104144294
824
+ name: Dot Recall@5
825
+ - type: dot_recall@10
826
+ value: 0.5741392786311136
827
+ name: Dot Recall@10
828
+ - type: dot_ndcg@10
829
+ value: 0.5181963556604454
830
+ name: Dot Ndcg@10
831
+ - type: dot_mrr@10
832
+ value: 0.5752588397996561
833
+ name: Dot Mrr@10
834
+ - type: dot_map@100
835
+ value: 0.4399993853318952
836
+ name: Dot Map@100
837
+ ---
838
+
839
+ # splade-distilbert-base-uncased trained on GooAQ
840
+
841
+ This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model finetuned from [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) on the [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) dataset using the [sentence-transformers](https://www.SBERT.net) library. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
842
+
843
+ ## Model Details
844
+
845
+ ### Model Description
846
+ - **Model Type:** SPLADE Sparse Encoder
847
+ - **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased) <!-- at revision 12040accade4e8a0f71eabdb258fecc2e7e948be -->
848
+ - **Maximum Sequence Length:** 256 tokens
849
+ - **Output Dimensionality:** 30522 dimensions
850
+ - **Similarity Function:** Dot Product
851
+ - **Training Dataset:**
852
+ - [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq)
853
+ - **Language:** en
854
+ - **License:** apache-2.0
855
+
856
+ ### Model Sources
857
+
858
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
859
+ - **Documentation:** [Sparse Encoder Documentation](https://www.sbert.net/docs/sparse_encoder/usage/usage.html)
860
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
861
+ - **Hugging Face:** [Sparse Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=sparse-encoder)
862
+
863
+ ### Full Model Architecture
864
+
865
+ ```
866
+ SparseEncoder(
867
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
868
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
869
+ )
870
+ ```
871
+
872
+ ## Usage
873
+
874
+ ### Direct Usage (Sentence Transformers)
875
+
876
+ First install the Sentence Transformers library:
877
+
878
+ ```bash
879
+ pip install -U sentence-transformers
880
+ ```
881
+
882
+ Then you can load this model and run inference.
883
+ ```python
884
+ from sentence_transformers import SparseEncoder
885
+
886
+ # Download from the 🤗 Hub
887
+ model = SparseEncoder("tomaarsen/splade-distilbert-base-uncased-gooaq")
888
+ # Run inference
889
+ sentences = [
890
+ 'what is the difference between 18 and 20 inch tires?',
891
+ 'The only real difference is a 20" rim would be more likely to be damaged, as you pointed out. Beyond looks, there is zero benefit for the 20" rim. Also, just the availability of tires will likely be much more limited for the larger rim. ... Tire selection is better for 18" wheels than 20" wheels.',
892
+ 'So extracurricular activities are just activities that you do outside of class. The Common App says that extracurricular activities "include arts, athletics, clubs, employment, personal commitments, and other pursuits."',
893
+ ]
894
+ embeddings = model.encode(sentences)
895
+ print(embeddings.shape)
896
+ # (3, 30522)
897
+
898
+ # Get the similarity scores for the embeddings
899
+ similarities = model.similarity(embeddings, embeddings)
900
+ print(similarities.shape)
901
+ # [3, 3]
902
+ ```
903
+
904
+ <!--
905
+ ### Direct Usage (Transformers)
906
+
907
+ <details><summary>Click to see the direct usage in Transformers</summary>
908
+
909
+ </details>
910
+ -->
911
+
912
+ <!--
913
+ ### Downstream Usage (Sentence Transformers)
914
+
915
+ You can finetune this model on your own dataset.
916
+
917
+ <details><summary>Click to expand</summary>
918
+
919
+ </details>
920
+ -->
921
+
922
+ <!--
923
+ ### Out-of-Scope Use
924
+
925
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
926
+ -->
927
+
928
+ ## Evaluation
929
+
930
+ ### Metrics
931
+
932
+ #### Sparse Information Retrieval
933
+
934
+ * Datasets: `NanoClimateFEVER`, `NanoDBPedia`, `NanoFEVER`, `NanoFiQA2018`, `NanoHotpotQA`, `NanoMSMARCO`, `NanoNFCorpus`, `NanoNQ`, `NanoQuoraRetrieval`, `NanoSCIDOCS`, `NanoArguAna`, `NanoSciFact` and `NanoTouche2020`
935
+ * Evaluated with [<code>SparseInformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseInformationRetrievalEvaluator)
936
+
937
+ | Metric | NanoClimateFEVER | NanoDBPedia | NanoFEVER | NanoFiQA2018 | NanoHotpotQA | NanoMSMARCO | NanoNFCorpus | NanoNQ | NanoQuoraRetrieval | NanoSCIDOCS | NanoArguAna | NanoSciFact | NanoTouche2020 |
938
+ |:-----------------|:-----------------|:------------|:-----------|:-------------|:-------------|:------------|:-------------|:-----------|:-------------------|:------------|:------------|:------------|:---------------|
939
+ | dot_accuracy@1 | 0.22 | 0.52 | 0.68 | 0.34 | 0.74 | 0.26 | 0.4 | 0.38 | 0.82 | 0.38 | 0.1 | 0.54 | 0.6531 |
940
+ | dot_accuracy@3 | 0.38 | 0.68 | 0.9 | 0.6 | 0.9 | 0.42 | 0.56 | 0.6 | 0.96 | 0.56 | 0.48 | 0.66 | 0.898 |
941
+ | dot_accuracy@5 | 0.48 | 0.72 | 0.92 | 0.6 | 0.92 | 0.56 | 0.62 | 0.7 | 0.96 | 0.62 | 0.58 | 0.72 | 0.898 |
942
+ | dot_accuracy@10 | 0.54 | 0.86 | 0.94 | 0.68 | 0.96 | 0.78 | 0.66 | 0.82 | 1.0 | 0.76 | 0.72 | 0.78 | 0.9796 |
943
+ | dot_precision@1 | 0.22 | 0.52 | 0.68 | 0.34 | 0.74 | 0.26 | 0.4 | 0.38 | 0.82 | 0.38 | 0.1 | 0.54 | 0.6531 |
944
+ | dot_precision@3 | 0.1333 | 0.4267 | 0.3067 | 0.2733 | 0.4467 | 0.14 | 0.34 | 0.2133 | 0.3733 | 0.24 | 0.16 | 0.2333 | 0.6327 |
945
+ | dot_precision@5 | 0.108 | 0.396 | 0.188 | 0.192 | 0.3 | 0.112 | 0.316 | 0.148 | 0.236 | 0.188 | 0.116 | 0.156 | 0.5347 |
946
+ | dot_precision@10 | 0.066 | 0.36 | 0.1 | 0.12 | 0.158 | 0.078 | 0.25 | 0.086 | 0.132 | 0.148 | 0.072 | 0.088 | 0.4612 |
947
+ | dot_recall@1 | 0.1167 | 0.0417 | 0.6567 | 0.1459 | 0.37 | 0.26 | 0.0237 | 0.36 | 0.734 | 0.0807 | 0.1 | 0.505 | 0.0452 |
948
+ | dot_recall@3 | 0.1707 | 0.1169 | 0.8567 | 0.3665 | 0.67 | 0.42 | 0.0767 | 0.57 | 0.9113 | 0.1497 | 0.48 | 0.635 | 0.1317 |
949
+ | dot_recall@5 | 0.2157 | 0.1515 | 0.8767 | 0.3961 | 0.75 | 0.56 | 0.0965 | 0.66 | 0.922 | 0.1947 | 0.58 | 0.69 | 0.1831 |
950
+ | dot_recall@10 | 0.254 | 0.2449 | 0.9167 | 0.5118 | 0.79 | 0.78 | 0.1197 | 0.77 | 0.9833 | 0.3037 | 0.72 | 0.77 | 0.2997 |
951
+ | **dot_ndcg@10** | **0.2262** | **0.4348** | **0.8076** | **0.403** | **0.7355** | **0.4857** | **0.3007** | **0.5619** | **0.9062** | **0.2913** | **0.4188** | **0.6443** | **0.5207** |
952
+ | dot_mrr@10 | 0.3172 | 0.6113 | 0.7872 | 0.4699 | 0.8237 | 0.3958 | 0.4807 | 0.5104 | 0.8922 | 0.4865 | 0.3219 | 0.616 | 0.7655 |
953
+ | dot_map@100 | 0.1814 | 0.3308 | 0.7674 | 0.3255 | 0.6711 | 0.4043 | 0.13 | 0.4968 | 0.8722 | 0.2168 | 0.3358 | 0.6035 | 0.3842 |
954
+
955
+ #### Sparse Nano BEIR
956
+
957
+ * Dataset: `NanoBEIR_mean`
958
+ * Evaluated with [<code>SparseNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sparse_encoder/evaluation.html#sentence_transformers.sparse_encoder.evaluation.SparseNanoBEIREvaluator) with these parameters:
959
+ ```json
960
+ {
961
+ "dataset_names": [
962
+ "climatefever",
963
+ "dbpedia",
964
+ "fever",
965
+ "fiqa2018",
966
+ "hotpotqa",
967
+ "msmarco",
968
+ "nfcorpus",
969
+ "nq",
970
+ "quoraretrieval",
971
+ "scidocs",
972
+ "arguana",
973
+ "scifact",
974
+ "touche2020"
975
+ ]
976
+ }
977
+ ```
978
+
979
+ | Metric | Value |
980
+ |:-----------------|:-----------|
981
+ | dot_accuracy@1 | 0.4641 |
982
+ | dot_accuracy@3 | 0.6614 |
983
+ | dot_accuracy@5 | 0.7152 |
984
+ | dot_accuracy@10 | 0.8061 |
985
+ | dot_precision@1 | 0.4641 |
986
+ | dot_precision@3 | 0.3015 |
987
+ | dot_precision@5 | 0.2301 |
988
+ | dot_precision@10 | 0.163 |
989
+ | dot_recall@1 | 0.2646 |
990
+ | dot_recall@3 | 0.4273 |
991
+ | dot_recall@5 | 0.4828 |
992
+ | dot_recall@10 | 0.5741 |
993
+ | **dot_ndcg@10** | **0.5182** |
994
+ | dot_mrr@10 | 0.5753 |
995
+ | dot_map@100 | 0.44 |
996
+
997
+ <!--
998
+ ## Bias, Risks and Limitations
999
+
1000
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
1001
+ -->
1002
+
1003
+ <!--
1004
+ ### Recommendations
1005
+
1006
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
1007
+ -->
1008
+
1009
+ ## Training Details
1010
+
1011
+ ### Training Dataset
1012
+
1013
+ #### gooaq
1014
+
1015
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1016
+ * Size: 3,011,496 training samples
1017
+ * Columns: <code>question</code> and <code>answer</code>
1018
+ * Approximate statistics based on the first 1000 samples:
1019
+ | | question | answer |
1020
+ |:--------|:-----------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|
1021
+ | type | string | string |
1022
+ | details | <ul><li>min: 18 characters</li><li>mean: 43.42 characters</li><li>max: 96 characters</li></ul> | <ul><li>min: 54 characters</li><li>mean: 252.96 characters</li><li>max: 426 characters</li></ul> |
1023
+ * Samples:
1024
+ | question | answer |
1025
+ |:-----------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1026
+ | <code>what is the difference between clay and mud mask?</code> | <code>The main difference between the two is that mud is a skin-healing agent, while clay is a cosmetic, drying agent. Clay masks are most useful for someone who has oily skin and is prone to breakouts of acne and blemishes.</code> |
1027
+ | <code>myki how much on card?</code> | <code>A full fare myki card costs $6 and a concession, seniors or child myki costs $3. For more information about how to use your myki, visit ptv.vic.gov.au or call 1800 800 007.</code> |
1028
+ | <code>how to find out if someone blocked your phone number on iphone?</code> | <code>If you get a notification like "Message Not Delivered" or you get no notification at all, that's a sign of a potential block. Next, you could try calling the person. If the call goes right to voicemail or rings once (or a half ring) then goes to voicemail, that's further evidence you may have been blocked.</code> |
1029
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1030
+ ```json
1031
+ {'loss': SparseMultipleNegativesRankingLoss(
1032
+ (model): SparseEncoder(
1033
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
1034
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1035
+ )
1036
+ (cross_entropy_loss): CrossEntropyLoss()
1037
+ ), 'lambda_corpus': 3e-05, 'lambda_query': 5e-05, 'corpus_regularizer': FlopsLoss(
1038
+ (model): SparseEncoder(
1039
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
1040
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1041
+ )
1042
+ ), 'query_regularizer': FlopsLoss(
1043
+ (model): SparseEncoder(
1044
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
1045
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1046
+ )
1047
+ )}
1048
+ ```
1049
+
1050
+ ### Evaluation Dataset
1051
+
1052
+ #### gooaq
1053
+
1054
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
1055
+ * Size: 1,000 evaluation samples
1056
+ * Columns: <code>question</code> and <code>answer</code>
1057
+ * Approximate statistics based on the first 1000 samples:
1058
+ | | question | answer |
1059
+ |:--------|:-----------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|
1060
+ | type | string | string |
1061
+ | details | <ul><li>min: 18 characters</li><li>mean: 43.17 characters</li><li>max: 98 characters</li></ul> | <ul><li>min: 51 characters</li><li>mean: 254.12 characters</li><li>max: 360 characters</li></ul> |
1062
+ * Samples:
1063
+ | question | answer |
1064
+ |:-----------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
1065
+ | <code>how do i program my directv remote with my tv?</code> | <code>['Press MENU on your remote.', 'Select Settings & Help > Settings > Remote Control > Program Remote.', 'Choose the device (TV, audio, DVD) you wish to program. ... ', 'Follow the on-screen prompts to complete programming.']</code> |
1066
+ | <code>are rodrigues fruit bats nocturnal?</code> | <code>Before its numbers were threatened by habitat destruction, storms, and hunting, some of those groups could number 500 or more members. Sunrise, sunset. Rodrigues fruit bats are most active at dawn, at dusk, and at night.</code> |
1067
+ | <code>why does your heart rate increase during exercise bbc bitesize?</code> | <code>During exercise there is an increase in physical activity and muscle cells respire more than they do when the body is at rest. The heart rate increases during exercise. The rate and depth of breathing increases - this makes sure that more oxygen is absorbed into the blood, and more carbon dioxide is removed from it.</code> |
1068
+ * Loss: [<code>SpladeLoss</code>](https://sbert.net/docs/package_reference/sparse_encoder/losses.html#spladeloss) with these parameters:
1069
+ ```json
1070
+ {'loss': SparseMultipleNegativesRankingLoss(
1071
+ (model): SparseEncoder(
1072
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
1073
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1074
+ )
1075
+ (cross_entropy_loss): CrossEntropyLoss()
1076
+ ), 'lambda_corpus': 3e-05, 'lambda_query': 5e-05, 'corpus_regularizer': FlopsLoss(
1077
+ (model): SparseEncoder(
1078
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
1079
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1080
+ )
1081
+ ), 'query_regularizer': FlopsLoss(
1082
+ (model): SparseEncoder(
1083
+ (0): MLMTransformer({'max_seq_length': 256, 'do_lower_case': False}) with MLMTransformer model: DistilBertForMaskedLM
1084
+ (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': None})
1085
+ )
1086
+ )}
1087
+ ```
1088
+
1089
+ ### Training Hyperparameters
1090
+ #### Non-Default Hyperparameters
1091
+
1092
+ - `eval_strategy`: steps
1093
+ - `per_device_train_batch_size`: 32
1094
+ - `per_device_eval_batch_size`: 32
1095
+ - `learning_rate`: 2e-05
1096
+ - `num_train_epochs`: 1
1097
+ - `bf16`: True
1098
+ - `load_best_model_at_end`: True
1099
+ - `batch_sampler`: no_duplicates
1100
+
1101
+ #### All Hyperparameters
1102
+ <details><summary>Click to expand</summary>
1103
+
1104
+ - `overwrite_output_dir`: False
1105
+ - `do_predict`: False
1106
+ - `eval_strategy`: steps
1107
+ - `prediction_loss_only`: True
1108
+ - `per_device_train_batch_size`: 32
1109
+ - `per_device_eval_batch_size`: 32
1110
+ - `per_gpu_train_batch_size`: None
1111
+ - `per_gpu_eval_batch_size`: None
1112
+ - `gradient_accumulation_steps`: 1
1113
+ - `eval_accumulation_steps`: None
1114
+ - `torch_empty_cache_steps`: None
1115
+ - `learning_rate`: 2e-05
1116
+ - `weight_decay`: 0.0
1117
+ - `adam_beta1`: 0.9
1118
+ - `adam_beta2`: 0.999
1119
+ - `adam_epsilon`: 1e-08
1120
+ - `max_grad_norm`: 1.0
1121
+ - `num_train_epochs`: 1
1122
+ - `max_steps`: -1
1123
+ - `lr_scheduler_type`: linear
1124
+ - `lr_scheduler_kwargs`: {}
1125
+ - `warmup_ratio`: 0.0
1126
+ - `warmup_steps`: 0
1127
+ - `log_level`: passive
1128
+ - `log_level_replica`: warning
1129
+ - `log_on_each_node`: True
1130
+ - `logging_nan_inf_filter`: True
1131
+ - `save_safetensors`: True
1132
+ - `save_on_each_node`: False
1133
+ - `save_only_model`: False
1134
+ - `restore_callback_states_from_checkpoint`: False
1135
+ - `no_cuda`: False
1136
+ - `use_cpu`: False
1137
+ - `use_mps_device`: False
1138
+ - `seed`: 42
1139
+ - `data_seed`: None
1140
+ - `jit_mode_eval`: False
1141
+ - `use_ipex`: False
1142
+ - `bf16`: True
1143
+ - `fp16`: False
1144
+ - `fp16_opt_level`: O1
1145
+ - `half_precision_backend`: auto
1146
+ - `bf16_full_eval`: False
1147
+ - `fp16_full_eval`: False
1148
+ - `tf32`: None
1149
+ - `local_rank`: 0
1150
+ - `ddp_backend`: None
1151
+ - `tpu_num_cores`: None
1152
+ - `tpu_metrics_debug`: False
1153
+ - `debug`: []
1154
+ - `dataloader_drop_last`: False
1155
+ - `dataloader_num_workers`: 0
1156
+ - `dataloader_prefetch_factor`: None
1157
+ - `past_index`: -1
1158
+ - `disable_tqdm`: False
1159
+ - `remove_unused_columns`: True
1160
+ - `label_names`: None
1161
+ - `load_best_model_at_end`: True
1162
+ - `ignore_data_skip`: False
1163
+ - `fsdp`: []
1164
+ - `fsdp_min_num_params`: 0
1165
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
1166
+ - `fsdp_transformer_layer_cls_to_wrap`: None
1167
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
1168
+ - `deepspeed`: None
1169
+ - `label_smoothing_factor`: 0.0
1170
+ - `optim`: adamw_torch
1171
+ - `optim_args`: None
1172
+ - `adafactor`: False
1173
+ - `group_by_length`: False
1174
+ - `length_column_name`: length
1175
+ - `ddp_find_unused_parameters`: None
1176
+ - `ddp_bucket_cap_mb`: None
1177
+ - `ddp_broadcast_buffers`: False
1178
+ - `dataloader_pin_memory`: True
1179
+ - `dataloader_persistent_workers`: False
1180
+ - `skip_memory_metrics`: True
1181
+ - `use_legacy_prediction_loop`: False
1182
+ - `push_to_hub`: False
1183
+ - `resume_from_checkpoint`: None
1184
+ - `hub_model_id`: None
1185
+ - `hub_strategy`: every_save
1186
+ - `hub_private_repo`: None
1187
+ - `hub_always_push`: False
1188
+ - `gradient_checkpointing`: False
1189
+ - `gradient_checkpointing_kwargs`: None
1190
+ - `include_inputs_for_metrics`: False
1191
+ - `include_for_metrics`: []
1192
+ - `eval_do_concat_batches`: True
1193
+ - `fp16_backend`: auto
1194
+ - `push_to_hub_model_id`: None
1195
+ - `push_to_hub_organization`: None
1196
+ - `mp_parameters`:
1197
+ - `auto_find_batch_size`: False
1198
+ - `full_determinism`: False
1199
+ - `torchdynamo`: None
1200
+ - `ray_scope`: last
1201
+ - `ddp_timeout`: 1800
1202
+ - `torch_compile`: False
1203
+ - `torch_compile_backend`: None
1204
+ - `torch_compile_mode`: None
1205
+ - `dispatch_batches`: None
1206
+ - `split_batches`: None
1207
+ - `include_tokens_per_second`: False
1208
+ - `include_num_input_tokens_seen`: False
1209
+ - `neftune_noise_alpha`: None
1210
+ - `optim_target_modules`: None
1211
+ - `batch_eval_metrics`: False
1212
+ - `eval_on_start`: False
1213
+ - `use_liger_kernel`: False
1214
+ - `eval_use_gather_object`: False
1215
+ - `average_tokens_across_devices`: False
1216
+ - `prompts`: None
1217
+ - `batch_sampler`: no_duplicates
1218
+ - `multi_dataset_batch_sampler`: proportional
1219
+
1220
+ </details>
1221
+
1222
+ ### Training Logs
1223
+ | Epoch | Step | Training Loss | Validation Loss | NanoClimateFEVER_dot_ndcg@10 | NanoDBPedia_dot_ndcg@10 | NanoFEVER_dot_ndcg@10 | NanoFiQA2018_dot_ndcg@10 | NanoHotpotQA_dot_ndcg@10 | NanoMSMARCO_dot_ndcg@10 | NanoNFCorpus_dot_ndcg@10 | NanoNQ_dot_ndcg@10 | NanoQuoraRetrieval_dot_ndcg@10 | NanoSCIDOCS_dot_ndcg@10 | NanoArguAna_dot_ndcg@10 | NanoSciFact_dot_ndcg@10 | NanoTouche2020_dot_ndcg@10 | NanoBEIR_mean_dot_ndcg@10 |
1224
+ |:----------:|:---------:|:-------------:|:---------------:|:----------------------------:|:-----------------------:|:---------------------:|:------------------------:|:------------------------:|:-----------------------:|:------------------------:|:------------------:|:------------------------------:|:-----------------------:|:-----------------------:|:-----------------------:|:--------------------------:|:-------------------------:|
1225
+ | 0.0213 | 2000 | 0.75 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1226
+ | 0.0425 | 4000 | 0.071 | 0.0924 | 0.1931 | 0.2903 | 0.5966 | 0.3079 | 0.6182 | 0.3378 | 0.1867 | 0.3781 | 0.3784 | 0.1966 | 0.2325 | 0.4148 | 0.5139 | 0.3573 |
1227
+ | 0.0638 | 6000 | 0.0578 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1228
+ | 0.0850 | 8000 | 0.0511 | 0.0589 | 0.1826 | 0.2911 | 0.5719 | 0.3820 | 0.6818 | 0.2417 | 0.2032 | 0.2925 | 0.4541 | 0.2090 | 0.2306 | 0.5240 | 0.5183 | 0.3679 |
1229
+ | 0.1063 | 10000 | 0.0464 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1230
+ | 0.1275 | 12000 | 0.0458 | 0.0795 | 0.1978 | 0.2958 | 0.6206 | 0.3664 | 0.6673 | 0.2691 | 0.1872 | 0.2327 | 0.6770 | 0.2008 | 0.3288 | 0.5384 | 0.5017 | 0.3911 |
1231
+ | 0.1488 | 14000 | 0.0427 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1232
+ | 0.1700 | 16000 | 0.0392 | 0.0581 | 0.2785 | 0.4104 | 0.8125 | 0.3832 | 0.7265 | 0.5093 | 0.2688 | 0.6075 | 0.7879 | 0.2760 | 0.3342 | 0.5722 | 0.5301 | 0.4998 |
1233
+ | 0.1913 | 18000 | 0.039 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1234
+ | 0.2125 | 20000 | 0.0366 | 0.0472 | 0.2319 | 0.3466 | 0.7349 | 0.3774 | 0.7174 | 0.4061 | 0.2189 | 0.4166 | 0.7486 | 0.2364 | 0.3560 | 0.5907 | 0.5211 | 0.4541 |
1235
+ | 0.2338 | 22000 | 0.0312 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1236
+ | 0.2550 | 24000 | 0.0322 | 0.0543 | 0.2169 | 0.4469 | 0.7618 | 0.4014 | 0.6831 | 0.4412 | 0.2707 | 0.5253 | 0.8104 | 0.2621 | 0.3581 | 0.6006 | 0.5037 | 0.4832 |
1237
+ | 0.2763 | 26000 | 0.0292 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1238
+ | 0.2975 | 28000 | 0.03 | 0.0529 | 0.2257 | 0.5070 | 0.7976 | 0.4014 | 0.7442 | 0.5165 | 0.3216 | 0.5799 | 0.8483 | 0.3318 | 0.3206 | 0.5665 | 0.5149 | 0.5135 |
1239
+ | 0.3188 | 30000 | 0.0294 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1240
+ | 0.3400 | 32000 | 0.0291 | 0.0512 | 0.1754 | 0.4539 | 0.8196 | 0.3903 | 0.7372 | 0.4689 | 0.2948 | 0.5548 | 0.8643 | 0.2791 | 0.4040 | 0.5229 | 0.5055 | 0.4977 |
1241
+ | 0.3613 | 34000 | 0.0284 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1242
+ | 0.3825 | 36000 | 0.0268 | 0.0404 | 0.2566 | 0.4462 | 0.8142 | 0.3737 | 0.7281 | 0.4418 | 0.2568 | 0.5135 | 0.8305 | 0.2749 | 0.3775 | 0.5485 | 0.5228 | 0.4912 |
1243
+ | 0.4038 | 38000 | 0.0262 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1244
+ | 0.4250 | 40000 | 0.0238 | 0.0416 | 0.2464 | 0.5235 | 0.8004 | 0.4016 | 0.7418 | 0.4483 | 0.2915 | 0.5771 | 0.8538 | 0.2523 | 0.3536 | 0.6227 | 0.4967 | 0.5084 |
1245
+ | 0.4463 | 42000 | 0.0253 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1246
+ | 0.4675 | 44000 | 0.0224 | 0.0360 | 0.2080 | 0.5100 | 0.8317 | 0.3775 | 0.7223 | 0.4447 | 0.2789 | 0.5586 | 0.8324 | 0.3151 | 0.4005 | 0.6089 | 0.5119 | 0.5077 |
1247
+ | 0.4888 | 46000 | 0.0225 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1248
+ | 0.5100 | 48000 | 0.0209 | 0.0232 | 0.2386 | 0.5045 | 0.8204 | 0.3746 | 0.7390 | 0.4662 | 0.2963 | 0.5380 | 0.8580 | 0.3292 | 0.4010 | 0.6336 | 0.5214 | 0.5170 |
1249
+ | 0.5313 | 50000 | 0.0225 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1250
+ | 0.5525 | 52000 | 0.0205 | 0.0380 | 0.2237 | 0.5114 | 0.7952 | 0.3583 | 0.6979 | 0.4310 | 0.2816 | 0.5364 | 0.8747 | 0.2703 | 0.4009 | 0.5947 | 0.5038 | 0.4984 |
1251
+ | 0.5738 | 54000 | 0.0204 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1252
+ | 0.5950 | 56000 | 0.0204 | 0.0365 | 0.2316 | 0.4676 | 0.8134 | 0.3754 | 0.7280 | 0.4536 | 0.2927 | 0.5205 | 0.8662 | 0.2859 | 0.3589 | 0.6281 | 0.5069 | 0.5022 |
1253
+ | 0.6163 | 58000 | 0.0199 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1254
+ | 0.6376 | 60000 | 0.0196 | 0.0365 | 0.2233 | 0.4897 | 0.8149 | 0.3385 | 0.7395 | 0.4778 | 0.2725 | 0.5365 | 0.8610 | 0.2836 | 0.4031 | 0.5380 | 0.5146 | 0.4995 |
1255
+ | 0.6588 | 62000 | 0.0187 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1256
+ | 0.6801 | 64000 | 0.0184 | 0.0453 | 0.2333 | 0.4792 | 0.7881 | 0.3653 | 0.7402 | 0.5062 | 0.3008 | 0.5607 | 0.8922 | 0.2857 | 0.4039 | 0.5972 | 0.5217 | 0.5134 |
1257
+ | 0.7013 | 66000 | 0.0182 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1258
+ | 0.7226 | 68000 | 0.0162 | 0.0323 | 0.2341 | 0.4678 | 0.8283 | 0.3855 | 0.7567 | 0.5229 | 0.3297 | 0.5445 | 0.8909 | 0.2787 | 0.3917 | 0.5904 | 0.5115 | 0.5179 |
1259
+ | 0.7438 | 70000 | 0.0195 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1260
+ | 0.7651 | 72000 | 0.0181 | 0.0243 | 0.2082 | 0.4374 | 0.7487 | 0.4010 | 0.7245 | 0.4712 | 0.3179 | 0.5168 | 0.8721 | 0.2794 | 0.4312 | 0.5801 | 0.5129 | 0.5001 |
1261
+ | 0.7863 | 74000 | 0.0171 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1262
+ | 0.8076 | 76000 | 0.0164 | 0.0284 | 0.2153 | 0.4654 | 0.7985 | 0.4027 | 0.7528 | 0.4871 | 0.3267 | 0.5385 | 0.9092 | 0.2997 | 0.3852 | 0.5979 | 0.5001 | 0.5138 |
1263
+ | 0.8288 | 78000 | 0.0169 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1264
+ | 0.8501 | 80000 | 0.0168 | 0.0244 | 0.2032 | 0.4466 | 0.7855 | 0.4042 | 0.7396 | 0.4971 | 0.2946 | 0.5485 | 0.9071 | 0.2983 | 0.3919 | 0.5862 | 0.5149 | 0.5091 |
1265
+ | 0.8713 | 82000 | 0.0144 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1266
+ | **0.8926** | **84000** | **0.0155** | **0.0229** | **0.2262** | **0.4348** | **0.8076** | **0.403** | **0.7355** | **0.4857** | **0.3007** | **0.5619** | **0.9062** | **0.2913** | **0.4188** | **0.6443** | **0.5207** | **0.5182** |
1267
+ | 0.9138 | 86000 | 0.0144 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1268
+ | 0.9351 | 88000 | 0.0149 | 0.0211 | 0.2259 | 0.4212 | 0.8098 | 0.3938 | 0.7309 | 0.4665 | 0.3051 | 0.5301 | 0.9061 | 0.2881 | 0.4086 | 0.6390 | 0.5199 | 0.5111 |
1269
+ | 0.9563 | 90000 | 0.013 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1270
+ | 0.9776 | 92000 | 0.0143 | 0.0231 | 0.2288 | 0.4224 | 0.8176 | 0.4130 | 0.7332 | 0.4807 | 0.3033 | 0.5424 | 0.9007 | 0.2772 | 0.4215 | 0.6354 | 0.5170 | 0.5149 |
1271
+ | 0.9988 | 94000 | 0.0129 | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |
1272
+ | -1 | -1 | - | - | 0.2262 | 0.4348 | 0.8076 | 0.4030 | 0.7355 | 0.4857 | 0.3007 | 0.5619 | 0.9062 | 0.2913 | 0.4188 | 0.6443 | 0.5207 | 0.5182 |
1273
+
1274
+ * The bold row denotes the saved checkpoint.
1275
+
1276
+ ### Environmental Impact
1277
+ Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
1278
+ - **Energy Consumed**: 1.338 kWh
1279
+ - **Carbon Emitted**: 0.520 kg of CO2
1280
+ - **Hours Used**: 3.894 hours
1281
+
1282
+ ### Training Hardware
1283
+ - **On Cloud**: No
1284
+ - **GPU Model**: 1 x NVIDIA GeForce RTX 3090
1285
+ - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
1286
+ - **RAM Size**: 31.78 GB
1287
+
1288
+ ### Framework Versions
1289
+ - Python: 3.11.6
1290
+ - Sentence Transformers: 4.2.0.dev0
1291
+ - Transformers: 4.49.0
1292
+ - PyTorch: 2.6.0+cu124
1293
+ - Accelerate: 1.5.1
1294
+ - Datasets: 2.21.0
1295
+ - Tokenizers: 0.21.1
1296
+
1297
+ ## Citation
1298
+
1299
+ ### BibTeX
1300
+
1301
+ #### Sentence Transformers
1302
+ ```bibtex
1303
+ @inproceedings{reimers-2019-sentence-bert,
1304
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
1305
+ author = "Reimers, Nils and Gurevych, Iryna",
1306
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
1307
+ month = "11",
1308
+ year = "2019",
1309
+ publisher = "Association for Computational Linguistics",
1310
+ url = "https://arxiv.org/abs/1908.10084",
1311
+ }
1312
+ ```
1313
+
1314
+ #### SpladeLoss
1315
+ ```bibtex
1316
+ @misc{formal2022distillationhardnegativesampling,
1317
+ title={From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective},
1318
+ author={Thibault Formal and Carlos Lassance and Benjamin Piwowarski and Stéphane Clinchant},
1319
+ year={2022},
1320
+ eprint={2205.04733},
1321
+ archivePrefix={arXiv},
1322
+ primaryClass={cs.IR},
1323
+ url={https://arxiv.org/abs/2205.04733},
1324
+ }
1325
+ ```
1326
+
1327
+ <!--
1328
+ ## Glossary
1329
+
1330
+ *Clearly define terms in order to be accessible across audiences.*
1331
+ -->
1332
+
1333
+ <!--
1334
+ ## Model Card Authors
1335
+
1336
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
1337
+ -->
1338
+
1339
+ <!--
1340
+ ## Model Card Contact
1341
+
1342
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
1343
+ -->
config.json ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "distilbert/distilbert-base-uncased",
3
+ "activation": "gelu",
4
+ "architectures": [
5
+ "DistilBertForMaskedLM"
6
+ ],
7
+ "attention_dropout": 0.1,
8
+ "dim": 768,
9
+ "dropout": 0.1,
10
+ "hidden_dim": 3072,
11
+ "initializer_range": 0.02,
12
+ "max_position_embeddings": 512,
13
+ "model_type": "distilbert",
14
+ "n_heads": 12,
15
+ "n_layers": 6,
16
+ "pad_token_id": 0,
17
+ "qa_dropout": 0.1,
18
+ "seq_classif_dropout": 0.2,
19
+ "sinusoidal_pos_embds": false,
20
+ "tie_weights_": true,
21
+ "torch_dtype": "float32",
22
+ "transformers_version": "4.49.0",
23
+ "vocab_size": 30522
24
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,11 @@
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "SparseEncoder",
3
+ "__version__": {
4
+ "sentence_transformers": "4.2.0.dev0",
5
+ "transformers": "4.49.0",
6
+ "pytorch": "2.6.0+cu124"
7
+ },
8
+ "prompts": {},
9
+ "default_prompt_name": null,
10
+ "similarity_fn_name": "dot"
11
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4e2d16e3eab1288cdb346a5986b6f4935802ff23abaf75182c67d7c9d1a1a04c
3
+ size 267954768
modules.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_SpladePooling",
12
+ "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
13
+ }
14
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 256,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": "[CLS]",
3
+ "mask_token": "[MASK]",
4
+ "pad_token": "[PAD]",
5
+ "sep_token": "[SEP]",
6
+ "unk_token": "[UNK]"
7
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "100": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "101": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "102": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "103": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": false,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": true,
47
+ "extra_special_tokens": {},
48
+ "mask_token": "[MASK]",
49
+ "model_max_length": 512,
50
+ "pad_token": "[PAD]",
51
+ "sep_token": "[SEP]",
52
+ "strip_accents": null,
53
+ "tokenize_chinese_chars": true,
54
+ "tokenizer_class": "DistilBertTokenizer",
55
+ "unk_token": "[UNK]"
56
+ }
vocab.txt ADDED
The diff for this file is too large to render. See raw diff