tomaarsen HF Staff commited on
Commit
2c1de37
·
verified ·
1 Parent(s): 1f3defc

Add new SentenceTransformer model

Browse files
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ tokenizer.json filter=lfs diff=lfs merge=lfs -text
1_Pooling/config.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "word_embedding_dimension": 1024,
3
+ "pooling_mode_cls_token": false,
4
+ "pooling_mode_mean_tokens": false,
5
+ "pooling_mode_max_tokens": false,
6
+ "pooling_mode_mean_sqrt_len_tokens": false,
7
+ "pooling_mode_weightedmean_tokens": false,
8
+ "pooling_mode_lasttoken": true,
9
+ "include_prompt": true
10
+ }
README.md ADDED
@@ -0,0 +1,856 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ tags:
5
+ - sentence-transformers
6
+ - sentence-similarity
7
+ - feature-extraction
8
+ - dense
9
+ - generated_from_trainer
10
+ - dataset_size:197462
11
+ - loss:MSELoss
12
+ base_model: Qwen/Qwen3-Embedding-0.6B
13
+ widget:
14
+ - source_sentence: 'Instruct: Given a web search query, retrieve relevant passages
15
+ that answer the query
16
+
17
+ Query:who sings the song i don''t want to work'
18
+ sentences:
19
+ - The Invisible Man Griffin is the surname of the story's protagonist. His name
20
+ is not mentioned until about halfway through the book. Consumed with his greed
21
+ for power and fame, he is the model of science without humanity. A gifted young
22
+ student, he becomes interested in the science of refraction. During his experiments,
23
+ he accidentally discovers chemicals (combined with an unspecified kind of radiation)
24
+ that would make living tissue invisible. Obsessed with his discovery, he tries
25
+ the experiment on himself and becomes invisible. However, he does not know how
26
+ to reverse the process, and he slowly discovers that the advantages of being invisible
27
+ do not outweigh the disadvantages and the problems he faces. Thus begins his downfall
28
+ as he takes the road to crime for his survival, revealing in the process his lack
29
+ of conscience, inhumanity and complete selfishness. He progresses from obsession
30
+ to fanaticism, to insanity, and finally to his fateful end.
31
+ - 'Instruct: Given a web search query, retrieve relevant passages that answer the
32
+ query
33
+
34
+ Query:who did the united states become independent from'
35
+ - Jordan Belfort Jordan Ross Belfort (/ˈbɛlfɔːrt/; born July 9, 1962) is an American
36
+ author, motivational speaker, and former stockbroker. In 1999, he pleaded guilty
37
+ to fraud and related crimes in connection with stock-market manipulation and running
38
+ a boiler room as part of a penny-stock scam. Belfort spent 22 months in prison
39
+ as part of an agreement under which he gave testimony against numerous partners
40
+ and subordinates in his fraud scheme.[5] He published the memoir The Wolf of Wall
41
+ Street, which was adapted into a film and released in 2013.
42
+ - source_sentence: London water supply infrastructure Most of London's water comes
43
+ from non-tidal parts of the Thames and Lea, with the remainder being abstracted
44
+ from underground sources.[22]
45
+ sentences:
46
+ - 'Instruct: Given a web search query, retrieve relevant passages that answer the
47
+ query
48
+
49
+ Query:what is the number on the hogwarts express'
50
+ - 'Instruct: Given a web search query, retrieve relevant passages that answer the
51
+ query
52
+
53
+ Query:when did roughing the kicker become a rule'
54
+ - Agora Early in Greek history (18th century–8th century BC), free-born citizens
55
+ would gather in the agora for military duty or to hear statements of the ruling
56
+ king or council. Later, the Agora also served as a marketplace where merchants
57
+ kept stalls or shops to sell their goods amid colonnades. This attracted artisans
58
+ who built workshops nearby.[2]
59
+ - source_sentence: 'Instruct: Given a web search query, retrieve relevant passages
60
+ that answer the query
61
+
62
+ Query:what is meant by lagging and leading current in ac circuit'
63
+ sentences:
64
+ - .org The domain name org is a generic top-level domain (gTLD) of the Domain Name
65
+ System (DNS) used in the Internet. The name is truncated from organization. It
66
+ was one of the original domains established in 1985, and has been operated by
67
+ the Public Interest Registry since 2003. The domain was originally intended for
68
+ non-profit entities, but this restriction was not enforced and has been removed.
69
+ The domain is commonly used by schools, open-source projects, and communities,
70
+ but also by some for-profit entities. The number of registered domains in org
71
+ has increased from fewer than one million in the 1990s, to ten million as of June
72
+ 2013.
73
+ - 'Instruct: Given a web search query, retrieve relevant passages that answer the
74
+ query
75
+
76
+ Query:how many episode in season 1 game of thrones'
77
+ - 'Instruct: Given a web search query, retrieve relevant passages that answer the
78
+ query
79
+
80
+ Query:when is season 11 of doctor who coming out'
81
+ - source_sentence: Gabriel Vlad (born April 9, 1969) in Bucharest, is a former Romanian
82
+ former rugby union football player.
83
+ sentences:
84
+ - As of May 2013, The Jewish Tribune had a circulation of 60,500 copies a week which
85
+ made it, for a time, the largest Jewish weekly publication in Canada.
86
+ - Cunjamba Dima is a city and commune of Angola, located in the province of Cuando
87
+ Cubango.
88
+ - He also acted in the National award winning Tamil movie Vazhakku Enn 18/9, directed
89
+ by Balaji Sakthivel.
90
+ - source_sentence: The actress was thirteen when she was offered the role of Annie.
91
+ sentences:
92
+ - All profits from the sale and streaming of the song go to music education supported
93
+ by the CMA Foundation.
94
+ - Narsingh Temple is situated at the across of the village just across confluence
95
+ of Magri State village.
96
+ - Contrasting significantly from other soccer leagues in the U.S., WLS intends to
97
+ be an open entry, promotion and relegation competition.
98
+ datasets:
99
+ - sentence-transformers/natural-questions
100
+ - sentence-transformers/gooaq
101
+ - sentence-transformers/wikipedia-en-sentences
102
+ pipeline_tag: sentence-similarity
103
+ library_name: sentence-transformers
104
+ metrics:
105
+ - cosine_accuracy@1
106
+ - cosine_accuracy@3
107
+ - cosine_accuracy@5
108
+ - cosine_accuracy@10
109
+ - cosine_precision@1
110
+ - cosine_precision@3
111
+ - cosine_precision@5
112
+ - cosine_precision@10
113
+ - cosine_recall@1
114
+ - cosine_recall@3
115
+ - cosine_recall@5
116
+ - cosine_recall@10
117
+ - cosine_ndcg@10
118
+ - cosine_mrr@10
119
+ - cosine_map@100
120
+ - negative_mse
121
+ model-index:
122
+ - name: SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B
123
+ results:
124
+ - task:
125
+ type: information-retrieval
126
+ name: Information Retrieval
127
+ dataset:
128
+ name: NanoMSMARCO
129
+ type: NanoMSMARCO
130
+ metrics:
131
+ - type: cosine_accuracy@1
132
+ value: 0.42
133
+ name: Cosine Accuracy@1
134
+ - type: cosine_accuracy@3
135
+ value: 0.64
136
+ name: Cosine Accuracy@3
137
+ - type: cosine_accuracy@5
138
+ value: 0.76
139
+ name: Cosine Accuracy@5
140
+ - type: cosine_accuracy@10
141
+ value: 0.82
142
+ name: Cosine Accuracy@10
143
+ - type: cosine_precision@1
144
+ value: 0.42
145
+ name: Cosine Precision@1
146
+ - type: cosine_precision@3
147
+ value: 0.21333333333333335
148
+ name: Cosine Precision@3
149
+ - type: cosine_precision@5
150
+ value: 0.15200000000000002
151
+ name: Cosine Precision@5
152
+ - type: cosine_precision@10
153
+ value: 0.08199999999999999
154
+ name: Cosine Precision@10
155
+ - type: cosine_recall@1
156
+ value: 0.42
157
+ name: Cosine Recall@1
158
+ - type: cosine_recall@3
159
+ value: 0.64
160
+ name: Cosine Recall@3
161
+ - type: cosine_recall@5
162
+ value: 0.76
163
+ name: Cosine Recall@5
164
+ - type: cosine_recall@10
165
+ value: 0.82
166
+ name: Cosine Recall@10
167
+ - type: cosine_ndcg@10
168
+ value: 0.620918816092183
169
+ name: Cosine Ndcg@10
170
+ - type: cosine_mrr@10
171
+ value: 0.5567777777777778
172
+ name: Cosine Mrr@10
173
+ - type: cosine_map@100
174
+ value: 0.5664067325709117
175
+ name: Cosine Map@100
176
+ - task:
177
+ type: information-retrieval
178
+ name: Information Retrieval
179
+ dataset:
180
+ name: NanoNFCorpus
181
+ type: NanoNFCorpus
182
+ metrics:
183
+ - type: cosine_accuracy@1
184
+ value: 0.38
185
+ name: Cosine Accuracy@1
186
+ - type: cosine_accuracy@3
187
+ value: 0.44
188
+ name: Cosine Accuracy@3
189
+ - type: cosine_accuracy@5
190
+ value: 0.52
191
+ name: Cosine Accuracy@5
192
+ - type: cosine_accuracy@10
193
+ value: 0.66
194
+ name: Cosine Accuracy@10
195
+ - type: cosine_precision@1
196
+ value: 0.38
197
+ name: Cosine Precision@1
198
+ - type: cosine_precision@3
199
+ value: 0.31333333333333335
200
+ name: Cosine Precision@3
201
+ - type: cosine_precision@5
202
+ value: 0.29200000000000004
203
+ name: Cosine Precision@5
204
+ - type: cosine_precision@10
205
+ value: 0.254
206
+ name: Cosine Precision@10
207
+ - type: cosine_recall@1
208
+ value: 0.041275151654868704
209
+ name: Cosine Recall@1
210
+ - type: cosine_recall@3
211
+ value: 0.06868331254409366
212
+ name: Cosine Recall@3
213
+ - type: cosine_recall@5
214
+ value: 0.08524350018847202
215
+ name: Cosine Recall@5
216
+ - type: cosine_recall@10
217
+ value: 0.11409038508225758
218
+ name: Cosine Recall@10
219
+ - type: cosine_ndcg@10
220
+ value: 0.30429750607308503
221
+ name: Cosine Ndcg@10
222
+ - type: cosine_mrr@10
223
+ value: 0.44163492063492066
224
+ name: Cosine Mrr@10
225
+ - type: cosine_map@100
226
+ value: 0.1254808602198398
227
+ name: Cosine Map@100
228
+ - task:
229
+ type: information-retrieval
230
+ name: Information Retrieval
231
+ dataset:
232
+ name: NanoNQ
233
+ type: NanoNQ
234
+ metrics:
235
+ - type: cosine_accuracy@1
236
+ value: 0.4
237
+ name: Cosine Accuracy@1
238
+ - type: cosine_accuracy@3
239
+ value: 0.72
240
+ name: Cosine Accuracy@3
241
+ - type: cosine_accuracy@5
242
+ value: 0.76
243
+ name: Cosine Accuracy@5
244
+ - type: cosine_accuracy@10
245
+ value: 0.82
246
+ name: Cosine Accuracy@10
247
+ - type: cosine_precision@1
248
+ value: 0.4
249
+ name: Cosine Precision@1
250
+ - type: cosine_precision@3
251
+ value: 0.24666666666666665
252
+ name: Cosine Precision@3
253
+ - type: cosine_precision@5
254
+ value: 0.16
255
+ name: Cosine Precision@5
256
+ - type: cosine_precision@10
257
+ value: 0.088
258
+ name: Cosine Precision@10
259
+ - type: cosine_recall@1
260
+ value: 0.39
261
+ name: Cosine Recall@1
262
+ - type: cosine_recall@3
263
+ value: 0.69
264
+ name: Cosine Recall@3
265
+ - type: cosine_recall@5
266
+ value: 0.73
267
+ name: Cosine Recall@5
268
+ - type: cosine_recall@10
269
+ value: 0.79
270
+ name: Cosine Recall@10
271
+ - type: cosine_ndcg@10
272
+ value: 0.6214012092294585
273
+ name: Cosine Ndcg@10
274
+ - type: cosine_mrr@10
275
+ value: 0.571047619047619
276
+ name: Cosine Mrr@10
277
+ - type: cosine_map@100
278
+ value: 0.564828869259454
279
+ name: Cosine Map@100
280
+ - task:
281
+ type: nano-beir
282
+ name: Nano BEIR
283
+ dataset:
284
+ name: NanoBEIR mean
285
+ type: NanoBEIR_mean
286
+ metrics:
287
+ - type: cosine_accuracy@1
288
+ value: 0.4000000000000001
289
+ name: Cosine Accuracy@1
290
+ - type: cosine_accuracy@3
291
+ value: 0.6
292
+ name: Cosine Accuracy@3
293
+ - type: cosine_accuracy@5
294
+ value: 0.68
295
+ name: Cosine Accuracy@5
296
+ - type: cosine_accuracy@10
297
+ value: 0.7666666666666666
298
+ name: Cosine Accuracy@10
299
+ - type: cosine_precision@1
300
+ value: 0.4000000000000001
301
+ name: Cosine Precision@1
302
+ - type: cosine_precision@3
303
+ value: 0.25777777777777783
304
+ name: Cosine Precision@3
305
+ - type: cosine_precision@5
306
+ value: 0.20133333333333336
307
+ name: Cosine Precision@5
308
+ - type: cosine_precision@10
309
+ value: 0.1413333333333333
310
+ name: Cosine Precision@10
311
+ - type: cosine_recall@1
312
+ value: 0.2837583838849562
313
+ name: Cosine Recall@1
314
+ - type: cosine_recall@3
315
+ value: 0.4662277708480312
316
+ name: Cosine Recall@3
317
+ - type: cosine_recall@5
318
+ value: 0.5250811667294907
319
+ name: Cosine Recall@5
320
+ - type: cosine_recall@10
321
+ value: 0.5746967950274192
322
+ name: Cosine Recall@10
323
+ - type: cosine_ndcg@10
324
+ value: 0.5155391771315755
325
+ name: Cosine Ndcg@10
326
+ - type: cosine_mrr@10
327
+ value: 0.5231534391534391
328
+ name: Cosine Mrr@10
329
+ - type: cosine_map@100
330
+ value: 0.41890548735006855
331
+ name: Cosine Map@100
332
+ - task:
333
+ type: knowledge-distillation
334
+ name: Knowledge Distillation
335
+ dataset:
336
+ name: Unknown
337
+ type: unknown
338
+ metrics:
339
+ - type: negative_mse
340
+ value: -0.016825005412101746
341
+ name: Negative Mse
342
+ ---
343
+
344
+ # SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B
345
+
346
+ This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) on the [nq](https://huggingface.co/datasets/sentence-transformers/natural-questions) dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
347
+
348
+ ## Model Details
349
+
350
+ ### Model Description
351
+ - **Model Type:** Sentence Transformer
352
+ - **Base model:** [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) <!-- at revision b22da495047858cce924d27d76261e96be6febc0 -->
353
+ - **Maximum Sequence Length:** 512 tokens
354
+ - **Output Dimensionality:** 1024 dimensions
355
+ - **Similarity Function:** Cosine Similarity
356
+ - **Training Dataset:**
357
+ - [nq](https://huggingface.co/datasets/sentence-transformers/natural-questions)
358
+ - **Language:** en
359
+ <!-- - **License:** Unknown -->
360
+
361
+ ### Model Sources
362
+
363
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
364
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
365
+ - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
366
+
367
+ ### Full Model Architecture
368
+
369
+ ```
370
+ SentenceTransformer(
371
+ (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: Qwen3Model
372
+ (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
373
+ (2): Normalize()
374
+ )
375
+ ```
376
+
377
+ ## Usage
378
+
379
+ ### Direct Usage (Sentence Transformers)
380
+
381
+ First install the Sentence Transformers library:
382
+
383
+ ```bash
384
+ pip install -U sentence-transformers
385
+ ```
386
+
387
+ Then you can load this model and run inference.
388
+ ```python
389
+ from sentence_transformers import SentenceTransformer
390
+
391
+ # Download from the 🤗 Hub
392
+ model = SentenceTransformer("tomaarsen/Qwen3-Embedding-0.6B-18-layers")
393
+ # Run inference
394
+ sentences = [
395
+ 'The actress was thirteen when she was offered the role of Annie.',
396
+ 'Contrasting significantly from other soccer leagues in the U.S., WLS intends to be an open entry, promotion and relegation competition.',
397
+ 'Narsingh Temple is situated at the across of the village just across confluence of Magri State village.',
398
+ ]
399
+ embeddings = model.encode(sentences)
400
+ print(embeddings.shape)
401
+ # [3, 1024]
402
+
403
+ # Get the similarity scores for the embeddings
404
+ similarities = model.similarity(embeddings, embeddings)
405
+ print(similarities.shape)
406
+ # [3, 3]
407
+ ```
408
+
409
+ <!--
410
+ ### Direct Usage (Transformers)
411
+
412
+ <details><summary>Click to see the direct usage in Transformers</summary>
413
+
414
+ </details>
415
+ -->
416
+
417
+ <!--
418
+ ### Downstream Usage (Sentence Transformers)
419
+
420
+ You can finetune this model on your own dataset.
421
+
422
+ <details><summary>Click to expand</summary>
423
+
424
+ </details>
425
+ -->
426
+
427
+ <!--
428
+ ### Out-of-Scope Use
429
+
430
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
431
+ -->
432
+
433
+ ## Evaluation
434
+
435
+ ### Metrics
436
+
437
+ #### Information Retrieval
438
+
439
+ * Datasets: `NanoMSMARCO`, `NanoNFCorpus` and `NanoNQ`
440
+ * Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters:
441
+ ```json
442
+ {
443
+ "query_prompt": "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:"
444
+ }
445
+ ```
446
+
447
+ | Metric | NanoMSMARCO | NanoNFCorpus | NanoNQ |
448
+ |:--------------------|:------------|:-------------|:-----------|
449
+ | cosine_accuracy@1 | 0.42 | 0.38 | 0.4 |
450
+ | cosine_accuracy@3 | 0.64 | 0.44 | 0.72 |
451
+ | cosine_accuracy@5 | 0.76 | 0.52 | 0.76 |
452
+ | cosine_accuracy@10 | 0.82 | 0.66 | 0.82 |
453
+ | cosine_precision@1 | 0.42 | 0.38 | 0.4 |
454
+ | cosine_precision@3 | 0.2133 | 0.3133 | 0.2467 |
455
+ | cosine_precision@5 | 0.152 | 0.292 | 0.16 |
456
+ | cosine_precision@10 | 0.082 | 0.254 | 0.088 |
457
+ | cosine_recall@1 | 0.42 | 0.0413 | 0.39 |
458
+ | cosine_recall@3 | 0.64 | 0.0687 | 0.69 |
459
+ | cosine_recall@5 | 0.76 | 0.0852 | 0.73 |
460
+ | cosine_recall@10 | 0.82 | 0.1141 | 0.79 |
461
+ | **cosine_ndcg@10** | **0.6209** | **0.3043** | **0.6214** |
462
+ | cosine_mrr@10 | 0.5568 | 0.4416 | 0.571 |
463
+ | cosine_map@100 | 0.5664 | 0.1255 | 0.5648 |
464
+
465
+ #### Nano BEIR
466
+
467
+ * Dataset: `NanoBEIR_mean`
468
+ * Evaluated with [<code>NanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.NanoBEIREvaluator) with these parameters:
469
+ ```json
470
+ {
471
+ "dataset_names": [
472
+ "msmarco",
473
+ "nfcorpus",
474
+ "nq"
475
+ ],
476
+ "query_prompts": {
477
+ "msmarco": "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:",
478
+ "nfcorpus": "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:",
479
+ "nq": "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:"
480
+ }
481
+ }
482
+ ```
483
+
484
+ | Metric | Value |
485
+ |:--------------------|:-----------|
486
+ | cosine_accuracy@1 | 0.4 |
487
+ | cosine_accuracy@3 | 0.6 |
488
+ | cosine_accuracy@5 | 0.68 |
489
+ | cosine_accuracy@10 | 0.7667 |
490
+ | cosine_precision@1 | 0.4 |
491
+ | cosine_precision@3 | 0.2578 |
492
+ | cosine_precision@5 | 0.2013 |
493
+ | cosine_precision@10 | 0.1413 |
494
+ | cosine_recall@1 | 0.2838 |
495
+ | cosine_recall@3 | 0.4662 |
496
+ | cosine_recall@5 | 0.5251 |
497
+ | cosine_recall@10 | 0.5747 |
498
+ | **cosine_ndcg@10** | **0.5155** |
499
+ | cosine_mrr@10 | 0.5232 |
500
+ | cosine_map@100 | 0.4189 |
501
+
502
+ #### Knowledge Distillation
503
+
504
+ * Evaluated with [<code>MSEEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.MSEEvaluator)
505
+
506
+ | Metric | Value |
507
+ |:-----------------|:------------|
508
+ | **negative_mse** | **-0.0168** |
509
+
510
+ <!--
511
+ ## Bias, Risks and Limitations
512
+
513
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
514
+ -->
515
+
516
+ <!--
517
+ ### Recommendations
518
+
519
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
520
+ -->
521
+
522
+ ## Training Details
523
+
524
+ ### Training Dataset
525
+
526
+ #### nq
527
+
528
+ * Dataset: [nq](https://huggingface.co/datasets/sentence-transformers/natural-questions) at [f9e894e](https://huggingface.co/datasets/sentence-transformers/natural-questions/tree/f9e894e1081e206e577b4eaa9ee6de2b06ae6f17)
529
+ * Size: 197,462 training samples
530
+ * Columns: <code>text</code> and <code>label</code>
531
+ * Approximate statistics based on the first 1000 samples:
532
+ | | text | label |
533
+ |:--------|:------------------------------------------------------------------------------------|:--------------------------------------|
534
+ | type | string | list |
535
+ | details | <ul><li>min: 27 tokens</li><li>mean: 89.38 tokens</li><li>max: 505 tokens</li></ul> | <ul><li>size: 1024 elements</li></ul> |
536
+ * Samples:
537
+ | text | label |
538
+ |:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------|
539
+ | <code>Instruct: Given a web search query, retrieve relevant passages that answer the query<br>Query:the movie bernie based on a true story</code> | <code>[-0.05126953125, -0.0020294189453125, 0.00152587890625, 0.060791015625, 0.022216796875, ...]</code> |
540
+ | <code>College World Series The College World Series, or CWS, is an annual June baseball tournament held in Omaha, Nebraska. The CWS is the culmination of the National Collegiate Athletic Association (NCAA) Division I Baseball Championship tournament—featuring 64 teams in the first round—which determines the NCAA Division I college baseball champion. The eight participating teams are split into two, four-team, double-elimination brackets, with the winners of each bracket playing in a best-of-three championship series.</code> | <code>[0.033935546875, -0.0908203125, -0.010498046875, 0.0625, -0.01263427734375, ...]</code> |
541
+ | <code>Instruct: Given a web search query, retrieve relevant passages that answer the query<br>Query:does the femoral nerve turn into the saphenous nerve</code> | <code>[0.052978515625, -0.0028228759765625, -0.0022430419921875, 0.0732421875, 0.044677734375, ...]</code> |
542
+ * Loss: [<code>MSELoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#mseloss)
543
+
544
+ ### Evaluation Datasets
545
+
546
+ #### nq
547
+
548
+ * Dataset: [nq](https://huggingface.co/datasets/sentence-transformers/natural-questions) at [f9e894e](https://huggingface.co/datasets/sentence-transformers/natural-questions/tree/f9e894e1081e206e577b4eaa9ee6de2b06ae6f17)
549
+ * Size: 3,000 evaluation samples
550
+ * Columns: <code>text</code> and <code>label</code>
551
+ * Approximate statistics based on the first 1000 samples:
552
+ | | text | label |
553
+ |:--------|:------------------------------------------------------------------------------------|:--------------------------------------|
554
+ | type | string | list |
555
+ | details | <ul><li>min: 21 tokens</li><li>mean: 87.24 tokens</li><li>max: 410 tokens</li></ul> | <ul><li>size: 1024 elements</li></ul> |
556
+ * Samples:
557
+ | text | label |
558
+ |:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------|
559
+ | <code>Instruct: Given a web search query, retrieve relevant passages that answer the query<br>Query:who was the heir apparent of the austro-hungarian empire in 1914</code> | <code>[0.0262451171875, 0.0556640625, -0.0, -0.03076171875, -0.05712890625, ...]</code> |
560
+ | <code>Instruct: Given a web search query, retrieve relevant passages that answer the query<br>Query:who played tommy in coward of the county</code> | <code>[-0.00848388671875, -0.02294921875, -0.00182342529296875, 0.060546875, -0.021240234375, ...]</code> |
561
+ | <code>Vertebra The vertebral arch is formed by pedicles and laminae. Two pedicles extend from the sides of the vertebral body to join the body to the arch. The pedicles are short thick processes that extend, one from each side, posteriorly, from the junctions of the posteriolateral surfaces of the centrum, on its upper surface. From each pedicle a broad plate, a lamina, projects backwards and medialwards to join and complete the vertebral arch and form the posterior border of the vertebral foramen, which completes the triangle of the vertebral foramen.[6] The upper surfaces of the laminae are rough to give attachment to the ligamenta flava. These ligaments connect the laminae of adjacent vertebra along the length of the spine from the level of the second cervical vertebra. Above and below the pedicles are shallow depressions called vertebral notches (superior and inferior). When the vertebrae articulate the notches align with those on adjacent vertebrae and these form the openings of the int...</code> | <code>[0.062255859375, -0.005706787109375, -0.009765625, 0.035400390625, -0.0125732421875, ...]</code> |
562
+ * Loss: [<code>MSELoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#mseloss)
563
+
564
+ #### gooaq
565
+
566
+ * Dataset: [gooaq](https://huggingface.co/datasets/sentence-transformers/gooaq) at [b089f72](https://huggingface.co/datasets/sentence-transformers/gooaq/tree/b089f728748a068b7bc5234e5bcf5b25e3c8279c)
567
+ * Size: 3,000 evaluation samples
568
+ * Columns: <code>text</code> and <code>label</code>
569
+ * Approximate statistics based on the first 1000 samples:
570
+ | | text | label |
571
+ |:--------|:------------------------------------------------------------------------------------|:--------------------------------------|
572
+ | type | string | list |
573
+ | details | <ul><li>min: 10 tokens</li><li>mean: 43.88 tokens</li><li>max: 117 tokens</li></ul> | <ul><li>size: 1024 elements</li></ul> |
574
+ * Samples:
575
+ | text | label |
576
+ |:-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------|
577
+ | <code>Instruct: Given a web search query, retrieve relevant passages that answer the query<br>Query:what essential oils are soothing?</code> | <code>[-0.025146484375, 0.06591796875, -0.0025634765625, 0.0732421875, -0.046630859375, ...]</code> |
578
+ | <code>Titles of books should be underlined or put in italics . (Titles of stories, essays and poems are in "quotation marks.") Refer to the text specifically as a novel, story, essay, memoir, or poem, depending on what it is.</code> | <code>[-0.006988525390625, -0.050537109375, -0.007476806640625, -0.07177734375, -0.049560546875, ...]</code> |
579
+ | <code>Dakine Cyclone Wet/Dry 32L Backpack. Born from the legacy of our most iconic surf pack, the Cyclone Collection is a family of super-technical and durable wet/dry packs and bags.</code> | <code>[0.0016632080078125, 0.04150390625, -0.01324462890625, 0.0234375, 0.03173828125, ...]</code> |
580
+ * Loss: [<code>MSELoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#mseloss)
581
+
582
+ #### wikipedia
583
+
584
+ * Dataset: [wikipedia](https://huggingface.co/datasets/sentence-transformers/wikipedia-en-sentences) at [4a0972d](https://huggingface.co/datasets/sentence-transformers/wikipedia-en-sentences/tree/4a0972dcb781b5b5d27799798f032606421dd422)
585
+ * Size: 3,000 evaluation samples
586
+ * Columns: <code>text</code> and <code>label</code>
587
+ * Approximate statistics based on the first 1000 samples:
588
+ | | text | label |
589
+ |:--------|:----------------------------------------------------------------------------------|:--------------------------------------|
590
+ | type | string | list |
591
+ | details | <ul><li>min: 5 tokens</li><li>mean: 28.1 tokens</li><li>max: 105 tokens</li></ul> | <ul><li>size: 1024 elements</li></ul> |
592
+ * Samples:
593
+ | text | label |
594
+ |:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------|
595
+ | <code>The daughter of Vice-admiral George Davies and Julia Hume, she spent her younger years on board the ship he was stationed, the Griper.</code> | <code>[0.0361328125, 0.01904296875, -0.003662109375, 0.0247802734375, 0.0140380859375, ...]</code> |
596
+ | <code>The impetus for the project began when Amalgamated Dynamics, hired to provide the practical effects for The Thing, a prequel to John Carpenter's 1982 classic film-renowned for its almost exclusive use of practical effects-became disillusioned upon discovering the theatrical release had the bulk of their effects digitally replaced with computer-generated imagery.</code> | <code>[-0.0106201171875, -0.0439453125, -0.01104736328125, 0.00946044921875, 0.0322265625, ...]</code> |
597
+ | <code>Lost Angeles, his second feature film, starring Joelle Carter and Kelly Blatz, had its world premiere at the Oldenburg International Film Festival in 2012.</code> | <code>[0.0272216796875, 0.0263671875, -0.007110595703125, 0.0294189453125, 0.01129150390625, ...]</code> |
598
+ * Loss: [<code>MSELoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#mseloss)
599
+
600
+ ### Training Hyperparameters
601
+ #### Non-Default Hyperparameters
602
+
603
+ - `eval_strategy`: steps
604
+ - `per_device_train_batch_size`: 16
605
+ - `per_device_eval_batch_size`: 16
606
+ - `learning_rate`: 0.0001
607
+ - `num_train_epochs`: 1
608
+ - `warmup_ratio`: 0.1
609
+ - `bf16`: True
610
+ - `load_best_model_at_end`: True
611
+
612
+ #### All Hyperparameters
613
+ <details><summary>Click to expand</summary>
614
+
615
+ - `overwrite_output_dir`: False
616
+ - `do_predict`: False
617
+ - `eval_strategy`: steps
618
+ - `prediction_loss_only`: True
619
+ - `per_device_train_batch_size`: 16
620
+ - `per_device_eval_batch_size`: 16
621
+ - `per_gpu_train_batch_size`: None
622
+ - `per_gpu_eval_batch_size`: None
623
+ - `gradient_accumulation_steps`: 1
624
+ - `eval_accumulation_steps`: None
625
+ - `torch_empty_cache_steps`: None
626
+ - `learning_rate`: 0.0001
627
+ - `weight_decay`: 0.0
628
+ - `adam_beta1`: 0.9
629
+ - `adam_beta2`: 0.999
630
+ - `adam_epsilon`: 1e-08
631
+ - `max_grad_norm`: 1.0
632
+ - `num_train_epochs`: 1
633
+ - `max_steps`: -1
634
+ - `lr_scheduler_type`: linear
635
+ - `lr_scheduler_kwargs`: {}
636
+ - `warmup_ratio`: 0.1
637
+ - `warmup_steps`: 0
638
+ - `log_level`: passive
639
+ - `log_level_replica`: warning
640
+ - `log_on_each_node`: True
641
+ - `logging_nan_inf_filter`: True
642
+ - `save_safetensors`: True
643
+ - `save_on_each_node`: False
644
+ - `save_only_model`: False
645
+ - `restore_callback_states_from_checkpoint`: False
646
+ - `no_cuda`: False
647
+ - `use_cpu`: False
648
+ - `use_mps_device`: False
649
+ - `seed`: 42
650
+ - `data_seed`: None
651
+ - `jit_mode_eval`: False
652
+ - `use_ipex`: False
653
+ - `bf16`: True
654
+ - `fp16`: False
655
+ - `fp16_opt_level`: O1
656
+ - `half_precision_backend`: auto
657
+ - `bf16_full_eval`: False
658
+ - `fp16_full_eval`: False
659
+ - `tf32`: None
660
+ - `local_rank`: 0
661
+ - `ddp_backend`: None
662
+ - `tpu_num_cores`: None
663
+ - `tpu_metrics_debug`: False
664
+ - `debug`: []
665
+ - `dataloader_drop_last`: False
666
+ - `dataloader_num_workers`: 0
667
+ - `dataloader_prefetch_factor`: None
668
+ - `past_index`: -1
669
+ - `disable_tqdm`: False
670
+ - `remove_unused_columns`: True
671
+ - `label_names`: None
672
+ - `load_best_model_at_end`: True
673
+ - `ignore_data_skip`: False
674
+ - `fsdp`: []
675
+ - `fsdp_min_num_params`: 0
676
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
677
+ - `tp_size`: 0
678
+ - `fsdp_transformer_layer_cls_to_wrap`: None
679
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
680
+ - `deepspeed`: None
681
+ - `label_smoothing_factor`: 0.0
682
+ - `optim`: adamw_torch
683
+ - `optim_args`: None
684
+ - `adafactor`: False
685
+ - `group_by_length`: False
686
+ - `length_column_name`: length
687
+ - `ddp_find_unused_parameters`: None
688
+ - `ddp_bucket_cap_mb`: None
689
+ - `ddp_broadcast_buffers`: False
690
+ - `dataloader_pin_memory`: True
691
+ - `dataloader_persistent_workers`: False
692
+ - `skip_memory_metrics`: True
693
+ - `use_legacy_prediction_loop`: False
694
+ - `push_to_hub`: False
695
+ - `resume_from_checkpoint`: None
696
+ - `hub_model_id`: None
697
+ - `hub_strategy`: every_save
698
+ - `hub_private_repo`: None
699
+ - `hub_always_push`: False
700
+ - `gradient_checkpointing`: False
701
+ - `gradient_checkpointing_kwargs`: None
702
+ - `include_inputs_for_metrics`: False
703
+ - `include_for_metrics`: []
704
+ - `eval_do_concat_batches`: True
705
+ - `fp16_backend`: auto
706
+ - `push_to_hub_model_id`: None
707
+ - `push_to_hub_organization`: None
708
+ - `mp_parameters`:
709
+ - `auto_find_batch_size`: False
710
+ - `full_determinism`: False
711
+ - `torchdynamo`: None
712
+ - `ray_scope`: last
713
+ - `ddp_timeout`: 1800
714
+ - `torch_compile`: False
715
+ - `torch_compile_backend`: None
716
+ - `torch_compile_mode`: None
717
+ - `include_tokens_per_second`: False
718
+ - `include_num_input_tokens_seen`: False
719
+ - `neftune_noise_alpha`: None
720
+ - `optim_target_modules`: None
721
+ - `batch_eval_metrics`: False
722
+ - `eval_on_start`: False
723
+ - `use_liger_kernel`: False
724
+ - `eval_use_gather_object`: False
725
+ - `average_tokens_across_devices`: False
726
+ - `prompts`: None
727
+ - `batch_sampler`: batch_sampler
728
+ - `multi_dataset_batch_sampler`: proportional
729
+
730
+ </details>
731
+
732
+ ### Training Logs
733
+ | Epoch | Step | Training Loss | nq loss | gooaq loss | wikipedia loss | NanoMSMARCO_cosine_ndcg@10 | NanoNFCorpus_cosine_ndcg@10 | NanoNQ_cosine_ndcg@10 | NanoBEIR_mean_cosine_ndcg@10 | negative_mse |
734
+ |:----------:|:--------:|:-------------:|:----------:|:----------:|:--------------:|:--------------------------:|:---------------------------:|:---------------------:|:----------------------------:|:------------:|
735
+ | -1 | -1 | - | - | - | - | 0.2033 | 0.0972 | 0.1638 | 0.1548 | -0.0985 |
736
+ | 0.0162 | 200 | 0.0008 | - | - | - | - | - | - | - | - |
737
+ | 0.0324 | 400 | 0.0004 | - | - | - | - | - | - | - | - |
738
+ | 0.0486 | 600 | 0.0003 | - | - | - | - | - | - | - | - |
739
+ | 0.0648 | 800 | 0.0003 | - | - | - | - | - | - | - | - |
740
+ | 0.0810 | 1000 | 0.0002 | 0.0002 | 0.0003 | 0.0003 | 0.5482 | 0.2864 | 0.5995 | 0.4780 | -0.0280 |
741
+ | 0.0972 | 1200 | 0.0002 | - | - | - | - | - | - | - | - |
742
+ | 0.1134 | 1400 | 0.0002 | - | - | - | - | - | - | - | - |
743
+ | 0.1296 | 1600 | 0.0002 | - | - | - | - | - | - | - | - |
744
+ | 0.1458 | 1800 | 0.0002 | - | - | - | - | - | - | - | - |
745
+ | 0.1620 | 2000 | 0.0002 | 0.0002 | 0.0003 | 0.0003 | 0.6136 | 0.2926 | 0.6028 | 0.5030 | -0.0218 |
746
+ | 0.1783 | 2200 | 0.0002 | - | - | - | - | - | - | - | - |
747
+ | 0.1945 | 2400 | 0.0001 | - | - | - | - | - | - | - | - |
748
+ | 0.2107 | 2600 | 0.0001 | - | - | - | - | - | - | - | - |
749
+ | 0.2269 | 2800 | 0.0001 | - | - | - | - | - | - | - | - |
750
+ | 0.2431 | 3000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6169 | 0.2990 | 0.5781 | 0.4980 | -0.0199 |
751
+ | 0.2593 | 3200 | 0.0001 | - | - | - | - | - | - | - | - |
752
+ | 0.2755 | 3400 | 0.0001 | - | - | - | - | - | - | - | - |
753
+ | 0.2917 | 3600 | 0.0001 | - | - | - | - | - | - | - | - |
754
+ | 0.3079 | 3800 | 0.0001 | - | - | - | - | - | - | - | - |
755
+ | 0.3241 | 4000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6137 | 0.3000 | 0.5987 | 0.5041 | -0.0187 |
756
+ | 0.3403 | 4200 | 0.0001 | - | - | - | - | - | - | - | - |
757
+ | 0.3565 | 4400 | 0.0001 | - | - | - | - | - | - | - | - |
758
+ | 0.3727 | 4600 | 0.0001 | - | - | - | - | - | - | - | - |
759
+ | 0.3889 | 4800 | 0.0001 | - | - | - | - | - | - | - | - |
760
+ | 0.4051 | 5000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6235 | 0.2945 | 0.6105 | 0.5095 | -0.0182 |
761
+ | 0.4213 | 5200 | 0.0001 | - | - | - | - | - | - | - | - |
762
+ | 0.4375 | 5400 | 0.0001 | - | - | - | - | - | - | - | - |
763
+ | 0.4537 | 5600 | 0.0001 | - | - | - | - | - | - | - | - |
764
+ | 0.4699 | 5800 | 0.0001 | - | - | - | - | - | - | - | - |
765
+ | 0.4861 | 6000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6183 | 0.2999 | 0.6141 | 0.5108 | -0.0175 |
766
+ | 0.5023 | 6200 | 0.0001 | - | - | - | - | - | - | - | - |
767
+ | 0.5186 | 6400 | 0.0001 | - | - | - | - | - | - | - | - |
768
+ | 0.5348 | 6600 | 0.0001 | - | - | - | - | - | - | - | - |
769
+ | 0.5510 | 6800 | 0.0001 | - | - | - | - | - | - | - | - |
770
+ | 0.5672 | 7000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6129 | 0.3005 | 0.6201 | 0.5112 | -0.0173 |
771
+ | 0.5834 | 7200 | 0.0001 | - | - | - | - | - | - | - | - |
772
+ | 0.5996 | 7400 | 0.0001 | - | - | - | - | - | - | - | - |
773
+ | 0.6158 | 7600 | 0.0001 | - | - | - | - | - | - | - | - |
774
+ | 0.6320 | 7800 | 0.0001 | - | - | - | - | - | - | - | - |
775
+ | 0.6482 | 8000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6258 | 0.3032 | 0.6099 | 0.5130 | -0.0170 |
776
+ | 0.6644 | 8200 | 0.0001 | - | - | - | - | - | - | - | - |
777
+ | 0.6806 | 8400 | 0.0001 | - | - | - | - | - | - | - | - |
778
+ | 0.6968 | 8600 | 0.0001 | - | - | - | - | - | - | - | - |
779
+ | 0.7130 | 8800 | 0.0001 | - | - | - | - | - | - | - | - |
780
+ | **0.7292** | **9000** | **0.0001** | **0.0001** | **0.0002** | **0.0002** | **0.6209** | **0.3043** | **0.6214** | **0.5155** | **-0.0168** |
781
+ | 0.7454 | 9200 | 0.0001 | - | - | - | - | - | - | - | - |
782
+ | 0.7616 | 9400 | 0.0001 | - | - | - | - | - | - | - | - |
783
+ | 0.7778 | 9600 | 0.0001 | - | - | - | - | - | - | - | - |
784
+ | 0.7940 | 9800 | 0.0001 | - | - | - | - | - | - | - | - |
785
+ | 0.8102 | 10000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6224 | 0.3015 | 0.6183 | 0.5141 | -0.0168 |
786
+ | 0.8264 | 10200 | 0.0001 | - | - | - | - | - | - | - | - |
787
+ | 0.8427 | 10400 | 0.0001 | - | - | - | - | - | - | - | - |
788
+ | 0.8589 | 10600 | 0.0001 | - | - | - | - | - | - | - | - |
789
+ | 0.8751 | 10800 | 0.0001 | - | - | - | - | - | - | - | - |
790
+ | 0.8913 | 11000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6224 | 0.3014 | 0.6155 | 0.5131 | -0.0167 |
791
+ | 0.9075 | 11200 | 0.0001 | - | - | - | - | - | - | - | - |
792
+ | 0.9237 | 11400 | 0.0001 | - | - | - | - | - | - | - | - |
793
+ | 0.9399 | 11600 | 0.0001 | - | - | - | - | - | - | - | - |
794
+ | 0.9561 | 11800 | 0.0001 | - | - | - | - | - | - | - | - |
795
+ | 0.9723 | 12000 | 0.0001 | 0.0001 | 0.0002 | 0.0002 | 0.6247 | 0.3020 | 0.6133 | 0.5133 | -0.0167 |
796
+ | 0.9885 | 12200 | 0.0001 | - | - | - | - | - | - | - | - |
797
+ | -1 | -1 | - | - | - | - | 0.6209 | 0.3043 | 0.6214 | 0.5155 | -0.0168 |
798
+
799
+ * The bold row denotes the saved checkpoint.
800
+
801
+ ### Framework Versions
802
+ - Python: 3.11.10
803
+ - Sentence Transformers: 4.2.0.dev0
804
+ - Transformers: 4.51.2
805
+ - PyTorch: 2.5.1+cu124
806
+ - Accelerate: 1.5.2
807
+ - Datasets: 3.5.0
808
+ - Tokenizers: 0.21.0
809
+
810
+ ## Citation
811
+
812
+ ### BibTeX
813
+
814
+ #### Sentence Transformers
815
+ ```bibtex
816
+ @inproceedings{reimers-2019-sentence-bert,
817
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
818
+ author = "Reimers, Nils and Gurevych, Iryna",
819
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
820
+ month = "11",
821
+ year = "2019",
822
+ publisher = "Association for Computational Linguistics",
823
+ url = "https://arxiv.org/abs/1908.10084",
824
+ }
825
+ ```
826
+
827
+ #### MSELoss
828
+ ```bibtex
829
+ @inproceedings{reimers-2020-multilingual-sentence-bert,
830
+ title = "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation",
831
+ author = "Reimers, Nils and Gurevych, Iryna",
832
+ booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
833
+ month = "11",
834
+ year = "2020",
835
+ publisher = "Association for Computational Linguistics",
836
+ url = "https://arxiv.org/abs/2004.09813",
837
+ }
838
+ ```
839
+
840
+ <!--
841
+ ## Glossary
842
+
843
+ *Clearly define terms in order to be accessible across audiences.*
844
+ -->
845
+
846
+ <!--
847
+ ## Model Card Authors
848
+
849
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
850
+ -->
851
+
852
+ <!--
853
+ ## Model Card Contact
854
+
855
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
856
+ -->
added_tokens.json ADDED
@@ -0,0 +1,28 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "</think>": 151668,
3
+ "</tool_call>": 151658,
4
+ "</tool_response>": 151666,
5
+ "<think>": 151667,
6
+ "<tool_call>": 151657,
7
+ "<tool_response>": 151665,
8
+ "<|box_end|>": 151649,
9
+ "<|box_start|>": 151648,
10
+ "<|endoftext|>": 151643,
11
+ "<|file_sep|>": 151664,
12
+ "<|fim_middle|>": 151660,
13
+ "<|fim_pad|>": 151662,
14
+ "<|fim_prefix|>": 151659,
15
+ "<|fim_suffix|>": 151661,
16
+ "<|im_end|>": 151645,
17
+ "<|im_start|>": 151644,
18
+ "<|image_pad|>": 151655,
19
+ "<|object_ref_end|>": 151647,
20
+ "<|object_ref_start|>": 151646,
21
+ "<|quad_end|>": 151651,
22
+ "<|quad_start|>": 151650,
23
+ "<|repo_name|>": 151663,
24
+ "<|video_pad|>": 151656,
25
+ "<|vision_end|>": 151653,
26
+ "<|vision_pad|>": 151654,
27
+ "<|vision_start|>": 151652
28
+ }
config.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "Qwen3Model"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 151643,
8
+ "eos_token_id": 151643,
9
+ "head_dim": 128,
10
+ "hidden_act": "silu",
11
+ "hidden_size": 1024,
12
+ "initializer_range": 0.02,
13
+ "intermediate_size": 3072,
14
+ "max_position_embeddings": 32768,
15
+ "max_window_layers": 28,
16
+ "model_type": "qwen3",
17
+ "num_attention_heads": 16,
18
+ "num_hidden_layers": 18,
19
+ "num_key_value_heads": 8,
20
+ "rms_norm_eps": 1e-06,
21
+ "rope_scaling": null,
22
+ "rope_theta": 1000000,
23
+ "sliding_window": null,
24
+ "tie_word_embeddings": true,
25
+ "torch_dtype": "bfloat16",
26
+ "transformers_version": "4.51.2",
27
+ "use_cache": true,
28
+ "use_sliding_window": false,
29
+ "vocab_size": 151669
30
+ }
config_sentence_transformers.json ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "prompts": {
3
+ "query": "Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:",
4
+ "document": ""
5
+ },
6
+ "default_prompt_name": null,
7
+ "similarity_fn_name": "cosine",
8
+ "model_type": "SentenceTransformer",
9
+ "__version__": {
10
+ "sentence_transformers": "4.2.0.dev0",
11
+ "transformers": "4.51.2",
12
+ "pytorch": "2.5.1+cu124"
13
+ }
14
+ }
merges.txt ADDED
The diff for this file is too large to render. See raw diff
 
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:4b20283ebb796d18157b657b928be43d34bb2a6863fbc1d264e2e07b468e582d
3
+ size 876955560
modules.json ADDED
@@ -0,0 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "idx": 0,
4
+ "name": "0",
5
+ "path": "",
6
+ "type": "sentence_transformers.models.Transformer"
7
+ },
8
+ {
9
+ "idx": 1,
10
+ "name": "1",
11
+ "path": "1_Pooling",
12
+ "type": "sentence_transformers.models.Pooling"
13
+ },
14
+ {
15
+ "idx": 2,
16
+ "name": "2",
17
+ "path": "2_Normalize",
18
+ "type": "sentence_transformers.models.Normalize"
19
+ }
20
+ ]
sentence_bert_config.json ADDED
@@ -0,0 +1,4 @@
 
 
 
 
 
1
+ {
2
+ "max_seq_length": 512,
3
+ "do_lower_case": false
4
+ }
special_tokens_map.json ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "additional_special_tokens": [
3
+ "<|im_start|>",
4
+ "<|im_end|>",
5
+ "<|object_ref_start|>",
6
+ "<|object_ref_end|>",
7
+ "<|box_start|>",
8
+ "<|box_end|>",
9
+ "<|quad_start|>",
10
+ "<|quad_end|>",
11
+ "<|vision_start|>",
12
+ "<|vision_end|>",
13
+ "<|vision_pad|>",
14
+ "<|image_pad|>",
15
+ "<|video_pad|>"
16
+ ],
17
+ "eos_token": {
18
+ "content": "<|im_end|>",
19
+ "lstrip": false,
20
+ "normalized": false,
21
+ "rstrip": false,
22
+ "single_word": false
23
+ },
24
+ "pad_token": {
25
+ "content": "<|endoftext|>",
26
+ "lstrip": false,
27
+ "normalized": false,
28
+ "rstrip": false,
29
+ "single_word": false
30
+ }
31
+ }
tokenizer.json ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a5b90ffcbd8fe896c9ee9fe56c5dd84116f876ad5cdbe0d1424fbe150f41ca6
3
+ size 11423970
tokenizer_config.json ADDED
@@ -0,0 +1,241 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "add_bos_token": false,
3
+ "add_prefix_space": false,
4
+ "added_tokens_decoder": {
5
+ "151643": {
6
+ "content": "<|endoftext|>",
7
+ "lstrip": false,
8
+ "normalized": false,
9
+ "rstrip": false,
10
+ "single_word": false,
11
+ "special": true
12
+ },
13
+ "151644": {
14
+ "content": "<|im_start|>",
15
+ "lstrip": false,
16
+ "normalized": false,
17
+ "rstrip": false,
18
+ "single_word": false,
19
+ "special": true
20
+ },
21
+ "151645": {
22
+ "content": "<|im_end|>",
23
+ "lstrip": false,
24
+ "normalized": false,
25
+ "rstrip": false,
26
+ "single_word": false,
27
+ "special": true
28
+ },
29
+ "151646": {
30
+ "content": "<|object_ref_start|>",
31
+ "lstrip": false,
32
+ "normalized": false,
33
+ "rstrip": false,
34
+ "single_word": false,
35
+ "special": true
36
+ },
37
+ "151647": {
38
+ "content": "<|object_ref_end|>",
39
+ "lstrip": false,
40
+ "normalized": false,
41
+ "rstrip": false,
42
+ "single_word": false,
43
+ "special": true
44
+ },
45
+ "151648": {
46
+ "content": "<|box_start|>",
47
+ "lstrip": false,
48
+ "normalized": false,
49
+ "rstrip": false,
50
+ "single_word": false,
51
+ "special": true
52
+ },
53
+ "151649": {
54
+ "content": "<|box_end|>",
55
+ "lstrip": false,
56
+ "normalized": false,
57
+ "rstrip": false,
58
+ "single_word": false,
59
+ "special": true
60
+ },
61
+ "151650": {
62
+ "content": "<|quad_start|>",
63
+ "lstrip": false,
64
+ "normalized": false,
65
+ "rstrip": false,
66
+ "single_word": false,
67
+ "special": true
68
+ },
69
+ "151651": {
70
+ "content": "<|quad_end|>",
71
+ "lstrip": false,
72
+ "normalized": false,
73
+ "rstrip": false,
74
+ "single_word": false,
75
+ "special": true
76
+ },
77
+ "151652": {
78
+ "content": "<|vision_start|>",
79
+ "lstrip": false,
80
+ "normalized": false,
81
+ "rstrip": false,
82
+ "single_word": false,
83
+ "special": true
84
+ },
85
+ "151653": {
86
+ "content": "<|vision_end|>",
87
+ "lstrip": false,
88
+ "normalized": false,
89
+ "rstrip": false,
90
+ "single_word": false,
91
+ "special": true
92
+ },
93
+ "151654": {
94
+ "content": "<|vision_pad|>",
95
+ "lstrip": false,
96
+ "normalized": false,
97
+ "rstrip": false,
98
+ "single_word": false,
99
+ "special": true
100
+ },
101
+ "151655": {
102
+ "content": "<|image_pad|>",
103
+ "lstrip": false,
104
+ "normalized": false,
105
+ "rstrip": false,
106
+ "single_word": false,
107
+ "special": true
108
+ },
109
+ "151656": {
110
+ "content": "<|video_pad|>",
111
+ "lstrip": false,
112
+ "normalized": false,
113
+ "rstrip": false,
114
+ "single_word": false,
115
+ "special": true
116
+ },
117
+ "151657": {
118
+ "content": "<tool_call>",
119
+ "lstrip": false,
120
+ "normalized": false,
121
+ "rstrip": false,
122
+ "single_word": false,
123
+ "special": false
124
+ },
125
+ "151658": {
126
+ "content": "</tool_call>",
127
+ "lstrip": false,
128
+ "normalized": false,
129
+ "rstrip": false,
130
+ "single_word": false,
131
+ "special": false
132
+ },
133
+ "151659": {
134
+ "content": "<|fim_prefix|>",
135
+ "lstrip": false,
136
+ "normalized": false,
137
+ "rstrip": false,
138
+ "single_word": false,
139
+ "special": false
140
+ },
141
+ "151660": {
142
+ "content": "<|fim_middle|>",
143
+ "lstrip": false,
144
+ "normalized": false,
145
+ "rstrip": false,
146
+ "single_word": false,
147
+ "special": false
148
+ },
149
+ "151661": {
150
+ "content": "<|fim_suffix|>",
151
+ "lstrip": false,
152
+ "normalized": false,
153
+ "rstrip": false,
154
+ "single_word": false,
155
+ "special": false
156
+ },
157
+ "151662": {
158
+ "content": "<|fim_pad|>",
159
+ "lstrip": false,
160
+ "normalized": false,
161
+ "rstrip": false,
162
+ "single_word": false,
163
+ "special": false
164
+ },
165
+ "151663": {
166
+ "content": "<|repo_name|>",
167
+ "lstrip": false,
168
+ "normalized": false,
169
+ "rstrip": false,
170
+ "single_word": false,
171
+ "special": false
172
+ },
173
+ "151664": {
174
+ "content": "<|file_sep|>",
175
+ "lstrip": false,
176
+ "normalized": false,
177
+ "rstrip": false,
178
+ "single_word": false,
179
+ "special": false
180
+ },
181
+ "151665": {
182
+ "content": "<tool_response>",
183
+ "lstrip": false,
184
+ "normalized": false,
185
+ "rstrip": false,
186
+ "single_word": false,
187
+ "special": false
188
+ },
189
+ "151666": {
190
+ "content": "</tool_response>",
191
+ "lstrip": false,
192
+ "normalized": false,
193
+ "rstrip": false,
194
+ "single_word": false,
195
+ "special": false
196
+ },
197
+ "151667": {
198
+ "content": "<think>",
199
+ "lstrip": false,
200
+ "normalized": false,
201
+ "rstrip": false,
202
+ "single_word": false,
203
+ "special": false
204
+ },
205
+ "151668": {
206
+ "content": "</think>",
207
+ "lstrip": false,
208
+ "normalized": false,
209
+ "rstrip": false,
210
+ "single_word": false,
211
+ "special": false
212
+ }
213
+ },
214
+ "additional_special_tokens": [
215
+ "<|im_start|>",
216
+ "<|im_end|>",
217
+ "<|object_ref_start|>",
218
+ "<|object_ref_end|>",
219
+ "<|box_start|>",
220
+ "<|box_end|>",
221
+ "<|quad_start|>",
222
+ "<|quad_end|>",
223
+ "<|vision_start|>",
224
+ "<|vision_end|>",
225
+ "<|vision_pad|>",
226
+ "<|image_pad|>",
227
+ "<|video_pad|>"
228
+ ],
229
+ "bos_token": null,
230
+ "chat_template": "{%- if tools %}\n {{- '<|im_start|>system\\n' }}\n {%- if messages[0].role == 'system' %}\n {{- messages[0].content + '\\n\\n' }}\n {%- endif %}\n {{- \"# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n {%- for tool in tools %}\n {{- \"\\n\" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {{- '<|im_start|>system\\n' + messages[0].content + '<|im_end|>\\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == \"user\" and not(message.content.startswith('<tool_response>') and message.content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n{%- endfor %}\n{%- for message in messages %}\n {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) %}\n {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n {%- elif message.role == \"assistant\" %}\n {%- set content = message.content %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is defined and message.reasoning_content is not none %}\n {%- set reasoning_content = message.reasoning_content %}\n {%- else %}\n {%- if '</think>' in message.content %}\n {%- set content = message.content.split('</think>')[-1].lstrip('\\n') %}\n {%- set reasoning_content = message.content.split('</think>')[0].rstrip('\\n').split('<think>')[-1].lstrip('\\n') %}\n {%- endif %}\n {%- endif %}\n {%- if loop.index0 > ns.last_query_index %}\n {%- if loop.last or (not loop.last and reasoning_content) %}\n {{- '<|im_start|>' + message.role + '\\n<think>\\n' + reasoning_content.strip('\\n') + '\\n</think>\\n\\n' + content.lstrip('\\n') }}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- else %}\n {{- '<|im_start|>' + message.role + '\\n' + content }}\n {%- endif %}\n {%- if message.tool_calls %}\n {%- for tool_call in message.tool_calls %}\n {%- if (loop.first and content) or (not loop.first) %}\n {{- '\\n' }}\n {%- endif %}\n {%- if tool_call.function %}\n {%- set tool_call = tool_call.function %}\n {%- endif %}\n {{- '<tool_call>\\n{\"name\": \"' }}\n {{- tool_call.name }}\n {{- '\", \"arguments\": ' }}\n {%- if tool_call.arguments is string %}\n {{- tool_call.arguments }}\n {%- else %}\n {{- tool_call.arguments | tojson }}\n {%- endif %}\n {{- '}\\n</tool_call>' }}\n {%- endfor %}\n {%- endif %}\n {{- '<|im_end|>\\n' }}\n {%- elif message.role == \"tool\" %}\n {%- if loop.first or (messages[loop.index0 - 1].role != \"tool\") %}\n {{- '<|im_start|>user' }}\n {%- endif %}\n {{- '\\n<tool_response>\\n' }}\n {{- message.content }}\n {{- '\\n</tool_response>' }}\n {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n {{- '<|im_end|>\\n' }}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if add_generation_prompt %}\n {{- '<|im_start|>assistant\\n' }}\n {%- if enable_thinking is defined and enable_thinking is false %}\n {{- '<think>\\n\\n</think>\\n\\n' }}\n {%- endif %}\n{%- endif %}",
231
+ "clean_up_tokenization_spaces": false,
232
+ "eos_token": "<|im_end|>",
233
+ "errors": "replace",
234
+ "extra_special_tokens": {},
235
+ "model_max_length": 131072,
236
+ "pad_token": "<|endoftext|>",
237
+ "padding_side": "left",
238
+ "split_special_tokens": false,
239
+ "tokenizer_class": "Qwen2Tokenizer",
240
+ "unk_token": null
241
+ }
vocab.json ADDED
The diff for this file is too large to render. See raw diff