Hugging Face's logo

Oysiyl
/
reranker-ModernBERT-base-gooaq-bce

Text Ranking
sentence-transformers
Safetensors
English
modernbert
cross-encoder
Generated from Trainer
dataset_size:578402
loss:BinaryCrossEntropyLoss
Eval Results
Model card Files Community
Oysiyl commited on
Commit
38c8644
·
verified ·
1 Parent(s): cd2b9ab

Add new CrossEncoder model

Browse files
Files changed (6) hide show
  1. README.md +512 -0
  2. config.json +55 -0
  3. model.safetensors +3 -0
  4. special_tokens_map.json +37 -0
  5. tokenizer.json +0 -0
  6. tokenizer_config.json +945 -0
README.md ADDED
@@ -0,0 +1,512 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: apache-2.0
5
+ tags:
6
+ - sentence-transformers
7
+ - cross-encoder
8
+ - generated_from_trainer
9
+ - dataset_size:578402
10
+ - loss:BinaryCrossEntropyLoss
11
+ base_model: answerdotai/ModernBERT-base
12
+ pipeline_tag: text-ranking
13
+ library_name: sentence-transformers
14
+ metrics:
15
+ - map
16
+ - mrr@10
17
+ - ndcg@10
18
+ model-index:
19
+ - name: ModernBERT-base trained on GooAQ
20
+ results:
21
+ - task:
22
+ type: cross-encoder-reranking
23
+ name: Cross Encoder Reranking
24
+ dataset:
25
+ name: gooaq dev
26
+ type: gooaq-dev
27
+ metrics:
28
+ - type: map
29
+ value: 0.7283
30
+ name: Map
31
+ - type: mrr@10
32
+ value: 0.7272
33
+ name: Mrr@10
34
+ - type: ndcg@10
35
+ value: 0.7716
36
+ name: Ndcg@10
37
+ - task:
38
+ type: cross-encoder-reranking
39
+ name: Cross Encoder Reranking
40
+ dataset:
41
+ name: NanoMSMARCO R100
42
+ type: NanoMSMARCO_R100
43
+ metrics:
44
+ - type: map
45
+ value: 0.4489
46
+ name: Map
47
+ - type: mrr@10
48
+ value: 0.4376
49
+ name: Mrr@10
50
+ - type: ndcg@10
51
+ value: 0.5096
52
+ name: Ndcg@10
53
+ - task:
54
+ type: cross-encoder-reranking
55
+ name: Cross Encoder Reranking
56
+ dataset:
57
+ name: NanoNFCorpus R100
58
+ type: NanoNFCorpus_R100
59
+ metrics:
60
+ - type: map
61
+ value: 0.3159
62
+ name: Map
63
+ - type: mrr@10
64
+ value: 0.4737
65
+ name: Mrr@10
66
+ - type: ndcg@10
67
+ value: 0.3176
68
+ name: Ndcg@10
69
+ - task:
70
+ type: cross-encoder-reranking
71
+ name: Cross Encoder Reranking
72
+ dataset:
73
+ name: NanoNQ R100
74
+ type: NanoNQ_R100
75
+ metrics:
76
+ - type: map
77
+ value: 0.4904
78
+ name: Map
79
+ - type: mrr@10
80
+ value: 0.5075
81
+ name: Mrr@10
82
+ - type: ndcg@10
83
+ value: 0.5388
84
+ name: Ndcg@10
85
+ - task:
86
+ type: cross-encoder-nano-beir
87
+ name: Cross Encoder Nano BEIR
88
+ dataset:
89
+ name: NanoBEIR R100 mean
90
+ type: NanoBEIR_R100_mean
91
+ metrics:
92
+ - type: map
93
+ value: 0.4184
94
+ name: Map
95
+ - type: mrr@10
96
+ value: 0.4729
97
+ name: Mrr@10
98
+ - type: ndcg@10
99
+ value: 0.4553
100
+ name: Ndcg@10
101
+ ---
102
+
103
+ # ModernBERT-base trained on GooAQ
104
+
105
+ This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
106
+
107
+ ## Model Details
108
+
109
+ ### Model Description
110
+ - **Model Type:** Cross Encoder
111
+ - **Base model:** [answerdotai/ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) <!-- at revision 8949b909ec900327062f0ebf497f51aef5e6f0c8 -->
112
+ - **Maximum Sequence Length:** 8192 tokens
113
+ - **Number of Output Labels:** 1 label
114
+ <!-- - **Training Dataset:** Unknown -->
115
+ - **Language:** en
116
+ - **License:** apache-2.0
117
+
118
+ ### Model Sources
119
+
120
+ - **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
121
+ - **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
122
+ - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
123
+ - **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
124
+
125
+ ## Usage
126
+
127
+ ### Direct Usage (Sentence Transformers)
128
+
129
+ First install the Sentence Transformers library:
130
+
131
+ ```bash
132
+ pip install -U sentence-transformers
133
+ ```
134
+
135
+ Then you can load this model and run inference.
136
+ ```python
137
+ from sentence_transformers import CrossEncoder
138
+
139
+ # Download from the 🤗 Hub
140
+ model = CrossEncoder("Oysiyl/reranker-ModernBERT-base-gooaq-bce")
141
+ # Get scores for pairs of texts
142
+ pairs = [
143
+ ['what do you do with a degree in criminal justice?', "['Police Patrol Officer. Job Description. ... ', 'Criminal Investigators & Special Agents. Job Description. ... ', 'Private Detective or Investigator. ... ', 'First-Line Police Supervisor. ... ', 'Correctional Officer. ... ', 'Probation and Parole Officers. ... ', 'Postsecondary Criminal Justice & Law Enforcement Teachers.']"],
144
+ ['what do you do with a degree in criminal justice?', "['Prison officer. ... ', 'Police officer. ... ', 'Detective. ... ', 'Criminologist. ... ', 'Probation officer. ... ', 'Forensic scientist. ... ', 'Crime Scene investigator. ... ', 'Court reporter.']"],
145
+ ['what do you do with a degree in criminal justice?', "A high school diploma is required to work as a detective. In some cases a bachelor's degree in criminal justice or law enforcement may be needed. Experience in law enforcement is usually required, but the amount varies by employer."],
146
+ ['what do you do with a degree in criminal justice?', "['Court Reporter. ... ', 'Criminal Intelligence Analyst. ... ', 'Forensic Accountant. ... ', 'Police Officer and Police Support Roles. ... ', 'Immigration, Customs and Border Roles. ... ', 'Prison Officer. ... ', 'Probation Officer. ... ', 'Scene of Crime Officer.']"],
147
+ ['what do you do with a degree in criminal justice?', "Crime Scene Investigator Education and Training Although a high school diploma or equivalent is a minimum requirement for some positions, many police departments and law enforcement agencies prefer a minimum of an associate's (two-year) or a bachelor's (four-year) degree in criminal justice or a natural science."],
148
+ ]
149
+ scores = model.predict(pairs)
150
+ print(scores.shape)
151
+ # (5,)
152
+
153
+ # Or rank different texts based on similarity to a single text
154
+ ranks = model.rank(
155
+ 'what do you do with a degree in criminal justice?',
156
+ [
157
+ "['Police Patrol Officer. Job Description. ... ', 'Criminal Investigators & Special Agents. Job Description. ... ', 'Private Detective or Investigator. ... ', 'First-Line Police Supervisor. ... ', 'Correctional Officer. ... ', 'Probation and Parole Officers. ... ', 'Postsecondary Criminal Justice & Law Enforcement Teachers.']",
158
+ "['Prison officer. ... ', 'Police officer. ... ', 'Detective. ... ', 'Criminologist. ... ', 'Probation officer. ... ', 'Forensic scientist. ... ', 'Crime Scene investigator. ... ', 'Court reporter.']",
159
+ "A high school diploma is required to work as a detective. In some cases a bachelor's degree in criminal justice or law enforcement may be needed. Experience in law enforcement is usually required, but the amount varies by employer.",
160
+ "['Court Reporter. ... ', 'Criminal Intelligence Analyst. ... ', 'Forensic Accountant. ... ', 'Police Officer and Police Support Roles. ... ', 'Immigration, Customs and Border Roles. ... ', 'Prison Officer. ... ', 'Probation Officer. ... ', 'Scene of Crime Officer.']",
161
+ "Crime Scene Investigator Education and Training Although a high school diploma or equivalent is a minimum requirement for some positions, many police departments and law enforcement agencies prefer a minimum of an associate's (two-year) or a bachelor's (four-year) degree in criminal justice or a natural science.",
162
+ ]
163
+ )
164
+ # [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
165
+ ```
166
+
167
+ <!--
168
+ ### Direct Usage (Transformers)
169
+
170
+ <details><summary>Click to see the direct usage in Transformers</summary>
171
+
172
+ </details>
173
+ -->
174
+
175
+ <!--
176
+ ### Downstream Usage (Sentence Transformers)
177
+
178
+ You can finetune this model on your own dataset.
179
+
180
+ <details><summary>Click to expand</summary>
181
+
182
+ </details>
183
+ -->
184
+
185
+ <!--
186
+ ### Out-of-Scope Use
187
+
188
+ *List how the model may foreseeably be misused and address what users ought not to do with the model.*
189
+ -->
190
+
191
+ ## Evaluation
192
+
193
+ ### Metrics
194
+
195
+ #### Cross Encoder Reranking
196
+
197
+ * Dataset: `gooaq-dev`
198
+ * Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
199
+ ```json
200
+ {
201
+ "at_k": 10,
202
+ "always_rerank_positives": false
203
+ }
204
+ ```
205
+
206
+ | Metric | Value |
207
+ |:------------|:---------------------|
208
+ | map | 0.7283 (+0.1972) |
209
+ | mrr@10 | 0.7272 (+0.2033) |
210
+ | **ndcg@10** | **0.7716 (+0.1804)** |
211
+
212
+ #### Cross Encoder Reranking
213
+
214
+ * Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
215
+ * Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
216
+ ```json
217
+ {
218
+ "at_k": 10,
219
+ "always_rerank_positives": true
220
+ }
221
+ ```
222
+
223
+ | Metric | NanoMSMARCO_R100 | NanoNFCorpus_R100 | NanoNQ_R100 |
224
+ |:------------|:---------------------|:---------------------|:---------------------|
225
+ | map | 0.4489 (-0.0407) | 0.3159 (+0.0549) | 0.4904 (+0.0708) |
226
+ | mrr@10 | 0.4376 (-0.0399) | 0.4737 (-0.0262) | 0.5075 (+0.0808) |
227
+ | **ndcg@10** | **0.5096 (-0.0309)** | **0.3176 (-0.0074)** | **0.5388 (+0.0382)** |
228
+
229
+ #### Cross Encoder Nano BEIR
230
+
231
+ * Dataset: `NanoBEIR_R100_mean`
232
+ * Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
233
+ ```json
234
+ {
235
+ "dataset_names": [
236
+ "msmarco",
237
+ "nfcorpus",
238
+ "nq"
239
+ ],
240
+ "rerank_k": 100,
241
+ "at_k": 10,
242
+ "always_rerank_positives": true
243
+ }
244
+ ```
245
+
246
+ | Metric | Value |
247
+ |:------------|:---------------------|
248
+ | map | 0.4184 (+0.0284) |
249
+ | mrr@10 | 0.4729 (+0.0049) |
250
+ | **ndcg@10** | **0.4553 (-0.0000)** |
251
+
252
+ <!--
253
+ ## Bias, Risks and Limitations
254
+
255
+ *What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
256
+ -->
257
+
258
+ <!--
259
+ ### Recommendations
260
+
261
+ *What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
262
+ -->
263
+
264
+ ## Training Details
265
+
266
+ ### Training Dataset
267
+
268
+ #### Unnamed Dataset
269
+
270
+ * Size: 578,402 training samples
271
+ * Columns: <code>question</code>, <code>answer</code>, and <code>label</code>
272
+ * Approximate statistics based on the first 1000 samples:
273
+ | | question | answer | label |
274
+ |:--------|:-----------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------|:------------------------------------------------|
275
+ | type | string | string | int |
276
+ | details | <ul><li>min: 20 characters</li><li>mean: 43.36 characters</li><li>max: 95 characters</li></ul> | <ul><li>min: 55 characters</li><li>mean: 252.47 characters</li><li>max: 386 characters</li></ul> | <ul><li>0: ~82.80%</li><li>1: ~17.20%</li></ul> |
277
+ * Samples:
278
+ | question | answer | label |
279
+ |:---------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------|
280
+ | <code>what do you do with a degree in criminal justice?</code> | <code>['Police Patrol Officer. Job Description. ... ', 'Criminal Investigators & Special Agents. Job Description. ... ', 'Private Detective or Investigator. ... ', 'First-Line Police Supervisor. ... ', 'Correctional Officer. ... ', 'Probation and Parole Officers. ... ', 'Postsecondary Criminal Justice & Law Enforcement Teachers.']</code> | <code>1</code> |
281
+ | <code>what do you do with a degree in criminal justice?</code> | <code>['Prison officer. ... ', 'Police officer. ... ', 'Detective. ... ', 'Criminologist. ... ', 'Probation officer. ... ', 'Forensic scientist. ... ', 'Crime Scene investigator. ... ', 'Court reporter.']</code> | <code>0</code> |
282
+ | <code>what do you do with a degree in criminal justice?</code> | <code>A high school diploma is required to work as a detective. In some cases a bachelor's degree in criminal justice or law enforcement may be needed. Experience in law enforcement is usually required, but the amount varies by employer.</code> | <code>0</code> |
283
+ * Loss: [<code>BinaryCrossEntropyLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#binarycrossentropyloss) with these parameters:
284
+ ```json
285
+ {
286
+ "activation_fn": "torch.nn.modules.linear.Identity",
287
+ "pos_weight": 5
288
+ }
289
+ ```
290
+
291
+ ### Training Hyperparameters
292
+ #### Non-Default Hyperparameters
293
+
294
+ - `eval_strategy`: steps
295
+ - `per_device_train_batch_size`: 16
296
+ - `per_device_eval_batch_size`: 16
297
+ - `learning_rate`: 2e-05
298
+ - `num_train_epochs`: 1
299
+ - `warmup_ratio`: 0.1
300
+ - `seed`: 12
301
+ - `bf16`: True
302
+ - `dataloader_num_workers`: 4
303
+ - `load_best_model_at_end`: True
304
+
305
+ #### All Hyperparameters
306
+ <details><summary>Click to expand</summary>
307
+
308
+ - `overwrite_output_dir`: False
309
+ - `do_predict`: False
310
+ - `eval_strategy`: steps
311
+ - `prediction_loss_only`: True
312
+ - `per_device_train_batch_size`: 16
313
+ - `per_device_eval_batch_size`: 16
314
+ - `per_gpu_train_batch_size`: None
315
+ - `per_gpu_eval_batch_size`: None
316
+ - `gradient_accumulation_steps`: 1
317
+ - `eval_accumulation_steps`: None
318
+ - `torch_empty_cache_steps`: None
319
+ - `learning_rate`: 2e-05
320
+ - `weight_decay`: 0.0
321
+ - `adam_beta1`: 0.9
322
+ - `adam_beta2`: 0.999
323
+ - `adam_epsilon`: 1e-08
324
+ - `max_grad_norm`: 1.0
325
+ - `num_train_epochs`: 1
326
+ - `max_steps`: -1
327
+ - `lr_scheduler_type`: linear
328
+ - `lr_scheduler_kwargs`: {}
329
+ - `warmup_ratio`: 0.1
330
+ - `warmup_steps`: 0
331
+ - `log_level`: passive
332
+ - `log_level_replica`: warning
333
+ - `log_on_each_node`: True
334
+ - `logging_nan_inf_filter`: True
335
+ - `save_safetensors`: True
336
+ - `save_on_each_node`: False
337
+ - `save_only_model`: False
338
+ - `restore_callback_states_from_checkpoint`: False
339
+ - `no_cuda`: False
340
+ - `use_cpu`: False
341
+ - `use_mps_device`: False
342
+ - `seed`: 12
343
+ - `data_seed`: None
344
+ - `jit_mode_eval`: False
345
+ - `use_ipex`: False
346
+ - `bf16`: True
347
+ - `fp16`: False
348
+ - `fp16_opt_level`: O1
349
+ - `half_precision_backend`: auto
350
+ - `bf16_full_eval`: False
351
+ - `fp16_full_eval`: False
352
+ - `tf32`: None
353
+ - `local_rank`: 0
354
+ - `ddp_backend`: None
355
+ - `tpu_num_cores`: None
356
+ - `tpu_metrics_debug`: False
357
+ - `debug`: []
358
+ - `dataloader_drop_last`: False
359
+ - `dataloader_num_workers`: 4
360
+ - `dataloader_prefetch_factor`: None
361
+ - `past_index`: -1
362
+ - `disable_tqdm`: False
363
+ - `remove_unused_columns`: True
364
+ - `label_names`: None
365
+ - `load_best_model_at_end`: True
366
+ - `ignore_data_skip`: False
367
+ - `fsdp`: []
368
+ - `fsdp_min_num_params`: 0
369
+ - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
370
+ - `tp_size`: 0
371
+ - `fsdp_transformer_layer_cls_to_wrap`: None
372
+ - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
373
+ - `deepspeed`: None
374
+ - `label_smoothing_factor`: 0.0
375
+ - `optim`: adamw_torch
376
+ - `optim_args`: None
377
+ - `adafactor`: False
378
+ - `group_by_length`: False
379
+ - `length_column_name`: length
380
+ - `ddp_find_unused_parameters`: None
381
+ - `ddp_bucket_cap_mb`: None
382
+ - `ddp_broadcast_buffers`: False
383
+ - `dataloader_pin_memory`: True
384
+ - `dataloader_persistent_workers`: False
385
+ - `skip_memory_metrics`: True
386
+ - `use_legacy_prediction_loop`: False
387
+ - `push_to_hub`: False
388
+ - `resume_from_checkpoint`: None
389
+ - `hub_model_id`: None
390
+ - `hub_strategy`: every_save
391
+ - `hub_private_repo`: None
392
+ - `hub_always_push`: False
393
+ - `gradient_checkpointing`: False
394
+ - `gradient_checkpointing_kwargs`: None
395
+ - `include_inputs_for_metrics`: False
396
+ - `include_for_metrics`: []
397
+ - `eval_do_concat_batches`: True
398
+ - `fp16_backend`: auto
399
+ - `push_to_hub_model_id`: None
400
+ - `push_to_hub_organization`: None
401
+ - `mp_parameters`:
402
+ - `auto_find_batch_size`: False
403
+ - `full_determinism`: False
404
+ - `torchdynamo`: None
405
+ - `ray_scope`: last
406
+ - `ddp_timeout`: 1800
407
+ - `torch_compile`: False
408
+ - `torch_compile_backend`: None
409
+ - `torch_compile_mode`: None
410
+ - `include_tokens_per_second`: False
411
+ - `include_num_input_tokens_seen`: False
412
+ - `neftune_noise_alpha`: None
413
+ - `optim_target_modules`: None
414
+ - `batch_eval_metrics`: False
415
+ - `eval_on_start`: False
416
+ - `use_liger_kernel`: False
417
+ - `eval_use_gather_object`: False
418
+ - `average_tokens_across_devices`: False
419
+ - `prompts`: None
420
+ - `batch_sampler`: batch_sampler
421
+ - `multi_dataset_batch_sampler`: proportional
422
+
423
+ </details>
424
+
425
+ ### Training Logs
426
+ | Epoch | Step | Training Loss | gooaq-dev_ndcg@10 | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10 | NanoBEIR_R100_mean_ndcg@10 |
427
+ |:----------:|:---------:|:-------------:|:--------------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
428
+ | -1 | -1 | - | 0.1374 (-0.4539) | 0.0392 (-0.5012) | 0.2824 (-0.0427) | 0.0371 (-0.4635) | 0.1196 (-0.3358) |
429
+ | 0.0000 | 1 | 1.3234 | - | - | - | - | - |
430
+ | 0.0277 | 1000 | 1.1925 | - | - | - | - | - |
431
+ | 0.0553 | 2000 | 0.9574 | - | - | - | - | - |
432
+ | 0.0830 | 3000 | 0.7563 | - | - | - | - | - |
433
+ | 0.1106 | 4000 | 0.7247 | 0.7171 (+0.1259) | 0.5045 (-0.0359) | 0.3093 (-0.0158) | 0.5701 (+0.0694) | 0.4613 (+0.0059) |
434
+ | 0.1383 | 5000 | 0.6744 | - | - | - | - | - |
435
+ | 0.1660 | 6000 | 0.6876 | - | - | - | - | - |
436
+ | 0.1936 | 7000 | 0.6446 | - | - | - | - | - |
437
+ | 0.2213 | 8000 | 0.6519 | 0.7319 (+0.1407) | 0.5106 (-0.0298) | 0.3346 (+0.0095) | 0.6054 (+0.1048) | 0.4835 (+0.0282) |
438
+ | 0.2490 | 9000 | 0.6339 | - | - | - | - | - |
439
+ | 0.2766 | 10000 | 0.6168 | - | - | - | - | - |
440
+ | 0.3043 | 11000 | 0.6005 | - | - | - | - | - |
441
+ | 0.3319 | 12000 | 0.6363 | 0.7463 (+0.1550) | 0.5087 (-0.0317) | 0.3075 (-0.0176) | 0.5757 (+0.0751) | 0.4640 (+0.0086) |
442
+ | 0.3596 | 13000 | 0.5882 | - | - | - | - | - |
443
+ | 0.3873 | 14000 | 0.5888 | - | - | - | - | - |
444
+ | 0.4149 | 15000 | 0.5824 | - | - | - | - | - |
445
+ | 0.4426 | 16000 | 0.5882 | 0.7487 (+0.1574) | 0.5277 (-0.0127) | 0.3215 (-0.0036) | 0.5609 (+0.0603) | 0.4701 (+0.0147) |
446
+ | 0.4702 | 17000 | 0.5661 | - | - | - | - | - |
447
+ | 0.4979 | 18000 | 0.5758 | - | - | - | - | - |
448
+ | 0.5256 | 19000 | 0.556 | - | - | - | - | - |
449
+ | 0.5532 | 20000 | 0.5524 | 0.7556 (+0.1644) | 0.5419 (+0.0014) | 0.3191 (-0.0059) | 0.6132 (+0.1125) | 0.4914 (+0.0360) |
450
+ | 0.5809 | 21000 | 0.5546 | - | - | - | - | - |
451
+ | 0.6086 | 22000 | 0.563 | - | - | - | - | - |
452
+ | 0.6362 | 23000 | 0.5369 | - | - | - | - | - |
453
+ | 0.6639 | 24000 | 0.5492 | 0.7570 (+0.1658) | 0.4779 (-0.0626) | 0.2881 (-0.0369) | 0.5079 (+0.0073) | 0.4246 (-0.0307) |
454
+ | 0.6915 | 25000 | 0.5443 | - | - | - | - | - |
455
+ | 0.7192 | 26000 | 0.5522 | - | - | - | - | - |
456
+ | 0.7469 | 27000 | 0.5323 | - | - | - | - | - |
457
+ | 0.7745 | 28000 | 0.5167 | 0.7695 (+0.1782) | 0.4941 (-0.0463) | 0.3167 (-0.0083) | 0.5194 (+0.0188) | 0.4434 (-0.0119) |
458
+ | 0.8022 | 29000 | 0.4998 | - | - | - | - | - |
459
+ | 0.8299 | 30000 | 0.5326 | - | - | - | - | - |
460
+ | 0.8575 | 31000 | 0.5262 | - | - | - | - | - |
461
+ | 0.8852 | 32000 | 0.5136 | 0.7682 (+0.1769) | 0.5004 (-0.0400) | 0.3273 (+0.0023) | 0.5318 (+0.0311) | 0.4531 (-0.0022) |
462
+ | 0.9128 | 33000 | 0.5145 | - | - | - | - | - |
463
+ | 0.9405 | 34000 | 0.5038 | - | - | - | - | - |
464
+ | 0.9682 | 35000 | 0.5236 | - | - | - | - | - |
465
+ | **0.9958** | **36000** | **0.5347** | **0.7716 (+0.1804)** | **0.5096 (-0.0309)** | **0.3176 (-0.0074)** | **0.5388 (+0.0382)** | **0.4553 (-0.0000)** |
466
+ | -1 | -1 | - | 0.7716 (+0.1804) | 0.5096 (-0.0309) | 0.3176 (-0.0074) | 0.5388 (+0.0382) | 0.4553 (-0.0000) |
467
+
468
+ * The bold row denotes the saved checkpoint.
469
+
470
+ ### Framework Versions
471
+ - Python: 3.10.10
472
+ - Sentence Transformers: 4.1.0
473
+ - Transformers: 4.51.3
474
+ - PyTorch: 2.7.0+cu128
475
+ - Accelerate: 1.7.0
476
+ - Datasets: 3.6.0
477
+ - Tokenizers: 0.21.1
478
+
479
+ ## Citation
480
+
481
+ ### BibTeX
482
+
483
+ #### Sentence Transformers
484
+ ```bibtex
485
+ @inproceedings{reimers-2019-sentence-bert,
486
+ title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
487
+ author = "Reimers, Nils and Gurevych, Iryna",
488
+ booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
489
+ month = "11",
490
+ year = "2019",
491
+ publisher = "Association for Computational Linguistics",
492
+ url = "https://arxiv.org/abs/1908.10084",
493
+ }
494
+ ```
495
+
496
+ <!--
497
+ ## Glossary
498
+
499
+ *Clearly define terms in order to be accessible across audiences.*
500
+ -->
501
+
502
+ <!--
503
+ ## Model Card Authors
504
+
505
+ *Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
506
+ -->
507
+
508
+ <!--
509
+ ## Model Card Contact
510
+
511
+ *Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
512
+ -->
config.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "architectures": [
3
+ "ModernBertForSequenceClassification"
4
+ ],
5
+ "attention_bias": false,
6
+ "attention_dropout": 0.0,
7
+ "bos_token_id": 50281,
8
+ "classifier_activation": "gelu",
9
+ "classifier_bias": false,
10
+ "classifier_dropout": 0.0,
11
+ "classifier_pooling": "mean",
12
+ "cls_token_id": 50281,
13
+ "decoder_bias": true,
14
+ "deterministic_flash_attn": false,
15
+ "embedding_dropout": 0.0,
16
+ "eos_token_id": 50282,
17
+ "global_attn_every_n_layers": 3,
18
+ "global_rope_theta": 160000.0,
19
+ "gradient_checkpointing": false,
20
+ "hidden_activation": "gelu",
21
+ "hidden_size": 768,
22
+ "id2label": {
23
+ "0": "LABEL_0"
24
+ },
25
+ "initializer_cutoff_factor": 2.0,
26
+ "initializer_range": 0.02,
27
+ "intermediate_size": 1152,
28
+ "label2id": {
29
+ "LABEL_0": 0
30
+ },
31
+ "layer_norm_eps": 1e-05,
32
+ "local_attention": 128,
33
+ "local_rope_theta": 10000.0,
34
+ "max_position_embeddings": 8192,
35
+ "mlp_bias": false,
36
+ "mlp_dropout": 0.0,
37
+ "model_type": "modernbert",
38
+ "norm_bias": false,
39
+ "norm_eps": 1e-05,
40
+ "num_attention_heads": 12,
41
+ "num_hidden_layers": 22,
42
+ "pad_token_id": 50283,
43
+ "position_embedding_type": "absolute",
44
+ "repad_logits_with_grad": false,
45
+ "sentence_transformers": {
46
+ "activation_fn": "torch.nn.modules.activation.Sigmoid",
47
+ "version": "4.1.0"
48
+ },
49
+ "sep_token_id": 50282,
50
+ "sparse_pred_ignore_index": -100,
51
+ "sparse_prediction": false,
52
+ "torch_dtype": "float32",
53
+ "transformers_version": "4.51.3",
54
+ "vocab_size": 50368
55
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:01d9990def24b460c191e27e4d4dbe2cc8b54e4a80dab851eae8c48364c3d2cf
3
+ size 598436708
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": true,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,945 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "|||IP_ADDRESS|||",
5
+ "lstrip": false,
6
+ "normalized": true,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": false
10
+ },
11
+ "1": {
12
+ "content": "<|padding|>",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "50254": {
20
+ "content": " ",
21
+ "lstrip": false,
22
+ "normalized": true,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": false
26
+ },
27
+ "50255": {
28
+ "content": " ",
29
+ "lstrip": false,
30
+ "normalized": true,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": false
34
+ },
35
+ "50256": {
36
+ "content": " ",
37
+ "lstrip": false,
38
+ "normalized": true,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": false
42
+ },
43
+ "50257": {
44
+ "content": " ",
45
+ "lstrip": false,
46
+ "normalized": true,
47
+ "rstrip": false,
48
+ "single_word": false,
49
+ "special": false
50
+ },
51
+ "50258": {
52
+ "content": " ",
53
+ "lstrip": false,
54
+ "normalized": true,
55
+ "rstrip": false,
56
+ "single_word": false,
57
+ "special": false
58
+ },
59
+ "50259": {
60
+ "content": " ",
61
+ "lstrip": false,
62
+ "normalized": true,
63
+ "rstrip": false,
64
+ "single_word": false,
65
+ "special": false
66
+ },
67
+ "50260": {
68
+ "content": " ",
69
+ "lstrip": false,
70
+ "normalized": true,
71
+ "rstrip": false,
72
+ "single_word": false,
73
+ "special": false
74
+ },
75
+ "50261": {
76
+ "content": " ",
77
+ "lstrip": false,
78
+ "normalized": true,
79
+ "rstrip": false,
80
+ "single_word": false,
81
+ "special": false
82
+ },
83
+ "50262": {
84
+ "content": " ",
85
+ "lstrip": false,
86
+ "normalized": true,
87
+ "rstrip": false,
88
+ "single_word": false,
89
+ "special": false
90
+ },
91
+ "50263": {
92
+ "content": " ",
93
+ "lstrip": false,
94
+ "normalized": true,
95
+ "rstrip": false,
96
+ "single_word": false,
97
+ "special": false
98
+ },
99
+ "50264": {
100
+ "content": " ",
101
+ "lstrip": false,
102
+ "normalized": true,
103
+ "rstrip": false,
104
+ "single_word": false,
105
+ "special": false
106
+ },
107
+ "50265": {
108
+ "content": " ",
109
+ "lstrip": false,
110
+ "normalized": true,
111
+ "rstrip": false,
112
+ "single_word": false,
113
+ "special": false
114
+ },
115
+ "50266": {
116
+ "content": " ",
117
+ "lstrip": false,
118
+ "normalized": true,
119
+ "rstrip": false,
120
+ "single_word": false,
121
+ "special": false
122
+ },
123
+ "50267": {
124
+ "content": " ",
125
+ "lstrip": false,
126
+ "normalized": true,
127
+ "rstrip": false,
128
+ "single_word": false,
129
+ "special": false
130
+ },
131
+ "50268": {
132
+ "content": " ",
133
+ "lstrip": false,
134
+ "normalized": true,
135
+ "rstrip": false,
136
+ "single_word": false,
137
+ "special": false
138
+ },
139
+ "50269": {
140
+ "content": " ",
141
+ "lstrip": false,
142
+ "normalized": true,
143
+ "rstrip": false,
144
+ "single_word": false,
145
+ "special": false
146
+ },
147
+ "50270": {
148
+ "content": " ",
149
+ "lstrip": false,
150
+ "normalized": true,
151
+ "rstrip": false,
152
+ "single_word": false,
153
+ "special": false
154
+ },
155
+ "50271": {
156
+ "content": " ",
157
+ "lstrip": false,
158
+ "normalized": true,
159
+ "rstrip": false,
160
+ "single_word": false,
161
+ "special": false
162
+ },
163
+ "50272": {
164
+ "content": " ",
165
+ "lstrip": false,
166
+ "normalized": true,
167
+ "rstrip": false,
168
+ "single_word": false,
169
+ "special": false
170
+ },
171
+ "50273": {
172
+ "content": " ",
173
+ "lstrip": false,
174
+ "normalized": true,
175
+ "rstrip": false,
176
+ "single_word": false,
177
+ "special": false
178
+ },
179
+ "50274": {
180
+ "content": " ",
181
+ "lstrip": false,
182
+ "normalized": true,
183
+ "rstrip": false,
184
+ "single_word": false,
185
+ "special": false
186
+ },
187
+ "50275": {
188
+ "content": " ",
189
+ "lstrip": false,
190
+ "normalized": true,
191
+ "rstrip": false,
192
+ "single_word": false,
193
+ "special": false
194
+ },
195
+ "50276": {
196
+ "content": " ",
197
+ "lstrip": false,
198
+ "normalized": true,
199
+ "rstrip": false,
200
+ "single_word": false,
201
+ "special": false
202
+ },
203
+ "50277": {
204
+ "content": "|||EMAIL_ADDRESS|||",
205
+ "lstrip": false,
206
+ "normalized": true,
207
+ "rstrip": false,
208
+ "single_word": false,
209
+ "special": false
210
+ },
211
+ "50278": {
212
+ "content": "|||PHONE_NUMBER|||",
213
+ "lstrip": false,
214
+ "normalized": true,
215
+ "rstrip": false,
216
+ "single_word": false,
217
+ "special": false
218
+ },
219
+ "50279": {
220
+ "content": "<|endoftext|>",
221
+ "lstrip": false,
222
+ "normalized": false,
223
+ "rstrip": false,
224
+ "single_word": false,
225
+ "special": true
226
+ },
227
+ "50280": {
228
+ "content": "[UNK]",
229
+ "lstrip": false,
230
+ "normalized": false,
231
+ "rstrip": false,
232
+ "single_word": false,
233
+ "special": true
234
+ },
235
+ "50281": {
236
+ "content": "[CLS]",
237
+ "lstrip": false,
238
+ "normalized": false,
239
+ "rstrip": false,
240
+ "single_word": false,
241
+ "special": true
242
+ },
243
+ "50282": {
244
+ "content": "[SEP]",
245
+ "lstrip": false,
246
+ "normalized": false,
247
+ "rstrip": false,
248
+ "single_word": false,
249
+ "special": true
250
+ },
251
+ "50283": {
252
+ "content": "[PAD]",
253
+ "lstrip": false,
254
+ "normalized": false,
255
+ "rstrip": false,
256
+ "single_word": false,
257
+ "special": true
258
+ },
259
+ "50284": {
260
+ "content": "[MASK]",
261
+ "lstrip": true,
262
+ "normalized": false,
263
+ "rstrip": false,
264
+ "single_word": false,
265
+ "special": true
266
+ },
267
+ "50285": {
268
+ "content": "[unused0]",
269
+ "lstrip": false,
270
+ "normalized": true,
271
+ "rstrip": false,
272
+ "single_word": false,
273
+ "special": false
274
+ },
275
+ "50286": {
276
+ "content": "[unused1]",
277
+ "lstrip": false,
278
+ "normalized": true,
279
+ "rstrip": false,
280
+ "single_word": false,
281
+ "special": false
282
+ },
283
+ "50287": {
284
+ "content": "[unused2]",
285
+ "lstrip": false,
286
+ "normalized": true,
287
+ "rstrip": false,
288
+ "single_word": false,
289
+ "special": false
290
+ },
291
+ "50288": {
292
+ "content": "[unused3]",
293
+ "lstrip": false,
294
+ "normalized": true,
295
+ "rstrip": false,
296
+ "single_word": false,
297
+ "special": false
298
+ },
299
+ "50289": {
300
+ "content": "[unused4]",
301
+ "lstrip": false,
302
+ "normalized": true,
303
+ "rstrip": false,
304
+ "single_word": false,
305
+ "special": false
306
+ },
307
+ "50290": {
308
+ "content": "[unused5]",
309
+ "lstrip": false,
310
+ "normalized": true,
311
+ "rstrip": false,
312
+ "single_word": false,
313
+ "special": false
314
+ },
315
+ "50291": {
316
+ "content": "[unused6]",
317
+ "lstrip": false,
318
+ "normalized": true,
319
+ "rstrip": false,
320
+ "single_word": false,
321
+ "special": false
322
+ },
323
+ "50292": {
324
+ "content": "[unused7]",
325
+ "lstrip": false,
326
+ "normalized": true,
327
+ "rstrip": false,
328
+ "single_word": false,
329
+ "special": false
330
+ },
331
+ "50293": {
332
+ "content": "[unused8]",
333
+ "lstrip": false,
334
+ "normalized": true,
335
+ "rstrip": false,
336
+ "single_word": false,
337
+ "special": false
338
+ },
339
+ "50294": {
340
+ "content": "[unused9]",
341
+ "lstrip": false,
342
+ "normalized": true,
343
+ "rstrip": false,
344
+ "single_word": false,
345
+ "special": false
346
+ },
347
+ "50295": {
348
+ "content": "[unused10]",
349
+ "lstrip": false,
350
+ "normalized": true,
351
+ "rstrip": false,
352
+ "single_word": false,
353
+ "special": false
354
+ },
355
+ "50296": {
356
+ "content": "[unused11]",
357
+ "lstrip": false,
358
+ "normalized": true,
359
+ "rstrip": false,
360
+ "single_word": false,
361
+ "special": false
362
+ },
363
+ "50297": {
364
+ "content": "[unused12]",
365
+ "lstrip": false,
366
+ "normalized": true,
367
+ "rstrip": false,
368
+ "single_word": false,
369
+ "special": false
370
+ },
371
+ "50298": {
372
+ "content": "[unused13]",
373
+ "lstrip": false,
374
+ "normalized": true,
375
+ "rstrip": false,
376
+ "single_word": false,
377
+ "special": false
378
+ },
379
+ "50299": {
380
+ "content": "[unused14]",
381
+ "lstrip": false,
382
+ "normalized": true,
383
+ "rstrip": false,
384
+ "single_word": false,
385
+ "special": false
386
+ },
387
+ "50300": {
388
+ "content": "[unused15]",
389
+ "lstrip": false,
390
+ "normalized": true,
391
+ "rstrip": false,
392
+ "single_word": false,
393
+ "special": false
394
+ },
395
+ "50301": {
396
+ "content": "[unused16]",
397
+ "lstrip": false,
398
+ "normalized": true,
399
+ "rstrip": false,
400
+ "single_word": false,
401
+ "special": false
402
+ },
403
+ "50302": {
404
+ "content": "[unused17]",
405
+ "lstrip": false,
406
+ "normalized": true,
407
+ "rstrip": false,
408
+ "single_word": false,
409
+ "special": false
410
+ },
411
+ "50303": {
412
+ "content": "[unused18]",
413
+ "lstrip": false,
414
+ "normalized": true,
415
+ "rstrip": false,
416
+ "single_word": false,
417
+ "special": false
418
+ },
419
+ "50304": {
420
+ "content": "[unused19]",
421
+ "lstrip": false,
422
+ "normalized": true,
423
+ "rstrip": false,
424
+ "single_word": false,
425
+ "special": false
426
+ },
427
+ "50305": {
428
+ "content": "[unused20]",
429
+ "lstrip": false,
430
+ "normalized": true,
431
+ "rstrip": false,
432
+ "single_word": false,
433
+ "special": false
434
+ },
435
+ "50306": {
436
+ "content": "[unused21]",
437
+ "lstrip": false,
438
+ "normalized": true,
439
+ "rstrip": false,
440
+ "single_word": false,
441
+ "special": false
442
+ },
443
+ "50307": {
444
+ "content": "[unused22]",
445
+ "lstrip": false,
446
+ "normalized": true,
447
+ "rstrip": false,
448
+ "single_word": false,
449
+ "special": false
450
+ },
451
+ "50308": {
452
+ "content": "[unused23]",
453
+ "lstrip": false,
454
+ "normalized": true,
455
+ "rstrip": false,
456
+ "single_word": false,
457
+ "special": false
458
+ },
459
+ "50309": {
460
+ "content": "[unused24]",
461
+ "lstrip": false,
462
+ "normalized": true,
463
+ "rstrip": false,
464
+ "single_word": false,
465
+ "special": false
466
+ },
467
+ "50310": {
468
+ "content": "[unused25]",
469
+ "lstrip": false,
470
+ "normalized": true,
471
+ "rstrip": false,
472
+ "single_word": false,
473
+ "special": false
474
+ },
475
+ "50311": {
476
+ "content": "[unused26]",
477
+ "lstrip": false,
478
+ "normalized": true,
479
+ "rstrip": false,
480
+ "single_word": false,
481
+ "special": false
482
+ },
483
+ "50312": {
484
+ "content": "[unused27]",
485
+ "lstrip": false,
486
+ "normalized": true,
487
+ "rstrip": false,
488
+ "single_word": false,
489
+ "special": false
490
+ },
491
+ "50313": {
492
+ "content": "[unused28]",
493
+ "lstrip": false,
494
+ "normalized": true,
495
+ "rstrip": false,
496
+ "single_word": false,
497
+ "special": false
498
+ },
499
+ "50314": {
500
+ "content": "[unused29]",
501
+ "lstrip": false,
502
+ "normalized": true,
503
+ "rstrip": false,
504
+ "single_word": false,
505
+ "special": false
506
+ },
507
+ "50315": {
508
+ "content": "[unused30]",
509
+ "lstrip": false,
510
+ "normalized": true,
511
+ "rstrip": false,
512
+ "single_word": false,
513
+ "special": false
514
+ },
515
+ "50316": {
516
+ "content": "[unused31]",
517
+ "lstrip": false,
518
+ "normalized": true,
519
+ "rstrip": false,
520
+ "single_word": false,
521
+ "special": false
522
+ },
523
+ "50317": {
524
+ "content": "[unused32]",
525
+ "lstrip": false,
526
+ "normalized": true,
527
+ "rstrip": false,
528
+ "single_word": false,
529
+ "special": false
530
+ },
531
+ "50318": {
532
+ "content": "[unused33]",
533
+ "lstrip": false,
534
+ "normalized": true,
535
+ "rstrip": false,
536
+ "single_word": false,
537
+ "special": false
538
+ },
539
+ "50319": {
540
+ "content": "[unused34]",
541
+ "lstrip": false,
542
+ "normalized": true,
543
+ "rstrip": false,
544
+ "single_word": false,
545
+ "special": false
546
+ },
547
+ "50320": {
548
+ "content": "[unused35]",
549
+ "lstrip": false,
550
+ "normalized": true,
551
+ "rstrip": false,
552
+ "single_word": false,
553
+ "special": false
554
+ },
555
+ "50321": {
556
+ "content": "[unused36]",
557
+ "lstrip": false,
558
+ "normalized": true,
559
+ "rstrip": false,
560
+ "single_word": false,
561
+ "special": false
562
+ },
563
+ "50322": {
564
+ "content": "[unused37]",
565
+ "lstrip": false,
566
+ "normalized": true,
567
+ "rstrip": false,
568
+ "single_word": false,
569
+ "special": false
570
+ },
571
+ "50323": {
572
+ "content": "[unused38]",
573
+ "lstrip": false,
574
+ "normalized": true,
575
+ "rstrip": false,
576
+ "single_word": false,
577
+ "special": false
578
+ },
579
+ "50324": {
580
+ "content": "[unused39]",
581
+ "lstrip": false,
582
+ "normalized": true,
583
+ "rstrip": false,
584
+ "single_word": false,
585
+ "special": false
586
+ },
587
+ "50325": {
588
+ "content": "[unused40]",
589
+ "lstrip": false,
590
+ "normalized": true,
591
+ "rstrip": false,
592
+ "single_word": false,
593
+ "special": false
594
+ },
595
+ "50326": {
596
+ "content": "[unused41]",
597
+ "lstrip": false,
598
+ "normalized": true,
599
+ "rstrip": false,
600
+ "single_word": false,
601
+ "special": false
602
+ },
603
+ "50327": {
604
+ "content": "[unused42]",
605
+ "lstrip": false,
606
+ "normalized": true,
607
+ "rstrip": false,
608
+ "single_word": false,
609
+ "special": false
610
+ },
611
+ "50328": {
612
+ "content": "[unused43]",
613
+ "lstrip": false,
614
+ "normalized": true,
615
+ "rstrip": false,
616
+ "single_word": false,
617
+ "special": false
618
+ },
619
+ "50329": {
620
+ "content": "[unused44]",
621
+ "lstrip": false,
622
+ "normalized": true,
623
+ "rstrip": false,
624
+ "single_word": false,
625
+ "special": false
626
+ },
627
+ "50330": {
628
+ "content": "[unused45]",
629
+ "lstrip": false,
630
+ "normalized": true,
631
+ "rstrip": false,
632
+ "single_word": false,
633
+ "special": false
634
+ },
635
+ "50331": {
636
+ "content": "[unused46]",
637
+ "lstrip": false,
638
+ "normalized": true,
639
+ "rstrip": false,
640
+ "single_word": false,
641
+ "special": false
642
+ },
643
+ "50332": {
644
+ "content": "[unused47]",
645
+ "lstrip": false,
646
+ "normalized": true,
647
+ "rstrip": false,
648
+ "single_word": false,
649
+ "special": false
650
+ },
651
+ "50333": {
652
+ "content": "[unused48]",
653
+ "lstrip": false,
654
+ "normalized": true,
655
+ "rstrip": false,
656
+ "single_word": false,
657
+ "special": false
658
+ },
659
+ "50334": {
660
+ "content": "[unused49]",
661
+ "lstrip": false,
662
+ "normalized": true,
663
+ "rstrip": false,
664
+ "single_word": false,
665
+ "special": false
666
+ },
667
+ "50335": {
668
+ "content": "[unused50]",
669
+ "lstrip": false,
670
+ "normalized": true,
671
+ "rstrip": false,
672
+ "single_word": false,
673
+ "special": false
674
+ },
675
+ "50336": {
676
+ "content": "[unused51]",
677
+ "lstrip": false,
678
+ "normalized": true,
679
+ "rstrip": false,
680
+ "single_word": false,
681
+ "special": false
682
+ },
683
+ "50337": {
684
+ "content": "[unused52]",
685
+ "lstrip": false,
686
+ "normalized": true,
687
+ "rstrip": false,
688
+ "single_word": false,
689
+ "special": false
690
+ },
691
+ "50338": {
692
+ "content": "[unused53]",
693
+ "lstrip": false,
694
+ "normalized": true,
695
+ "rstrip": false,
696
+ "single_word": false,
697
+ "special": false
698
+ },
699
+ "50339": {
700
+ "content": "[unused54]",
701
+ "lstrip": false,
702
+ "normalized": true,
703
+ "rstrip": false,
704
+ "single_word": false,
705
+ "special": false
706
+ },
707
+ "50340": {
708
+ "content": "[unused55]",
709
+ "lstrip": false,
710
+ "normalized": true,
711
+ "rstrip": false,
712
+ "single_word": false,
713
+ "special": false
714
+ },
715
+ "50341": {
716
+ "content": "[unused56]",
717
+ "lstrip": false,
718
+ "normalized": true,
719
+ "rstrip": false,
720
+ "single_word": false,
721
+ "special": false
722
+ },
723
+ "50342": {
724
+ "content": "[unused57]",
725
+ "lstrip": false,
726
+ "normalized": true,
727
+ "rstrip": false,
728
+ "single_word": false,
729
+ "special": false
730
+ },
731
+ "50343": {
732
+ "content": "[unused58]",
733
+ "lstrip": false,
734
+ "normalized": true,
735
+ "rstrip": false,
736
+ "single_word": false,
737
+ "special": false
738
+ },
739
+ "50344": {
740
+ "content": "[unused59]",
741
+ "lstrip": false,
742
+ "normalized": true,
743
+ "rstrip": false,
744
+ "single_word": false,
745
+ "special": false
746
+ },
747
+ "50345": {
748
+ "content": "[unused60]",
749
+ "lstrip": false,
750
+ "normalized": true,
751
+ "rstrip": false,
752
+ "single_word": false,
753
+ "special": false
754
+ },
755
+ "50346": {
756
+ "content": "[unused61]",
757
+ "lstrip": false,
758
+ "normalized": true,
759
+ "rstrip": false,
760
+ "single_word": false,
761
+ "special": false
762
+ },
763
+ "50347": {
764
+ "content": "[unused62]",
765
+ "lstrip": false,
766
+ "normalized": true,
767
+ "rstrip": false,
768
+ "single_word": false,
769
+ "special": false
770
+ },
771
+ "50348": {
772
+ "content": "[unused63]",
773
+ "lstrip": false,
774
+ "normalized": true,
775
+ "rstrip": false,
776
+ "single_word": false,
777
+ "special": false
778
+ },
779
+ "50349": {
780
+ "content": "[unused64]",
781
+ "lstrip": false,
782
+ "normalized": true,
783
+ "rstrip": false,
784
+ "single_word": false,
785
+ "special": false
786
+ },
787
+ "50350": {
788
+ "content": "[unused65]",
789
+ "lstrip": false,
790
+ "normalized": true,
791
+ "rstrip": false,
792
+ "single_word": false,
793
+ "special": false
794
+ },
795
+ "50351": {
796
+ "content": "[unused66]",
797
+ "lstrip": false,
798
+ "normalized": true,
799
+ "rstrip": false,
800
+ "single_word": false,
801
+ "special": false
802
+ },
803
+ "50352": {
804
+ "content": "[unused67]",
805
+ "lstrip": false,
806
+ "normalized": true,
807
+ "rstrip": false,
808
+ "single_word": false,
809
+ "special": false
810
+ },
811
+ "50353": {
812
+ "content": "[unused68]",
813
+ "lstrip": false,
814
+ "normalized": true,
815
+ "rstrip": false,
816
+ "single_word": false,
817
+ "special": false
818
+ },
819
+ "50354": {
820
+ "content": "[unused69]",
821
+ "lstrip": false,
822
+ "normalized": true,
823
+ "rstrip": false,
824
+ "single_word": false,
825
+ "special": false
826
+ },
827
+ "50355": {
828
+ "content": "[unused70]",
829
+ "lstrip": false,
830
+ "normalized": true,
831
+ "rstrip": false,
832
+ "single_word": false,
833
+ "special": false
834
+ },
835
+ "50356": {
836
+ "content": "[unused71]",
837
+ "lstrip": false,
838
+ "normalized": true,
839
+ "rstrip": false,
840
+ "single_word": false,
841
+ "special": false
842
+ },
843
+ "50357": {
844
+ "content": "[unused72]",
845
+ "lstrip": false,
846
+ "normalized": true,
847
+ "rstrip": false,
848
+ "single_word": false,
849
+ "special": false
850
+ },
851
+ "50358": {
852
+ "content": "[unused73]",
853
+ "lstrip": false,
854
+ "normalized": true,
855
+ "rstrip": false,
856
+ "single_word": false,
857
+ "special": false
858
+ },
859
+ "50359": {
860
+ "content": "[unused74]",
861
+ "lstrip": false,
862
+ "normalized": true,
863
+ "rstrip": false,
864
+ "single_word": false,
865
+ "special": false
866
+ },
867
+ "50360": {
868
+ "content": "[unused75]",
869
+ "lstrip": false,
870
+ "normalized": true,
871
+ "rstrip": false,
872
+ "single_word": false,
873
+ "special": false
874
+ },
875
+ "50361": {
876
+ "content": "[unused76]",
877
+ "lstrip": false,
878
+ "normalized": true,
879
+ "rstrip": false,
880
+ "single_word": false,
881
+ "special": false
882
+ },
883
+ "50362": {
884
+ "content": "[unused77]",
885
+ "lstrip": false,
886
+ "normalized": true,
887
+ "rstrip": false,
888
+ "single_word": false,
889
+ "special": false
890
+ },
891
+ "50363": {
892
+ "content": "[unused78]",
893
+ "lstrip": false,
894
+ "normalized": true,
895
+ "rstrip": false,
896
+ "single_word": false,
897
+ "special": false
898
+ },
899
+ "50364": {
900
+ "content": "[unused79]",
901
+ "lstrip": false,
902
+ "normalized": true,
903
+ "rstrip": false,
904
+ "single_word": false,
905
+ "special": false
906
+ },
907
+ "50365": {
908
+ "content": "[unused80]",
909
+ "lstrip": false,
910
+ "normalized": true,
911
+ "rstrip": false,
912
+ "single_word": false,
913
+ "special": false
914
+ },
915
+ "50366": {
916
+ "content": "[unused81]",
917
+ "lstrip": false,
918
+ "normalized": true,
919
+ "rstrip": false,
920
+ "single_word": false,
921
+ "special": false
922
+ },
923
+ "50367": {
924
+ "content": "[unused82]",
925
+ "lstrip": false,
926
+ "normalized": true,
927
+ "rstrip": false,
928
+ "single_word": false,
929
+ "special": false
930
+ }
931
+ },
932
+ "clean_up_tokenization_spaces": true,
933
+ "cls_token": "[CLS]",
934
+ "extra_special_tokens": {},
935
+ "mask_token": "[MASK]",
936
+ "model_input_names": [
937
+ "input_ids",
938
+ "attention_mask"
939
+ ],
940
+ "model_max_length": 8192,
941
+ "pad_token": "[PAD]",
942
+ "sep_token": "[SEP]",
943
+ "tokenizer_class": "PreTrainedTokenizer",
944
+ "unk_token": "[UNK]"
945
+ }