Add new CrossEncoder model

README.md

@@ -0,0 +1,520 @@
+---
+language:
+- en
+tags:
+- sentence-transformers
+- cross-encoder
+- generated_from_trainer
+- dataset_size:78704
+- loss:ListNetLoss
+base_model: microsoft/MiniLM-L12-H384-uncased
+datasets:
+- microsoft/ms_marco
+pipeline_tag: text-ranking
+library_name: sentence-transformers
+metrics:
+- map
+- mrr@10
+- ndcg@10
+co2_eq_emissions:
+  emissions: 92.35489230616244
+  energy_consumed: 0.23759819168968113
+  source: codecarbon
+  training_type: fine-tuning
+  on_cloud: false
+  cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K
+  ram_total_size: 31.777088165283203
+  hours_used: 0.977
+  hardware_used: 1 x NVIDIA GeForce RTX 3090
+model-index:
+- name: CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+  results:
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoMSMARCO R100
+      type: NanoMSMARCO_R100
+    metrics:
+    - type: map
+      value: 0.506
+      name: Map
+    - type: mrr@10
+      value: 0.49
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.5497
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNFCorpus R100
+      type: NanoNFCorpus_R100
+    metrics:
+    - type: map
+      value: 0.3383
+      name: Map
+    - type: mrr@10
+      value: 0.5705
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.3736
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNQ R100
+      type: NanoNQ_R100
+    metrics:
+    - type: map
+      value: 0.5939
+      name: Map
+    - type: mrr@10
+      value: 0.6004
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.6574
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-nano-beir
+      name: Cross Encoder Nano BEIR
+    dataset:
+      name: NanoBEIR R100 mean
+      type: NanoBEIR_R100_mean
+    metrics:
+    - type: map
+      value: 0.4794
+      name: Map
+    - type: mrr@10
+      value: 0.5536
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.5269
+      name: Ndcg@10
+---
+
+# CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+
+This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) on the [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Cross Encoder
+- **Base model:** [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) <!-- at revision 44acabbec0ef496f6dbc93adadea57f376b7c0ec -->
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Output Labels:** 1 label
+- **Training Dataset:**
+    - [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco)
+- **Language:** en
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import CrossEncoder
+
+# Download from the 🤗 Hub
+model = CrossEncoder("tomaarsen/reranker-msmarco-v1.1-MiniLM-L12-H384-uncased-listnet-seeded")
+# Get scores for pairs of texts
+pairs = [
+    ['How many calories in an egg', 'There are on average between 55 and 80 calories in an egg depending on its size.'],
+    ['How many calories in an egg', 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.'],
+    ['How many calories in an egg', 'Most of the calories in an egg come from the yellow yolk in the center.'],
+]
+scores = model.predict(pairs)
+print(scores.shape)
+# (3,)
+
+# Or rank different texts based on similarity to a single text
+ranks = model.rank(
+    'How many calories in an egg',
+    [
+        'There are on average between 55 and 80 calories in an egg depending on its size.',
+        'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.',
+        'Most of the calories in an egg come from the yellow yolk in the center.',
+    ]
+)
+# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Cross Encoder Reranking
+
+* Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
+* Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
+  ```json
+  {
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | NanoMSMARCO_R100     | NanoNFCorpus_R100    | NanoNQ_R100          |
+|:------------|:---------------------|:---------------------|:---------------------|
+| map         | 0.5060 (+0.0164)     | 0.3383 (+0.0773)     | 0.5939 (+0.1743)     |
+| mrr@10      | 0.4900 (+0.0125)     | 0.5705 (+0.0707)     | 0.6004 (+0.1737)     |
+| **ndcg@10** | **0.5497 (+0.0093)** | **0.3736 (+0.0485)** | **0.6574 (+0.1568)** |
+
+#### Cross Encoder Nano BEIR
+
+* Dataset: `NanoBEIR_R100_mean`
+* Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
+  ```json
+  {
+      "dataset_names": [
+          "msmarco",
+          "nfcorpus",
+          "nq"
+      ],
+      "rerank_k": 100,
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | Value                |
+|:------------|:---------------------|
+| map         | 0.4794 (+0.0894)     |
+| mrr@10      | 0.5536 (+0.0856)     |
+| **ndcg@10** | **0.5269 (+0.0715)** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 78,704 training samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                          | docs                                                                                   | labels                                                                                 |
+  |:--------|:-----------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                         | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 11 characters</li><li>mean: 34.13 characters</li><li>max: 88 characters</li></ul> | <ul><li>min: 2 elements</li><li>mean: 6.00 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 2 elements</li><li>mean: 6.00 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                                     | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:----------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>food that are vegetarian that have vitamin a</code> | <code>['Vitamin A is a fat soluble vitamin, and therefore, needs to be consumed with fat in order to have optimal absorption. High vitamin A foods include sweet potatoes, carrots, dark leafy greens, winter squashes, lettuce, dried apricots, cantaloupe, bell peppers, fish, liver, and tropical fruits. The current daily value for Vitamin A is 5000 international units (IU).', 'Unlike some other B vitamins, B12 is not found in any plant food other than fortified cereals. It is, however, abundant in many meats and fish, and in smaller amounts in milk and eggs. This makes it difficult for people following a strict vegetarian diet to get the necessary amount of vitamin B12.', 'They found that 92% of the vegans they studied -- those who ate the strictest vegetarian diet, which shuns all animal products, including milk and eggs -- had vitamin B12 deficiency. But two in three people who followed a vegetarian diet that included milk and eggs as their only animal foods also were deficient.', 'Vitamin B 1...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>what is trilobar prostatic enlargement</code>       | <code>["Prostate enlargement: Most prostatic enlargement is due to benign prostatic hyperplasia (BPH), a problem that bothers men increasingly with advancing age. The process of BPH generally begins in a man's 30s, evolves very slowly and usually causes symptoms only after he has passed the half-century mark. It is not a precursor (a forerunner) to prostate cancer. Treatment of BPH is usually reserved for men with significant symptoms. Watchful waiting with medical monitoring once a year is appropriate for most men with BPH. The medical therapy of BPH includes medication.", '1 A benign (noncancerous) condition in which an overgrowth of prostate tissue pushes against the urethra and the bladder, blocking the flow of urine. 2  Increase in constituent cells in the prostate, leading to enlargement of the organ (hypertrophy) and adverse impact on the lower urinary tract function. 1 Increase in constituent cells in the prostate, leading to enlargement of the organ (hypertrophy) and adverse impact ...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>what is the classification of seasoning</code>      | <code>['Artificial kiln seasoning. Its the most traditional way of seasoning wood or timber. In this method wood is dryed usually by the keeping the wood exposed to air, so that the moisture evaporates and wood is seasoned. This method is very economical is a sense that no operational charges exists but the process is too slow & ..... [Read More].', 'Vegetables used in seasoning such as onions, garlic, and celery may also be included in this category in some circumstances. Some people break types of spices up by what one does when it is added to food. Sweet, hot, pungent, and tangy are the four primary categories. I think the most common herbs and spices are garlic, onions, and Italian spices like Oregano. Garlic and onion can also be found in salts and powders for simpler things like marinades and just for basic baking.', 'Spices and herbs at a grocery shop in Goa, India. A spice is a seed, fruit, root, bark, berry, bud or vegetable substance primarily used for flavoring, coloring or preser...</code> | <code>[1, 1, 1, 0, 0, ...]</code> |
+* Loss: [<code>ListNetLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#listnetloss) with these parameters:
+  ```json
+  {
+      "activation_fct": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": 16
+  }
+  ```
+
+### Evaluation Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 1,000 evaluation samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                          | docs                                                                                   | labels                                                                                 |
+  |:--------|:-----------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                         | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 11 characters</li><li>mean: 33.79 characters</li><li>max: 95 characters</li></ul> | <ul><li>min: 2 elements</li><li>mean: 6.00 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 2 elements</li><li>mean: 6.00 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                             | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:--------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>absolute viscosity definition</code>        | <code>['Noun. 1. absolute viscosity-a measure of the resistance to flow of a fluid under an applied force. coefficient of viscosity, dynamic viscosity. coefficient-a constant number that serves as a measure of some property or characteristic.', '1 the state or property of being viscous. 2 (Physics). a the extent to which a fluid resists a tendency to flow. b (Also called) absolute viscosity a measure of this resistance, equal to the tangential stress on a liquid undergoing streamline flow divided by its velocity gradient. It is measured in newton seconds per metre squared. , (Symbol) η.', 'Kinematic Viscosity. Kinematic viscosity is the ratio of-absolute (or dynamic) viscosity to density-a quantity in which no force is involved. Kinematic viscosity can be obtained by dividing the absolute viscosity of a fluid with the fluid mass density.', '2] = shear stress acted by fluid on lower surface of the blank element du = velocity of the blank element relative to blank holder and die surface [mu] =...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>meaning of chartered engineer</code>        | <code>['noun. ( 1 in Britain) an engineer who is registered with the Engineering Council as having the scientific and technical knowledge and practical experience to satisfy its professional requirements.', '1 Trends. ( 2 in Britain) an engineer who is registered with the Engineering Council as having the scientific and technical knowledge and practical experience to satisfy its professional requirements.', '1 (in Britain) an engineer who is registered with the Engineering Council as having the scientific and technical knowledge and practical experience to satisfy its professional requirements. 2  Abbreviation: CEng.', 'chartered engineer n (in Britain) an engineer who is registered with the Engineering Council as having the scientific and technical knowledge and practical experience to satisfy its professional requirements, (Abbrev.) CEng. chartered engineer.', 'chartered engineer. ( 1 in Britain) an engineer who is registered with the Engineering Council as having the scientific and techni...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>how much do personal assistants make</code> | <code>['States That Pay the Most. Without exception, the personal assistants wanting to earn the highest dollar amount per year should live on the East Coast. New York tops the list, with an annual salary range of over $66,000 per year or just over $31.00 per hour. PAs in Maryland make the least per year at nearly $59,000. According to the Bureau of Labor Statistics, as of 2013, executive assistants/secretaries earn the highest salaries at nearly $51,870 per year on average. Other highly trained assistants include legal and medical secretaries, who can expect to earn just over $45,000 and $33,000 respectively.', "Before an agreement on pay is reached, research the national and local averages for full time personal assistant pay. According to the US Bureau of Labor Statistics (BLS), an assistant in California made an average hourly rate of $27.01 as of May 2012, while the same position in Florida earned a rate of $20.60. Determine what your budget for an assistant is and compare that number t...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>ListNetLoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#listnetloss) with these parameters:
+  ```json
+  {
+      "activation_fct": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": 16
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 2e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `seed`: 12
+- `bf16`: True
+- `load_best_model_at_end`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 12
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch      | Step     | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10  | NanoBEIR_R100_mean_ndcg@10 |
+|:----------:|:--------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
+| -1         | -1       | -             | -               | 0.0300 (-0.5104)         | 0.2528 (-0.0723)          | 0.0168 (-0.4839)     | 0.0999 (-0.3555)           |
+| 0.0002     | 1        | 2.0665        | -               | -                        | -                         | -                    | -                          |
+| 0.0508     | 250      | 2.0907        | -               | -                        | -                         | -                    | -                          |
+| 0.1016     | 500      | 2.0889        | 2.0762          | 0.4880 (-0.0524)         | 0.3157 (-0.0094)          | 0.5145 (+0.0139)     | 0.4394 (-0.0160)           |
+| 0.1525     | 750      | 2.0817        | -               | -                        | -                         | -                    | -                          |
+| 0.2033     | 1000     | 2.0771        | 2.0739          | 0.5346 (-0.0058)         | 0.3581 (+0.0331)          | 0.5875 (+0.0869)     | 0.4934 (+0.0380)           |
+| 0.2541     | 1250     | 2.0813        | -               | -                        | -                         | -                    | -                          |
+| 0.3049     | 1500     | 2.073         | 2.0730          | 0.5088 (-0.0316)         | 0.3440 (+0.0189)          | 0.5719 (+0.0713)     | 0.4749 (+0.0195)           |
+| 0.3558     | 1750     | 2.0698        | -               | -                        | -                         | -                    | -                          |
+| 0.4066     | 2000     | 2.0752        | 2.0725          | 0.5421 (+0.0017)         | 0.3741 (+0.0490)          | 0.6318 (+0.1311)     | 0.5160 (+0.0606)           |
+| 0.4574     | 2250     | 2.073         | -               | -                        | -                         | -                    | -                          |
+| 0.5082     | 2500     | 2.0712        | 2.0725          | 0.5311 (-0.0094)         | 0.3506 (+0.0256)          | 0.6258 (+0.1252)     | 0.5025 (+0.0471)           |
+| 0.5591     | 2750     | 2.0682        | -               | -                        | -                         | -                    | -                          |
+| 0.6099     | 3000     | 2.0738        | 2.0727          | 0.5682 (+0.0277)         | 0.3634 (+0.0384)          | 0.6241 (+0.1235)     | 0.5186 (+0.0632)           |
+| 0.6607     | 3250     | 2.0702        | -               | -                        | -                         | -                    | -                          |
+| 0.7115     | 3500     | 2.0722        | 2.0721          | 0.5591 (+0.0187)         | 0.3563 (+0.0312)          | 0.6453 (+0.1446)     | 0.5202 (+0.0649)           |
+| 0.7624     | 3750     | 2.0714        | -               | -                        | -                         | -                    | -                          |
+| **0.8132** | **4000** | **2.0632**    | **2.0724**      | **0.5497 (+0.0093)**     | **0.3736 (+0.0485)**      | **0.6574 (+0.1568)** | **0.5269 (+0.0715)**       |
+| 0.8640     | 4250     | 2.0681        | -               | -                        | -                         | -                    | -                          |
+| 0.9148     | 4500     | 2.066         | 2.0720          | 0.5510 (+0.0106)         | 0.3718 (+0.0468)          | 0.6483 (+0.1476)     | 0.5237 (+0.0683)           |
+| 0.9656     | 4750     | 2.0736        | -               | -                        | -                         | -                    | -                          |
+| -1         | -1       | -             | -               | 0.5497 (+0.0093)         | 0.3736 (+0.0485)          | 0.6574 (+0.1568)     | 0.5269 (+0.0715)           |
+
+* The bold row denotes the saved checkpoint.
+
+### Environmental Impact
+Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon).
+- **Energy Consumed**: 0.238 kWh
+- **Carbon Emitted**: 0.092 kg of CO2
+- **Hours Used**: 0.977 hours
+
+### Training Hardware
+- **On Cloud**: No
+- **GPU Model**: 1 x NVIDIA GeForce RTX 3090
+- **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K
+- **RAM Size**: 31.78 GB
+
+### Framework Versions
+- Python: 3.11.6
+- Sentence Transformers: 3.5.0.dev0
+- Transformers: 4.49.0
+- PyTorch: 2.6.0+cu124
+- Accelerate: 1.5.1
+- Datasets: 3.3.2
+- Tokenizers: 0.21.0
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### ListNetLoss
+```bibtex
+@inproceedings{cao2007learning,
+    title={Learning to rank: from pairwise approach to listwise approach},
+    author={Cao, Zhe and Qin, Tao and Liu, Tie-Yan and Tsai, Ming-Feng and Li, Hang},
+    booktitle={Proceedings of the 24th international conference on Machine learning},
+    pages={129--136},
+    year={2007}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,34 @@
+{
+  "_name_or_path": "microsoft/MiniLM-L12-H384-uncased",
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "sentence_transformers": {
+    "activation_fn": "torch.nn.modules.activation.Sigmoid"
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2681c144a0130fccc27fc1b1d16e3cfc2ef64b60fc3fd36bd7943070d397bd91
+size 133464836

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}