Add new CrossEncoder model

README.md

@@ -0,0 +1,502 @@
+---
+language:
+- en
+tags:
+- sentence-transformers
+- cross-encoder
+- generated_from_trainer
+- dataset_size:78704
+- loss:ListMLELoss
+base_model: microsoft/MiniLM-L12-H384-uncased
+datasets:
+- microsoft/ms_marco
+pipeline_tag: text-ranking
+library_name: sentence-transformers
+metrics:
+- map
+- mrr@10
+- ndcg@10
+model-index:
+- name: CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+  results:
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoMSMARCO R100
+      type: NanoMSMARCO_R100
+    metrics:
+    - type: map
+      value: 0.4776
+      name: Map
+    - type: mrr@10
+      value: 0.4665
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.533
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNFCorpus R100
+      type: NanoNFCorpus_R100
+    metrics:
+    - type: map
+      value: 0.3041
+      name: Map
+    - type: mrr@10
+      value: 0.4803
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.317
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-reranking
+      name: Cross Encoder Reranking
+    dataset:
+      name: NanoNQ R100
+      type: NanoNQ_R100
+    metrics:
+    - type: map
+      value: 0.479
+      name: Map
+    - type: mrr@10
+      value: 0.4831
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.5407
+      name: Ndcg@10
+  - task:
+      type: cross-encoder-nano-beir
+      name: Cross Encoder Nano BEIR
+    dataset:
+      name: NanoBEIR R100 mean
+      type: NanoBEIR_R100_mean
+    metrics:
+    - type: map
+      value: 0.4202
+      name: Map
+    - type: mrr@10
+      value: 0.4766
+      name: Mrr@10
+    - type: ndcg@10
+      value: 0.4636
+      name: Ndcg@10
+---
+
+# CrossEncoder based on microsoft/MiniLM-L12-H384-uncased
+
+This is a [Cross Encoder](https://www.sbert.net/docs/cross_encoder/usage/usage.html) model finetuned from [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) on the [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) dataset using the [sentence-transformers](https://www.SBERT.net) library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Cross Encoder
+- **Base model:** [microsoft/MiniLM-L12-H384-uncased](https://huggingface.co/microsoft/MiniLM-L12-H384-uncased) <!-- at revision 44acabbec0ef496f6dbc93adadea57f376b7c0ec -->
+- **Maximum Sequence Length:** 512 tokens
+- **Number of Output Labels:** 1 label
+- **Training Dataset:**
+    - [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco)
+- **Language:** en
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Documentation:** [Cross Encoder Documentation](https://www.sbert.net/docs/cross_encoder/usage/usage.html)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Cross Encoders on Hugging Face](https://huggingface.co/models?library=sentence-transformers&other=cross-encoder)
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import CrossEncoder
+
+# Download from the 🤗 Hub
+model = CrossEncoder("yjoonjang/reranker-msmarco-v1.1-MiniLM-L12-H384-uncased-plistmle-customweight")
+# Get scores for pairs of texts
+pairs = [
+    ['How many calories in an egg', 'There are on average between 55 and 80 calories in an egg depending on its size.'],
+    ['How many calories in an egg', 'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.'],
+    ['How many calories in an egg', 'Most of the calories in an egg come from the yellow yolk in the center.'],
+]
+scores = model.predict(pairs)
+print(scores.shape)
+# (3,)
+
+# Or rank different texts based on similarity to a single text
+ranks = model.rank(
+    'How many calories in an egg',
+    [
+        'There are on average between 55 and 80 calories in an egg depending on its size.',
+        'Egg whites are very low in calories, have no fat, no cholesterol, and are loaded with protein.',
+        'Most of the calories in an egg come from the yellow yolk in the center.',
+    ]
+)
+# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Cross Encoder Reranking
+
+* Datasets: `NanoMSMARCO_R100`, `NanoNFCorpus_R100` and `NanoNQ_R100`
+* Evaluated with [<code>CrossEncoderRerankingEvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderRerankingEvaluator) with these parameters:
+  ```json
+  {
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | NanoMSMARCO_R100     | NanoNFCorpus_R100    | NanoNQ_R100          |
+|:------------|:---------------------|:---------------------|:---------------------|
+| map         | 0.4776 (-0.0120)     | 0.3041 (+0.0431)     | 0.4790 (+0.0594)     |
+| mrr@10      | 0.4665 (-0.0110)     | 0.4803 (-0.0195)     | 0.4831 (+0.0564)     |
+| **ndcg@10** | **0.5330 (-0.0074)** | **0.3170 (-0.0081)** | **0.5407 (+0.0401)** |
+
+#### Cross Encoder Nano BEIR
+
+* Dataset: `NanoBEIR_R100_mean`
+* Evaluated with [<code>CrossEncoderNanoBEIREvaluator</code>](https://sbert.net/docs/package_reference/cross_encoder/evaluation.html#sentence_transformers.cross_encoder.evaluation.CrossEncoderNanoBEIREvaluator) with these parameters:
+  ```json
+  {
+      "dataset_names": [
+          "msmarco",
+          "nfcorpus",
+          "nq"
+      ],
+      "rerank_k": 100,
+      "at_k": 10,
+      "always_rerank_positives": true
+  }
+  ```
+
+| Metric      | Value                |
+|:------------|:---------------------|
+| map         | 0.4202 (+0.0302)     |
+| mrr@10      | 0.4766 (+0.0086)     |
+| **ndcg@10** | **0.4636 (+0.0082)** |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 78,704 training samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                           | docs                                                                                   | labels                                                                                 |
+  |:--------|:------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                          | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 10 characters</li><li>mean: 34.15 characters</li><li>max: 100 characters</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                         | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:----------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>is dna acidic</code>                    | <code>['Great question. Deoxyribonucleic Acid is made up of a sugar (the deoxyribose), a phosphate ion, and a nitrogenous base. the phosphate ion is where the acidity is, its also called phosphoric acid while in your cells.', 'Confidence votes 134. Great question. Deoxyribonucleic Acid is made up of a sugar (the deoxyribose), a phosphate ion, and a nitrogenous base. the phosphate ion is where the acidity is, its also called phosphoric acid while in your cells.', 'Yes, DNA is an acid. In the structure of DNA, there is sugar (deoxyribose), a phosphate ion, and a nitrogenous base. the phosphate ion gives the acidic property. Fri Jan 14, 2005 5:07 pm. Which is kind of funny because the nitrogen base really IS a base.', 'First of all, DNA is not made up of nucleotide bases but of nucleotides. These consist of a sugar bound to one of the 4 nucleobases Adenine, Cytosine, Guanine or Thymine (Uracil in the case of RNA) and a phosphate group.', 'As you already know, the letters in DNA stand for Deoxyr...</code> | <code>[1, 1, 0, 0, 0, ...]</code> |
+  | <code>what is a nerve conduction study</code> | <code>['A nerve conduction study (NCS), also called a nerve conduction velocity (NCV) test--is a measurement of the speed of conduction of an electrical impulse through a nerve. NCS can determine nerve damage and destruction. During the test, the nerve is stimulated, usually with surface electrode patches attached to the skin. The nerve conduction velocity (speed) is then calculated by measuring the distance between electrodes and the time it takes for electrical impulses to travel between electrodes. A related procedure that may be performed is electromyography (EMG).', 'Nerve conduction studies and needle EMG are commonly performed by physical medicine and rehabilitation or neurology specialists to assess the ability of the nervous system to conduct electrical impulses and to evaluate nerve/muscle function to determine if neuromuscular disease is present. Motor nerve conduction studies. In motor nerve conduction studies, motor nerves are stimulated and the compound muscle action potential ...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>average nhl salary</code>               | <code>['Most NHL teams, 17 of them, paid an average salary per player ranging from $1.8 million to $1 million. Only five teams--Philadelphia Flyers, Colorado Avalanche, New York Islanders, Anaheim Ducks and the Tampa Bay Lightning paid average salaries of under a million dollars per player. The minimum NHL player salary, per the collective bargaining agreement, is $500,000 in 2009-10 and 2010-11. The maximum seasonal salary for players new to the NHL is $900,000 for 2009 and 2010; and $925,000 for 2011 draftees.', 'Sports NHL NHL Salaries 2015. National Hockey League salary cap is what every team has to spend a limited amount of money on purchasing players every year. So that makes it more compatible to earn good players in all the teams and make it more balanced in strength. This led some teams to trade away well paid star players to fit the cap. But the teams were allowed to spend $70.4 Million in a year for the prorated shorted season length. The highest NHL career salary earns as of now ...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>ListMLELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#listmleloss) with these parameters:
+  ```json
+  {
+      "lambda_weight": "sentence_transformers.cross_encoder.losses.ListMLELoss.ListMLELambdaWeight",
+      "activation_fct": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": 16,
+      "respect_input_order": true
+  }
+  ```
+
+### Evaluation Dataset
+
+#### ms_marco
+
+* Dataset: [ms_marco](https://huggingface.co/datasets/microsoft/ms_marco) at [a47ee7a](https://huggingface.co/datasets/microsoft/ms_marco/tree/a47ee7aae8d7d466ba15f9f0bfac3b3681087b3a)
+* Size: 1,000 evaluation samples
+* Columns: <code>query</code>, <code>docs</code>, and <code>labels</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                                         | docs                                                                                   | labels                                                                                 |
+  |:--------|:----------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------|
+  | type    | string                                                                                        | list                                                                                   | list                                                                                   |
+  | details | <ul><li>min: 12 characters</li><li>mean: 33.4 characters</li><li>max: 93 characters</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> | <ul><li>min: 3 elements</li><li>mean: 6.50 elements</li><li>max: 10 elements</li></ul> |
+* Samples:
+  | query                                                                | docs                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | labels                            |
+  |:---------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------|
+  | <code>what is contamination</code>                                   | <code>['Contamination is the presence of an unwanted constituent, contaminant or impurity in a material, physical body, natural environment, workplace, etc. Contamination may include residual radioactive material remaining at a site after the completion of decommissioning of a site where there was a nuclear reactor, such as a power plant, experimental reactor, isotope reactor or a nuclear powered ship or submarine.', 'contamination. 1. the soiling or making inferior by contact or mixture, as by introduction of infectious organisms into a wound, into water, milk, food or onto the external surface of the body or on bandages and other dressings. 2. the deposition of radioactive material in any place where it is not desired. See Cross contamination Public health The presence of any foreign or undesired material in a system–eg, toxic contamination of the ground water in an ecosystem or untreated sewage into a stream Radiation physics The deposition of radioactive material in any place where it is...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+  | <code>what does crisis representation in social research mean</code> | <code>['The crisis of representation that now seems so apparent after the writing of Baudrillard was also the result of a convergence of historical conditions both inside and outside of art. As a result, art’s capacity to depict the world was effected. Digging through all of my archived materials continues to be a good distraction from being productive. I stumbled upon this diatribe… but can’t figure out if I wrote this for a class or for my own geekful bliss…. Issues surrounding representation have played a key role in the development of postmodern art.', 'CRISIS OF REPRESENTATION. This phrase was coined by George Marcus and Michael Fischer to refer specifically to the uncertainty within the human sciences about adequate means of describing social reality. This crisis arises from the (noncontroversial) claim that no interpretive account can ever directly or completely capture lived experience. Broadly conceived, the crisis is part of a more general set of ideas across the human sciences tha...</code> | <code>[1, 1, 0, 0, 0, ...]</code> |
+  | <code>what was moche</code>                                          | <code>['The Moche civilization (alternatively, the Mochica culture, Early Chimu, Pre-Chimu, Proto-Chimu, etc.) flourished in northern Peru with its capital near present-day Moche and Trujillo, from about 100 AD to 800 AD, during the Regional Development Epoch.', 'Moche Politics and Economy. The Moche were a stratified society with a powerful elite and an elaborate, well-codified ritual process. The political economy was based on the presence of large civic-ceremonial centers that produced a wide range of goods which were marketed to rural agrarian villages.', 'The Moche civilization (also known as the Mochica) flourished along the northern coast and valleys of ancient Peru, in particular, in the Chicama and Trujillo Valleys, between 1 CE and 800 CE.', 'While Mochica has been used in place of Moche by people describing this culture, the word Mochica actually refers to a particular dialect. This dialect, however, was not proven to be the dialect of the Moche Civilization and culture.', 'Moche ...</code> | <code>[1, 0, 0, 0, 0, ...]</code> |
+* Loss: [<code>ListMLELoss</code>](https://sbert.net/docs/package_reference/cross_encoder/losses.html#listmleloss) with these parameters:
+  ```json
+  {
+      "lambda_weight": "sentence_transformers.cross_encoder.losses.ListMLELoss.ListMLELambdaWeight",
+      "activation_fct": "torch.nn.modules.linear.Identity",
+      "mini_batch_size": 16,
+      "respect_input_order": true
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `learning_rate`: 2e-05
+- `num_train_epochs`: 1
+- `warmup_ratio`: 0.1
+- `seed`: 12
+- `bf16`: True
+- `load_best_model_at_end`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 16
+- `per_device_eval_batch_size`: 16
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 2e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.1
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 12
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: True
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: True
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `dispatch_batches`: None
+- `split_batches`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+
+</details>
+
+### Training Logs
+| Epoch      | Step     | Training Loss | Validation Loss | NanoMSMARCO_R100_ndcg@10 | NanoNFCorpus_R100_ndcg@10 | NanoNQ_R100_ndcg@10  | NanoBEIR_R100_mean_ndcg@10 |
+|:----------:|:--------:|:-------------:|:---------------:|:------------------------:|:-------------------------:|:--------------------:|:--------------------------:|
+| -1         | -1       | -             | -               | 0.0284 (-0.5121)         | 0.2663 (-0.0587)          | 0.0359 (-0.4647)     | 0.1102 (-0.3452)           |
+| 0.0002     | 1        | 11.2052       | -               | -                        | -                         | -                    | -                          |
+| 0.0508     | 250      | 10.1581       | -               | -                        | -                         | -                    | -                          |
+| 0.1016     | 500      | 9.1585        | 9.0436          | 0.0470 (-0.4934)         | 0.3321 (+0.0071)          | 0.0242 (-0.4764)     | 0.1344 (-0.3209)           |
+| 0.1525     | 750      | 9.0556        | -               | -                        | -                         | -                    | -                          |
+| 0.2033     | 1000     | 8.9995        | 8.9401          | 0.2576 (-0.2828)         | 0.2456 (-0.0795)          | 0.3359 (-0.1648)     | 0.2797 (-0.1757)           |
+| 0.2541     | 1250     | 8.9878        | -               | -                        | -                         | -                    | -                          |
+| 0.3049     | 1500     | 8.9811        | 8.8985          | 0.4518 (-0.0886)         | 0.2943 (-0.0307)          | 0.3790 (-0.1217)     | 0.3750 (-0.0803)           |
+| 0.3558     | 1750     | 8.9185        | -               | -                        | -                         | -                    | -                          |
+| 0.4066     | 2000     | 8.863         | 8.9124          | 0.4213 (-0.1191)         | 0.2972 (-0.0278)          | 0.4477 (-0.0530)     | 0.3887 (-0.0667)           |
+| 0.4574     | 2250     | 8.8962        | -               | -                        | -                         | -                    | -                          |
+| 0.5082     | 2500     | 8.9063        | 8.8869          | 0.5117 (-0.0287)         | 0.3135 (-0.0116)          | 0.5208 (+0.0202)     | 0.4487 (-0.0067)           |
+| 0.5591     | 2750     | 8.9379        | -               | -                        | -                         | -                    | -                          |
+| 0.6099     | 3000     | 8.869         | 8.8610          | 0.5208 (-0.0196)         | 0.3203 (-0.0048)          | 0.4566 (-0.0440)     | 0.4326 (-0.0228)           |
+| 0.6607     | 3250     | 8.8965        | -               | -                        | -                         | -                    | -                          |
+| 0.7115     | 3500     | 8.8487        | 8.8466          | 0.5024 (-0.0380)         | 0.3007 (-0.0243)          | 0.4827 (-0.0179)     | 0.4286 (-0.0267)           |
+| 0.7624     | 3750     | 8.8695        | -               | -                        | -                         | -                    | -                          |
+| 0.8132     | 4000     | 8.8732        | 8.8497          | 0.5207 (-0.0197)         | 0.3247 (-0.0003)          | 0.5292 (+0.0285)     | 0.4582 (+0.0028)           |
+| 0.8640     | 4250     | 8.9325        | -               | -                        | -                         | -                    | -                          |
+| **0.9148** | **4500** | **8.8244**    | **8.8205**      | **0.5330 (-0.0074)**     | **0.3170 (-0.0081)**      | **0.5407 (+0.0401)** | **0.4636 (+0.0082)**       |
+| 0.9656     | 4750     | 8.858         | -               | -                        | -                         | -                    | -                          |
+| -1         | -1       | -             | -               | 0.5330 (-0.0074)         | 0.3170 (-0.0081)          | 0.5407 (+0.0401)     | 0.4636 (+0.0082)           |
+
+* The bold row denotes the saved checkpoint.
+
+### Framework Versions
+- Python: 3.11.11
+- Sentence Transformers: 3.5.0.dev0
+- Transformers: 4.49.0
+- PyTorch: 2.6.0+cu124
+- Accelerate: 1.5.2
+- Datasets: 3.4.0
+- Tokenizers: 0.21.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### ListMLELoss
+```bibtex
+@inproceedings{lan2013position,
+    title={Position-aware ListMLE: a sequential learning process for ranking},
+    author={Lan, Yanyan and Guo, Jiafeng and Cheng, Xueqi and Liu, Tie-Yan},
+    booktitle={Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence},
+    pages={333--342},
+    year={2013}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,34 @@
+{
+  "_name_or_path": "microsoft/MiniLM-L12-H384-uncased",
+  "architectures": [
+    "BertForSequenceClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 384,
+  "id2label": {
+    "0": "LABEL_0"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 1536,
+  "label2id": {
+    "LABEL_0": 0
+  },
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "sentence_transformers": {
+    "activation_fn": "torch.nn.modules.activation.Sigmoid"
+  },
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:934dbeb88c9b7b908fcf3576917e546f342b90935d3d721057e722f71a3da443
+size 133464836

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}