README.md

---
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:19089
- loss:MultipleNegativesRankingLoss
base_model: sentence-transformers/all-MiniLM-L6-v2

pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: val ir eval
      type: val-ir-eval
    metrics:
    - type: cosine_accuracy@1
      value: 0.37753510140405616
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.5858034321372855
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.6809672386895476
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.7753510140405616
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.37753510140405616
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.19552782111284447
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.13634945397815915
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.07769110764430577
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.37675507020280813
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.5854134165366615
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.6801872074882995
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.7739209568382736
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.5691636886714377
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.5045034420424439
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.512220040783084
      name: Cosine Map@100
---

# SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2). It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2) <!-- at revision fa97f6e7cb1a59073dff9e6b13e2715cf7475ac9 -->
- **Maximum Sequence Length:** 256 tokens
- **Output Dimensionality:** 384 dimensions
- **Similarity Function:** Cosine Similarity
<!-- - **Training Dataset:** Unknown -->
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tjohn327/scion-minilm-v2")
# Run inference
sentences = [
    "What role does the function 'target' play in modeling network topology?",
    'The document introduces a formal framework for verifying secure forwarding protocols within the SCION Internet architecture, specifically focusing on the ICING data plane protocol. It employs event systems as labeled transition systems, defining an event system E as a tuple comprising a set of states S, an initial state s0, a set of events E, and a transition relation e−→. The framework formalizes the concepts of reachability and invariants, establishing that a state property P is an invariant if the reachable states from the initial state are contained within P. The refinement of abstract event systems to concrete systems is articulated through mappings that preserve invariants. The document emphasizes parametrization, allowing models to incorporate assumptions on parameters, which are highlighted in gray. An abstract model is defined for a path-aware network architecture, excluding cryptographic elements, and is proven to satisfy path authorization and detectability. The network topology is modeled as a multigraph, with nodes representing Autonomous Systems (ASes) and edges representing links, characterized by a partial bijective function target that facilitates multiple links and forwarding policies. Paths in the network are defined as finite sequences of hop fields, encapsulating local routing information.',
    "The document chunk presents a testbed architecture for evaluating Secure In-Band Network Telemetry (ID-INT) within the SCION Internet Architecture, utilizing a Tofino 2 switch as the ID-INT enabled border routers for two SCION Autonomous Systems (ASes). The Dynamic Multi-Path Transport Protocol (DMTP) is adapted to send probe packets and retrieve telemetry data, focusing on instantaneous queue length at the egress of the border router. The experiment assesses DMTP's ability to adjust sending rates on Path 2 based on ID-INT telemetry, with initial path capacities set at 100 Mbps, later reduced to 75 Mbps. Results indicate that DMTP with ID-INT-enabled congestion control aligns sending rates more closely with available link capacity, achieving a 2% gain in goodput despite ID-INT overhead. The adaptation speed of DMTP using ID-INT telemetry is 35% faster than traditional congestion window-based control, demonstrating improved bandwidth utilization and congestion prevention. Related work includes a software implementation of the SCION reference border router in Go, and a hardware implementation on a NetFPGA SUME card capable of 10 Gbps throughput, highlighting the need for efficient high-bandwidth traffic handling in SCION.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Dataset: `val-ir-eval`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.3775     |
| cosine_accuracy@3   | 0.5858     |
| cosine_accuracy@5   | 0.681      |
| cosine_accuracy@10  | 0.7754     |
| cosine_precision@1  | 0.3775     |
| cosine_precision@3  | 0.1955     |
| cosine_precision@5  | 0.1363     |
| cosine_precision@10 | 0.0777     |
| cosine_recall@1     | 0.3768     |
| cosine_recall@3     | 0.5854     |
| cosine_recall@5     | 0.6802     |
| cosine_recall@10    | 0.7739     |
| **cosine_ndcg@10**  | **0.5692** |
| cosine_mrr@10       | 0.5045     |
| cosine_map@100      | 0.5122     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details


### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: steps
- `per_device_train_batch_size`: 192
- `per_device_eval_batch_size`: 192
- `fp16`: True
- `multi_dataset_batch_sampler`: round_robin

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: steps
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 192
- `per_device_eval_batch_size`: 192
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 1
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 5e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1
- `num_train_epochs`: 3
- `max_steps`: -1
- `lr_scheduler_type`: linear
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.0
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: False
- `fp16`: True
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: None
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: False
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: False
- `resume_from_checkpoint`: None
- `hub_model_id`: None
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `dispatch_batches`: None
- `split_batches`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: batch_sampler
- `multi_dataset_batch_sampler`: round_robin

</details>

### Training Logs
| Epoch | Step | val-ir-eval_cosine_ndcg@10 |
|:-----:|:----:|:--------------------------:|
| 1.0   | 50   | 0.5375                     |
| 2.0   | 100  | 0.5626                     |
| 3.0   | 150  | 0.5692                     |


### Framework Versions
- Python: 3.12.3
- Sentence Transformers: 3.4.1
- Transformers: 4.49.0
- PyTorch: 2.6.0+cu124
- Accelerate: 1.4.0
- Datasets: 3.3.2
- Tokenizers: 0.21.0

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->