README.md

---
language:
- en
license: apache-2.0
tags:
- sentence-transformers
- sentence-similarity
- feature-extraction
- generated_from_trainer
- dataset_size:6300
- loss:MatryoshkaLoss
- loss:MultipleNegativesRankingLoss
base_model: BAAI/bge-base-en-v1.5
widget:
- source_sentence: What are the components of Comcast's domestic distribution revenue?
  sentences:
  - Cash used in investing activities was $2.3 billion for fiscal 2023, compared to
    $2.1 billion for fiscal 2022.
  - Domestic distribution revenue primarily includes revenue generated from the distribution
    of our television networks operating predominantly in the United States to traditional
    and virtual multichannel video providers, and from NBC-affiliated and Telemundo-affiliated
    local broadcast television stations. Our revenue from distribution agreements
    is generally based on the number of subscribers receiving the programming on our
    television networks and a per subscriber fee. Distribution revenue also includes
    Peacock subscription fees.
  - In January 2023, Alphabet Inc. announced a reduction of its workforce, consequently
    recording employee severance and related charges of $2.1 billion for the year.
- source_sentence: What was the noncash pre-tax impairment charge recorded due to
    the disposal of Vrio's operations in 2021, and what are the main components contributing
    to this amount?
  sentences:
  - The cash equities rate per contract (per 100 shares) for NYSE increased by 6%,
    from $0.045 in 2022 to $0.048 in 2023.
  - In the second quarter of 2021, we classified the Vrio disposal group as held-for-sale
    and reported the disposal group at fair value less cost to sell, which resulted
    in a noncash, pre-tax impairment charge of $4,555, including approximately $2,100
    related to accumulated foreign currency translation adjustments and $2,500 related
    to property, plant and equipment and intangible assets.
  - 'SECRET LAIR - our internet-based storefront where MAGIC: THE GATHERING fans can
    purchase exclusive and limited versions of cards.'
- source_sentence: What does the Corporate and Other segment include in its composition?
  sentences:
  - The segment consists of unallocated corporate expenses and administrative costs
    and activities not considered when evaluating segment performance as well as certain
    assets benefiting more than one segment. In addition, intersegment transactions
    are eliminated within the Corporate and Other segment.
  - Net cash provided by (used in) operating activities was recorded at $20,930 million
    for the reported year.
  - Forward-Looking Statements Certain statements in this report, other than purely
    historical information, including estimates, projections, statements relating
    to our business plans, objectives and expected operating results, and the assumptions
    upon which those statements are based, are “forward-looking statements” within
    the meaning of the Private Securities Litigation Reform Act of 1995, Section 27A
    of the Securities Act of 1933 and Section 21E of the Securities Exchange Act of
    1934.
- source_sentence: What was the purchase price for the repurchase of Mobility preferred
    interests by AT&T in 2023?
  sentences:
  - Net revenue increased $1.5 billion, or 19%, to $9.6 billion in 2023 from $8.1
    billion in 2022. On a constant dollar basis, net revenue increased 20%. Comparable
    sales increased 13%, or 14% on a constant dollar basis. The increase in net revenue
    was primarily due to increased Americas net revenue. China Mainland and Rest of
    World net revenue also increased.
  - Google Services includes products and services such as ads, Android, Chrome, devices,
    Google Maps, Google Play, Search, and YouTube. Google Services generates revenues
    primarily from advertising; fees received for consumer subscription-based products
    such. as YouTube TV, YouTube Music and Premium, and NFL Sunday Ticket; and the
    sale of apps and in-app purchases and devices.
  - In April 2023, we also accepted the December 2022 put option notice from the AT&T
    pension trust and repurchased the remaining 213 million Mobility preferred interests
    for a purchase price, including accrued and unpaid distributions, of $5,414.
- source_sentence: What is the maximum leverage ratio allowed before default under
    the company's credit facility?
  sentences:
  - If the company's leverage ratio exceeds 3.50 to 1, it would be in default of its
    revolving credit facility, impairing its ability to borrow under the facility.
  - Research and Development Because the industries in which the Company competes
    are characterized by rapid technological advances, the Company’s ability to compete
    successfully depends heavily upon its ability to ensure a continual and timely
    flow of competitive products, services and technologies to the marketplace.
  - Visa is focused on extending, enhancing and investing in VisaNet, their proprietary
    advanced transaction processing network, to offer a single connection point for
    facilitating payment transactions to multiple endpoints through various form factors.
datasets:
- philschmid/finanical-rag-embedding-dataset
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
- cosine_accuracy@1
- cosine_accuracy@3
- cosine_accuracy@5
- cosine_accuracy@10
- cosine_precision@1
- cosine_precision@3
- cosine_precision@5
- cosine_precision@10
- cosine_recall@1
- cosine_recall@3
- cosine_recall@5
- cosine_recall@10
- cosine_ndcg@10
- cosine_mrr@10
- cosine_map@100
model-index:
- name: BGE base Financial Matryoshka
  results:
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 768
      type: dim_768
    metrics:
    - type: cosine_accuracy@1
      value: 0.6771428571428572
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8371428571428572
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8685714285714285
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9185714285714286
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6771428571428572
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.27904761904761904
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17371428571428568
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09185714285714283
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6771428571428572
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8371428571428572
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8685714285714285
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9185714285714286
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.800782444183487
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.762721088435374
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7655884035994069
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 512
      type: dim_512
    metrics:
    - type: cosine_accuracy@1
      value: 0.6828571428571428
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8371428571428572
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8757142857142857
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.92
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6828571428571428
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.27904761904761904
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17514285714285713
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09199999999999998
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6828571428571428
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8371428571428572
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8757142857142857
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.92
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.80444342170685
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7670583900226756
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7699510134898729
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 256
      type: dim_256
    metrics:
    - type: cosine_accuracy@1
      value: 0.6757142857142857
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8228571428571428
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8642857142857143
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9185714285714286
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6757142857142857
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2742857142857143
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.17285714285714285
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09185714285714283
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6757142857142857
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8228571428571428
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8642857142857143
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9185714285714286
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7984105242762846
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7599024943310656
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7625291382895937
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 128
      type: dim_128
    metrics:
    - type: cosine_accuracy@1
      value: 0.6714285714285714
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.8114285714285714
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.8485714285714285
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.9014285714285715
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6714285714285714
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.2704761904761904
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.16971428571428568
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.09014285714285714
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6714285714285714
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.8114285714285714
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.8485714285714285
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.9014285714285715
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7872870842648211
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7507193877551018
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7542921487122674
      name: Cosine Map@100
  - task:
      type: information-retrieval
      name: Information Retrieval
    dataset:
      name: dim 64
      type: dim_64
    metrics:
    - type: cosine_accuracy@1
      value: 0.6242857142857143
      name: Cosine Accuracy@1
    - type: cosine_accuracy@3
      value: 0.7842857142857143
      name: Cosine Accuracy@3
    - type: cosine_accuracy@5
      value: 0.82
      name: Cosine Accuracy@5
    - type: cosine_accuracy@10
      value: 0.8828571428571429
      name: Cosine Accuracy@10
    - type: cosine_precision@1
      value: 0.6242857142857143
      name: Cosine Precision@1
    - type: cosine_precision@3
      value: 0.26142857142857145
      name: Cosine Precision@3
    - type: cosine_precision@5
      value: 0.16399999999999998
      name: Cosine Precision@5
    - type: cosine_precision@10
      value: 0.08828571428571429
      name: Cosine Precision@10
    - type: cosine_recall@1
      value: 0.6242857142857143
      name: Cosine Recall@1
    - type: cosine_recall@3
      value: 0.7842857142857143
      name: Cosine Recall@3
    - type: cosine_recall@5
      value: 0.82
      name: Cosine Recall@5
    - type: cosine_recall@10
      value: 0.8828571428571429
      name: Cosine Recall@10
    - type: cosine_ndcg@10
      value: 0.7546358861091382
      name: Cosine Ndcg@10
    - type: cosine_mrr@10
      value: 0.7135277777777775
      name: Cosine Mrr@10
    - type: cosine_map@100
      value: 0.7174129354945035
      name: Cosine Map@100
---

# BGE base Financial Matryoshka

This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) on the [finanical-rag-embedding-dataset](https://huggingface.co/datasets/philschmid/finanical-rag-embedding-dataset) dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

## Model Details

### Model Description
- **Model Type:** Sentence Transformer
- **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) <!-- at revision a5beb1e3e68b9ab74eb54cfd186867f64f240e1a -->
- **Maximum Sequence Length:** 512 tokens
- **Output Dimensionality:** 768 dimensions
- **Similarity Function:** Cosine Similarity
- **Training Dataset:**
    - [finanical-rag-embedding-dataset](https://huggingface.co/datasets/philschmid/finanical-rag-embedding-dataset)
- **Language:** en
- **License:** apache-2.0

### Model Sources

- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)

### Full Model Architecture

```
SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)
```

## Usage

### Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

```bash
pip install -U sentence-transformers
```

Then you can load this model and run inference.
```python
from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("bnkc123/bge-base-financial-matryoshka")
# Run inference
sentences = [
    "What is the maximum leverage ratio allowed before default under the company's credit facility?",
    "If the company's leverage ratio exceeds 3.50 to 1, it would be in default of its revolving credit facility, impairing its ability to borrow under the facility.",
    'Research and Development Because the industries in which the Company competes are characterized by rapid technological advances, the Company’s ability to compete successfully depends heavily upon its ability to ensure a continual and timely flow of competitive products, services and technologies to the marketplace.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]
```

<!--
### Direct Usage (Transformers)

<details><summary>Click to see the direct usage in Transformers</summary>

</details>
-->

<!--
### Downstream Usage (Sentence Transformers)

You can finetune this model on your own dataset.

<details><summary>Click to expand</summary>

</details>
-->

<!--
### Out-of-Scope Use

*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->

## Evaluation

### Metrics

#### Information Retrieval

* Dataset: `dim_768`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters:
  ```json
  {
      "truncate_dim": 768
  }
  ```

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6771     |
| cosine_accuracy@3   | 0.8371     |
| cosine_accuracy@5   | 0.8686     |
| cosine_accuracy@10  | 0.9186     |
| cosine_precision@1  | 0.6771     |
| cosine_precision@3  | 0.279      |
| cosine_precision@5  | 0.1737     |
| cosine_precision@10 | 0.0919     |
| cosine_recall@1     | 0.6771     |
| cosine_recall@3     | 0.8371     |
| cosine_recall@5     | 0.8686     |
| cosine_recall@10    | 0.9186     |
| **cosine_ndcg@10**  | **0.8008** |
| cosine_mrr@10       | 0.7627     |
| cosine_map@100      | 0.7656     |

#### Information Retrieval

* Dataset: `dim_512`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters:
  ```json
  {
      "truncate_dim": 512
  }
  ```

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6829     |
| cosine_accuracy@3   | 0.8371     |
| cosine_accuracy@5   | 0.8757     |
| cosine_accuracy@10  | 0.92       |
| cosine_precision@1  | 0.6829     |
| cosine_precision@3  | 0.279      |
| cosine_precision@5  | 0.1751     |
| cosine_precision@10 | 0.092      |
| cosine_recall@1     | 0.6829     |
| cosine_recall@3     | 0.8371     |
| cosine_recall@5     | 0.8757     |
| cosine_recall@10    | 0.92       |
| **cosine_ndcg@10**  | **0.8044** |
| cosine_mrr@10       | 0.7671     |
| cosine_map@100      | 0.77       |

#### Information Retrieval

* Dataset: `dim_256`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters:
  ```json
  {
      "truncate_dim": 256
  }
  ```

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6757     |
| cosine_accuracy@3   | 0.8229     |
| cosine_accuracy@5   | 0.8643     |
| cosine_accuracy@10  | 0.9186     |
| cosine_precision@1  | 0.6757     |
| cosine_precision@3  | 0.2743     |
| cosine_precision@5  | 0.1729     |
| cosine_precision@10 | 0.0919     |
| cosine_recall@1     | 0.6757     |
| cosine_recall@3     | 0.8229     |
| cosine_recall@5     | 0.8643     |
| cosine_recall@10    | 0.9186     |
| **cosine_ndcg@10**  | **0.7984** |
| cosine_mrr@10       | 0.7599     |
| cosine_map@100      | 0.7625     |

#### Information Retrieval

* Dataset: `dim_128`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters:
  ```json
  {
      "truncate_dim": 128
  }
  ```

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6714     |
| cosine_accuracy@3   | 0.8114     |
| cosine_accuracy@5   | 0.8486     |
| cosine_accuracy@10  | 0.9014     |
| cosine_precision@1  | 0.6714     |
| cosine_precision@3  | 0.2705     |
| cosine_precision@5  | 0.1697     |
| cosine_precision@10 | 0.0901     |
| cosine_recall@1     | 0.6714     |
| cosine_recall@3     | 0.8114     |
| cosine_recall@5     | 0.8486     |
| cosine_recall@10    | 0.9014     |
| **cosine_ndcg@10**  | **0.7873** |
| cosine_mrr@10       | 0.7507     |
| cosine_map@100      | 0.7543     |

#### Information Retrieval

* Dataset: `dim_64`
* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters:
  ```json
  {
      "truncate_dim": 64
  }
  ```

| Metric              | Value      |
|:--------------------|:-----------|
| cosine_accuracy@1   | 0.6243     |
| cosine_accuracy@3   | 0.7843     |
| cosine_accuracy@5   | 0.82       |
| cosine_accuracy@10  | 0.8829     |
| cosine_precision@1  | 0.6243     |
| cosine_precision@3  | 0.2614     |
| cosine_precision@5  | 0.164      |
| cosine_precision@10 | 0.0883     |
| cosine_recall@1     | 0.6243     |
| cosine_recall@3     | 0.7843     |
| cosine_recall@5     | 0.82       |
| cosine_recall@10    | 0.8829     |
| **cosine_ndcg@10**  | **0.7546** |
| cosine_mrr@10       | 0.7135     |
| cosine_map@100      | 0.7174     |

<!--
## Bias, Risks and Limitations

*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->

<!--
### Recommendations

*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->

## Training Details

### Training Dataset

#### finanical-rag-embedding-dataset

* Dataset: [finanical-rag-embedding-dataset](https://huggingface.co/datasets/philschmid/finanical-rag-embedding-dataset) at [e0b1781](https://huggingface.co/datasets/philschmid/finanical-rag-embedding-dataset/tree/e0b17819cf52d444066c99f4a176f5717e066300)
* Size: 6,300 training samples
* Columns: <code>anchor</code> and <code>positive</code>
* Approximate statistics based on the first 1000 samples:
  |         | anchor                                                                           | positive                                                                           |
  |:--------|:---------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
  | type    | string                                                                           | string                                                                             |
  | details | <ul><li>min: 7 tokens</li><li>mean: 20.5 tokens</li><li>max: 43 tokens</li></ul> | <ul><li>min: 9 tokens</li><li>mean: 46.09 tokens</li><li>max: 512 tokens</li></ul> |
* Samples:
  | anchor                                                                                                                                                    | positive                                                                                                                                                                                                                                                                                                                 |
  |:----------------------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
  | <code>What was the amount of premiums written by Berkshire Hathaway's Insurance Underwriting in 2023, and how did it compare to the previous year?</code> | <code>Premiums written increased $3.5 billion (24.1%) in 2023 compared to 2022. The increase was primarily due to RSUI and CapSpecialty ($2.1 billion), as well as comparative increases from BHSI and BH Direct, and to a lesser extent the other businesses. Premiums written | $ | 18,142 | | | | $ | 14,619 |</code> |
  | <code>What types of transportation equipment does XTRA Corporation manage in its fleet?</code>                                                            | <code>XTRA manages a diverse fleet of approximately 90,000 units located at 47 facilities throughout the U.S. The fleet includes over-the-road and storage trailers, chassis, temperature-controlled vans and flatbed trailers.</code>                                                                                   |
  | <code>What seasonal trends affect the company's sales volumes?</code>                                                                                     | <code>Sales volumes for the company are highest in the second fiscal quarter due to seasonal influences, particularly during the spring season in the regions it serves.</code>                                                                                                                                          |
* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
  ```json
  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          768,
          512,
          256,
          128,
          64
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  ```

### Training Hyperparameters
#### Non-Default Hyperparameters

- `eval_strategy`: epoch
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `gradient_accumulation_steps`: 16
- `learning_rate`: 2e-05
- `num_train_epochs`: 4
- `lr_scheduler_type`: cosine
- `warmup_ratio`: 0.1
- `bf16`: True
- `tf32`: True
- `load_best_model_at_end`: True
- `optim`: adamw_torch_fused
- `push_to_hub`: True
- `hub_model_id`: bnkc123/bge-base-financial-matryoshka
- `batch_sampler`: no_duplicates

#### All Hyperparameters
<details><summary>Click to expand</summary>

- `overwrite_output_dir`: False
- `do_predict`: False
- `eval_strategy`: epoch
- `prediction_loss_only`: True
- `per_device_train_batch_size`: 32
- `per_device_eval_batch_size`: 16
- `per_gpu_train_batch_size`: None
- `per_gpu_eval_batch_size`: None
- `gradient_accumulation_steps`: 16
- `eval_accumulation_steps`: None
- `torch_empty_cache_steps`: None
- `learning_rate`: 2e-05
- `weight_decay`: 0.0
- `adam_beta1`: 0.9
- `adam_beta2`: 0.999
- `adam_epsilon`: 1e-08
- `max_grad_norm`: 1.0
- `num_train_epochs`: 4
- `max_steps`: -1
- `lr_scheduler_type`: cosine
- `lr_scheduler_kwargs`: {}
- `warmup_ratio`: 0.1
- `warmup_steps`: 0
- `log_level`: passive
- `log_level_replica`: warning
- `log_on_each_node`: True
- `logging_nan_inf_filter`: True
- `save_safetensors`: True
- `save_on_each_node`: False
- `save_only_model`: False
- `restore_callback_states_from_checkpoint`: False
- `no_cuda`: False
- `use_cpu`: False
- `use_mps_device`: False
- `seed`: 42
- `data_seed`: None
- `jit_mode_eval`: False
- `use_ipex`: False
- `bf16`: True
- `fp16`: False
- `fp16_opt_level`: O1
- `half_precision_backend`: auto
- `bf16_full_eval`: False
- `fp16_full_eval`: False
- `tf32`: True
- `local_rank`: 0
- `ddp_backend`: None
- `tpu_num_cores`: None
- `tpu_metrics_debug`: False
- `debug`: []
- `dataloader_drop_last`: False
- `dataloader_num_workers`: 0
- `dataloader_prefetch_factor`: None
- `past_index`: -1
- `disable_tqdm`: False
- `remove_unused_columns`: True
- `label_names`: None
- `load_best_model_at_end`: True
- `ignore_data_skip`: False
- `fsdp`: []
- `fsdp_min_num_params`: 0
- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
- `tp_size`: 0
- `fsdp_transformer_layer_cls_to_wrap`: None
- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
- `deepspeed`: None
- `label_smoothing_factor`: 0.0
- `optim`: adamw_torch_fused
- `optim_args`: None
- `adafactor`: False
- `group_by_length`: False
- `length_column_name`: length
- `ddp_find_unused_parameters`: None
- `ddp_bucket_cap_mb`: None
- `ddp_broadcast_buffers`: False
- `dataloader_pin_memory`: True
- `dataloader_persistent_workers`: False
- `skip_memory_metrics`: True
- `use_legacy_prediction_loop`: False
- `push_to_hub`: True
- `resume_from_checkpoint`: None
- `hub_model_id`: bnkc123/bge-base-financial-matryoshka
- `hub_strategy`: every_save
- `hub_private_repo`: None
- `hub_always_push`: False
- `gradient_checkpointing`: False
- `gradient_checkpointing_kwargs`: None
- `include_inputs_for_metrics`: False
- `include_for_metrics`: []
- `eval_do_concat_batches`: True
- `fp16_backend`: auto
- `push_to_hub_model_id`: None
- `push_to_hub_organization`: None
- `mp_parameters`: 
- `auto_find_batch_size`: False
- `full_determinism`: False
- `torchdynamo`: None
- `ray_scope`: last
- `ddp_timeout`: 1800
- `torch_compile`: False
- `torch_compile_backend`: None
- `torch_compile_mode`: None
- `include_tokens_per_second`: False
- `include_num_input_tokens_seen`: False
- `neftune_noise_alpha`: None
- `optim_target_modules`: None
- `batch_eval_metrics`: False
- `eval_on_start`: False
- `use_liger_kernel`: False
- `eval_use_gather_object`: False
- `average_tokens_across_devices`: False
- `prompts`: None
- `batch_sampler`: no_duplicates
- `multi_dataset_batch_sampler`: proportional

</details>

### Training Logs
| Epoch     | Step   | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 |
|:---------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:|
| 0.8122    | 10     | 25.483        | -                      | -                      | -                      | -                      | -                     |
| 1.0       | 13     | -             | 0.7890                 | 0.7887                 | 0.7815                 | 0.7647                 | 0.7280                |
| 1.5685    | 20     | 9.1323        | -                      | -                      | -                      | -                      | -                     |
| 2.0       | 26     | -             | 0.7952                 | 0.7982                 | 0.7933                 | 0.7801                 | 0.7477                |
| 2.3249    | 30     | 6.7535        | -                      | -                      | -                      | -                      | -                     |
| 3.0       | 39     | -             | 0.8019                 | 0.8048                 | 0.7989                 | 0.7865                 | 0.7547                |
| 3.0812    | 40     | 6.5646        | -                      | -                      | -                      | -                      | -                     |
| **3.731** | **48** | **-**         | **0.8008**             | **0.8044**             | **0.7984**             | **0.7873**             | **0.7546**            |

* The bold row denotes the saved checkpoint.

### Framework Versions
- Python: 3.12.6
- Sentence Transformers: 4.1.0
- Transformers: 4.51.3
- PyTorch: 2.7.0+cu126
- Accelerate: 1.6.0
- Datasets: 3.5.1
- Tokenizers: 0.21.1

## Citation

### BibTeX

#### Sentence Transformers
```bibtex
@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}
```

#### MatryoshkaLoss
```bibtex
@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}
```

#### MultipleNegativesRankingLoss
```bibtex
@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}
```

<!--
## Glossary

*Clearly define terms in order to be accessible across audiences.*
-->

<!--
## Model Card Authors

*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->

<!--
## Model Card Contact

*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
-->