metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - generated_from_trainer
  - dataset_size:20792
  - loss:MatryoshkaLoss
  - loss:MultipleNegativesRankingLoss
base_model: kokojake/modernbert-embed-base-fitness-health-matryoshka-8-epochs-25k
widget:
  - source_sentence: >-
      caregiver, or family member after receiving training in the appropriate
      technique.

      Vestibular training

      Vestibular functions are specific sensory functions of the inner ear
      related to position, balance and movement. Vestibular therapy includes
      exercises and techniques 

      to address symptoms of vestibular dysfunction, such as dizziness, visual
      or gaze 

      disturbances and balance disorders. The exercises and techniques are
      practised
    sentences:
      - STZ-induced diabetes model for DCM research
      - vestibular training for family caregivers
      - duration and frequency of acupuncture sessions for back pain
  - source_sentence: >-
      Favours Control

      FIGURE 8

      Forest plot of the effects of exercise training versus control on visceral
      fat. Data are reported as SMD (95% conﬁdence limits). SMD, standardized

      mean difference.
    sentences:
      - chronic symptoms and quality of life after traumatic brain injury
      - supported education intervention in school and university settings
      - exercise vs control on visceral fat SMD
  - source_sentence: >-
      tions that do not appear to have a relationship with exercise.

      Consider the following case: a 30-year-old female at ABC fit-

      ness center would like guidance on setting up an aerobic exercise program
      of moderate intensity. She has a heart rate monitor

      and is interested in using heart rate to gauge her intensity. Hav-

      ing completed the health screening questionnaire and other nec- essary
      documents, she meets with an exercise professional to de-

      termine what heart rate range would be appropriate. Based on

      the ACSM Guidelines, 64% to 76% of heart rate max is suggested for
      moderate intensity (1). An estimated maximal heart rate is

      calculated based on her age (for simplicity in this example,

      220 −age is used; for more information on estimation of maxi- mal heart
      rate see “Estimating Maximal Heart Rate” (23)). This re-

      sults in an estimated maximal heart rate of 190 (calculated as 220–30 =
      190) and a suggested range of 122–144 beats per minute

      TABLE: Select Physiologic Responses and Examples of

      Medication Class Effects (12)

      Areas of Potential

      Impact

      Medication Class
    sentences:
      - IBS and mental health disorders research
      - ACSM Guidelines for moderate intensity aerobic exercise heart rate range
      - >-
        ginger extract and omega-3 fatty acids supplementation for diabetic
        cardiomyopathy
  - source_sentence: |-
      ● Overall assessment and training in 
      abilities of self-care, managing the 
      changing environment, returning to 
      roles in the family and community.
      © Handicap International
    sentences:
      - difference between exogenous and endogenous ketones
      - Bisphosphonates for heterotopic ossification
      - self-care and community reintegration training
  - source_sentence: "Kasperczyk A, Kasperczyk S, Vendemiale G. An open-label, single-center \npilot study to test the effects of an amino acid mixture in older patients admitted to internal medicine wards. Nutrition. 2020;69:110588.\n\t38.\t Paddon-Jones D, Rasmussen BB. Dietary protein recommendations and the prevention of sarcopenia. Curr Opin Clin Nutr Metab Care. \n2009;12(1):86–90. 39.\t MacKenzie-Shalders KL, King NA, Byrne NM, Slater GJ. Increasing Protein \nDistribution Has No Effect on Changes in Lean Mass During a Rugby Preseason. Int J Sport Nutr Exerc Metab. 2016;26(1):1–7. 40.\t Zanini B, Simonetto A, Zubani M, Castellano M, Gilioli G: The Effects of \nCow-Milk Protein Supplementation in Elderly Population: Systematic Review and Narrative Synthesis. Nutrients. 2020, 12(9)."
    sentences:
      - effect of sitting time on obesity and diabetes
      - dietary protein recommendations for sarcopenia prevention
      - alternative treatments for chronic lower back pain in older adults
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: ModernBERT Embed base fitness health Matryoshka
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 768
          type: dim_768
        metrics:
          - type: cosine_accuracy@1
            value: 0.5577672003461704
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5633924707918649
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5768065772392903
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6572912159238425
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5577672003461704
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.5587768642723208
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.5546516659454782
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.488749459108611
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.07671206122845729
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.22954893017730485
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.37230530075054374
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.607515754908188
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5889008778621518
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5732042405884898
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6480888290704816
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 512
          type: dim_512
        metrics:
          - type: cosine_accuracy@1
            value: 0.5508437905668542
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5573344872349633
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5703158805711813
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6512332323669408
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5508437905668542
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.552141929900476
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.548334054521852
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.4848117697966249
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.07565244485079678
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.22624747899614794
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3669823430843505
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.6015669568225397
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5828389278226814
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5666920799763724
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6430351850654122
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 256
          type: dim_256
        metrics:
          - type: cosine_accuracy@1
            value: 0.5499783643444396
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5543054954565123
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5707485936823886
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6538295110341843
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5499783643444396
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.5505553151593826
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.5471224578104718
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.4856771960190394
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.07515575798786531
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.22481099241159822
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.36529494112791383
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.6017858069285081
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5820878299131311
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5658907021628786
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6428043114206009
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 128
          type: dim_128
        metrics:
          - type: cosine_accuracy@1
            value: 0.5196884465599307
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.5222847252271744
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5348334054521852
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6205106014712246
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.5196884465599307
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.5196884465599307
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.5151882302033751
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.45867589787970575
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.07059233617000817
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.21080992904878668
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.34166460612804184
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.564437701406452
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5472355538402842
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.5342628079646684
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.6136096665025225
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: dim 64
          type: dim_64
        metrics:
          - type: cosine_accuracy@1
            value: 0.45261791432280396
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.4552141929900476
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.475118996105582
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.5551709216789269
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.45261791432280396
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.4527621520265397
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.45114668974469935
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.4054954565123324
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.06187471238315029
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.18494725021120523
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.3014507909034088
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.5026563777104669
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.4836333347992592
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.46809753903003604
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.5520511526692713
            name: Cosine Map@100

ModernBERT Embed base fitness health Matryoshka

This is a sentence-transformers model finetuned from kokojake/modernbert-embed-base-fitness-health-matryoshka-8-epochs-25k on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: kokojake/modernbert-embed-base-fitness-health-matryoshka-8-epochs-25k
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- json
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False}) with Transformer model: ModernBertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("kokojake/modernbert-embed-base-fitness-health-matryoshka-epoch-15")
# Run inference
sentences = [
    'Kasperczyk A, Kasperczyk S, Vendemiale G. An open-label, single-center \npilot study to test the effects of an amino acid mixture in older patients admitted to internal medicine wards. Nutrition. 2020;69:110588.\n\t38.\t Paddon-Jones D, Rasmussen BB. Dietary protein recommendations and the prevention of sarcopenia. Curr Opin Clin Nutr Metab Care. \n2009;12(1):86–90. 39.\t MacKenzie-Shalders KL, King NA, Byrne NM, Slater GJ. Increasing Protein \nDistribution Has No Effect on Changes in Lean Mass During a Rugby Preseason. Int J Sport Nutr Exerc Metab. 2016;26(1):1–7. 40.\t Zanini B, Simonetto A, Zubani M, Castellano M, Gilioli G: The Effects of \nCow-Milk Protein Supplementation in Elderly Population: Systematic Review and Narrative Synthesis. Nutrients. 2020, 12(9).',
    'dietary protein recommendations for sarcopenia prevention',
    'effect of sitting time on obesity and diabetes',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 768
}
```

Metric	Value
cosine_accuracy@1	0.5578
cosine_accuracy@3	0.5634
cosine_accuracy@5	0.5768
cosine_accuracy@10	0.6573
cosine_precision@1	0.5578
cosine_precision@3	0.5588
cosine_precision@5	0.5547
cosine_precision@10	0.4887
cosine_recall@1	0.0767
cosine_recall@3	0.2295
cosine_recall@5	0.3723
cosine_recall@10	0.6075
cosine_ndcg@10	0.5889
cosine_mrr@10	0.5732
cosine_map@100	0.6481

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 512
}
```

Metric	Value
cosine_accuracy@1	0.5508
cosine_accuracy@3	0.5573
cosine_accuracy@5	0.5703
cosine_accuracy@10	0.6512
cosine_precision@1	0.5508
cosine_precision@3	0.5521
cosine_precision@5	0.5483
cosine_precision@10	0.4848
cosine_recall@1	0.0757
cosine_recall@3	0.2262
cosine_recall@5	0.367
cosine_recall@10	0.6016
cosine_ndcg@10	0.5828
cosine_mrr@10	0.5667
cosine_map@100	0.643

Information Retrieval

Dataset: dim_256
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 256
}
```

Metric	Value
cosine_accuracy@1	0.55
cosine_accuracy@3	0.5543
cosine_accuracy@5	0.5707
cosine_accuracy@10	0.6538
cosine_precision@1	0.55
cosine_precision@3	0.5506
cosine_precision@5	0.5471
cosine_precision@10	0.4857
cosine_recall@1	0.0752
cosine_recall@3	0.2248
cosine_recall@5	0.3653
cosine_recall@10	0.6018
cosine_ndcg@10	0.5821
cosine_mrr@10	0.5659
cosine_map@100	0.6428

Information Retrieval

Dataset: dim_128
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 128
}
```

Metric	Value
cosine_accuracy@1	0.5197
cosine_accuracy@3	0.5223
cosine_accuracy@5	0.5348
cosine_accuracy@10	0.6205
cosine_precision@1	0.5197
cosine_precision@3	0.5197
cosine_precision@5	0.5152
cosine_precision@10	0.4587
cosine_recall@1	0.0706
cosine_recall@3	0.2108
cosine_recall@5	0.3417
cosine_recall@10	0.5644
cosine_ndcg@10	0.5472
cosine_mrr@10	0.5343
cosine_map@100	0.6136

Information Retrieval

Dataset: dim_64
Evaluated with InformationRetrievalEvaluator with these parameters:
```
{
    "truncate_dim": 64
}
```

Metric	Value
cosine_accuracy@1	0.4526
cosine_accuracy@3	0.4552
cosine_accuracy@5	0.4751
cosine_accuracy@10	0.5552
cosine_precision@1	0.4526
cosine_precision@3	0.4528
cosine_precision@5	0.4511
cosine_precision@10	0.4055
cosine_recall@1	0.0619
cosine_recall@3	0.1849
cosine_recall@5	0.3015
cosine_recall@10	0.5027
cosine_ndcg@10	0.4836
cosine_mrr@10	0.4681
cosine_map@100	0.5521

Training Details

Training Dataset

json

Dataset: json
Size: 20,792 training samples
Columns: positive and anchor
Approximate statistics based on the first 1000 samples:
positive anchor
type string string
details
min: 4 tokens
mean: 220.24 tokens
max: 415 tokens

min: 5 tokens
mean: 11.15 tokens
max: 41 tokens

	positive	anchor
type	string	string
details	min: 4 tokens mean: 220.24 tokens max: 415 tokens	min: 5 tokens mean: 11.15 tokens max: 41 tokens

Samples:

positive	anchor
`interpretations, if a common framework like the ICF is used”23, the unit recommends using the ICF for communications outside the association, particularly in research contexts. Health conditions (disorder or disease) Activities © WHO, International Classification of Functioning, Disability and Health, 2001 Participation Body Functions and Structures Environmental Factors Personal Factors`	`ICF usage in research communications for health disorders`
`Physiol. Regul. Integr. Comp. Physiol. 2015, 309, R767–R779. [CrossRef] 39. Laurentino, G.C.; Ugrinowitsch, C.; Roschel, H.; Aoki, M.S.; Soares, A.G.; Neves, M.; Aihara, A.Y.; Fernandes`	`Laurentino et al. research on integrative physiology`
Telling your client to “push through your heels” when performing a squat or “explode through your hips or push through your feet” when performing jumping and sprinting movements are examples of internal cues. You also may utilize external cues to enhance motor learning and performance in all populations. External cues—or external focus of attention—direct a client’s attention toward the effect the movement will have on the surrounding environment and the movement outcome, as it relates to the exercise being performed (Winkelman et al., 2017;	`effect of external focus of attention on motor learning`

Loss: MatryoshkaLoss with these parameters:

{
    "loss": "MultipleNegativesRankingLoss",
    "matryoshka_dims": [
        768,
        512,
        256,
        128,
        64
    ],
    "matryoshka_weights": [
        1,
        1,
        1,
        1,
        1
    ],
    "n_dims_per_step": -1
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
gradient_accumulation_steps: 16
learning_rate: 2e-05
num_train_epochs: 4
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 4
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: True
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	dim_768_cosine_ndcg@10	dim_512_cosine_ndcg@10	dim_256_cosine_ndcg@10	dim_128_cosine_ndcg@10	dim_64_cosine_ndcg@10
0.2462	10	7.2777	-	-	-	-	-
0.4923	20	7.6341	-	-	-	-	-
0.7385	30	7.1497	-	-	-	-	-
0.9846	40	6.8322	0.5820	0.5741	0.5679	0.5308	0.4724
1.2462	50	6.779	-	-	-	-	-
1.4923	60	5.5133	-	-	-	-	-
1.7385	70	6.1867	-	-	-	-	-
1.9846	80	6.0276	0.5829	0.5798	0.5769	0.5409	0.4897
2.2462	90	4.971	-	-	-	-	-
2.4923	100	5.0184	-	-	-	-	-
2.7385	110	5.1473	-	-	-	-	-
2.9846	120	5.6456	0.5880	0.5830	0.5780	0.5472	0.4872
3.2462	130	5.0487	-	-	-	-	-
3.4923	140	4.7154	-	-	-	-	-
3.7385	150	5.1362	-	-	-	-	-
3.9846	160	4.931	0.5889	0.5828	0.5821	0.5472	0.4836

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.11.12
Sentence Transformers: 4.0.2
Transformers: 4.51.2
PyTorch: 2.6.0+cu124
Accelerate: 1.5.2
Datasets: 3.5.0
Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}