metadata

language:
  - en
license: apache-2.0
tags:
  - sentence-transformers
  - sentence-similarity
  - feature-extraction
  - dense
  - generated_from_trainer
  - dataset_size:20420717
  - loss:CachedMultipleNegativesRankingLoss
  - loss:CachedMultipleNegativesSymmetricRankingLoss
widget:
  - source_sentence: what is the difference between body spray and eau de toilette?
    sentences:
      - >-
        Eau de Toilette (EDT) is ideal for those that may find the EDP or
        Perfume oil too strong, with 7%-12% fragrance concentration in alcohol.
        Gives four to five hours wear. Body Mist is a light refreshing fragrance
        perfect for layering with other products from the same family. 3-5%
        fragrance concentration in alcohol.
      - >-
        To join the Army as an enlisted member you must usually take the Armed
        Services Vocational Aptitude Battery (ASVAB) test and get a good score.
        The maximum ASVAB score is 99. For enlistment into the Army you must get
        a minimum ASVAB score of 31.
      - >-
        Points needed to redeem rewards with Redbox Perks: 1,500 points = FREE
        1-night DVD rental. 1,750 points = FREE Blu-ray rental. 2,500 points =
        FREE 1-night Game rental.
  - source_sentence: hurtles definition
    sentences:
      - >-
        BCIP-NBT Solution is a 5-bromo, 4-chloro, 3-indolylphosphate
        (BCIP)/Nitro-Blue Tetrazolium (NBT) substrate, used for the localization
        of alkaline phosphatase (AP) labeled probes on western, northern,
        southern and dot blots.
      - >-
        Definition of hurtle for English Language Learners. : 1  to move or fall
        with great speed and force. : 2  to cause (something or someone) to move
        or go with great speed and force.
      - >-
        10 most extreme places on Earth. A dongle is a piece of hardware that
        attaches to a computer and allows a piece of secured software to run.
        The device does not contain the software in its entirety, but rather is
        an electronic key that unlocks the program on a computer.0 most extreme
        places on Earth. A dongle is a piece of hardware that attaches to a
        computer and allows a piece of secured software to run. The device does
        not contain the software in its entirety, but rather is an electronic
        key that unlocks the program on a computer.
  - source_sentence: what is nbt solution used for
    sentences:
      - >-
        Nembutal Sodium Solution [edit]. Nembutal Sodium Solution (pentobarbital
        sodium injection) is a sterile solution for intravenous or intramuscular
        injection. Each mL contains pentobarbital sodium 50 mg, in a vehicle of
        propylene glycol, 40%, alcohol, 10% and water for injection, to volume.
      - >-
        BCIP-NBT Solution is a 5-bromo, 4-chloro, 3-indolylphosphate
        (BCIP)/Nitro-Blue Tetrazolium (NBT) substrate, used for the localization
        of alkaline phosphatase (AP) labeled probes on western, northern,
        southern and dot blots.
      - >-
        Definition of hurtle for English Language Learners. : 1  to move or fall
        with great speed and force. : 2  to cause (something or someone) to move
        or go with great speed and force.
  - source_sentence: definition doula
    sentences:
      - "The Trivago Guy, aka Tim Williams, the Houston-born actor and Berlin resident who stars in hotel-search site Trivagoâ\x80\x99s TV commercials in North America, says heâ\x80\x99s perplexed and amused about all the attention heâ\x80\x99s getting from the TV spots, and heâ\x80\x99s enjoying every minute of it. â\x80\x9CBring it on, I love it,â\x80\x9D Williams says."
      - >-
        Average Length of Doula Study. Typically, a birth doula needs to finish
        7 to 12 hours of childbirth education, 16 hours of birth doula training,
        and attend at two to five births. A postpartum doula usually attends
        about 27 hours of postpartum doula education and assists two or more
        women with postpartum support.
      - >-
        Doula. A doula, is a nonmedical person who assists a woman before,
        during, or after childbirth, as well as her partner and/or family by
        providing information, physical assistance, and emotional support.
  - source_sentence: who is catch me if you can based on
    sentences:
      - >-
        Catch Me If You Can Catch Me If You Can is a 2002 American biographical
        crime film directed and produced by Steven Spielberg from a screenplay
        by Jeff Nathanson. The film is based on the life of Frank Abagnale, who,
        before his 19th birthday, successfully performed cons worth millions of
        dollars by posing as a Pan American World Airways pilot, a Georgia
        doctor and a Louisiana parish prosecutor. His primary crime was check
        fraud; he became so experienced that the FBI eventually turned to him
        for help in catching other checking forgers. The film stars Leonardo
        DiCaprio and Tom Hanks, with Christopher Walken, Martin Sheen, and
        Nathalie Baye in supporting roles.
      - >-
        Ghost Rider (Johnny Blaze) Centurious appeared, stealing Blaze's soul
        into his soul crystal. Zarathos, weakened from the ordeal used the last
        of his strength to shatter the crystal, freeing Blaze's soul and many
        others contained inside of the crystal as well. Before the crystal was
        reformed, Centurious was absorbed into the crystal. Zarathos followed
        him into the crystal, freeing Blaze from the curse, restoring his soul,
        and ending his time as the Ghost Rider.[31]
      - >-
        Colonial Brazil In contrast to the neighboring Spanish possessions,
        which had several viceroyalties with jurisdiction initially over New
        Spain (Mexico) and Peru, and in the eighteenth century expanded to
        viceroyalties of Rio de la Plata and New Granada, the Portuguese colony
        of Brazil was settled mainly in the coastal area by the Portuguese and a
        large black slave population working sugar plantations and mines. The
        boom and bust economic cycles were linked to export products. Brazil's
        sugar age, with the development of plantation slavery, merchants serving
        as middle men between production sites, Brazilian ports, and Europe was
        undermined by the growth of the sugar industry in the Caribbean on
        islands that European powers seized from Spain. Gold and diamonds were
        discovered and mined in southern Brazil through the end of the colonial
        era. Brazilian cities were largely port cities and the colonial
        administrative capital was moved several times in response to the rise
        and fall of export products' importance. Unlike Spanish America that
        fragmented in many republics, Brazil remained as a single administrative
        unit with a monarch, giving rise to the largest country in Latin
        America. Like Spanish America with European Spanish, Brazil had
        linguistic integrity of Portuguese. Both Spanish America and Brazil were
        Roman Catholic.
datasets:
  - sentence-transformers/msmarco-msmarco-distilbert-base-v3
  - sentence-transformers/gooaq
  - sentence-transformers/natural-questions
pipeline_tag: sentence-similarity
library_name: sentence-transformers
metrics:
  - cosine_accuracy@1
  - cosine_accuracy@3
  - cosine_accuracy@5
  - cosine_accuracy@10
  - cosine_precision@1
  - cosine_precision@3
  - cosine_precision@5
  - cosine_precision@10
  - cosine_recall@1
  - cosine_recall@3
  - cosine_recall@5
  - cosine_recall@10
  - cosine_ndcg@10
  - cosine_mrr@10
  - cosine_map@100
model-index:
  - name: ModernBERT-small-1.5-Retrieval-BEIR-Tuned
    results:
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoMSMARCO
          type: NanoMSMARCO
        metrics:
          - type: cosine_accuracy@1
            value: 0.08
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.3
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.36
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.5
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.08
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.09999999999999998
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.07200000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.05000000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.08
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.36
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.5
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.2774734521140046
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.20799206349206348
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.22779015501462538
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoNQ
          type: NanoNQ
        metrics:
          - type: cosine_accuracy@1
            value: 0.16
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.32
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.44
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.16
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.10666666666666666
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.08800000000000001
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.06000000000000001
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.14
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.29
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.4
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.55
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3273011574644101
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.27730158730158727
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.2638075437451129
            name: Cosine Map@100
      - task:
          type: information-retrieval
          name: Information Retrieval
        dataset:
          name: NanoHotpotQA
          type: NanoHotpotQA
        metrics:
          - type: cosine_accuracy@1
            value: 0.52
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.72
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.78
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.84
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.52
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.29333333333333333
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.19199999999999995
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.11799999999999997
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.26
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.44
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.48
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.59
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.5097756918840387
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.625547619047619
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.4238786723152704
            name: Cosine Map@100
      - task:
          type: nano-beir
          name: Nano BEIR
        dataset:
          name: NanoBEIR mean
          type: NanoBEIR_mean
        metrics:
          - type: cosine_accuracy@1
            value: 0.25333333333333335
            name: Cosine Accuracy@1
          - type: cosine_accuracy@3
            value: 0.4466666666666666
            name: Cosine Accuracy@3
          - type: cosine_accuracy@5
            value: 0.5266666666666667
            name: Cosine Accuracy@5
          - type: cosine_accuracy@10
            value: 0.6466666666666666
            name: Cosine Accuracy@10
          - type: cosine_precision@1
            value: 0.25333333333333335
            name: Cosine Precision@1
          - type: cosine_precision@3
            value: 0.16666666666666666
            name: Cosine Precision@3
          - type: cosine_precision@5
            value: 0.11733333333333333
            name: Cosine Precision@5
          - type: cosine_precision@10
            value: 0.076
            name: Cosine Precision@10
          - type: cosine_recall@1
            value: 0.16
            name: Cosine Recall@1
          - type: cosine_recall@3
            value: 0.3433333333333333
            name: Cosine Recall@3
          - type: cosine_recall@5
            value: 0.41333333333333333
            name: Cosine Recall@5
          - type: cosine_recall@10
            value: 0.5466666666666667
            name: Cosine Recall@10
          - type: cosine_ndcg@10
            value: 0.3715167671541511
            name: Cosine Ndcg@10
          - type: cosine_mrr@10
            value: 0.37028042328042327
            name: Cosine Mrr@10
          - type: cosine_map@100
            value: 0.3051587903583362
            name: Cosine Map@100

ModernBERT-small-1.5-Retrieval-BEIR-Tuned

This is a sentence-transformers model trained on the msmarco, gooaq and natural_questions datasets. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Maximum Sequence Length: 1024 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity
Training Datasets:
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'who is catch me if you can based on',
    'Catch Me If You Can Catch Me If You Can is a 2002 American biographical crime film directed and produced by Steven Spielberg from a screenplay by Jeff Nathanson. The film is based on the life of Frank Abagnale, who, before his 19th birthday, successfully performed cons worth millions of dollars by posing as a Pan American World Airways pilot, a Georgia doctor and a Louisiana parish prosecutor. His primary crime was check fraud; he became so experienced that the FBI eventually turned to him for help in catching other checking forgers. The film stars Leonardo DiCaprio and Tom Hanks, with Christopher Walken, Martin Sheen, and Nathalie Baye in supporting roles.',
    "Colonial Brazil In contrast to the neighboring Spanish possessions, which had several viceroyalties with jurisdiction initially over New Spain (Mexico) and Peru, and in the eighteenth century expanded to viceroyalties of Rio de la Plata and New Granada, the Portuguese colony of Brazil was settled mainly in the coastal area by the Portuguese and a large black slave population working sugar plantations and mines. The boom and bust economic cycles were linked to export products. Brazil's sugar age, with the development of plantation slavery, merchants serving as middle men between production sites, Brazilian ports, and Europe was undermined by the growth of the sugar industry in the Caribbean on islands that European powers seized from Spain. Gold and diamonds were discovered and mined in southern Brazil through the end of the colonial era. Brazilian cities were largely port cities and the colonial administrative capital was moved several times in response to the rise and fall of export products' importance. Unlike Spanish America that fragmented in many republics, Brazil remained as a single administrative unit with a monarch, giving rise to the largest country in Latin America. Like Spanish America with European Spanish, Brazil had linguistic integrity of Portuguese. Both Spanish America and Brazil were Roman Catholic.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.5555, -0.1763],
#         [ 0.5555,  1.0000, -0.0652],
#         [-0.1763, -0.0652,  1.0000]])

Evaluation

Metrics

Information Retrieval

Datasets: NanoMSMARCO, NanoNQ and NanoHotpotQA
Evaluated with InformationRetrievalEvaluator

Metric	NanoMSMARCO	NanoNQ	NanoHotpotQA
cosine_accuracy@1	0.08	0.16	0.52
cosine_accuracy@3	0.3	0.32	0.72
cosine_accuracy@5	0.36	0.44	0.78
cosine_accuracy@10	0.5	0.6	0.84
cosine_precision@1	0.08	0.16	0.52
cosine_precision@3	0.1	0.1067	0.2933
cosine_precision@5	0.072	0.088	0.192
cosine_precision@10	0.05	0.06	0.118
cosine_recall@1	0.08	0.14	0.26
cosine_recall@3	0.3	0.29	0.44
cosine_recall@5	0.36	0.4	0.48
cosine_recall@10	0.5	0.55	0.59
cosine_ndcg@10	0.2775	0.3273	0.5098
cosine_mrr@10	0.208	0.2773	0.6255
cosine_map@100	0.2278	0.2638	0.4239

Nano BEIR

Dataset: NanoBEIR_mean

Evaluated with NanoBEIREvaluator with these parameters:

{
    "dataset_names": [
        "MSMARCO",
        "NQ",
        "HotpotQA"
    ]
}

Metric	Value
cosine_accuracy@1	0.2533
cosine_accuracy@3	0.4467
cosine_accuracy@5	0.5267
cosine_accuracy@10	0.6467
cosine_precision@1	0.2533
cosine_precision@3	0.1667
cosine_precision@5	0.1173
cosine_precision@10	0.076
cosine_recall@1	0.16
cosine_recall@3	0.3433
cosine_recall@5	0.4133
cosine_recall@10	0.5467
cosine_ndcg@10	0.3715
cosine_mrr@10	0.3703
cosine_map@100	0.3052

Training Details

Training Datasets

msmarco

Dataset: msmarco at 28ff31e
Size: 17,307,990 training samples
Columns: anchor, positive, and negative

Approximate statistics based on the first 1000 samples:

	anchor	positive	negative
type	string	string	string
details	min: 5 tokens mean: 9.57 tokens max: 16 tokens	min: 44 tokens mean: 83.39 tokens max: 211 tokens	min: 24 tokens mean: 83.62 tokens max: 268 tokens

Samples:

anchor	positive	negative
`what are the liberal arts?`	`liberal arts. 1. the academic course of instruction at a college intended to provide general knowledge and comprising the arts, humanities, natural sciences, and social sciences, as opposed to professional or technical subjects.`	`The liberal arts education at the secondary school level prepares the student for higher education at a university. They are thus meant for the more academically minded students. In addition to the usual curriculum, students of a liberal arts education often study Latin and Ancient Greek. Some liberal arts education provide general education, others have a specific focus.`
`what are the liberal arts?`	`liberal arts. 1. the academic course of instruction at a college intended to provide general knowledge and comprising the arts, humanities, natural sciences, and social sciences, as opposed to professional or technical subjects.`	`Liberal Arts. Upon completion of the Liberal Arts degree, students will be able to express ideas in coherent, creative, and appropriate forms, orally and in writing. Students will be able to apply their reading abilities in order to interconnect an understanding of resources to academic, professional, and personal interests.`
`what are the liberal arts?`	`liberal arts. 1. the academic course of instruction at a college intended to provide general knowledge and comprising the arts, humanities, natural sciences, and social sciences, as opposed to professional or technical subjects.`	`Rather than preparing students for a specific career, liberal arts programs focus on cultural literacy and hone communication and analytical skills. They often cover various disciplines, ranging from the humanities to social sciences. 1 Program Levels in Liberal Arts: Associate degree, Bachelor's degree, Master's degree.`

Loss: CachedMultipleNegativesRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "mini_batch_size": 64
}

gooaq

Dataset: gooaq at b089f72
Size: 3,012,496 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 8 tokens
mean: 12.19 tokens
max: 22 tokens

min: 13 tokens
mean: 58.34 tokens
max: 124 tokens

	anchor	positive
type	string	string
details	min: 8 tokens mean: 12.19 tokens max: 22 tokens	min: 13 tokens mean: 58.34 tokens max: 124 tokens

Samples:

anchor	positive
`is toprol xl the same as metoprolol?`	`Metoprolol succinate is also known by the brand name Toprol XL. It is the extended-release form of metoprolol. Metoprolol succinate is approved to treat high blood pressure, chronic chest pain, and congestive heart failure.`
`are you experienced cd steve hoffman?`	`The Are You Experienced album was apparently mastered from the original stereo UK master tapes (according to Steve Hoffman - one of the very few who has heard both the master tapes and the CDs produced over the years). ... The CD booklets were a little sparse, but at least they stayed true to the album's original design.`
`how are babushka dolls made?`	`Matryoshka dolls are made of wood from lime, balsa, alder, aspen, and birch trees; lime is probably the most common wood type. ... After cutting, the trees are stripped of most of their bark, although a few inner rings of bark are left to bind the wood and keep it from splitting.`

Loss: CachedMultipleNegativesSymmetricRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "mini_batch_size": 64
}

natural_questions

Dataset: natural_questions at f9e894e
Size: 100,231 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 10 tokens
mean: 12.47 tokens
max: 23 tokens

min: 17 tokens
mean: 138.32 tokens
max: 556 tokens

	anchor	positive
type	string	string
details	min: 10 tokens mean: 12.47 tokens max: 23 tokens	min: 17 tokens mean: 138.32 tokens max: 556 tokens

Samples:

anchor	positive
`when did richmond last play in a preliminary final`	Richmond Football Club Richmond began 2017 with 5 straight wins, a feat it had not achieved since 1995. A series of close losses hampered the Tigers throughout the middle of the season, including a 5-point loss to the Western Bulldogs, 2-point loss to Fremantle, and a 3-point loss to the Giants. Richmond ended the season strongly with convincing victories over Fremantle and St Kilda in the final two rounds, elevating the club to 3rd on the ladder. Richmond's first final of the season against the Cats at the MCG attracted a record qualifying final crowd of 95,028; the Tigers won by 51 points. Having advanced to the first preliminary finals for the first time since 2001, Richmond defeated Greater Western Sydney by 36 points in front of a crowd of 94,258 to progress to the Grand Final against Adelaide, their first Grand Final appearance since 1982. The attendance was 100,021, the largest crowd to a grand final since 1986. The Crows led at quarter time and led by as many as 13, but the Tig...
`who sang what in the world's come over you`	Jack Scott (singer) At the beginning of 1960, Scott again changed record labels, this time to Top Rank Records.[1] He then recorded four Billboard Hot 100 hits – "What in the World's Come Over You" (#5), "Burning Bridges" (#3) b/w "Oh Little One" (#34), and "It Only Happened Yesterday" (#38).[1] "What in the World's Come Over You" was Scott's second gold disc winner.[6] Scott continued to record and perform during the 1960s and 1970s.[1] His song "You're Just Gettin' Better" reached the country charts in 1974.[1] In May 1977, Scott recorded a Peel session for BBC Radio 1 disc jockey, John Peel.
`who produces the most wool in the world`	Wool Global wool production is about 2 million tonnes per year, of which 60% goes into apparel. Wool comprises ca 3% of the global textile market, but its value is higher owing to dying and other modifications of the material.[1] Australia is a leading producer of wool which is mostly from Merino sheep but has been eclipsed by China in terms of total weight.[30] New Zealand (2016) is the third-largest producer of wool, and the largest producer of crossbred wool. Breeds such as Lincoln, Romney, Drysdale, and Elliotdale produce coarser fibers, and wool from these sheep is usually used for making carpets.

Loss: CachedMultipleNegativesSymmetricRankingLoss with these parameters:

{
    "scale": 20.0,
    "similarity_fct": "cos_sim",
    "mini_batch_size": 64
}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 256
weight_decay: 0.01
num_train_epochs: 1
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
bf16_full_eval: True
load_best_model_at_end: True

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 256
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: True
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	NanoMSMARCO_cosine_ndcg@10	NanoNQ_cosine_ndcg@10	NanoHotpotQA_cosine_ndcg@10	NanoBEIR_mean_cosine_ndcg@10
-1	-1	-	0.2397	0.3332	0.4767	0.3499
0.0063	500	1.6152	-	-	-	-
0.0125	1000	1.6149	-	-	-	-
0.0188	1500	1.5833	-	-	-	-
0.0251	2000	1.58	0.2449	0.3259	0.4793	0.3500
0.0313	2500	1.5475	-	-	-	-
0.0376	3000	1.5432	-	-	-	-
0.0439	3500	1.5067	-	-	-	-
0.0501	4000	1.4909	0.2474	0.3124	0.4786	0.3461
0.0564	4500	1.4532	-	-	-	-
0.0627	5000	1.425	-	-	-	-
0.0689	5500	1.4394	-	-	-	-
0.0752	6000	1.42	0.2532	0.3149	0.4895	0.3525
0.0815	6500	1.3737	-	-	-	-
0.0878	7000	1.3755	-	-	-	-
0.0940	7500	1.3194	-	-	-	-
0.1003	8000	1.3143	0.2660	0.3163	0.4823	0.3548
0.1066	8500	1.3038	-	-	-	-
0.1128	9000	1.2815	-	-	-	-
0.1191	9500	1.2291	-	-	-	-
0.1254	10000	1.24	0.2687	0.3407	0.4876	0.3657
0.1316	10500	1.2383	-	-	-	-
0.1379	11000	1.2116	-	-	-	-
0.1442	11500	1.1967	-	-	-	-
0.1504	12000	1.1712	0.2697	0.3436	0.4888	0.3674
0.1567	12500	1.1781	-	-	-	-
0.1630	13000	1.1798	-	-	-	-
0.1692	13500	1.1486	-	-	-	-
0.1755	14000	1.156	0.2761	0.3490	0.4895	0.3716
0.1818	14500	1.1622	-	-	-	-
0.1880	15000	1.1638	-	-	-	-
0.1943	15500	1.1447	-	-	-	-
0.2006	16000	1.1353	0.2783	0.3427	0.4967	0.3726
0.2068	16500	1.1397	-	-	-	-
0.2131	17000	1.1346	-	-	-	-
0.2194	17500	1.1345	-	-	-	-
0.2256	18000	1.13	0.2697	0.3424	0.5051	0.3724
0.2319	18500	1.1145	-	-	-	-
0.2382	19000	1.1215	-	-	-	-
0.2445	19500	1.1193	-	-	-	-
0.2507	20000	1.1329	0.2795	0.3363	0.4992	0.3717
0.2570	20500	1.1239	-	-	-	-
0.2633	21000	1.0929	-	-	-	-
0.2695	21500	1.1079	-	-	-	-
0.2758	22000	1.1192	0.2792	0.3278	0.5054	0.3708
0.2821	22500	1.1252	-	-	-	-
0.2883	23000	1.1089	-	-	-	-
0.2946	23500	1.1032	-	-	-	-
0.3009	24000	1.0974	0.2769	0.3372	0.5043	0.3728
0.3071	24500	1.1129	-	-	-	-
0.3134	25000	1.0901	-	-	-	-
0.3197	25500	1.1087	-	-	-	-
0.3259	26000	1.0921	0.2769	0.3349	0.5038	0.3719
0.3322	26500	1.0881	-	-	-	-
0.3385	27000	1.0984	-	-	-	-
0.3447	27500	1.105	-	-	-	-
0.3510	28000	1.1022	0.2766	0.3353	0.5043	0.3721
0.3573	28500	1.0925	-	-	-	-
0.3635	29000	1.1009	-	-	-	-
0.3698	29500	1.1043	-	-	-	-
0.3761	30000	1.0893	0.2772	0.3279	0.5055	0.3702
0.3823	30500	1.1084	-	-	-	-
0.3886	31000	1.0885	-	-	-	-
0.3949	31500	1.1046	-	-	-	-
0.4012	32000	1.0925	0.2775	0.3273	0.5052	0.3700
0.4074	32500	1.1126	-	-	-	-
0.4137	33000	1.0946	-	-	-	-
0.4200	33500	1.0979	-	-	-	-
0.4262	34000	1.0852	0.2775	0.3273	0.5098	0.3715
0.4325	34500	1.0925	-	-	-	-
0.4388	35000	1.0919	-	-	-	-
0.4450	35500	1.0878	-	-	-	-
0.4513	36000	1.0775	0.2781	0.3273	0.5098	0.3717
0.4576	36500	1.0898	-	-	-	-
0.4638	37000	1.0858	-	-	-	-
0.4701	37500	1.0822	-	-	-	-
0.4764	38000	1.0831	0.2849	0.3273	0.5098	0.3740
0.4826	38500	1.0886	-	-	-	-
0.4889	39000	1.089	-	-	-	-
0.4952	39500	1.0986	-	-	-	-
0.5014	40000	1.0885	0.2775	0.3276	0.5101	0.3717
0.5077	40500	1.0903	-	-	-	-
0.5140	41000	1.0883	-	-	-	-
0.5202	41500	1.1009	-	-	-	-
0.5265	42000	1.0909	0.2781	0.3276	0.5101	0.3719
0.5328	42500	1.0843	-	-	-	-
0.5390	43000	1.086	-	-	-	-
0.5453	43500	1.0762	-	-	-	-
0.5516	44000	1.0781	0.2781	0.3276	0.5101	0.3719
0.5579	44500	1.0952	-	-	-	-
0.5641	45000	1.0814	-	-	-	-
0.5704	45500	1.0815	-	-	-	-
0.5767	46000	1.0889	0.2849	0.3276	0.5101	0.3742
0.5829	46500	1.087	-	-	-	-
0.5892	47000	1.0786	-	-	-	-
0.5955	47500	1.0846	-	-	-	-
0.6017	48000	1.095	0.2849	0.3273	0.5101	0.3741
0.6080	48500	1.0839	-	-	-	-
0.6143	49000	1.0899	-	-	-	-
0.6205	49500	1.0903	-	-	-	-
0.6268	50000	1.0915	0.2775	0.3273	0.5098	0.3715
0.6331	50500	1.0764	-	-	-	-
0.6393	51000	1.1006	-	-	-	-
0.6456	51500	1.0968	-	-	-	-
0.6519	52000	1.084	0.2849	0.3273	0.5101	0.3741
0.6581	52500	1.0892	-	-	-	-
0.6644	53000	1.09	-	-	-	-
0.6707	53500	1.0946	-	-	-	-
0.6769	54000	1.0861	0.2775	0.3273	0.5098	0.3715
0.6832	54500	1.0962	-	-	-	-
0.6895	55000	1.0841	-	-	-	-
0.6958	55500	1.0894	-	-	-	-
0.7020	56000	1.082	0.2775	0.3273	0.5098	0.3715
0.7083	56500	1.0939	-	-	-	-
0.7146	57000	1.096	-	-	-	-
0.7208	57500	1.1048	-	-	-	-
0.7271	58000	1.0853	0.2849	0.3273	0.5098	0.3740
0.7334	58500	1.0893	-	-	-	-
0.7396	59000	1.0946	-	-	-	-
0.7459	59500	1.0985	-	-	-	-
0.7522	60000	1.099	0.2849	0.3273	0.5098	0.3740
0.7584	60500	1.0972	-	-	-	-
0.7647	61000	1.0812	-	-	-	-
0.7710	61500	1.0744	-	-	-	-
0.7772	62000	1.0781	0.2775	0.3273	0.5101	0.3716
0.7835	62500	1.0823	-	-	-	-
0.7898	63000	1.0819	-	-	-	-
0.7960	63500	1.0911	-	-	-	-
0.8023	64000	1.1069	0.2775	0.3273	0.5098	0.3715
0.8086	64500	1.0786	-	-	-	-
0.8148	65000	1.0872	-	-	-	-
0.8211	65500	1.0776	-	-	-	-
0.8274	66000	1.0849	0.2849	0.3273	0.5098	0.3740
0.8336	66500	1.0778	-	-	-	-
0.8399	67000	1.0972	-	-	-	-
0.8462	67500	1.0835	-	-	-	-
0.8525	68000	1.0927	0.2849	0.3273	0.5098	0.3740
0.8587	68500	1.082	-	-	-	-
0.8650	69000	1.0742	-	-	-	-
0.8713	69500	1.0886	-	-	-	-
0.8775	70000	1.0828	0.2775	0.3273	0.5098	0.3715
0.8838	70500	1.0863	-	-	-	-
0.8901	71000	1.0905	-	-	-	-
0.8963	71500	1.0856	-	-	-	-
0.9026	72000	1.0946	0.2775	0.3273	0.5098	0.3715
0.9089	72500	1.102	-	-	-	-
0.9151	73000	1.0819	-	-	-	-
0.9214	73500	1.0884	-	-	-	-
0.9277	74000	1.0888	0.2775	0.3273	0.5098	0.3715
0.9339	74500	1.0756	-	-	-	-
0.9402	75000	1.0767	-	-	-	-
0.9465	75500	1.0821	-	-	-	-
0.9527	76000	1.0891	0.2775	0.3273	0.5098	0.3715
0.9590	76500	1.0923	-	-	-	-
0.9653	77000	1.0765	-	-	-	-
0.9715	77500	1.075	-	-	-	-
0.9778	78000	1.0902	0.2775	0.3273	0.5098	0.3715
0.9841	78500	1.0833	-	-	-	-
0.9903	79000	1.0746	-	-	-	-
0.9966	79500	1.0872	-	-	-	-

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.11.13
Sentence Transformers: 5.0.0
Transformers: 4.53.1
PyTorch: 2.7.1+cu128
Accelerate: 1.8.1
Datasets: 4.0.0
Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CachedMultipleNegativesRankingLoss

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}