--- language: - en license: apache-2.0 tags: - sentence-transformers - sentence-similarity - feature-extraction - generated_from_trainer - dataset_size:6300 - loss:MatryoshkaLoss - loss:MultipleNegativesRankingLoss base_model: BAAI/bge-base-en-v1.5 widget: - source_sentence: What are the main components of technology and infrastructure costs? sentences: - As of January 29, 2023, from the total aggregate lease obligations of $14.7 billion, $1.5 billion was payable within 12 months. - Technology and infrastructure costs include payroll and related expenses for employees involved in the research and development of new and existing products and services, development, design, and maintenance of our stores, curation and display of products and services made available in our online stores, and infrastructure costs. - '''Note 13 — Commitments and Contingencies — Litigation and Other Legal Matters'' is stated to be part of Part IV, Item 15 of the consolidated financial statements within an Annual Report on Form 10-K.' - source_sentence: How is Meta's workforce comprised in terms of diversity as of December 31, 2022? sentences: - As of December 31, 2022, our global employee base was composed of 45.4% underrepresented people, with 47.9% underrepresented people in the U.S., and 43.1% of our leaders in the U.S. being people of color. - IBM's 2023 Annual Report to Stockholders includes the Financial Statements and Supplementary Data on pages 44 through 121. - Factors affecting the overall effective tax rate include acquisitions, changes in corporate structures, location of business functions, the mix and amount of income, agreements with tax authorities, and variations in estimated and actual pre-tax income. - source_sentence: What was the valuation allowance against deferred tax assets at the end of 2023, and what changes may affect its realization? sentences: - At December 31, 2020, valuation allowances against deducted assets were $7.0 billion. The ability to realize deductible benefits in future is contingent on producing any estimated sufficient values in cash-generating, with effects are modifications in trade situations, political of force, or those actions meaningfully impacting on the values. - Amazon considers its intellectual property essential for its success, utilizing trademark, copyright, and patent law, trade-secret protection, and confidentiality and/or license agreements to protect these rights. - 'During 2023, AMC served as the theatrical distributor for two theatrical releases: TAYLOR SWIFT | THE ERAS TOUR and RENAISSANCE: A FILM BY BEYONCÉ.' - source_sentence: What significant services are included in Iron Mountain's service revenues? sentences: - The decrease in net income in 2022 was primarily due to an increase in selling, general and administrative expenses of $532.4 million, an impairment charge recognized in 2022 of $407.9 million, an increase in income tax expense of $119.2 million, partially offset by an increase in gross profit of $883.8 million, a decrease in acquisition-related expenses of $41.4 million, a gain on disposal of assets of $10.2 million, and an increase in other income (expense), net of $3.6 million. - Service revenues include charges for the handling of records, destruction services, digital solutions, and data center services. - The total operating expenses for Chipotle Mexican Grill in 2023 amounted to $8,313,836. - source_sentence: In which part and item of the Annual Report on Form 10-K can the consolidated financial statements be found? sentences: - In order to maintain leadership, we optimize our portfolio with organic and inorganic innovations and effective resource allocation. These investments not only drive current performance but will extend our innovation leadership into the future. - Our Consumer Wireline business unit offers AT&T Internet Air, which is a fixed wireless access product that provides home internet services delivered over our 5G wireless network where available. - The consolidated financial statements and accompanying notes listed in Part IV, Item 15(a)(1) of this Annual Report on Form 10-K are included elsewhere in this Annual Report on For... 10-K. pipeline_tag: sentence-similarity library_name: sentence-transformers metrics: - cosine_accuracy@1 - cosine_accuracy@3 - cosine_accuracy@5 - cosine_accuracy@10 - cosine_precision@1 - cosine_precision@3 - cosine_precision@5 - cosine_precision@10 - cosine_recall@1 - cosine_recall@3 - cosine_recall@5 - cosine_recall@10 - cosine_ndcg@10 - cosine_mrr@10 - cosine_map@100 model-index: - name: BGE base Financial Matryoshka results: - task: type: information-retrieval name: Information Retrieval dataset: name: dim 768 type: dim_768 metrics: - type: cosine_accuracy@1 value: 0.7114285714285714 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.8371428571428572 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.87 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9057142857142857 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7114285714285714 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.27904761904761904 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.174 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09057142857142855 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7114285714285714 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8371428571428572 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.87 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9057142857142857 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8110932340412786 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7804977324263039 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.784240984630403 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 512 type: dim_512 metrics: - type: cosine_accuracy@1 value: 0.7157142857142857 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.83 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.87 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9071428571428571 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7157142857142857 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.27666666666666667 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.174 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.0907142857142857 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7157142857142857 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.83 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.87 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9071428571428571 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8116485651477514 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7810300453514737 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7845397715740386 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 256 type: dim_256 metrics: - type: cosine_accuracy@1 value: 0.7128571428571429 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.8214285714285714 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.86 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.9042857142857142 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.7128571428571429 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.27380952380952384 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.17199999999999996 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.09042857142857143 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.7128571428571429 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.8214285714285714 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.86 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.9042857142857142 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.8071701520591847 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7762494331065761 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7797123012827435 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 128 type: dim_128 metrics: - type: cosine_accuracy@1 value: 0.71 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.81 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8442857142857143 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.8985714285714286 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.71 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.27 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.16885714285714284 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08985714285714284 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.71 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.81 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8442857142857143 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.8985714285714286 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.801264041144764 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7705725623582764 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7744092505881914 name: Cosine Map@100 - task: type: information-retrieval name: Information Retrieval dataset: name: dim 64 type: dim_64 metrics: - type: cosine_accuracy@1 value: 0.6685714285714286 name: Cosine Accuracy@1 - type: cosine_accuracy@3 value: 0.78 name: Cosine Accuracy@3 - type: cosine_accuracy@5 value: 0.8257142857142857 name: Cosine Accuracy@5 - type: cosine_accuracy@10 value: 0.8757142857142857 name: Cosine Accuracy@10 - type: cosine_precision@1 value: 0.6685714285714286 name: Cosine Precision@1 - type: cosine_precision@3 value: 0.25999999999999995 name: Cosine Precision@3 - type: cosine_precision@5 value: 0.16514285714285715 name: Cosine Precision@5 - type: cosine_precision@10 value: 0.08757142857142856 name: Cosine Precision@10 - type: cosine_recall@1 value: 0.6685714285714286 name: Cosine Recall@1 - type: cosine_recall@3 value: 0.78 name: Cosine Recall@3 - type: cosine_recall@5 value: 0.8257142857142857 name: Cosine Recall@5 - type: cosine_recall@10 value: 0.8757142857142857 name: Cosine Recall@10 - type: cosine_ndcg@10 value: 0.7698003192070297 name: Cosine Ndcg@10 - type: cosine_mrr@10 value: 0.7363242630385484 name: Cosine Mrr@10 - type: cosine_map@100 value: 0.7409337390692949 name: Cosine Map@100 --- # BGE base Financial Matryoshka This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more. ## Model Details ### Model Description - **Model Type:** Sentence Transformer - **Base model:** [BAAI/bge-base-en-v1.5](https://huggingface.co/BAAI/bge-base-en-v1.5) - **Maximum Sequence Length:** 512 tokens - **Output Dimensionality:** 768 dimensions - **Similarity Function:** Cosine Similarity - **Training Dataset:** - json - **Language:** en - **License:** apache-2.0 ### Model Sources - **Documentation:** [Sentence Transformers Documentation](https://sbert.net) - **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers) - **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers) ### Full Model Architecture ``` SentenceTransformer( (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) (2): Normalize() ) ``` ## Usage ### Direct Usage (Sentence Transformers) First install the Sentence Transformers library: ```bash pip install -U sentence-transformers ``` Then you can load this model and run inference. ```python from sentence_transformers import SentenceTransformer # Download from the 🤗 Hub model = SentenceTransformer("TatvaRA/bge-base-financial-matryoshka") # Run inference sentences = [ 'In which part and item of the Annual Report on Form 10-K can the consolidated financial statements be found?', 'The consolidated financial statements and accompanying notes listed in Part IV, Item 15(a)(1) of this Annual Report on Form 10-K are included elsewhere in this Annual Report on For... 10-K.', 'Our Consumer Wireline business unit offers AT&T Internet Air, which is a fixed wireless access product that provides home internet services delivered over our 5G wireless network where available.', ] embeddings = model.encode(sentences) print(embeddings.shape) # [3, 768] # Get the similarity scores for the embeddings similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] ``` ## Evaluation ### Metrics #### Information Retrieval * Dataset: `dim_768` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters: ```json { "truncate_dim": 768 } ``` | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7114 | | cosine_accuracy@3 | 0.8371 | | cosine_accuracy@5 | 0.87 | | cosine_accuracy@10 | 0.9057 | | cosine_precision@1 | 0.7114 | | cosine_precision@3 | 0.279 | | cosine_precision@5 | 0.174 | | cosine_precision@10 | 0.0906 | | cosine_recall@1 | 0.7114 | | cosine_recall@3 | 0.8371 | | cosine_recall@5 | 0.87 | | cosine_recall@10 | 0.9057 | | **cosine_ndcg@10** | **0.8111** | | cosine_mrr@10 | 0.7805 | | cosine_map@100 | 0.7842 | #### Information Retrieval * Dataset: `dim_512` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters: ```json { "truncate_dim": 512 } ``` | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7157 | | cosine_accuracy@3 | 0.83 | | cosine_accuracy@5 | 0.87 | | cosine_accuracy@10 | 0.9071 | | cosine_precision@1 | 0.7157 | | cosine_precision@3 | 0.2767 | | cosine_precision@5 | 0.174 | | cosine_precision@10 | 0.0907 | | cosine_recall@1 | 0.7157 | | cosine_recall@3 | 0.83 | | cosine_recall@5 | 0.87 | | cosine_recall@10 | 0.9071 | | **cosine_ndcg@10** | **0.8116** | | cosine_mrr@10 | 0.781 | | cosine_map@100 | 0.7845 | #### Information Retrieval * Dataset: `dim_256` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters: ```json { "truncate_dim": 256 } ``` | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.7129 | | cosine_accuracy@3 | 0.8214 | | cosine_accuracy@5 | 0.86 | | cosine_accuracy@10 | 0.9043 | | cosine_precision@1 | 0.7129 | | cosine_precision@3 | 0.2738 | | cosine_precision@5 | 0.172 | | cosine_precision@10 | 0.0904 | | cosine_recall@1 | 0.7129 | | cosine_recall@3 | 0.8214 | | cosine_recall@5 | 0.86 | | cosine_recall@10 | 0.9043 | | **cosine_ndcg@10** | **0.8072** | | cosine_mrr@10 | 0.7762 | | cosine_map@100 | 0.7797 | #### Information Retrieval * Dataset: `dim_128` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters: ```json { "truncate_dim": 128 } ``` | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.71 | | cosine_accuracy@3 | 0.81 | | cosine_accuracy@5 | 0.8443 | | cosine_accuracy@10 | 0.8986 | | cosine_precision@1 | 0.71 | | cosine_precision@3 | 0.27 | | cosine_precision@5 | 0.1689 | | cosine_precision@10 | 0.0899 | | cosine_recall@1 | 0.71 | | cosine_recall@3 | 0.81 | | cosine_recall@5 | 0.8443 | | cosine_recall@10 | 0.8986 | | **cosine_ndcg@10** | **0.8013** | | cosine_mrr@10 | 0.7706 | | cosine_map@100 | 0.7744 | #### Information Retrieval * Dataset: `dim_64` * Evaluated with [InformationRetrievalEvaluator](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator) with these parameters: ```json { "truncate_dim": 64 } ``` | Metric | Value | |:--------------------|:-----------| | cosine_accuracy@1 | 0.6686 | | cosine_accuracy@3 | 0.78 | | cosine_accuracy@5 | 0.8257 | | cosine_accuracy@10 | 0.8757 | | cosine_precision@1 | 0.6686 | | cosine_precision@3 | 0.26 | | cosine_precision@5 | 0.1651 | | cosine_precision@10 | 0.0876 | | cosine_recall@1 | 0.6686 | | cosine_recall@3 | 0.78 | | cosine_recall@5 | 0.8257 | | cosine_recall@10 | 0.8757 | | **cosine_ndcg@10** | **0.7698** | | cosine_mrr@10 | 0.7363 | | cosine_map@100 | 0.7409 | ## Training Details ### Training Dataset #### json * Dataset: json * Size: 6,300 training samples * Columns: anchor and positive * Approximate statistics based on the first 1000 samples: | | anchor | positive | |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------| | type | string | string | | details | | | * Samples: | anchor | positive | |:--------------------------------------------------------------------------------------------------------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | What percentage of total revenues did STELARA account for in fiscal 2023 for the Company? | Sales of the Company’s largest product, STELARA (ustekinumab), accounted for approximately 12.8% of the Company's total revenues for fiscal 2023. | | What is the effective date for the new accounting standard ASU No. 2022-04 regarding liabilities in supplier finance programs? | In September 2022, the FASB issued ASU No. 2022-04, “Liabilities—Supplier Finance Programs (Topic 405-50) - Disclosure of Supplier Finance Program Obligations,” which is effective for fiscal years beginning after December 15, 2022, including interim periods within those fiscal years. | | What was the pre-tax net favorable prior period development for 2022 and what factors contributed to it? | The pre-tax net favorable prior period development for 2022 was $876 million. Adverse development factors like molestation claims, primarily reviver statute-related compromising $155 million, and $113 million related to legacy asbestos and environmental exposures significantly influenced this outcome. | * Loss: [MatryoshkaLoss](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters: ```json { "loss": "MultipleNegativesRankingLoss", "matryoshka_dims": [ 768, 512, 256, 128, 64 ], "matryoshka_weights": [ 1, 1, 1, 1, 1 ], "n_dims_per_step": -1 } ``` ### Training Hyperparameters #### Non-Default Hyperparameters - `eval_strategy`: epoch - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `gradient_accumulation_steps`: 16 - `learning_rate`: 2e-05 - `num_train_epochs`: 4 - `lr_scheduler_type`: cosine - `warmup_ratio`: 0.1 - `fp16`: True - `load_best_model_at_end`: True - `optim`: adamw_torch_fused - `batch_sampler`: no_duplicates #### All Hyperparameters
Click to expand - `overwrite_output_dir`: False - `do_predict`: False - `eval_strategy`: epoch - `prediction_loss_only`: True - `per_device_train_batch_size`: 32 - `per_device_eval_batch_size`: 16 - `per_gpu_train_batch_size`: None - `per_gpu_eval_batch_size`: None - `gradient_accumulation_steps`: 16 - `eval_accumulation_steps`: None - `learning_rate`: 2e-05 - `weight_decay`: 0.0 - `adam_beta1`: 0.9 - `adam_beta2`: 0.999 - `adam_epsilon`: 1e-08 - `max_grad_norm`: 1.0 - `num_train_epochs`: 4 - `max_steps`: -1 - `lr_scheduler_type`: cosine - `lr_scheduler_kwargs`: {} - `warmup_ratio`: 0.1 - `warmup_steps`: 0 - `log_level`: passive - `log_level_replica`: warning - `log_on_each_node`: True - `logging_nan_inf_filter`: True - `save_safetensors`: True - `save_on_each_node`: False - `save_only_model`: False - `restore_callback_states_from_checkpoint`: False - `no_cuda`: False - `use_cpu`: False - `use_mps_device`: False - `seed`: 42 - `data_seed`: None - `jit_mode_eval`: False - `use_ipex`: False - `bf16`: False - `fp16`: True - `fp16_opt_level`: O1 - `half_precision_backend`: auto - `bf16_full_eval`: False - `fp16_full_eval`: False - `tf32`: None - `local_rank`: 0 - `ddp_backend`: None - `tpu_num_cores`: None - `tpu_metrics_debug`: False - `debug`: [] - `dataloader_drop_last`: False - `dataloader_num_workers`: 0 - `dataloader_prefetch_factor`: None - `past_index`: -1 - `disable_tqdm`: False - `remove_unused_columns`: True - `label_names`: None - `load_best_model_at_end`: True - `ignore_data_skip`: False - `fsdp`: [] - `fsdp_min_num_params`: 0 - `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False} - `fsdp_transformer_layer_cls_to_wrap`: None - `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None} - `deepspeed`: None - `label_smoothing_factor`: 0.0 - `optim`: adamw_torch_fused - `optim_args`: None - `adafactor`: False - `group_by_length`: False - `length_column_name`: length - `ddp_find_unused_parameters`: None - `ddp_bucket_cap_mb`: None - `ddp_broadcast_buffers`: False - `dataloader_pin_memory`: True - `dataloader_persistent_workers`: False - `skip_memory_metrics`: True - `use_legacy_prediction_loop`: False - `push_to_hub`: False - `resume_from_checkpoint`: None - `hub_model_id`: None - `hub_strategy`: every_save - `hub_private_repo`: False - `hub_always_push`: False - `gradient_checkpointing`: False - `gradient_checkpointing_kwargs`: None - `include_inputs_for_metrics`: False - `eval_do_concat_batches`: True - `fp16_backend`: auto - `push_to_hub_model_id`: None - `push_to_hub_organization`: None - `mp_parameters`: - `auto_find_batch_size`: False - `full_determinism`: False - `torchdynamo`: None - `ray_scope`: last - `ddp_timeout`: 1800 - `torch_compile`: False - `torch_compile_backend`: None - `torch_compile_mode`: None - `dispatch_batches`: None - `split_batches`: None - `include_tokens_per_second`: False - `include_num_input_tokens_seen`: False - `neftune_noise_alpha`: None - `optim_target_modules`: None - `batch_eval_metrics`: False - `prompts`: None - `batch_sampler`: no_duplicates - `multi_dataset_batch_sampler`: proportional
### Training Logs | Epoch | Step | Training Loss | dim_768_cosine_ndcg@10 | dim_512_cosine_ndcg@10 | dim_256_cosine_ndcg@10 | dim_128_cosine_ndcg@10 | dim_64_cosine_ndcg@10 | |:----------:|:------:|:-------------:|:----------------------:|:----------------------:|:----------------------:|:----------------------:|:---------------------:| | 0.8122 | 10 | 1.6789 | - | - | - | - | - | | 0.9746 | 12 | - | 0.7976 | 0.8019 | 0.7944 | 0.7781 | 0.7387 | | 1.6244 | 20 | 0.6377 | - | - | - | - | - | | 1.9492 | 24 | - | 0.8071 | 0.8080 | 0.8016 | 0.7940 | 0.7594 | | 2.4365 | 30 | 0.5295 | - | - | - | - | - | | 2.9239 | 36 | - | 0.8110 | 0.8122 | 0.8067 | 0.8000 | 0.7697 | | 3.2487 | 40 | 0.4367 | - | - | - | - | - | | **3.8985** | **48** | **-** | **0.8111** | **0.8116** | **0.8072** | **0.8013** | **0.7698** | * The bold row denotes the saved checkpoint. ### Framework Versions - Python: 3.11.12 - Sentence Transformers: 4.1.0 - Transformers: 4.41.2 - PyTorch: 2.1.2+cu121 - Accelerate: 1.5.2 - Datasets: 2.19.1 - Tokenizers: 0.19.1 ## Citation ### BibTeX #### Sentence Transformers ```bibtex @inproceedings{reimers-2019-sentence-bert, title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks", author = "Reimers, Nils and Gurevych, Iryna", booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing", month = "11", year = "2019", publisher = "Association for Computational Linguistics", url = "https://arxiv.org/abs/1908.10084", } ``` #### MatryoshkaLoss ```bibtex @misc{kusupati2024matryoshka, title={Matryoshka Representation Learning}, author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi}, year={2024}, eprint={2205.13147}, archivePrefix={arXiv}, primaryClass={cs.LG} } ``` #### MultipleNegativesRankingLoss ```bibtex @misc{henderson2017efficient, title={Efficient Natural Language Response Suggestion for Smart Reply}, author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil}, year={2017}, eprint={1705.00652}, archivePrefix={arXiv}, primaryClass={cs.CL} } ```