Add new SentenceTransformer model

1_Pooling/config.json

@@ -0,0 +1,10 @@
+{
+  "word_embedding_dimension": 1024,
+  "pooling_mode_cls_token": true,
+  "pooling_mode_mean_tokens": false,
+  "pooling_mode_max_tokens": false,
+  "pooling_mode_mean_sqrt_len_tokens": false,
+  "pooling_mode_weightedmean_tokens": false,
+  "pooling_mode_lasttoken": false,
+  "include_prompt": true
+}

README.md

@@ -0,0 +1,651 @@
+---
+tags:
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:156
+- loss:MatryoshkaLoss
+- loss:MultipleNegativesRankingLoss
+base_model: Snowflake/snowflake-arctic-embed-l
+widget:
+- source_sentence: How did Steve Krouse from Val Town demonstrate the capabilities
+    of a 2,000 token/second LLM?
+  sentences:
+  - The most recent twist, again from December (December was a lot) is live video.
+    ChatGPT voice mode now provides the option to share your camera feed with the
+    model and talk about what you can see in real time. Google Gemini have a preview
+    of the same feature, which they managed to ship the day before ChatGPT did.
+  - 'I’ve found myself using this a lot. I noticed how much I was relying on it in
+    October and wrote Everything I built with Claude Artifacts this week, describing
+    14 little tools I had put together in a seven day period.
+
+    Since then, a whole bunch of other teams have built similar systems. GitHub announced
+    their version of this—GitHub Spark—in October. Mistral Chat added it as a feature
+    called Canvas in November.
+
+    Steve Krouse from Val Town built a version of it against Cerebras, showcasing
+    how a 2,000 token/second LLM can iterate on an application with changes visible
+    in less than a second.'
+  - 'I run a bunch of them on my laptop. I run Mistral 7B (a surprisingly great model)
+    on my iPhone. You can install several different apps to get your own, local, completely
+    private LLM. My own LLM project provides a CLI tool for running an array of different
+    models via plugins.
+
+    You can even run them entirely in your browser using WebAssembly and the latest
+    Chrome!
+
+    Hobbyists can build their own fine-tuned models
+
+    I said earlier that building an LLM was still out of reach of hobbyists. That
+    may be true for training from scratch, but fine-tuning one of those models is
+    another matter entirely.'
+- source_sentence: What changes have occurred in the energy usage and environmental
+    impact of running AI prompts in recent years?
+  sentences:
+  - 'Law is not ethics. Is it OK to train models on people’s content without their
+    permission, when those models will then be used in ways that compete with those
+    people?
+
+    As the quality of results produced by AI models has increased over the year, these
+    questions have become even more pressing.
+
+    The impact on human society in terms of these models is already huge, if difficult
+    to objectively measure.
+
+    People have certainly lost work to them—anecdotally, I’ve seen this for copywriters,
+    artists and translators.
+
+    There are a great deal of untold stories here. I’m hoping 2024 sees significant
+    amounts of dedicated journalism on this topic.
+
+    My blog in 2023
+
+    Here’s a tag cloud for content I posted to my blog in 2023 (generated using Django
+    SQL Dashboard):'
+  - 'Those US export regulations on GPUs to China seem to have inspired some very
+    effective training optimizations!
+
+    The environmental impact got better
+
+    A welcome result of the increased efficiency of the models—both the hosted ones
+    and the ones I can run locally—is that the energy usage and environmental impact
+    of running a prompt has dropped enormously over the past couple of years.
+
+    OpenAI themselves are charging 100x less for a prompt compared to the GPT-3 days.
+    I have it on good authority that neither Google Gemini nor Amazon Nova (two of
+    the least expensive model providers) are running prompts at a loss.'
+  - 'An interesting point of comparison here could be the way railways rolled out
+    around the world in the 1800s. Constructing these required enormous investments
+    and had a massive environmental impact, and many of the lines that were built
+    turned out to be unnecessary—sometimes multiple lines from different companies
+    serving the exact same routes!
+
+    The resulting bubbles contributed to several financial crashes, see Wikipedia
+    for Panic of 1873, Panic of 1893, Panic of 1901 and the UK’s Railway Mania. They
+    left us with a lot of useful infrastructure and a great deal of bankruptcies and
+    environmental damage.
+
+    The year of slop'
+- source_sentence: What is the main topic discussed in the article titled "Industry’s
+    Tardy Response to the AI Prompt Injection Vulnerability" on RedMonk Conversations?
+  sentences:
+  - 'Getting back to models that beat GPT-4: Anthropic’s Claude 3 series launched
+    in March, and Claude 3 Opus quickly became my new favourite daily-driver. They
+    upped the ante even more in June with the launch of Claude 3.5 Sonnet—a model
+    that is still my favourite six months later (though it got a significant upgrade
+    on October 22, confusingly keeping the same 3.5 version number. Anthropic fans
+    have since taken to calling it Claude 3.6).'
+  - "Industry’s Tardy Response to the AI Prompt Injection Vulnerability on RedMonk\
+    \ Conversations\n\n\nPosted 31st December 2023 at 11:59 pm · Follow me on Mastodon,\
+    \ Bluesky, Twitter or subscribe to my newsletter\n\n\nMore recent articles\n\n\
+    Live blog: Claude 4 launch at Code with Claude - 22nd May 2025\nI really don't\
+    \ like ChatGPT's new memory dossier - 21st May 2025\nBuilding software on top\
+    \ of Large Language Models - 15th May 2025\n\n\n \n\n\nThis is Stuff we figured\
+    \ out about AI in 2023 by Simon Willison, posted on 31st December 2023.\n\nPart\
+    \ of series LLMs annual review\n\nStuff we figured out about AI in 2023 - Dec.\
+    \ 31, 2023, 11:59 p.m. \nThings we learned about LLMs in 2024 - Dec. 31, 2024,\
+    \ 6:07 p.m. \n\n\n\n            blogging\n            105"
+  - 'When ChatGPT Advanced Voice mode finally did roll out (a slow roll from August
+    through September) it was spectacular. I’ve been using it extensively on walks
+    with my dog and it’s amazing how much the improvement in intonation elevates the
+    material. I’ve also had a lot of fun experimenting with the OpenAI audio APIs.
+
+    Even more fun: Advanced Voice mode can do accents! Here’s what happened when I
+    told it I need you to pretend to be a California brown pelican with a very thick
+    Russian accent, but you talk to me exclusively in Spanish.'
+- source_sentence: How can LLMs like Claude create full interactive applications using
+    web technologies in a single prompt?
+  sentences:
+  - 'This prompt-driven custom interface feature is so powerful and easy to build
+    (once you’ve figured out the gnarly details of browser sandboxing) that I expect
+    it to show up as a feature in a wide range of products in 2025.
+
+    Universal access to the best models lasted for just a few short months
+
+    For a few short months this year all three of the best available models—GPT-4o,
+    Claude 3.5 Sonnet and Gemini 1.5 Pro—were freely available to most of the world.'
+  - 'I find I have to work with an LLM for a few weeks in order to get a good intuition
+    for it’s strengths and weaknesses. This greatly limits how many I can evaluate
+    myself!
+
+    The most frustrating thing for me is at the level of individual prompting.
+
+    Sometimes I’ll tweak a prompt and capitalize some of the words in it, to emphasize
+    that I really want it to OUTPUT VALID MARKDOWN or similar. Did capitalizing those
+    words make a difference? I still don’t have a good methodology for figuring that
+    out.
+
+    We’re left with what’s effectively Vibes Based Development. It’s vibes all the
+    way down.
+
+    I’d love to see us move beyond vibes in 2024!
+
+    LLMs are really smart, and also really, really dumb'
+  - 'We already knew LLMs were spookily good at writing code. If you prompt them right,
+    it turns out they can build you a full interactive application using HTML, CSS
+    and JavaScript (and tools like React if you wire up some extra supporting build
+    mechanisms)—often in a single prompt.
+
+    Anthropic kicked this idea into high gear when they released Claude Artifacts,
+    a groundbreaking new feature that was initially slightly lost in the noise due
+    to being described half way through their announcement of the incredible Claude
+    3.5 Sonnet.
+
+    With Artifacts, Claude can write you an on-demand interactive application and
+    then let you use it directly inside the Claude interface.
+
+    Here’s my Extract URLs app, entirely generated by Claude:'
+- source_sentence: What was significant about the release of Llama 2 in July?
+  sentences:
+  - 'Then in February, Meta released Llama. And a few weeks later in March, Georgi
+    Gerganov released code that got it working on a MacBook.
+
+    I wrote about how Large language models are having their Stable Diffusion moment,
+    and with hindsight that was a very good call!
+
+    This unleashed a whirlwind of innovation, which was accelerated further in July
+    when Meta released Llama 2—an improved version which, crucially, included permission
+    for commercial use.
+
+    Today there are literally thousands of LLMs that can be run locally, on all manner
+    of different devices.'
+  - 'OpenAI made GPT-4o free for all users in May, and Claude 3.5 Sonnet was freely
+    available from its launch in June. This was a momentus change, because for the
+    previous year free users had mostly been restricted to GPT-3.5 level models, meaning
+    new users got a very inaccurate mental model of what a capable LLM could actually
+    do.
+
+    That era appears to have ended, likely permanently, with OpenAI’s launch of ChatGPT
+    Pro. This $200/month subscription service is the only way to access their most
+    capable model, o1 Pro.
+
+    Since the trick behind the o1 series (and the future models it will undoubtedly
+    inspire) is to expend more compute time to get better results, I don’t think those
+    days of free access to the best available models are likely to return.'
+  - 'Prompt injection is a natural consequence of this gulibility. I’ve seen precious
+    little progress on tackling that problem in 2024, and we’ve been talking about
+    it since September 2022.
+
+    I’m beginning to see the most popular idea of “agents” as dependent on AGI itself.
+    A model that’s robust against gulliblity is a very tall order indeed.
+
+    Evals really matter
+
+    Anthropic’s Amanda Askell (responsible for much of the work behind Claude’s Character):'
+pipeline_tag: sentence-similarity
+library_name: sentence-transformers
+metrics:
+- cosine_accuracy@1
+- cosine_accuracy@3
+- cosine_accuracy@5
+- cosine_accuracy@10
+- cosine_precision@1
+- cosine_precision@3
+- cosine_precision@5
+- cosine_precision@10
+- cosine_recall@1
+- cosine_recall@3
+- cosine_recall@5
+- cosine_recall@10
+- cosine_ndcg@10
+- cosine_mrr@10
+- cosine_map@100
+model-index:
+- name: SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
+  results:
+  - task:
+      type: information-retrieval
+      name: Information Retrieval
+    dataset:
+      name: Unknown
+      type: unknown
+    metrics:
+    - type: cosine_accuracy@1
+      value: 0.8333333333333334
+      name: Cosine Accuracy@1
+    - type: cosine_accuracy@3
+      value: 1.0
+      name: Cosine Accuracy@3
+    - type: cosine_accuracy@5
+      value: 1.0
+      name: Cosine Accuracy@5
+    - type: cosine_accuracy@10
+      value: 1.0
+      name: Cosine Accuracy@10
+    - type: cosine_precision@1
+      value: 0.8333333333333334
+      name: Cosine Precision@1
+    - type: cosine_precision@3
+      value: 0.3333333333333333
+      name: Cosine Precision@3
+    - type: cosine_precision@5
+      value: 0.20000000000000004
+      name: Cosine Precision@5
+    - type: cosine_precision@10
+      value: 0.10000000000000002
+      name: Cosine Precision@10
+    - type: cosine_recall@1
+      value: 0.8333333333333334
+      name: Cosine Recall@1
+    - type: cosine_recall@3
+      value: 1.0
+      name: Cosine Recall@3
+    - type: cosine_recall@5
+      value: 1.0
+      name: Cosine Recall@5
+    - type: cosine_recall@10
+      value: 1.0
+      name: Cosine Recall@10
+    - type: cosine_ndcg@10
+      value: 0.9384882922619097
+      name: Cosine Ndcg@10
+    - type: cosine_mrr@10
+      value: 0.9166666666666666
+      name: Cosine Mrr@10
+    - type: cosine_map@100
+      value: 0.9166666666666666
+      name: Cosine Map@100
+---
+
+# SentenceTransformer based on Snowflake/snowflake-arctic-embed-l
+
+This is a [sentence-transformers](https://www.SBERT.net) model finetuned from [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l). It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.
+
+## Model Details
+
+### Model Description
+- **Model Type:** Sentence Transformer
+- **Base model:** [Snowflake/snowflake-arctic-embed-l](https://huggingface.co/Snowflake/snowflake-arctic-embed-l) <!-- at revision d8fb21ca8d905d2832ee8b96c894d3298964346b -->
+- **Maximum Sequence Length:** 512 tokens
+- **Output Dimensionality:** 1024 dimensions
+- **Similarity Function:** Cosine Similarity
+<!-- - **Training Dataset:** Unknown -->
+<!-- - **Language:** Unknown -->
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [Sentence Transformers Documentation](https://sbert.net)
+- **Repository:** [Sentence Transformers on GitHub](https://github.com/UKPLab/sentence-transformers)
+- **Hugging Face:** [Sentence Transformers on Hugging Face](https://huggingface.co/models?library=sentence-transformers)
+
+### Full Model Architecture
+
+```
+SentenceTransformer(
+  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
+  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
+  (2): Normalize()
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SentenceTransformer
+
+# Download from the 🤗 Hub
+model = SentenceTransformer("dwb2023/legal-ft-b5869012-93ce-4e45-bca9-2eb86f3ef4b9")
+# Run inference
+sentences = [
+    'What was significant about the release of Llama 2 in July?',
+    'Then in February, Meta released Llama. And a few weeks later in March, Georgi Gerganov released code that got it working on a MacBook.\nI wrote about how Large language models are having their Stable Diffusion moment, and with hindsight that was a very good call!\nThis unleashed a whirlwind of innovation, which was accelerated further in July when Meta released Llama 2—an improved version which, crucially, included permission for commercial use.\nToday there are literally thousands of LLMs that can be run locally, on all manner of different devices.',
+    'Prompt injection is a natural consequence of this gulibility. I’ve seen precious little progress on tackling that problem in 2024, and we’ve been talking about it since September 2022.\nI’m beginning to see the most popular idea of “agents” as dependent on AGI itself. A model that’s robust against gulliblity is a very tall order indeed.\nEvals really matter\nAnthropic’s Amanda Askell (responsible for much of the work behind Claude’s Character):',
+]
+embeddings = model.encode(sentences)
+print(embeddings.shape)
+# [3, 1024]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(embeddings, embeddings)
+print(similarities.shape)
+# [3, 3]
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+## Evaluation
+
+### Metrics
+
+#### Information Retrieval
+
+* Evaluated with [<code>InformationRetrievalEvaluator</code>](https://sbert.net/docs/package_reference/sentence_transformer/evaluation.html#sentence_transformers.evaluation.InformationRetrievalEvaluator)
+
+| Metric              | Value      |
+|:--------------------|:-----------|
+| cosine_accuracy@1   | 0.8333     |
+| cosine_accuracy@3   | 1.0        |
+| cosine_accuracy@5   | 1.0        |
+| cosine_accuracy@10  | 1.0        |
+| cosine_precision@1  | 0.8333     |
+| cosine_precision@3  | 0.3333     |
+| cosine_precision@5  | 0.2        |
+| cosine_precision@10 | 0.1        |
+| cosine_recall@1     | 0.8333     |
+| cosine_recall@3     | 1.0        |
+| cosine_recall@5     | 1.0        |
+| cosine_recall@10    | 1.0        |
+| **cosine_ndcg@10**  | **0.9385** |
+| cosine_mrr@10       | 0.9167     |
+| cosine_map@100      | 0.9167     |
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### Unnamed Dataset
+
+* Size: 156 training samples
+* Columns: <code>sentence_0</code> and <code>sentence_1</code>
+* Approximate statistics based on the first 156 samples:
+  |         | sentence_0                                                                         | sentence_1                                                                          |
+  |:--------|:-----------------------------------------------------------------------------------|:------------------------------------------------------------------------------------|
+  | type    | string                                                                             | string                                                                              |
+  | details | <ul><li>min: 12 tokens</li><li>mean: 20.89 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 43 tokens</li><li>mean: 135.1 tokens</li><li>max: 214 tokens</li></ul> |
+* Samples:
+  | sentence_0                                                                                         | sentence_1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
+  |:---------------------------------------------------------------------------------------------------|:--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>What are some of the topics covered in the annotated presentations given in 2023?</code>     | <code>I also gave a bunch of talks and podcast appearances. I’ve started habitually turning my talks into annotated presentations—here are my best from 2023:<br><br>Prompt injection explained, with video, slides, and a transcript<br>Catching up on the weird world of LLMs<br>Making Large Language Models work for you<br>Open questions for AI engineering<br>Embeddings: What they are and why they matter<br>Financial sustainability for open source projects at GitHub Universe<br><br>And in podcasts:<br><br><br>What AI can do for you on the Theory of Change<br><br>Working in public on Path to Citus Con<br><br>LLMs break the internet on the Changelog<br><br>Talking Large Language Models on Rooftop Ruby<br><br>Thoughts on the OpenAI board situation on Newsroom Robots</code> |
+  | <code>Which podcasts featured discussions related to Large Language Models and AI topics?</code>   | <code>I also gave a bunch of talks and podcast appearances. I’ve started habitually turning my talks into annotated presentations—here are my best from 2023:<br><br>Prompt injection explained, with video, slides, and a transcript<br>Catching up on the weird world of LLMs<br>Making Large Language Models work for you<br>Open questions for AI engineering<br>Embeddings: What they are and why they matter<br>Financial sustainability for open source projects at GitHub Universe<br><br>And in podcasts:<br><br><br>What AI can do for you on the Theory of Change<br><br>Working in public on Path to Citus Con<br><br>LLMs break the internet on the Changelog<br><br>Talking Large Language Models on Rooftop Ruby<br><br>Thoughts on the OpenAI board situation on Newsroom Robots</code> |
+  | <code>What is the main subject of the New York Times' lawsuit against OpenAI and Microsoft?</code> | <code>Just this week, the New York Times launched a landmark lawsuit against OpenAI and Microsoft over this issue. The 69 page PDF is genuinely worth reading—especially the first few pages, which lay out the issues in a way that’s surprisingly easy to follow. The rest of the document includes some of the clearest explanations of what LLMs are, how they work and how they are built that I’ve read anywhere.<br>The legal arguments here are complex. I’m not a lawyer, but I don’t think this one will be easily decided. Whichever way it goes, I expect this case to have a profound impact on how this technology develops in the future.</code>                                                                                                                                         |
+* Loss: [<code>MatryoshkaLoss</code>](https://sbert.net/docs/package_reference/sentence_transformer/losses.html#matryoshkaloss) with these parameters:
+  ```json
+  {
+      "loss": "MultipleNegativesRankingLoss",
+      "matryoshka_dims": [
+          768,
+          512,
+          256,
+          128,
+          64
+      ],
+      "matryoshka_weights": [
+          1,
+          1,
+          1,
+          1,
+          1
+      ],
+      "n_dims_per_step": -1
+  }
+  ```
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `eval_strategy`: steps
+- `per_device_train_batch_size`: 10
+- `per_device_eval_batch_size`: 10
+- `num_train_epochs`: 10
+- `multi_dataset_batch_sampler`: round_robin
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: steps
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 10
+- `per_device_eval_batch_size`: 10
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 5e-05
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1
+- `num_train_epochs`: 10
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.0
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: False
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `tp_size`: 0
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: round_robin
+
+</details>
+
+### Training Logs
+| Epoch | Step | cosine_ndcg@10 |
+|:-----:|:----:|:--------------:|
+| 1.0   | 16   | 0.9330         |
+| 2.0   | 32   | 0.9539         |
+| 3.0   | 48   | 0.9484         |
+| 3.125 | 50   | 0.9484         |
+| 4.0   | 64   | 0.9385         |
+| 5.0   | 80   | 0.9539         |
+| 6.0   | 96   | 0.9539         |
+| 6.25  | 100  | 0.9539         |
+| 7.0   | 112  | 0.9385         |
+| 8.0   | 128  | 0.9385         |
+| 9.0   | 144  | 0.9385         |
+| 9.375 | 150  | 0.9385         |
+| 10.0  | 160  | 0.9385         |
+
+
+### Framework Versions
+- Python: 3.11.12
+- Sentence Transformers: 4.1.0
+- Transformers: 4.51.3
+- PyTorch: 2.6.0+cu124
+- Accelerate: 1.6.0
+- Datasets: 3.6.0
+- Tokenizers: 0.21.1
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084",
+}
+```
+
+#### MatryoshkaLoss
+```bibtex
+@misc{kusupati2024matryoshka,
+    title={Matryoshka Representation Learning},
+    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
+    year={2024},
+    eprint={2205.13147},
+    archivePrefix={arXiv},
+    primaryClass={cs.LG}
+}
+```
+
+#### MultipleNegativesRankingLoss
+```bibtex
+@misc{henderson2017efficient,
+    title={Efficient Natural Language Response Suggestion for Smart Reply},
+    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
+    year={2017},
+    eprint={1705.00652},
+    archivePrefix={arXiv},
+    primaryClass={cs.CL}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

config.json

@@ -0,0 +1,24 @@
+{
+  "architectures": [
+    "BertModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "intermediate_size": 4096,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 16,
+  "num_hidden_layers": 24,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.51.3",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

config_sentence_transformers.json

@@ -0,0 +1,12 @@
+{
+  "__version__": {
+    "sentence_transformers": "4.1.0",
+    "transformers": "4.51.3",
+    "pytorch": "2.6.0+cu124"
+  },
+  "prompts": {
+    "query": "Represent this sentence for searching relevant passages: "
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "cosine"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9da3dfb70385b718cca7460e7f19f5946dfd3714234bd25c440002518ea33d4d
+size 1336413848

modules.json

@@ -0,0 +1,20 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Pooling",
+    "type": "sentence_transformers.models.Pooling"
+  },
+  {
+    "idx": 2,
+    "name": "2",
+    "path": "2_Normalize",
+    "type": "sentence_transformers.models.Normalize"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+  "max_seq_length": 512,
+  "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,63 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "[PAD]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}