sentence_transformers_support (#2)

- Add support for Sentence Transformer (0abe4aafd9afb6244fd6e400d537a013370413f3)

1_SpladePooling/config.json

@@ -0,0 +1,5 @@
+{
+    "pooling_strategy": "max",
+    "activation_function": "relu",
+    "word_embedding_dimension": null
+}

README.md

@@ -1,5 +1,94 @@
 ---
 license: cc-by-nc-sa-4.0
+language: "en"
+tags:
+- splade
+- query-expansion
+- document-expansion
+- bag-of-words
+- passage-retrieval
+- knowledge-distillation
+- sentence-transformers
+- sparse-encoder
+- sparse
+pipeline_tag: feature-extraction
+library_name: sentence-transformers
+datasets:
+- ms_marco
 ---
 
-Also called SPLADE-v2 in some papers
+Also called SPLADE-v2 in some papers
+
+## SPLADE_v2_distill
+
+SPLADE model for passage retrieval. For additional details, please visit:
+* paper: https://arxiv.org/abs/2109.10086
+* code: https://github.com/naver/splade
+
+## Model Details
+
+This is a [SPLADE Sparse Encoder](https://www.sbert.net/docs/sparse_encoder/usage/usage.html) model. It maps sentences & paragraphs to a 30522-dimensional sparse vector space and can be used for semantic search and sparse retrieval.
+
+### Model Description
+- **Model Type:** SPLADE Sparse Encoder
+- **Base model:** [distilbert/distilbert-base-uncased](https://huggingface.co/distilbert/distilbert-base-uncased)
+- **Maximum Sequence Length:** 512 tokens (256 for evaluation reproduction)
+- **Output Dimensionality:** 30522 dimensions
+- **Similarity Function:** Dot Product
+
+### Full Model Architecture
+
+```
+SparseEncoder(
+  (0): MLMTransformer({'max_seq_length': 512, 'do_lower_case': False}) with MLMTransformer model: BertForMaskedLM 
+  (1): SpladePooling({'pooling_strategy': 'max', 'activation_function': 'relu', 'word_embedding_dimension': 30522})
+)
+```
+
+## Usage
+
+### Direct Usage (Sentence Transformers)
+
+First install the Sentence Transformers library:
+
+```bash
+pip install -U sentence-transformers
+```
+
+Then you can load this model and run inference.
+```python
+from sentence_transformers import SparseEncoder
+
+# Download from the 🤗 Hub
+model = SparseEncoder("naver/splade_v2_distil")
+# Run inference
+queries = ["what causes aging fast"]
+documents = [
+    "UV-A light, specifically, is what mainly causes tanning, skin aging, and cataracts, UV-B causes sunburn, skin aging and skin cancer, and UV-C is the strongest, and therefore most effective at killing microorganisms. Again â\x80\x93 single words and multiple bullets.",
+    "Answers from Ronald Petersen, M.D. Yes, Alzheimer's disease usually worsens slowly. But its speed of progression varies, depending on a person's genetic makeup, environmental factors, age at diagnosis and other medical conditions. Still, anyone diagnosed with Alzheimer's whose symptoms seem to be progressing quickly â\x80\x94 or who experiences a sudden decline â\x80\x94 should see his or her doctor.",
+    "Bell's palsy and Extreme tiredness and Extreme fatigue (2 causes) Bell's palsy and Extreme tiredness and Hepatitis (2 causes) Bell's palsy and Extreme tiredness and Liver pain (2 causes) Bell's palsy and Extreme tiredness and Lymph node swelling in children (2 causes)",
+]
+query_embeddings = model.encode_query(queries)
+document_embeddings = model.encode_document(documents)
+print(query_embeddings.shape, document_embeddings.shape)
+# [1, 30522] [3, 30522]
+
+# Get the similarity scores for the embeddings
+similarities = model.similarity(query_embeddings, document_embeddings)
+print(similarities)
+# tensor([[15.0450, 13.3137,  5.3127]])
+
+```
+
+## Citation
+
+If you use our checkpoint, please cite our work:
+
+```
+@article{formal2021splade,
+  title={SPLADE v2: Sparse lexical and expansion model for information retrieval},
+  author={Formal, Thibault and Lassance, Carlos and Piwowarski, Benjamin and Clinchant, St{\'e}phane},
+  journal={arXiv preprint arXiv:2109.10086},
+  year={2021}
+}
+```

config_sentence_transformers.json

@@ -0,0 +1,14 @@
+{
+  "model_type": "SparseEncoder",
+  "__version__": {
+    "sentence_transformers": "5.0.0",
+    "transformers": "4.50.3",
+    "pytorch": "2.6.0+cu124"
+  },
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "dot"
+}

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.sparse_encoder.models.MLMTransformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_SpladePooling",
+    "type": "sentence_transformers.sparse_encoder.models.SpladePooling"
+  }
+]

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+    "max_seq_length": 512,
+    "do_lower_case": false
+}