Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+onnx/model.onnx_data filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,288 @@
+---
+license: apache-2.0
+datasets:
+- HIT-TMG/KaLM-embedding-pretrain-data
+- KaLM-Embedding/KaLM-embedding-finetuning-data
+language:
+- en
+- zh
+base_model:
+- KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5
+pipeline_tag: feature-extraction
+library_name: transformers.js
+tags:
+- Retrieval
+- STS
+- Classification
+- Clustering
+- Reranking
+- vllm
+---
+
+
+
+# KaLM-embedding-multilingual-mini-instruct-v2.5 (ONNX)
+
+
+This is an ONNX version of [KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5](https://huggingface.co/KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5). It was automatically converted and uploaded using [this Hugging Face Space](https://huggingface.co/spaces/onnx-community/convert-to-onnx).
+
+
+## Usage with Transformers.js
+
+
+See the pipeline documentation for `feature-extraction`: https://huggingface.co/docs/transformers.js/api/pipelines#module_pipelines.FeatureExtractionPipeline
+
+
+---
+
+
+<!-- <p align="center">
+    <img src="imgs/logo.jpg" width="800"/>
+<p>
+ -->
+<h1 align="center">KaLM-Embedding-V2.5</h1>
+
+<p align="center">
+  <a href="https://huggingface.co/KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5">
+    <img src="https://img.shields.io/badge/%F0%9F%A4%97_HuggingFace-Model-ffbd45.svg" alt="HuggingFace">
+  </a>
+  <a href="https://huggingface.co/spaces/nazib61/qdarnt">
+    <img src="https://img.shields.io/badge/%F0%9F%A4%97_HuggingFace-Demo-90EE90?style=flat" alt="Demo">
+  </a>
+  <a href="https://github.com/HITsz-TMG/KaLM-Embedding">
+    <img src="https://img.shields.io/badge/GitHub-Code-blue.svg?logo=github&" alt="GitHub Code">
+  </a>
+  <a href="https://arxiv.org/abs/2506.20923">
+    <img src="https://img.shields.io/badge/Paper-KaLM--Embedding-d4333f?logo=arxiv&logoColor=white&colorA=cccccc&colorB=d4333f&style=flat" alt="Paper">
+  </a>
+</p>
+
+## Short Description
+
+**KaLM-Embedding-V2.5** is a versatile and compact embedding model, which achieves SOTA performance among models of comparable size and competes with models 3–26x larger by leveraging superior training techniques and data.
+![perf](./imgs/perf.jpg)
+
+
+## Model Details
+- Model Size: 0.5B
+- Embedding Dimension: 896
+- Max Input Tokens: 32k
+- MRL dimensions: 896, 512, 256, 128, and 64
+- Attn: Bidirectional attention
+- Pooling: Mean pooling
+
+![archi](./imgs/archi.jpg)
+
+## Training Recipe
+- Large-scale weakly supervised pretraining
+- High-quality supervised finetuning
+- Contrastive distillation with fine-grained soft labels
+
+Additionally, focal-style sample reweighting and online hard-negative mixing are employed to emphasize difficult samples and enrich hard negatives.
+
+
+## 📑 Open-source Plan
+
+- [x] Model Checkpoint 
+    - [x] [KaLM-embedding-multilingual-mini-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-v1)
+    - [x] [KaLM-embedding-multilingual-mini-instruct-v1](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1)
+    - [x] [KaLM-embedding-multilingual-mini-instruct-v1.5](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v1.5)
+    - [x] [KaLM-embedding-multilingual-mini-instruct-v2](https://huggingface.co/HIT-TMG/KaLM-embedding-multilingual-mini-instruct-v2)
+    - [x] [KaLM-embedding-multilingual-mini-instruct-v2.5](https://huggingface.co/KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5)
+- [x] Training and Evaluation Code: [HITsz-TMG/KaLM-Embedding](https://github.com/HITsz-TMG/KaLM-Embedding)
+- [x] Technical Report: [KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model](https://arxiv.org/abs/2506.20923v4)
+- [x] Pre-training Data: [Pre-training Data](https://huggingface.co/datasets/HIT-TMG/KaLM-embedding-pretrain-data)
+- [x] Fine-tuning Data: [Fine-tuning Data](https://huggingface.co/datasets/KaLM-Embedding/KaLM-embedding-finetuning-data)
+
+## Evaluation
+### Overall results on MTEB (cmn, v1) and MTEB (eng, v1).
+![overall](./imgs/overall.jpg)
+
+### Detailed model performance on MTEB (cmn, v1).
+![mteb_cmn](./imgs/mteb_cmn.jpg)
+
+### Detailed model performance on MTEB (eng, v1).
+![mteb_cmn](./imgs/mteb_eng.jpg)
+
+### OOD evaluation: KaLM-Embedding-V2.5 exhibits strong OOD generalization, competing with the 15x larger model in real-world retrieval scenarios.
+![ood](./imgs/ood.jpg)
+
+### Matryoshka embedding evaluation: KaLM-Embedding-V2.5 maintains robust performance with matryoshka embeddings even at smaller dimensions.
+![matry](./imgs/matry.jpg)
+
+## Requirements
+Since we have used the Qwen2 model, we advise you to install `transformers>=4.37.0`, or you might encounter the following error:
+```
+KeyError: 'qwen2'
+```
+
+## Usage
+### sentence-transformers support
+Using this model becomes easy when you have [sentence-transformers](https://www.SBERT.net) installed:
+
+```
+pip install -U sentence-transformers
+```
+
+Then you can use the model like this:
+
+```python
+from sentence_transformers import SentenceTransformer
+import torch
+
+model = SentenceTransformer(
+    "KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5",
+    trust_remote_code=True,
+    model_kwargs={
+        "torch_dtype": torch.bfloat16,
+        "attn_implementation": "flash_attention_2",  # Optional
+    },
+)
+model.max_seq_length = 512
+
+sentences = ["This is an example sentence", "Each sentence is converted"]
+embeddings = model.encode(
+    sentences,
+    normalize_embeddings=True,
+    batch_size=256,
+    show_progress_bar=True,
+)
+print(embeddings)
+'''
+[[-0.01043701 -0.02172852  0.0100708  ... -0.02807617  0.00157166
+  -0.03637695]
+ [-0.00424194  0.02966309  0.03686523 ... -0.02587891  0.01953125
+  -0.00125122]]
+'''
+```
+
+We add task instructions for asymmetric tasks: retrieval, reranking, classification, and clustering.
+And, we add task instructions for both queries and passages in symmetric tasks, including STS and pair classification.
+If you want to add task instructions to the query, you can use the model like this:
+
+```python
+from sentence_transformers import SentenceTransformer
+import torch
+
+model = SentenceTransformer(
+    "KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5",
+    trust_remote_code=True,
+    model_kwargs={
+        "torch_dtype": torch.bfloat16,
+        "attn_implementation": "flash_attention_2",  # Optional
+    },
+)
+model.max_seq_length = 512
+
+sentences = ["This is an example sentence", "Each sentence is converted"]
+prompt = "Instruct: Classifying the category of french news.\nQuery:"
+embeddings = model.encode(
+    sentences,
+    prompt=prompt,
+    normalize_embeddings=True,
+    batch_size=256,
+    show_progress_bar=True,
+)
+print(embeddings)
+'''
+[[-0.01867676  0.02319336  0.00280762 ... -0.02075195  0.00196838
+  -0.0703125 ]
+ [-0.0067749   0.03491211  0.01434326 ... -0.0043335   0.00509644
+  -0.04174805]]
+'''
+```
+
+Or you can use `encode_query` and `encode_document` to automatically add the default prompt for queries (`"Instruct: Given a query, retrieve documents that answer the query \n Query: "`) and documents (`""`), respectively.
+
+```python
+from sentence_transformers import SentenceTransformer
+import torch
+
+model = SentenceTransformer(
+    "KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5",
+    trust_remote_code=True,
+    model_kwargs={
+        "torch_dtype": torch.bfloat16,
+        "attn_implementation": "flash_attention_2",  # Optional
+    },
+)
+model.max_seq_length = 512
+
+queries = [
+    "What is the capital of China?",
+    "Explain gravity",
+]
+documents = [
+    "The capital of China is Beijing.",
+    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun.",
+]
+
+query_embeddings = model.encode_query(queries)
+document_embeddings = model.encode_document(documents)
+
+similarities = model.similarity(query_embeddings, document_embeddings)
+print(similarities)
+'''
+tensor([[0.9034, 0.2563],
+        [0.3153, 0.7396]])
+'''
+```
+
+### vllm support
+```
+pip install -U vllm==0.8.5
+```
+```python
+import torch
+import vllm
+from vllm import LLM
+def get_detailed_instruct(task_description: str, query: str) -> str:
+    return f'Instruct: {task_description}\nQuery:{query}'
+
+task = 'Given a query, retrieve documents that answer the query'
+queries = [
+    get_detailed_instruct(task, 'What is the capital of China?'),
+    get_detailed_instruct(task, 'Explain gravity')
+]
+documents = [
+    "The capital of China is Beijing.",
+    "Gravity is a force that attracts two bodies towards each other. It gives weight to physical objects and is responsible for the movement of planets around the sun."
+]
+input_texts = queries + documents
+
+model = LLM(model="{MODEL_NAME_OR_PATH}", task="embed", trust_remote_code=True, dtype="float16")
+
+outputs = model.embed(input_texts)
+embeddings = torch.tensor([o.outputs.embedding for o in outputs])
+scores = (embeddings[:2] @ embeddings[2:].T)
+print(scores.tolist())
+```
+
+
+## Citation
+If you find this model useful, please consider giving a star and citation.
+```
+@misc{zhao2025kalmembeddingv2,
+      title={KaLM-Embedding-V2: Superior Training Techniques and Data Inspire A Versatile Embedding Model}, 
+      author={Xinping Zhao and Xinshuo Hu and Zifei Shan and Shouzheng Huang and Yao Zhou and Xin Zhang and Zetian Sun and Zhenyu Liu and Dongfang Li and Xinyuan Wei and Youcheng Pan and Yang Xiang and Meishan Zhang and Haofen Wang and Jun Yu and Baotian Hu and Min Zhang},
+      year={2025},
+      eprint={2506.20923},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2506.20923}, 
+}
+
+@misc{hu2025kalmembedding,
+      title={KaLM-Embedding: Superior Training Data Brings A Stronger Embedding Model}, 
+      author={Xinshuo Hu and Zifei Shan and Xinping Zhao and Zetian Sun and Zhenyu Liu and Dongfang Li and Shaolin Ye and Xinyuan Wei and Qian Chen and Baotian Hu and Haofen Wang and Jun Yu and Min Zhang},
+      year={2025},
+      eprint={2501.01028},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2501.01028}, 
+}
+```
+
+
+## Contact
+If you encounter any issue, feel free to contact us via the email: <[email protected]>, <[email protected]>

added_tokens.json

@@ -0,0 +1,5 @@
+{
+  "<|endoftext|>": 151643,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644
+}

config.json

@@ -0,0 +1,35 @@
+{
+  "_attn_implementation_autoset": true,
+  "_name_or_path": "KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5",
+  "architectures": [
+    "Qwen2Model"
+  ],
+  "attention_dropout": 0.0,
+  "auto_map": {
+    "AutoModel": "KaLM-Embedding/KaLM-embedding-multilingual-mini-instruct-v2.5--modeling.Qwen2Model"
+  },
+  "bos_token_id": 151643,
+  "eos_token_id": 151643,
+  "export_model_type": "transformer",
+  "hidden_act": "silu",
+  "hidden_size": 896,
+  "initializer_range": 0.02,
+  "intermediate_size": 4864,
+  "is_causal": false,
+  "max_position_embeddings": 131072,
+  "max_window_layers": 24,
+  "model_type": "qwen2",
+  "num_attention_heads": 14,
+  "num_hidden_layers": 24,
+  "num_key_value_heads": 2,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.49.0",
+  "use_cache": false,
+  "use_sliding_window": false,
+  "vocab_size": 151936
+}

onnx/model.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7aad5285192007982fc0a13bcf35148247bad2a3d63480d65d6db0e865c266d9
+size 825306

onnx/model.onnx_data

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8843a6b9a1eb48423dac7284d0d3da2fd99468df7d4d758cb49abe385e470196
+size 2043215360

onnx/model_bnb4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc1349bd3e217e2403b48591f942246eb8dcdf9304a7943d373ce46595564ecc
+size 814022610

onnx/model_fp16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:975eecdfd194a34bd2a54ba9f8b07b3c4cec228c5686ecb727d3c2cc1ba0867f
+size 1022406024

onnx/model_int8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24b1592de862610a04152900c08fc5ccafc8c9f020c7c3d4ff3815f22426aec5
+size 562341284

onnx/model_q4.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8fbdabd440c231dd0bb6482e78bba8a75fe0bcd7eb667aebc68afc4f134aad37
+size 836385426

onnx/model_q4f16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c5eb8abd440e7778cead911606521f52e1b35067bb648484f2928d83f2b314b4
+size 508055107

onnx/model_quantized.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24b1592de862610a04152900c08fc5ccafc8c9f020c7c3d4ff3815f22426aec5
+size 562341284

onnx/model_uint8.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca56ccd7877c920845ddaaaa57cdfa5f2086d1dfd70b381abc3596a612310144
+size 562341369

quantize_config.json

@@ -0,0 +1,18 @@
+{
+    "modes": [
+        "fp16",
+        "q8",
+        "int8",
+        "uint8",
+        "q4",
+        "q4f16",
+        "bnb4"
+    ],
+    "per_channel": false,
+    "reduce_range": false,
+    "block_size": null,
+    "is_symmetric": true,
+    "accuracy_level": null,
+    "quant_type": 1,
+    "op_block_list": null
+}

special_tokens_map.json

@@ -0,0 +1,20 @@
+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "eos_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f79052deba517b0663d877714e117a31a4a6243cddb85fc4443c80a2fa65a20
+size 11419302

tokenizer_config.json

@@ -0,0 +1,51 @@
+{
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>"
+  ],
+  "bos_token": null,
+  "chat_template": "{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %}",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|endoftext|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "max_length": 512,
+  "model_max_length": 32768,
+  "pad_to_multiple_of": null,
+  "pad_token": "<|endoftext|>",
+  "pad_token_type_id": 0,
+  "padding_side": "left",
+  "split_special_tokens": false,
+  "stride": 0,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": null
+}