initial commit

.gitattributes

@@ -0,0 +1,36 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,160 @@
+---
+quantized_by: Nomic
+pipeline_tag: sentence-similarity
+base_model: nomic-ai/nomic-embed-code
+base_model_relation: quantized
+tags:
+- sentence-similarity
+- feature-extraction
+license: apache-2.0
+---
+
+# Llama.cpp Quantizations of Nomic Embed Code: A State-of-the-Art Code Retriever
+
+[Blog](https://www.nomic.ai/blog/posts/introducing-state-of-the-art-nomic-embed-code) | [Technical Report](https://arxiv.org/abs/2412.01007) | [AWS SageMaker](https://aws.amazon.com/marketplace/seller-profile?id=seller-tpqidcj54zawi) | [Atlas Embedding and Unstructured Data Analytics Platform](https://atlas.nomic.ai)
+
+Using <a href="https://github.com/ggml-org/llama.cpp">llama.cpp</a> commit <a href="https://github.com/ggml-org/llama.cpp/tree/11683f579b2e0efac27955945a4bcadd8e123af3">11683f579</a> for quantization.
+
+Original model: [nomic-embed-code](https://huggingface.co/nomic-ai/nomic-embed-code)
+
+## Usage
+
+This model can be used with the [llama.cpp server](https://github.com/ggml-org/llama.cpp/blob/master/examples/embedding#post-v1embeddings-openai-compatible-embeddings-api) and other software that supports llama.cpp embedding models.
+
+Queries embedded with `nomic-embed-code` must begin with the following prefix:
+```
+Represent this query for searching relevant code:
+```
+
+For example, the code below shows how to use the prefix to embed user questions, e.g. in a RAG application.
+
+Start a llama.cpp server:
+```
+llama-server -m nomic-embed-code.Q4_0.gguf --embeddings --pooling last
+```
+
+And run this code:
+```python
+import requests
+from textwrap import dedent
+
+def dot(va, vb):
+    return sum(a*b for a, b in zip(va, vb))
+def embed(texts):
+    resp = requests.post('http://localhost:8080/v1/embeddings', json={'input': texts}).json()
+    return [d['embedding'] for d in resp['data']]
+
+docs = [
+    dedent("""\
+    def fn(n):
+        if n < 0:
+            raise ValueError
+        return 1 if n == 0 else n * fn(n - 1)
+    """).strip(),
+    dedent("""\
+    def fn(n):
+        print(("Fizz" * (n % 3 == 0) + "Buzz" * (n % 5 == 0)) or n)
+    """).strip(),
+]
+docs_embed = embed(docs)
+
+query = 'Calculate the n-th factorial'
+query_embed = embed(['Represent this query for searching relevant code: ' + query])[0]
+print(f'query: {query!r}')
+for d, e in zip(docs, docs_embed):
+    print(f'\nsimilarity {dot(query_embed, e):.2f}:\n{d}')
+```
+
+You should see output similar to this:
+```
+query: 'Calculate the n-th factorial'
+
+similarity 0.49:
+def fn(n):
+    if n < 0:
+        raise ValueError
+    return 1 if n == 0 else n * fn(n - 1)
+
+similarity 0.32:
+def fn(n):
+    print(("Fizz" * (n % 3 == 0) + "Buzz" * (n % 5 == 0)) or n)
+```
+
+## Download a file (not the whole branch) from below:
+
+| Filename                                                                                                                       | Quant Type | File Size | Description                                                                                    |
+| --------                                                                                                                       | ---------- | --------: | -----------                                                                                    |
+| [nomic-embed-code.f32.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.f32.gguf)         | f32        |  26.35GiB | Full FP32 weights.                                                                             |
+| [nomic-embed-code.f16.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.f16.gguf)         | f16        |  13.18GiB | Full FP16 weights.                                                                             |
+| [nomic-embed-code.bf16.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.bf16.gguf)       | bf16       |  13.18GiB | Full BF16 weights.                                                                             |
+| [nomic-embed-code.Q8\_0.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q8_0.gguf)      | Q8\_0      |   7.00GiB | Extremely high quality, generally unneeded but max available quant.                            |
+| [nomic-embed-code.Q6\_K.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q6_K.gguf)      | Q6\_K      |   5.41GiB | Very high quality, near perfect, *recommended*.                                                |
+| [nomic-embed-code.Q5\_K\_M.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q5_K_M.gguf) | Q5\_K\_M   |   4.72GiB | High quality, *recommended*.                                                                   |
+| [nomic-embed-code.Q5\_K\_S.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q5_K_S.gguf) | Q5\_K\_S   |   4.60GiB | High quality, *recommended*.                                                                   |
+| [nomic-embed-code.Q4\_1.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q4_1.gguf)      | Q4\_1      |   4.22GiB | Legacy format, similar performance to Q4\_K\_S but with improved tokens/watt on Apple silicon. |
+| [nomic-embed-code.Q4\_K\_M.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q4_K_M.gguf) | Q4\_K\_M   |   4.08GiB | Good quality, default size for most use cases, *recommended*.                                  |
+| [nomic-embed-code.Q4\_K\_S.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q4_K_S.gguf) | Q4\_K\_S   |   3.87GiB | Slightly lower quality with more space savings, *recommended*.                                 |
+| [nomic-embed-code.Q4\_0.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q4_0.gguf)      | Q4\_0      |   3.84GiB | Legacy format, offers online repacking for ARM and AVX CPU inference.                          |
+| [nomic-embed-code.Q3\_K\_L.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q3_K_L.gguf) | Q3\_K\_L   |   3.59GiB | Lower quality but usable, good for low RAM availability.                                       |
+| [nomic-embed-code.Q3\_K\_M.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q3_K_M.gguf) | Q3\_K\_M   |   3.33GiB | Low quality.                                                                                   |
+| [nomic-embed-code.Q3\_K\_S.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q3_K_S.gguf) | Q3\_K\_S   |   3.03GiB | Low quality, not recommended.                                                                  |
+| [nomic-embed-code.Q2\_K.gguf](https://huggingface.co/nomic-ai/nomic-embed-code-GGUF/blob/main/nomic-embed-code.Q2_K.gguf)      | Q2\_K      |   2.64GiB | Very low quality but surprisingly usable.                                                      |
+
+## Model Overview
+`nomic-embed-code` is a state-of-the-art code embedding model that excels at code retrieval tasks:
+
+- **High Performance**: Outperforms Voyage Code 3 and OpenAI Embed 3 Large on CodeSearchNet
+- **Multilingual Code Support**: Trained for multiple programming languages (Python, Java, Ruby, PHP, JavaScript, Go)
+- **Advanced Architecture**: 7B parameter code embedding model
+- **Fully Open-Source**: Model weights, training data, and [evaluation code](https://github.com/gangiswag/cornstack/) released
+
+| Model | Python | Java | Ruby | PHP | JavaScript | Go |
+|-------|--------|------|------|-----|------------|-----|
+| **Nomic Embed Code** | **81.7** | **80.5** | 81.8 | **72.3** | 77.1 | **93.8** |
+| Voyage Code 3 | 80.8 | **80.5** | **84.6** | 71.7 | **79.2** | 93.2 |
+| OpenAI Embed 3 Large | 70.8 | 72.9 | 75.3 | 59.6 | 68.1 | 87.6 |
+| Nomic CodeRankEmbed-137M | 78.4 | 76.9 | 79.3 | 68.8 | 71.4 | 92.7 |
+| CodeSage Large v2 (1B) | 74.2 | 72.3 | 76.7 | 65.2 | 72.5 | 84.6 |
+| CodeSage Large (1B) | 70.8 | 70.2 | 71.9 | 61.3 | 69.5 | 83.7 |
+| Qodo Embed 1 7B | 59.9 | 61.6 | 68.4 | 48.5 | 57.0 | 81.4 |
+
+## Model Architecture
+
+- **Total Parameters**: 7B
+- **Training Approach**: Trained on the CoRNStack dataset with dual-consistency filtering and progressive hard negative mining
+- **Supported Languages**: Python, Java, Ruby, PHP, JavaScript, and Go
+
+### CoRNStack Dataset Curation
+
+Starting with the deduplicated Stackv2, we create text-code pairs from function docstrings and respective code. We filtered out low-quality pairs where the docstring wasn't English, too short, or that contained URLs, HTML tags, or invalid characters. We additionally kept docstrings with text lengths of 256 tokens or longer to help the model learn long-range dependencies.
+
+
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/607997c83a565c15675055b3/rb-J54KLgg21f59Ba1jWp.png)
+
+After the initial filtering, we used dual-consistency filtering to remove potentially noisy examples. We embed each docstring and code pair and compute the similarity between each docstring and every code example. We remove pairs from the dataset if the corresponding code example is not found in the top-2 most similar examples for a given docstring.
+
+During training, we employ a novel curriculum-based hard negative mining strategy to ensure the model learns from challenging examples. We use a softmax-based sampling strategy to progressively sample hard negatives with increasing difficulty over time.
+
+
+## Join the Nomic Community
+
+- Nomic Embed Ecosystem: [https://www.nomic.ai/embed](https://www.nomic.ai/embed)
+- Website: [https://nomic.ai](https://nomic.ai)
+- Twitter: [https://twitter.com/nomic\_ai](https://twitter.com/nomic_ai)
+- Discord: [https://discord.gg/myY5YDR8z8](https://discord.gg/myY5YDR8z8)
+
+# Citation
+
+If you find the model, dataset, or training code useful, please cite our work:
+
+```bibtex
+@misc{suresh2025cornstackhighqualitycontrastivedata,
+      title={CoRNStack: High-Quality Contrastive Data for Better Code Retrieval and Reranking},
+      author={Tarun Suresh and Revanth Gangi Reddy and Yifei Xu and Zach Nussbaum and Andriy Mulyar and Brandon Duderstadt and Heng Ji},
+      year={2025},
+      eprint={2412.01007},
+      archivePrefix={arXiv},
+      primaryClass={cs.CL},
+      url={https://arxiv.org/abs/2412.01007},
+}
+```

nomic-embed-code.Q2_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:afaf7fb764e3bbd850629291a52d3955b3376381a4662544b860d71f1e7338a9
+size 2837112128

nomic-embed-code.Q3_K_L.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:848ea553eb9f788c32962ba336435b2ae60f1d449d572760ee42a55ce0141797
+size 3854280000

nomic-embed-code.Q3_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b9dbe007d13d967ed2135120b4c60350dce9d02af01dec29f6c26409c32f62b5
+size 3574211904

nomic-embed-code.Q3_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6d1d69385827e3cd28e44fb14cf56e4a4156aed72a38ea2f5dd96bebebb8b527
+size 3258189120

nomic-embed-code.Q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0999f5a97eb15363c4cad76cfa71c413db96f61e8a32a91166e9f45db30b895e
+size 4124828992

nomic-embed-code.Q4_1.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:859e801467a37922e0e5a3157a6498a9d33cb2b0ee08daed42e6988b1d1f14de
+size 4532659520

nomic-embed-code.Q4_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4354a73ee9ff5d811efe552a515dfd518667ff25fdfc4ee9e10af3f617f96eec
+size 4376511808

nomic-embed-code.Q4_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:be589a33ae8ad094e634adc05a7d186de41da6052a38b85608dc147b39ffab11
+size 4151207232

nomic-embed-code.Q5_K_M.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f234c58a5be4c5e89f71e3b7131150a568b9618d32a34ebb625e9c0f6e0be9fb
+size 5070144832

nomic-embed-code.Q5_K_S.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a75989528b69c854da8d3a9f2f6011c649dc190b7e7b7d81d1356feb395832fa
+size 4940490048

nomic-embed-code.Q6_K.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b179b7e381cb1c7d00a82f5c26858c94e19916257cbd6146c7e21c9240b3dd86
+size 5807129920

nomic-embed-code.Q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80bbd617cc55bad52ceb0f7dfad5d2f5b8080c61015cad7063ffc438b6561832
+size 7519464768

nomic-embed-code.bf16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09340ae55bc36f8d5e8a423084253c6460be99fee91d855d086767f40638abcd
+size 14147857728

nomic-embed-code.f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:33d214c79b97308e82b84f97ebdbba58bc1d8a841ccaaa9053848a06a091b33e
+size 14147857728

nomic-embed-code.f32.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:99694fed45c5fd1badea488f7998bce7fa4c4d7e1e1b9e4f58b3d3e6892040e4
+size 28288429376