Update README.md

README.md

@@ -5,7 +5,7 @@ datasets:
 - bigcode/the-stack-v2
 - bigcode/starcoderdata
 - bigcode/commitpack
-library_name: transformers
+library_name: exllamav2
 tags:
 - code
 model-index:
@@ -33,7 +33,7 @@ model-index:
     metrics:
     - name: EM
       type: exact_match
-      value: 0.2820
+      value: 0.282
       verified: false
   - task:
       type: text-generation
@@ -73,7 +73,7 @@ model-index:
     metrics:
     - name: EM
       type: exact_match
-      value: 0.2110
+      value: 0.211
       verified: false
   - task:
       type: text-generation
@@ -117,7 +117,7 @@ model-index:
     metrics:
     - name: EM
       type: exact_match
-      value: 0.2910
+      value: 0.291
       verified: false
   - task:
       type: text-generation
@@ -157,7 +157,7 @@ model-index:
     metrics:
     - name: pass@1
       type: pass@1
-      value: 0.2530
+      value: 0.253
       verified: false
   - task:
       type: text-generation
@@ -209,8 +209,24 @@ model-index:
       type: pass@1
       value: 0.2969
       verified: false
+base_model:
+- JetBrains/Mellum-4b-base
 ---
-
+# Mellum-4b-base-exl2
+Original model: [Mellum-4b-base](https://huggingface.co/JetBrains/Mellum-4b-base) by [JetBrains](https://huggingface.co/JetBrains)
+## Quants
+[4bpw h6 (main)](https://huggingface.co/cgus/Mellum-4b-base-exl2/tree/main)  
+[4.5bpw h6](https://huggingface.co/cgus/Mellum-4b-base-exl2/tree/4.5bpw-h6)  
+[5bpw h6](https://huggingface.co/cgus/Mellum-4b-base-exl2/tree/5bpw-h6)  
+[6bpw h6](https://huggingface.co/cgus/Mellum-4b-base-exl2/tree/6bpw-h6)  
+[8bpw h8](https://huggingface.co/cgus/Mellum-4b-base-exl2/tree/8bpw-h8)  
+## Quantization notes
+Made with Exllamav2 0.2.9 dev branch with default dataset. But it should work with older versions too.  
+It can be used with RTX GPUs on Windows or RTX/ROCm on Linux.  
+Usable with TabbyAPI or full version of Text-Generation-WebUI (not portable).  
+It's a text-completion model meant to be used for code autocompletion with dev tools, e.g. with VSCode + Continue extension.  
+On TabbyAPI v1/chat/completions endpoint will be disabled, it's normal as it lacks instruction template and isn't meant to be used in this manner anyway.
+# Original model card
 # Model Description
 Mellum-4b-base is JetBrains' first open-source large language model (LLM) optimized for code-related tasks.