Update README.md (#12)
Browse files- Update README.md (f9afe38eb6d83e0bc54ca24ff122a3803daf031f)
README.md
CHANGED
|
@@ -20,7 +20,7 @@ inference: false
|
|
| 20 |
model_creator: MaziyarPanahi
|
| 21 |
quantized_by: MaziyarPanahi
|
| 22 |
model-index:
|
| 23 |
-
- name:
|
| 24 |
results:
|
| 25 |
- task:
|
| 26 |
type: text-generation
|
|
@@ -37,7 +37,7 @@ model-index:
|
|
| 37 |
value: 63.48
|
| 38 |
name: normalized accuracy
|
| 39 |
source:
|
| 40 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
| 41 |
name: Open LLM Leaderboard
|
| 42 |
- task:
|
| 43 |
type: text-generation
|
|
@@ -53,7 +53,7 @@ model-index:
|
|
| 53 |
value: 80.86
|
| 54 |
name: normalized accuracy
|
| 55 |
source:
|
| 56 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
| 57 |
name: Open LLM Leaderboard
|
| 58 |
- task:
|
| 59 |
type: text-generation
|
|
@@ -70,7 +70,7 @@ model-index:
|
|
| 70 |
value: 69.24
|
| 71 |
name: accuracy
|
| 72 |
source:
|
| 73 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
| 74 |
name: Open LLM Leaderboard
|
| 75 |
- task:
|
| 76 |
type: text-generation
|
|
@@ -86,7 +86,7 @@ model-index:
|
|
| 86 |
- type: mc2
|
| 87 |
value: 60.66
|
| 88 |
source:
|
| 89 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
| 90 |
name: Open LLM Leaderboard
|
| 91 |
- task:
|
| 92 |
type: text-generation
|
|
@@ -103,7 +103,7 @@ model-index:
|
|
| 103 |
value: 72.77
|
| 104 |
name: accuracy
|
| 105 |
source:
|
| 106 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
| 107 |
name: Open LLM Leaderboard
|
| 108 |
- task:
|
| 109 |
type: text-generation
|
|
@@ -120,7 +120,7 @@ model-index:
|
|
| 120 |
value: 74.53
|
| 121 |
name: accuracy
|
| 122 |
source:
|
| 123 |
-
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/
|
| 124 |
name: Open LLM Leaderboard
|
| 125 |
- task:
|
| 126 |
type: text-generation
|
|
@@ -135,7 +135,7 @@ model-index:
|
|
| 135 |
value: 49.26
|
| 136 |
name: strict accuracy
|
| 137 |
source:
|
| 138 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
| 139 |
name: Open LLM Leaderboard
|
| 140 |
- task:
|
| 141 |
type: text-generation
|
|
@@ -150,7 +150,7 @@ model-index:
|
|
| 150 |
value: 37.66
|
| 151 |
name: normalized accuracy
|
| 152 |
source:
|
| 153 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
| 154 |
name: Open LLM Leaderboard
|
| 155 |
- task:
|
| 156 |
type: text-generation
|
|
@@ -165,7 +165,7 @@ model-index:
|
|
| 165 |
value: 2.95
|
| 166 |
name: exact match
|
| 167 |
source:
|
| 168 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
| 169 |
name: Open LLM Leaderboard
|
| 170 |
- task:
|
| 171 |
type: text-generation
|
|
@@ -180,7 +180,7 @@ model-index:
|
|
| 180 |
value: 9.06
|
| 181 |
name: acc_norm
|
| 182 |
source:
|
| 183 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
| 184 |
name: Open LLM Leaderboard
|
| 185 |
- task:
|
| 186 |
type: text-generation
|
|
@@ -195,7 +195,7 @@ model-index:
|
|
| 195 |
value: 7.75
|
| 196 |
name: acc_norm
|
| 197 |
source:
|
| 198 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
| 199 |
name: Open LLM Leaderboard
|
| 200 |
- task:
|
| 201 |
type: text-generation
|
|
@@ -212,23 +212,23 @@ model-index:
|
|
| 212 |
value: 31.42
|
| 213 |
name: accuracy
|
| 214 |
source:
|
| 215 |
-
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/
|
| 216 |
name: Open LLM Leaderboard
|
| 217 |
---
|
| 218 |
|
| 219 |
<img src="./phi-3-instruct.webp" alt="Phi-3 Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
| 220 |
|
| 221 |
|
| 222 |
-
# MaziyarPanahi/
|
| 223 |
|
| 224 |
This model is a fine-tune (DPO) of `microsoft/Phi-3-mini-4k-instruct` model.
|
| 225 |
|
| 226 |
# ⚡ Quantized GGUF
|
| 227 |
|
| 228 |
-
All GGUF models are available here: [MaziyarPanahi/
|
| 229 |
|
| 230 |
# 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
| 231 |
-
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/
|
| 232 |
|
| 233 |
|
| 234 |
|
|
@@ -259,7 +259,7 @@ Detailed results can be found [here](https://huggingface.co/datasets/open-llm-le
|
|
| 259 |
|
| 260 |
|
| 261 |
|
| 262 |
-
`MaziyarPanahi/
|
| 263 |
|
| 264 |

|
| 265 |
|
|
@@ -280,7 +280,7 @@ This model uses `ChatML` prompt template:
|
|
| 280 |
|
| 281 |
# How to use
|
| 282 |
|
| 283 |
-
You can use this model by using `MaziyarPanahi/
|
| 284 |
transformers library.
|
| 285 |
|
| 286 |
```python
|
|
@@ -288,7 +288,7 @@ from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
|
|
| 288 |
from transformers import pipeline
|
| 289 |
import torch
|
| 290 |
|
| 291 |
-
model_id = "MaziyarPanahi/
|
| 292 |
|
| 293 |
model = AutoModelForCausalLM.from_pretrained(
|
| 294 |
model_id,
|
|
|
|
| 20 |
model_creator: MaziyarPanahi
|
| 21 |
quantized_by: MaziyarPanahi
|
| 22 |
model-index:
|
| 23 |
+
- name: calme-2.3-phi3-4b
|
| 24 |
results:
|
| 25 |
- task:
|
| 26 |
type: text-generation
|
|
|
|
| 37 |
value: 63.48
|
| 38 |
name: normalized accuracy
|
| 39 |
source:
|
| 40 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 41 |
name: Open LLM Leaderboard
|
| 42 |
- task:
|
| 43 |
type: text-generation
|
|
|
|
| 53 |
value: 80.86
|
| 54 |
name: normalized accuracy
|
| 55 |
source:
|
| 56 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 57 |
name: Open LLM Leaderboard
|
| 58 |
- task:
|
| 59 |
type: text-generation
|
|
|
|
| 70 |
value: 69.24
|
| 71 |
name: accuracy
|
| 72 |
source:
|
| 73 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 74 |
name: Open LLM Leaderboard
|
| 75 |
- task:
|
| 76 |
type: text-generation
|
|
|
|
| 86 |
- type: mc2
|
| 87 |
value: 60.66
|
| 88 |
source:
|
| 89 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 90 |
name: Open LLM Leaderboard
|
| 91 |
- task:
|
| 92 |
type: text-generation
|
|
|
|
| 103 |
value: 72.77
|
| 104 |
name: accuracy
|
| 105 |
source:
|
| 106 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 107 |
name: Open LLM Leaderboard
|
| 108 |
- task:
|
| 109 |
type: text-generation
|
|
|
|
| 120 |
value: 74.53
|
| 121 |
name: accuracy
|
| 122 |
source:
|
| 123 |
+
url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 124 |
name: Open LLM Leaderboard
|
| 125 |
- task:
|
| 126 |
type: text-generation
|
|
|
|
| 135 |
value: 49.26
|
| 136 |
name: strict accuracy
|
| 137 |
source:
|
| 138 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 139 |
name: Open LLM Leaderboard
|
| 140 |
- task:
|
| 141 |
type: text-generation
|
|
|
|
| 150 |
value: 37.66
|
| 151 |
name: normalized accuracy
|
| 152 |
source:
|
| 153 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 154 |
name: Open LLM Leaderboard
|
| 155 |
- task:
|
| 156 |
type: text-generation
|
|
|
|
| 165 |
value: 2.95
|
| 166 |
name: exact match
|
| 167 |
source:
|
| 168 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 169 |
name: Open LLM Leaderboard
|
| 170 |
- task:
|
| 171 |
type: text-generation
|
|
|
|
| 180 |
value: 9.06
|
| 181 |
name: acc_norm
|
| 182 |
source:
|
| 183 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 184 |
name: Open LLM Leaderboard
|
| 185 |
- task:
|
| 186 |
type: text-generation
|
|
|
|
| 195 |
value: 7.75
|
| 196 |
name: acc_norm
|
| 197 |
source:
|
| 198 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 199 |
name: Open LLM Leaderboard
|
| 200 |
- task:
|
| 201 |
type: text-generation
|
|
|
|
| 212 |
value: 31.42
|
| 213 |
name: accuracy
|
| 214 |
source:
|
| 215 |
+
url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=MaziyarPanahi/calme-2.3-phi3-4b
|
| 216 |
name: Open LLM Leaderboard
|
| 217 |
---
|
| 218 |
|
| 219 |
<img src="./phi-3-instruct.webp" alt="Phi-3 Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
|
| 220 |
|
| 221 |
|
| 222 |
+
# MaziyarPanahi/calme-2.3-phi3-4b
|
| 223 |
|
| 224 |
This model is a fine-tune (DPO) of `microsoft/Phi-3-mini-4k-instruct` model.
|
| 225 |
|
| 226 |
# ⚡ Quantized GGUF
|
| 227 |
|
| 228 |
+
All GGUF models are available here: [MaziyarPanahi/calme-2.3-phi3-4b-GGUF](https://huggingface.co/MaziyarPanahi/calme-2.3-phi3-4b-GGUF)
|
| 229 |
|
| 230 |
# 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
|
| 231 |
+
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_MaziyarPanahi__calme-2.3-phi3-4b)
|
| 232 |
|
| 233 |
|
| 234 |
|
|
|
|
| 259 |
|
| 260 |
|
| 261 |
|
| 262 |
+
`MaziyarPanahi/calme-2.3-phi3-4b` is the best-performing Phi-3-mini-4k model on the Open LLM Leaderboard. (03/06/2024).
|
| 263 |
|
| 264 |

|
| 265 |
|
|
|
|
| 280 |
|
| 281 |
# How to use
|
| 282 |
|
| 283 |
+
You can use this model by using `MaziyarPanahi/calme-2.3-phi3-4b` as the model name in Hugging Face's
|
| 284 |
transformers library.
|
| 285 |
|
| 286 |
```python
|
|
|
|
| 288 |
from transformers import pipeline
|
| 289 |
import torch
|
| 290 |
|
| 291 |
+
model_id = "MaziyarPanahi/calme-2.3-phi3-4b"
|
| 292 |
|
| 293 |
model = AutoModelForCausalLM.from_pretrained(
|
| 294 |
model_id,
|