Create README.md

README.md

@@ -0,0 +1,62 @@
+---
+base_model: onekq-ai/OneSQL-v0.1-Qwen-3B
+tags:
+- text-generation-inference
+- transformers
+- qwen2
+- gguf
+license: apache-2.0
+language:
+- en
+---
+
+# Introduction
+
+This model is the GGUF version of [OneSQL-v0.1-Qwen-3B](https://huggingface.co/onekq-ai/OneSQL-v0.1-Qwen-3B). You can also find it on [Ollama](https://ollama.com/onekq/OneSQL-v0.1-Qwen).
+
+# Performances
+
+The self-evaluation EX score of the original model is **43.35** (compared to **63.33** by the 32B model on the [BIRD leaderboard](https://bird-bench.github.io/).
+Below is the self-evaluation results for each quantization.
+
+| Quantization |EX score|
+|------------|------|
+| Q4_0       | 16.83 |
+| Q4_1       | 21.85 |
+| Q4_K_S     | 22.49 |
+| Q4_K_M     | 21.85 |
+| Q5_0       | 23.40 |
+| Q5_1       | 23.53 |
+| Q5_K_S     | 22.77 |
+| Q5_K_M     | 23.73 |
+| Q6_K       | 24.51 |
+| **Q8_0**       | **24.90** |
+
+# Quick start
+
+To use this model, craft your prompt to start with your database schema in the form of **CREATE TABLE**, followed by your natural language query preceded by **--**.
+Make sure your prompt ends with **SELECT** in order for the model to finish the query for you. There is no need to set other parameters like temperature or max token limit.
+
+```sh
+PROMPT="CREATE TABLE students (
+    id INTEGER PRIMARY KEY,
+    name TEXT,
+    age INTEGER,
+    grade TEXT
+);
+
+-- Find the three youngest students
+SELECT "
+
+ollama run onekq-ai/OneSQL-v0.1-Qwen:3B-Q8_0 "$PROMPT"
+```
+
+The model response is the finished SQL query without **SELECT**
+```sql
+* FROM students ORDER BY age ASC LIMIT 3
+```
+
+# Caveats
+
+* The performance drop from the original model is due to quantization itself, and the lack of beam search support in llama.cpp framework. Use at your own discretion.
+* The 2-bit and 3-bit quantizations suffer from repetitive and unrelevant output token, hence are not recommended for usage.