jefowers commited on
Commit
dcc0e9c
·
verified ·
1 Parent(s): 236a935

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +32 -0
README.md ADDED
@@ -0,0 +1,32 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: Qwen/Qwen2.5-Coder-7B-Instruct
4
+ tags:
5
+ - gguf
6
+ - quantized
7
+ - q4_k_m
8
+ ---
9
+
10
+ # Qwen2.5-Coder-7B-Instruct-iat-05-1-GGUF
11
+
12
+ This is a GGUF quantized version (q4_k_m) of Qwen/Qwen2.5-Coder-7B-Instruct fine-tuned with the 'iat-05-1' adapter.
13
+
14
+ ## Model Details
15
+
16
+ - **Base Model:** Qwen/Qwen2.5-Coder-7B-Instruct
17
+ - **Adapter:** iat-05-1
18
+ - **Quantization:** q4_k_m
19
+ - **Format:** GGUF
20
+
21
+ ## Usage
22
+
23
+ This model can be used with llama.cpp or any compatible inference engine that supports GGUF format.
24
+
25
+ ```bash
26
+ # Example with llama.cpp
27
+ ./llama-cli -m Qwen2.5-Coder-7B-Instruct-iat-05-1-q4_k_m.gguf -p "Your prompt here"
28
+ ```
29
+
30
+ ## Files
31
+
32
+ - `Qwen2.5-Coder-7B-Instruct-iat-05-1-q4_k_m.gguf` - Quantized model in GGUF format (q4_k_m)