Files changed (1) hide show
  1. README.md +49 -37
README.md CHANGED
@@ -1,37 +1,49 @@
1
- ---
2
- language:
3
- - en
4
- license: apache-2.0
5
- tags:
6
- - chat
7
- - mlx
8
- base_model: Qwen/Qwen2.5-0.5B-Instruct
9
- license_link: https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE
10
- pipeline_tag: text-generation
11
- ---
12
-
13
- # madroid/Qwen2.5-0.5B-Instruct-mlx-4bit
14
-
15
- The Model [madroid/Qwen2.5-0.5B-Instruct-mlx-4bit](https://huggingface.co/madroid/Qwen2.5-0.5B-Instruct-mlx-4bit) was converted to MLX format from [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) using mlx-lm version **0.18.2**.
16
-
17
- ## Use with mlx
18
-
19
- ```bash
20
- pip install mlx-lm
21
- ```
22
-
23
- ```python
24
- from mlx_lm import load, generate
25
-
26
- model, tokenizer = load("madroid/Qwen2.5-0.5B-Instruct-mlx-4bit")
27
-
28
- prompt="hello"
29
-
30
- if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
31
- messages = [{"role": "user", "content": prompt}]
32
- prompt = tokenizer.apply_chat_template(
33
- messages, tokenize=False, add_generation_prompt=True
34
- )
35
-
36
- response = generate(model, tokenizer, prompt=prompt, verbose=True)
37
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - zho
4
+ - eng
5
+ - fra
6
+ - spa
7
+ - por
8
+ - deu
9
+ - ita
10
+ - rus
11
+ - jpn
12
+ - kor
13
+ - vie
14
+ - tha
15
+ - ara
16
+ license: apache-2.0
17
+ tags:
18
+ - chat
19
+ - mlx
20
+ base_model: Qwen/Qwen2.5-0.5B-Instruct
21
+ license_link: https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct/blob/main/LICENSE
22
+ pipeline_tag: text-generation
23
+ ---
24
+
25
+ # madroid/Qwen2.5-0.5B-Instruct-mlx-4bit
26
+
27
+ The Model [madroid/Qwen2.5-0.5B-Instruct-mlx-4bit](https://huggingface.co/madroid/Qwen2.5-0.5B-Instruct-mlx-4bit) was converted to MLX format from [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) using mlx-lm version **0.18.2**.
28
+
29
+ ## Use with mlx
30
+
31
+ ```bash
32
+ pip install mlx-lm
33
+ ```
34
+
35
+ ```python
36
+ from mlx_lm import load, generate
37
+
38
+ model, tokenizer = load("madroid/Qwen2.5-0.5B-Instruct-mlx-4bit")
39
+
40
+ prompt="hello"
41
+
42
+ if hasattr(tokenizer, "apply_chat_template") and tokenizer.chat_template is not None:
43
+ messages = [{"role": "user", "content": prompt}]
44
+ prompt = tokenizer.apply_chat_template(
45
+ messages, tokenize=False, add_generation_prompt=True
46
+ )
47
+
48
+ response = generate(model, tokenizer, prompt=prompt, verbose=True)
49
+ ```