Files changed (1) hide show
  1. README.md +82 -70
README.md CHANGED
@@ -1,71 +1,83 @@
1
- ---
2
- base_model: Qwen/Qwen2.5-7B-Instruct
3
- language:
4
- - en
5
- library_name: transformers
6
- license: apache-2.0
7
- license_link: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct/blob/main/LICENSE
8
- pipeline_tag: text-generation
9
- tags:
10
- - chat
11
- - mlc-ai
12
- - MLC-Weight-Conversion
13
- ---
14
- ---
15
- library_name: mlc-llm
16
- base_model: Qwen/Qwen2.5-7B-Instruct
17
- tags:
18
- - mlc-llm
19
- - web-llm
20
- ---
21
-
22
- # AMKCode/Qwen2.5-7B-Instruct-q4f16_1-MLC
23
-
24
- This is the [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) model in MLC format `q4f16_1`.
25
- The conversion was done using the [MLC-Weight-Conversion](https://huggingface.co/spaces/mlc-ai/MLC-Weight-Conversion) space.
26
- The model can be used for projects [MLC-LLM](https://github.com/mlc-ai/mlc-llm) and [WebLLM](https://github.com/mlc-ai/web-llm).
27
-
28
- ## Example Usage
29
-
30
- Here are some examples of using this model in MLC LLM.
31
- Before running the examples, please install MLC LLM by following the [installation documentation](https://llm.mlc.ai/docs/install/mlc_llm.html#install-mlc-packages).
32
-
33
- ### Chat
34
-
35
- In command line, run
36
- ```bash
37
- mlc_llm chat HF://mlc-ai/AMKCode/Qwen2.5-7B-Instruct-q4f16_1-MLC
38
- ```
39
-
40
- ### REST Server
41
-
42
- In command line, run
43
- ```bash
44
- mlc_llm serve HF://mlc-ai/AMKCode/Qwen2.5-7B-Instruct-q4f16_1-MLC
45
- ```
46
-
47
- ### Python API
48
-
49
- ```python
50
- from mlc_llm import MLCEngine
51
-
52
- # Create engine
53
- model = "HF://mlc-ai/AMKCode/Qwen2.5-7B-Instruct-q4f16_1-MLC"
54
- engine = MLCEngine(model)
55
-
56
- # Run chat completion in OpenAI API.
57
- for response in engine.chat.completions.create(
58
- messages=[{"role": "user", "content": "What is the meaning of life?"}],
59
- model=model,
60
- stream=True,
61
- ):
62
- for choice in response.choices:
63
- print(choice.delta.content, end="", flush=True)
64
- print("\n")
65
-
66
- engine.terminate()
67
- ```
68
-
69
- ## Documentation
70
-
 
 
 
 
 
 
 
 
 
 
 
 
71
  For more information on MLC LLM project, please visit our [documentation](https://llm.mlc.ai/docs/) and [GitHub repo](http://github.com/mlc-ai/mlc-llm).
 
1
+ ---
2
+ base_model: Qwen/Qwen2.5-7B-Instruct
3
+ language:
4
+ - zho
5
+ - eng
6
+ - fra
7
+ - spa
8
+ - por
9
+ - deu
10
+ - ita
11
+ - rus
12
+ - jpn
13
+ - kor
14
+ - vie
15
+ - tha
16
+ - ara
17
+ library_name: transformers
18
+ license: apache-2.0
19
+ license_link: https://huggingface.co/Qwen/Qwen2.5-7B-Instruct/blob/main/LICENSE
20
+ pipeline_tag: text-generation
21
+ tags:
22
+ - chat
23
+ - mlc-ai
24
+ - MLC-Weight-Conversion
25
+ ---
26
+ ---
27
+ library_name: mlc-llm
28
+ base_model: Qwen/Qwen2.5-7B-Instruct
29
+ tags:
30
+ - mlc-llm
31
+ - web-llm
32
+ ---
33
+
34
+ # AMKCode/Qwen2.5-7B-Instruct-q4f16_1-MLC
35
+
36
+ This is the [Qwen2.5-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct) model in MLC format `q4f16_1`.
37
+ The conversion was done using the [MLC-Weight-Conversion](https://huggingface.co/spaces/mlc-ai/MLC-Weight-Conversion) space.
38
+ The model can be used for projects [MLC-LLM](https://github.com/mlc-ai/mlc-llm) and [WebLLM](https://github.com/mlc-ai/web-llm).
39
+
40
+ ## Example Usage
41
+
42
+ Here are some examples of using this model in MLC LLM.
43
+ Before running the examples, please install MLC LLM by following the [installation documentation](https://llm.mlc.ai/docs/install/mlc_llm.html#install-mlc-packages).
44
+
45
+ ### Chat
46
+
47
+ In command line, run
48
+ ```bash
49
+ mlc_llm chat HF://mlc-ai/AMKCode/Qwen2.5-7B-Instruct-q4f16_1-MLC
50
+ ```
51
+
52
+ ### REST Server
53
+
54
+ In command line, run
55
+ ```bash
56
+ mlc_llm serve HF://mlc-ai/AMKCode/Qwen2.5-7B-Instruct-q4f16_1-MLC
57
+ ```
58
+
59
+ ### Python API
60
+
61
+ ```python
62
+ from mlc_llm import MLCEngine
63
+
64
+ # Create engine
65
+ model = "HF://mlc-ai/AMKCode/Qwen2.5-7B-Instruct-q4f16_1-MLC"
66
+ engine = MLCEngine(model)
67
+
68
+ # Run chat completion in OpenAI API.
69
+ for response in engine.chat.completions.create(
70
+ messages=[{"role": "user", "content": "What is the meaning of life?"}],
71
+ model=model,
72
+ stream=True,
73
+ ):
74
+ for choice in response.choices:
75
+ print(choice.delta.content, end="", flush=True)
76
+ print("\n")
77
+
78
+ engine.terminate()
79
+ ```
80
+
81
+ ## Documentation
82
+
83
  For more information on MLC LLM project, please visit our [documentation](https://llm.mlc.ai/docs/) and [GitHub repo](http://github.com/mlc-ai/mlc-llm).