Update README.md
Browse files
README.md
CHANGED
@@ -46,7 +46,7 @@ pip install -U vllm \
|
|
46 |
```
|
47 |
|
48 |
### vllm
|
49 |
-
Please load the model into vllm and sglang as float16 data type for AWQ support and use `tensor_parallel_size <= 2` i.e
|
50 |
```
|
51 |
vllm serve cpatonn/GLM-4.5-Air-AWQ --dtype float16 --tensor-parallel-size 2 --pipeline-parallel-size 2
|
52 |
```
|
|
|
46 |
```
|
47 |
|
48 |
### vllm
|
49 |
+
Please load the model into vllm and sglang as float16 data type for AWQ support and use `tensor_parallel_size <= 2` i.e.,
|
50 |
```
|
51 |
vllm serve cpatonn/GLM-4.5-Air-AWQ --dtype float16 --tensor-parallel-size 2 --pipeline-parallel-size 2
|
52 |
```
|