Update README.md (#11)
Browse files- Update README.md (ea591333b7dcf7210a3e3481e1f2883c81f12c08)
Co-authored-by: Yuchen Xie <[email protected]>
README.md
CHANGED
@@ -290,7 +290,7 @@ YaRN is currently supported by several inference frameworks, e.g., `transformers
|
|
290 |
|
291 |
For `vllm`, you can use
|
292 |
```shell
|
293 |
-
vllm serve ... --rope-scaling '{"
|
294 |
```
|
295 |
|
296 |
For `sglang`, you can use
|
|
|
290 |
|
291 |
For `vllm`, you can use
|
292 |
```shell
|
293 |
+
vllm serve ... --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072
|
294 |
```
|
295 |
|
296 |
For `sglang`, you can use
|