|
--- |
|
license: apache-2.0 |
|
--- |
|
|
|
A version of https://huggingface.co/xai-org/grok-1 that is compatible with the latest SGLang. |
|
|
|
The tokenizer is from https://huggingface.co/Xenova/grok-1-tokenizer |
|
|
|
``` |
|
# Load real weights |
|
python3 -m sglang.launch_server --model lmzheng/grok-1 --tp 8 --quantization fp8 |
|
|
|
# Load dummy weights |
|
python3 -m sglang.launch_server --model lmzheng/grok-1 --tp 8 --quantization fp8 --load-format dummy |
|
|
|
# Test accuracy on gsm-8k (sanity check), it should be around 0.85 |
|
python3 -m sglang.test.few_shot_gsm8k --num-questions 200 |
|
``` |
