Are there any updates to the recommended commands?
#27
by
NaiveYan
- opened
I tested the command in the current README with vLLM v0.8.0 (on 8 x A800 GPUs), but it only returns garbled text.
Are there any updates to the recommended commands, or are there other inference engines you would suggest?
Merge these three PRs, then build it yourself, then it should work.