multi-gpu setup on vllm
#15
by
Mitke15
- opened
Thanks for the cool model!
I can't seem to get this to run on 2 GPUs.
vllm version 0.9.2.
Seems to fail on:
self.NCCL_CHECK(self._funcs"ncclCommInitRank",
Thanks for the cool model!
I can't seem to get this to run on 2 GPUs.
vllm version 0.9.2.
Seems to fail on:
self.NCCL_CHECK(self._funcs"ncclCommInitRank",