multi-gpu setup on vllm

#15
by Mitke15 - opened

Thanks for the cool model!

I can't seem to get this to run on 2 GPUs.

vllm version 0.9.2.

Seems to fail on:

self.NCCL_CHECK(self._funcs"ncclCommInitRank",

Sign up or log in to comment