Issue in using it with VLLM

#1
by raghavgg - opened

"RuntimeError: Unsupported FusedMoe scheme: num_bits=8 type='int' symmetric=True group_size=None strategy='channel' block_structure=None dynamic=False actorder=None observer='minmax' observer_kwargs={}, num_bits=8 type='int' symmetric=True group_size=None strategy='token' block_structure=None dynamic=True actorder=None observer=None observer_kwargs={}"

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment