Issue in using it with VLLM
#1
by
raghavgg
- opened
"RuntimeError: Unsupported FusedMoe scheme: num_bits=8 type='int' symmetric=True group_size=None strategy='channel' block_structure=None dynamic=False actorder=None observer='minmax' observer_kwargs={}, num_bits=8 type='int' symmetric=True group_size=None strategy='token' block_structure=None dynamic=True actorder=None observer=None observer_kwargs={}"