Apple Silicon support ?
#5
by
Novell
- opened
Hello, first of all this release is amazing! The architecture and the fact it supports streaming in multiple modalities is impressive.
I would like to know if there is any plan to make this model compatible with MLX/MacOS backends ? Flash Attn 2 isn't supported on Mac and the inference speed is rather slow when running the model on mac without flash.
?