3 7

Maciej Gad PRO

div0-space

https://div0.space

AI & ML interests

Recent Activity

new activity 18 days ago

mlx-community/DeepSeek-R1-0528-4bit:Is there a chance to get the AWQ or DWQ model? Mac Studio (512GB) has a 3-bit honey point.

new activity 18 days ago

mlx-community/Llama-3_3-Nemotron-Super-49B-v1-mlx-4bit:[LM Studio]Error in iterating prediction stream: AttributeError: 'NoneType' object has no attribute 'shape'. Error Data: n/a, Additional Data: n/a

updated a model 18 days ago

LibraxisAI/Llama-3_1-Nemotron-Ultra-253B-v1-MLX-Q5

View all activity

Organizations

Posts 2

Post

254

Hello community!

TL;DR
We are proudly announcing that the SoTA Llama-3_1-Nemotron-Ultra-253B is now available for Apple Silicon 192gb+ machines thanks to mlx>=0.26.1 and newest mlx-lm runtime!
Enjoy!
LibraxisAI/Llama-3_1-Nemotron-Ultra-253B-v1-MLX-Q5

All the details covered in README.md file (As we hope).
You can try to run on LMStudio if you are not comfortable with the native mlx_lm.server runner using our modified .jinja template and custom tokenizer_config.json.

@mlx-community @prince-canuma @bartowski @lmstudio-community

Post

227

Hello Community!
I am experimenting with q5 quantization on mlx==0.26.2 and it is so promising! Our beloved ( CohereLabs/c4ai-command-a-03-2025) the model we use for our medical-nlp-use : 200gb+ bfloat16 vs. ~70gb q5 mlx with perfect quality ( LibraxisAI/c4ai-command-a-03-2025-q5-mlx)

The q5 and mixed quants came up with the release of mlx>=0.26.0. This quant is not yet supported by lmstudio but we didn't check the Ollama compatibility. For inference at this moment we us native mlx_lm.chat or mlx_lm.server, which works perfectly with python mx.generate pipeline. Repository consists of complete safetensors and config files **plus** we put there also our everyday command-generation scripts - convert-to-mlx.sh for mlx_lm.convert with variable parameters customization, and mlx-serve.sh for easy server building. Enjoy

@lmstudio-community @mlx-community
@bartowski

View all Posts