Hugging Face's logo
Michael Goin's picture
75 15 20

Michael Goin PRO

mgoin
muhammadzeeshan007's profile picture nicoboss's profile picture 21world's profile picture
·

AI & ML interests

LLM inference optimization, compression, quantization, pruning, distillation

Recent Activity

published a model 2 days ago
neuralmagic/Qwen2.5-3B-quantized.w8a8
published a model 2 days ago
neuralmagic/Qwen2.5-14B-quantized.w8a8
published a model 2 days ago
neuralmagic/Qwen2.5-14B-FP8-dynamic
View all activity

Organizations

Neural Magic's profile picture garage-bAInd's profile picture Blog-explorers's profile picture Revel Labs's profile picture ZeroGPU Explorers's profile picture NM Testing's profile picture MLX Community's profile picture Social Post Explorers's profile picture Red Hat's profile picture

mgoin's activity

New activity in neuralmagic/Meta-Llama-3-8B-Instruct-FP8-KV 3 days ago

How to load this model?

2
#1 opened 7 months ago by
Frz614
New activity in neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic about 1 month ago

Model does not run with VLLM

2
#3 opened about 1 month ago by
aswad546
New activity in nm-testing/Llama-3.3-70B-Instruct-FP8-dynamic about 2 months ago

Nice model, any info on scripts used to quantize?

1
#1 opened about 2 months ago by
RonanMcGovern
New activity in neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic about 2 months ago

Thanks!
#2 opened about 2 months ago by
Jindows
New activity in mistralai/Pixtral-Large-Instruct-2411 2 months ago

Add config_format and load_format to vLLM args
#5 opened 2 months ago by
mgoin

Update config.json to use null for sliding_window
#4 opened 2 months ago by
mgoin
New activity in mgoin/nemotron-3-8b-chat-4k-sft-hf 3 months ago

Adding `safetensors` variant of this model
#1 opened 3 months ago by
SFconvertbot
New activity in neuralmagic/Meta-Llama-3.1-70B-Instruct-quantized.w8a16 3 months ago

Is this the standard GPTQ quantization?

1
#5 opened 3 months ago by
molereddy
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w8a16 3 months ago

Model weights are not loaded

4
#3 opened 5 months ago by
MarvelousMouse
New activity in neuralmagic/pixtral-12b-FP8-dynamic 3 months ago

Update model card
#1 opened 3 months ago by
nm-research
New activity in nm-testing/llava-1.5-7b-hf-FP8-dynamic 3 months ago

Add chat_template to tokenizer_config.json
#1 opened 3 months ago by
nm-research
New activity in neuralmagic/Mistral-Nemo-Instruct-2407-FP8 3 months ago

7900xtx torch._scaled_mm is only supported on CUDA devices with compute capability >= 9.0 or 8.9, or ROCm MI300+

1
#3 opened 3 months ago by
aaaaaaaaaasdf
New activity in mistral-community/pixtral-12b 3 months ago

Why is the Pixtral activation function "gelu" when the reference code uses "silu"?

2
#10 opened 4 months ago by
mgoin

Update tokenizer_config.json with chat_template

3
#11 opened 4 months ago by
mgoin
New activity in neuralmagic/Llama-3.2-90B-Vision-Instruct-FP8-dynamic 4 months ago

Any chance your team is working on a 4-bit Llama-3.2-90B-Vision-Instruct-quantized.w4a16 version?

1
#1 opened 4 months ago by
mrhendrey
New activity in neuralmagic/Llama-3.2-11B-Vision-Instruct-FP8-dynamic 4 months ago

Oom with 24g vram

3
#1 opened 4 months ago by
Klopez
New activity in neuralmagic/Phi-3.5-mini-instruct-FP8-KV 4 months ago

latest vllm docker (v0.6.2) fail to load

2
#1 opened 4 months ago by
choronz333
New activity in neuralmagic/Meta-Llama-3.1-8B-Instruct-quantized.w4a16 5 months ago

Issue with loading model

1
#1 opened 5 months ago by
xSumukhax
New activity in neuralmagic/DeepSeek-Coder-V2-Instruct-FP8 5 months ago

Can it run on A100/A800 with VLLM?

3
#1 opened 6 months ago by
Parkerlambert123
New activity in neuralmagic/Meta-Llama-3.1-405B-Instruct-quantized.w4a16 6 months ago

weights does not exist when trying to deploy in sagemaker endpoint

1
#1 opened 6 months ago by
LorenzoCevolaniAXA