Nemotron models that have been converted and/or quantized to work well in vLLM
Michael Goin
mgoin
AI & ML interests
LLM inference optimization, compression, quantization, pruning, distillation
Recent Activity
updated
a model
1 day ago
RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-FP8
published
a model
1 day ago
RedHatAI/Mistral-Small-3.2-24B-Instruct-2506-FP8
updated
a model
11 days ago
mgoin/SEMIKONG-70B-W4A16-G128
Organizations
models
97
mgoin/SEMIKONG-70B-W4A16-G128
Updated
•
11
mgoin/llama-4-tiny-random
Text Generation
•
Updated
•
8
mgoin/Qwen1.5-14B-Chat-GPTQ
Text Generation
•
Updated
•
45
mgoin/pixtral-12b
Image-Text-to-Text
•
Updated
•
2.05k
•
1
mgoin/Llama-3.2-1B-Instruct-FP8-ATTN
Updated
•
13
mgoin/Llama-3.2-1B-Instruct-FP8-dynamic-ATTN
Updated
•
12
mgoin/Pixtral-Large-Instruct-2411
Updated
mgoin/Qwen2.5-Coder-32B-Instruct-fp8
Updated
mgoin/nemotron-3-8b-chat-4k-sft-hf
Text Generation
•
Updated
•
48
mgoin/llava-onevision-qwen2-7b-ov-hf-bnb-full-4bit
Image-Text-to-Text
•
Updated
•
24