Models

RedHatAI/gemma-2-2b-quantized.w8a16

Text Generation • 2B • Updated Oct 9, 2024 • 21

nm-testing/Mixtral-8x7B-Instruct-v0.1-W4A16-channel-quantized

6B • Updated Oct 9, 2024 • 936

RedHatAI/Phi-3-small-128k-instruct-quantized.w8a16

Text Generation • 3B • Updated Oct 9, 2024 • 18

RedHatAI/gemma-2-9b-it-quantized.w4a16

Text Generation • 3B • Updated Oct 9, 2024 • 858 • 2

RedHatAI/gemma-2-2b-it-quantized.w4a16

Text Generation • 1B • Updated Oct 9, 2024 • 883 • 1

RedHatAI/SmolLM-1.7B-Instruct-quantized.w8a16

Text Generation • 0.6B • Updated Oct 9, 2024 • 35

RedHatAI/Mistral-Nemo-Instruct-2407-quantized.w4a16

Text Generation • 3B • Updated Oct 9, 2024 • 80 • 4

RedHatAI/gemma-2-2b-it-quantized.w8a8

Text Generation • 3B • Updated Oct 9, 2024 • 67

RedHatAI/gemma-2-9b-it-quantized.w8a8

Text Generation • 10B • Updated Oct 9, 2024 • 16 • 2

RedHatAI/Meta-Llama-3.1-405B-Instruct-quantized.w8a8

Text Generation • 406B • Updated Dec 3, 2024 • 43 • 1

RedHatAI/Meta-Llama-3.1-405B-Instruct-quantized.w8a16

Text Generation • 105B • Updated Oct 9, 2024 • 13 • 1

nm-testing/Meta-Llama-3-8B-Instruct-fp8-hf_compat

8B • Updated Aug 30, 2024 • 20

RedHatAI/gemma-2-27b-it-quantized.w8a16

Text Generation • 9B • Updated Oct 9, 2024 • 10

RedHatAI/SmolLM-135M-Instruct-quantized.w8a16

Text Generation • 0.1B • Updated Oct 9, 2024 • 206

RedHatAI/SmolLM-360M-Instruct-quantized.w8a8

Text Generation • 0.4B • Updated Oct 9, 2024 • 10

RedHatAI/SmolLM-135M-Instruct-quantized.w8a8

Text Generation • 0.2B • Updated Oct 9, 2024 • 11 • 1

nm-testing/SmolLM-135M-Instruct-quantized.w4a16

Text Generation • 0.1B • Updated Oct 9, 2024 • 2

nm-testing/SmolLM-360M-Instruct-quantized.w4a16

0.1B • Updated Oct 9, 2024 • 2

nm-testing/SmolLM-1.7B-Instruct-quantized.w4a16

Text Generation • 0.4B • Updated Oct 9, 2024 • 2

RedHatAI/SmolLM-1.7B-Instruct-quantized.w8a8

Text Generation • 2B • Updated Oct 9, 2024 • 21 • 1

nm-testing/tinyllama-fp8-dynamic-compressed

1B • Updated Oct 9, 2024 • 1.4k

nm-testing/tinyllama-w4a16-compressed

0.3B • Updated Oct 9, 2024 • 4

nm-testing/tinyllama-w8a8-compressed

1B • Updated Oct 9, 2024 • 379

nm-testing/tinyllama-w8a16-dense

1B • Updated Oct 9, 2024 • 4

nm-testing/TinyLlama-1.1B-Chat-v1.0-actorder-group

Text Generation • 0.3B • Updated Oct 9, 2024 • 1.13k

nm-testing/TinyLlama-1.1B-Chat-v1.0-actorder-weight

Text Generation • 0.3B • Updated Oct 9, 2024 • 2

nm-testing/tinyllama-fp8-dynamic-compressed-hf-quantizer

1B • Updated Sep 11, 2024 • 2

nm-testing/tinyllama-w4a16-compressed-hf-quantizer

0.3B • Updated Sep 11, 2024 • 539

nm-testing/tinyllama-w8a8-compressed-hf-quantizer

1B • Updated Sep 11, 2024 • 500

nm-testing/tinyllama-w8a16-dense-hf-quantizer

1B • Updated Sep 11, 2024 • 19