My GGUF-IQ-ARM-Imatrix quants for inflatebot/MN-12B-Mag-Mell-R1.

Mag Mell R1 was tested with Temp 1.25 and MinP 0.2. This was fairly stable up to 10K, but this might be too "hot". If issues with coherency occur, try increasing MinP or decreasing Temperature.

Use the ChatML prompt format.

Downloads last month: 1,916

GGUF

Model size

12.2B params

Architecture

llama

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

View +2 files

Inference Providers NEW

This model is not currently available via any of the supported third-party Inference Providers, and HF Inference API has been turned off for this model.

Model tree for Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix

Base model

inflatebot/MN-12B-Mag-Mell-R1

Quantized

(16)

this model

Collection including Lewdiculous/MN-12B-Mag-Mell-R1-GGUF-IQ-ARM-Imatrix

Quantized Models (GGUF, IQ, Imatrix)

Collection

Various quantizations of models in the GGUF format. Models with a "checkmark" are personal favorites. An "orange arrow" means it's being uploaded. • 93 items • Updated 26 days ago • 51