---
license: apache-2.0
language:
- pl
base_model:
- CYFRAGOVPL/PLLuM-8x7B-chat
---
# PLLuM-8x7B-chat GGUF Quantizations by Nondzu

DISCLAIMER: This is a quantized version of an existing model [PLLuM-8x7B-chat](https://huggingface.co/CYFRAGOVPL/PLLuM-8x7B-chat). I am not the author of the original model. I am only hosting the quantized models. I do not take any responsibility for the models.

## Prompt Format

Use the following prompt structure:
```
???
```

## Available Files

Below is a list of available quantized model files along with their quantization type, file size, and a short description.

| Filename                                                                              | Quant Type | File Size | Description                                                                                   |
| ------------------------------------------------------------------------------------- | ---------- | --------- | --------------------------------------------------------------------------------------------- |
| [PLLuM-8x7B-chat-Q2_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)    | Q2_K       | 17 GB     | Very low quality but surprisingly usable.                                                   |
| [PLLuM-8x7B-chat-Q3_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)    | Q3_K       | 21 GB     | Low quality, suitable for setups with very limited RAM.                                       |
| [PLLuM-8x7B-chat-Q3_K_L.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)  | Q3_K_L     | 23 GB     | High quality; recommended for quality-focused usage.                                          |
| [PLLuM-8x7B-chat-Q3_K_M.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)  | Q3_K_M     | 21 GB     | Very high quality, near perfect output – recommended.                                         |
| [PLLuM-8x7B-chat-Q3_K_S.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)  | Q3_K_S     | 20 GB     | Moderate quality with improved space efficiency.                                              |
| [PLLuM-8x7B-chat-Q4_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)    | Q4_K       | 27 GB     | Good quality for standard use.                                                                |
| [PLLuM-8x7B-chat-Q4_K_M.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)  | Q4_K_M     | 27 GB     | Default quality for most use cases – recommended.                                             |
| [PLLuM-8x7B-chat-Q4_K_S.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)  | Q4_K_S     | 25 GB     | Slightly lower quality with enhanced space savings – recommended when size is a priority.       |
| [PLLuM-8x7B-chat-Q5_0.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)    | Q5_0       | 31 GB     | Extremely high quality – the maximum quant available.                                         |
| [PLLuM-8x7B-chat-Q5_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)    | Q5_K       | 31 GB     | Very high quality – recommended for demanding use cases.                                      |
| [PLLuM-8x7B-chat-Q5_K_M.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)  | Q5_K_M     | 31 GB     | High quality – recommended.                                                                   |
| [PLLuM-8x7B-chat-Q5_K_S.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)  | Q5_K_S     | 31 GB     | High quality, offered as an alternative with minimal quality loss.                            |
| [PLLuM-8x7B-chat-Q6_K.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)    | Q6_K       | 36 GB     | Very high quality with quantized embed/output weights.                                        |
| [PLLuM-8x7B-chat-Q8_0.gguf](https://huggingface.co/Nondzu/PLLuM-8x7B-chat-GGUF/tree/main)    | Q8_0       | 47 GB     | Maximum quality quantization.                                                                 |

## Downloading Using Hugging Face CLI

<details>
  <summary>Click to view download instructions</summary>

First, ensure you have the Hugging Face CLI installed:

```bash
pip install -U "huggingface_hub[cli]"
```

Then, target a specific file to download:

```bash
huggingface-cli download Nondzu/PLLuM-8x7B-chat-GGUF --include "PLLuM-8x7B-chat-Q4_K_M.gguf" --local-dir ./
```

For larger files, you can specify a new local directory (e.g., `PLLuM-8x7B-chat-Q8_0`) or download them directly into the current directory (`./`).

</details>