|
--- |
|
quantized_by: Doctor-Shotgun |
|
license: apache-2.0 |
|
language: |
|
- en |
|
tags: |
|
- music |
|
- llama |
|
base_model: m-a-p/YuE-s1-7B-anneal-en-cot |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
# YuE-s1-7B-anneal-en-cot-exl2 |
|
|
|
[m-a-p/YuE-s1-7B-anneal-en-cot](https://huggingface.co/m-a-p/YuE-s1-7B-anneal-en-cot) quantized with Exllamav2. It appears to remain coherent using the default calibration data without adding audio tokens. |
|
|
|
Intended to be used with the [WIP exl2 inference repository for YuE](https://github.com/sgsdxzy/YuE-exllamav2). |
|
|
|
Links: |
|
- [4.25bpw-h6](https://huggingface.co/Doctor-Shotgun/YuE-s1-7B-anneal-en-cot-exl2/tree/4.25bpw-h6) |
|
- [5.0bpw-h6](https://huggingface.co/Doctor-Shotgun/YuE-s1-7B-anneal-en-cot-exl2/tree/5.0bpw-h6) |
|
- [6.0bpw-h6](https://huggingface.co/Doctor-Shotgun/YuE-s1-7B-anneal-en-cot-exl2/tree/6.0bpw-h6) |
|
- [8.0bpw-h8](https://huggingface.co/Doctor-Shotgun/YuE-s1-7B-anneal-en-cot-exl2/tree/8.0bpw-h8) |
