Semantic Relationship: Q4 Quantized model is broken Moving towards AWQ Q4_K_M model
#1
by
TheFireHacker
- opened
Due to aggrressive quantization Attention layers are broken , moving towards better quant model.
Workflow is still
- Rank One and Static Analysis update of base model
- Quantization