Qwen2.5-Coder-7B-Instruct-iat-05-1-GGUF

This is a GGUF quantized version (q4_k_m) of Qwen/Qwen2.5-Coder-7B-Instruct fine-tuned with the 'iat-05-1' adapter.

Model Details

• Base Model: Qwen/Qwen2.5-Coder-7B-Instruct
• Adapter: iat-05-1
• Quantization: q4_k_m
• Format: GGUF

Usage

This model can be used with llama.cpp or any compatible inference engine that supports GGUF format.

# Example with llama.cpp
./llama-cli -m Qwen2.5-Coder-7B-Instruct-iat-05-1-q4_k_m.gguf -p "Your prompt here"

Files

• Qwen2.5-Coder-7B-Instruct-iat-05-1-q4_k_m.gguf - Quantized model in GGUF format (q4_k_m)