asynctales/Qwen2.5-Coder-0.5B-Instruct-Q4_K_M-GGUF

This model was converted to GGUF format from Qwen/Qwen2.5-Coder-0.5B-Instruct using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

path\to\llama-server.exe -m path\to\qwen2.5-coder-0.5b-instruct-q4_k_m.gguf -ngl 99 -fa -ub 1024 -b 1024 --ctx-size 0 --cache-reuse 256 -np 2 --port [port]