Renugadevi82/cisco-nx-ai-gguf-q4_k_m

Cisco NX-AI GGUF Q4_K_M

4-bit quantized (smallest, ~600MB)

Download

wget https://huggingface.co/Renugadevi82/cisco-nx-ai-gguf-q4_k_m/resolve/main/cisco-nx-ai-q4_k_m.gguf

Usage with llama.cpp

./llama-cli -m cisco-nx-ai-q4_k_m.gguf -p "Configure VLAN 100" -n 100

Usage with llama-cpp-python

from llama_cpp import Llama
llm = Llama(model_path="cisco-nx-ai-q4_k_m.gguf")
output = llm("Configure VLAN 100", max_tokens=100)
print(output['choices'][0]['text'])

File size: 0.62 GB