Renugadevi82/cisco-nx-ai-gguf-q8_0

Cisco NX-AI GGUF Q8_0

8-bit quantized (high quality, ~1.2GB)

Download

wget https://huggingface.co/Renugadevi82/cisco-nx-ai-gguf-q8_0/resolve/main/cisco-nx-ai-q8_0.gguf

Usage with llama.cpp

./llama-cli -m cisco-nx-ai-q8_0.gguf -p "Configure VLAN 100" -n 100

Usage with llama-cpp-python

from llama_cpp import Llama
llm = Llama(model_path="cisco-nx-ai-q8_0.gguf")
output = llm("Configure VLAN 100", max_tokens=100)
print(output['choices'][0]['text'])

File size: 1.09 GB