metadata
license: llama2
base_model:
- lmsys/vicuna-13b-v1.5-16k
This is a half precision (float16) gguf file for vicuna-13b-v1.5-16k.
Conversion code:
import os import subprocess
from huggingface_hub import snapshot_download
model_id="lmsys/vicuna-13b-v1.5-16k" snapshot_download(repo_id=model_id, local_dir="vicuna-hf", local_dir_use_symlinks=False, revision="main")
subprocess.run(["git", "clone", "https://github.com/ggerganov/llama.cpp.git"])
subprocess.run(["pip", "install", "-r", "llama.cpp/requirements.txt"])
subprocess.run(["python", "llama.cpp/convert_hf_to_gguf.py", "vicuna-hf", "--outfile", "vicuna-13b-v1.5-16k.gguf", "--outtype", "f16"])
(modified from https://www.substratus.ai/blog/converting-hf-model-gguf-model)