Please help! Unable to load the model
I downloaded Llama-3.2-3B-Instruct-F16.gguf and the model load fails with the following error.
error loading model: create_tensor: tensor 'output.weight' not found
llama_load_model_from_file: failed to load model
Traceback (most recent call last):
File "/teamspace/studios/this_studio/backend/llm.py", line 3, in
llm = Llama(model_path='model/llama3.2-3b-gguf/Llama-3.2-3B-Instruct-F16.gguf')
File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/llama_cpp/llama.py", line 962, in init
self._n_vocab = self.n_vocab()
File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/llama_cpp/llama.py", line 2266, in n_vocab
return self._model.n_vocab()
File "/home/zeus/miniconda3/envs/cloudspace/lib/python3.10/site-packages/llama_cpp/llama.py", line 251, in n_vocab
assert self.model is not None
AssertionError
ChatGPT suggested upgrading the version of llama-cpp -python and I am having lot of problems upgrading it and running in circles trying to resolve environment errors. The current version of llama-cpp-python is 0.2.24.
Can you please let me know how I can resolve the model load error?
Thanks!