ValueError During SFT Fine-tuning with Gamma3 Model
Problem Description
Encountering ValueError: Processor was not found error when performing SFT (Supervised Fine-Tuning) on Gamma3 model using LLaMA-Factory.
Environment
LLaMA-Factory Version: 0.9.3
Python Version: 3.10
Operating System: Linux
Model: Gamma3
Training Type: SFT (Supervised Fine-Tuning)
Hardware: GPU
Error Details
Error Message
Full Stack Trace
Error Location
The error occurs during the data preprocessing stage of SFT workflow, specifically in the _get_preprocessed_dataset function within data/loader.py.
You might need to use gemma2 template for fine-tuning the gemma3-270m-it model because it is not multimodal
Hi @rfhshfhs ,
Thank you so much for reaching out to us, the issue might be caused based on different reasons. Please find the some of the following common areas needs to check to avoid the above kind of issues.
Clear the Cache: The issue might be a corrupted cache.
Run the commandtransformers-cli envto find the cache directory.
Manually delete the folder corresponding to the Gemma model from the cache directory.
Restart the training process. The transformers library will re-download the model and its processor.Incorrect model_name_or_path: In your LLaMA-Factory configuration file or command-line arguments, the model_name_or_path parameter might be set to the wrong value. It should point to a valid Hugging Face model repository or a local path containing the model files (if the model is loaded into your local and trying to use from local repo).
Please following the instruction-tuned template which is specifically used for
itmodels for you experiment or fine-tuning purpose.
Thanks.