official finetune example?
can we have an official finetune example please?
I saw a couple of repos out there
- https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen2_VL_(7B)-Vision.ipynb
- https://github.com/zhangfaen/finetune-Qwen2.5-VL
- https://github.com/sandy1990418/Finetune-Qwen2.5-VL
- https://github.com/2U1/Qwen2-VL-Finetune
but it would be nice to have an official example from Qwen team
I finetuned 72B model by deepspeed ZeRO-3 on 8 * A800 GPUs. https://llamafactory.readthedocs.io/en/latest/advanced/distributed.html#id12
That's interesting! qLoRA, I suppose?
I also am using Llama Factory, but, I was hoping to see an official (provided by Qwen), minimal example that just directly calls huggingface trainer or lower level than that.
Llama Factory is a very heavy wrapper and hides many details, where I was hoping to see the essential bits.
hello, do you find the fine-tuning examples? Would you like to share some?
OK, seems they put the training script using torchrun in the official qwen vl github repo.
I couldn’t find an official fine-tuning tutorial, but this project can be refered: https://github.com/zhangfaen/finetune-Qwen2.5-VL.