best setting of num_train_epochs
Hi !
I have completed the hyperparameter search and am now conducting the formal fine-tuning training. My dataset is quite large, with several million cells. I would like to ask what a suitable number of num_train_epochs would be. During the hyperparameter search, the code was set to 1 epoch to avoid overfitting. However, for the formal training, the default parameter you provided is 8 epochs. Based on your experience, how many epochs would be appropriate? Or, is there any way to output the model's checkpoint at the end of each epoch?
Best regards
Thank you for your question. The number of epochs is task and data specific, so there is no default value that we recommend. However, as a general rule of thumb, 1 epoch is usually sufficient for fine-tuning and to prevent overfitting. Checkpoints can be saved at the end of each epoch or more frequently by setting the save_strategy on the TrainingArguments.