metadata
base_model:
- Qwen/Qwen2.5-VL-3B-Instruct
- vidore/colqwen2.5-v0.2
Finetune on MPDocVQA for one epoch with following parameters on H200
QUANTIZATION_STRATEGY ="bf16" SEED = 42
Training hyperparameters
EPOCHS = 1 BATCH_SIZE_TRAIN = 16 BATCH_SIZE_EVAL = 16 GRADIENT_ACCUM_STEPS = 2 WARMUP_STEPS = 20 LEARNING_RATE = 5e-5 SAVE_STEPS = 100 EVAL_STEPS = 10 LOGGING_STEPS = 1 SAVE_TOTAL_LIMIT = 5 REPORT_TO = ["wandb"] GRADIENT_CHECKPOINTING = False EVAL_STRATEGY = "steps"