unidisc / configs /lr_scheduler /constant_warmup_cosine_decay.yaml
aswerdlow's picture
Initial commit
131da64
raw
history blame contribute delete
105 Bytes
_target_: transformers.get_cosine_schedule_with_warmup
num_warmup_steps: 2500
num_training_steps: 1000000