See axolotl config
axolotl version: 0.9.2
base_model: timarni/qwen3_dpo
# Automatically upload checkpoint and final model to HF
# hub_model_id: username/custom_model_name
plugins:
- axolotl.integrations.cut_cross_entropy.CutCrossEntropyPlugin
strict: false
chat_template: qwen3
datasets:
- path: timarni/MNLP_STEM_IT_HARD
type: alpaca
split: train
shuffle_merged_datasets: true
val_set_size: 0.1
output_dir: ./outputs/dpo_stem_it_hard
dataset_prepared_path: last_run_prepared
sequence_len: 4096 #2048
sample_packing: true # was true -> need to check if it actually learns on the samples or not (better understand te hyperparam and event. install axolotl to debug)
eval_sample_packing: false
pad_to_sequence_len: true
# train_on_inputs: true # NEW
# group_by_length: false NEW?
# To be sure that no LORA is done
adapter: null
lora: false
merge_lora: false
wandb_project: mnlp_project
wandb_entity: tim-arni
wandb_watch:
wandb_name: dpo_stem_it_hard
wandb_log_model:
gradient_accumulation_steps: 16 # 2
micro_batch_size: 2 # 1
num_epochs: 20
optimizer: adamw_torch
lr_scheduler: cosine
learning_rate: 0.00005 # 0.00005
# cosine_min_lr_ratio: 0.1
warmup_ratio: 0.05
weight_decay: 0.01
bf16: auto
tf32: true
gradient_checkpointing: offload
gradient_checkpointing_kwargs:
use_reentrant: false
resume_from_checkpoint:
logging_steps: 1
gradient_clipping: 1.0 # or max_grad_norm?
flash_attention: true
evals_per_epoch: 4
saves_per_epoch: 1
save_total_limit: 10
special_tokens:
outputs/dpo_stem_it_hard
This model is a fine-tuned version of timarni/qwen3_dpo on the timarni/MNLP_STEM_IT_HARD dataset. It achieves the following results on the evaluation set:
- Loss: 0.1842
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 2
- eval_batch_size: 2
- seed: 42
- distributed_type: multi-GPU
- num_devices: 4
- gradient_accumulation_steps: 16
- total_train_batch_size: 128
- total_eval_batch_size: 8
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 2
- num_epochs: 20.0
Training results
| Training Loss | Epoch | Step | Validation Loss |
|---|---|---|---|
| 0.7556 | 0.3404 | 1 | 0.7317 |
| 0.7451 | 0.6809 | 2 | 0.2301 |
| 0.2038 | 1.0 | 3 | 0.1533 |
| 0.1309 | 1.3404 | 4 | 0.2209 |
| 0.2067 | 1.6809 | 5 | 0.1463 |
| 0.1065 | 2.0 | 6 | 0.1469 |
| 0.1173 | 2.3404 | 7 | 0.1395 |
| 0.1027 | 2.6809 | 8 | 0.1370 |
| 0.0756 | 3.0 | 9 | 0.1491 |
| 0.0852 | 3.3404 | 10 | 0.1476 |
| 0.0776 | 3.6809 | 11 | 0.1421 |
| 0.0546 | 4.0 | 12 | 0.1391 |
| 0.0557 | 4.3404 | 13 | 0.1376 |
| 0.0523 | 4.6809 | 14 | 0.1408 |
| 0.0369 | 5.0 | 15 | 0.1497 |
| 0.0384 | 5.3404 | 16 | 0.1581 |
| 0.0382 | 5.6809 | 17 | 0.1622 |
| 0.0258 | 6.0 | 18 | 0.1660 |
| 0.0273 | 6.3404 | 19 | 0.1670 |
| 0.0255 | 6.6809 | 20 | 0.1672 |
| 0.0185 | 7.0 | 21 | 0.1678 |
| 0.0207 | 7.3404 | 22 | 0.1689 |
| 0.0197 | 7.6809 | 23 | 0.1698 |
| 0.0147 | 8.0 | 24 | 0.1717 |
| 0.0167 | 8.3404 | 25 | 0.1734 |
| 0.0167 | 8.6809 | 26 | 0.1754 |
| 0.0126 | 9.0 | 27 | 0.1769 |
| 0.0149 | 9.3404 | 28 | 0.1790 |
| 0.015 | 9.6809 | 29 | 0.1800 |
| 0.0115 | 10.0 | 30 | 0.1814 |
| 0.0139 | 10.3404 | 31 | 0.1823 |
| 0.0141 | 10.6809 | 32 | 0.1830 |
| 0.011 | 11.0 | 33 | 0.1835 |
| 0.0135 | 11.3404 | 34 | 0.1833 |
| 0.0139 | 11.6809 | 35 | 0.1840 |
| 0.0109 | 12.0 | 36 | 0.1836 |
| 0.0134 | 12.3404 | 37 | 0.1838 |
| 0.0138 | 12.6809 | 38 | 0.1844 |
| 0.0109 | 13.0 | 39 | 0.1837 |
| 0.0133 | 13.3404 | 40 | 0.1842 |
Framework versions
- Transformers 4.51.3
- Pytorch 2.5.1+cu121
- Datasets 3.5.1
- Tokenizers 0.21.1
- Downloads last month
- 19