answerdotai-ModernBERT-base-atomic-anion-1e-06-256

This model is a fine-tuned version of answerdotai/ModernBERT-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.3983

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 256
• eval_batch_size: 256
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
0.5623	1.0	1152	0.5391
0.4968	2.0	2304	0.4940
0.4624	3.0	3456	0.4727
0.4443	4.0	4608	0.4542
0.4246	5.0	5760	0.4470
0.4141	6.0	6912	0.4378
0.3985	7.0	8064	0.4288
0.393	8.0	9216	0.4221
0.3788	9.0	10368	0.4176
0.3696	10.0	11520	0.4137
0.3621	11.0	12672	0.4114
0.3542	12.0	13824	0.4051
0.3475	13.0	14976	0.4049
0.3418	14.0	16128	0.4005
0.3351	15.0	17280	0.4005
0.3295	16.0	18432	0.3979
0.3259	17.0	19584	0.3970
0.3217	18.0	20736	0.3958
0.3136	19.0	21888	0.3965
0.3125	20.0	23040	0.3971
0.3075	21.0	24192	0.3957
0.3055	22.0	25344	0.3964
0.3011	23.0	26496	0.3968
0.2971	24.0	27648	0.3983

Framework versions

• Transformers 4.49.0
• Pytorch 2.6.0+cu124
• Datasets 3.3.2
• Tokenizers 0.21.1