bert-24.2

This model is a fine-tuned version of hung200504/bert-24.1 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 11.4596

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-07
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 2

Training results

Training Loss	Epoch	Step	Validation Loss
10.5094	0.09	5	11.5096
10.6572	0.18	10	11.5053
10.3097	0.27	15	11.5006
10.4711	0.36	20	11.4964
10.505	0.45	25	11.4927
10.5575	0.55	30	11.4890
9.9085	0.64	35	11.4854
10.5509	0.73	40	11.4820
10.1511	0.82	45	11.4788
10.0779	0.91	50	11.4760
9.8849	1.0	55	11.4735
10.1105	1.09	60	11.4713
10.2491	1.18	65	11.4692
10.4967	1.27	70	11.4671
10.0624	1.36	75	11.4654
10.5219	1.45	80	11.4642
10.2786	1.55	85	11.4627
10.2518	1.64	90	11.4618
10.1067	1.73	95	11.4609
10.36	1.82	100	11.4603
9.9273	1.91	105	11.4598
10.2964	2.0	110	11.4596

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1