DistilBert-finetuned-Hackaton

This model is a fine-tuned version of distilbert-base-uncased-finetuned-sst-2-english on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.1456
• Accuracy: 0.4283
• F1: 0.4344

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
2.3155	1.0	338	2.6640	0.33	0.3161
2.2064	2.0	676	2.5991	0.3283	0.3094
2.0703	3.0	1014	2.5172	0.3467	0.3347
2.0222	4.0	1352	2.4497	0.3567	0.3434
1.9197	5.0	1690	2.3951	0.375	0.3639
1.8334	6.0	2028	2.3398	0.375	0.3646
1.7327	7.0	2366	2.3231	0.3833	0.3749
1.6621	8.0	2704	2.3040	0.3867	0.3787
1.5902	9.0	3042	2.2702	0.3883	0.3809
1.5554	10.0	3380	2.2230	0.4167	0.4143
1.5008	11.0	3718	2.2277	0.4067	0.3999
1.4451	12.0	4056	2.2023	0.4033	0.4025
1.3788	13.0	4394	2.1953	0.41	0.4066
1.3418	14.0	4732	2.1774	0.4083	0.4036
1.2689	15.0	5070	2.1798	0.41	0.4123
1.2495	16.0	5408	2.1700	0.4233	0.4228
1.1946	17.0	5746	2.1653	0.42	0.4241
1.1652	18.0	6084	2.1672	0.4283	0.4279
1.1428	19.0	6422	2.1631	0.4217	0.4259
1.1027	20.0	6760	2.1501	0.4133	0.4189
1.063	21.0	7098	2.1522	0.4183	0.4244
1.0621	22.0	7436	2.1480	0.42	0.4258
1.0412	23.0	7774	2.1491	0.4217	0.4285
1.0311	24.0	8112	2.1493	0.4267	0.4333
1.0195	25.0	8450	2.1456	0.4283	0.4344

Framework versions

• Transformers 4.24.0
• Pytorch 1.12.1+cu113
• Datasets 2.7.1
• Tokenizers 0.13.2