vit-pretraining-2023_12_11-cxr-classification

This model was trained from scratch on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 1.6726
• Accuracy: 0.6033

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-06
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.4135	1.0	228	1.2518	0.6426
1.2038	2.0	457	1.2201	0.6415
1.3041	3.0	685	1.1914	0.6251
1.2537	4.0	914	1.1506	0.6219
1.2731	5.0	1142	1.1575	0.6175
1.226	6.0	1371	1.1689	0.6262
1.1369	7.0	1599	1.1385	0.6317
1.0285	8.0	1828	1.1477	0.6328
0.8827	9.0	2056	1.1554	0.6262
0.8787	10.0	2285	1.1614	0.6317
0.7872	11.0	2513	1.1644	0.6339
0.5644	12.0	2742	1.1884	0.6361
0.6407	13.0	2970	1.2171	0.5967
0.4576	14.0	3199	1.2272	0.6284
0.4908	15.0	3427	1.2462	0.6186
0.3767	16.0	3656	1.2672	0.6142
0.2817	17.0	3884	1.3112	0.6142
0.2429	18.0	4113	1.3251	0.6109
0.1766	19.0	4341	1.3853	0.6208
0.2002	20.0	4570	1.4216	0.5858
0.1292	21.0	4798	1.4720	0.5934
0.1414	22.0	5027	1.4895	0.6077
0.1132	23.0	5255	1.5467	0.6033
0.0864	24.0	5484	1.5533	0.6120
0.0748	25.0	5712	1.5990	0.6055
0.0708	26.0	5941	1.6205	0.6098
0.0502	27.0	6169	1.6452	0.5978
0.0505	28.0	6398	1.6634	0.5967
0.0748	29.0	6626	1.6689	0.6011
0.0434	29.93	6840	1.6726	0.6033

Framework versions

• Transformers 4.36.0.dev0
• Pytorch 2.1.1+cu121
• Datasets 2.15.0
• Tokenizers 0.15.0