long_t5_test

This model is a fine-tuned version of google/long-t5-tglobal-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.7929
• Rouge1: 0.5445
• Rouge2: 0.3112
• Rougel: 0.3469
• Rougelsum: 0.346
• Gen Len: 410.5957

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Gen Len	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
1.3932	1.0	1263	483.3617	0.9704	0.4329	0.2051	0.273	0.2724
1.2153	2.0	2526	452.3191	0.9323	0.4598	0.2276	0.2861	0.2856
1.1271	3.0	3789	394.5674	0.8961	0.4943	0.2629	0.3176	0.3171
1.0753	4.0	5052	424.6312	0.8926	0.4901	0.2611	0.3147	0.3146
1.0467	5.0	6315	409.1489	0.8780	0.504	0.2715	0.3249	0.3249
1.0262	6.0	7578	417.8298	0.8753	0.5117	0.2839	0.335	0.3354
1.0023	7.0	8841	416.0993	0.8620	0.507	0.2793	0.3288	0.3291
0.9851	8.0	10104	389.539	0.8556	0.5178	0.2891	0.3386	0.3382
0.9943	9.0	11367	409.2482	0.8570	0.5248	0.292	0.3405	0.3408
0.9463	10.0	12630	396.8511	0.7550	0.5243	0.2906	0.3329	0.3327
0.9385	11.0	13893	0.7894	0.5377	0.3003	0.3442	0.3439	407.3333
0.9157	12.0	15156	0.7918	0.5449	0.3036	0.3424	0.342	415.4255
0.9378	13.0	16419	0.7920	0.5332	0.2935	0.3368	0.3365	421.4326
0.9194	14.0	17682	0.7898	0.5509	0.3087	0.3476	0.3474	406.3688
0.911	15.0	18945	0.7956	0.5361	0.2991	0.3403	0.3398	415.9362
0.8769	16.0	20208	0.7918	0.5433	0.3058	0.3459	0.3453	414.4184
0.8808	17.0	21471	0.7901	0.5445	0.3085	0.3492	0.3484	400.5177
0.8908	18.0	22734	0.7926	0.5404	0.3043	0.3427	0.3419	404.7801
0.8868	19.0	23997	0.7919	0.5449	0.3104	0.3494	0.3489	407.461
0.8868	20.0	25260	0.7929	0.5445	0.3112	0.3469	0.346	410.5957

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0
• Datasets 3.6.0
• Tokenizers 0.19.1