flan-t5-large-Imagine-WQ

This model is a fine-tuned version of google/flan-t5-large on an unknown dataset. It achieves the following results on the evaluation set:

• Gen Len: 509.77
• Loss: 3.4156
• Rouge1: 20.6216
• Rouge2: 4.1953
• Rougel: 16.091
• Rougelsum: 17.4067

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.08
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Gen Len	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
No log	1.0	218	508.68	3.5702	16.525	3.2484	14.5949	13.3919
No log	2.0	436	509.62	3.5013	17.6208	3.7153	14.7229	14.9556
No log	3.0	654	509.9767	3.4592	19.4782	3.9388	15.7853	16.3468
No log	4.0	872	509.8233	3.4337	20.8148	4.287	16.3887	17.3557
No log	5.0	1090	509.77	3.4156	20.6216	4.1953	16.091	17.4067

Framework versions

• Transformers 4.35.2
• Pytorch 2.0.1+cu117
• Datasets 2.15.0
• Tokenizers 0.15.0