ibama_29102024_20241029175942

This model is a fine-tuned version of pierreguillou/bert-base-cased-squad-v1.1-portuguese.

Model description

Dataset com 1750 registros. Média do tamanho dos contextos: 2467.439831104856

["train"] : 1421 registros

["test"] : 329 registros

{'exact_match': 6.990881458966565, 'f1': 41.36428322707063}

Resultados:

:: Filtrando registros de ['test'] onde o contexto possuia até 6697 caracteres.

Modelo: ibama_29102024_20241029175942 :

'exact_match': 3.9755351681957185, 'f1': 38.429269059347

Modelo: pierreguillou/bert-base-cased-squad-v1.1-portuguese :

'exact_match': 6.422018348623853, 'f1': 37.47550481021018

Modelo: neuralmind/bert-base-portuguese-cased :

'exact_match': 0.0, 'f1': 21.520346204352514

:: Filtrando registros de ['test'] onde o contexto possuia até 512 caracteres.

Modelo: ibama_29102024_20241029175942 :

'exact_match': 12.67605633802817, 'f1': 70.76635146201694 

Modelo: pierreguillou/bert-base-cased-squad-v1.1-portuguese :

'exact_match': 1.408450704225352, 'f1': 38.42469128241023

Modelo: neuralmind/bert-base-portuguese-cased :

'exact_match': 0.0, 'f1': 15.264048430063177

Training results

It achieves the following results on the evaluation set:

• Loss: 4.1817

Epoch Training Loss Validation Loss 1 No log 4.598662 2 No log 4.266841 3 No log 4.225364 4 No log 4.181730

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 4
• mixed_precision_training: Native AMP

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.0+cu121
• Datasets 3.0.2
• Tokenizers 0.19.1