---
license: apache-2.0
tags:
- generated_from_trainer
metrics:
- precision
- recall
- f1
- accuracy
model-index:
- name: BERTModified-finetuned-wikitext-test
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# BERTModified-finetuned-wikitext-test

This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 18.8994
- Precision: 0.25
- Recall: 0.25
- F1: 0.25
- Accuracy: 0.25

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 5e-05
- train_batch_size: 4
- eval_batch_size: 4
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 50

### Training results

| Training Loss | Epoch | Step  | Validation Loss | Precision | Recall | F1     | Accuracy |
|:-------------:|:-----:|:-----:|:---------------:|:---------:|:------:|:------:|:--------:|
| 19.9877       | 1.0   | 250   | 19.8070         | 0.0385    | 0.0385 | 0.0385 | 0.0385   |
| 15.4776       | 2.0   | 500   | 20.2930         | 0.0577    | 0.0577 | 0.0577 | 0.0577   |
| 13.1238       | 3.0   | 750   | 20.1112         | 0.0769    | 0.0769 | 0.0769 | 0.0769   |
| 11.1387       | 4.0   | 1000  | 19.9105         | 0.0897    | 0.0897 | 0.0897 | 0.0897   |
| 9.5317        | 5.0   | 1250  | 19.9108         | 0.1282    | 0.1282 | 0.1282 | 0.1282   |
| 8.037         | 6.0   | 1500  | 19.6093         | 0.1410    | 0.1410 | 0.1410 | 0.1410   |
| 6.7498        | 7.0   | 1750  | 19.1636         | 0.1474    | 0.1474 | 0.1474 | 0.1474   |
| 5.6472        | 8.0   | 2000  | 19.6709         | 0.1538    | 0.1538 | 0.1538 | 0.1538   |
| 4.6665        | 9.0   | 2250  | 19.2537         | 0.1667    | 0.1667 | 0.1667 | 0.1667   |
| 3.9107        | 10.0  | 2500  | 19.1982         | 0.1474    | 0.1474 | 0.1474 | 0.1474   |
| 3.1874        | 11.0  | 2750  | 18.9938         | 0.1731    | 0.1731 | 0.1731 | 0.1731   |
| 2.5846        | 12.0  | 3000  | 18.7462         | 0.2115    | 0.2115 | 0.2115 | 0.2115   |
| 2.1464        | 13.0  | 3250  | 19.0017         | 0.1667    | 0.1667 | 0.1667 | 0.1667   |
| 1.7521        | 14.0  | 3500  | 18.4513         | 0.1859    | 0.1859 | 0.1859 | 0.1859   |
| 1.4561        | 15.0  | 3750  | 18.7532         | 0.2051    | 0.2051 | 0.2051 | 0.2051   |
| 1.2254        | 16.0  | 4000  | 18.3970         | 0.2179    | 0.2179 | 0.2179 | 0.2179   |
| 1.0416        | 17.0  | 4250  | 18.9764         | 0.1859    | 0.1859 | 0.1859 | 0.1859   |
| 0.8923        | 18.0  | 4500  | 18.3271         | 0.2244    | 0.2244 | 0.2244 | 0.2244   |
| 0.7803        | 19.0  | 4750  | 18.5893         | 0.2436    | 0.2436 | 0.2436 | 0.2436   |
| 0.6839        | 20.0  | 5000  | 18.3505         | 0.2051    | 0.2051 | 0.2051 | 0.2051   |
| 0.6175        | 21.0  | 5250  | 18.6798         | 0.2051    | 0.2051 | 0.2051 | 0.2051   |
| 0.5491        | 22.0  | 5500  | 18.7426         | 0.2115    | 0.2115 | 0.2115 | 0.2115   |
| 0.4952        | 23.0  | 5750  | 18.3955         | 0.2179    | 0.2179 | 0.2179 | 0.2179   |
| 0.4441        | 24.0  | 6000  | 18.5502         | 0.2564    | 0.2564 | 0.2564 | 0.2564   |
| 0.4047        | 25.0  | 6250  | 18.9599         | 0.2244    | 0.2244 | 0.2244 | 0.2244   |
| 0.3768        | 26.0  | 6500  | 18.8141         | 0.2308    | 0.2308 | 0.2308 | 0.2308   |
| 0.3435        | 27.0  | 6750  | 18.9732         | 0.2436    | 0.2436 | 0.2436 | 0.2436   |
| 0.3164        | 28.0  | 7000  | 18.9216         | 0.2372    | 0.2372 | 0.2372 | 0.2372   |
| 0.2954        | 29.0  | 7250  | 18.6152         | 0.1987    | 0.1987 | 0.1987 | 0.1987   |
| 0.2736        | 30.0  | 7500  | 18.6001         | 0.25      | 0.25   | 0.25   | 0.25     |
| 0.2491        | 31.0  | 7750  | 19.1374         | 0.2436    | 0.2436 | 0.2436 | 0.2436   |
| 0.2359        | 32.0  | 8000  | 18.8624         | 0.25      | 0.25   | 0.25   | 0.25     |
| 0.2222        | 33.0  | 8250  | 18.3201         | 0.2308    | 0.2308 | 0.2308 | 0.2308   |
| 0.212         | 34.0  | 8500  | 18.7708         | 0.2179    | 0.2179 | 0.2179 | 0.2179   |
| 0.1864        | 35.0  | 8750  | 18.8994         | 0.2372    | 0.2372 | 0.2372 | 0.2372   |
| 0.1771        | 36.0  | 9000  | 18.3130         | 0.2308    | 0.2308 | 0.2308 | 0.2308   |
| 0.1703        | 37.0  | 9250  | 18.6183         | 0.2436    | 0.2436 | 0.2436 | 0.2436   |
| 0.1554        | 38.0  | 9500  | 18.8593         | 0.2372    | 0.2372 | 0.2372 | 0.2372   |
| 0.1469        | 39.0  | 9750  | 18.8936         | 0.2628    | 0.2628 | 0.2628 | 0.2628   |
| 0.1407        | 40.0  | 10000 | 18.9002         | 0.2372    | 0.2372 | 0.2372 | 0.2372   |
| 0.1328        | 41.0  | 10250 | 19.1827         | 0.2564    | 0.2564 | 0.2564 | 0.2564   |
| 0.1297        | 42.0  | 10500 | 18.5465         | 0.25      | 0.25   | 0.25   | 0.25     |
| 0.1226        | 43.0  | 10750 | 18.9125         | 0.2308    | 0.2308 | 0.2308 | 0.2308   |
| 0.1218        | 44.0  | 11000 | 19.0831         | 0.2308    | 0.2308 | 0.2308 | 0.2308   |
| 0.1136        | 45.0  | 11250 | 18.7969         | 0.2372    | 0.2372 | 0.2372 | 0.2372   |
| 0.1075        | 46.0  | 11500 | 18.7629         | 0.25      | 0.25   | 0.25   | 0.25     |
| 0.1044        | 47.0  | 11750 | 18.9700         | 0.2115    | 0.2115 | 0.2115 | 0.2115   |
| 0.1042        | 48.0  | 12000 | 18.7211         | 0.2628    | 0.2628 | 0.2628 | 0.2628   |
| 0.1008        | 49.0  | 12250 | 18.9104         | 0.2244    | 0.2244 | 0.2244 | 0.2244   |
| 0.1014        | 50.0  | 12500 | 18.7892         | 0.25      | 0.25   | 0.25   | 0.25     |


### Framework versions

- Transformers 4.24.0
- Pytorch 1.13.0
- Datasets 2.6.1
- Tokenizers 0.13.2