---
library_name: transformers
license: cc-by-nc-sa-4.0
base_model: stulcrad/Robeczech-PRETRAINED4-CERED3
tags:
- generated_from_trainer
datasets:
- generator
metrics:
- accuracy
model-index:
- name: Robeczech-PRETRAINED43-CERED2
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# Robeczech-PRETRAINED43-CERED2

This model is a fine-tuned version of [stulcrad/Robeczech-PRETRAINED4-CERED3](https://huggingface.co/stulcrad/Robeczech-PRETRAINED4-CERED3) on the generator dataset.
It achieves the following results on the evaluation set:
- Loss: 1.1137
- Accuracy: 0.8807
- Micro Precision: 0.8807
- Micro Recall: 0.8807
- Micro F1: 0.8807
- Macro Precision: 0.8503
- Macro Recall: 0.8424
- Macro F1: 0.8426

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 0.0001
- train_batch_size: 12
- eval_batch_size: 12
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 24
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
- lr_scheduler_type: linear
- lr_scheduler_warmup_steps: 500
- num_epochs: 10

### Training results

| Training Loss | Epoch | Step   | Validation Loss | Accuracy | Micro Precision | Micro Recall | Micro F1 | Macro Precision | Macro Recall | Macro F1 |
|:-------------:|:-----:|:------:|:---------------:|:--------:|:---------------:|:------------:|:--------:|:---------------:|:------------:|:--------:|
| 0.4181        | 1.0   | 15074  | 0.5844          | 0.8550   | 0.8550          | 0.8550       | 0.8550   | 0.8202          | 0.7858       | 0.7946   |
| 0.3727        | 2.0   | 30148  | 0.5572          | 0.8620   | 0.8620          | 0.8620       | 0.8620   | 0.8293          | 0.8058       | 0.8102   |
| 0.282         | 3.0   | 45222  | 0.6841          | 0.8567   | 0.8567          | 0.8567       | 0.8567   | 0.8174          | 0.8071       | 0.8010   |
| 0.2209        | 4.0   | 60296  | 0.6510          | 0.8672   | 0.8672          | 0.8672       | 0.8672   | 0.8171          | 0.8205       | 0.8132   |
| 0.1918        | 5.0   | 75370  | 0.7609          | 0.8665   | 0.8665          | 0.8665       | 0.8665   | 0.8254          | 0.8171       | 0.8162   |
| 0.13          | 6.0   | 90444  | 0.8197          | 0.8724   | 0.8724          | 0.8724       | 0.8724   | 0.8347          | 0.8345       | 0.8302   |
| 0.0959        | 7.0   | 105518 | 0.8901          | 0.8721   | 0.8721          | 0.8721       | 0.8721   | 0.8304          | 0.8256       | 0.8236   |
| 0.0799        | 8.0   | 120592 | 1.0162          | 0.8749   | 0.8749          | 0.8749       | 0.8749   | 0.8364          | 0.8361       | 0.8316   |
| 0.0454        | 9.0   | 135666 | 1.0664          | 0.8747   | 0.8747          | 0.8747       | 0.8747   | 0.8280          | 0.8363       | 0.8284   |
| 0.0274        | 10.0  | 150740 | 1.1455          | 0.8768   | 0.8768          | 0.8768       | 0.8768   | 0.8326          | 0.8369       | 0.8313   |


### Framework versions

- Transformers 4.46.2
- Pytorch 2.5.1+cu124
- Datasets 3.1.0
- Tokenizers 0.20.3