End of training

README.md

@@ -0,0 +1,81 @@
+---
+library_name: transformers
+license: cc-by-nc-sa-4.0
+base_model: stulcrad/Robeczech-PRETRAINED4-CERED3
+tags:
+- generated_from_trainer
+datasets:
+- generator
+metrics:
+- accuracy
+model-index:
+- name: Robeczech-PRETRAINED43-CERED2
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# Robeczech-PRETRAINED43-CERED2
+
+This model is a fine-tuned version of [stulcrad/Robeczech-PRETRAINED4-CERED3](https://huggingface.co/stulcrad/Robeczech-PRETRAINED4-CERED3) on the generator dataset.
+It achieves the following results on the evaluation set:
+- Loss: 1.1137
+- Accuracy: 0.8807
+- Micro Precision: 0.8807
+- Micro Recall: 0.8807
+- Micro F1: 0.8807
+- Macro Precision: 0.8503
+- Macro Recall: 0.8424
+- Macro F1: 0.8426
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0001
+- train_batch_size: 12
+- eval_batch_size: 12
+- seed: 42
+- gradient_accumulation_steps: 2
+- total_train_batch_size: 24
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- num_epochs: 10
+
+### Training results
+
+| Training Loss | Epoch | Step   | Validation Loss | Accuracy | Micro Precision | Micro Recall | Micro F1 | Macro Precision | Macro Recall | Macro F1 |
+|:-------------:|:-----:|:------:|:---------------:|:--------:|:---------------:|:------------:|:--------:|:---------------:|:------------:|:--------:|
+| 0.4181        | 1.0   | 15074  | 0.5844          | 0.8550   | 0.8550          | 0.8550       | 0.8550   | 0.8202          | 0.7858       | 0.7946   |
+| 0.3727        | 2.0   | 30148  | 0.5572          | 0.8620   | 0.8620          | 0.8620       | 0.8620   | 0.8293          | 0.8058       | 0.8102   |
+| 0.282         | 3.0   | 45222  | 0.6841          | 0.8567   | 0.8567          | 0.8567       | 0.8567   | 0.8174          | 0.8071       | 0.8010   |
+| 0.2209        | 4.0   | 60296  | 0.6510          | 0.8672   | 0.8672          | 0.8672       | 0.8672   | 0.8171          | 0.8205       | 0.8132   |
+| 0.1918        | 5.0   | 75370  | 0.7609          | 0.8665   | 0.8665          | 0.8665       | 0.8665   | 0.8254          | 0.8171       | 0.8162   |
+| 0.13          | 6.0   | 90444  | 0.8197          | 0.8724   | 0.8724          | 0.8724       | 0.8724   | 0.8347          | 0.8345       | 0.8302   |
+| 0.0959        | 7.0   | 105518 | 0.8901          | 0.8721   | 0.8721          | 0.8721       | 0.8721   | 0.8304          | 0.8256       | 0.8236   |
+| 0.0799        | 8.0   | 120592 | 1.0162          | 0.8749   | 0.8749          | 0.8749       | 0.8749   | 0.8364          | 0.8361       | 0.8316   |
+| 0.0454        | 9.0   | 135666 | 1.0664          | 0.8747   | 0.8747          | 0.8747       | 0.8747   | 0.8280          | 0.8363       | 0.8284   |
+| 0.0274        | 10.0  | 150740 | 1.1455          | 0.8768   | 0.8768          | 0.8768       | 0.8768   | 0.8326          | 0.8369       | 0.8313   |
+
+
+### Framework versions
+
+- Transformers 4.46.2
+- Pytorch 2.5.1+cu124
+- Datasets 3.1.0
+- Tokenizers 0.20.3

added_tokens.json

@@ -0,0 +1,7 @@
+{
+  "[unused1]": 51997,
+  "[unused2]": 51998,
+  "[unused3]": 51999,
+  "[unused4]": 52000,
+  "[unused5]": 52001
+}

config.json

@@ -0,0 +1,160 @@
+{
+  "_name_or_path": "stulcrad/Robeczech-PRETRAINED4-CERED3",
+  "architectures": [
+    "RelationExtractionModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "P0 other",
+    "1": "P19 m\u00edsto narozen\u00ed",
+    "2": "P20 m\u00edsto \u00famrt\u00ed",
+    "3": "P21 pohlav\u00ed",
+    "4": "P22 otec",
+    "5": "P25 matka",
+    "6": "P26 cho\u0165",
+    "7": "P27 st\u00e1tn\u00ed ob\u010danstv\u00ed",
+    "8": "P36 hlavn\u00ed m\u011bsto",
+    "9": "P39 ve funkci",
+    "10": "P47 hrani\u010d\u00ed s",
+    "11": "P50 autor",
+    "12": "P53 rodina",
+    "13": "P54 \u010dlen sportovn\u00edho dru\u017estva",
+    "14": "P97 \u0161lechtick\u00fd titul",
+    "15": "P101 zam\u011b\u0159en\u00ed",
+    "16": "P102 stranick\u00e1 p\u0159\u00edslu\u0161nost",
+    "17": "P105 taxonomick\u00e9 za\u0159azen\u00ed",
+    "18": "P106 povol\u00e1n\u00ed",
+    "19": "P118 liga",
+    "20": "P131 nach\u00e1z\u00ed se v administrativn\u00ed jednotce",
+    "21": "P136 \u017e\u00e1nr",
+    "22": "P137 oper\u00e1tor",
+    "23": "P138 pojmenov\u00e1no po",
+    "24": "P140 vyzn\u00e1n\u00ed",
+    "25": "P150 ni\u017e\u0161\u00ed spr\u00e1vn\u00ed celky",
+    "26": "P155 p\u0159edchoz\u00ed",
+    "27": "P159 s\u00eddlo",
+    "28": "P161 hraje",
+    "29": "P171 nad\u0159azen\u00fd taxon",
+    "30": "P175 interpret",
+    "31": "P176 v\u00fdrobce",
+    "32": "P179 s\u00e9rie",
+    "33": "P206 u vodn\u00ed plochy",
+    "34": "P264 hudebn\u00ed vydavatelstv\u00ed",
+    "35": "P276 m\u00edsto",
+    "36": "P279 nadt\u0159\u00edda",
+    "37": "P360 seznam (\u010deho)",
+    "38": "P361 \u010d\u00e1st (\u010deho)",
+    "39": "P413 pozice hr\u00e1\u010de",
+    "40": "P425 obor tohoto povol\u00e1n\u00ed",
+    "41": "P460 \u00fadajn\u011b tot\u00e9\u017e co",
+    "42": "P461 protiklad",
+    "43": "P463 \u010dlen (\u010deho)",
+    "44": "P495 zem\u011b p\u016fvodu",
+    "45": "P527 skl\u00e1d\u00e1 se z",
+    "46": "P530 diplomatick\u00fd vztah",
+    "47": "P641 sport",
+    "48": "P669 ulice",
+    "49": "P706 oblast",
+    "50": "P708 diec\u00e9ze",
+    "51": "P734 p\u0159\u00edjmen\u00ed",
+    "52": "P735 rodn\u00e9 jm\u00e9no",
+    "53": "P740 m\u00edsto vzniku",
+    "54": "P800 d\u00edlo",
+    "55": "P921 hlavn\u00ed t\u00e9ma d\u00edla",
+    "56": "P974 p\u0159\u00edtok",
+    "57": "P1056 produkuje",
+    "58": "P1303 hudebn\u00ed n\u00e1stroj",
+    "59": "P1376 hlavn\u00ed s\u00eddlo \u010deho",
+    "60": "P1383 zahrnuje s\u00eddlo",
+    "61": "P1889 rozd\u00edln\u00e9 od",
+    "62": "P3373 sourozenec",
+    "63": "P4552 poho\u0159\u00ed"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "P0 other": 0,
+    "P101 zam\u011b\u0159en\u00ed": 15,
+    "P102 stranick\u00e1 p\u0159\u00edslu\u0161nost": 16,
+    "P105 taxonomick\u00e9 za\u0159azen\u00ed": 17,
+    "P1056 produkuje": 57,
+    "P106 povol\u00e1n\u00ed": 18,
+    "P118 liga": 19,
+    "P1303 hudebn\u00ed n\u00e1stroj": 58,
+    "P131 nach\u00e1z\u00ed se v administrativn\u00ed jednotce": 20,
+    "P136 \u017e\u00e1nr": 21,
+    "P137 oper\u00e1tor": 22,
+    "P1376 hlavn\u00ed s\u00eddlo \u010deho": 59,
+    "P138 pojmenov\u00e1no po": 23,
+    "P1383 zahrnuje s\u00eddlo": 60,
+    "P140 vyzn\u00e1n\u00ed": 24,
+    "P150 ni\u017e\u0161\u00ed spr\u00e1vn\u00ed celky": 25,
+    "P155 p\u0159edchoz\u00ed": 26,
+    "P159 s\u00eddlo": 27,
+    "P161 hraje": 28,
+    "P171 nad\u0159azen\u00fd taxon": 29,
+    "P175 interpret": 30,
+    "P176 v\u00fdrobce": 31,
+    "P179 s\u00e9rie": 32,
+    "P1889 rozd\u00edln\u00e9 od": 61,
+    "P19 m\u00edsto narozen\u00ed": 1,
+    "P20 m\u00edsto \u00famrt\u00ed": 2,
+    "P206 u vodn\u00ed plochy": 33,
+    "P21 pohlav\u00ed": 3,
+    "P22 otec": 4,
+    "P25 matka": 5,
+    "P26 cho\u0165": 6,
+    "P264 hudebn\u00ed vydavatelstv\u00ed": 34,
+    "P27 st\u00e1tn\u00ed ob\u010danstv\u00ed": 7,
+    "P276 m\u00edsto": 35,
+    "P279 nadt\u0159\u00edda": 36,
+    "P3373 sourozenec": 62,
+    "P36 hlavn\u00ed m\u011bsto": 8,
+    "P360 seznam (\u010deho)": 37,
+    "P361 \u010d\u00e1st (\u010deho)": 38,
+    "P39 ve funkci": 9,
+    "P413 pozice hr\u00e1\u010de": 39,
+    "P425 obor tohoto povol\u00e1n\u00ed": 40,
+    "P4552 poho\u0159\u00ed": 63,
+    "P460 \u00fadajn\u011b tot\u00e9\u017e co": 41,
+    "P461 protiklad": 42,
+    "P463 \u010dlen (\u010deho)": 43,
+    "P47 hrani\u010d\u00ed s": 10,
+    "P495 zem\u011b p\u016fvodu": 44,
+    "P50 autor": 11,
+    "P527 skl\u00e1d\u00e1 se z": 45,
+    "P53 rodina": 12,
+    "P530 diplomatick\u00fd vztah": 46,
+    "P54 \u010dlen sportovn\u00edho dru\u017estva": 13,
+    "P641 sport": 47,
+    "P669 ulice": 48,
+    "P706 oblast": 49,
+    "P708 diec\u00e9ze": 50,
+    "P734 p\u0159\u00edjmen\u00ed": 51,
+    "P735 rodn\u00e9 jm\u00e9no": 52,
+    "P740 m\u00edsto vzniku": 53,
+    "P800 d\u00edlo": 54,
+    "P921 hlavn\u00ed t\u00e9ma d\u00edla": 55,
+    "P97 \u0161lechtick\u00fd titul": 14,
+    "P974 p\u0159\u00edtok": 56
+  },
+  "layer_norm_eps": 1e-05,
+  "max_position_embeddings": 514,
+  "model_type": "roberta",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 1,
+  "position_embedding_type": "absolute",
+  "torch_dtype": "float32",
+  "transformers_version": "4.46.2",
+  "type_vocab_size": 1,
+  "use_cache": true,
+  "vocab_size": 51997
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:45d8f06f2b6a986b6d025a5c9b356695bdc8b3fda9edc7a1c9846b827e6255d2
+size 504532408

runs/Apr04_09-16-35_dgx10/events.out.tfevents.1743750997.dgx10.2636948.2

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b4d09f4f11ff5169d2c24c8794ac08cadadaa31be8264655c3a13736aa24f5d
+size 81581

runs/Apr04_09-16-35_dgx10/events.out.tfevents.1743765966.dgx10.2636948.3

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:14f55ec080a5d5ddf2c7933b930b75a075e1fa717dad1d9f6dc2a0c89b37497f
+size 757

special_tokens_map.json

@@ -0,0 +1,88 @@
+{
+  "additional_special_tokens": [
+    {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    },
+    {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false
+    }
+  ],
+  "bos_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eos_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": true,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": true,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,108 @@
+{
+  "add_prefix_space": true,
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "1": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "2": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "3": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "51960": {
+      "content": "[MASK]",
+      "lstrip": true,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "51997": {
+      "content": "[unused1]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "51998": {
+      "content": "[unused2]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "51999": {
+      "content": "[unused3]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "52000": {
+      "content": "[unused4]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "52001": {
+      "content": "[unused5]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "additional_special_tokens": [
+    "[unused1]",
+    "[unused2]",
+    "[unused3]",
+    "[unused4]",
+    "[unused5]"
+  ],
+  "bos_token": "[CLS]",
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "eos_token": "[SEP]",
+  "errors": "replace",
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6ba9e29f5bdfcb3ac2df1d9600b70ba9e6bfa1da73df3926b87c2c8adea29c9d
+size 5368