eskayML/bert_interview_new

README.md

@@ -18,8 +18,8 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.9106
-- Accuracy: 0.7697
+- Loss: 2.6712
+- Accuracy: 0.3534
 
 ## Model description
 
@@ -42,7 +42,7 @@ The following hyperparameters were used during training:
 - train_batch_size: 2
 - eval_batch_size: 2
 - seed: 42
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
 - lr_scheduler_type: linear
 - num_epochs: 10
 
@@ -50,21 +50,21 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Accuracy |
 |:-------------:|:-----:|:----:|:---------------:|:--------:|
-| 2.5257        | 1.0   | 759  | 2.1260          | 0.4079   |
-| 1.9969        | 2.0   | 1518 | 1.7308          | 0.5066   |
-| 1.7415        | 3.0   | 2277 | 1.4204          | 0.6316   |
-| 1.5223        | 4.0   | 3036 | 1.2385          | 0.6743   |
-| 1.3484        | 5.0   | 3795 | 1.1238          | 0.7039   |
-| 1.2576        | 6.0   | 4554 | 1.0478          | 0.7336   |
-| 1.1427        | 7.0   | 5313 | 0.9814          | 0.7467   |
-| 1.0499        | 8.0   | 6072 | 0.9471          | 0.75     |
-| 1.0145        | 9.0   | 6831 | 0.9234          | 0.7632   |
-| 0.911         | 10.0  | 7590 | 0.9106          | 0.7697   |
+| No log        | 1.0   | 463  | 2.2196          | 0.3319   |
+| 2.1608        | 2.0   | 926  | 2.1235          | 0.3534   |
+| 1.816         | 3.0   | 1389 | 2.1393          | 0.3879   |
+| 1.533         | 4.0   | 1852 | 2.1836          | 0.3578   |
+| 1.2761        | 5.0   | 2315 | 2.2730          | 0.3664   |
+| 1.122         | 6.0   | 2778 | 2.3939          | 0.3578   |
+| 0.9403        | 7.0   | 3241 | 2.4908          | 0.3578   |
+| 0.8317        | 8.0   | 3704 | 2.5671          | 0.3448   |
+| 0.7571        | 9.0   | 4167 | 2.6484          | 0.3491   |
+| 0.693         | 10.0  | 4630 | 2.6712          | 0.3534   |
 
 
 ### Framework versions
 
-- Transformers 4.44.2
-- Pytorch 2.4.0+cu121
-- Datasets 3.0.0
-- Tokenizers 0.19.1
+- Transformers 4.47.1
+- Pytorch 2.5.1+cu121
+- Datasets 3.2.0
+- Tokenizers 0.21.0

config.json

@@ -64,6 +64,6 @@
   "sinusoidal_pos_embds": false,
   "tie_weights_": true,
   "torch_dtype": "float32",
-  "transformers_version": "4.44.2",
+  "transformers_version": "4.47.1",
   "vocab_size": 30522
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:149a9d8e3b01a3b88776b9549a68c36e9a8e0d5d9045d8623e488bfa81770a85
+oid sha256:6d9dbccaa14803162baafd5476241487de48fe774526ae379c9906da80c002d2
 size 267887936

tokenizer_config.json

@@ -41,9 +41,10 @@
       "special": true
     }
   },
-  "clean_up_tokenization_spaces": true,
+  "clean_up_tokenization_spaces": false,
   "cls_token": "[CLS]",
   "do_lower_case": true,
+  "extra_special_tokens": {},
   "mask_token": "[MASK]",
   "max_length": 512,
   "model_max_length": 512,

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d11b28af5d635c99b0ece872a85d26404ddaef6bb47baa37d9a9e428a8ccbae8
-size 5240
+oid sha256:4be4389c28dca71b9cdc03c0481c5cb743949f9a9768a3f5fa6af0031c050bdb
+size 5304