adhi29/openhermes-mistral-dpo-gptq

README.md

@@ -17,15 +17,15 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [TheBloke/OpenHermes-2-Mistral-7B-GPTQ](https://huggingface.co/TheBloke/OpenHermes-2-Mistral-7B-GPTQ) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.3035
-- Rewards/chosen: -0.1587
-- Rewards/rejected: 0.2546
-- Rewards/accuracies: 0.5
-- Rewards/margins: -0.4133
-- Logps/rejected: -174.7579
-- Logps/chosen: -224.3590
-- Logits/rejected: -2.3662
-- Logits/chosen: -2.4097
+- Loss: 0.6522
+- Rewards/chosen: 0.0746
+- Rewards/rejected: 0.0188
+- Rewards/accuracies: 0.5625
+- Rewards/margins: 0.0558
+- Logps/rejected: -85.2534
+- Logps/chosen: -70.1958
+- Logits/rejected: -2.6062
+- Logits/chosen: -2.5855
 
 ## Model description
 
@@ -58,11 +58,11 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
 |:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.6721        | 0.01  | 10   | 0.7773          | 0.0134         | 0.0815           | 0.4375             | -0.0681         | -176.4889      | -222.6381    | -2.3576         | -2.4077       |
-| 0.7142        | 0.01  | 20   | 1.0003          | -0.1104        | 0.1182           | 0.375              | -0.2286         | -176.1221      | -223.8764    | -2.3560         | -2.4059       |
-| 0.7088        | 0.01  | 30   | 1.1927          | -0.1594        | 0.1949           | 0.4375             | -0.3543         | -175.3545      | -224.3664    | -2.3615         | -2.4090       |
-| 0.6932        | 0.02  | 40   | 1.2898          | -0.1780        | 0.2348           | 0.4375             | -0.4128         | -174.9561      | -224.5523    | -2.3625         | -2.4082       |
-| 0.6738        | 0.03  | 50   | 1.3035          | -0.1587        | 0.2546           | 0.5                | -0.4133         | -174.7579      | -224.3590    | -2.3662         | -2.4097       |
+| 0.6707        | 0.01  | 10   | 0.6784          | 0.0243         | 0.0096           | 0.375              | 0.0147          | -85.3454       | -70.6989     | -2.6043         | -2.5836       |
+| 0.6955        | 0.01  | 20   | 0.6748          | 0.0239         | 0.0210           | 0.3125             | 0.0029          | -85.2314       | -70.7031     | -2.6048         | -2.5849       |
+| 0.7423        | 0.01  | 30   | 0.6740          | 0.0542         | 0.0384           | 0.5625             | 0.0158          | -85.0572       | -70.4001     | -2.6053         | -2.5858       |
+| 0.7023        | 0.02  | 40   | 0.6620          | 0.0731         | 0.0323           | 0.5625             | 0.0407          | -85.1185       | -70.2116     | -2.6061         | -2.5863       |
+| 0.685         | 0.03  | 50   | 0.6522          | 0.0746         | 0.0188           | 0.5625             | 0.0558          | -85.2534       | -70.1958     | -2.6062         | -2.5855       |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6468b21e45d7e7d451b65a9109ec3ce1f7f620e5e5852d97827059332318d0e3
+oid sha256:ed2bd803830f05aed17991050c376ef747a8829b0d969d4141a8810e47adaa2b
 size 13648432

runs/Jan09_02-10-47_31d4a5c829e1/events.out.tfevents.1704766337.31d4a5c829e1.122.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bedb5f4083ef1ec4d5d4937a330545ed552fb5d7373e2f24d64f8ae04d5b1b3e
+size 12594

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fec25297cc0415bc1bea7e329eeabc63520f98291332920894380203922a7ee0
+oid sha256:accf2b53d9c6289fc381d015c0585ad7bcbb44c69f6389d73604727ffdd55f6c
 size 4155